Visual inspection and measurement method for reinforcing mesh

Abstract

The present invention relates to the technical field of buildings, and in particular, to a visual inspection and measurement method for a reinforcing mesh, comprising the steps of: using an RGBD camera to perform data acquisition on a building site; performing Mask R-CNN instance segmentation on RGB, preliminarily identifying a workpiece, and segmenting a 2D workpiece from the background; converting the obtained 2D workpiece into a 3D point cloud on the basis of an intrinsic mapping relationship of the RGBD camera; and for on-site acquired data, obtaining a segmentation result of rebar details by means of FPFH feature matching with an offline model; and on the basis of an identified rebar point cloud, performing cylinder fitting to obtain the number of rebars within the field of view. According to the present invention, preliminary identification of rebars is performed on a 2D site image by means of Mask R-CNN instance segmentation, and then rebar 3D geometry is precisely identified by using FPFH feature consistency.

Description

Reinforcing mesh visual detection and measurement method TECHNICAL FIELD

[0001] The present application relates to the technical field of building, in particular to a reinforcing mesh visual detection and measurement method. BACKGROUND

[0002] The mesh structure made of steel bars is usually used in concrete structures, which plays a role in strengthening the strength, stability and crack resistance of concrete. In civil construction, reinforcing mesh is widely used in bridge, tunnel, high-rise building and other complex projects. It can effectively disperse stress, prevent concrete cracking and improve the durability of the structure. The specifications and quality of the building reinforcing mesh need to be strictly managed, such as the number, diameter and spacing of reinforcing mesh, and currently manual measurement is mostly used, which is low in efficiency and difficult to ensure the measurement accuracy.

[0003] SUMMARY

[0004] The present application relates to the technical field of building, in particular to a reinforcing mesh visual detection and measurement method.

[0005] In order to achieve the above-mentioned purpose, the technical scheme adopted by the present application is as follows:

[0006] A reinforcing mesh visual detection and measurement method, comprising the following steps:

[0007] Step S1: using an RGBD camera to collect data on the construction site;

[0008] Step S2: performing Mask R-CNN instance segmentation on RGB, which not only preliminarily identifies the workpiece, but also separates the 2D workpiece from the background;

[0009] Step S3: based on the internal mapping relationship of the RGBD camera, converting the segmented 2D workpiece into 3D point cloud;

[0010] Step S4: for the collected data, obtaining the segmentation result of the reinforcing bar details by matching the FPFH features of the paired and offline models;

[0011] Step S5: according to the reinforcing bar point cloud identified in step S4, performing cylinder fitting to obtain the number count of reinforcing bars in the field of view.

[0012] The following is a further defined technical scheme of the present application, the data acquisition device is an RGBD camera, the color image and depth image obtained by the RGBD camera are one-to-one corresponding in the image coordinate system, therefore the 2D color workpiece image is converted into 3D point cloud.

[0013] The following is a further limited technical solution of the application, because Mask R-CNN cannot distinguish the background of the workpiece hollow area, so that the converted workpiece point cloud contains noise points, and through conditional filtering, that is, filtering the background according to the Z direction distance of the workpiece to the camera.

[0014] The following is a further limited technical solution of the application, based on the point cloud obtained by the RGBD camera, the point cloud data contains noise points or outliers due to the influence of the performance of the acquisition device and the surface characteristics of the object, and because the noise points or outliers in the point cloud are sparse and isolated, statistical filtering is used to eliminate invalid point clouds with large density differences, that is, the mean and variance of the Gaussian distribution in the neighborhood of the point cloud are used to determine outliers and filter them out.

[0015] The following is a further limited technical solution of the application, the data collected by the RGBD camera belongs to dense point cloud, and the number of point clouds is large, and because each point cloud needs to be calculated in the feature extraction process, voxel filtering is used to downsample the point cloud data to dilute the point cloud density and improve the execution efficiency of the calculation.

[0016] The following is a further limited technical solution of the application, when calculating the FPFH feature, the original point cloud should be used as the neighborhood of the key point to ensure that the extracted feature has more local performance and distinguishability.

[0017] The following is a further limited technical solution of the application, for offline workpiece models, the same method as the online workpiece is used to extract the FHFP feature of each workpiece Wherein, 0<j≤N, N represents the type, m represents the number of key points, and an offline feature library F is established m :

[0018] Next, the FPFH features of the online workpiece and the offline model are matched, that is, the FPFH feature matching relationship between the on-site reinforcement mesh and the candidate categories is determined.

[0019] The following is a further limited technical solution of the application, according to the central axis fitted by the cylinder, the center position of different steel bars is located, and the average spacing and the number of steel bars within a unit distance are calculated.

[0020] Compared with the prior art, the application has the following technical effects:

[0021] The application performs preliminary identification of steel bars on a 2D live image through instance segmentation of Mask R-CNN, and then performs accurate identification of the 3D shape of the steel bars by using the consistency of FPFH features. In this way, the strong distinguishing characteristics of the FPFH features on local details can make up for the deficiency of the Mask R-CNN model in identifying 3D workpieces with similar local structures and heterogeneous perspectives, and the rough identification by Mask R-CNN can narrow the candidate range and thus improve the efficiency of the identification based on the FPFH features.

[0022] The application will be further described below in combination with the drawings and embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the drawings needed to be used in the embodiments or the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without any creative effort on the basis of these drawings.

[0024] FIG. 1 is a flowchart of the method of the application. DETAILED DESCRIPTION

[0025] In order to make the above-mentioned purposes, features and advantages of the application more apparent and easy to understand, the specific embodiments of the application will be described in detail below in combination with the drawings. In the following description, a large number of specific details are set forth in order to fully understand the application. However, the application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without departing from the connotation of the application, so the application is not limited by the specific embodiments disclosed below.

[0026] As shown in FIG. 1, the embodiment provides a method for visual detection and measurement of a steel mesh, which detects and measures the steel mesh based on Mask R-CNN and point cloud feature pairing FPFH algorithm. The method first takes a two-dimensional photo of the scene by an RGBD camera, performs neural network scanning, extracts point cloud data therefrom, extracts feature point sets of the required detection area from the point cloud data, and calculates a rotation matrix and a translation vector by using point cloud registration.

[0027] The method performs preliminary identification of steel bars on a 2D live image through instance segmentation of Mask R-CNN, and then performs accurate identification of the 3D shape of the steel bars by using the consistency of FPFH features. In this way, the strong distinguishing characteristics of the FPFH features on local details can make up for the deficiency of the Mask R-CNN model in identifying 3D workpieces with similar local structures and heterogeneous perspectives, and the rough identification by Mask R-CNN can narrow the candidate range and thus improve the efficiency of the identification based on the FPFH features.

[0028] The workpiece recognition algorithm combining Mask R-CNN and FPFH feature pairing mainly includes two processes of instance segmentation based on Mask R-CNN and steel bar recognition based on 3D feature consistency pairing. First, the data of the construction site is collected by using an RGBD camera, and Mask R-CNN instance segmentation is performed on the RGB, which can not only preliminarily identify the workpiece, but also segment the 2D workpiece from the background. Based on the internal mapping relationship of the RGBD camera, the segmented 2D workpiece can be converted into 3D point cloud. Finally, the distinguishability of the FPFH feature is used to further accurately identify the steel bar point cloud and measure the features.

[0029] In general, the method mainly includes the following steps: point cloud preprocessing, FPFH feature calculation and pairing, steel bar feature measurement and extraction, etc.

[0030] 1. Point cloud preprocessing: the data acquisition device is an RGBD camera, and the color image and depth image obtained by the camera can realize one-to-one correspondence in the image coordinate system, so the 2D color workpiece image can be converted into 3D point cloud. Because Mask R-CNN cannot distinguish the background of the hollow area of the workpiece, the converted workpiece point cloud contains noise points. Through conditional filtering, that is, according to the distance of the workpiece to the camera in the Z direction, the background is filtered out. In addition, based on the point cloud obtained by the RGBD camera, the point cloud data contains noise points or outliers, especially in the edge area with obvious feature distinguishability. In addition, the data collected by the RGBD camera belongs to dense point cloud, and the number of point clouds is large, and the feature extraction process needs to calculate each point cloud, which not only consumes a lot of time, but also causes the lack of computer computing power. In view of the above situation, it is necessary to filter and down-sample the point cloud for preprocessing. Considering that the noise points or outliers in the point cloud are sparse and relatively isolated, statistical filtering is used to eliminate invalid point clouds with relatively large density differences, that is, the mean and variance of the Gaussian distribution in the neighborhood of the point cloud are used to determine the outliers and filter them out. Then, voxel filtering is used to down-sample the point cloud data to dilute the point cloud density and improve the execution efficiency of the algorithm.

[0031] 2. FPFH feature extraction and pairing: for the data collected on site, the FPFH features of the steel bar details are further obtained by pairing with the offline model. When calculating the FPFH feature, the original point cloud should be used as the neighborhood of the key point to ensure that the extracted feature has more local performance and distinguishability. For the offline workpiece model, the FHFP feature of each workpiece is extracted in the same way as the online workpiece (0 < j ≤ N, where N represents the type and m represents the number of key points), and an offline feature library F m :

[0032] Next, the FPFH features of the online workpiece and the offline model are paired, that is, the FPFH feature pairing relationship between the field reinforcement mesh and the candidate categories is determined.

[0033] 3. Reinforcement measurement feature extraction: According to the accurately recognized reinforcement point cloud, further cylinder fitting is performed to obtain the reinforcement quantity count in the field of view. According to the central axis of the cylinder fitting, the center positions of different reinforcements can be located, so as to calculate the average spacing and the reinforcement distribution quantity per unit distance.

[0034] The above is only a preferred embodiment of the present application, and does not limit the present application in any form. Any person skilled in the art can make many possible changes and modifications to the technical solution of the present application, or modify it into equivalent embodiments, without departing from the scope of the technical solution of the present application, by using the disclosed methods and technical contents. Therefore, any equivalent changes made according to the shape, structure and principle of the present application, without departing from the technical solution of the present application, should be covered by the protection scope of the present application.

Claims

1. A method of visual inspection and measurement of reinforcement mesh, characterised by, Comprise the following steps: Step S1: data collection of construction site using RGBD camera; Step S2: Mask R-CNN instance segmentation on RGB, both preliminary identification of workpieces and segmentation of 2D workpieces from the background; Step S3: based on the internal mapping relationship of RGBD camera, convert the segmented 2D workpieces into 3D point cloud; Step S4: for the data collected on site, obtain the segmentation result of steel bar details by matching the FPFH features of the offline model; Step S5: according to the steel bar point cloud identified in step S4, perform cylinder fitting to obtain the number of steel bars in the field of view.

2. The method of visual inspection and measurement of a reinforcing mesh as claimed in claim 1, wherein, The data collection device is RGBD camera, and the color image and depth image obtained are one-to-one corresponding in the image coordinate system, so the 2D color workpiece image is converted into 3D point cloud.

3. The method of visual inspection and measurement of a reinforcing mesh according to claim 1, wherein, Because Mask R-CNN cannot distinguish the background of the hollow area of the workpiece, the converted workpiece point cloud contains noise points, which are filtered out by conditional filtering, that is, according to the distance of the workpiece to the Z direction of the camera.

4. The method of visual inspection and measurement of a reinforcing mesh as claimed in claim 1, wherein, Based on the point cloud obtained by RGBD camera, the performance of the collection device and the surface characteristics of the object are affected, and there are noise points or outliers in the point cloud data. Because the noise points or outliers in the point cloud are sparse and isolated, statistical filtering is used to eliminate invalid point clouds with large density difference, that is, to determine outliers and filter them out based on the mean and variance of the Gaussian distribution in the neighborhood of the point cloud.

5. The method of visual inspection and measurement of a reinforcing mesh as claimed in claim 1, wherein, The data collected by RGBD camera belongs to dense point cloud, and the number of point clouds is large. Because each point cloud needs to be calculated in the feature extraction process, voxel filtering is used to downsample the point cloud data to dilute the point cloud density and improve the execution efficiency of the calculation.

6. The method of visual inspection and measurement of a reinforcing mesh as claimed in claim 1, wherein, When calculating the FPFH feature, the original point cloud should be used as the neighborhood of the key point to ensure that the extracted feature has more local performance and distinguishability.

7. The method of visual inspection and measurement of a reinforcing mesh as claimed in claim 1, wherein, For offline workpiece models, the FHFP features of each workpiece are extracted in the same way as the online workpieces where 0 < j < N, N represents the category, and m represents the number of key points, an offline feature library F is established m : Next, the FPFH features of the online workpiece and the offline model are matched, that is, the FPFH feature matching relationship between the steel bar mesh on site and the candidate categories is determined.

8. The method of visual inspection and measurement of a reinforcing mesh as claimed in claim 1, wherein, According to the central axis of the cylinder fitting, the center position of different steel bars is located, and the average spacing and the number of steel bars distributed in the unit distance are calculated.

Images