Steel bar spacing detection method and system based on computer vision

Abstract

The invention discloses a steel bar spacing detection method and system based on computer vision, which can realize steel bar spacing detection in building industry production and engineering construction by combining a camera calibration technology and a deep neural network model in a computer vision technology, and comprises the following steps: collecting steel bar pictures, marking the pictures to make a data set, and training a model by using the manufactured data set and adjusting parameters to optimal configuration. Camera parameters are obtained through a camera calibration method, a camera is erected to a designated position to shoot an image containing a marker to obtain a size conversion relation, then a reinforcing steel bar image is shot, reinforcing steel bar recognition is achieved through a trained model, and recognition content is output and converted into a binary image; and extracting the pixel pitch of the central axis detection steel bar and converting the pixel pitch into a real size. The steel bar spacing detection can be practically and effectively realized, and the method has a wide application prospect.

Description

Technical Field

[0001] The present invention belongs to the technical field of civil engineering and artificial intelligence interaction, and specifically relates to a steel bar spacing detection method and system based on computer vision. Background Art

[0002] Reinforced concrete engineering technology is the most widely used and largest-volume engineering construction technology in the world. Various structures developed based on reinforced concrete are widely used in housing construction, roads, bridges, railways and other engineering construction fields. As an important component of reinforced concrete, steel bars provide high-strength tensile properties, while enhancing the crack resistance and stability of reinforced concrete. The spacing of steel bars has a huge impact on the quality of reinforced concrete products. Rebar binding is a hidden project and is covered by the next process after binding. The spacing detection of steel bars is often carried out immediately after binding is completed.

[0003] Traditional detection methods mainly rely on manual measurement, which is time-consuming, labor-intensive and prone to large measurement errors. If there are omissions in the detection, it is found that it does not meet the design specifications during the final acceptance process, and it needs to be readjusted, which consumes construction time. If the omissions are not discovered in time during the acceptance process, it will affect the strength of the finished reinforced concrete product, burying quality defects and safety hazards. Using laser point cloud scanning, it is possible to extract the steel bar point cloud model and accurately measure its spacing, but laser point cloud scanning requires frequent collection of point cloud data. The process is time-consuming and technically difficult. It has certain requirements on the skills of construction industry production or construction practitioners, and it is difficult to apply on a large scale. Traditional image processing technology can also be used to detect the spacing between steel bars, but the detection method based on edge detection or contour extraction has very limited detection effect in complex construction scenes, and the recognition effect is often not ideal. There are many omissions or recognition errors in the detection. Summary of the invention

[0004] To solve the above problems, the present invention discloses a method for detecting steel bar spacing based on computer vision, which can realize automatic detection of steel bar spacing, reduce labor costs, and greatly improve engineering efficiency.

[0005] To achieve the above object, the technical solution of the present invention is as follows:

[0006] A method for detecting steel bar spacing based on computer vision relies on a neural network framework and image processing technology to complete the steel bar spacing detection, and specifically includes the following steps:

[0007] S1: Use a camera to capture images of constructed steel meshes, steel cages, or various types of individual steel bars, reduce interference factors similar to steel bar features in the image, and process the image;

[0008] S2: Annotate the images collected in S1, with the annotated objects being steel bars, to form a complete data set that can be used for neural network training. The data set is divided into a training set, a validation set, and a test set according to a reasonable ratio for training the model and testing the model effect.

[0009] S3: Use the Mask2Former deep learning neural network model to train the data set obtained in step S1, perform multiple trainings by setting different training hyperparameter modules, find the hyperparameter module with the best effect, perform training with the hyperparameter module configuration, and save the training parameters;

[0010] S4: Set up a camera directly above the detection target, calibrate the camera parameters, and obtain the internal and external parameters of the camera through calibration. Take an image of the detection target and eliminate the distortion in the image;

[0011] S5: Process the image, adjust the image content to the front view direction through projection transformation or affine transformation, and obtain the conversion relationship between the pixel size and the real size;

[0012] S6: Use the neural network model that has been trained in S3 to recognize the processed image in S5, output the recognition content, which is the steel bar mask and steel bar category, extract the recognition content, and perform image enhancement processing;

[0013] S7: The pixel size between the steel bars in the target image is obtained by extracting the center line of the skeleton, and the actual size is obtained by conversion. The steel bar spacing detection can be completed by comparing it with the design specifications.

[0014] Furthermore, the images collected to establish the data set in step S1 should ensure that the images are clear and the types of steel bars are diverse. The image size is set to 1333*800 pixels. When shooting, it should be ensured that the interference objects similar to the characteristics of steel bars are reduced. In order to increase the diversity of images, the images are processed, including: flipping, mirroring, cropping, and adjusting the contrast. The image shooting perspective is mainly based on the front view of the shooting target, supplemented by other perspectives to increase the diversity of image types.

[0015] Furthermore, the dataset in step S2 is divided into three parts: training set, validation set, and test set in a ratio of 8:1:1 or 7:2:1. The annotated object is steel bars, and the labels are set to rebar_v and rebar_h to distinguish between longitudinal and transverse steel bars. The dataset is saved in the format of the MS COCO instance segmentation dataset.

[0016] Furthermore, the hyperparameter module configuration of the Mask2Former model in step S3, the hyperparameters that need to be adjusted should include: learning rate LR, training batch size BATCH_SIZE, number of iterations ITERATION, gradient update interval STEPS, weight decay WEIGHT_DECAY, optimizer OPTIMIZER, and number of threads NUM_WORKERS.

[0017] Furthermore, the camera calibration in step S4 is performed using a checkerboard calibration plate, and the specific calibration process is as follows:

[0018] S4-1: First, change the camera position to shoot the calibration plate from multiple perspectives to ensure that the calibration plate is clearly photographed.

[0019] S4-2: Set up the camera at the designated location and fix it, ensuring that the shooting angle makes the detection target image basically present the front view direction, and can completely capture all detection targets. Re-shoot the calibration plate image and ensure that the calibration plate can occupy more than 2 / 3 of the image content.

[0020] S4-3: Use the image captured in step S4-1 to identify corner points for calibration, and select the camera position set up in step S4-2 as the camera calibration posture.

[0021] S4-4: After the calibration is completed, the image can be captured without changing the camera position, and the image can be detected using the camera parameters at the specified position.

[0022] Furthermore, in step S5, the central square (or multiple adjacent squares) of the checkerboard calibration plate is selected as the marker, and the marker is projected to the center of the image through affine transformation or projection transformation, so that the image is in the direction of the front view, and the pixels of the marked object after projection are set and compared with the actual size of the marker to obtain the conversion relationship λ between the image size after processing and the actual size.

[0023] Furthermore, the output identification content in step S6 is the steel bar category: rebar_v, rebar_h, and the mask for the steel bar. A specific mask is extracted according to the different steel bar categories to obtain mask images of the transverse steel bars and the vertical steel bars, respectively. The mask images are binarized to obtain binary images containing information about the transverse steel bars and the vertical positions, respectively.

[0024] Furthermore, in step S7, the acquired binary image of the transverse or longitudinal steel bars is processed to extract the skeleton of the steel bars distributed in the same direction, the center line of the steel bar skeleton is fitted as a straight line by least squares transformation, and the steel bar center distance is calculated by calculating the distance between the center lines of the steel bar skeleton.

[0025] In a second aspect, a steel bar spacing detection system based on computer vision is provided, comprising:

[0026] The model training module trains the steel bar detection model by collecting batch images. The image collection and processing requirements refer to the specific contents described in step S2 and step S3. The model training operation refers to the specific contents described in step S4, and the output data is the trained optimal model parameters.

[0027] The camera calibration module is used to calibrate the image, obtain the internal and external parameters of the camera that took the image, and eliminate image distortion. The calibration operation refers to the specific content described in step S5.

[0028] The image processing module is used to adjust the calibrated steel bar image to the front view, set the conversion relationship between the pixel size and the actual size, and output the processed image to be detected.

[0029] The steel bar recognition module is used to identify the steel bars in the image to be detected, identify the steel bars and output a mask image containing category information and position information.

[0030] The spacing detection module is used to extract the steel bar mask skeleton line in the mask image, obtain the pixel spacing between the steel bars according to the distance between the calculated skeleton lines, and obtain the actual steel bar spacing through the conversion relationship between the pixel size and the actual size.

[0031] The beneficial effects of the present invention are:

[0032] The present invention is a method and system for detecting steel bar spacing based on computer vision, which overcomes the inherent defects of traditional steel bar spacing detection methods, such as time-consuming and labor-intensive, excessive detection error fluctuations, and unstable results. The camera calibration is completed by using computer vision technology, and the camera is set up to shoot steel bar images to obtain the conversion relationship between the image size and the actual size. The deep learning model Mask2Former is used to identify steel bars and output a mask image containing position information and steel bar category information and its binary image. The pixel spacing between the steel bars is obtained by extracting the central axis, and the pixel spacing is converted to finally realize the steel bar spacing measurement. Efficient measurement can be achieved through one-time arrangement in conjunction with the construction process, and the automation of steel bar spacing measurement in the reinforced concrete construction process can be realized, which has broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 It is a flow chart of a steel bar spacing detection method based on computer vision.

[0034] Figure 2 This is a schematic diagram of model hyperparameter configuration under an embodiment.

[0035] Figure 3 The present invention is a schematic diagram of calibration based on computer vision technology under an embodiment.

[0036] Figure 4 The present invention is a steel bar identification effect diagram under an embodiment.

[0037] Figure 5 The present invention is a schematic diagram of steel bar mask extraction under an embodiment.

[0038] Figure 6 It is a flow chart of a steel bar spacing detection system based on computer vision. DETAILED DESCRIPTION

[0039] The present invention will be further explained below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention.

[0040] like Figure 1 As shown, the steel bar spacing detection method based on computer vision described in the present invention specifically includes the following steps:

[0041] S1: Use a camera to shoot the built steel mesh, steel cage or various types of individual steel images. The image should be clear and the types of steel bars should be diverse. The image size should be set to 1333*800 pixels. When shooting, it should be ensured that interference objects similar to steel bar features appear less. To increase image diversity, the image is flipped, mirrored, cropped, and contrast adjusted. The image shooting perspective is mainly based on the front view of the shooting target, supplemented by other perspectives to increase the diversity of image types. In this embodiment, a total of 245 images are collected, which are expanded to 300 after image processing. The data set has a total of 300 images.

[0042] S2: Annotate the images collected in S1, with the annotated objects being steel bars, to form a complete data set that can be used for neural network training, and divide them into training set, validation set, and test set according to a reasonable ratio for training models and testing model effects. In this embodiment, a 7:2:1 ratio is used, and the labels are set to rebar_v and rebar_h to distinguish between longitudinal steel bars and transverse steel bars. The data set is saved in the format of the MS COCO instance segmentation data set.

[0043] S3: Use the Mask2Former deep learning neural network model to train the data set obtained in step S1, perform multiple trainings by setting different training hyperparameter modules, find the hyperparameter module with the best effect, perform training with the hyperparameter module configuration, and save the training parameters;

[0044] The detailed steps of the hyperparameter module configuration of the Mask2Former model are as follows:

[0045] Use Optuna to adjust hyperparameters and set appropriate adjustment steps according to the upper and lower limits of conventional hyperparameter fluctuations. That is, give several sampling points within the range of hyperparameter variation and use a configuration each time in the specified order.

[0046] The model training is performed by increasing the step size and combining the hyperparameter configurations, so that basically every configuration combination can be implemented. Considering the number of hyperparameter models that need to be adjusted, the number of combinations should be set to more than 50 times;

[0047] Use Optuna to monitor training, set up an early stopping mechanism, cut off combination branches with poor training effects during training, and select the most appropriate hyperparameter configuration based on the training results.

[0048] The hyperparameters that need to be adjusted include: learning rate LR, training batch size BATCH_SIZE, number of iterations ITERATION, gradient update interval STEPS, weight decay WEIGHT_DECAY, optimizer OPTIMIZER, and number of threads NUM_WORKERS provided by the CPU.

[0049] In this embodiment, the training is performed on a server with a CPU model of Intel(R) Xeon(R) Platinum 8336C CPU@2.30GHz, a GPU model of NVIDIA GeForce RTX 4090(24G), and an operating system of Ubuntu 20.04. The optimal configuration is as follows Figure 2 as shown in .

[0050] S4: Set up a camera directly above the detection target, calibrate the camera parameters, and obtain the internal and external parameters of the camera through calibration. Take an image of the detection target and eliminate the distortion in the image;

[0051] The camera calibration is performed using a checkerboard calibration plate, and the calibration process is as follows:

[0052] S4-1: First, change the camera position to shoot the calibration plate from multiple perspectives. You can also adjust the position of the calibration plate to ensure that the calibration plate is clearly photographed. During the calibration process, the calibration plate should occupy more than 2 / 3 of the image content. For the specific process, refer to Figure 3 shown.

[0053] S4-2: Set up the camera at the designated location and fix it, ensure that the shooting angle makes the image of the test target basically present the front view direction, and can completely capture all the test targets. The camera should be located in front of the steel mesh (when arranged vertically) or above (when arranged horizontally). Finally, retake the calibration plate image and ensure that the calibration plate can occupy more than 2 / 3 of the image content.

[0054] S4-3: Use the image captured in step S4-1 to identify the corner points for calibration, and use the calibration plate image last captured in S4-2 to determine the camera pose and set it as the camera calibration pose.

[0055] S4-4: After the calibration is completed, the image can be captured without changing the camera position, and the image can be detected using the camera parameters at the specified position.

[0056] S5: Process the image, adjust the image content to the front view direction through projection transformation or affine transformation, and obtain the conversion relationship between the pixel size and the real size.

[0057] The central square (or multiple adjacent squares) of the checkerboard calibration plate is selected as the marker, and the marker is projected to the center of the image through affine transformation or projection transformation, so that the image is in the front view direction, and the pixels of the projected marker are set, and compared with the actual size of the marker to obtain the conversion relationship between the image size and the actual size after processing;

[0058] The conversion relationship is:

[0059] Among them, λ is the conversion factor, b actual is the actual size of the marker, b pixel is the pixel size of the marker, the former can be obtained by measuring the actual marker, and the latter can be obtained based on the captured image data.

[0060] S6: Use the neural network model that has been trained in S3 to recognize the processed image in S5, output the recognition content, which is the steel bar mask and steel bar category, extract the recognition content, and perform image enhancement processing.

[0061] The output recognition content is the steel bar category: rebar_v, rebar_h, and the mask for the steel bar. The steel bar image reference obtained after recognition Figure 4 . According to the different types of steel bars, specific masks are extracted to obtain the mask images of transverse steel bars and vertical steel bars respectively. The mask images are binarized to obtain binary images containing information about transverse steel bars and vertical positions respectively. The binary images can be used as effective input for extracting the central axis of steel bars. The binary image effect can be referenced Figure 5 ,The steel bars are exported in two different categories: horizontal and vertical.

[0062] S7: Processing is performed based on the acquired binary image of the transverse or longitudinal steel bars, the skeleton of the steel bars distributed in the same direction is extracted, the center line of the steel bar skeleton is fitted as a straight line through least square transformation, and the steel bar center distance is calculated by calculating the distance between the center lines of the steel bar skeleton.

[0063] The calculation formula is: actual =dpixel ·λ

[0064] Among them, d actual is the actual reinforcement spacing required, d pixel is the pixel steel bar spacing obtained according to the image data, and λ is the conversion factor obtained in step S5.

[0065] like Figure 6 As shown, this embodiment provides a steel bar spacing detection system based on computer vision, including:

[0066] The model training module 101 trains the steel bar detection model by collecting batch images. The image collection and processing requirements refer to the specific contents described in step S2 and step S3. The model training operation refers to the specific contents described in step S4, and the output data is the trained optimal model parameters.

[0067] The camera calibration module 102 is used to calibrate the image, obtain the internal and external parameters of the camera that took the image, and eliminate image distortion. The calibration operation is described in detail in step S5.

[0068] The image processing module 103 is used to adjust the calibrated steel bar image to the front view, set the conversion relationship between the pixel size and the actual size, and output the processed image to be detected.

[0069] The steel bar identification module 104 is used to identify the steel bars in the image to be detected, identify the steel bars and output a mask image containing category information and position information.

[0070] The spacing detection module 105 is used to extract the steel bar mask skeleton line in the mask image, obtain the pixel spacing between the steel bars according to the distance between the calculated skeleton lines, and obtain the actual steel bar spacing through the conversion relationship between the pixel size and the actual size.

[0071] It should be noted that the above content only illustrates the technical idea of ​​the present invention and cannot be used to limit the protection scope of the present invention. For ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications all fall within the protection scope of the claims of the present invention.

Claims

1. A method for detecting steel bar spacing based on computer vision, characterized in that: Relying on the neural network framework and image processing technology to complete the steel bar spacing detection, the specific steps include the following: S1: Use a camera to capture images of constructed steel meshes, steel cages, or various types of individual steel bars, reduce interference factors similar to steel bar features in the image, and process the image; S2: Annotate the images collected in S1, with the annotated objects being steel bars, to form a complete data set that can be used for neural network training. The data set is divided into a training set, a validation set, and a test set according to a reasonable ratio for training the model and testing the model effect. S3: Use the Mask2Former deep learning neural network model to train the data set obtained in step S1, perform multiple trainings by setting different training hyperparameter modules, find the hyperparameter module with the best effect, perform training with the hyperparameter module configuration, and save the training parameters; S4: Set up a camera directly above the detection target, calibrate the camera parameters, and obtain the internal and external parameters of the camera through calibration; take an image of the detection target and eliminate the distortion in the image; S5: Process the image, adjust the image content to the front view direction through projection transformation or affine transformation, and obtain the conversion relationship between the pixel size and the real size; S6: Use the neural network model that has been trained in S3 to recognize the processed image in S5, output the recognition content, which is the steel bar mask and steel bar category, extract the recognition content, and perform image enhancement processing; S7: The pixel size between the steel bars in the target image is obtained by extracting the center line of the skeleton, and the actual size is obtained by conversion. The steel bar spacing detection can be completed by comparing it with the design specifications.

2. According to the method for detecting steel bar spacing based on computer vision described in claim 1, it is characterized in that: The images collected to establish the data set described in step S1 should ensure that the images are clear and the types of steel bars are diverse. The captured images are set to 1333*800 pixels in size. When shooting, it should be ensured that interference objects similar to steel bar features are reduced. In order to increase image diversity, the images are flipped, mirrored, cropped, and the contrast is adjusted. The image shooting perspective is mainly based on the front view of the shooting target, supplemented by other perspectives to increase the diversity of image types.

3. The method for detecting steel bar spacing based on computer vision according to claim 1, characterized in that: The dataset in step S2 is divided into three parts: training set, validation set, and test set in a ratio of 8:1:1 or 7:2:

1. The labeled objects are steel bars, and the labels are set to rebar_v and rebar_h to distinguish between longitudinal and transverse steel bars. The dataset is saved in the format of the MS COCO instance segmentation dataset.

4. The method for detecting steel bar spacing based on computer vision according to claim 1, characterized in that: The hyperparameter module configuration of the Mask2Former model described in step S3, the hyperparameters that need to be adjusted should include: learning rate LR, training batch size BATCH_SIZE, number of iterations ITERATION, gradient update interval STEPS, weight decay WEIGHT_DECAY, optimizer OPTIMIZER, number of threads NUM_WORKERS.

5. The method for detecting steel bar spacing based on computer vision according to claim 1, characterized in that: The camera calibration in step S4 is performed using a checkerboard calibration plate. The specific calibration process is as follows: (1) First, change the camera position to shoot the calibration plate from multiple perspectives to ensure that the calibration plate is clearly photographed; (2) Set up the camera at the designated location and fix it, ensuring that the shooting angle makes the image of the detection target basically present the front view direction and can completely capture all the detection targets; re-shoot the calibration plate image and ensure that the calibration plate can occupy more than 2 / 3 of the image content; (3) Using the image captured in step (1) to identify corner points for calibration, and selecting the camera position set up in step (2) as the camera calibration posture; (4) After calibration, the image can be captured without changing the camera position, and the image can be detected using the camera parameters at the specified position.

6. The method for detecting steel bar spacing based on computer vision according to claim 1, characterized in that: In step S5, the central square of the checkerboard calibration plate is selected as the marker, and the marker is projected to the center of the image through affine transformation or projection transformation, so that the image is in the direction of the front view, and the pixels of the marked object after projection are set and compared with the actual size of the marker to obtain the conversion relationship between the image size and the actual size after processing.

7. The method for detecting steel bar spacing based on computer vision according to claim 1, characterized in that: The output identification content described in step S6 is the steel bar category: rebar_v, rebar_h, and the mask for the steel bar. Specific masks are extracted according to different steel bar categories to obtain mask images of transverse steel bars and vertical steel bars respectively. The mask images are binarized to obtain binary images containing information about the transverse steel bars and vertical positions respectively.

8. The method for detecting steel bar spacing based on computer vision according to claim 1, characterized in that: In step S7, the acquired binary image of the transverse or longitudinal steel bars is processed to extract the skeleton of the steel bars distributed in the same direction, the center line of the steel bar skeleton is fitted as a straight line by least squares transformation, and the steel bar center distance is calculated by calculating the distance between the center lines of the steel bar skeleton.

9. A steel bar spacing detection system based on computer vision, characterized in that: include: The model training module trains the steel bar detection model by collecting batch images, and the output data is the best model parameters after training. The camera calibration module is used to calibrate the image, obtain the internal and external parameters of the camera that took the image, and eliminate image distortion. The image processing module is used to adjust the calibrated steel bar image to the front view, set the conversion relationship between the pixel size and the actual size, and output the processed image to be detected. The steel bar recognition module is used to identify the steel bars in the image to be detected, identify the steel bars and output a mask image containing category information and location information. The spacing detection module is used to extract the steel bar mask skeleton line in the mask image, obtain the pixel spacing between the steel bars according to the distance between the calculated skeleton lines, and obtain the actual steel bar spacing through the conversion relationship between the pixel size and the actual size.

Images