A computer vision-based steel bar spacing detection method and system

Abstract

The application discloses a kind of steel bar spacing detection method and system based on computer vision, in combination with camera calibration technology in computer vision technology and deep neural network model, steel bar spacing detection in building industrial production and engineering construction construction can be realized, method includes: collection steel bar picture, label picture production dataset, utilize the dataset of production completion training model and adjustment parameter to optimal configuration.By camera calibration method, obtain camera parameter, erect camera to specified position and shoot image containing marker to obtain size conversion relationship, then shoot steel bar image and realize the identification of steel bar by trained model, output identification content and convert into binary image, extract central axis and detect steel bar pixel spacing and convert into real size.The application can effectively realize steel bar spacing detection, and has broad application prospect.

Description

Technical Field

[0001] The present invention belongs to the technical field of civil engineering and artificial intelligence interaction, and particularly relates to a method and system for detecting steel bar spacing based on computer vision. Background Art

[0002] Reinforced concrete engineering technology is the world's most widely used and largest-volume engineering construction technology. Various structures developed based on reinforced concrete are widely used in housing, roads, bridges, railways, and other engineering construction fields. As a key component of reinforced concrete, rebar provides high tensile strength while enhancing crack resistance and stability. The spacing and arrangement of rebar significantly impacts the quality of the finished reinforced concrete product. Rebar tying is a concealed process, obscured by the next step after tying. Therefore, rebar spacing testing is often performed immediately after tying.

[0003] Traditional detection methods mainly rely on manual measurement, which is time-consuming, labor-intensive, and prone to large measurement errors. If there are any inspection omissions and they are found to be non-compliant with the design specifications during the final acceptance process, readjustments will be required, which will waste construction time. If omissions are not discovered in time during the acceptance process, the strength of the finished reinforced concrete product will be affected, and quality defects and safety hazards will be buried. Using laser point cloud scanning, it is possible to extract the point cloud model of the steel bar and accurately measure its spacing, but laser point cloud scanning requires frequent collection of point cloud data. The process is time-consuming and technically difficult, and it requires certain skills from construction industry production or construction practitioners, making it difficult to apply on a large scale. Traditional image processing technology can also be used to detect the spacing between steel bars, but detection methods based on edge detection or contour extraction have very limited detection effects in complex construction scenarios, and the recognition effect is often unsatisfactory. There are many omissions or recognition errors in the detection. Summary of the Invention

[0004] To solve the above problems, the present invention discloses a method for detecting steel bar spacing based on computer vision, which can realize automatic detection of steel bar spacing, reduce labor costs, and greatly improve engineering efficiency.

[0005] To achieve the above object, the technical solution of the present invention is as follows:

[0006] A computer vision-based method for detecting steel bar spacing relies on a neural network framework and image processing technology to complete the detection of steel bar spacing. The method includes the following steps:

[0007] S1: Use a camera to capture images of constructed steel meshes, steel cages, or various types of individual steel bars, reduce interference factors similar to steel bar features in the image, and process the image;

[0008] S2: Label the images collected in S1, with the labeled objects being steel bars, to form a complete dataset that can be used for neural network training. The dataset is then divided into training, validation, and test sets in reasonable proportions for model training and testing.

[0009] S3: Use the Mask2Former deep learning neural network model to train the data set obtained in step S1. Perform multiple trainings by setting different training hyperparameter modules to find the hyperparameter module with the best effect. Perform training with this hyperparameter module configuration and save the training parameters.

[0010] S4: Set up a camera directly above the target and calibrate the camera parameters. Obtain the internal and external parameters of the camera through calibration. Take an image of the target and eliminate image distortion.

[0011] S5: Process the image, adjust the image content to the front view direction through projection transformation or affine transformation, and calculate the conversion relationship between pixel size and real size;

[0012] S6: Use the neural network model trained in S3 to recognize the processed image in S5, output the recognition content, which is the steel bar mask and steel bar category, extract the recognition content, and perform image enhancement processing;

[0013] S7: The pixel size between the steel bars in the target image is obtained by extracting the center line of the skeleton. The actual size is obtained through conversion and compared with the design specifications to complete the steel bar spacing detection.

[0014] Furthermore, the images collected to establish the dataset in step S1 should be clear and contain a variety of rebar types. The captured images were set to 1333 x 800 pixels. Interference objects similar to rebar features should be minimized during capture. To increase image diversity, image processing was performed, including flipping, mirroring, cropping, and contrast adjustment. The primary viewing angle for capturing images was a frontal view of the target, supplemented by other viewing angles to increase image diversity.

[0015] Furthermore, the dataset prepared in step S2 is divided into three parts: training set, validation set, and test set in a ratio of 8:1:1 or 7:2:1. The labeled objects are steel bars, and the labels are set to rebar_v and rebar_h to distinguish between longitudinal and transverse steel bars. The dataset is saved in the format of the MS COCO instance segmentation dataset.

[0016] Further, the hyperparameter module configuration of the Mask2Former model in step S3 needs to adjust the hyperparameters, including: learning rate LR, training batch size BATCH_SIZE, iteration number ITERATION, gradient update interval STEPS, weight decay WEIGHT_DECAY, optimizer OPTIMIZER, and thread number NUM_WORKERS.

[0017] Further, in step S4, the camera calibration is performed using a checkerboard calibration board. The specific calibration process is as follows:

[0018] S4-1: First, change the camera position to take multiple-angle calibration board shots to ensure that the calibration board is clearly shot.

[0019] S4-2: Set up the camera at the specified position and fix it to ensure that the detection target image is basically in the front view direction and can be completely shot. Re-shoot the calibration board picture and make sure that the calibration board occupies more than 2 / 3 of the image content.

[0020] S4-3: Use the image recognition corner points shot in step S4-1 to calibrate and select the camera calibration position as the camera position in step S4-2.

[0021] S4-4: After completing the calibration, do not change the camera position, and then shoot the image and use the camera parameters at the specified position to detect the image.

[0022] Further, in step S5, the center square (or multiple adjacent squares) of the checkerboard calibration board is selected as the marker, and the marker is projected to the image center through affine transformation or projection transformation to make the image in the front view direction. Set the pixel after the marker projection to compare with the actual size of the marker to obtain the conversion relationship λ between the image size after processing and the real size.

[0023] Further, in step S6, the output recognition content is the steel bar category: rebar_v and rebar_h, and the mask for the steel bar. According to the different steel bar categories, specific masks are extracted to obtain the mask graph of horizontal and vertical steel bars. The mask graph is binarized to obtain a binary graph containing information about the horizontal and vertical positions of the steel bars.

[0024] Further, in step S7, the horizontal or vertical steel bar binary image is processed to extract the skeleton of the steel bars in the same direction, and the center line of the steel bar skeleton is fitted as a straight line through least squares transformation. The distance between the center lines of the steel bar skeletons is calculated to obtain the steel bar center distance.

[0025] In a second aspect, a steel bar spacing detection system based on computer vision is provided, which comprises:

[0026] The model training module trains the rebar detection model by collecting batches of images. The image acquisition and processing requirements refer to the specific contents described in steps S2 and S3. The model training operation refers to the specific contents described in step S4. The output data is the trained optimal model parameters.

[0027] The camera calibration module is used to calibrate the image, obtain the internal and external parameters of the camera that captured the image, and eliminate image distortion. The calibration operation refers to the specific content described in step S5.

[0028] Image processing module, used to adjust the calibrated steel bar image to the front view, set the conversion relationship between pixel size and actual size, and output the processed image to be detected

[0029] The steel bar recognition module is used to identify the steel bars in the image to be detected, identify the steel bars and output a mask image containing category information and position information.

[0030] The spacing detection module is used to extract the steel bar mask skeleton line in the mask image, obtain the pixel spacing between the steel bars according to the distance between the calculated skeleton lines, and obtain the actual steel bar spacing through the conversion relationship between the pixel size and the actual size.

[0031] The beneficial effects of the present invention are:

[0032] The present invention is a method and system for detecting steel bar spacing based on computer vision, which overcomes the inherent defects of traditional steel bar spacing detection methods, such as time-consuming and labor-intensive methods, excessive detection errors, and unstable results. By utilizing computer vision technology to complete camera calibration, and setting up a camera to capture steel bar images, the conversion relationship between image size and actual size is obtained. The deep learning model Mask2Former is used to identify steel bars and output a mask image containing position information and steel bar category information and its binary image. The pixel spacing between steel bars is obtained by extracting the central axis, and the pixel spacing is converted to finally achieve steel bar spacing measurement. Efficient measurement can be achieved through a one-time arrangement in conjunction with the construction process, and the automation of steel bar spacing measurement in the reinforced concrete construction process can be realized, which has broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 It is a flow chart of a steel bar spacing detection method based on computer vision.

[0034] Figure 2 This is a schematic diagram of model hyperparameter configuration under an embodiment.

[0035] Figure 3 The present invention is a schematic diagram of calibration based on computer vision technology under an embodiment.

[0036] Figure 4 This is a steel bar identification effect diagram under an embodiment.

[0037] Figure 5 This is a schematic diagram of steel bar mask extraction under an embodiment.

[0038] Figure 6 It is a flow chart of a steel bar spacing detection system based on computer vision. DETAILED DESCRIPTION

[0039] The present invention will be further explained below with reference to the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention.

[0040] like Figure 1 As shown, the method for detecting steel bar spacing based on computer vision according to the present invention specifically includes the following steps:

[0041] S1: Use a camera to capture images of constructed steel meshes, steel cages, or various individual steel bars. Ensure the images are clear and diverse, and set the image size to 1333*800 pixels. Avoid any interference with features similar to those of steel bars. To increase image diversity, perform image processing operations such as flipping, mirroring, cropping, and adjusting contrast. The primary viewing angle for capturing images is the front view of the target, supplemented by other viewing angles to increase image diversity. In this example, a total of 245 images were collected, which were expanded to 300 after image processing, for a total of 300 images in the dataset.

[0042] S2: Label the images collected in S1, with the labeled objects being steel bars, to form a complete dataset that can be used for neural network training. The dataset is then divided into a training set, a validation set, and a test set according to a reasonable ratio for training the model and testing the model's effectiveness. In this embodiment, a 7:2:1 ratio is used for the division, and the labels are set to rebar_v and rebar_h to distinguish between longitudinal and transverse steel bars. The dataset is saved in the format of the MS COCO instance segmentation dataset.

[0043] S3: Use the Mask2Former deep learning neural network model to train the data set obtained in step S1. Perform multiple trainings by setting different training hyperparameter modules to find the hyperparameter module with the best effect. Perform training with this hyperparameter module configuration and save the training parameters.

[0044] The detailed steps for configuring the hyperparameter module of the Mask2Former model are as follows:

[0045] Use Optuna to adjust hyperparameters, setting appropriate adjustment steps based on the upper and lower limits of conventional hyperparameter fluctuations. That is, give several sampling points within the hyperparameter variation range and select a configuration each time in the specified order.

[0046] Use incremental steps and combine hyperparameter configurations to train the model, so that almost every configuration combination can be implemented. Considering the number of hyperparameter models that need to be adjusted, the number of combinations should be set to more than 50 times;

[0047] Use Optuna to monitor training, set up an early stopping mechanism, cut off combination branches with poor training results during training, and select the most appropriate hyperparameter configuration based on the training results.

[0048] The hyperparameters that need to be adjusted include: learning rate LR, training batch size BATCH_SIZE, number of iterations ITERATION, gradient update interval STEPS, weight decay WEIGHT_DECAY, optimizer OPTIMIZER, and number of threads NUM_WORKERS provided by the CPU.

[0049] In this embodiment, the training is performed on a server with an Intel(R) Xeon(R) Platinum 8336C CPU @ 2.30GHz, a NVIDIA GeForce RTX 4090 (24G) GPU, and an Ubuntu 20.04 operating system. Figure 2 As shown in .

[0050] S4: Set up a camera directly above the target and calibrate the camera parameters. Obtain the internal and external parameters of the camera through calibration. Take an image of the target and eliminate image distortion.

[0051] The camera calibration is performed using a checkerboard calibration plate, and the calibration process is as follows:

[0052] S4-1: First, change the camera position to shoot the calibration plate from multiple angles. You can also adjust the position of the calibration plate to ensure that the calibration plate is clearly photographed. During the calibration process, the calibration plate should occupy more than 2 / 3 of the image content. For the specific process, refer to Figure 3 shown.

[0053] S4-2: Set up the camera at the designated location and secure it. Ensure that the camera's viewing angle allows the target image to be viewed from the front and captures all targets. Ensure the camera is directly in front of the steel mesh (when arranged vertically) or directly above it (when arranged horizontally). Finally, retake the calibration plate image, ensuring that the calibration plate occupies at least two-thirds of the image content.

[0054] S4-3: Use the image captured in step S4-1 to identify corner points for calibration, and use the calibration plate image captured last in S4-2 to determine the camera pose and set it as the camera calibration pose.

[0055] S4-4: After calibration, the camera position is not changed, and the image can be captured and detected using the camera parameters at the specified position.

[0056] S5: Process the image, adjust the image content to the front view direction through projection transformation or affine transformation, and obtain the conversion relationship between the pixel size and the real size.

[0057] The central square (or multiple adjacent squares) of the checkerboard calibration plate is selected as the marker, and the marker is projected to the center of the image through affine transformation or projection transformation so that the image is in the front view direction. The projected pixels of the marker are set and compared with the actual size of the marker to obtain the conversion relationship between the processed image size and the actual size;

[0058] The conversion relationship is:

[0059] Where λ is the conversion factor, b actual is the actual size of the marker, b pixel is the pixel size of the marker. The former can be obtained by measuring the actual marker, and the latter can be obtained based on the captured image data.

[0060] S6: Use the neural network model that has been trained in S3 to recognize the processed image in S5, output the recognition content, which is the steel bar mask and steel bar category, extract the recognition content, and perform image enhancement processing.

[0061] The output recognition content is the steel bar category: rebar_v, rebar_h, and the mask for the steel bar. The steel bar image reference obtained after recognition Figure 4 Specific masks are extracted according to different types of steel bars, and mask images of horizontal and vertical steel bars are obtained respectively. The mask images are binarized to obtain binary images containing information about the horizontal and vertical positions of steel bars. The binary images can be used as effective input for extracting the center axis of steel bars. The binary image effect can be referenced. Figure 5 ,The steel bars are output in two different categories of horizontal and vertical directions.

[0062] S7: Process the acquired binary image of the transverse or longitudinal steel bars to extract the skeleton of the steel bars distributed in the same direction, fit the center line of the steel bar skeleton into a straight line through least square transformation, and calculate the steel bar center distance by calculating the distance between the center lines of the steel bar skeleton.

[0063] The calculation formula is: actual =dpixel ·λ

[0064] wherein d actual is the actual steel bar spacing to be obtained, d pixel is the pixel steel bar spacing obtained according to the image data, and λ is the conversion factor obtained in step S5.

[0065] As Figure 6 shown, the embodiment provides a steel bar spacing detection system based on computer vision, comprising:

[0066] A model training module 101 is configured to train a steel bar detection model by collecting batches of images, and the image collection and processing requirements are described in detail in steps S2 and S3. The model training operation is described in detail in step S4, and the output data is the best model parameter after training.

[0067] A camera calibration module 102 is configured to calibrate the images, obtain the internal and external parameters of the camera for shooting the images, and eliminate image distortion. The calibration operation is described in detail in step S5.

[0068] An image processing module 103 is configured to adjust the steel bar image after calibration to a front view, set the conversion relationship between the pixel size and the actual size, and output the processed image to be detected

[0069] A steel bar recognition module 104 is configured to recognize the steel bar in the image to be detected, recognize the steel bar, and output a mask image containing category information and position information.

[0070] A spacing detection module 105 is configured to extract the steel bar mask skeleton line in the mask image, obtain the pixel spacing between the steel bars according to the distance between the skeleton lines, and obtain the actual steel bar spacing through the conversion relationship between the pixel size and the actual size.

[0071] It should be noted that the above content only illustrates the technical idea of the present application, and cannot be used to limit the protection scope of the present application. For ordinary skilled persons in the technical field, they can make several improvements and refinements without departing from the principles of the present application, and these improvements and refinements all fall within the protection scope of the claims of the present application.

Claims

1. A method for detecting steel bar spacing based on computer vision, characterized in that: Rebar spacing detection is accomplished by relying on a neural network framework and image processing technology, specifically including the following steps: S1: Use a camera to capture images of constructed steel meshes, steel cages, or various types of individual steel bars, reduce interference factors similar to steel bar features in the image, and process the image; S2: Label the images collected in S1, with the labeled objects being steel bars, to form a complete dataset that can be used for neural network training. The dataset is then divided into training, validation, and test sets in reasonable proportions for model training and testing. S3: Use the Mask2Former deep learning neural network model to train the data set obtained in step S1. Perform multiple trainings by setting different training hyperparameter modules to find the hyperparameter module with the best effect. Perform training with this hyperparameter module configuration and save the training parameters. S4: Set up a camera directly above the detection target, calibrate the camera parameters, and obtain the internal and external parameters of the camera through calibration; capture the detection target image and eliminate the distortion in the image; S5: Process the image, adjust the image content to the front view direction through projection transformation or affine transformation, and calculate the conversion relationship between pixel size and real size; S6: Use the neural network model trained in S3 to recognize the processed image in S5, output the recognition content, which is the steel bar mask and steel bar category, extract the recognition content, and perform image enhancement processing; S7: The pixel size between the steel bars in the target image is obtained by extracting the center line of the skeleton. The actual size is obtained through conversion and compared with the design specifications to complete the steel bar spacing detection.

2. The method for detecting steel bar spacing based on computer vision according to claim 1, wherein: The images collected to establish the dataset in step S1 should be clear and diverse in types. The captured images should be set to 1333*800 pixels in size. When shooting, interference objects similar to the characteristics of steel bars should be reduced. To increase image diversity, the images are flipped, mirrored, cropped, and the contrast is adjusted. The image shooting perspective is mainly based on the front view of the target, supplemented by other perspectives to increase the diversity of image types.

3. The method for detecting steel bar spacing based on computer vision according to claim 1, wherein: The dataset prepared in step S2 is divided into three parts: training set, validation set, and test set in a ratio of 8:1:1 or 7:2:

1. The annotated objects are steel bars, and the labels are set to rebar_v and rebar_h to distinguish between longitudinal and transverse steel bars. The dataset is saved in the format of the MS COCO instance segmentation dataset.

4. The method for detecting steel bar spacing based on computer vision according to claim 1, wherein: The hyperparameter module configuration of the Mask2Former model described in step S3 should include the following hyperparameters: learning rate LR, training batch size BATCH_SIZE, number of iterations ITERATION, gradient update interval STEPS, weight decay WEIGHT_DECAY, optimizer OPTIMIZER, and number of threads NUM_WORKERS.

5. The method for detecting steel bar spacing based on computer vision according to claim 1, wherein: The camera calibration in step S4 is performed using a checkerboard calibration plate. The specific calibration process is as follows: (1) First, change the camera position to shoot the calibration plate from multiple angles to ensure that the calibration plate is clearly photographed; (2) Set up the camera at the designated location and fix it, ensuring that the shooting angle makes the detection target image basically present the front view direction and can completely capture all detection targets; re-shoot the calibration plate image and ensure that the calibration plate can occupy more than 2 / 3 of the image content; (3) Using the image captured in step (1) to identify corner points for calibration, and selecting the camera position set up in step (2) as the camera calibration pose; (4) After calibration, the image can be captured without changing the camera position, and the image can be detected using the camera parameters at the specified position.

6. The method for detecting steel bar spacing based on computer vision according to claim 1, wherein: In step S5, the central square of the checkerboard calibration plate is selected as the marker, and the marker is projected to the center of the image through affine transformation or projection transformation so that the image is in the direction of the front view. The pixels after projection of the marker are set and compared with the actual size of the marker to obtain the conversion relationship between the image size and the actual size after processing.

7. The method for detecting steel bar spacing based on computer vision according to claim 1, wherein: The output recognition content described in step S6 is the steel bar category: rebar_v, rebar_h, and the mask for the steel bar. Specific masks are extracted according to different steel bar categories to obtain mask images of transverse steel bars and vertical steel bars respectively. The mask images are binarized to obtain binary images containing information about the transverse steel bars and vertical positions respectively.

8. The method for detecting steel bar spacing based on computer vision according to claim 1, wherein: In step S7, the acquired binary image of the transverse or longitudinal steel bars is processed to extract the skeleton of the steel bars distributed in the same direction, the center line of the steel bar skeleton is fitted into a straight line by least square transformation, and the steel bar center distance is calculated by calculating the distance between the center lines of the steel bar skeleton.

9. A steel bar spacing detection system based on computer vision, characterized in that: include: The model training module trains the steel bar detection model by collecting batch images, and the output data is the trained optimal model parameters. The camera calibration module is used to calibrate the image, obtain the internal and external parameters of the camera that took the image, and eliminate image distortion. The image processing module is used to adjust the calibrated steel bar image to the positive view, set the conversion relationship between pixel size and actual size, and output the processed image to be detected. The steel bar recognition module is used to identify the steel bars in the image to be detected, identify the steel bars and output a mask image containing category information and location information. The spacing detection module is used to extract the steel bar mask skeleton line in the mask image, obtain the pixel spacing between the steel bars according to the distance between the calculated skeleton lines, and obtain the actual steel bar spacing through the conversion relationship between the pixel size and the actual size.

Images