Method for automatically detecting quality of concealed engineering reinforcing steel bars based on computer vision technology

Abstract

The invention discloses a computer vision technology-based hidden engineering reinforcing steel bar quality automatic detection method, which comprises the following steps of: acquiring an image of a binding reinforcing steel bar, performing data enhancement processing on the acquired image, and performing position and size marking on the processed image to obtain a binding reinforcing steel bar data set; using the data set to train a YOLOv8 target detection network to obtain a target detection model; inputting the preprocessed image needing to be detected into a target detection model for target identification to obtain the number and position information of all steel bars in the image; extracting a contour according to the steel bar position information, and performing size calculation by using a size calculation algorithm; and comparing the size and the number of the reinforcing steel bars with reinforcing steel bar parameters indicated in a design file, and judging whether the index error meets the acceptance standard or not. The device is high in automation degree and low in cost, multiple steel bars can be measured at the same time, and the detection efficiency and precision are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of concealed project quality inspection, and particularly to an automatic detection method for the quality of steel bars in concealed projects based on computer vision technology. Background Art

[0002] The acceptance of concealed projects is of great significance in projects. As an important part of concealed projects, whether the steel bars meet the specification requirements will affect the quality of components and even the safety and stability of the project structure. Therefore, during the construction process, it is necessary to ensure that the specifications, quantities, and positions of the steel bars meet the design requirements to avoid potential structural risks.

[0003] The acceptance content of steel bars mainly includes the spacing, quantity, diameter, and anchorage length of the steel bars. Currently, the traditional acceptance method uses manual assistance with simple measuring devices (tape measures, vernier calipers) to detect various indicators of the steel bars, which has problems of low efficiency and poor accuracy.

[0004] With the rapid development of deep learning technology, it has been widely used in the fields of computer vision and image recognition. However, in the current field of steel bar acceptance, the application and development of computer vision technology are still in the primary stage, and there are still problems such as insufficient model generalization ability and the inability to achieve real-time detection in actual operations. Summary of the Invention

[0005] In order to overcome the above problems existing in the prior art, the present invention proposes an automatic detection method for the quality of steel bars in concealed projects based on computer vision technology.

[0006] The technical solution adopted by the present invention to solve its technical problems is: an automatic detection method for the quality of steel bars in concealed projects based on computer vision technology, including the following steps: Step 1, collect steel bar sample images with different rib shapes and sizes, perform data enhancement processing on the collected images, and label the rib regions and the positions and sizes of the steel bar center regions for the processed images to obtain a bound steel bar data set; Step 2, use the data set obtained in Step 1 to train the YOLOv8 object detection network to obtain an object detection model; Step 3, input the preprocessed image to be detected into the object detection model obtained in Step 2 for object recognition to obtain the quantity and position information of all steel bars in the image; Step 4, extract the contours based on the steel bar position information obtained in Step 3 and perform size calculation using a size calculation algorithm; Step 5, compare the sizes obtained in Step 4 and the steel bar quantity obtained in Step 3 with the steel bar specifications stated in the design document to determine whether the index error meets the acceptance standard; When training the YOLOv8 object detection network in step 2, a rib suppression loss function is added to adjust the yolov8 loss function and reduce the impact of the rib area on model training.

[0007] In the above-mentioned automatic detection method for the quality of concealed project steel bars based on computer vision technology, the size calculation in step 4 is specifically as follows: perform distortion correction on the camera, select an object with a known size, calculate the conversion ratio between pixels and the actual size, and the diameter of the steel bar is the product of the bounding box width and the conversion ratio between pixels and the actual size.

[0008] In the above-mentioned automatic detection method for the quality of concealed project steel bars based on computer vision technology, the camera distortion correction process includes: preparing a standard checkerboard pattern, where the size of each small square is known, as the calibration benchmark; collecting checkerboard images at different angles and positions, with the corner points of the checkerboard distributed at different positions and orientations in each checkerboard image; using the findChessboardCorners function in the OpenCV library to detect the corner points of the checkerboard in each image; and using the calibrateCamera function to calculate the internal and external parameters of the camera.

[0009] In the above-mentioned automatic detection method for the quality of concealed project steel bars based on computer vision technology, the data augmentation processing in step 1 includes but is not limited to rotation, scaling, flipping, cropping, translation, and color transformation.

[0010] In the above-mentioned automatic detection method for the quality of concealed project steel bars based on computer vision technology, in step 1, the labelme tool is used to mark the minimum bounding rectangle of each steel bar on the image and mark the corresponding position information and size information.

[0011] In the above-mentioned automatic detection method for the quality of concealed project steel bars based on computer vision technology, the loss function of the YOLOv8 object detection network is: ; Where, is the total loss, is the value of the weight map; is the weight of the localization loss, is the weight of the classification loss; is the localization loss, is the cross-entropy loss; is the predicted bounding box, is the ground truth bounding box; is the predicted class, is the ground truth class.

[0012] The beneficial effects of the present invention are as follows: (1) The present invention proposes a steel bar acceptance technology using computer vision. This technology is characterized by low cost, high precision, high efficiency, and automation, representing a new method in the field of steel bar acceptance.

[0013] (2) The detection efficiency of the present invention is significantly improved. Compared with traditional manual measurement, it usually takes 10 to 30 seconds for manual use of tools such as calipers and tape measures to measure the dimensions of a steel bar, while the present patented technology only takes 16 milliseconds to 50 milliseconds to complete the dimension measurement of a steel bar. In addition, the traditional method requires measurement one by one, while the present patented technology can achieve simultaneous measurement of multiple steel bars.

[0014] (3) The detection accuracy of the present invention is significantly improved. Due to the influence of subjective factors and objective factors of the instrument in traditional manual measurement, there are often large deviations in the measurement results. The present patented technology controls the measurement error within the range of ±1 mm to ±5 mm through the use of high-resolution cameras, accurate sample data annotation, and correction of camera distortion, with a relative error of 1% to 3%, greatly improving the measurement accuracy.

[0015] (4) The present invention has a high degree of automation. In traditional manual acceptance, acceptance personnel need to manually record the position, dimensions, and quantity of each steel bar, which is time-consuming and error-prone. The present patented technology realizes the complete automation of steel bar index measurement. The computer automatically identifies the position of the steel bar, automatically assigns a bounding box to each steel bar, and accurately measures its diameter and length without manual operation intervention. Brief Description of the Drawings

[0016] Figure 1 is the schematic flow chart of the present invention; Figure 2 is the work flow chart of the acceptance personnel in the embodiment of the present invention; Figure 3 is the model training flow chart in the embodiment of the present invention; Figure 4 is the schematic diagram of dimension calculation in the embodiment of the present invention; Figure 5 is the schematic diagram of the server system before and after the present invention. Detailed Embodiments

[0017] To enable those skilled in the art to better understand the technical solution of the present invention, the present invention will be described in detail below with reference to the drawings and specific embodiments.

[0018] The present invention discloses an automatic detection method for the quality of steel bars in concealed works based on computer vision technology. The specific steps are as Figure 1 shown and include the following steps: Step 1: Collect images of steel bar samples with different rib shapes and sizes, perform data augmentation on the collected images, and label the rib regions and mark the positions and sizes of the centers of the steel bars on the processed images to obtain a dataset of tied steel bars.

[0019] Step 2: Use the dataset obtained in Step 1 to train the YOLOv8 object detection network to obtain an object detection model.

[0020] When training the YOLOv8 object detection network, add a rib suppression loss function to adjust the yolov8 loss function and reduce the impact of the rib region on model training.

[0021] Step 3: Input the preprocessed image to be detected into the object detection model obtained in Step 2 for object recognition to obtain the quantity and position information of all steel bars in the image.

[0022] Step 4: Extract the contours based on the steel bar position information obtained in Step 3 and perform size calculation using a size calculation algorithm.

[0023] Step 5: Compare the sizes obtained in Step 4 and the quantity of steel bars obtained in Step 3 with the steel bar specifications noted in the design document to determine whether the index error meets the acceptance standard.

[0024] Based on the above detection method, when performing automatic detection of steel bar quality, the following preparatory work needs to be done: I. Technical preparatory work: The specific process is as Figure 2 shown. First, the acceptance personnel equip a high-resolution industrial network camera (IP camera) and establish a real-time data transmission channel between the camera and the computer.

[0025] Construct the transmission channel between the camera and the computer. The specific operation steps are described as follows:

[0026] (1) Camera network access: First, connect the IP camera to the local area network or Wi-Fi network through physical connection or wireless means to ensure that the camera is successfully connected and obtains a valid IP address to enable network communication.

[0027] (2) Configure the IP address to OpenCV: Subsequently, configure the video IP address of the camera to the OpenCV program so that OpenCV can identify and receive the video stream data transmitted by the camera through this IP address.

[0028] The acceptance personnel fix the camera in the acceptance area and collect images of tied steel bars in a handheld or fixed manner. Through the established transmission channel, the backend server can receive the collected picture data in real time.

[0029] The backend server processes the collected data. After the front-end server receives the information processed by the backend, it marks the minimum bounding rectangle of each steel bar on the original steel bar binding image, and marks the corresponding position information and dimension information beside it. The processed result image is finally presented on the front-end display.

[0030] The acceptance personnel will compare the steel bar index measurement results displayed on the front-end display with the steel bar specifications noted in the design document in detail to determine whether the index error is within the range allowed by the acceptance standard.

[0031] II. Preparation work for the backend server model training: First, construct a dataset specifically for the target to be measured (i.e., the target steel bar), which should include three parts: the training set, the test set, and the validation set. Subsequently, use this dataset to conduct centralized training on the target steel bar to construct a yolov8 object detection network with strong generalization ability.

[0032] For the training of the high-strength object detection network, the specific operation steps are as Figure 3 shown and are described as follows: (1) First, perform systematic data augmentation on the collected steel bar images, including but not limited to rotation, scaling, flipping, cropping, translation, and color transformation, etc., to expand the dataset, aiming to improve the generalization ability of the model and simulate the performance of ribbed steel bars at different positions and angles, helping the model better adapt to the morphological changes of steel bars. Ensure that it can still maintain good detection performance in a diverse environment and eliminate the influence of steel bar ribs on measurement errors.

[0033] (2) Then, use the labelme tool to accurately annotate the pre-prepared steel bar images to ensure that the position, dimension, and rib area of each steel bar are accurately recorded.

[0034] (3) After that, summarize the annotation files corresponding to the images of the training set, test set, and validation set and integrate them into a standardized annotation file for subsequent processing and analysis.

[0035] (4) Finally, execute the program in the PyCharm integrated development environment to convert all the json-format annotation files into txt format for the smooth progress of the training process.

[0036] (5) Add a rib suppression loss function. By adjusting the yolov8 loss function, reduce the influence of the rib area on model training. Finally, use the improved yolov8 model for training.

[0037] The specific process of adding a rib suppression loss function and adjusting the yolov8 loss function is as follows: 1) Data preparation and rib mask generation Generate a mask image: Use manual annotation to generate a rib mask; Annotation of the rib region: Mark the central region of the steel bar binary mask and the rib region. Thus, a weight map is created, with the weight of the central region set to 1.0 and the weight of the rib region set to 0.2; Specifically expressed as: Central region of the steel bar: weight_map = 1.0; Rib region of the steel bar: weight_map = 0.2.

[0038] 2) Design a rib suppression loss function

[0039] weight_map is the weight map for each pixel or region, where the weight of the central region of the steel bar is set to 1.0 and the weight of the rib region of the steel bar is set to 0.2. In the yolov8 loss function, rib suppression is added through a weighted loss term, that is, this weight map is used to adjust the conventional loss function (including localization loss and classification loss).

[0040] 3) Integrate into the yolov8 model training (call the rib suppression loss function)

[0041] In the training code of YOLOv8, modify yolo_loss() and add the rib suppression loss function.

[0042] In the training loop, pass in the weight map weight_map and calculate the weighted loss.

[0043] The adjusted loss function is: ; where, is the total loss, is the value of the weight map. The weight of the rib region is smaller (0.2), and the weight of the target region (central region of the steel bar) is larger (1). is the weight of the localization loss, is the weight of the classification loss. is the localization loss, is the cross-entropy loss. is the predicted bounding box, is the ground truth bounding box. is the predicted class, is the ground truth class.

[0044] The backend server receives the steel bar images transmitted from the front end and inputs them into the well-trained yolov8 detection network for target recognition. This process can accurately identify all the steel bar targets in the image and obtain the position and quantity information of these steel bars. ​

[0045] Extract the contour of the steel bar based on its position information, and use the dimension calculation algorithm to measure the relevant dimensions. Subsequently, transmit the test results and the dimension calculation results back to the front-end server through the network for subsequent display and comparison.

[0046] To ensure high-precision dimension measurement, the following dimension calculation steps are implemented (as Figure 4 shown): First, perform distortion correction on the camera to ensure the accuracy of measurement. The specific calibration process is as follows:

[0047] (1) Carefully prepare a standard chessboard pattern, where the size of each small square is known, as the calibration reference.

[0048] (2) Take ten chessboard images from multiple angles and positions, ensuring that the corner points of the chessboard are distributed at different positions and orientations in each image for subsequent calibration calculations.

[0049] (3) Use the findChessboardCorners function in the OpenCV library to accurately detect the corner points of the chessboard in each image.

[0050] (4) Use the calibrateCamera function to calculate the internal and external parameters of the camera.

[0051] (5) Correct the distortion of the camera according to the results obtained from the calibration.

[0052] (6) After the distortion correction is completed, select an object with a known size, and then calculate the conversion ratio between pixels and the actual size.

[0053] (7) After obtaining the conversion ratio, the diameter of the steel bar can be calculated according to the following formula: Steel bar diameter (mm) = Bounding box width (pixels) × Pixel-to-millimeter ratio.

[0054] III. Verify the feasibility of the method

[0055] Select and prepare a batch of steel bar materials with a diameter of 20 mm, bind them strictly in accordance with industry specifications, and configure a high-definition camera to capture and collect detailed images of the steel bars.

[0056] The back-end server uses the efficient yolov8 detection network to identify the targets to be measured in the images. The average accuracy of the results exceeds 60% under the map@0.5 metric and exceeds 40% under the map@0.75 metric, indicating that the model has high detection accuracy and reliability.

[0057] The operator reads the detection data output by the backend server on the front-end display device, compares it with the actual size of 20 millimeters, and the resulting error range is between 1% and 3%, thus verifying the practical feasibility of the patented technology.

[0058] Based on the above detection method, the detection system of this embodiment includes a front-end device and a backend server. As Figure 5 shown, the front-end device includes a camera and a computer. The computer includes a controller, a display screen, a keyboard, and a mouse. The backend server is used to process the images collected by the front-end device, and perform contour extraction and size calculation. The camera captures images and transmits them to the backend server through a data transmission channel. The backend server processes the captured images. The computer receives the information processed by the backend server, marks the minimum circumscribed rectangle of each steel bar on the original image, and marks the corresponding position and size information beside it, and displays the processing result on the display screen.

[0059] The above embodiments are only exemplary embodiments of the present invention and are not used to limit the present invention. Those skilled in the art can make various modifications or equivalent replacements to the present invention within the essence and protection scope of the present invention, and such modifications or equivalent replacements should also be regarded as falling within the protection scope of the present invention.

Claims

1. The method for automatically detecting the quality of steel bars in concealed engineering based on computer vision technology is characterized in that: The steps include: Step 1, collect steel bar sample images with different rib shapes and sizes, perform data enhancement processing on the collected images, mark the rib area and the position and size of the steel bar center area on the processed images, and obtain a tied steel bar data set; Step 2: Use the data set obtained in step 1 to train the YOLOv8 target detection network to obtain a target detection model; Step 3, inputting the preprocessed image to be detected into the target detection model obtained in step 2 to perform target recognition, and obtaining the quantity and position information of all steel bars in the image; Step 4, extracting the contour according to the steel bar position information obtained in step 3, and performing size calculation using a size calculation algorithm; Step 5, comparing the size obtained in step 4 and the number of steel bars obtained in step 3 with the steel bar specifications indicated in the design document to determine whether the index error meets the acceptance criteria; In the step 2, when training the YOLOv8 target detection network, a rib suppression loss function is added, the yolov8 loss function is adjusted, and the influence of the rib area on the model training is reduced.

2. The method for automatically detecting the quality of steel bars in concealed works based on computer vision technology according to claim 1, characterized in that: The size calculation in step 4 is specifically as follows: performing distortion correction on the camera, selecting an object of known size, calculating the conversion ratio between pixels and actual size, and the diameter of the steel bar is the product of the width of the bounding box and the conversion ratio between pixels and actual size.

3. The method for automatically detecting the quality of steel bars in concealed works based on computer vision technology according to claim 2, characterized in that: The camera distortion correction process includes: preparing a standard chessboard pattern, in which the size of each small square is known, as a calibration reference; collecting chessboard images at different angles and positions, and the corner points of the chessboard in each chessboard image are distributed in different positions and directions; using the findChessboardCorners function in the OpenCV library to detect the chessboard corner points in each image; and using the calibrateCamera function to calculate the internal and external parameters of the camera.

4. The method for automatically detecting the quality of steel bars in concealed works based on computer vision technology according to claim 1, characterized in that: The data enhancement processing in step 1 includes but is not limited to rotation, scaling, flipping, cropping, translation and color transformation.

5. The method for automatically detecting the quality of steel bars in concealed works based on computer vision technology according to claim 1, characterized in that: In step 1, the labelme tool is used to mark the minimum circumscribed rectangle of each steel bar on the image, and the corresponding position information and size information are marked.

6. The method for automatically detecting the quality of steel bars in concealed works based on computer vision technology according to claim 1, characterized in that: The YOLOv8 target detection network loss function is: ； in, is the total loss, is the value of the weight map; is the weight of the positioning loss, is the weight of classification loss; is the positioning loss, is the cross entropy loss; is the predicted bounding box, It is a real border; is the predicted category, is the real category.

Images