Prefabricated laminated slab and reinforcing steel bar accurate detection method fusing multi-scale features

Abstract

The invention discloses a multi-scale feature fused prefabricated laminated slab and reinforcing steel bar accurate detection method, which comprises the following steps: step 1, setting a shooting distance and a field of view, and obtaining an image shot by a camera; step 2, carrying out camera calibration and distortion correction; 3, collecting corrected image data, dividing the corrected image data into a data set, and sending the data set into an improved YOLOv8 model for reasoning to obtain a preliminary detection result; step 4, according to a detection result, pairing the laminated slab, the reinforcing steel bar and the wire box; 5, correcting a detection result; step 6, re-detecting the reinforcing steel bars subjected to identification missing detection and only partial detection, and updating a detection result; 7, converting the pixel distance and the actual distance to obtain a conversion ratio; and 8, calculating the actual physical size of the target according to the conversion proportion and the detection result. According to the invention, the problems of insufficient detection precision, distortion influence and large measurement error in the prior art are effectively solved, and the detection efficiency and the automation level are improved.

Description

Technical Field

[0001] The present invention belongs to the fields of machine vision and computer vision, and relates to a precise detection method for precast composite slabs and steel bars that integrates multi-scale features. Background Art

[0002] In the process of traditional precast composite slab and steel bar detection, operations such as manually measuring dimensions and recording the number of steel bars are usually relied on. The whole process is time-consuming and cumbersome, and it is difficult to meet the requirements of modern construction for efficiency and precision. With the rapid development of machine vision and computer vision technologies, vision-based automated detection systems have begun to become an effective solution to replace traditional manual operations, especially having obvious advantages in improving detection efficiency and realizing digital storage of data.

[0003] In the detection of precast composite slabs, existing technologies have tried to use the YOLOv5 deep learning object detection algorithm to perform automated detection on targets such as composite slabs and steel bars. However, there are still the following key problems in the existing technologies: 1. Insufficient object detection accuracy: Although YOLOv5 has achieved good results in object detection tasks, when facing large-size targets such as large composite slabs, or dense small targets such as steel bars and junction boxes, there is still room for improvement in its accuracy and recall rate. Especially in the case of occlusion between these targets, missing details, and complex lighting environments, the detection accuracy of YOLOv5 is particularly insufficient, resulting in common situations of false detection or missed detection.

[0004] 2. Camera distortion problem: Traditional detection methods usually require the camera to be installed at a relatively high position of about 8 meters to capture large-size composite slabs. However, this setting cannot completely eliminate camera lens distortion. Especially when shooting at a large angle of view, the distortion problem still exists, resulting in deviations in the shape and size of the targets in the image, affecting subsequent detection and analysis results.

[0005] 3. Distance conversion error: In the application of object detection, small-size targets such as magnet boxes are often used as reference objects to perform the conversion between pixel distance and actual physical distance. However, this conversion method has certain errors. Especially when the proportional relationship between the reference object and the detection target is relatively complex, the accuracy of size measurement is greatly affected, further increasing the measurement error.

[0006] In summary, there are significant problems in the existing technologies in terms of large-size target detection, camera distortion correction, and accurate size measurement methods. Summary of the Invention

[0007] In order to solve the above-mentioned technical problems existing in the prior art, the present invention proposes a precise detection method for precast composite slabs and steel bars that integrates multi-scale features, and its specific technical solutions are as follows: A precise detection method for precast composite slabs and steel bars that integrates multi-scale features, comprising the following steps: Step 1: Set the shooting distance and field of view to obtain the images captured by the camera; Step 2: Perform camera calibration and distortion correction; Step 3: Collect the corrected image data, divide the dataset, and feed it into the improved YOLOv8 model for inference to obtain the preliminary detection results, including the rectangular box positions and class information of the composite slabs, steel bars, and junction boxes; Step 4: Pair the composite slabs with their steel bars and junction boxes according to the detection results; Step 5: Correct the detection results; Step 6: Re-detect the steel bars that are missed or only partially detected during recognition and update the detection results; Step 7: Convert the pixel distance to the actual distance to obtain the conversion ratio; Step 8: Calculate the actual physical size of the target according to the conversion ratio and the detection results.

[0008] Further, in Step 1, the setting of the shooting distance and field of view is specifically as follows: According to the formula D = f * L / l = f * W / w, where f is the focal length of the camera lens, D is the shooting distance, l and w are the length and width of the camera sensor, and L and W are the maximum length and width of the precast composite slab.

[0009] Further, Step 2 is specifically as follows: Customize a large-size camera calibration board with a size of 2m × 1.5m, take several calibration images at different angles, and use the camera calibration tool to calculate the lens distortion parameters and complete the distortion correction.

[0010] Further, the improved YOLOv8 model adds a P6 feature extraction layer in the Backbone network; adds a P2 feature layer in the Neck structure and fuses it with the P3, P4, P5, and P6 feature layers; and adds feature maps with sizes of 160×160 and 10×10 in the Head structure.

[0011] Further, Step 4 specifically includes: Step 4.1: Traverse the composite slab data detected by the model in sequence, including the coordinates of the upper left and lower right corners of the composite slab and the corresponding confidence levels, and determine whether the confidence level, area, and position of the composite slab meet the set range. If they meet, save the composite slab data; Step 4.2: If the number of saved composite slabs is not zero, then continue to judge the steel bars and junction boxes; Step 4.3: Traverse the steel bar data detected by the model in sequence. Determine whether the length of the steel bar is greater than the width. If so, it is considered a horizontal steel bar; otherwise, it is a vertical steel bar. Then, continue to traverse the saved composite slab data for this steel bar. If it is a horizontal steel bar, first determine whether the y-coordinate value of the center coordinate of the steel bar is greater than the y-coordinate value of the upper left corner of the composite slab and less than the y-coordinate value of the lower right corner of the composite slab. If it meets the condition, use the x-coordinate values of the upper left corner and the lower right corner of the steel bar to compare with the x-coordinate values of the lower right corner and the upper left corner of the composite slab respectively. If the distance between the upper left corner of the steel bar and the lower right corner of the composite slab is within the set range, it is determined that the steel bar belongs to the right-side steel bar of the composite slab, and save the steel bar data to the composite slab data; similarly, judge the affiliation matching problems of the left-side steel bars and the vertical steel bars; where the upper left corner of the set image is the coordinate origin, the horizontal direction is the x-axis, and the vertical direction is the y-axis. Step 4.4: Traverse the junction box data detected by the model in sequence, and then traverse the saved composite slab data for each junction box data. Determine whether the junction box is included inside the composite slab through the upper left corner and the lower right corner coordinates of the junction box and the upper left corner and the lower right corner coordinates of the composite slab. If so, the junction box matches the composite slab, and save the junction box data to the composite slab data.

[0012] Furthermore, the fifth step specifically includes: First, traverse and read the composite slab data, and sequentially obtain all the steel bars on each side included in the composite slab; then use the interquartile range method to judge abnormal steel bars. Sort all the y-coordinates of the steel bars on one side from small to large, take the data near the 25% position as Q1, take the data near the 75% position as Q3, calculate the interquartile range IQR, IQR = Q3 - Q1, calculate the abnormal value boundary, the lower bound = Q1 - 1.5 * IQR, the upper bound = Q3 + 1.5 * IQR, that is, the data below the lower bound and the data above the upper bound are judged as abnormal; then calculate the absolute error between the average value of the normal steel bars and the corresponding composite slab edge. If the error is within the set range, the steel bars on this side do not need to be adjusted. If the error exceeds the set range, the composite slab edge needs to be adjusted to the average value position, and then continue to judge the steel bars on other sides.

[0013] Furthermore, in the sixth step, based on the prior position knowledge that the exposed steel bars are in a symmetric position relationship, if the distance between two steel bars is too large or the length of a certain steel bar is significantly abnormal compared with other steel bars, the image is cropped and then sent to the steel bar detection model for re-detection, and the detection results are updated.

[0014] Furthermore, in the seventh step, the conversion method of target grading is adopted. For the composite slab category, by actually measuring the length and width of a series of composite slabs and comparing them with the lengths detected in the image, calculate the average value of the length-width ratio as the conversion ratio.

[0015] Beneficial effects: The present invention realizes high-precision detection and dimensional measurement of precast composite slabs, steel bars and wire ducts, effectively solves the problems of insufficient detection accuracy, distortion influence and large measurement error in the prior art, and at the same time improves the detection efficiency and automation level, meeting the actual engineering requirements. Description of the Drawings

[0016] Figure 1 is a schematic flow chart of a method for accurately detecting precast composite slabs and steel bars by fusing multi-scale features in this embodiment; Figure 2 is a schematic diagram of the improved YOLOV8 model architecture in this embodiment; Figure 3 is a schematic flow chart of pairing the composite slab, its steel bars and wire ducts in this embodiment; Figure 4 is a flow chart for correcting the detection results in this embodiment. Detailed Embodiments

[0017] In order to make the objectives, technical solutions and technical effects of the present invention clearer and more understandable, the present invention will be further described in detail below with reference to the accompanying drawings of the specification and embodiments.

[0018] As Figure 1 shown, this embodiment provides a method for accurately detecting precast composite slabs and steel bars by fusing multi-scale features. This method can accurately measure the real-time dimensions of precast large-size composite slabs and small-size steel bars in the factory, and specifically includes the following steps: Step 1: Set the shooting distance and field of view, and obtain the images captured by the camera.

[0019] Among them, the setting of the shooting distance and field of view is specifically as follows: According to the formula D = f * L / l = f * W / w, where f is the focal length of the camera lens, D is the shooting distance, l and w are the length and width of the camera sensor, and L and W are the maximum length and width of the precast composite slab. Given that the maximum size of the composite slab is 5m × 3m and an industrial camera with a focal length of 12mm is selected, the shooting distance is calculated to be 5.4m, and the field of view size is 5910mm × 3940mm.

[0020] Step 2: Camera calibration and distortion correction.

[0021] Specifically, a large-size camera calibration board is customized with a size of 2m × 1.5m. 15 - 20 calibration images at different angles are taken, and the lens distortion parameters are calculated using a camera calibration tool and distortion correction is completed to ensure the geometric accuracy of the subsequent captured images.

[0022] Step 3: Collect the corrected image data, divide the data set, and send it into the improved model for inference to obtain the preliminary detection results.

[0023] Specifically, 700 images of laminated plates and steel bars are collected using a camera and divided into a training set, a validation set, and a test set according to a certain ratio to provide diverse sample data for model training.

[0024] As Figure 2 shown, the improved model has an architecture improvement based on the YOLOv8 model. A P6 feature extraction layer is newly added in the Backbone network to enhance the detection ability for large-size targets; a P2 feature layer is newly added in the Neck structure and fused with the P3, P4, P5, and P6 feature layers to improve the detection effect for small targets; feature maps with sizes of 160×160 and 10×10 are newly added in the Head structure to improve the detection ability for multi-scale targets.

[0025] Among them, a variety of data augmentation methods, such as Mosaic, Mixup, Gaussian blur, median filtering, etc., are used to expand the diversity of training data, and training is carried out on the improved YOLOv8 model to improve the robustness and detection accuracy of the model.

[0026] The captured image is input into the trained improved YOLOv8 model to obtain preliminary detection results, including the rectangular frame positions and class information of laminated plates, steel bars, and junction boxes.

[0027] Step Four: Matching and optimization of detection results. According to the detection results, pair the laminated plates with their steel bars and junction boxes. As Figure 3 shown, it specifically includes: Step 4.1: Traverse the laminated plate data detected by the model in sequence, including the coordinates of the upper left corner and the lower right corner of the laminated plate and the corresponding confidence level, and judge whether the confidence level, area, and position of the laminated plate meet the set range. If they meet, save the laminated plate data; Step 4.2: If the number of saved laminated plates is not zero, it is necessary to continue to judge the steel bars and junction boxes; Step 4.3: Traverse the steel bar data detected by the model in sequence. Judge whether the length of the steel bar is greater than the width. If so, it is considered a horizontal steel bar, otherwise it is a vertical steel bar. Continuing for this steel bar, traverse the saved laminated plate data in sequence. If it is a horizontal steel bar, first judge whether the y coordinate value of the center coordinate of the steel bar is greater than the y coordinate value of the upper left corner of the laminated plate and less than the y coordinate value of the lower right corner of the laminated plate. If it meets, use the x coordinate values of the upper left corner and the lower right corner of the steel bar to compare with the x coordinate values of the lower right corner and the upper left corner of the laminated plate respectively. If the distance between the upper left corner of the steel bar and the lower right corner of the laminated plate is within the set range, it is determined that the steel bar belongs to the right-side steel bar of the laminated plate, and save the steel bar data into the laminated plate data. Similarly, the membership matching problems of the left-side steel bars and vertical steel bars can be judged. Among them, the upper left corner of the image is set as the coordinate origin, the horizontal direction is the x-axis, and the vertical direction is the y-axis.

[0028] Step 4.4: Traverse the wire box data detected by the model in sequence, and then traverse the saved laminated slab data for each wire box data. Determine whether the wire box is included inside the laminated slab by the upper left and lower right coordinates of the wire box and the upper left and lower right coordinates of the laminated slab. If so, the wire box matches the laminated slab, and save the wire box data into the laminated slab data.

[0029] Step Five: Correction of the detection results. As Figure 4 shown, first traverse and read the laminated slab data, and sequentially obtain all the steel bars on each edge included in the laminated slab. Then use the interquartile range method to judge abnormal steel bars. Taking the judgment process of the upper side steel bars as an example: sort all the y coordinates of the upper side steel bars from small to large, take the data near the 25% position as Q1, and take the data near the 75% position as Q3. Calculate the interquartile range IQR, IQR = Q3 - Q1. Calculate the outlier boundary, lower bound = Q1 - 1.5 * IQR, upper bound = Q3 + 1.5 * IQR. That is, the data below the lower bound and the data above the upper bound are judged as abnormal. Then calculate the absolute error between the average value of the normal steel bars and the corresponding laminated slab edge. If the error is within the set range, the steel bars on this edge do not need to be adjusted. If the error exceeds the set range, adjust the laminated slab edge to the average value position, and then continue to judge the steel bars on other edges.

[0030] Step Six: Re-detection of steel bars. Based on the prior position knowledge of steel bars, that is, the exposed steel bars are often symmetrical, identify the undetected and only partially detected steel bars: if the distance between two steel bars is too large or the length of a certain steel bar is significantly abnormal compared with other steel bars, crop the relevant area, send it into the steel bar detection model for re-detection, and update the detection results.

[0031] Step Seven: Conversion between pixel distance and actual distance. Considering the influence of camera distortion, adopt a conversion method of target classification: for the laminated slab category, by actually measuring the length and width of a series of laminated slabs and comparing with the length detected in the image, calculate the average value of the length-width ratio as the conversion ratio. For the steel bar and wire box categories, use a similar method to calculate the conversion ratio, and adjust the conversion result according to the actual size of the detection target.

[0032] Step Eight: Calculation of the actual length. According to the conversion ratio and the detection results, accurately calculate the actual physical size of the target, providing high-precision data support for subsequent size measurement and recording.

[0033] Through the above improvements and optimizations, the present invention realizes high-precision detection and size measurement of precast laminated slabs, steel bars and wire boxes, effectively solves the problems of insufficient detection accuracy, distortion influence and large measurement error in the prior art, improves the detection efficiency at the same time, and meets the actual engineering requirements.

[0034] The above are only the preferred embodiments of the present invention and do not impose any form of limitation on the present invention. Although the implementation process of the present invention has been described in detail above, those skilled in the art can still modify the technical solutions recorded in the foregoing examples or make equivalent replacements for some of the technical features. Any modifications, equivalent replacements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for accurately detecting prefabricated composite slabs and steel bars by integrating multi-scale features, characterized in that: The following steps are involved: Step 1: Set the shooting distance and field of view to obtain the image captured by the camera; Step 2: Perform camera calibration and distortion correction; Step 3: Collect the corrected image data and divide the data set, send it to the improved YOLOv8 model for reasoning, and obtain preliminary detection results; Step 4: Pair the composite slabs, their steel bars and wire boxes according to the test results; Step 5: Correct the test results; Step 6: Re-test the steel bars that were missed or only partially tested, and update the test results; Step 7: Convert the pixel distance to the actual distance to obtain a conversion ratio; Step 8. Calculate the actual physical size of the target based on the conversion ratio and the detection results.

2. The detection method according to claim 1, characterized in that In step 1, the shooting distance and field of view are set specifically as follows: according to the formula D=f*L / l=f*W / w, wherein f is the focal length of the camera lens, D is the shooting distance, l and w are the length and width of the camera photosensitive element, and L and W are the maximum length and width of the prefabricated composite board.

3. The detection method according to claim 1, characterized in that The step 2 specifically includes: customizing a large-size camera calibration plate with a size of 2m×1.5m, taking a number of calibration images at different angles, and using a camera calibration tool to calculate lens distortion parameters and complete distortion correction.

4. The detection method according to claim 1, characterized in that The improved YOLOv8 model adds a P6 feature extraction layer in the Backbone network; adds a P2 feature layer in the Neck structure and merges it with the P3, P4, P5, and P6 feature layers; and adds feature maps of 160×160 and 10×10 in the Head structure.

5. The detection method according to claim 1, characterized in that The step 4 specifically includes: Step 4.1, sequentially traverse the superimposed plate data detected by the model, including the coordinates of the upper left corner and the lower right corner of the superimposed plate and the corresponding confidence, and determine whether the confidence, area and position of the superimposed plate meet the set range. If yes, save the superimposed plate data; Step 4.2: If the number of saved composite plates is not zero, it is necessary to continue to determine the steel bars and wire boxes; Step 4.3, traverse the steel bar data detected by the model in sequence, and determine whether the length of the steel bar is greater than the width. If so, it is considered to be a transverse steel bar, otherwise it is a longitudinal steel bar. Continue to traverse the saved composite plate data for the steel bar in sequence. If it is a transverse steel bar, first determine whether the y coordinate value of the center coordinate of the steel bar is greater than the y coordinate value of the upper left corner of the composite plate, and less than the y coordinate value of the lower right corner of the composite plate. If it meets the requirements, use the x coordinate values ​​of the upper left corner and lower right corner of the steel bar to correspond to the x coordinate values ​​of the lower right corner and upper left corner of the composite plate respectively for comparison. If the distance between the upper left corner of the steel bar and the lower right corner of the composite plate is within the set range, it is determined that the steel bar belongs to the right steel bar of the composite plate, and the steel bar data is saved in the composite plate data; similarly, determine the belonging matching problem of the left steel bar and the longitudinal steel bar; wherein, the upper left corner of the image is set as the coordinate origin, the horizontal direction is the x-axis, and the vertical direction is the y-axis; Step 4.4, traverse the wire box data detected by the model in sequence, and then traverse the saved superimposed plate data for each wire box data, and determine whether the wire box is contained in the superimposed plate through the coordinates of the upper left corner and lower right corner of the wire box and the upper left corner and lower right corner of the superimposed plate. If so, the wire box matches the superimposed plate, and the wire box data is saved in the superimposed plate data.

6. The detection method according to claim 5, characterized in that The step five specifically includes: first, traverse and read the composite plate data, and obtain all the steel bars of each side of the composite plate in turn; then use the interquartile range method to judge the abnormal steel bars, sort all the y coordinates of the steel bars on one side from small to large, take the data near the 25% position as Q1, take the data near the 75% position as Q3, calculate the interquartile range IQR, IQR=Q3-Q1, calculate the abnormal value boundary, the lower boundary=Q1-1.5*IQR, the upper boundary=Q3+1.5*IQR, that is, the data below the lower boundary and the data above the upper boundary are judged as abnormal; then calculate the average value of the normal steel bars and the absolute error of the corresponding composite plate edge. If the error is within the set range, the steel bars on this side do not need to be adjusted. If the error exceeds the set range, the edge of the composite plate needs to be adjusted to the average value position, and then continue to judge the steel bars on other sides.

7. The detection method according to claim 5, characterized in that: In step six, based on the prior knowledge that the exposed steel bars are in a symmetrical position, if the distance between two steel bars is too large or the length of a certain steel bar is obviously abnormal from the other steel bars, the image is cropped and then sent to the steel bar detection model for re-detection, and the detection result is updated.

8. The detection method according to claim 1, characterized in that In the step seven, a conversion method of target classification is adopted to classify the laminated boards by actually measuring the length and width of a series of laminated boards and comparing them with the length detected in the image, and calculating the average value of the length-to-width ratio as the conversion ratio.

Images