A method and system for quality inspection of precast beam reinforcement cage based on depth vision

Abstract

This invention discloses a method and system for quality inspection of precast beam steel reinforcement cages based on depth vision. The method includes: acquiring depth maps and RGB images of the steel reinforcement cage using a binocular structured light camera; collecting steel reinforcement cage image data, performing preprocessing operations on the raw data, and completing data annotation; dividing the annotated dataset into training, validation, and test sets; using ResNet-FPN as the backbone network of Mask R-CNN to extract multi-scale features of the images; constructing an edge detection model based on ResNet-FPN; integrating the remaining parts of the edge detection model (excluding the backbone network) into the original Mask R-CNN model as edge detection branches; introducing a feature fusion module into the mask branch, which combines shape and position features from the edge detection branch; calculating the actual diameter of the steel reinforcement and the distance between adjacent steel reinforcements based on the steel reinforcement instance segmentation results, and evaluating according to design standards. This method overcomes the problems of existing methods, such as being only applicable to single-layer steel reinforcement meshes and having low computational efficiency.

Description

Technical Field

[0001] This invention relates to the field of bridge engineering, and more particularly to a quality inspection technology for precast beam steel reinforcement cages. Specifically, it relates to a method and system for quality inspection of precast beam steel reinforcement cages based on depth vision. Background Technology

[0002] In modern bridge construction, precast structures have demonstrated numerous advantages, and precast concrete beams have been widely used. The quality of the steel reinforcement in precast beams is crucial to the overall quality of the bridge project, directly affecting key performance indicators such as structural strength, safety, and durability.

[0003] Regarding industry standards, current standards provide clear and specific regulations for key parameters in precast beam reinforcement engineering, such as rebar spacing, rebar cage dimensions, and protective layer thickness. These regulations aim to ensure the quality and performance of precast beams, guaranteeing their safety and reliability during use.

[0004] In the actual production process of precast beams, according to the construction procedure, the reinforcement work must be inspected before concrete pouring. The quality inspection of the reinforcement cage is a crucial part of this inspection process. Currently, in this field, the quality inspection of the reinforcement cage mostly relies on manual methods.

[0005] However, manual inspection has significant drawbacks. First, in terms of production tasks, precast beam plants typically handle a large volume of precast beam production, often requiring the inspection of the steel reinforcement cages of thousands of beams. Manual operation is extremely inefficient, severely hindering production progress and failing to meet the high-efficiency demands of large-scale precast beam production. Second, regarding the accuracy and objectivity of the inspection, manual inspection is inevitably affected by the subjective factors of the inspectors. Different inspectors may produce significantly different results, and due to the limitations of manual measurement, its accuracy is difficult to guarantee. This introduces considerable uncertainty into the quality control of precast beams.

[0006] With the development of technology, computer vision technology has brought new possibilities for solving the above problems, and some research and patents have attempted to apply this technology to the quality inspection of steel reinforcement cages. However, existing inspection schemes based on computer vision technology have many shortcomings. For example, many published patent applications mainly focus on the quality inspection of single-layer steel mesh, while actual precast beams often have multi-layer steel reinforcement cages, and this method cannot meet the actual inspection needs. In addition, some patents use point cloud data to detect steel reinforcement spacing. Although it is theoretically possible to obtain the required data, in practical applications, the complexity of point cloud data processing affects the detection speed, significantly reducing the overall detection efficiency and failing to meet the requirements of fast and efficient inspection in the precast beam production process. Summary of the Invention

[0007] In summary, existing technologies for inspecting the quality of precast beam reinforcement cages are severely inadequate in terms of applicability and efficiency, failing to meet the requirements of high-quality, large-scale production of precast beams in bridge construction. There is an urgent need for an innovative inspection method and system to address the problems of existing technologies and improve the quality and efficiency of precast beam reinforcement cage quality inspection.

[0008] To address the aforementioned deficiencies or improvement needs of existing technologies, as a first aspect of this invention, the present invention provides a method for quality inspection of precast beam reinforcement cages based on depth vision, comprising:

[0009] S1. Obtain depth and RGB images of the steel reinforcement skeleton using a binocular structured light camera;

[0010] S2. Collect steel reinforcement cage image data, perform preprocessing operations such as cropping and noise reduction on the raw data, and then complete data annotation; divide the annotated dataset into training set, validation set and test set.

[0011] S3. Model construction, the specific steps are as follows:

[0012] ResNet-FPN is used as the backbone network of Mask R-CNN to extract multi-scale features of images;

[0013] An edge detection model based on ResNet-FPN was constructed.

[0014] The remaining parts of the edge detection model, excluding the backbone network, are treated as edge detection branches and directly integrated into the original Mask R-CNN model to avoid redundant feature extraction calculations.

[0015] A feature fusion module is introduced into the mask branch, which combines shape and position features from the edge detection branch;

[0016] S4. Quality inspection of the reinforcing steel cage, the specific steps are as follows:

[0017] By improving the mask branch of the Mask R-CNN model, combining the shape and position features provided by the edge detection branch, and using the feature fusion module to perform mask prediction, a mask for each rebar instance is generated, separating the rebar from the background and other interfering elements.

[0018] Based on the results of the reinforcement sample segmentation, the actual diameter of the reinforcement and the distance between adjacent reinforcements are calculated, and an evaluation is performed according to the design standards.

[0019] Furthermore, the depth map described in S1 is divided into three layers: upper layer of reinforcing bars, lower layer of reinforcing bars, and bottom layer of ground or frame. When the reinforcing bar skeleton is photographed from a top-down angle, the plane of the top layer of reinforcing bar mesh is closest to the camera, followed by the plane of the lower layer of reinforcing bars, and the plane of the ground or frame is furthest away.

[0020] Furthermore, the depth map can be modified by extracting the depth range of the lower layer of reinforcing bars and removing the pixels of the lower layer of reinforcing bars to eliminate interference between the upper and lower layers of reinforcing bars when calculating the spacing between them, as shown in the following formula:

[0021] S={(x,y)|d min ≤depth(x,y)≤d max ,(x,y)∈Depth}

[0022]

[0023] In the formula, (x,y) are the pixel coordinates of the depth map, depth(x,y) is the depth value at those coordinates, Mask is a mask of the same size as the RGB image, rgb(x,y) is the pixel value at those coordinates, and d min With d max These represent the upper and lower limits of the depth range where the lower layer of reinforcing bars are located, respectively.

[0024] Furthermore, the output of the ResNet-FPN backbone network in S3 is a feature map of layers P2-P5, where P2 has the highest resolution and P5 has the lowest resolution but contains higher-level semantic information.

[0025] Furthermore, after the corresponding deconvolution operation, the feature maps of layers P2-P5 are restored to their original input size. The four feature maps are merged in the second dimension, and after a 1×1 convolution operation, the number of channels is converted to 1 and input into the sigmoid function to output the edge detection result.

[0026] Furthermore, the feature fusion module introduced in the mask branch as described in S3 is specifically expressed by the following formula:

[0027] F = f(F) b )+F m

[0028] Where F represents the output fusion feature, F b For the input mask features, F m For mask branch features, f represents 1×1 convolution and ReLU activation operations.

[0029] Furthermore, the method for detecting the distance between adjacent reinforcing bars in S4 is as follows:

[0030] After extracting all the top-level steel reinforcement instances, the Zhang-Suen image thinning algorithm is used to extract the center lines of the steel reinforcement instances. Then, one steel reinforcement instance is selected, and the intersection point of the center line of the instance with other center lines is calculated to obtain the pixel coordinates (x, y) of the intersection point.

[0031] The depth value Z of the point is obtained from the depth map acquired by the binocular structured light camera. Substituting it into the following formula, the three-dimensional physical coordinates (X,Y,Z) of the point can be obtained.

[0032] The spacing information of the reinforcing bars can be obtained by calculating the distance between two points in three-dimensional space.

[0033]

[0034] Among them, f x f y c represents the focal length of the camera in the x and y directions, respectively. x and c y The coordinates of the principal point of the camera are given, and the focal length and principal point can be obtained from the camera's intrinsic parameter matrix. u and v are the pixel coordinates of the intersection point.

[0035] Furthermore, the method for detecting the actual diameter of the reinforcing bars in S4 is as follows:

[0036] In the calculation of the diameter of the reinforcing bar, the eight-direction search method is used. Several points are selected on the center line of each reinforcing bar instance. The number of pixels belonging to the reinforcing bar mask is searched in the directions of 0 degrees, 45 degrees, 90 degrees, 135 degrees, 180 degrees, 235 degrees, 270 degrees and 315 degrees respectively, until the edge of the reinforcing bar. The minimum value of the reinforcing bar edge point is calculated according to the following formula.

[0037] Convert the edge to 3D physical coordinates to determine the diameter of the rebar at that point. Average the diameters of non-zero rebars to complete the diameter detection.

[0038] d1 = n1 + n5 + 1

[0039] d2=2 1 / 2 (n² + n⁶ + 1)

[0040] d3 = n3 + n7 + 1

[0041] d4=2 1 / 2 (n⁴ + n⁸ + 1)

[0042] d = min(d1, d2, d3, d4)

[0043] Where n1-n8 represent the number of pixels in 8 directions, d1 represents the pixel width in the 0-degree and 180-degree directions, d2 represents the pixel width in the 45-degree and 235-degree directions, d3 represents the pixel width in the 90-degree and 270-degree directions, d4 represents the pixel width in the 135-degree and 315-degree directions, and d represents the final pixel width.

[0044] According to a second aspect of the present invention, a precast beam reinforcement cage quality inspection system based on depth vision is provided, comprising:

[0045] The image acquisition unit is used to simultaneously acquire depth maps and RGB images of the steel reinforcement skeleton based on a binocular structured light camera;

[0046] The data processing unit is used to collect image data of steel reinforcement cages, perform preprocessing operations such as cropping and noise reduction on the raw data, and then complete data annotation; and divide the annotated dataset into training set, validation set and test set.

[0047] Model building units are used to complete the following steps:

[0048] ResNet-FPN is used as the backbone network of Mask R-CNN to extract multi-scale features of images;

[0049] An edge detection model based on ResNet-FPN was constructed.

[0050] The remaining parts of the edge detection model, excluding the backbone network, are treated as edge detection branches and directly integrated into the original Mask R-CNN model to avoid redundant feature extraction calculations.

[0051] A feature fusion module is introduced into the mask branch, which combines shape and position features from the edge detection branch to help with accurate mask prediction;

[0052] The steel reinforcement cage quality inspection unit is used to complete the following steps:

[0053] By improving the mask branch of the Mask R-CNN model and combining the shape and position features provided by the edge detection branch, the feature fusion module is used to perform mask prediction, generating a mask for each rebar instance, separating the rebar from the background and other interfering elements, and realizing rebar instance segmentation.

[0054] Based on the results of the reinforcement sample segmentation, the actual diameter of the reinforcement and the distance between adjacent reinforcements are calculated, and an evaluation is performed according to the design standards.

[0055] As a third aspect of the invention, a computer-readable storage medium is also provided, having a computer program stored thereon, which is executed by a processor of any step of the method for quality inspection of precast beam reinforcement cages based on depth vision.

[0056] In summary, compared with the prior art, the above-described technical solutions conceived by this invention can achieve the following beneficial effects:

[0057] 1. The depth vision-based precast beam rebar skeleton quality inspection method of the present invention utilizes feature maps of different levels output by ResNet-FPN, and through deconvolution, feature fusion and convolution operations, combined with information from edge detection branches, can more accurately delineate the outline of rebar instances during rebar instance segmentation. Especially for edge details, it effectively overcomes the problem of edge feature loss caused by traditional convolution and pooling operations, improves segmentation accuracy to a new level, and provides a reliable image segmentation foundation for accurate rebar quality inspection.

[0058] 2. The depth vision-based precast beam rebar skeleton quality inspection method of this invention introduces a unique feature fusion module into the mask branch of the improved MaskR-CNN model. This feature fusion module can fuse the shape and position features obtained by the edge detection branch with the features of the mask branch, achieving efficient information integration. In this way, the rebar instance segmentation process is optimized, allowing the model to fully utilize the detailed information obtained by edge detection during rebar instance segmentation. While ensuring overall computational efficiency, it greatly improves the accuracy of instance segmentation. Whether for complete rebar instances or incomplete lower-layer rebar instances that may appear in actual scenarios, it can more accurately delineate their outlines, avoiding segmentation errors caused by missing edge features. Ultimately, it achieves significant technical effects such as improving instance segmentation accuracy, avoiding redundant calculations, and improving model computation and segmentation efficiency, providing a more reliable and efficient tool for the quality inspection of precast beam rebar skeletons. Attached Figure Description

[0059] Figure 1 A flowchart illustrating a better implementation of a depth vision-based method for inspecting the quality of precast beam reinforcement cages;

[0060] Figure 2 A flowchart for the best implementation of steel reinforcement cage quality inspection;

[0061] Figure 3 A schematic diagram of a steel reinforcement cage model for optimal implementation;

[0062] Figure 4 A schematic diagram illustrating the optimal implementation of lower-layer rebar removal;

[0063] Figure 5A schematic diagram of the depth value distribution range for optimal implementation;

[0064] Figure 6 A schematic diagram of a better implementation of the rebar instance segmentation dataset;

[0065] Figure 7 A schematic diagram of a ResNet-FPN-based edge detection model for optimal implementation;

[0066] Figure 8 A schematic diagram illustrating the edge detection results of a better implementation;

[0067] Figure 9 A schematic diagram of the improved Mask R-CNN model for better implementation;

[0068] Figure 10 A comparative schematic diagram of rebar segmentation for better implementation;

[0069] Figure 11 A schematic diagram illustrating the quality inspection results of the reinforced concrete cage for optimal implementation;

[0070] Figure 12 A schematic diagram of a depth vision-based precast beam reinforcement cage quality inspection system for optimal implementation. Detailed Implementation

[0071] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0072] Example 1

[0073] Please refer to Figure 1 This embodiment 1 provides a method for quality inspection of precast beam reinforcement cages based on depth vision, including:

[0074] (1) Image Acquisition

[0075] The depth map and RGB image of the steel reinforcement skeleton are obtained using a binocular structured light camera. The specific method is as follows:

[0076] At the precast beam production site, a binocular structured light camera was installed. The camera's position and angle were carefully adjusted to ensure a complete and clear image of the precast beam's steel reinforcement cage from a top-down perspective. The binocular structured light camera simultaneously acquired depth and RGB images of the steel reinforcement cage.

[0077] Please refer to Figure 3 as well as Figure 4The depth map clearly shows a three-layer structure: upper reinforcing steel, lower reinforcing steel, and the ground or support frame at the bottom. Due to the shooting angle, the top layer of reinforcing steel mesh is closest to the camera, followed by the lower layer, and the ground or support frame is furthest away. Figure 4 The left-middle image shows the point cloud generated from the depth map and RGB map, which is only for demonstration purposes. It can be clearly observed that the point cloud of the steel skeleton is divided into three layers.

[0078] Please refer to Figure 5 When photographing the reinforcing steel frame from a fixed position, the distances between the top layer of reinforcing steel, the bottom layer of reinforcing steel, the ground, or the frame and the camera can be measured, allowing the depth range [dmin, dmax] of the bottom layer of reinforcing steel to be obtained. When photographing from a non-fixed position, the depth range [dmin, dmax] of the bottom layer of reinforcing steel can be obtained by generating a depth value distribution range.

[0079] Then, by utilizing the depth map information and extracting the depth range of the lower-layer rebar, pixels of the lower-layer rebar are removed to eliminate interference between the upper and lower layers of rebar during subsequent rebar spacing calculations. The specific steps are as follows:

[0080] Given a depth map named Depth with pixel coordinates (x, y) and a depth value of depth(x, y), and a mask of the same size as the RGB image named Mask with pixel values ​​of rgb(x, y) in the RGB image at the same coordinates, remove the lower layer rebar pixels using the following formula:

[0081]

[0082] Furthermore, the specific process can be expressed by the following formula:

[0083] S={(x,y)|d min ≤depth(x,y)≤d max ,(x,y)∈Depth}

[0084]

[0085] In the formula, (x,y) are the pixel coordinates of the depth map, depth(x,y) is the depth value at those coordinates, Mask is a mask of the same size as the RGB image, rgb(x,y) is the pixel value at those coordinates, and d min With d max These represent the upper and lower limits of the depth range where the lower layer of reinforcing bars are located, respectively.

[0086] (2) Data processing

[0087] Please refer to Figure 6We collected images of steel reinforcement cages and preprocessed the raw data, including cropping and noise reduction. We then used the LabelMe annotation tool to perform semantic segmentation and annotation on the raw data. After completing the annotation, we checked the accuracy of the annotations. Finally, we split the dataset into training, validation, and test sets.

[0088] Cropping is an important preprocessing step for the acquired raw images. Because images may contain a lot of background information during capture, such as the supporting structure around the rebar cage, ground debris, and the workshop environment, this information is not necessary for rebar cage quality inspection and may interfere with subsequent analysis. Therefore, cropping removes these irrelevant areas, retaining only the core area containing the rebar cage.

[0089] Specifically, image cropping algorithms are employed, cropping based on pre-defined boundary conditions or automatically identified boundary ranges. For example, by analyzing the image's grayscale histogram and using edge detection algorithms (such as Canny edge detection), the approximate range of the reinforcing steel skeleton is identified. The image is then precisely cropped based on this range, ensuring that the cropped image contains only the main reinforcing steel skeleton information. During the cropping process, variations in the size of different precast beams and the position of the reinforcing steel skeleton need to be considered. Therefore, it may be necessary to dynamically adjust the cropping parameters to ensure effective removal of irrelevant areas under different conditions.

[0090] In some other preferred embodiments, due to the complex environment of actual production workshops, the acquired images are often affected by noise. Noise may originate from factors such as uneven lighting, thermal noise of the camera sensor, and electromagnetic interference, which can reduce image quality and affect subsequent feature extraction and model processing. In other embodiments, Gaussian filtering algorithms are used for noise reduction.

[0091] Gaussian filtering is a linear smoothing filter that smooths out noise by performing a weighted average of each pixel and its neighboring pixels in an image. For each pixel, its value is updated based on the weighted average of its neighboring pixels. The weights are determined by a Gaussian function, with pixels closer to the center pixel having a higher weight and pixels farther away having a lower weight. This allows for the removal of noise while preserving as much edge and detail information as possible in the image.

[0092] In some preferred embodiments, after cropping and denoising, the labeled dataset is divided into a training set, a validation set, and a test set. The division ratio is usually determined based on actual conditions and experience; some embodiments use a 7:2:1 ratio.

[0093] The training set was used to train the improved Mask R-CNN model. Through a large amount of data input, the model learned the feature representations and patterns of the rebar skeleton. During training, the model continuously adjusted its parameters to minimize the loss function and improve its ability to identify and segment the rebar skeleton.

[0094] The validation set is used to evaluate the model's performance during training and monitor whether the model is overfitting or underfitting. At each training cycle or stage, the validation set is input into the model, and the model's hyperparameters, such as learning rate, number of network layers, and regularization parameters, are adjusted based on the model's performance metrics on the validation set (such as accuracy, recall, F1 score, etc.) to ensure that the model has good generalization ability.

[0095] The test set is used to finally evaluate the model's performance. After the model is trained, the test set is used to test the model and evaluate its performance on unseen data, ensuring that the model can have reliable performance in actual steel reinforcement cage quality inspection tasks.

[0096] (3) Model building

[0097] Please refer to Figure 7 This method uses Mask R-CNN as the base model to achieve instance segmentation of rebar. It detects the spacing between rebar skeletons and the diameter of the rebars, requiring higher accuracy in the rebar instance segmentation results and ensuring clear edge details. In image segmentation, edge features, as shallow features, are gradually lost during multiple convolution and pooling operations, resulting in relatively coarse edge details in the segmentation results.

[0098] In this embodiment, ResNet-FPN is used as the backbone network for Mask R-CNN, which outputs feature maps at layers P2-P5. Among them, the feature map at layer P2 has the highest resolution and can preserve rich details of the image; the feature map at layer P5 has the lowest resolution but contains higher-level semantic information, which helps to understand the overall structure of the steel reinforcement.

[0099] Furthermore, in this embodiment 1, the edge detection model is constructed by performing corresponding deconvolution operations on the feature maps of layers P2-P5 to restore the feature map size to the original input size. Then, these four feature maps are merged in the second dimension, followed by a 1×1 convolution operation to convert the number of channels to 1, and finally input into the sigmoid function to output the edge detection result, thus completing the construction of the ResNet-FPN-based edge detection model.

[0100] Please refer to Figure 8 The actual effect of the edge detection model in Embodiment 1 can be clearly seen.

[0101] Please refer to Figure 9 Furthermore, the remaining parts of the constructed edge detection model, excluding the backbone network, are directly integrated into the original Mask R-CNN model as edge detection branches. This effectively avoids redundant feature extraction calculations and improves the computational efficiency of the model.

[0102] Since the edges and the mask are closely related, a feature fusion module is introduced into the mask branch. The shape and position features in the edge detection branch help to make accurate predictions of the mask.

[0103] F = f(F) b )+F m

[0104] Where F represents the output fusion feature, F b For the input mask features, F m This represents the mask branch features. `f` denotes a 1×1 convolution and ReLU activation operation. This feature fusion module combines shape and position features from the edge detection branch to achieve accurate mask prediction.

[0105] Please refer to Figure 10 Due to limitations in depth camera accuracy and environmental conditions, pixels from the lower-layer rebar cannot be completely removed. The improved Mask R-CNN model combines low-level edge features with high-level semantic features in its mask detection branch, making the rebar instance segmentation result more reliant on the edge features of complete rebar instances. For lower-layer rebar instances where pixels were not completely removed, the edge features of the entire rebar cannot be obtained, leading to the loss of some edge features during mask branch prediction. This results in the lower-layer rebar instances not being classified as rebars. Therefore, the introduced edge detection branch has a suppressive effect on the segmentation results of incomplete lower-layer rebar instances.

[0106] (4) Quality inspection of steel reinforcement cage

[0107] 4.1 Reinforcing bar example segmentation

[0108] By improving the mask branch of the Mask R-CNN model and combining the shape and position features provided by the edge detection branch, the feature fusion module is used to perform mask prediction, generating a mask for each rebar instance, separating the rebar from the background and other interfering elements, and realizing rebar instance segmentation.

[0109] 4.2 Rebar Spacing Inspection

[0110] In this embodiment 1, after all the top-level steel reinforcement instances are extracted, the Zhang-Suen image thinning algorithm is used to extract the center line of the steel reinforcement instances. Then, one steel reinforcement instance is selected, and the intersection point of the center line of the instance with other center lines is calculated to obtain the pixel coordinates (x, y) of the intersection point.

[0111] The depth value Z of the point is obtained from the depth map acquired by the binocular structured light camera. Substituting it into the following formula, the three-dimensional physical coordinates (X,Y,Z) of the point can be obtained.

[0112] The spacing information of the reinforcing bars can be obtained by calculating the distance between two points in three-dimensional space.

[0113]

[0114] Among them, f x f y c represents the focal length of the camera in the x and y directions, respectively. x and c y The coordinates of the principal point of the camera are given, and the focal length and principal point can be obtained from the camera's intrinsic parameter matrix. u and v are the pixel coordinates of the intersection point.

[0115] In other preferred embodiments, deep learning techniques are utilized, such as training a specialized convolutional neural network (CNN) to automatically detect feature points on the rebar, rather than relying solely on the Zhang-Suen image thinning algorithm. Annotated rebar images can be used as the training set, where feature points can be manually annotated at key locations on the rebar (such as rebar endpoints, bends, or equally spaced markers).

[0116] The trained CNN network can be a similar object detection network (such as YOLO, SSD, etc.), modified specifically for detecting feature points on rebar. The network's input is a preprocessed image of the rebar, and its output is the location and confidence score of the feature points. By learning from a large amount of training data, the network can automatically learn the feature representation of the rebar and accurately locate the feature points.

[0117] After detecting feature points on different rebar instances, these feature points need to be matched and grouped to determine which feature points belong to the same rebar or the same row of rebars. Deep learning-based feature descriptors (such as the Siamese network) can be used to generate a feature vector for each feature point.

[0118] Furthermore, for feature vectors of different feature points, similarity metrics (such as cosine similarity, Euclidean distance, etc.) are used for matching. Feature points with high similarity are grouped to ensure that feature points belonging to the same rebar or the same row of rebars are correctly associated.

[0119] For steel bars that are partially occluded or deformed, deep learning methods are more robust because the network can learn the features of the steel bars under different conditions, rather than relying on simple image thinning algorithms, thereby improving the accuracy of feature point detection and matching.

[0120] 4.3 Rebar Diameter Inspection

[0121] In this embodiment 1, the eight-direction search method is used to calculate the diameter of the reinforcing bar. Several points are selected on the center line of each reinforcing bar instance. The number of pixels belonging to the reinforcing bar mask is searched in the directions of 0 degrees, 45 degrees, 90 degrees, 135 degrees, 180 degrees, 235 degrees, 270 degrees and 315 degrees for each point until the edge of the reinforcing bar. The minimum value of the reinforcing bar edge point is calculated according to the following formula.

[0122] Convert the edge to 3D physical coordinates to determine the diameter of the rebar at that point. Average the diameters of non-zero rebars to complete the diameter detection.

[0123] d1 = n1 + n5 + 1

[0124] d2=2 1 / 2 (n² + n⁶ + 1)

[0125] d3 = n3 + n7 + 1

[0126] d4=2 1 / 2 (n⁴ + n⁸ + 1)

[0127] d = min(d1, d2, d3, d4)

[0128] Where n1-n8 represent the number of pixels in 8 directions, d1 represents the pixel width in the 0-degree and 180-degree directions, d2 represents the pixel width in the 45-degree and 235-degree directions, d3 represents the pixel width in the 90-degree and 270-degree directions, d4 represents the pixel width in the 135-degree and 315-degree directions, and d represents the final pixel width.

[0129] Example 2

[0130] Please refer to Figure 12 This embodiment 2 provides a precast beam reinforcement cage quality inspection system based on depth vision, including:

[0131] The image acquisition unit is used to simultaneously acquire depth maps and RGB images of the steel reinforcement skeleton based on a binocular structured light camera;

[0132] The data processing unit is used to collect image data of steel reinforcement cages, perform preprocessing operations such as cropping and noise reduction on the raw data, and then complete data annotation; and divide the annotated dataset into training set, validation set and test set.

[0133] Model building units are used to complete the following steps:

[0134] ResNet-FPN is used as the backbone network of Mask R-CNN to extract multi-scale features of images;

[0135] An edge detection model based on ResNet-FPN was constructed.

[0136] The remaining parts of the edge detection model, excluding the backbone network, are treated as edge detection branches and directly integrated into the original Mask R-CNN model to avoid redundant feature extraction calculations.

[0137] A feature fusion module is introduced into the mask branch, which combines shape and position features from the edge detection branch to help with accurate mask prediction;

[0138] The steel reinforcement cage quality inspection unit is used to complete the following steps:

[0139] By improving the mask branch of the Mask R-CNN model and combining the shape and position features provided by the edge detection branch, the feature fusion module is used to perform mask prediction, generating a mask for each rebar instance, separating the rebar from the background and other interfering elements, and realizing rebar instance segmentation.

[0140] Based on the results of the reinforcement sample segmentation, the actual diameter of the reinforcement and the distance between adjacent reinforcements are calculated, and an evaluation is performed according to the design standards.

[0141] Example 3

[0142] This embodiment 3 also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can realize a method for quality inspection of precast beam steel reinforcement skeleton based on depth vision.

[0143] The computer-readable storage medium may include various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0144] For a description of the computer-readable storage medium provided in this application, please refer to the above method embodiments; further details will not be repeated here.

[0145] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for quality inspection of precast beam reinforcement cage based on depth vision, characterized in that, include: S1. Obtain depth and RGB images of the steel reinforcement skeleton using a binocular structured light camera; S2. Collect steel reinforcement cage image data, perform preprocessing operations such as cropping and noise reduction on the raw data, and then complete data annotation; divide the annotated dataset into training set, validation set and test set. S3. Model construction, the specific steps are as follows: ResNet-FPN is used as the backbone network of Mask R-CNN to extract multi-scale features of images; An edge detection model is built based on ResNet-FPN; The remaining parts of the edge detection model, excluding the backbone network, are treated as edge detection branches and directly integrated into the original Mask R-CNN model to avoid redundant feature extraction calculations. A feature fusion module is introduced into the mask branch, which combines shape and position features from the edge detection branch; S4. Quality inspection of the reinforcing steel cage, the specific steps are as follows: By improving the mask branch of the Mask R-CNN model, combining the shape and position features provided by the edge detection branch, and using the feature fusion module to perform mask prediction, a mask for each rebar instance is generated, separating the rebar from the background and other interfering elements. Based on the results of the reinforcement sample segmentation, the actual diameter of the reinforcement and the distance between adjacent reinforcements are calculated, and an evaluation is performed according to the design standards.

2. The method for quality inspection of precast beam reinforcement cage based on depth vision according to claim 1, characterized in that, The depth map described in S1 is divided into three layers: upper layer of reinforcing bars, lower layer of reinforcing bars, and bottom layer of ground or frame. When the reinforcing bar skeleton is photographed from a top-down angle, the plane of the top layer of reinforcing bar mesh is closest to the camera, followed by the plane of the lower layer of reinforcing bars, and the plane of the ground or frame is furthest away.

3. The method for quality inspection of precast beam reinforcement cage based on depth vision according to claim 2, characterized in that, The depth map extracts the depth range of the lower layer of reinforcing bars and removes the pixels of the lower layer of reinforcing bars to eliminate interference between the upper and lower layers of reinforcing bars when calculating the reinforcing bar spacing, as shown in the following formula: S={(x,y)|d min ≤depth(x,y)≤d max ,(x,y)∈Depth} In the formula, (x,y) are the pixel coordinates of the depth map, depth(x,y) is the depth value at those coordinates, Mask(x,y) is a mask of the same size as the RGB image, rgb(x,y) is the pixel value at those coordinates, and d min With d max These represent the upper and lower limits of the depth range where the lower layer of reinforcing bars are located, respectively.

4. The method for quality inspection of precast beam reinforcement cage based on depth vision according to claim 1, characterized in that, The output of the ResNet-FPN backbone network in S3 is the feature map of layers P2-P5, where P2 has the highest resolution and P5 has the lowest resolution but contains higher-level semantic information.

5. The method for quality inspection of precast beam reinforcement cage based on depth vision according to claim 4, characterized in that, After the corresponding deconvolution operation, the feature maps of layers P2-P5 are restored to their original input size. The four feature maps are merged in the second dimension, and after a 1×1 convolution operation, the number of channels is converted to 1 and input into the sigmoid function to output the edge detection result.

6. The method for quality inspection of precast beam reinforcement cage based on depth vision according to claim 1, characterized in that, The feature fusion module introduced in the mask branch as described in S3 is specifically expressed by the following formula: F=f(F b )+F m Where F represents the output fusion feature, F b For the input mask features, F m For mask branch features, f represents 1×1 convolution and ReLU activation operations.

7. The method for quality inspection of precast beam reinforcement cage based on depth vision according to claim 1, characterized in that, The method for detecting the distance between adjacent reinforcing bars in S4 is as follows: After extracting all the top-level steel reinforcement instances, the Zhang-Suen image thinning algorithm is used to extract the center lines of the steel reinforcement instances. Then, one steel reinforcement instance is selected, and the intersection point of the center line of the instance with other center lines is calculated to obtain the pixel coordinates (x, y) of the intersection point. The depth value Z of the point is obtained from the depth map acquired by the binocular structured light camera. Substituting it into the following formula, the three-dimensional physical coordinates (X,Y,Z) of the point can be obtained. The spacing information of the reinforcing bars can be obtained by calculating the distance between two points in three-dimensional space. Among them, f x f y c represents the focal length of the camera in the x and y directions, respectively. x and c y The coordinates of the camera's principal point are given, and the focal length and principal point can be obtained from the camera's intrinsic parameter matrix. u and v are the pixel coordinates of the intersection point.

8. The method for quality inspection of precast beam reinforcement cage based on depth vision according to claim 1, characterized in that, The method for calculating the actual diameter of the reinforcing bar in S4 is as follows: In the calculation of the diameter of the reinforcing bar, the eight-direction search method is used. Several points are selected on the center line of each reinforcing bar instance. The number of pixels belonging to the reinforcing bar mask is searched in the directions of 0 degrees, 45 degrees, 90 degrees, 135 degrees, 180 degrees, 235 degrees, 270 degrees and 315 degrees respectively, until the edge of the reinforcing bar. The minimum value of the reinforcing bar edge point is calculated according to the following formula. Convert the edge to three-dimensional physical coordinates, calculate the diameter of the rebar at that point, and take the average of the diameters of non-zero rebars to complete the diameter detection: d1 = n1 + n5 + 1 d2=2 1 / 2 (n2+n6+1) d3 = n3 + n7 + 1 d4=2 1 / 2 (n4+n8+1) d = min(d1, d2, d3, d4) Where n1-n8 represent the number of pixels in 8 directions, d1 represents the pixel width in the 0-degree and 180-degree directions, d2 represents the pixel width in the 45-degree and 235-degree directions, d3 represents the pixel width in the 90-degree and 270-degree directions, d4 represents the pixel width in the 135-degree and 315-degree directions, and d represents the final pixel width.

9. A quality inspection system for the reinforcing steel skeleton of precast beams based on depth vision, characterized in that, include: The image acquisition unit is used to simultaneously acquire depth maps and RGB images of the steel reinforcement skeleton based on a binocular structured light camera; The data processing unit is used to collect image data of steel reinforcement cages, perform preprocessing operations such as cropping and noise reduction on the raw data, and then complete data annotation; and divide the annotated dataset into training set, validation set and test set. Model building units are used to complete the following steps: ResNet-FPN is used as the backbone network of Mask R-CNN to extract multi-scale features of images; An edge detection model is built based on ResNet-FPN; The remaining parts of the edge detection model, excluding the backbone network, are treated as edge detection branches and directly integrated into the original Mask R-CNN model to avoid redundant feature extraction calculations. A feature fusion module is introduced into the mask branch, which combines shape and position features from the edge detection branch to help with accurate mask prediction; The steel reinforcement cage quality inspection unit is used to complete the following steps: By improving the mask branch of the Mask R-CNN model and combining the shape and position features provided by the edge detection branch, the feature fusion module is used to perform mask prediction, generating a mask for each rebar instance, separating the rebar from the background and other interfering elements, and realizing rebar instance segmentation. Based on the results of the reinforcement sample segmentation, the actual diameter of the reinforcement and the distance between adjacent reinforcements are calculated, and an evaluation is performed according to the design standards.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, The computer program is executed by a processor using the method for quality inspection of precast beam reinforcement skeleton based on depth vision, as described in any one of claims 1-8.

Images