Intelligent reinforcement detection method and system based on convolutional neural network and binocular vision

Abstract

The application discloses an intelligent reinforcement detection method and system based on a convolutional neural network and binocular vision. The method comprises the following steps: S1: using a depth camera to collect RGB image and depth image data of the reinforcement; S2: inputting the RGB image of the reinforcement into a convolutional neural network for reinforcement recognition, and obtaining a predicted bounding box and a mask of the reinforcement; and S3: based on the reinforcement recognition result, using binocular vision technology for reinforcement detection, and outputting a visual reinforcement quality acceptance result. The application uses the convolutional neural network for reinforcement recognition, improves the accuracy of reinforcement target detection and segmentation, and combines the binocular vision technology, so that the intelligent reinforcement detection has real-time performance, simultaneously outputs a visual result, assists workers in quality acceptance in a reinforcement hidden engineering, greatly improves work efficiency, and reduces labor cost.

Description

Technical Field

[0001] This invention relates to the fields of deep learning and computer vision technology, specifically to an intelligent reinforcement detection method and system based on convolutional neural networks and binocular vision. Background Technology

[0002] Reinforced concrete structures are the most widely used structural form in engineering projects due to their advantages such as readily available materials, high plasticity, reasonable material stress distribution, simple construction technology, and low cost. During structural design, the load-bearing capacity of components is ensured by controlling the diameter and spacing of the reinforcing bars. Before concrete pouring, a concealed works inspection of the reinforcing bars is required, which involves checking whether the specifications, quantity, and spacing of the reinforcing bars meet the design requirements. Traditional reinforcing bar inspection mainly relies on manual measurement methods, which not only have limitations in accuracy and scope but also pose safety hazards during on-site inspections. Given the labor shortage and aging workforce, traditional reinforcing bar inspection methods urgently need to be transformed into intelligent systems.

[0003] Rebar detection methods based on digital image processing technology are easily affected by factors such as lighting, background, and occlusion, resulting in accuracy that cannot meet the requirements of practical engineering projects and poor real-time detection performance. With the development of laser scanning technology and equipment, methods using 3D point clouds to achieve high-precision measurement have been widely used in the field of civil engineering. However, the high cost of laser scanning equipment and the cumbersome data acquisition and calculation process limit the practical application of this method.

[0004] In recent years, due to their powerful feature learning capabilities, object detection and instance segmentation algorithms based on convolutional neural networks have been widely used and have achieved good results, such as precast component identification, rebar tying point positioning, and rebar section counting.

[0005] The patent specification with publication number CN113269718A discloses a method for detecting cracks in precast concrete components based on deep learning. The method involves collecting crack image data, preprocessing the collected crack image samples, and manually annotating them; augmenting the annotated data samples and dividing them into training, validation, and test sets; building a convolutional neural network model; training, validating, and testing the convolutional neural network model to obtain the final algorithm model; and using the algorithm model to detect the crack images to be detected to obtain the detection results.

[0006] The patent specification with publication number CN115222652A discloses a method for identifying, counting, and centering the end faces of bundled steel bars. The method includes S1, capturing an image of the end face of the steel bars and processing it to obtain an image to be identified; S2, performing data augmentation on the image to be identified using a first preset algorithm; S3, forming a final detection box in the image to be identified using a second preset algorithm with a lightweight convolutional neural network and calculating the number of the final detection boxes; and S4, generating a counting result. Summary of the Invention

[0007] To address the shortcomings of existing rebar detection technologies, this invention provides an intelligent rebar detection method based on convolutional neural networks and binocular vision. It utilizes an improved Mask R-CNN instance segmentation model to enhance the accuracy of rebar identification and combines binocular vision technology to output visualized rebar detection results. This method assists workers in quality acceptance during concealed rebar projects, significantly improving work efficiency and reducing labor costs.

[0008] A smart reinforcement detection method based on convolutional neural networks and binocular vision includes the following steps:

[0009] S1: Use a depth camera to acquire RGB and depth image data of the reinforcing bars;

[0010] S2: Input the RGB image of the rebar into the convolutional neural network to identify the rebar and obtain the predicted bounding box and mask of the rebar;

[0011] S3: Based on the rebar identification results, use binocular vision technology to detect the reinforcement and output visualized rebar quality acceptance results.

[0012] In a preferred embodiment, in step S1, depth image data of the reinforcing bars is obtained through the following steps:

[0013] S1.1: Perform stereo matching on the left and right eye images of the depth camera to obtain a disparity map;

[0014] S1.2: Based on the relationship between depth and disparity, the disparity map is converted into a depth map. The formula for calculating depth z is as follows:

[0015]

[0016] Where f is the focal length of the depth camera, b is the baseline length of the depth camera, d is the parallax between the left and right eye images, and x l Let x be the x-coordinate of the projection point of the left eye camera. r The x-coordinate of the projection point of the right eye camera.

[0017] In a preferred embodiment, step S2 specifically includes the following steps:

[0018] S2.1: Use camera equipment to collect original images of steel bars, use manual annotation to create steel bar mask labels, divide the dataset into training set and test set, and augment the dataset through data augmentation;

[0019] S2.2: Pre-train the improved Mask R-CNN model using the public dataset COCO2017, and initialize the network parameters based on the principle of transfer learning;

[0020] S2.3: Train the improved Mask R-CNN model established in step S2.2 using the dataset from step S2.1 to construct a rebar instance segmentation model;

[0021] S2.4: Input the RGB image of the rebar acquired by the depth camera into the rebar instance segmentation model to obtain the predicted bounding box and mask of the rebar.

[0022] In step S2.1, the data augmentation refers to geometric transformation operations such as random translation, rotation, mirroring, and affine transformation, as well as pixel transformation operations such as randomly adjusting brightness, contrast, HSV, adding Gaussian noise, and adding salt-and-pepper noise.

[0023] More preferably, in step S2.2, the improved Mask R-CNN model includes an optimized feature extraction module, an RPN module, an ROI alignment module, and an output branch;

[0024] The optimized feature extraction module is a CA-SA module, consisting of a bottom-up propagation path and a channel attention (CA) module and a spatial attention (SA) module, added to the ResNet-FPN feature pyramid structure based on residual networks. In the CA module, a feature map with height H, width W, and number of channels C is input to a global average pooling layer. The spatial dimensions W and H are compressed to unit 1. The resulting 1×1×C feature map is then convolved and processed with softmax to make the sum of the channels equal to 1. The output at this point is the attention mechanism weight for each channel. This weight is multiplied by the corresponding channel of the input feature map to obtain the output feature map. In the SA module, after the feature map undergoes 1×1 convolution and softmax processing, the channel dimensions are compressed. For unit 1, the SA module learns the weight matrix of the H×W size feature map on the two-dimensional plane. This weight matrix corresponds to the spatial attention mechanism weight of each pixel, representing the importance of spatial location information. The input feature map is given this weight matrix to amplify important features and weaken background information, thereby achieving the effect of feature selection and enhancement. The bottom-up propagation path specifically refers to: based on the {P2, P3, P4, P5} feature map obtained by ResNet-FPN, the feature information of P2 is passed to N2. N2 performs a 3×3 convolution to downsample the height and width to the size of P3, and then adds it element by element to P3 and sends it to the CA-SA module to obtain N3. In this way, N4 and N5 are further extracted from the P4 and P5 feature maps to obtain the {N2, N3, N4, N5} feature map.

[0025] The image is processed by the optimized feature extraction module to generate a feature map. The RPN module further generates strong prior anchor boxes for each point on the feature map, and obtains the classification score and bounding box regression value of the anchor boxes through 1×1 convolution, thereby selecting a set of better candidate boxes and inputting them into the ROI alignment module. The ROI alignment module transforms the feature map generated by the optimized feature extraction module and the candidate boxes selected by the RPN module to the same dimension to meet the requirements of the subsequent fully convolutional network for input features. Finally, the features obtained by the ROI alignment module are input into the fully connected layer, and the predicted class score, bounding box regression value and pixel mask of the object are output in the classification branch, bounding box regression branch and mask branch, respectively, thereby completing the entire detection and segmentation task.

[0026] Further preferred, step S2.3 specifically includes the following steps:

[0027] S2.3.1: Set the evaluation metric for the improved Mask R-CNN model to the mAP calculated using the defined method on the COCO2017 dataset, which is the mean of the sum of all average precision APs under different intersection-over-union (IoU) thresholds (0.50: 0.05: 0.95). The specific formula for calculating AP is as follows:

[0028]

[0029] Where P represents the proportion of all predicted steel reinforcement targets that are correctly predicted, and R represents the proportion of all actually correct steel reinforcement targets that are predicted as positive samples.

[0030] S2.3.2: Input and output the rebar images and rebar mask labels from the training set into the improved Mask R-CNN model to obtain the weight parameters of the rebar instance segmentation model; use the obtained weight parameters to predict the bounding boxes and masks of the rebar images in the test set, calculate the loss and mAP index by comparing the results with the ground truth values ​​of the rebar mask labels, and adjust the weight parameters accordingly; until the training is completed, obtain the weight parameter corresponding to the maximum mAP, and use this weight coefficient for subsequent rebar recognition.

[0031] In a preferred embodiment, step S3 specifically includes the following steps:

[0032] S3.1: Based on the steel bar recognition results, the mask images are binarized, the edge detection algorithm is used to extract the pixel coordinates of the edges, and then the pixel coordinates of the centerline of each mask are calculated using the median transformation principle.

[0033] S3.2: Calculate the normal vector of the centerline of each mask using the k-nearest neighbor algorithm. Assign masks with the horizontal component of the centerline normal vector less than the vertical component to the vertical direction, and assign masks with the horizontal component of the centerline normal vector greater than or equal to the vertical component to the horizontal direction. Then, using the centerline coordinates, automatically sort the rebar masks in the order from top to bottom and from left to right.

[0034] S3.3: Extend the normal vector of the pixel point along the center line of the mask to both sides of the edge, and use linear interpolation to extract the paired pixels of the edge, which are used to calculate the diameter of the rebar; similarly, extend the normal vector of the pixel point along the center line of the mask to the center line of the adjacent mask, and extract the paired pixels of the center line of the mask, which are used to calculate the spacing of the rebar.

[0035] S3.4: Align the RGB image of the steel bar and the depth image acquired by the depth camera to obtain the depth information of each pixel in the RGB image; using the camera intrinsic parameter matrix, transform the paired pixels of the edge and centerline extracted in S3.3 from the pixel coordinate system to the camera coordinate system;

[0036] S3.5: Substitute the camera coordinates of the paired pixels of the rebar edge and centerline obtained in S3.4 into the spatial distance formula to calculate the actual rebar diameter and spacing, and output the visualized rebar quality acceptance results.

[0037] More preferably, in step S3.4, the specific formula for the camera intrinsic parameter matrix is:

[0038]

[0039] Where M represents the camera intrinsic parameter matrix, (u0, v0) are the coordinates of the center point of the RGB image in the pixel coordinate system, and d x d y Here, f represents the length of a single pixel along the x and y axes, and f is the focal length of the depth camera. x f y This represents the number of pixels per length f on the x-axis and y-axis of the imaging plane.

[0040] More preferably, in step S3.5, the spatial distance formula is specifically as follows:

[0041]

[0042] Where D is the distance between any two points in space, x1, y1, z1 are the x, y, z coordinates of point 1, and x2, y2, z2 are the x, y, z coordinates of point 2.

[0043] The present invention also provides an intelligent reinforcement detection system based on convolutional neural networks and binocular vision, the system being able to execute the aforementioned intelligent reinforcement detection method based on convolutional neural networks and binocular vision.

[0044] Compared with the prior art, the beneficial effects of this invention are as follows:

[0045] (1) This invention improves Mask R-CNN by adding a bottom-up propagation path to its feature extraction module and embedding a CA-SA module that combines channel attention and spatial attention. While strengthening the fusion of shallow and deep feature information, the channel attention mechanism assigns a larger weight coefficient to the channel with high target response, and the spatial attention mechanism makes the target pixel the focus of feature extraction, thereby improving the accuracy of steel bar target detection and segmentation.

[0046] (2) This invention combines binocular vision technology to enable real-time intelligent reinforcement detection and output visualized results, which assists workers in quality acceptance in the concealed reinforcement project, greatly improves work efficiency and reduces labor costs. Attached Figure Description

[0047] Figure 1 This is a flowchart illustrating the intelligent reinforcement detection method based on convolutional neural networks and binocular vision according to the present invention.

[0048] Figure 2 The image data of the reinforcing bars acquired by the depth camera in the example are shown in (a) and (b).

[0049] Figure 3 This is a schematic diagram of the Mask R-CNN network structure;

[0050] Figure 4 Here is a structural diagram of the channel attention mechanism CA module for an example;

[0051] Figure 5 Here is a structural diagram of the spatial attention mechanism (SA) module in the embodiment;

[0052] Figure 6 This is a bottom-up attention mechanism path structure diagram for an example.

[0053] Figure 7 The output results of the intelligent reinforcement detection method in the example are shown in Figure (a), where (a) is the reinforcement prediction result based on the improved Mask R-CNN, and (b) is the visualization result of the reinforcement quality detection, which intuitively shows the diameter and spacing of the reinforcement. Detailed Implementation

[0054] The present invention will be further described below with reference to the accompanying drawings and specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention.

[0055] like Figure 1 As shown, an intelligent reinforcement detection method based on convolutional neural networks and binocular vision includes the following steps:

[0056] S1: Use a depth camera to acquire RGB and depth image data of the reinforcing bars. Specifically, the depth image data of the reinforcing bars is obtained through the following steps:

[0057] S1.1: Perform stereo matching on the left and right eye images of the depth camera to obtain a disparity map;

[0058] S1.2: Based on the relationship between depth and disparity, the disparity map is converted into a depth map. The formula for calculating depth z is as follows:

[0059]

[0060] Where f is the focal length of the depth camera, b is the baseline length of the depth camera, d is the parallax between the left and right eye images, and x l Let x be the x-coordinate of the projection point of the left eye camera. r The x-coordinate of the projection point of the right eye camera.

[0061] Example, Figure 2 (a) is the RGB image of the reinforcing steel. Figure 2 (b) is a depth diagram of the reinforcing bars.

[0062] S2: Input the RGB image of the rebar into a convolutional neural network for rebar recognition, and obtain the predicted bounding box and mask of the rebar. Specific steps include:

[0063] S2.1: Create a rebar dataset; acquire original images of rebar using a camera, create rebar mask labels using manual annotation, randomly divide the data into training and testing sets in a 7:3 ratio, and augment the dataset using data augmentation. Data augmentation refers to geometric transformation operations such as random translation, rotation, mirroring, and affine transformations, as well as pixel transformation operations such as randomly adjusting brightness, contrast, HSV, adding Gaussian noise, and adding salt-and-pepper noise.

[0064] S2.2: Pre-train the improved Mask R-CNN model using the public dataset COCO2017, and initialize the network parameters based on the principle of transfer learning.

[0065] The network structure of Mask R-CNN is as follows: Figure 3 As shown, the Mask R-CNN model includes a feature extraction module, an RPN module, a ROI alignment module, and an output branch. The feature extraction module uses a ResNet-FPN feature pyramid structure based on residual networks to generate feature maps from the image. The RPN module further generates strong prior anchor boxes for each point on the feature map, and obtains the classification score and bounding box regression value of the anchor boxes through 1×1 convolutions, thereby selecting a set of better candidate boxes to input into the ROI alignment module. The ROI alignment module transforms the optimized feature map generated by the feature extraction module and the candidate boxes selected by the RPN module to the same dimension to meet the input feature requirements of the subsequent fully convolutional network. Finally, the features obtained from the ROI alignment module are input into a fully connected layer, which outputs the predicted class score of the object, the bounding box regression value, and the pixel mask in the classification branch, bounding box regression branch, and mask branch, respectively, thus completing the entire detection and segmentation task.

[0066] This embodiment optimizes the feature extraction module of the Mask R-CNN network structure described above, forming an improved Mask R-CNN model. The optimized feature extraction module is a CA-SA module, consisting of a bottom-up propagation path and a channel attention (CA) module and a spatial attention (SA) mechanism, added to the ResNet-FPN feature pyramid structure based on residual networks. The structure of the CA module is as follows... Figure 4 As shown, in the CA module, after a feature map with height H, width W, and number of channels C is input into the global average pooling layer, the spatial dimensions W and H are compressed to a unit of 1. The resulting 1×1×C feature map is then convolved and processed through softmax to make the sum of the channels equal to 1. The output at this point is the attention mechanism weight for each channel. This weight is multiplied by the corresponding channel of the input feature map to obtain the output feature map. The structure of the SA module is as follows: Figure 5As shown, in the SA module, after the feature map undergoes 1×1 convolution and softmax processing, the channel dimension is compressed to a unit of 1. The SA module learns the weight matrix of the H×W size feature map on the two-dimensional plane. This weight matrix corresponds to the spatial attention mechanism weights of each pixel, representing the importance of spatial location information. Assigning this weight matrix to the input feature map amplifies important features and weakens background information, thereby achieving feature selection and enhancement. The structure of the bottom-up propagation path is as follows: Figure 6 As shown, based on the feature maps {P2, P3, P4, P5} obtained from ResNet-FPN, the feature information of P2 is passed to N2. N2 performs a 3×3 convolution to downsample the height and width to the size of P3, and then adds it element-wise to P3 and sends it to the CA-SA module to obtain N3. Similarly, N4 and N5 are further extracted from the feature maps P4 and P5 to obtain the feature maps {N2, N3, N4, N5}.

[0067] S2.3: Train the improved Mask R-CNN model established in step S2.2 using the dataset from step S2.1 to construct a rebar instance segmentation model. The training batch size is set to 4, the initial learning rate to 0.0005, and the training epochs to 50. Specific steps include:

[0068] S2.3.1: Set the evaluation metric for the improved Mask R-CNN model to the mAP calculated using the defined method on the COCO2017 dataset, which is the mean of the sum of all average precision APs under different intersection-over-union (IoU) thresholds (0.50: 0.05: 0.95). The specific formula for calculating AP is as follows:

[0069]

[0070] Where P represents the proportion of all predicted steel reinforcement targets that are correctly predicted, and R represents the proportion of all actually correct steel reinforcement targets that are predicted as positive samples.

[0071] S2.3.2: Input and output the rebar images and rebar mask labels from the training set into the improved Mask R-CNN model to obtain the weight parameters of the rebar instance segmentation model; use the obtained weight parameters to predict the bounding boxes and masks of the rebar images in the test set, calculate the loss and mAP index by comparing the results with the ground truth values ​​of the rebar mask labels, and adjust the weight parameters accordingly; until the training is completed, obtain the weight parameter corresponding to the maximum mAP, and use this weight coefficient for subsequent rebar recognition.

[0072] S2.4: Input the RGB image of the rebar acquired by the depth camera into the rebar instance segmentation model to obtain the predicted bounding box and mask of the rebar. The result is as follows: Figure 7 As shown in (a).

[0073] S3: Based on the rebar identification results, use binocular vision technology to perform reinforcement detection and output visualized rebar quality acceptance results. Specific steps include:

[0074] S3.1: Based on the steel bar recognition results, the mask images are binarized, and the pixel coordinates of their edges are extracted using the edge detection algorithm. Then, the pixel coordinates of the center line of each mask are calculated using the median transformation principle. The upper left corner of the image is the origin of the pixel coordinates (0,0), and the positive directions of the u-axis and v-axis are to the right and down.

[0075] S3.2: Calculate the normal vector of the centerline of each mask using the k-nearest neighbor algorithm. Assign masks with a horizontal component of the centerline normal vector less than the vertical component to the vertical direction, and assign masks with a horizontal component of the centerline normal vector greater than or equal to the vertical component to the horizontal direction. Then, using the centerline coordinates, automatically sort the rebar masks in a top-to-bottom and left-to-right order. Calculate the average value of the (u,v) coordinates of all pixels along each centerline. Sort the masks from top to bottom using the average v coordinate from smallest to largest, and sort the masks from left to right using the average u coordinate from smallest to largest.

[0076] S3.3: Extend the normal vector of the pixel at the center line of the mask to both sides of the edge, and use linear interpolation to extract the paired pixels at the edge to calculate the diameter of the rebar. Specifically, 20 pixels are evenly selected for each center line, and the normal vector of these 20 pixels is extended to both sides of the edge. The paired pixels at the edge are extracted using linear interpolation, and the average length of the line connecting these 20 pairs of paired pixels represents the diameter of each rebar. Similarly, extend the normal vector of the pixel at the center line of the mask to the center line of the adjacent mask, and extract the paired pixels of the center line of the mask to calculate the rebar spacing.

[0077] S3.4: Align the RGB image of the steel reinforcement and the depth image acquired by the depth camera to obtain the depth information of each pixel in the RGB image; using the camera intrinsic parameter matrix, transform the paired pixels of the edges and centerlines extracted in S3.3 from the pixel coordinate system to the camera coordinate system; the specific formula of the camera intrinsic parameter matrix is ​​as follows:

[0078]

[0079] Where M represents the camera intrinsic parameter matrix, (u0, v0) are the coordinates of the center point of the RGB image in the pixel coordinate system, and d x d y Here, f represents the length of a single pixel along the x and y axes, and f is the focal length of the depth camera. x f y This represents the number of pixels per length f on the x and y axes of the imaging plane.

[0080] S3.5: Substitute the camera coordinates of the paired pixels of the rebar edge and centerline obtained in S3.4 into the spatial distance formula to calculate the actual rebar diameter and spacing, and output a visualized rebar quality acceptance result, such as... Figure 7 As shown in (b); the spatial distance formula is specifically as follows:

[0081]

[0082] Where D is the distance between any two points in space, x1, y1, z1 are the x, y, z coordinates of point 1, and x2, y2, z2 are the x, y, z coordinates of point 2.

[0083] Furthermore, it should be understood that after reading the above description of the present invention, those skilled in the art can make various alterations or modifications to the present invention, and these equivalent forms also fall within the scope defined by the appended claims.

Claims

1. A smart reinforcement detection method based on convolutional neural networks and binocular vision, characterized in that, Including the following steps: S1: Use a depth camera to acquire RGB and depth image data of the reinforcing bars; S2: Input the RGB image of the rebar into the convolutional neural network to identify the rebar and obtain the predicted bounding box and mask of the rebar; S3: Based on the rebar identification results, use binocular vision technology to detect the reinforcement and output visualized rebar quality acceptance results; Step S2 specifically includes the following steps: S2.1: Use camera equipment to collect original images of steel bars, use manual annotation to create steel bar mask labels, divide the dataset into training set and test set, and augment the dataset through data augmentation; S2.2: Pre-train the improved Mask R-CNN model using the public dataset COCO2017, and initialize the network parameters based on the principle of transfer learning; S2.3: Train the improved Mask R-CNN model established in step S2.2 using the dataset from step S2.1 to construct a rebar instance segmentation model; S2.4: Input the RGB image of the rebar acquired by the depth camera into the rebar instance segmentation model to obtain the predicted bounding box and mask of the rebar; In step S2.2, the improved Mask R-CNN model includes an optimized feature extraction module, RPN module, ROI alignment module, and output branch; The optimized feature extraction module is a CA-SA module, consisting of a bottom-up propagation path and a channel attention (CA) module and a spatial attention (SA) module, added to the ResNet-FPN feature pyramid structure based on residual networks. In the CA module, a feature map with height H, width W, and number of channels C is input to a global average pooling layer. The spatial dimensions W and H are compressed to unit 1. The resulting 1×1×C feature map is then convolved and processed with softmax to make the sum of the channels equal to 1. The output at this point is the attention mechanism weight for each channel. This weight is multiplied by the corresponding channel of the input feature map to obtain the output feature map. In the SA module, after the feature map undergoes 1×1 convolution and softmax processing, the channel dimensions are compressed. For unit 1, the SA module learns the weight matrix of the H×W size feature map on the two-dimensional plane. This weight matrix corresponds to the spatial attention mechanism weight of each pixel, representing the importance of spatial location information. The input feature map is given this weight matrix to amplify important features and weaken background information, thereby achieving the effect of feature selection and enhancement. The bottom-up propagation path specifically refers to: based on the {P2, P3, P4, P5} feature map obtained by ResNet-FPN, the feature information of P2 is passed to N2. N2 performs a 3×3 convolution to downsample the height and width to the size of P3, and then adds it element by element to P3 and sends it to the CA-SA module to obtain N3. In this way, N4 and N5 are further extracted from the P4 and P5 feature maps to obtain the {N2, N3, N4, N5} feature map. The image is processed by the optimized feature extraction module to generate a feature map; the RPN module further generates a strong prior anchor box for each point on the feature map, and obtains the classification score and bounding box regression value of the anchor box through 1×1 convolution, thereby selecting a set of better candidate boxes to input into the ROI alignment module. The ROI alignment module transforms the feature map generated by the optimized feature extraction module and the candidate boxes selected by the RPN module to the same dimension to meet the requirements of the subsequent fully convolutional network for input features. Finally, the features obtained from the ROI alignment module are input into the fully connected layer, which outputs the predicted class score of the object, the bounding box regression value, and the pixel mask in the classification branch, bounding box regression branch, and mask branch, respectively, thus completing the entire detection and segmentation task.

2. The intelligent reinforcement detection method based on convolutional neural network and binocular vision according to claim 1, characterized in that, In step S1, depth image data of the reinforcing bars is obtained through the following steps: S1.1: Perform stereo matching on the left and right eye images of the depth camera to obtain a disparity map; S1.2: Based on the relationship between depth and disparity, the disparity map is converted into a depth map. The formula for calculating depth z is as follows: Where f is the focal length of the depth camera, b is the baseline length of the depth camera, d is the parallax between the left and right eye images, and x l Let x be the x-coordinate of the projection point of the left eye camera. r The x-coordinate of the projection point of the right eye camera.

3. The intelligent reinforcement detection method based on convolutional neural network and binocular vision according to claim 1, characterized in that, Step S2.3 specifically includes the following steps: S2.3.1: Set the evaluation metric for the improved Mask R-CNN model to the mAP calculated using the method defined in the COCO2017 dataset, which is the mean of the sum of all average precision APs under different intersection-over-union (IoU) thresholds of 0.50, 0.05, and 0.

95. The specific formula for calculating AP is as follows: Where P represents the proportion of all predicted steel reinforcement targets that are correctly predicted, and R represents the proportion of all actually correct steel reinforcement targets that are predicted as positive samples. S2.3.2: Input and output the images of rebars and rebar mask labels in the training set into the improved MaskR-CNN model to obtain the weight parameters of the rebar instance segmentation model; use the obtained weight parameters to predict the bounding boxes and masks of the rebar images in the test set, calculate the loss and mAP index by comparing the results with the ground truth values ​​of the rebar mask labels, and adjust the weight parameters accordingly. Until training is complete, obtain the weight parameter corresponding to the maximum mAP, and use this weight parameter for subsequent rebar identification.

4. The intelligent reinforcement detection method based on convolutional neural network and binocular vision according to claim 1, characterized in that, Step S3 specifically includes the following steps: S3.1: Based on the steel bar recognition results, the mask images are binarized, the edge detection algorithm is used to extract the pixel coordinates of the edges, and then the pixel coordinates of the centerline of each mask are calculated using the median transformation principle. S3.2: Calculate the normal vector of the centerline of each mask using the k-nearest neighbor algorithm. Assign masks with the horizontal component of the centerline normal vector less than the vertical component to the vertical direction, and assign masks with the horizontal component of the centerline normal vector greater than or equal to the vertical component to the horizontal direction. Then, using the centerline coordinates, automatically sort the rebar masks in the order from top to bottom and from left to right. S3.3: Extend the normal vector of the pixel point along the center line of the mask to both sides of the edge, and use linear interpolation to extract the paired pixels of the edge, which are used to calculate the diameter of the rebar; similarly, extend the normal vector of the pixel point along the center line of the mask to the center line of the adjacent mask, and extract the paired pixels of the center line of the mask, which are used to calculate the spacing of the rebar. S3.4: Align the RGB image of the steel bar and the depth image acquired by the depth camera to obtain the depth information of each pixel in the RGB image; using the camera intrinsic parameter matrix, transform the paired pixels of the edge and centerline extracted in S3.3 from the pixel coordinate system to the camera coordinate system; S3.5: Substitute the camera coordinates of the paired pixels of the rebar edge and centerline obtained in S3.4 into the spatial distance formula to calculate the actual rebar diameter and spacing, and output the visualized rebar quality acceptance results.

5. The intelligent reinforcement detection method based on convolutional neural network and binocular vision according to claim 4, characterized in that, In step S3.4, the specific formula for the camera intrinsic parameter matrix is ​​as follows: Where M represents the camera intrinsic parameter matrix, (u0, v0) are the coordinates of the center point of the RGB image in the pixel coordinate system, and d x d y Here, f represents the length of a single pixel along the x and y axes, and f is the focal length of the depth camera. x f y This represents the number of pixels per length f on the x-axis and y-axis of the imaging plane.

6. The intelligent reinforcement detection method based on convolutional neural network and binocular vision according to claim 4, characterized in that, In step S3.5, the spatial distance formula is specifically as follows: Where D is the distance between any two points in space, x1, y1, z1 are the x, y, z coordinates of point 1, and x2, y2, z2 are the x, y, z coordinates of point 2.

Images