Double-layer steel bar binding point detection method and device, computer equipment and storage medium

Abstract

The invention relates to a double-layer steel bar binding point detection method and device, computer equipment and a storage medium. The method comprises the following steps: acquiring an image of a to-be-bound area in a double-layer steel bar structure; a target detection model is adopted to detect binding points in the to-be-bound area image, and a rectangular detection frame where each binding point is located is obtained; depth information corresponding to each rectangular detection frame is determined, and the level of each binding point in the double-layer steel bar structure is determined based on the depth information; and determining a target binding point in the double-layer steel bar structure according to the confidence coefficient of the rectangular detection frame, the depth information, the hierarchy and a preset steel bar spacing parameter. By adopting the method, the detected binding points can be screened in combination with depth information, hierarchy and steel bar spacing parameters meeting the requirements of the steel bar binding industry, so that the interference of steel bar overlapping, shielding or uneven illumination on image detection is eliminated, and the detection accuracy of the target binding point is improved.

Description

Methods, devices, computer equipment, and storage media for detecting double-layer rebar tying points Technical Field

[0001] This application relates to the fields of computer vision and building intelligence technology, and in particular to a method, device, computer equipment, computer-readable storage medium and computer program product for detecting double-layer rebar tying points. Background Technology

[0002] In the manufacturing and pre-assembly process of precast beams, the identification of rebar tying points is crucial. Existing automation technologies mainly employ two methods to identify rebar tying points: binocular vision-based tying point localization technology and traditional vision-based tying point recognition algorithms.

[0003] A binocular vision-based technique for tying point localization is proposed: a multi-scale binocular stereo matching model for rebar mesh based on detail enhancement is used to match stereo features of rebar mesh images from rebar engineering sites. This model introduces a multi-scale feature extraction network and an attention mechanism on top of the AnyNet network to obtain high-precision rebar mesh depth information. Depth filtering of the rebar mesh depth information yields the target tying working surface, thereby locating the rebar tying points within the target tying working surface. However, this technique suffers from high computational cost, complex calibration, and difficulty in matching in weakly textured regions (such as smooth rebar surfaces).

[0004] Traditional vision-based ties-in-reinforcement (TIR) ​​recognition algorithms acquire the positional information and intersection coordinates of the reinforcing bars in the rebar image through edge detection and line detection. The TIR intersection region image is cropped from the rebar image to form a dataset. A lightweight image classification network is used to classify the dataset, obtaining "tied" and "untied" identification results. Based on the identification results, the TIR tie points are located and marked in the rebar image. However, this technique is only applicable to single-layer rebar meshes. When rebars overlap, are occluded, or there is uneven lighting, this technique is prone to a high false detection rate when identifying TIR tie points.

[0005] Therefore, there is an urgent need for a more accurate method for detecting rebar tying points suitable for double-layer rebar mesh structures. Summary of the Invention

[0006] Therefore, it is necessary to provide a method, device, computer equipment, computer-readable storage medium, and computer program product for detecting double-layer rebar tying points that can improve detection accuracy, thereby providing visual guidance for subsequent intelligent execution of rebar tying actions by robotic arms, in response to the aforementioned technical problems.

[0007] Firstly, this application provides a method for detecting double-layer rebar tying points, including:

[0008] Obtain an image of the area to be tied in a double-layer steel reinforcement structure;

[0009] The binding points in the image of the region to be bound are detected using a target detection model, and a rectangular detection box is obtained for each binding point.

[0010] Determine the depth information corresponding to each of the rectangular detection boxes, and determine the layer to which each of the binding points belongs in the double-layer steel structure based on the depth information;

[0011] The target binding point in the double-layer rebar structure is determined based on the confidence level of the rectangular detection frame, the depth information, the layer, and the preset rebar spacing parameters.

[0012] In one embodiment, determining the depth information corresponding to each of the rectangular detection frames, and determining the layer to which each tying point belongs in the double-layer steel reinforcement structure based on the depth information, includes:

[0013] A depth estimation model is used to predict the depth of each pixel in the image of the region to be bound, and a pixel-level depth map corresponding to the image of the region to be bound is output. The resolution of the pixel-level depth map is the same as the resolution of the image of the region to be bound.

[0014] Determine the depth information corresponding to each rectangular detection box based on the pixel-level depth map;

[0015] The depth information corresponding to each rectangular detection box is statistically analyzed to obtain the depth statistics result for each rectangular detection box.

[0016] The layer to which each of the binding points belongs is determined using the depth statistics results.

[0017] In one embodiment, determining the layer to which each ligation point belongs using the depth statistics includes:

[0018] When the depth statistics result includes the mean depth value corresponding to each of the rectangular detection boxes, the binding points are clustered based on the mean depth value to determine the level to which the binding points belong;

[0019] Alternatively, if the depth statistics include the depth mean and depth variance corresponding to each rectangular detection box, a Gaussian mixture model can be used to divide the binding points based on the depth mean and depth variance to obtain the layer to which the binding points belong.

[0020] In one embodiment, determining the target tying point in the double-layer rebar structure based on the confidence level of the rectangular detection frame, the depth information, the layer, and preset rebar spacing parameters includes:

[0021] The depth difference between adjacent rectangular detection boxes is determined based on the depth information of adjacent rectangular detection boxes and their corresponding levels.

[0022] The confidence level of the rectangular detection frame, the depth difference, and the rebar spacing parameters are used to perform cross-validation on each binding point to obtain the cross-validation result of each binding point;

[0023] The binding points whose cross-validation results meet preset conditions are taken as the target binding points. The preset conditions include the confidence level being greater than a preset confidence threshold, the depth difference being greater than a preset difference threshold, and the spacing between the binding points matching the rebar spacing parameter.

[0024] In one embodiment, acquiring an image of the area to be tied in a double-layer rebar structure includes:

[0025] Color images of the area to be tied in the double-layer steel reinforcement structure are acquired using a monocular camera.

[0026] The color image is subjected to Gaussian filtering for denoising to obtain a denoised color image;

[0027] The denoised color image is subjected to contrast enhancement processing to obtain an enhanced color image;

[0028] The enhanced color image is subjected to distortion correction processing to obtain a corrected color image, which is then used as the image of the area to be bound.

[0029] In one embodiment, the method further includes:

[0030] The two-dimensional coordinates of the target binding point in the image of the area to be bound, along with the depth information, are combined with the intrinsic parameter matrix of the monocular camera to perform perspective transformation processing, thereby obtaining the three-dimensional spatial coordinates of the target binding point in the camera coordinate system of the monocular camera. The monocular camera is the camera that acquires the image of the area to be bound. The three-dimensional spatial coordinates are used to generate operation instructions that instruct the robotic arm to perform binding actions on the target binding point.

[0031] Secondly, this application also provides a double-layer rebar tying point detection device, comprising:

[0032] The image acquisition module is used to acquire images of the area to be tied in a double-layer steel structure.

[0033] The binding point detection module is used to detect binding points in the image of the region to be bound using a target detection model, and obtain a rectangular detection box where each binding point is located.

[0034] The depth segmentation module is used to determine the depth information corresponding to each of the rectangular detection boxes, and to determine the layer to which each of the binding points belongs in the double-layer steel structure based on the depth information;

[0035] The binding point determination module is used to determine the target binding point in the double-layer steel structure based on the confidence level of the rectangular detection frame, the depth information, the layer, and the preset steel bar spacing parameters.

[0036] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the double-layer rebar tying point detection method described in any of the embodiments of the first aspect.

[0037] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the double-layer rebar tying point detection method described in any of the embodiments of the first aspect.

[0038] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the double-layer rebar tying point detection method described in any of the embodiments of the first aspect.

[0039] The aforementioned double-layer rebar tying point detection method, apparatus, computer equipment, computer-readable storage medium, and computer program product acquire an image of the area to be tied in a double-layer rebar structure; use a target detection model to detect tying points in the image of the area to be tied, obtaining a rectangular detection box for each tying point; determine the depth information corresponding to each rectangular detection box, and determine the layer to which each tying point belongs in the double-layer rebar structure based on the depth information; and determine the target tying point in the double-layer rebar structure based on the confidence level of the rectangular detection box, the depth information, the layer, and the preset rebar spacing parameters. This method can combine the three-dimensional depth information, layer, and rebar spacing parameters that meet the industrial requirements for rebar tying to filter tying points detected in the two-dimensional plane, thereby eliminating interference from rebar overlap, occlusion, or uneven lighting on image detection and improving the detection accuracy of the target tying point. Furthermore, the aforementioned double-layer rebar tying point detection method, by using vision-based target detection and depth prediction to obtain detection box information in the two-dimensional plane dimension and depth information in the three-dimensional solid dimension, can reduce detection costs, simplify detection difficulty, accurately distinguish the spatial hierarchy of tying points, and solve the problem of matching difficulties in areas with weak texture (such as smooth rebar surfaces). Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 shows the application environment of a double-layer rebar tying point detection method in one embodiment;

[0042] Figure 2 is a flowchart illustrating a method for detecting double-layer rebar tying points in one embodiment;

[0043] Figure 3 is a flowchart illustrating the steps for determining the level in one embodiment;

[0044] Figure 4 is a flowchart illustrating the steps for determining the target binding point in one embodiment;

[0045] Figure 5 is a flowchart illustrating the image preprocessing steps in one embodiment;

[0046] Figure 6 is a flowchart illustrating the double-layer rebar tying point detection method in another embodiment;

[0047] Figure 7 is a structural block diagram of a double-layer rebar tying point detection device 700 in one embodiment;

[0048] Figure 8 is an internal structure diagram of a computer device in one embodiment. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0050] It should be noted that the data involved in this application (including but not limited to data used for analysis, data stored, data displayed, etc.) are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0051] The double-layer rebar tying point detection method provided in this application embodiment can be applied to the application environment shown in Figure 1. The terminal 102 can communicate with the monocular camera 104 via a wired connection or wireless communication signal. The monocular camera 104 can be used to acquire images of the double-layer rebar structure 10. For example, the terminal 102 can acquire images of the area to be tied in the double-layer rebar structure 10 captured by the monocular camera 104; use a target detection model to detect tying points in the image of the area to be tied, obtaining a rectangular detection box for each tying point; determine the depth information corresponding to each rectangular detection box, and determine the layer to which each tying point belongs in the double-layer rebar structure 10 based on the depth information; determine the target tying point in the double-layer rebar structure 10 based on the confidence level of the rectangular detection box, the depth information, the layer, and preset rebar spacing parameters. Subsequently, the terminal 102 can also generate corresponding tying action control commands based on the location of the target tying point, and send the tying action control commands to the robotic arm for execution, so that the robotic arm performs the tying action at the target tying point. The terminal 102 can be, but is not limited to, various personal computers, laptops, industrial control computers, robotic arm controllers, embodied intelligent robots, edge computing terminals, microcontrollers, embedded terminals, etc. The monocular camera 104 can be a standalone camera device, or it can be a camera device mounted on the robotic arm.

[0052] In an exemplary embodiment, as shown in FIG2, a method for detecting double-layer rebar tying points is provided. Taking the application of this method to terminal 102 in FIG1 as an example, the method includes the following steps S202 to S208. Wherein:

[0053] Step S202: Obtain an image of the area to be tied in the double-layer steel reinforcement structure.

[0054] Among them, double-layer reinforced concrete structure usually refers to a building structure in which steel bars are arranged in the upper and lower layers of the concrete section.

[0055] Images of the area to be tied refer to images taken of the area to be tied in a double-layer steel reinforcement structure.

[0056] For example, the terminal can acquire an image of the area to be tied in a double-layer steel structure, captured by a monocular camera. Alternatively, the terminal can acquire video data recorded by the monocular camera and extract image frames from the video data as the image of the area to be tied.

[0057] Step S204: Use the target detection model to detect the binding points in the image of the region to be bound, and obtain the rectangular detection box where each binding point is located.

[0058] For example, the terminal can use an object detection model to detect binding points in the image of the area to be bound, determine the rectangular detection box where each binding point is located, and output the position information and confidence score of the rectangular detection box. The object detection model can be an object detection model trained using images of binding points in a double-layer steel structure as training samples.

[0059] Step S206: Determine the depth information corresponding to each rectangular detection box, and determine the layer to which each binding point belongs in the double-layer steel structure based on the depth information.

[0060] Depth information can be used to characterize the distance of a pixel from the monocular camera. That is, the smaller the depth, the closer the pixel is to the monocular camera, and the larger the depth, the farther the pixel is from the monocular camera.

[0061] Optionally, in some embodiments, when the image of the area to be bound includes multiple consecutively acquired image frames, the depth information of the pixels can be calculated using the parallax between each acquired image frame and the feature point matching results between adjacent image frames obtained by a monocular camera. The depth information corresponding to the rectangular detection box is determined using the depth information of the pixels within the rectangular detection box. Alternatively, in other embodiments, the perspective relationship between the image of the area to be bound and the double-layer steel structure can be determined based on the pixel size of the steel bars in the image of the area to be bound and the actual size of the steel bars. The depth information corresponding to the rectangular detection box is determined based on the perspective relationship and the pixel size within the rectangular detection box. Alternatively, in other embodiments, a pre-trained neural network model can be used to predict the depth information of the rectangular detection box.

[0062] Based on the depth information of the rectangular detection frame containing the tying point, the layer to which the tying point belongs in the double-layer reinforced concrete structure can be determined. The layer can include the first layer and the second layer. If the first layer is the preceding layer, then the second layer can be the following layer. If the first layer is the following layer, then the second layer can be the preceding layer.

[0063] Step S208: Determine the target binding point in the double-layer rebar structure based on the confidence level, depth information, level, and preset rebar spacing parameters of the rectangular detection box.

[0064] For example, the terminal can use preset conditions to determine whether the confidence level, depth information, layer level of the binding tape, and spacing of the binding points match the rebar spacing parameters of the rectangular detection frame where the binding point is located. Binding points that meet the preset conditions are taken as valid target binding points. Subsequently, the terminal can control the robotic arm to perform binding actions at the target binding points.

[0065] In the above-mentioned double-layer rebar tying point detection method, an image of the area to be tied in the double-layer rebar structure is acquired; a target detection model is used to detect the tying points in the image of the area to be tied, and a rectangular detection box is obtained for each tying point; the depth information corresponding to each rectangular detection box is determined, and the layer to which each tying point belongs in the double-layer rebar structure is determined based on the depth information; the target tying point in the double-layer rebar structure is determined according to the confidence level of the rectangular detection box, the depth information, the layer, and the preset rebar spacing parameters. This method can combine the depth information, layer, and rebar spacing parameters that meet the industrial requirements of rebar tying to filter the tying points detected in the two-dimensional plane, thereby eliminating the interference of rebar overlap, occlusion, or uneven lighting on image detection and improving the detection accuracy of the target tying point. Furthermore, the aforementioned double-layer rebar tying point detection method, by using vision-based target detection and depth prediction to obtain detection box information in the two-dimensional plane dimension and depth information in the three-dimensional solid dimension, can reduce detection costs, simplify detection difficulty, accurately distinguish the spatial hierarchy of tying points, and solve the problem of matching difficulties in areas with weak texture (such as smooth rebar surfaces).

[0066] In an exemplary embodiment, as shown in FIG3, step S206 may include steps S302 to S308. Wherein:

[0067] Step S302: Use the depth estimation model to predict the depth of each pixel in the image of the region to be bound, and output the pixel-level depth map corresponding to the image of the region to be bound.

[0068] The depth estimation model can be used to predict a depth value for each pixel in a color image (RGB image). The depth estimation model can be a monocular model trained using images of tethered points labeled with depth information as training samples.

[0069] The resolution of the pixel-level depth map is the same as that of the image of the region to be ligated. The pixel-level depth map can be used to reflect the predicted depth value corresponding to each pixel.

[0070] For example, the terminal can run a depth estimation model. The image of the region to be ligated is input into the depth estimation model, which predicts the depth value corresponding to each pixel, resulting in a predicted depth value for each pixel. Based on the pixels and their corresponding predicted depth values, a pixel-level depth map with the same resolution as the image of the region to be ligated is generated and output.

[0071] Step S304: Determine the depth information corresponding to each rectangular detection box based on the pixel-level depth map.

[0072] For example, the terminal can read the predicted depth value of each pixel within the rectangular detection box from the pixel-level depth map based on the pixel region coordinates corresponding to the rectangular detection box. The predicted depth values ​​of each pixel within the rectangular detection box are then aggregated as the depth information corresponding to the rectangular detection box.

[0073] Step S306: Calculate the depth information corresponding to each rectangular detection box to obtain the depth statistics result corresponding to each rectangular detection box.

[0074] For example, the terminal can separately calculate the depth information corresponding to each rectangular detection box to determine the depth statistics for each rectangular detection box. The depth statistics may include, but are not limited to, data such as the mean depth and the variance depth.

[0075] Step S308: Use depth statistics to determine the level to which each binding point belongs.

[0076] For example, the terminal can compare the depth statistics with the depth of each layer of steel bars in the double-layer steel structure relative to the monocular camera to determine the layer to which each tying point belongs. Alternatively, the terminal can cluster the tying points based on the depth statistics to determine tying points of different layers.

[0077] In this embodiment, the depth value of each pixel in the image of the area to be tied is predicted by using a depth estimation model, and a pixel-level depth map with the same resolution as the image of the area to be tied is constructed. Based on the pixel-level depth map, the depth information corresponding to the rectangular detection box where the tying point is located is determined. The depth statistics corresponding to the depth information are used to divide the layer to which the tying point belongs, which can accurately distinguish the spatial layer where the tying point is located. At the same time, compared with the tying point localization technology based on binocular vision, this embodiment obtains depth information based on monocular vision, which can also reduce detection costs, simplify detection difficulty, and avoid interference from the smooth surface of the weak textured steel bar.

[0078] In an exemplary embodiment, step S308 may include: if the depth statistics result includes the mean depth value corresponding to each rectangular detection box, clustering the binding points based on the mean depth value to determine the level to which the binding points belong.

[0079] For example, the terminal can cluster the binding points corresponding to each rectangular detection box based on the mean depth of each rectangular detection box to obtain the clustering results of the binding points. Binding points in the same clustering result are determined to be binding points of the same level. Since the binding points with a smaller mean depth are closer to the monocular camera, when the monocular camera is shooting from above, the binding points in the clustering result with a smaller mean depth can be determined as binding points in the front layer of the double-layer steel structure, that is, the binding points in this clustering result belong to the front layer. The binding points in the clustering result with a larger mean depth belong to the rear layer. Conversely, if the monocular camera is shooting from below, the binding points in the clustering result with a smaller mean depth belong to the rear layer, and the binding points in the clustering result with a larger mean depth belong to the front layer.

[0080] In this embodiment, tying points belonging to the same level are determined by depth mean clustering, and then the level to which the tying points belong is divided based on the magnitude of the depth mean. This can quickly find tying points at the same level and improve the efficiency of determining the level.

[0081] In an exemplary embodiment, step S308 may include: when the depth statistics result includes the depth mean and depth variance corresponding to each rectangular detection box, using a Gaussian mixture model to divide the binding points based on the depth mean and depth variance to obtain the level to which the binding points belong.

[0082] For example, when the depth statistics include the depth mean and depth variance of each rectangular detection box, the terminal can construct the covariance matrix of each rectangular detection box using the depth mean and depth variance. A Gaussian distribution corresponding to each rectangular detection box is then constructed based on the depth mean and covariance matrix. A Gaussian mixture model is then used to divide the binding points corresponding to each rectangular detection box based on the Gaussian distribution, determining the layer to which each binding point belongs.

[0083] In this embodiment, a Gaussian distribution is constructed using the depth mean and depth variance. A Gaussian mixture model is then used to divide the binding points based on the Gaussian distribution to determine the level to which the binding points belong, thereby improving the accuracy of level determination.

[0084] In an exemplary embodiment, as shown in FIG4, step S208 may include steps S402 to S406. Wherein:

[0085] Step S402: Determine the depth difference between adjacent rectangular detection boxes based on the depth information of adjacent rectangular detection boxes and their corresponding levels.

[0086] For example, the terminal can calculate the distance between rectangular detection boxes using the pixel coordinates corresponding to the rectangular detection boxes. For each rectangular detection box, the rectangular detection box with the smallest distance is taken as its neighboring rectangular detection box. The depth information and corresponding layer of the neighboring rectangular detection boxes are compared, and the differences in depth information and the differences in layer are determined as the depth difference.

[0087] Step S404: Cross-validate each binding point using the confidence level, depth difference, and rebar spacing parameters of the rectangular detection frame to obtain the cross-validation result for each binding point.

[0088] Step S406: The binding points whose cross-validation results meet the preset conditions are taken as target binding points. The preset conditions include a confidence level greater than a preset confidence threshold, a depth difference greater than a preset difference threshold, and the binding point spacing matching the rebar spacing parameter.

[0089] For example, the terminal can combine prior information such as the confidence level of the rectangular detection box, the depth difference between adjacent rectangular detection boxes, and the rebar spacing parameters to perform cross-validation on each binding point detected by the target detection model, and select binding points whose cross-validation results meet the preset conditions of a confidence level greater than a preset confidence threshold, a depth difference greater than a preset difference threshold, and a binding point spacing matching the rebar spacing parameters as target binding points.

[0090] In this embodiment, by combining prior information such as the confidence level of the rectangular detection box, the depth difference between adjacent rectangular detection boxes, and the rebar spacing parameters, each binding point is cross-validated to determine the effective target binding point. This can eliminate interference from rebar overlap, occlusion, and uneven lighting, improve the matching accuracy in weak texture areas, and thus obtain more accurate target binding points.

[0091] In an exemplary embodiment, as shown in FIG5, step S202 may include steps S502 to S508. Wherein:

[0092] Step S502: Acquire color images of the area to be tied in the double-layer steel structure using a monocular camera.

[0093] Step S504: Perform Gaussian filtering on the color image to denoise it, and obtain the denoised color image.

[0094] Step S506: Perform contrast enhancement processing on the denoised color image to obtain the enhanced color image.

[0095] Step S508: Perform distortion correction processing on the enhanced color image to obtain a corrected color image, and use the corrected color image as the image of the area to be bound.

[0096] For example, the terminal can acquire a color image (also known as an RGB image) of the area to be tied in a double-layer steel structure using a monocular camera. The pixels in the color image are convolved using a preset Gaussian kernel, and the processed pixel values ​​are used as the values ​​after Gaussian filtering for denoising, resulting in a denoised color image. The denoised color image is then converted to a grayscale image, and the grayscale values ​​of the pixels in the grayscale image are transformed using histogram adaptive equalization. The transformed pixel grayscale values ​​are then truncated to the RGB value range of (0, 255) (RGB values ​​exceeding 255 are 255), forming an enhanced color image. The distortion parameters and intrinsic parameters of the monocular camera are used to correct the distortion of the pixel coordinates in the enhanced color image, and the corrected color image is used as the image of the area to be tied for tying point detection.

[0097] In this embodiment, by performing preprocessing operations such as Gaussian filtering for noise reduction, contrast enhancement, and distortion correction on the original color image, image noise interference can be reduced, image resolution can be improved, and image distortion can be eliminated, thereby obtaining a higher quality image of the area to be bound.

[0098] In an exemplary embodiment, the double-layer rebar tying point detection method provided in this application may further include: using the two-dimensional coordinates and depth information of the target tying point in the image of the area to be tied, combined with the intrinsic parameter matrix of a monocular camera, to perform perspective transformation processing to obtain the three-dimensional spatial coordinates of the target tying point in the camera coordinate system of the monocular camera. The monocular camera is a camera that acquires images of the area to be tied, and the three-dimensional spatial coordinates are used to generate operation instructions that instruct the robotic arm to perform tying actions on the target tying point.

[0099] For example, the terminal can establish a perspective transformation matrix between the imaging coordinate system and the camera coordinate system of the monocular camera using the principle of perspective transformation and the depth information and intrinsic parameter matrix of the monocular camera. The perspective transformation matrix is ​​then used to process the two-dimensional coordinates of the target binding point in the imaging coordinate system to obtain the three-dimensional spatial coordinates of the target binding point in the camera coordinate system. Subsequently, the terminal can generate operation commands based on these three-dimensional spatial coordinates to instruct the robotic arm to perform binding actions on the target binding point.

[0100] In this embodiment, by converting the two-dimensional coordinates of the target binding point in the imaging coordinate system into three-dimensional spatial coordinates in the camera coordinate system, the target binding point can be located and identified in three-dimensional space, which facilitates faster and more accurate subsequent manipulation of the robotic arm to perform binding actions at the target binding point.

[0101] In an exemplary embodiment, as shown in FIG6, this application also provides a method for detecting double-layer rebar tying points, including the following steps S602 to S612. Wherein:

[0102] Step S602: Acquire a color image of the area to be tied in the double-layer steel structure using a monocular camera, and preprocess the color image to obtain an image of the area to be tied. The preprocessing includes Gaussian filtering for noise reduction, contrast enhancement, and distortion correction.

[0103] For example, the terminal can acquire a color image of the area to be tied in the double-layer steel structure using a monocular camera, and perform preprocessing operations such as Gaussian filtering for noise reduction, contrast enhancement, and distortion correction on the color image according to the operations of steps S504 to S508 above, and use the preprocessed color image as the image of the area to be tied.

[0104] Step S604: Use the target detection model to detect the binding points in the image of the region to be bound, and obtain the rectangular detection box where each binding point is located.

[0105] Step S606: Use the depth estimation model to predict the depth of each pixel in the image of the region to be bound, and output a pixel-level depth map with the same resolution as the image of the region to be bound.

[0106] Step S608: Calculate the depth information corresponding to each rectangular detection box, and determine the level to which each binding point belongs based on the depth statistics results.

[0107] Step S610: Combine the confidence level of the rectangular detection box, the depth difference between adjacent rectangular detection boxes, and the preset rebar spacing parameters to perform cross-validation on the binding points, and take the binding points whose cross-validation results meet the preset conditions as the target binding points.

[0108] For example, the terminal can execute steps S604 and S606 in parallel, respectively using a target detection model to identify the rectangular detection box where the binding point is located, and using a depth estimation model to generate the corresponding pixel-level depth map. The depth information of the pixels within the rectangular detection box is statistically analyzed to obtain the mean depth and depth variance. Clustering or a Gaussian mixture model is used to determine whether the binding point belongs to the previous or subsequent layer based on the mean depth and depth variance. The binding point is cross-validated by combining the rectangular detection box, the pixel-level depth map, and the rebar spacing parameters that meet the requirements of the rebar binding process. Binding points with a confidence level higher than a preset confidence threshold, a depth difference greater than a preset difference threshold, and a spacing that matches the indirect parameters of the rebar are selected as valid target binding points using preset conditions.

[0109] Step S612: Using the two-dimensional coordinates and depth information of the target binding point, combined with the intrinsic parameter matrix of the monocular camera, perspective transformation is performed to obtain the three-dimensional spatial coordinates of the target binding point in the camera coordinate system.

[0110] For example, the terminal can use the depth information corresponding to the target binding point and the intrinsic parameter matrix of the monocular camera to establish a perspective transformation matrix between the imaging coordinate system of the monocular camera and the camera coordinate system. The perspective transformation matrix is ​​then used to convert the two-dimensional coordinates of the target binding point into three-dimensional spatial coordinates in the camera coordinate system. Subsequently, the terminal can use these three-dimensional spatial coordinates to instruct the robotic arm to perform binding actions at the target binding point.

[0111] In this embodiment, by using a monocular vision-based method to accurately predict the target binding points of the front and rear layers in a double-layer steel structure, visual guidance can be provided for the subsequent robotic arm to perform binding actions at the target binding points, reducing detection costs. At the same time, it can also improve the detection accuracy in scenarios such as overlapping steel bars, occlusion, or uneven lighting, and improve the processing capability and adaptability to complex scenarios.

[0112] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0113] Based on the same inventive concept, this application also provides a double-layer rebar tying point detection device for implementing the above-mentioned double-layer rebar tying point detection method. The solution provided by this device is similar to the solution described in the above method. Therefore, the specific limitations of one or more embodiments of the double-layer rebar tying point detection device provided below can be found in the limitations of the double-layer rebar tying point detection method above, and will not be repeated here.

[0114] In an exemplary embodiment, as shown in FIG7, a double-layer rebar tying point detection device 700 is provided, comprising: an image acquisition module 702, a tying point detection module 704, a depth segmentation module 706, and a tying point determination module 708, wherein:

[0115] Image acquisition module 702 is used to acquire images of the area to be tied in a double-layer steel structure.

[0116] The binding point detection module 704 is used to detect binding points in the image of the region to be bound using a target detection model, and obtain a rectangular detection box for each binding point.

[0117] The depth segmentation module 706 is used to determine the depth information corresponding to each rectangular detection box, and to determine the layer to which each binding point belongs in the double-layer steel structure based on the depth information.

[0118] The binding point determination module 708 is used to determine the target binding point in the double-layer steel structure based on the confidence level, depth information, level of the rectangular detection frame and the preset steel bar spacing parameters.

[0119] In an exemplary embodiment, the depth segmentation module 706 is further configured to use a depth estimation model to predict the depth of each pixel in the image of the region to be bound, and output a pixel-level depth map corresponding to the image of the region to be bound, wherein the resolution of the pixel-level depth map is consistent with the resolution of the image of the region to be bound; determine the depth information corresponding to each rectangular detection box based on the pixel-level depth map; statistically analyze the depth information corresponding to each rectangular detection box to obtain the depth statistics result corresponding to each rectangular detection box; and use the depth statistics result to determine the layer to which each binding point belongs.

[0120] In an exemplary embodiment, the depth segmentation module 706 is further configured to cluster the tying points based on the depth mean when the depth statistics result includes the depth mean corresponding to each rectangular detection box, and determine the level to which the tying points belong; or, when the depth statistics result includes the depth mean and depth variance corresponding to each rectangular detection box, use a Gaussian mixture model to segment the tying points based on the depth mean and depth variance, and obtain the level to which the tying points belong.

[0121] In an exemplary embodiment, the binding point determination module 708 is further configured to determine the depth difference between adjacent rectangular detection boxes based on the depth information of adjacent rectangular detection boxes and the corresponding level; to perform cross-validation on each binding point using the confidence level, depth difference, and rebar spacing parameters of the rectangular detection boxes, and to obtain the cross-validation result of each binding point; and to use the binding points whose cross-validation results meet preset conditions as target binding points, the preset conditions including a confidence level greater than a preset confidence threshold, a depth difference greater than a preset difference threshold, and the spacing of the binding point matching the rebar spacing parameter.

[0122] In an exemplary embodiment, the image acquisition module 702 is further configured to acquire a color image of the area to be tied in the double-layer steel structure using a monocular camera; perform Gaussian filtering on the color image to denoise it, thereby obtaining a denoised color image; perform contrast enhancement on the denoised color image, thereby obtaining an enhanced color image; perform distortion correction on the enhanced color image, thereby obtaining a corrected color image, and use the corrected color image as the image of the area to be tied.

[0123] In an exemplary embodiment, the double-layer rebar tying point detection device 700 further includes a three-dimensional positioning module, which is used to perform perspective transformation processing by combining the two-dimensional coordinates and depth information of the target tying point in the image of the area to be tied with the intrinsic parameter matrix of the monocular camera, so as to obtain the three-dimensional spatial coordinates of the target tying point in the camera coordinate system of the monocular camera. The monocular camera is the camera that acquires the image of the area to be tied, and the three-dimensional spatial coordinates are used to generate operation instructions that instruct the robotic arm to perform tying actions on the target tying point.

[0124] Each module in the aforementioned double-layer rebar tying point detection device 700 can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0125] In an exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram is shown in Figure 8. The computer device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor of the computer device provides computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The input / output interface of the computer device is used for exchanging information between the processor and external devices. The communication interface of the computer device is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When the computer program is executed by the processor, it implements a method for detecting double-layer rebar tying points. The display unit of the computer device is used to form a visually visible image and may be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0126] Those skilled in the art will understand that the structure shown in Figure 8 is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or may combine certain components, or may have different component arrangements.

[0127] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0128] In one exemplary embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above-described method embodiments.

[0129] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0130] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0131] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0132] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for detecting double-layer rebar tying points, characterized in that, The method includes: acquiring an image of the area to be tied in a double-layer steel structure; using a target detection model to detect tying points in the image of the area to be tied, obtaining a rectangular detection box where each tying point is located; determining the depth information corresponding to each rectangular detection box, and determining the layer to which each tying point belongs in the double-layer steel structure based on the depth information; and determining the target tying point in the double-layer steel structure according to the confidence level of the rectangular detection box, the depth information, the layer, and preset steel bar spacing parameters.

2. The method according to claim 1, characterized in that, The step of determining the depth information corresponding to each rectangular detection box and determining the layer to which each tying point belongs in the double-layer steel reinforcement structure based on the depth information includes: using a depth estimation model to predict the depth of each pixel in the image of the area to be tied, and outputting a pixel-level depth map corresponding to the image of the area to be tied, wherein the resolution of the pixel-level depth map is consistent with the resolution of the image of the area to be tied; determining the depth information corresponding to each rectangular detection box based on the pixel-level depth map; statistically analyzing the depth information corresponding to each rectangular detection box to obtain a depth statistical result corresponding to each rectangular detection box; and using the depth statistical result to determine the layer to which each tying point belongs.

3. The method according to claim 2, characterized in that, The step of determining the level to which each tying point belongs using the depth statistics results includes: when the depth statistics results include the depth mean corresponding to each rectangular detection box, clustering the tying points based on the depth mean to determine the level to which the tying point belongs; or, when the depth statistics results include the depth mean and depth variance corresponding to each rectangular detection box, using a Gaussian mixture model to divide the tying points based on the depth mean and the depth variance to obtain the level to which the tying point belongs.

4. The method according to claim 1, characterized in that, The step of determining the target binding point in the double-layer rebar structure based on the confidence level of the rectangular detection frame, the depth information, the layer, and the preset rebar spacing parameters includes: determining the depth difference between adjacent rectangular detection frames based on the depth information and corresponding layer of adjacent rectangular detection frames; performing cross-validation on each binding point using the confidence level of the rectangular detection frame, the depth difference, and the rebar spacing parameters to obtain the cross-validation result for each binding point; and selecting binding points whose cross-validation results meet preset conditions as the target binding points, wherein the preset conditions include the confidence level being greater than a preset confidence threshold, the depth difference being greater than a preset difference threshold, and the spacing of the binding point matching the rebar spacing parameters.

5. The method according to claim 1, characterized in that, The process of acquiring an image of the area to be tied in a double-layer steel reinforcement structure includes: acquiring a color image of the area to be tied in the double-layer steel reinforcement structure using a monocular camera; performing Gaussian filtering denoising on the color image to obtain a denoised color image; performing contrast enhancement on the denoised color image to obtain an enhanced color image; performing distortion correction on the enhanced color image to obtain a corrected color image; and using the corrected color image as the image of the area to be tied.

6. The method according to claim 1, characterized in that, The method further includes: using the two-dimensional coordinates of the target binding point in the image of the area to be bound and the depth information, combined with the intrinsic parameter matrix of the monocular camera, to perform perspective transformation processing to obtain the three-dimensional spatial coordinates of the target binding point in the camera coordinate system of the monocular camera, wherein the monocular camera is the camera that acquires the image of the area to be bound, and the three-dimensional spatial coordinates are used to generate operation instructions that instruct the robotic arm to perform binding actions on the target binding point.

7. A double-layer rebar tying point detection device, characterized in that, The device includes: an image acquisition module for acquiring an image of the area to be tied in a double-layer steel structure; a tying point detection module for detecting tying points in the image of the area to be tied using a target detection model to obtain a rectangular detection box for each tying point; a depth segmentation module for determining the depth information corresponding to each rectangular detection box and determining the layer to which each tying point belongs in the double-layer steel structure based on the depth information; and a tying point determination module for determining the target tying point in the double-layer steel structure based on the confidence level of the rectangular detection box, the depth information, the layer, and preset steel bar spacing parameters.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images