Rebar binding point detection method and device based on depth rendering, equipment and medium

Abstract

The application provides a steel bar binding point detection method and device based on deep rendering, equipment and a medium, wherein the method comprises: acquiring an RGB-D image of a steel bar binding scene, the RGB-D image comprising an RGB-D image and a corresponding depth map; performing pseudo-color coding on the depth map to obtain a pseudo-color depth map; performing spatial alignment and fusion on the pseudo-color depth map and the RGB-D image to construct an RGB-depth superimposed image; inputting the RGB-depth superimposed image into a pre-trained binding point detection network to obtain a heat map of a binding point candidate position, and extracting a candidate binding point based on the heat map; performing texture consistency checking and depth layer consistency checking on each candidate binding point to screen out a binding point meeting double constraints as a final binding point, so as to solve the problems of poor robustness, low precision and easy interference of background and multi-layer steel bars in first-layer steel bar binding point detection in a complex construction scene.

Description

Technical Field

[0001] This invention relates to the field of building construction automation technology, and in particular to a method, device, equipment and medium for detecting rebar tying points based on depth rendering. Background Technology

[0002] In large-scale steel reinforcement engineering and prefabricated building construction, rebar tying is a core process for ensuring the strength and stability of concrete structures. Its construction efficiency and quality directly determine the overall project's progress and safety performance. With the development of industrialized and intelligent construction, manual rebar tying operations, which require a large investment of human resources, involve high labor intensity, have long construction cycles, and are easily affected by adverse weather and site conditions, resulting in inconsistent tying quality, can no longer meet the needs of modern construction. Promoting the automation of rebar tying operations towards robots and intelligent equipment has become an inevitable trend in the industry. Achieving accurate and stable detection of tying points is the core visual perception foundation for rebar tying robots to complete automated operations.

[0003] Currently, the mainstream technical solutions for rebar tying robot tying point detection in the industry are based on a two-stage strategy of preprocessing and detection using RGB or RGB-D images. The preprocessing stage removes interference from background rebar, formwork, supports, and other components, retaining only the first layer of rebar being processed by the robot. This typically employs depth thresholding or point cloud plane fitting methods. The detection stage, based on the preprocessed foreground data, identifies the tying positions of intersecting rebars using object detection or keypoint detection networks. However, the depth thresholding method filters the first layer of rebar by setting a fixed depth range. This method struggles to adapt to dynamic adjustments in camera installation height and tilt angle at construction sites, leading to misjudgments of rebar layers. Furthermore, it is sensitive to depth noise and voids on the rebar's metal surface, resulting in incomplete foreground region extraction. The point cloud plane fitting method achieves layering by fitting the main plane of the first layer of rebar, but it is easily affected by multi-planar background structures at construction sites, and outliers generated by depth noise significantly reduce the accuracy of plane fitting. The algorithm model in the detection stage is highly dependent on the preprocessing results. If the rebar separation error occurs in the preprocessing stage, the detection network will infer based on the incorrect input, resulting in distorted detection results of the binding points. At the same time, the pure data-driven detection model is a black box model, and the decision-making process lacks interpretability. It is difficult to locate the root cause of the problem after the detection fails, and the overall robustness and engineering reliability are insufficient. The above problems mean that although the existing technical solutions can achieve certain detection results in experimental environments or simple scenarios, they cannot achieve stable and accurate detection of binding points in actual construction scenarios with multiple layers of rebar and complex backgrounds. This has become a key bottleneck restricting the engineering application of rebar binding robots.

[0004] Therefore, there is an urgent need to develop a new method for detecting rebar tying points to solve the technical problems of poor robustness, low accuracy, and susceptibility to background and multi-layer rebar interference in the detection of first-layer rebar tying points under complex construction scenarios. Summary of the Invention

[0005] To overcome the problems existing in related technologies, this disclosure provides a method, device, equipment and medium for detecting rebar tying points based on depth rendering, so as to solve the technical problems of poor robustness, low accuracy and susceptibility to background and multi-layer rebar interference in the detection of first-layer rebar tying points in complex construction scenarios in related technologies.

[0006] This specification provides one or more embodiments of a method for detecting rebar tying points based on depth rendering, including the following steps: Acquire an RGB-D image of the rebar tying scene, wherein the RGB-D image includes an RGB-D image and a corresponding depth map; The depth map is pseudo-color encoded to obtain a pseudo-color depth map. The pseudo-color depth map is spatially aligned and fused with the RGB-D image to construct an RGB-depth overlay map. The RGB-depth overlay image is input into a pre-trained ligation point detection network to obtain a heat map of candidate ligation point locations, and candidate ligation points are extracted based on the heat map. For each candidate binding point, perform texture consistency checks and depth layer consistency checks, and select the binding points that satisfy the dual constraints as the final binding points. Preferably, the step of performing pseudo-color encoding on the depth map to obtain a pseudo-color depth map specifically includes the following steps: Construct a depth-effective mask that retains only pixel regions with valid depth values; The distribution of depth values ​​within the valid pixel region is statistically analyzed, and the lower and upper bounds of a preset percentile range are selected as truncation thresholds to cut the depth values ​​to the range of the truncation thresholds. Linear normalization is performed on the truncated depth values ​​to obtain a normalized grayscale image; The normalized grayscale image is converted into a three-channel pseudo-color image using a preset pseudo-color mapping function to obtain a pseudo-color depth map.

[0007] Preferably, the step of spatially aligning and fusing the pseudo-color depth map with the RGB-D image to construct an RGB-depth overlay map specifically includes the following steps: The resolution of the pseudo-color depth map is adjusted to match that of the RGB-D image using an interpolation method, resulting in an aligned pseudo-color depth map. The aligned pseudo-color depth map and the RGB-D image are linearly weighted and fused according to preset weights to generate an RGB-depth overlay map.

[0008] Preferably, the step of inputting the RGB-depth overlay image into a pre-trained ligation point detection network to obtain a heatmap of candidate ligation point locations specifically includes the following steps: A binding point detection network based on Lab bi-branch and color attention is constructed. The specific processing steps include: The RGB-depth overlay image is converted from the RGB color space to the Lab color space to obtain the luminance component and chrominance component; Construct a brightness branch, extract the features of the steel structure using the brightness component as input, and output a brightness feature map; Construct a chroma branch, extract depth pseudocolor difference features using chroma components as input, and output a chroma feature map; The luminance feature map and the chrominance feature map are fused along the channel dimension to obtain the fused feature; A color attention module is applied to the fused features, and channel weights are generated through global average pooling and nonlinear mapping. The fused features are then weighted to obtain the enhanced features. The enhanced features are input into the backbone network for multi-scale feature extraction, and the final output is a heatmap of tying point prediction.

[0009] Preferably, the extraction of candidate binding points based on the heatmap specifically includes the following steps: Discrete candidate binding point coordinates are obtained by searching for local maxima and filtering by confidence threshold on the heat map.

[0010] Preferably, the texture consistency check includes: A local image patch is extracted centered on the candidate ligation point; Determine whether the local image patch contains an approximately orthogonal linear structure and whether the candidate point is located near the intersection region; If not, then the corresponding candidate point is removed.

[0011] Preferably, the depth layer consistency check includes: Obtain the depth values ​​within the neighborhood of the candidate tying point and statistically analyze their distribution; Determine whether the depth values ​​within the neighborhood are mainly concentrated within the main depth range of the first layer of foreground reinforcement; If not, then the corresponding candidate point is removed.

[0012] This specification provides one or more embodiments of a rebar tying point detection device based on depth rendering, including: The data acquisition module is used to acquire RGB-D images of the rebar tying scene, wherein the RGB-D image includes an RGB-D image and a corresponding depth map; The image processing module is used to perform pseudo-color encoding on the depth map to obtain a pseudo-color depth map, and to spatially align and fuse the pseudo-color depth map with the RGB-D image to construct an RGB-depth overlay map; The detection module is used to input the RGB-depth overlay image into a pre-trained tying point detection network to obtain a heat map of candidate tying point locations, and extract candidate tying points based on the heat map; The filtering module performs texture consistency checks and depth layer consistency checks on each candidate binding point, and selects binding points that meet the dual constraints as the final binding points.

[0013] This specification provides one or more embodiments of a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the depth-rendering-based rebar tying point detection method described above.

[0014] This specification provides one or more embodiments of a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the depth-rendering-based rebar tying point detection method described above.

[0015] This disclosure provides a method, apparatus, device, and medium for detecting rebar tying points based on depth rendering. Its advantages lie in acquiring an RGB-D image containing both an RGB-D image and a corresponding depth map, simultaneously collecting texture color information and 3D spatial depth information of the rebar scene. This provides fundamental data support for subsequent depth rendering, image fusion, and tying point detection, while also considering the acquisition of raw data for both the appearance features and spatial hierarchy features of the rebar. The depth map is pseudo-color encoded and spatially aligned with and fused with the RGB-D image to obtain an RGB-depth overlay image. This visually integrates depth hierarchy information with texture information, effectively enhancing the visual separability between the first layer of rebar in the foreground, the subsequent layers, and the background structure. This provides a more interpretable and easily learned input representation for the tying point detection network, avoiding the feature loss problem caused by relying solely on RGB or depth information. The problem involves inputting an overlay image into a pre-trained tying point detection network to generate a heatmap, and extracting candidate tying points based on the heatmap. Utilizing the network's learning ability on the texture-depth joint features of the overlay image, the confidence distribution of rebar intersections is accurately identified. Local maxima and threshold filtering are used to discretize continuous thermal responses into candidate locations, providing target objects for subsequent precise selection and improving the efficiency and targeting of initial tying point identification. The candidate tying points undergo dual checks for texture and depth layer consistency to filter the final tying points. Texture checks eliminate false candidate points without cross-shaped structures, and depth layer checks eliminate candidate points from later layers / backgrounds that deviate from the main depth range of the first layer of rebar, effectively reducing false detections and missed detections, significantly improving the accuracy and stability of tying point positioning, and ensuring that the detection results match the actual first-layer rebar tying requirements of the robot's operation. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in one or more embodiments of this specification or in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 A flowchart illustrating a method for detecting rebar tying points based on depth rendering, provided for one or more embodiments of this specification; Figure 2 A flowchart illustrating the depth rendering and RGB-depth overlay map construction process provided in one or more embodiments of this specification; Figure 3 A schematic diagram illustrating the depth rendering visualization results provided in one or more embodiments of this specification; Figure 4 A flowchart of a method for generating and geometrically checking tie-up point candidates based on overlay images, provided for one or more embodiments of this specification; Figure 5 This is a schematic diagram of the binding point detection provided in one or more embodiments of this specification; Figure 6 A schematic diagram of a rebar tying point detection device based on depth rendering, provided for one or more embodiments of this specification; Figure 7 This is a schematic diagram of the structure of a computer device provided for one or more embodiments of this specification. Detailed Implementation

[0018] To enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the technical solutions in one or more embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of the embodiments. Based on one or more embodiments of this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of this invention.

[0019] The present invention will now be described in detail with reference to specific embodiments and accompanying drawings.

[0020] Method Implementation Examples According to embodiments of the present invention, to address the problem of unstable detection of rebar tying points under conditions of multi-layered rebar stacking and complex backgrounds, a rebar tying point detection method based on depth rendering is provided. This method fuses the original RGB image with a depth image to form a superimposed image that combines texture and depth layer information. Based on this, a tying point detection network is designed for this superimposed image, and the detection results are filtered by combining lightweight texture and depth constraints, thereby achieving stable and reliable tying point detection in complex construction scenarios. Figure 1 The diagram shown is a flowchart of the rebar tying point detection method based on depth rendering provided in this embodiment. The rebar tying point detection method based on depth rendering according to this embodiment includes the following steps: S110. Obtain the RGB-D image of the rebar tying scene. The RGB-D image includes the RGB-D image and the corresponding depth map.

[0021] S120. Perform pseudo-color encoding on the depth map to obtain a pseudo-color depth map. Spatially align and fuse the pseudo-color depth map with the RGB-D image to construct an RGB-depth overlay map.

[0022] S130. Input the RGB-depth overlay image into the pre-trained tying point detection network to obtain a heat map of candidate tying point locations, and extract candidate tying points based on the heat map.

[0023] S140. Perform texture consistency check and depth layer consistency check on each candidate binding point, and select the binding points that satisfy the dual constraints as the final binding points.

[0024] The method provided in this embodiment acquires an RGB-D image containing an RGB-D image and a corresponding depth map, simultaneously collecting texture color information and three-dimensional spatial depth information of the rebar scene. This provides basic data support for subsequent depth rendering, image fusion, and tie-point detection, taking into account the acquisition of raw data for both the appearance features and spatial hierarchy features of the rebar. The depth map is pseudo-color encoded and spatially aligned with the RGB-D image to obtain an RGB-depth overlay map. This visually integrates depth hierarchy information with texture information, effectively enhancing the visual separability between the first layer of rebar in the foreground, the next layer, and the background structure. This provides a more interpretable and easily learned input representation for the tie-point detection network, avoiding the feature loss problem caused by relying solely on RGB or depth information. The overlay map is then input into the pre-trained tie-point detection network. A point detection network generates a heatmap and extracts candidate binding points based on the heatmap. Utilizing the network's learning ability on the texture-depth joint features of the overlay image, it accurately identifies the confidence distribution of rebar intersections. Through local maxima and threshold filtering, continuous thermal responses are discretized into candidate locations, providing target objects for subsequent precise selection and improving the efficiency and targeting of initial binding point identification. The candidate binding points undergo dual checks for texture and depth layer consistency to filter the final binding points. Texture checks eliminate false candidate points without cross-shaped structures, while depth layer checks eliminate candidate points from later layers / backgrounds that deviate from the main depth range of the first layer of rebar. This effectively reduces false detections and missed detections, significantly improving the accuracy and stability of binding point positioning and ensuring that the detection results match the actual first-layer rebar binding requirements of the robot's operation.

[0025] In one embodiment, RGB-D data is used as input, and the depth values ​​are normalized and pseudo-color encoded to visually represent the depth differences between different rebar layers, forming a clearly structured pseudo-color depth map. Furthermore, the pseudo-color depth map is spatially aligned and weighted and fused with the original RGB image to obtain a superimposed image containing both texture and depth hierarchy information. This rendering method effectively enhances the depth separation between the first layer of foreground rebar and multiple layers of background rebar in the image domain, thus providing a more interpretable and easier-to-learn input representation for the subsequent tie-point detection network. Figure 2 The diagram shown is a flowchart of the depth rendering and RGB-depth overlay map construction process provided in this embodiment. The process involves pseudo-color encoding of the depth map to obtain a pseudo-color depth map, specifically including the following steps: Raw depth maps acquired by RGB-D cameras often contain holes, noise points, or outlier measurements, especially noticeable on reflective surfaces such as metal and steel reinforcement. Directly normalizing and visualizing the raw depth map can lead to outliers significantly compressing the effective depth range, impacting the stability of subsequent depth representation. Therefore, we first perform a depth map validity processing, constructing a depth validity mask that retains only pixel regions with valid depth values, such as pixels with depths greater than zero and not NaN. All subsequent depth statistics and mapping operations are performed within these valid regions. Locations with invalid or missing depth are uniformly marked as background regions and assigned fixed background values ​​in subsequent processing to prevent interference with visual representation and model learning. This step effectively constrains the quality of depth data without introducing complex depth restoration algorithms, providing a reliable foundation for subsequent depth normalization and fusion operations.

[0026] Depth maps may contain outliers at extremely close or distant locations, which can compress the grayscale dynamic range of normal depth regions and affect visualization quality. To improve contrast, the distribution of depth values ​​within the valid pixel region is statistically analyzed, and lower and upper bounds of a preset percentile range are selected as truncation thresholds, such as using the lower bounds corresponding to 1% and 99% percentages. and the Upper Realm By truncating the depth value to the truncation threshold range, the influence of outliers can be effectively eliminated.

[0027] The truncated depth values ​​are linearly normalized to obtain a normalized grayscale image.

[0028] Normalized representation is: ; in, I gary (x',y') This represents the normalized grayscale intensity.

[0029] The normalized grayscale image is converted into a three-channel pseudo-color image using a predefined standard pseudo-color mapping function. This involves mapping each grayscale value to an RGB color vector using a predefined color lookup table. The image after pseudo-color mapping is denoted as... Each pixel contains three components: red, green, and blue, used to encode different depth levels. For invalid or missing pixel locations in the depth map, their pseudocolor values ​​are uniformly set to zero or a specified background color to obtain a pseudocolor depth map.

[0030] The method provided in this embodiment ensures accurate subsequent statistics by constructing an effective mask to remove invalid depth values; adaptively truncating to remove outliers and enhance depth contrast; linearly normalizing to map depth to a unified dimension; and using pseudocolor conversion to generate a color depth map to visualize depth information, facilitating subsequent RGB fusion and feature extraction by the detection network.

[0031] In one embodiment, to achieve an integrated representation of depth structure and texture information, the pseudo-color depth map and the RGB-D image are spatially aligned and fused to construct an RGB-depth overlay map, specifically including the following steps: The resolution of the pseudo-color depth map is adjusted to match that of the RGB-D image using an interpolation method, resulting in an aligned pseudo-color depth map.

[0032] The aligned pseudo-color depth map and the RGB-D image are linearly weighted and fused according to preset weights to generate an RGB-depth overlay map.

[0033] Specifically, let the original RGB image be... Due to depth map The depth map may differ from the original RGB image in resolution and coordinate system. An interpolation method is used to scale the depth map to make it perfectly match the RGB image in resolution, resulting in an aligned depth map. The two images were then fused using a linear weighting method to obtain an overlay image. The calculation formula is as follows: ; in represents the weighting coefficient of the depth pseudo-color image in the fusion result. When... When the depth is large, the depth information in the overlay image is more prominent, which is beneficial for observing the spatial relationship between different steel reinforcement layers; when When the image size is smaller, the texture and color details of the RGB image are more apparent, which is beneficial for identifying local features on the surface of the reinforcing steel. Experiments show that when... When the value is in the middle range of 0.4 to 0.6, a good balance can be achieved between texture and depth, so that the first layer of reinforcement has both a clear outline and a clear sense of depth separation in the overlay image. To visually demonstrate the processing effect of this step, as shown below... Figure 3 The image shown is a schematic diagram of the depth rendering visualization result provided in this embodiment. Figure 3 'a' represents the original RGB image. Figure 3 b is the overlay image.

[0034] The method provided in this embodiment achieves pixel-level correspondence between the pseudo-color depth map and the RGB image through interpolation alignment, ensuring spatial consistency; then, it constructs an RGB-depth overlay map by weighted fusion, effectively combining color and depth information, enhancing the multimodal feature expression of the binding points, and improving the recognition capability of the detection network.

[0035] In one embodiment, the overlay image I overlay (u,v)It simultaneously preserves the texture and depth information of the rebar surface, providing a unified image representation that combines appearance and geometric features. Based on this, a tying point annotation dataset is constructed, and a tying point detection and inspection method using overlay images as input is designed, enabling the detection network to directly learn the salient patterns of the tying points, such as... Figure 4 The diagram shows the flowchart of the method for generating candidate tying points and performing geometric checks based on overlay images provided in this embodiment. In the overlay image, the first layer of reinforcing bars, due to its smallest distance from the camera, exhibits a more concentrated and prominent color and structural pattern. Meanwhile, the interference from subsequent layers of reinforcing bars and the background structure is significantly weakened in the depth pseudo-color dimension. Based on these characteristics, the method is directly applied to the overlay image... I overlay (u,v) Manual annotation of tying points is performed on the overlay image. For each overlay image, the annotator selects the desired tying position in the rebar intersection area and records the corresponding pixel coordinates, forming tying point annotation data. Based on this annotation set, training, validation, and test samples are further constructed to provide a unified input for the detection and inspection modules of the subsequent deep learning model. The RGB-depth overlay image is then used to... I overlay (u,v) The pre-trained ligation point detection network is fed with a standard three-channel input image to obtain a heatmap of ligation point candidate locations, so as to make full use of the texture-depth joint features contained in the overlay image. The specific steps include: A binding point detection network based on Lab bi-branch and color attention is constructed. The specific processing steps include: Convert the RGB-depth overlay image from the RGB color space to the Lab color space to obtain the luminance component. and chromaticity components , ,in, L(u,v) It represents pixel brightness and darkness, and can be used to capture structural features such as steel bar edges and textures. a(u,v) and It can depict color shifts in the red-green and yellow-blue directions, and more directly reflect color differences caused by pseudo-color depth mapping.

[0036] Construct a luminance branch using a single-channel luminance component. L(u,v) Extracting the steel reinforcement structure features F from the input L It outputs a brightness feature map, mainly capturing structural information such as the edges and intersection shapes of the reinforcing bars.

[0037] Construct a chroma branch using dual-channel chroma components. [ ] Extract depth pseudocolor difference features from the input, and output chromaticity feature map F. C The focus is on modeling the differences in color-depth encoding between the foreground steel reinforcement layer and the background.

[0038] The luminance feature map and chrominance feature map are fused along the channel dimension to obtain the fused feature F. LC .

[0039] To highlight the salience of the first layer of foreground reinforcement in the color-depth dimension, a color attention module is introduced. This is applied to the fused feature F. LC A color attention module is applied, generating channel weights through global average pooling and non-linear mapping; that is, by applying F... LC Global average pooling is performed to obtain channel-level statistical vectors. Then, a lightweight nonlinear mapping layer is used to generate channel weights to weight the fused features, resulting in enhanced features. This process can be understood as allowing the network to select the color-depth mode that is more important for judging the binding points, thereby explicitly enhancing the foreground steel reinforcement layer related channels and suppressing the background and noise channels at the feature level.

[0040] Enhanced features The input backbone network extracts high-level features at multiple scales through multi-scale convolution and context fusion, and finally outputs a heatmap of tether point prediction. Each pixel value represents the confidence level of that location as a tying point.

[0041] The method provided in this embodiment converts the RGB-depth overlay map to Lab space, separates the luminance and chrominance components, extracts the steel reinforcement structure features and depth pseudocolor difference features respectively, and after fusion, enhances key information with color attention. Then, through multi-scale feature extraction, the generated heat map can more accurately and robustly locate the binding points.

[0042] In one embodiment, extracting candidate binding points based on a heatmap specifically includes the following steps: In the heat map The discrete candidate binding point coordinates are obtained through local maximum search and confidence threshold filtering. This step discretizes the continuous thermal response distribution into a finite number of candidate spatial locations, providing candidate inputs for subsequent tethering point selection based on texture and depth consistency constraints.

[0043] In one embodiment, texture consistency checking includes: A local image patch is extracted centered on the candidate binding point.

[0044] Determine whether a local image patch contains approximately orthogonal linear structures—a typical steel bar "cross" texture pattern—and whether the candidate points are located near the intersection region.

[0045] If not, candidate binding points that only exhibit unidirectional linear features or lack obvious cross structures are eliminated.

[0046] In one embodiment, the deep layer consistency check includes: Statistical analysis is performed on the depth distribution of the neighborhood of candidate tying points using the depth values ​​corresponding to the overlay image. The depth values ​​within the neighborhood of candidate tying points are obtained, and their distribution is statistically analyzed.

[0047] Determine whether the depth values ​​within the neighborhood are mainly concentrated within the main depth range of the first layer of foreground reinforcement; If not, that is, if the neighborhood depth of a candidate tying point deviates significantly from this range, it is more likely to belong to the subsequent reinforcement or background components, and the corresponding candidate point is removed.

[0048] After filtering based on both texture and depth constraints, a reliable set of final binding points can be obtained. For example... Figure 5 The diagram shown is a schematic diagram of the binding point detection provided in this embodiment.

[0049] Device Examples According to embodiments of the present invention, a rebar tying point detection device based on depth rendering is provided, such as... Figure 6 The diagram shown is a structural schematic of the rebar tying point detection device based on depth rendering provided in this embodiment. The rebar tying point detection device based on depth rendering according to this embodiment of the invention includes: The data acquisition module 61 is used to acquire RGB-D images of the rebar tying scene. The RGB-D images include RGB-D images and corresponding depth maps.

[0050] Image processing module 62 is used to perform pseudo-color encoding on the depth map to obtain a pseudo-color depth map, and to spatially align and fuse the pseudo-color depth map with the RGB-D image to construct an RGB-depth overlay map.

[0051] The detection module 63 is used to input the RGB-depth overlay map into the pre-trained tying point detection network to obtain a heat map of the candidate tying point locations, and extract candidate tying points based on the heat map.

[0052] The filtering module 64 is used to perform texture consistency checks and depth layer consistency checks on each candidate binding point, and filter out the binding points that meet the dual constraints as the final binding points.

[0053] The device provided in this embodiment acquires an RGB-D image containing an RGB-D image and a corresponding depth map through the data acquisition module 61. It can simultaneously collect the texture color information and three-dimensional spatial depth information of the rebar scene, providing basic data support for subsequent depth rendering, image fusion, and tie point detection, while taking into account the acquisition of raw data of both the appearance features and spatial layer features of the rebar. The image processing module 62 performs pseudo-color encoding on the depth map and spatially aligns and fuses it with the RGB-D image to obtain an RGB-depth overlay image. It integrates the depth layer information with the texture information in a visual form, effectively enhancing the visual separability of the first layer of rebar in the foreground, the next layer of rebar, and the background structure. This provides a more interpretable and easy-to-learn input expression for the tie point detection network, avoiding the feature loss problem of relying solely on RGB or depth information. The detection module 63 processes the overlay image... The pre-trained binding point detection network generates a heat map, and candidate binding points are extracted based on the heat map. By utilizing the network's ability to learn the joint features of texture and depth in the overlay image, the confidence distribution of rebar intersections is accurately identified. Continuous thermal responses are discretized into candidate positions through local maxima and threshold filtering, providing target objects for subsequent accurate filtering and improving the efficiency and targeting of the initial binding point identification. The filtering module 64 performs dual checks on the consistency of texture and depth layers on the candidate binding points and filters the final binding points. False candidate points without cross-shaped structures are eliminated through texture checks, and candidate points in the later layers / background that deviate from the main depth range of the first layer of rebar are eliminated through depth layer checks. This effectively reduces false detections and missed detections, significantly improves the accuracy and stability of binding point positioning, and ensures that the detection results match the actual first layer rebar binding requirements of the robot's operation.

[0054] The embodiments of the present invention are device embodiments corresponding to the above method embodiments. The specific operations of each module processing step can be understood with reference to the description of the method embodiments, and will not be repeated here.

[0055] like Figure 7 As shown, the present invention also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, it implements the rebar tying point detection method based on depth rendering in the above embodiments, or when the computer program is executed by a processor, it implements the rebar tying point detection method based on depth rendering in the above embodiments.

[0056] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0057] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for apparatus or system embodiments, since they are basically similar to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. The apparatus and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0058] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features. These modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention, and the contents not described in detail in the specification of the present invention are known to those skilled in the art.

Claims

1. A method for detecting rebar tying points based on depth rendering, characterized in that, Includes the following steps: Acquire an RGB-D image of the rebar tying scene, wherein the RGB-D image includes an RGB-D image and a corresponding depth map; The depth map is pseudo-color encoded to obtain a pseudo-color depth map. The pseudo-color depth map is spatially aligned and fused with the RGB-D image to construct an RGB-depth overlay map. The RGB-depth overlay image is input into a pre-trained ligation point detection network to obtain a heat map of candidate ligation point locations, and candidate ligation points are extracted based on the heat map. For each candidate binding point, perform texture consistency checks and depth layer consistency checks, and select the binding points that satisfy the dual constraints as the final binding points.

2. The method for detecting rebar tying points based on depth rendering as described in claim 1, characterized in that, The process of performing pseudo-color encoding on the depth map to obtain a pseudo-color depth map specifically includes the following steps: Construct a depth-effective mask that retains only pixel regions with valid depth values; The distribution of depth values ​​within the valid pixel region is statistically analyzed, and the lower and upper bounds of a preset percentile range are selected as truncation thresholds to cut the depth values ​​to the range of the truncation thresholds. Linear normalization is performed on the truncated depth values ​​to obtain a normalized grayscale image; The normalized grayscale image is converted into a three-channel pseudo-color image using a preset pseudo-color mapping function to obtain a pseudo-color depth map.

3. The method for detecting rebar tying points based on depth rendering as described in claim 1, characterized in that, The step of spatially aligning and fusing the pseudo-color depth map with the RGB-D image to construct an RGB-depth overlay map specifically includes the following steps: The resolution of the pseudo-color depth map is adjusted to match that of the RGB-D image using an interpolation method, resulting in an aligned pseudo-color depth map. The aligned pseudo-color depth map and the RGB-D image are linearly weighted and fused according to preset weights to generate an RGB-depth overlay map.

4. The method for detecting rebar tying points based on depth rendering as described in claim 1, characterized in that, The step of inputting the RGB-depth overlay map into a pre-trained ligation point detection network to obtain a heatmap of candidate ligation point locations specifically includes the following steps: A binding point detection network based on Lab bi-branch and color attention is constructed. The specific processing steps include: The RGB-depth overlay image is converted from the RGB color space to the Lab color space to obtain the luminance component and chrominance component; Construct a brightness branch, extract the features of the steel structure using the brightness component as input, and output a brightness feature map; Construct a chroma branch, extract depth pseudocolor difference features using chroma components as input, and output a chroma feature map; The luminance feature map and the chrominance feature map are fused along the channel dimension to obtain the fused feature; A color attention module is applied to the fused features, and channel weights are generated through global average pooling and nonlinear mapping. The fused features are then weighted to obtain the enhanced features. The enhanced features are input into the backbone network for multi-scale feature extraction, and the final output is a heatmap of tying point prediction.

5. The method for detecting rebar tying points based on depth rendering as described in claim 1, characterized in that, The extraction of candidate binding points based on the heatmap specifically includes the following steps: Discrete candidate binding point coordinates are obtained by searching for local maxima and filtering by confidence threshold on the heat map.

6. The method for detecting rebar tying points based on depth rendering as described in claim 1, characterized in that, The texture consistency check includes: A local image patch is extracted centered on the candidate ligation point; Determine whether the local image patch contains an approximately orthogonal linear structure and whether the candidate point is located near the intersection region; If not, then the corresponding candidate point is removed.

7. The method for detecting rebar tying points based on depth rendering as described in claim 1, characterized in that, The deep layer consistency check includes: Obtain the depth values ​​within the neighborhood of the candidate tying point and statistically analyze their distribution; Determine whether the depth values ​​within the neighborhood are mainly concentrated within the main depth range of the first layer of foreground reinforcement; If not, then the corresponding candidate point is removed.

8. A rebar tying point detection device based on depth rendering, characterized in that, include: The data acquisition module is used to acquire RGB-D images of the rebar tying scene, wherein the RGB-D image includes an RGB-D image and a corresponding depth map; The image processing module is used to perform pseudo-color encoding on the depth map to obtain a pseudo-color depth map, and to spatially align and fuse the pseudo-color depth map with the RGB-D image to construct an RGB-depth overlay map; The detection module is used to input the RGB-depth overlay image into a pre-trained tying point detection network to obtain a heat map of candidate tying point locations, and extract candidate tying points based on the heat map; The filtering module performs texture consistency checks and depth layer consistency checks on each candidate binding point, and selects binding points that meet the dual constraints as the final binding points.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the rebar tying point detection method based on depth rendering as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the rebar tying point detection method based on depth rendering as described in any one of claims 1 to 7.

Images