A multi-layer steel bar intelligent counting method and system based on deep learning

Abstract

The application relates to a kind of multi-layer steel bar intelligent counting method and system based on deep learning, method includes: the overhead image of the multi-layer steel bar to be counted is collected, and two-dimensional image is obtained;Two-dimensional image is input into monocular depth estimation model, and three-dimensional point cloud is generated;The ground plane is fitted in three-dimensional point cloud, the distance of each point to ground plane is calculated, three-dimensional point cloud is segmented into different layers, and the corresponding primary steel bar image of each layer is output;For the missing area in the next layer steel bar image due to the shielding of the last layer, pixel repair is carried out, and the complete steel bar image corresponding to each layer after repair is obtained;Obtain steel bar image, train instance segmentation model;Complete steel bar image is input into instance segmentation model, the number of steel bars in each layer is counted, and visual output is carried out;The application is based on monocular image multi-layer steel bar automatic layering, shielding repair, accurate segmentation, layer counting and visual output, which greatly reduces the equipment deployment cost and difficulty in the field, and improves the intelligent level of steel bar concealed engineering acceptance.

Description

Technical Field

[0001] This application relates to the field of building engineering testing, and in particular to a deep learning-based intelligent counting method and system for multi-layer steel reinforcement. Background Technology

[0002] Driven by the demand for resilient infrastructure construction, reinforced concrete structures, with their excellent stress distribution, ease of construction, and economy, have become an indispensable structural form in modern engineering. In these structures, the internal steel reinforcement cage, as the main load-bearing component, fundamentally determines the structure's load-bearing capacity and long-term durability. Before concrete pouring, a comprehensive inspection of the concealed reinforcement work must be conducted to ensure that the quantity, specifications, spacing, and arrangement of the reinforcement bars strictly conform to the design drawings. Inadequate reinforcement quantity can seriously endanger structural safety, trigger major construction accidents, cause irreversible performance degradation, and shorten the structure's service life.

[0003] While rebar inspection is crucial, on-site manual verification remains a challenging and labor-intensive task. In modern large-scale projects such as bridge decks and heavy foundations, rebar is often densely bundled and stacked in multiple layers to meet high strength requirements, which easily leads to visual fatigue for inspectors and makes it difficult to accurately count the lower layers of rebar. Furthermore, manual on-site inspection exposes workers to serious safety risks such as falls, mechanical injuries, and rebar cuts. Therefore, there is an urgent need to develop intelligent and automated inspection technologies to reliably verify the quantity of complex, overlapping rebar.

[0004] Computer vision and 3D reconstruction technologies have been widely applied in civil engineering, but achieving low-cost, spatially perceptive rebar detection remains a significant challenge. Currently used technologies, on the one hand, rely on low-cost RGB cameras to identify rebar in a two-dimensional plane. However, monocular images inherently lack spatial depth information, making it impossible to decouple and count multi-layered rebar meshes in severely occluded scenes. On the other hand, 3D equipment, such as depth cameras or 3D laser scanners, can accurately obtain scene depth and acquire the spatial geometry of multi-layered rebar. However, these devices are expensive, difficult to operate, and computationally costly, making them difficult to deploy flexibly in dynamic construction sites. Summary of the Invention

[0005] This application provides a deep learning-based intelligent counting method and system for multi-layer rebar, which realizes automatic layering, occlusion repair, accurate segmentation, layer counting and visualization output of multi-layer rebar based on monocular images, significantly reducing the on-site equipment deployment cost and difficulty and improving the intelligent level of rebar concealed works acceptance.

[0006] This application provides a deep learning-based intelligent counting method for multi-layer steel reinforcement, including the following steps: S201, acquiring a top view image of the multi-layer steel reinforcement mesh to be counted to obtain a two-dimensional image; S202. Input the acquired 2D image into a monocular depth estimation model to generate a corresponding 3D point cloud and recover the 3D structure of the multi-layer steel mesh; S203. Fit the 3D point cloud to the ground plane, calculate the distance from each point to the ground plane, segment the 3D point cloud into different layers, and output the primary steel mesh image corresponding to each layer; S204. For the missing areas in the next layer of steel mesh image caused by the occlusion of the previous layer, use the fast traversal method to perform pixel repair, and obtain the complete steel mesh image corresponding to each layer after repair; S205. Acquire images of steel bars and steel mesh, construct and train an instance segmentation model to perform steel bar image segmentation and realize steel bar counting; S206. Input the complete steel bar image into the instance segmentation model, count the number of steel bars in each layer, and output the visualization.

[0007] Optionally, S201 includes: based on a monocular depth estimation model, inferring the relative depth value of each pixel in the two-dimensional image according to the relative relationship between pixels in the two-dimensional image, using the relative depth value as the z coordinate of each pixel, and the x and y coordinates as the pixel coordinates of that point in the two-dimensional image, to obtain the three-dimensional coordinates of each pixel, and combining the three-dimensional coordinates of each pixel to generate a three-dimensional point cloud, thereby recovering the three-dimensional structure of the multi-layer steel mesh.

[0008] Optionally, S203 includes: fitting a ground plane in the generated 3D point cloud based on a random sampling consensus algorithm to obtain a ground plane equation, calculating the distance from each pixel in the 3D point cloud to the ground plane; modeling the distance distribution from each pixel to the ground plane based on a Gaussian mixture model, determining the layer to which each pixel belongs, realizing automatic layering of multi-layer steel reinforcement, and outputting the primary steel reinforcement image corresponding to each layer.

[0009] Optionally, in step S203, the modeling of the distance distribution from each pixel to the ground plane based on the Gaussian mixture model includes: taking the number of steel mesh layers as the distribution number of the Gaussian mixture model; using the expectation-maximization algorithm to iteratively estimate the mean, variance, and weight of each sub-distribution; wherein, the sub-distribution refers to the pixel distribution of each layer of steel mesh; selecting the average of the means of two adjacent sub-distributions as the layering threshold, and assigning pixels with a distance less than the layering threshold to the ground plane to the upper layer and pixels with a distance greater than the layering threshold to the lower layer; and inversely mapping the distance back to the coordinates of each pixel to determine the layer to which each pixel belongs.

[0010] Optionally, in S204, the pixel repair using the fast-moving method includes: using the effective edge of the occluded area of ​​the lower layer of steel reinforcement image as the initial evolution condition, and gradually filling all content within the boundary; when repairing pixels, using the normalized weighted sum of all known pixels in the neighborhood to replace unknown pixels; and determining the degree of influence of each known pixel on the new pixel value through a weight function to achieve a smooth repair effect.

[0011] Optionally, in S205, the step of acquiring images of reinforcing bars and reinforcing mesh, constructing and training an instance segmentation model to perform reinforcing bar image segmentation and achieve reinforcing bar counting includes: acquiring images of reinforcing bars and reinforcing mesh; unifying the resolution of the images of reinforcing bars and reinforcing mesh and dividing them into training sets and validation sets; performing brightness adjustment, contrast adjustment, histogram equalization, Gaussian blurring, and random erasure data enhancement processing on the images in the training set to obtain an image dataset; and saving the image dataset in the format of the MSCOCO instance segmentation dataset.

[0012] Optionally, in S205, the step of acquiring images of reinforcing bars and reinforcing mesh, constructing and training an instance segmentation model to perform reinforcing bar image segmentation and achieve reinforcing bar counting further includes: preserving reinforcing bar texture details based on a high-frequency feature extraction branch; strengthening the reinforcing bar contour boundaries based on an edge feature enhancement branch; and adaptively weighting effective feature channels and suppressing background noise based on a channel attention branch.

[0013] Optionally, step S206 includes: fitting the central axis of the rebar segmentation mask using the least squares method and superimposing it onto the original image to complete the visualization output; extracting the boundary point coordinates of a certain rebar mask output by the instance segmentation model; calculating the geometric center of the boundary point coordinates; subtracting the geometric center coordinates from all boundary point coordinates to achieve alignment; calculating the covariance matrix and performing eigenvalue decomposition, taking the eigenvector corresponding to the largest eigenvalue as the direction of the rebar central axis; projecting the boundary points onto the direction of the rebar central axis, finding the two farthest endpoints of the projection range as the endpoints of the central axis; drawing line segments on the original color image based on the endpoint coordinates of the central axis, and superimposing them onto the original image to complete the visualization output.

[0014] Secondly, this application provides a deep learning-based intelligent counting system for multi-layer rebar meshes. The system includes: an acquisition module for acquiring a top-view image of the multi-layer rebar mesh to be counted, obtaining a two-dimensional image; a recovery module for inputting the acquired two-dimensional image into a monocular depth estimation model to generate a corresponding three-dimensional point cloud, recovering the three-dimensional structure of the multi-layer rebar mesh; a layering module for fitting a ground plane to the three-dimensional point cloud, calculating the distance from each point to the ground plane, segmenting the three-dimensional point cloud into different layers, and outputting the primary rebar image corresponding to each layer; a repair module for performing pixel repair on missing areas in the next layer's rebar image caused by occlusion in the previous layer using a fast traversal method, obtaining the repaired complete rebar image corresponding to each layer; a counting module for acquiring images of rebars and rebar meshes, constructing and training an instance segmentation model to perform rebar image segmentation and achieve rebar counting; and an output module for inputting the complete rebar image into the instance segmentation model, counting the number of rebars in each layer, and providing a visual output.

[0015] This application has at least the following advantages: The above steps primarily involve acquiring 2D images using a mobile phone or camera, then processing them with a monocular depth estimation model and 3D point clouds to reconstruct the 3D structure of multi-layered steel reinforcement mesh. This achieves precise decoupling of the multi-layered steel reinforcement, eliminating reliance on specialized equipment like depth cameras. Multi-layered steel reinforcement can be achieved simply by taking photos with a mobile phone. A rapid traversal method effectively repairs occluded areas, minimizing blurring or illusions during the repair of small targets, resulting in good repair quality. The detection and visualization output can be completed using only conventional imaging equipment. The overall solution is low-cost, easy to deploy and use, and can reliably handle complex construction site environments and densely overlapping steel reinforcement scenarios, providing key technical support for the intelligent acceptance of concealed steel reinforcement projects. Attached Figure Description

[0016] Figure 1 This is a diagram illustrating the application environment of a deep learning-based intelligent counting method for multi-layer rebar in one embodiment. Figure 2 This is a flowchart illustrating the steps of a deep learning-based intelligent counting method for multi-layer rebar in one embodiment. Figure 3 Here is a flowchart illustrating the process structure of a deep learning-based intelligent counting method for multi-layer rebar in one embodiment; Figure 4 This is a schematic diagram illustrating the structure for generating a 3D point cloud map based on a monocular depth estimation model from a 2D image in one embodiment. Figure 5 This is a schematic diagram illustrating a structure for automatic layering of multi-layer steel reinforcement based on a Gaussian mixture model in one embodiment. Figure 6This is a schematic diagram illustrating the structure of obtaining a complete image by repairing missing pixels based on the fast traversal method in one embodiment; Figure 7 This is a schematic diagram illustrating rebar image segmentation based on an instance segmentation model in one embodiment. Figure 8 This is a structural schematic diagram showing the statistical output of the number of steel bars in each layer in one embodiment; Figure 9 The following is a structural block diagram of a deep learning-based intelligent rebar counting system in one embodiment; Figure 10 This is a schematic structural diagram of a computer device in one embodiment. Detailed Implementation

[0017] The present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the scope of the present application.

[0018] For ease of understanding, the system to which this application applies is first described. This application provides a deep learning-based intelligent counting method for multi-layer rebar, which can be applied to, for example... Figure 1 The system architecture shown includes a user-space file server 103 and a terminal device 101. The terminal device 101 communicates with the user-space file server 103 via a network. The user-space file server 103 can be a file server based on the NFSv3 / v4 protocol, running in a Linux environment. NFS (Network File System) is a network abstraction on top of a file system, allowing remote clients running on the terminal device 101 to access the file system over the network in a manner similar to a local file system. The terminal device 101 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, etc. The user-space file server 103 can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0019] Figure 2 A flowchart illustrating a deep learning-based intelligent counting method for multi-layer rebar, provided in this application embodiment, includes the following steps: S201. Acquire a top-view image of the multi-layer steel mesh to be counted to obtain a two-dimensional image; S202. Input the acquired 2D image into the monocular depth estimation model to generate the corresponding 3D point cloud and recover the 3D structure of the multi-layer steel mesh; S203. Fit the ground plane to the 3D point cloud, calculate the distance from each point to the ground plane, segment the 3D point cloud into different layers, and output the primary steel mesh image corresponding to each layer; S204. For the missing areas in the next layer of steel mesh image caused by the occlusion of the previous layer, use the fast traversal method to perform pixel repair, and obtain the complete steel mesh image corresponding to each layer after repair; S205. Acquire images of steel bars and steel mesh, construct and train an instance segmentation model to perform steel bar image segmentation and realize steel bar counting; S206. Input the complete rebar image into the instance segmentation model, count the number of rebars in each layer, and output the visualization.

[0020] In this embodiment, it should be noted that the three-dimensional structure of the multi-layer steel mesh is recovered by combining the monocular depth estimation model with the three-dimensional point cloud, realizing the precise decoupling of the multi-layer steel mesh. The occluded area is effectively repaired by the fast travel method, and the segmentation reliability is improved by combining the instance segmentation model. The detection and visualization output can be completed using only conventional imaging equipment. The overall solution is low-cost, easy to deploy and use, and can stably cope with the complex environment and densely overlapping steel mesh scene of the construction site, providing key technical support for the intelligent acceptance of steel mesh concealed works.

[0021] The following is a detailed explanation of each step: Please refer to Figure 2 , Figure 3 As shown, in step S201, a top view image of the multi-layer steel mesh to be counted is acquired to obtain a two-dimensional image.

[0022] In this embodiment, it should be noted that an industrial camera or a high-resolution smartphone is used to capture a vertically overhead view of the multi-layered steel mesh to be inspected, obtaining an RGB two-dimensional image. The overhead view of the target is prioritized because there is less angular distortion in this view, resulting in higher accuracy for subsequent algorithms. An image resolution of at least 1920×1080 pixels is recommended to ensure the accuracy of subsequent depth estimation and segmentation.

[0023] Please refer to Figure 2 , Figure 3 As shown, in step S202, the acquired two-dimensional image is input into the monocular depth estimation model to generate the corresponding three-dimensional point cloud and recover the three-dimensional structure of the multi-layer steel mesh.

[0024] In this embodiment, it should be noted that the acquired 2D image is input into a pre-trained monocular depth estimation model, Pixel-perfect depth. This model is trained on a large-scale depth dataset and can infer the relative depth value of each pixel based on monocular cues such as perspective patterns, texture, shadows, and relative size in the image.

[0025] Specifically, in one example, such as Figure 4 As shown, the relative depth value output by the monocular depth estimation model is used as the z-coordinate of each pixel, and the column index of the pixel in the image is used as the x-coordinate and the row index as the y-coordinate, thus obtaining the three-dimensional coordinates (x, y, z) of each pixel. The three-dimensional coordinates of all pixels together constitute a sparse or dense three-dimensional point cloud, which intuitively reflects the hierarchical structure of the multi-layer steel mesh in the vertical direction, thereby recovering the three-dimensional structure of the multi-layer steel mesh.

[0026] Please continue to refer to Figure 2 , Figure 3 As shown, in step S203, the three-dimensional point cloud is fitted to the ground plane, the distance from each point to the ground plane is calculated, the three-dimensional point cloud is divided into different layers, and the primary reinforcement image corresponding to each layer is output.

[0027] In this embodiment, it should be noted that calculating the distance to the ground plane is to eliminate perspective gradient errors introduced by the camera's tilted shooting. Since the depth value obtained from depth estimation is the distance from the target point to the camera, when the camera tilts to shoot the steel mesh, the depth of steel pixels at the same physical level will vary depending on their distance from the camera. By calculating the orthogonal distance from each pixel in the point cloud to the fitted ground plane (reference plane), it can be ensured that the point cloud feature values ​​of steel at the same level are realigned to the same reference plane, thus providing a reliable physical prior for subsequent accurate layering using a unified threshold. Figure 5 As shown, the ground plane fitting uses the Random Sample Consensus (RANSAC) algorithm to fit the ground plane in the generated 3D point cloud. The specific steps are as follows: three points are randomly selected to determine a candidate plane; the number of interior points within a set distance threshold is counted; after multiple iterations, the plane with the most interior points is selected as the ground plane, resulting in the ground plane equation ax + by + cz + d = 0, where a, b, and c represent the three components of the ground plane normal vector; and d represents the signed distance from the plane to the origin (the offset in the direction of the normal vector). The vertical distance from each pixel in the 3D point cloud to this ground plane is calculated.

[0028] Please continue to refer to Figure 5 As shown, a Gaussian mixture model (GMM) is used to model the distance distribution of each pixel to the ground plane to achieve layering. In this embodiment, it should be noted that a common approach is to first obtain a 3D point cloud using a depth camera, then perform clustering based on the z-coordinate to achieve layering, or to project the 3D point cloud obtained from the depth camera onto the depth direction to determine a layering threshold and achieve layering. However, this approach cannot adaptively and dynamically fit the threshold based on data characteristics to achieve automatic layering. Here, a Gaussian Mixture Model (GMM) is used to model the distances from all rebar pixels to the reference plane and from the rebar mesh pixels to the ground plane, thereby finding an adaptive dynamic threshold that can achieve rebar layering.

[0029] Specifically, we first assume that the distance from the pixels of each layer of the multi-layer steel mesh to the ground plane is basically the same within the same layer, but there are significant differences between different layers. We then take the number of mesh layers as the distribution number of the Gaussian mixture model, such as... Figure 5 As shown, the number of steel mesh layers is set as the distribution number K of the Gaussian sub-distribution. For example, if there are two layers, upper and lower, then K=2. The mean μ of each Gaussian sub-distribution is estimated iteratively using the Expectation-Maximization (EM) algorithm. k σ k² and weight π k Each sub-distribution corresponds to the pixel distance distribution of a layer of steel mesh, which is described by the formula: In the formula, t represents the number of iterations. The weights of the Gaussian subdistribution are represented by x, where N represents the total number of samples. i This represents the pixel of the i-th sample, i.e., the steel mesh. Indicates sample x i The probability of it occurring under the k-th Gaussian distribution.

[0030] In one example, the average of the means of two adjacent sub-distributions is chosen as the stratification threshold, described by the formula: T=(μ k +μ k+1 ) / 2, In the formula, μ k This represents the mean distance distribution of pixels corresponding to each sub-distribution of the rebar mesh. Pixels with a distance less than T are assigned to the upper layer, and those greater than T are assigned to the lower layer. The pixel coordinates are inversely mapped back to the original image space, and the primary images of the upper and lower rebar layers are output respectively. By using a Gaussian mixture model, the threshold can be dynamically fitted adaptively according to the data characteristics, resulting in stronger interpretability and minimizing the impact of noise in depth estimation, thus making the results more stable.

[0031] Please continue to refer to Figure 2 , Figure 3As shown, in step S204, the missing areas in the next layer of steel reinforcement image caused by the occlusion of the previous layer are repaired by using the fast traversal method to obtain the complete steel reinforcement images corresponding to each layer after repair.

[0032] In this embodiment, it should be noted that, as Figure 6 As shown, the upper layer of reinforcing bars can create shadows or occlusions in the image of the lower layer of reinforcing bars, causing the lower layer of reinforcing bars to appear broken or missing. This embodiment uses the Fast Marching Method to repair the primary image of the lower layer of reinforcing bars. The Fast Marching Method has a fast processing speed and can minimize blurring or illusions when repairing small targets, resulting in better repair effects.

[0033] Specifically, the repair process is as follows: The boundary of the area to be repaired is determined, and the effective edge of the occluded area of ​​the lower layer of steel reinforcement image is used as the initial evolution condition. Since the edge of the occluded area of ​​the lower layer of steel reinforcement image often retains some steel reinforcement texture information, using this as the initial evolution condition can restore some steel reinforcement shape, which helps with subsequent steel reinforcement segmentation. Following the order of distance from the boundary from nearest to farthest, the value of the unknown pixel is calculated by replacing the unknown pixel with a normalized weighted sum of all known pixels in the neighborhood, and described by the formula: In the formula, I(p) represents the pixel value of the unknown point p, and I(q) represents the pixel value of the known point q. This represents the image gradient at point q. Let w(p,q) represent a small neighborhood of point p. Let w(p,q) represent the weight function of point q relative to point p based on distance and direction. Let dir(p,q) measure whether the direction from point q to p is consistent with the direction of the repair boundary advancement. The more consistent the direction, the greater the weight. Let dst(p,q) measure the distance of point q from p. The closer the distance, the greater the weight. Let lev(p,q) measure whether the distances from point q and point p to the boundary of the area to be repaired are similar. The closer they are, the greater the weight.

[0034] The process proceeds pixel by pixel, using a weighting function to determine the influence of each known pixel on the new pixel value, until the entire missing area is filled, achieving a smoother repair effect and obtaining a complete image of the lower layer of steel reinforcement. Repeat the above repair steps for the (N+1)th layer, with the repair process based on the image after repair of the Nth layer.

[0035] Please continue to refer to Figure 2 , Figure 3 As shown, in step S205, images of reinforcing bars and reinforcing mesh are acquired, and an instance segmentation model is constructed and trained to perform reinforcing bar image segmentation and achieve reinforcing bar counting.

[0036] In this embodiment, it should be noted that the instance segmentation model Mask2Former is constructed. Mask2Former adopts the transformer framework, whose self-attention mechanism can better capture global contextual relationships and achieve more refined and accurate segmentation. Specifically, the dataset is first prepared by collecting images of rebar mesh under different lighting conditions, angles, and rebar densities. These images are then uniformly scaled to a fixed resolution, such as 1333*800, and divided into training and validation sets in an 8:2 ratio. The outline of each rebar is labeled using a polygon annotation tool (such as LabelMe), with the label "rebar". The training set is then subjected to enhancement operations such as random brightness adjustment (0.6-1.4 times), contrast adjustment, histogram equalization, Gaussian blur (kernel size 3×3 or 5×5), and random erasure to improve the instance segmentation model.

[0037] A rebar feature enhancement module is added to the feature processing part of the Mask2Former instance segmentation model. The rebar feature enhancement module includes: a high-frequency feature extraction branch, which extracts the texture details of the rebar surface through the Laplacian operator or the Sobel operator; an edge feature enhancement branch, which enhances the rebar contour through Canny edge detection or a learnable edge convolutional layer; and a channel attention branch, which uses the SE module or the ECA module to adaptively weight the feature channels and suppress background noise.

[0038] The Mask2Former instance segmentation model was trained and constructed. The dataset was saved in MSCOCO format. The AdamW optimizer was used during model training with an initial learning rate of 0.0001, momentum of 0.9, and weight decay of 0.1. Training continued until the training set loss tended to converge.

[0039] Reference Figure 2 , Figure 3 As shown, in step S206, the complete rebar image is input into the instance segmentation model, the number of rebars in each layer is counted, and the result is visualized.

[0040] In this embodiment, it should be noted that, as Figure 7 As shown, the repaired complete rebar images (upper and lower layers) are input into the trained instance segmentation model, and the model outputs a mask for each rebar. The number of masks in each layer is counted to obtain the number of rebars in each layer.

[0041] To facilitate manual review, such as Figure 8As shown, the least squares method is used to fit the central axis of the rebar segmentation mask to find a straight line that best represents the centerline position and direction of the rebar area. This allows for the visualization of each layer of rebar. Specifically, the coordinates of all boundary points of a rebar mask are extracted; the geometric center C of the boundary points is calculated, and the coordinates of all boundary points are subtracted from the geometric center coordinates to achieve alignment; then, the covariance matrix S of the coordinate matrix is ​​calculated, and eigenvalue decomposition is performed. The eigenvector corresponding to the largest eigenvalue is taken as the direction of the rebar central axis; all boundary points are projected onto this rebar central axis direction, and the farthest two endpoints of the projected line segment are taken as the endpoints of the central axis; the line segment is directly drawn on the original color image using the endpoint coordinates of the central axis and labeled with a number. The final output includes a statistical table of the number of rebars in each layer and a visualization image with axis labels, obtaining the number of rebars in each layer, described by a formula: In the formula, C represents the geometric center of the boundary point, (x i ,y i Let represent the coordinates of the i-th boundary point, N represent the total number of boundary points, S represent the covariance matrix of the coordinate matrix, λ represent the eigenvalues, and u represent the eigenvectors.

[0042] The implementation principle of this embodiment is as follows: The above steps mainly involve acquiring 2D images using a mobile phone or camera, and then processing these images using a monocular depth estimation model combined with 3D point clouds to reconstruct the 3D structure of a multi-layered steel reinforcement mesh. This achieves precise decoupling of the multi-layered steel reinforcement, eliminating reliance on specialized equipment like depth cameras. Multi-layered steel reinforcement can be achieved simply by taking photos with a mobile phone. A fast-travel method effectively repairs occluded areas, minimizing blurring or illusions during the repair of small targets, resulting in good repair performance. An instance segmentation model with an embedded steel reinforcement feature enhancement module further enhances segmentation reliability. Detection and visualization can be completed using only conventional imaging equipment. The overall solution is low-cost, easy to deploy and use, and can stably handle complex construction site environments and densely overlapping steel reinforcement scenarios, providing key technical support for the intelligent acceptance of concealed steel reinforcement projects.

[0043] Reference Figure 9 As shown in the figure, this application embodiment also provides a multi-layer rebar intelligent counting system based on deep learning. The system may include: a data acquisition module 301, a recovery module 302, a layering module 303, a repair module 304, a counting module 305, and an output module 306; wherein the main functions of each component module are as follows: The acquisition module 301 is used to acquire a top view image of the multi-layer steel mesh to be counted, and obtain a two-dimensional image. The recovery module 302 is used to input the acquired 2D image into the monocular depth estimation model to generate the corresponding 3D point cloud and recover the 3D structure of the multi-layer steel mesh; the layering module 303 is used to fit the ground plane on the 3D point cloud, calculate the distance from each point to the ground plane, segment the 3D point cloud into different layers, and output the primary steel mesh image corresponding to each layer; the repair module 304 is used to perform pixel repair on the missing areas in the next layer of steel mesh image caused by the occlusion of the previous layer using the fast traversal method to obtain the complete steel mesh image corresponding to each layer after repair; the counting module 305 is used to acquire images of steel bars and steel mesh, construct and train an instance segmentation model to perform steel bar image segmentation and realize steel bar counting. The output module 306 is used to input the complete rebar image into the instance segmentation model, count the number of rebars in each layer, and output the visualization.

[0044] like Figure 10 The diagram shown is a block diagram of a computer device according to an embodiment of this application. The term "computer device" is intended to represent various forms of digital computers or mobile devices. The digital computer may include a desktop computer, a portable computer, a workbench, a personal digital assistant, a server, a mainframe computer, and other suitable computers. The mobile device may include a tablet computer, a smartphone, a wearable device, etc.

[0045] like Figure 10 As shown, device 600 includes a computing unit 601, a ROM 602, a RAM 603, a bus 604, and an I / O interface 605. The computing unit 601, ROM 602, and RAM 603 are interconnected via the bus 604. The I / O interface 605 is also connected to the bus 604.

[0046] The computing unit 601 can execute various processes in the method embodiments of this application according to computer instructions stored in the read-only memory (ROM) 602 or computer instructions loaded from the storage unit 608 into the random access memory (RAM) 603. The computing unit 601 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. The computing unit 601 can include, but is not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. In some embodiments, the methods provided in the embodiments of this application can be implemented as computer software programs, which are tangibly contained in a computer-readable storage medium, such as the storage unit 608.

[0047] RAM 603 can also store various programs and data required for the operation of device 600. Part or all of the computer program can be loaded and / or installed on device 600 via ROM 602 and / or communication unit 609.

[0048] The input unit 606, output unit 607, storage unit 608, and communication unit 609 in device 600 can be connected to I / O interface 605. The input unit 606 can be, for example, a keyboard, mouse, touchscreen, or microphone; the output unit 607 can be, for example, a display, speaker, or indicator light. Device 600 can exchange information and data with other devices through the communication unit 609.

[0049] It should be noted that the device may also include other components necessary for normal operation. It may also include only the components necessary for implementing the solution of this application, without necessarily including all the components shown in the figures.

[0050] Various implementations of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SOCs), payload programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof.

[0051] The computer instructions used to implement the methods of this application may be written in any combination of one or more programming languages. These computer instructions may be provided to the computing unit 601 such that when executed by the computing unit 601, such as a processor, the computer instructions cause the execution of the steps involved in the embodiments of the methods of this application.

[0052] The computer-readable storage medium provided in this application can be a tangible medium that can contain or store computer instructions for performing the steps involved in the method embodiments of this application. The computer-readable storage medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, and other forms of storage media.

[0053] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A deep learning-based intelligent counting method for multi-layer rebar, characterized in that, Includes the following steps: S201. Acquire a top-view image of the multi-layer steel mesh to be counted to obtain a two-dimensional image; S202. Input the acquired 2D image into a monocular depth estimation model to generate a corresponding 3D point cloud and recover the 3D structure of the multi-layer steel mesh; S203. Fit the 3D point cloud to the ground plane, calculate the distance from each point to the ground plane, segment the 3D point cloud into different layers, and output the primary steel mesh image corresponding to each layer; S204. For the missing areas in the next layer of steel mesh image caused by the occlusion of the previous layer, use the fast traversal method to perform pixel repair, and obtain the complete steel mesh image corresponding to each layer after repair; S205. Acquire images of steel bars and steel mesh, construct and train an instance segmentation model to perform steel bar image segmentation and realize steel bar counting; S206. Input the complete rebar image into the instance segmentation model, count the number of rebars in each layer, and output the visualization.

2. The method according to claim 1, characterized in that, S202 includes: Based on the monocular depth estimation model, the relative depth value of each pixel in the two-dimensional image is inferred according to the relative relationship between pixels in the two-dimensional image. The relative depth value is used as the z coordinate of each pixel, and the x and y coordinates are the pixel coordinates of that point in the two-dimensional image. The three-dimensional coordinates of each pixel are obtained. The three-dimensional point cloud is generated by combining the three-dimensional coordinates of each pixel, and the three-dimensional structure of the multi-layer steel mesh is recovered.

3. The method according to claim 1, characterized in that, S203 includes: The ground plane is fitted in the generated 3D point cloud based on the random sampling consensus algorithm to obtain the ground plane equation, and the distance from each pixel in the 3D point cloud to the ground plane is calculated. The distance distribution from each pixel to the ground plane is modeled based on the Gaussian mixture model to determine the layer to which each pixel belongs, realize the automatic layering of multi-layer steel reinforcement, and output the primary steel reinforcement image corresponding to each layer.

4. The method according to claim 3, characterized in that, In step S203, the modeling of the distance distribution from each pixel to the ground plane based on the Gaussian mixture model includes: The number of steel mesh layers is taken as the distribution number of the Gaussian mixture model; The mean, variance, and weight of each sub-distribution are estimated iteratively using the expectation-maximization algorithm; wherein, the sub-distribution refers to the pixel distribution of each layer of steel mesh. The average of the means of two adjacent sub-distributions is selected as the layering threshold. The pixels of the steel mesh that are less than the layering threshold from the ground plane are in the upper layer, and those that are greater than the layering threshold are in the lower layer. The coordinates of each pixel are mapped back to the distance to determine the layer to which each pixel belongs.

5. The method according to claim 1, characterized in that, In step S204, pixel restoration using the fast-travel method includes: Using the effective edges of the occluded area in the next layer of the steel reinforcement image as the initial evolution condition, all content within the boundary is gradually filled in; When repairing pixels, the normalized weighted sum of all known pixels in the neighborhood is used to replace unknown pixels; By using a weighting function to determine the degree of influence of each known pixel on the new pixel value, a smooth restoration effect can be achieved.

6. The method according to claim 1, characterized in that, In S205, the step of acquiring images of reinforcing bars and reinforcing mesh, constructing and training an instance segmentation model to perform reinforcing bar image segmentation and achieve reinforcing bar counting includes: Images of reinforcing bars and reinforcing mesh are acquired, the resolution of the images is standardized, and they are divided into training and validation sets. The images in the training set were subjected to brightness adjustment, contrast adjustment, histogram equalization, Gaussian blur, and random erasure data augmentation to obtain the image dataset. The image dataset is saved in the format of an MSCOCO instance segmentation dataset.

7. The method according to claim 6, characterized in that, In S205, the step of acquiring images of reinforcing bars and reinforcing mesh, constructing and training an instance segmentation model to perform reinforcing bar image segmentation and achieve reinforcing bar counting further includes: Based on high-frequency feature extraction branches, the details of the steel reinforcement texture are preserved; Strengthen branches based on edge features to enhance the outline boundary of reinforcing bars; Based on channel attention branching, the effective feature channels are adaptively weighted and background noise is suppressed.

8. The method according to any one of claims 1-7, characterized in that, S206 includes: The center axis of the rebar segmentation mask is fitted using the least squares method and then superimposed onto the original image to complete the visualization output. Extract the boundary point coordinates of a certain rebar mask output by the instance segmentation model; Calculate the geometric center of the boundary point coordinates, and subtract the geometric center coordinates from the coordinates of all boundary points to achieve alignment; Calculate the covariance matrix and perform eigenvalue decomposition. Take the eigenvector corresponding to the largest eigenvalue as the direction of the central axis of the reinforcing bar. Project the boundary points onto the direction of the central axis of the reinforcing bar, and find the two farthest endpoints of the projection range as the endpoints of the central axis; Draw line segments on the original color image based on the endpoint coordinates of the central axis, and overlay them onto the original image to complete the visualization output.

9. A multi-layer rebar intelligent counting system based on deep learning, characterized in that: The acquisition module is used to acquire a top-view image of the multi-layer steel mesh to be counted, and obtain a two-dimensional image. The recovery module is used to input the acquired two-dimensional image into the monocular depth estimation model to generate the corresponding three-dimensional point cloud and recover the three-dimensional structure of the multi-layer steel mesh. The layering module is used to fit the ground plane to the three-dimensional point cloud, calculate the distance from each point to the ground plane, divide the three-dimensional point cloud into different layers, and output the primary reinforcement image corresponding to each layer. The repair module is used to repair the missing areas in the next layer of rebar image caused by the occlusion of the previous layer using a fast traversal method, so as to obtain the complete rebar image corresponding to each layer after repair. The counting module is used to acquire images of reinforcing bars and reinforcing mesh, build and train an instance segmentation model to perform reinforcing bar image segmentation and realize reinforcing bar counting; The output module is used to input the complete rebar image into the instance segmentation model, count the number of rebars in each layer, and output the visualization.

Images