A Machine Vision-Based Method for Identifying Intersections of Tunnel Secondary Lining Reinforcement Mesh
By using multimodal image enhancement and deformation sensing networks, combined with geometric consistency constraints and phased training, the problems of low efficiency, insufficient accuracy and safety risks in the detection of tunnel reinforcement mesh intersections are solved, and high-precision and robust automatic identification of tunnel secondary lining reinforcement mesh intersections is achieved.
Patent Information
- Application Number
- CN202610464698.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2026-04-09
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2046-04-09
AI Technical Summary
Existing technologies for detecting steel mesh intersections in tunnel engineering suffer from low efficiency, insufficient accuracy, high safety risks, and unstable model training. In particular, it is difficult to achieve high-precision and robust automatic identification under complex lighting and perspective distortion conditions.
We employ multimodal image enhancement, fine-grained intersection feature enhancement, and deformation-aware networks, combined with geometric consistency constraints and phased progressive training. By constructing a dataset for detecting intersections of the tunnel secondary lining steel mesh, we use a conditional image generator and a deformation foreground mask for data enhancement, extract features using multi-scale Gaussian convolution kernels, and optimize network parameters through a total loss function to achieve sub-pixel-level accurate localization.
It achieves high-precision and robust identification of steel mesh intersections in complex tunnel environments, improving detection efficiency and intelligent supervision of construction quality, and ensuring sub-pixel-level positioning accuracy and system stability.
Smart Images

Figure CN122435310B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of machine vision technology, specifically a machine vision-based method for identifying intersections of steel mesh reinforcement in tunnel secondary lining. Background Technology
[0002] In tunnel construction, the construction quality of the secondary lining reinforcement mesh directly affects the long-term safety and stability of the tunnel structure. The location, density, and distribution of the reinforcement mesh intersections are key indicators for assessing the binding quality and whether the spacing meets design specifications. Traditionally, this inspection work mainly relies on manual visual inspection or string line measurement, which is not only inefficient and heavily influenced by personnel experience, but also poses safety risks and makes it difficult to guarantee the comprehensiveness and accuracy of measurements in the harsh environment of tunnels, which are often high-altitude, dimly lit, and damp. With the development of machine vision technology, automatically identifying reinforcement mesh intersections using images has become a research direction in the industry.
[0003] However, existing technologies also have the following drawbacks: 1. Conventional data augmentation techniques mainly simulate apparent changes, making it difficult to generate complex physical conditions unique to tunnels, such as point light sources, mixed lighting, wet reflections, and perspective distortion. This results in insufficient generalization ability of the model to real complex working conditions.
[0004] 2. Conventional feature extraction networks are not designed for pixel-level micro-targets with specific directions and spatial structures, such as steel bar intersections. They have limited discriminative power for intersection features affected by corrosion, occlusion, and deformation, and are easily interfered with by complex background textures.
[0005] 3. Conventional target detection or key point detection methods lack effective geometric prior guidance when dealing with perspective distortion caused by shooting angle, oblique view, etc. in tunnel scenes. This can easily lead to positioning errors or missed detections. In addition, they usually only output pixel-level coordinates, which is difficult to meet the engineering measurement requirements of sub-pixel accuracy.
[0006] 4. For complex models involving data generation, feature engineering, and recognition tasks, end-to-end one-time training is often adopted. This can easily lead to training instability, difficulty in convergence, or getting stuck in local optima due to insufficient initialization of components, making it difficult to systematically optimize the overall pipeline performance. Summary of the Invention
[0007] To address the shortcomings of existing technologies, this invention provides a machine vision-based method for identifying intersections of steel mesh reinforcement in tunnel secondary lining. By employing multimodal image enhancement, fine-grained intersection feature enhancement, and deformation perception networks, combined with geometric consistency constraints and phased progressive training, this method effectively overcomes challenges such as lighting interference, cluttered backgrounds, and steel mesh deformation in complex tunnel environments. It achieves high-precision, robust, and sub-pixel-level automatic identification of steel mesh intersections, significantly improving detection efficiency and the level of intelligent supervision of construction quality.
[0008] To achieve the above objectives, the present invention employs the following technical solution: A machine vision-based method for identifying intersections of steel reinforcement mesh in tunnel secondary lining includes the following steps: S1. Constructing detection data for intersections of the tunnel secondary lining steel mesh: Collecting original image data of the tunnel secondary lining steel mesh, labeling the original image data, and constructing a training dataset; S2. Construct a multimodal data augmentation module based on a coarse segmentation mask: separate the foreground steel reinforcement from the background lining generation, and synthesize the augmented image through a conditional image generator function and a background generator function; S3. Construct a fine-grained intersection feature enhancement extraction module: preprocess the enhanced image to generate an enhanced feature map that can highlight the specific geometric structure of the intersection. S4. Construct a deformation-aware intersection recognition network: Utilize the deformation foreground mask as an implicit geometric prior constraint, guide heat map prediction through weighted fusion, and jointly predict the sub-pixel level offset vector field to achieve sub-pixel level accurate positioning. S5. Design the total loss function for multi-task joint optimization: To jointly optimize the tasks of heatmap prediction and offset prediction, a geometric consistency constraint in the tunnel scenario is introduced. The heatmap focusing loss, offset smoothing L1 loss and deformation consistency loss are weighted and fused to construct the total loss function to drive the iterative update of network parameters. S6. Implement a phased progressive training strategy: decompose the complex system, which includes generation, filtering, and recognition, into five ordered stages: pre-trained generator, optimized filter, trained recognition network, end-to-end fine-tuning, and adversarial training. S7. Implement online identification of intersections of the secondary lining steel mesh in tunnels: Load the trained model parameters and output the set of precise coordinates of all identified steel mesh intersections in the original input image to complete the automated identification task of intersections of the secondary lining steel mesh in tunnels.
[0009] Furthermore, the conditional image generator function uses a dual-path structure to extract structural features and rendering lighting features respectively, and then fuses them to obtain an enhanced foreground rebar image.
[0010] Furthermore, S3 specifically refers to: Define a multi-scale anisotropic Gaussian convolution kernel group: taking into account the directional and specific spatial structure characteristics of steel bar intersections, a guideable and anisotropic two-dimensional Gaussian partial derivative convolution kernel is used as a feature enhancement filter. Feature enhancement convolution operation: Multi-scale convolution operation is performed on the enhanced image using multiple two-dimensional Gaussian partial derivative convolution kernels of different scales, and the convolution responses of different scales are weighted and fused to generate enhanced feature maps.
[0011] Furthermore, S4 specifically refers to: Deformation-aware intersection heatmap prediction: A deformation foreground mask is used as a geometric guide. Multi-channel features and deformation mask information are fused through weighted convolution to generate an intersection heatmap, which adaptively enhances the response in the deformation region. Subpixel-level crosspoint offset prediction: Predicts a two-dimensional offset vector for each pixel location, used to further refine the precise coordinates of crosspoints from the peak locations of the heatmap.
[0012] Furthermore, the total loss function in S5 is specifically expressed as follows: ; In the formula, Represents the total loss function. The weighting coefficients represent the focus loss of the heatmap. The weighting coefficients represent the offset smoothing L1 loss. The weighting coefficients representing the deformation consistency loss The heatmap indicates that the loss is focused. This indicates offset smoothing L1 loss; This indicates the loss of deformation consistency.
[0013] Furthermore, S5 also includes an improved form of focus loss to address the problem of extreme pixel imbalance between positive and negative samples at intersections and to enhance attention to samples that are difficult to classify.
[0014] Furthermore, S1 also includes data annotation, specifically: Pixel-level semantic segmentation and annotation are performed on the steel reinforcement area in the image to generate a binary mask with the same size as the original image. The steel reinforcement pixels are marked as foreground, and the concrete lining and other background pixels are marked as background. This mask is used for foreground and background separation in the multimodal enhancement module. The intersections of the steel mesh are precisely marked, with a pixel-level point marked at the center of each steel mesh intersection, and a heat map label of the actual intersection is generated based on this.
[0015] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This invention employs a multimodal enhancement module based on a coarse segmentation mask to separate and generate foreground steel bars from background lining. Through a condition generator and latent variables, it controlslably synthesizes enhanced images that simulate complex tunnel lighting, texture, and perspective deformation, overcoming the limitations of conventional data enhancement (such as rotation, cropping, and color jitter) in realistically simulating such complex physical scene changes.
[0016] 2. This invention employs a fine-grained intersection feature enhancement extraction module, which uses a set of learnable multi-scale anisotropic Gaussian partial derivative convolution kernels as a dedicated filter to convolve the enhanced image, generating an enhanced feature map that strongly responds to small cross-shaped targets, significantly amplifying the distinguishability of intersections from steel reinforcement and complex backgrounds at the feature level.
[0017] 3. This invention constructs a deformation-aware intersection recognition network, which innovatively uses a deformation foreground mask that reflects changes in shooting perspective as a geometric prior. It guides heatmap prediction through weighted fusion and jointly predicts sub-pixel-level offset vector fields, thereby achieving high-precision positioning of intersections affected by perspective deformation and effectively compensating for systematic biases.
[0018] 4. This invention adopts a phased progressive training strategy, which decomposes the complex system including generation, filtering and recognition into five orderly stages: pre-trained generator, optimized filter, trained recognition network, end-to-end fine-tuning and adversarial training. This ensures that the parameters of each component are stably and fully initialized and co-optimized, thereby improving the convergence and final performance of the overall model. Attached Figure Description
[0019] Appendix Figure 1 This is a flowchart of the present invention; Appendix Figure 2 It is the original input image; Appendix Figure 3 It is a mask image for segmenting the rough steel reinforcement area; Appendix Figure 4 It is a deformed foreground mask image; Appendix Figure 5 It enhances the image; Appendix Figure 6 These are the first four channels of the structural feature tensor. Appendix Figure 7 These are the first four channels of the lighting rendering feature tensor. Detailed Implementation
[0020] The present invention will be further illustrated below with reference to specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. Furthermore, it should be understood that after reading the teachings of this invention, those skilled in the art can make various alterations or modifications to the invention, and these equivalent forms also fall within the scope defined in this application.
[0021] like Figure 1 As shown, a machine vision-based method for identifying intersections of steel mesh reinforcement in tunnel secondary lining is disclosed. S1. Collect raw image data of the tunnel secondary lining steel mesh, and annotate the raw image data to construct a training dataset; Image data of the steel mesh for tunnel secondary lining was collected, and a corresponding training dataset was constructed. Data collection was conducted at the tunnel construction site or inside a completed tunnel using high-resolution digital or industrial cameras. Under safe conditions, images were taken from different longitudinal positions, circumferential angles, and distances from the lining surface. The images should cover a variety of complex real-world conditions, including different lighting conditions (such as only local point light sources, mixed light sources, or insufficient lighting), different shooting angles (such as frontal, oblique, and upward views), and different surface conditions (such as wet concrete reflections, partial occlusion, surface stains, or interference from formwork joints), in order to comprehensively capture the morphological diversity of the steel mesh in the tunnel scene.
[0022] The acquired raw images constitute the initial data source for model learning. These raw images need to undergo fine data annotation to construct the training dataset. The annotation work mainly includes two aspects: 1) Performing pixel-level semantic segmentation annotation on the rebar regions in the images to generate a binary mask with the same size as the original image, namely the "rough rebar region segmentation mask". In this mask, rebar pixels are marked as foreground (value 1), and concrete lining and other background pixels are marked as background (value 0). This mask is used for foreground and background separation in the multimodal enhancement module; 2) Accurately annotating the intersections of the rebar mesh. The annotator needs to mark a pixel-level point at the center of each rebar intersection and generate a "real intersection heatmap label" based on this. A two-dimensional Gaussian kernel can be used to render the marked point position, forming a bright spot on the heatmap with the intersection as the center and the intensity decreasing outward.
[0023] In addition, to support subpixel-level positioning, a "true offset vector" needs to be calculated for each positive sample pixel near the peak of the intersection in the heat map. This vector points to the true subpixel coordinates of its respective intersection.
[0024] The constructed dataset includes the original images, the corresponding coarse rebar region segmentation masks, real intersection heatmaps, and pixel-level real offset vector fields, providing comprehensive supervision signals for the training of subsequent models.
[0025] S2. Construct a multimodal data augmentation module based on a coarse segmentation mask. Images of tunnel secondary lining steel mesh are characterized by complex background textures, extremely small targets at steel bar intersections, irregular changes in steel bar shape due to perspective and lighting, and local occlusion and reflection interference. Conventional data augmentation techniques such as random rotation, cropping, and color jitter can only provide limited appearance changes and cannot simulate the complex lighting conditions inside the tunnel or the non-uniform deformation of the steel mesh caused by the shooting angle, resulting in insufficient robustness of the model to variations in real-world scenarios.
[0026] This invention, based on a rough reinforcement region segmentation mask, separates the generation of foreground reinforcement from the generation of background lining. An enhanced image is synthesized using a conditional image generator function and a background generator function, as shown below: ; In the formula, The augmented image is an image generated through a multimodal augmentation strategy that can simulate the complex lighting, texture and deformation in a real tunnel and is used to augment the training dataset. The original input image is a three-channel RGB image of the tunnel secondary lining steel mesh, with dimensions of [dimension missing]. , serving as a reference starting point for image generation; This represents the image height, which is the number of pixels in the vertical direction of the image. This represents the image width, which is the number of pixels in the horizontal direction of the image; The rough reinforcement region segmentation mask is a binary matrix with the same spatial dimensions as the original image. This is used to initially distinguish the foreground steel reinforcement area from the background lining area in the image, where the value 1 represents the foreground area and the value 0 represents the background area. It can be obtained by semantic segmentation of the original image through a pre-trained lightweight segmentation network (such as U-Net). The conditional image generator function is defined as follows: A deep neural network with a set of trainable parameters is used to generate an enhanced foreground steel reinforcement image block by fusing latent variables of illumination and texture, guided by the original image and segmentation mask. This represents a latent variable related to illumination conditions. A continuous vector of dimension is used to control and simulate the complex and varied lighting effects such as point light sources, diffuse reflection, and shadows within the tunnel. It is preferably sampled from a standard Gaussian distribution or a uniform distribution. Example value: 32; This represents the implicit variable of the background texture. A continuous vector of dimension is used to control the generation of detailed textures such as water stains, cracks, and dirt on the concrete lining surface, increasing the realism and diversity of the background. It is preferably sampled from a standard Gaussian distribution. Example value: 64; This represents the element-wise multiplication operator, also known as the Hadamard product; The foreground mask representing deformation is achieved by segmenting the area of rough steel reinforcement into a mask. The foreground mask obtained by applying perspective transformation is represented as ; in Perspective transformation operations can be implemented using a 3×3 perspective transformation matrix, with parameters... The elements in the perspective transformation matrix are used to simulate the nonlinear deformation caused by different shooting angles, so that the mask can better fit the shape of the steel bars under the real shooting perspective. The background generator function is represented by... A deep neural network with a set of trainable parameters, which is based on texture latent variables Generating realistic images of tunnel lining backgrounds can be achieved, for example, through a generator using a generative adversarial network. Indicates and A matrix of all ones with the same dimension This represents a background area mask, which is a deformable foreground mask. The complement is used to preserve the area output by the background generator when compositing the final image.
[0027] In one implementation, to ensure that the generated foreground rebar region maintains geometric realism while incorporating diverse lighting effects, the conditional image generator function employs a dual-path structure to extract structural features and render lighting features separately, and then fuses them to obtain the enhanced foreground rebar image, represented as follows: ; In the formula, This represents the Sigmoid activation function, used to map linear output values to... An interval is used to represent the normalized pixel intensity, which conforms to the range of image pixel values; Represent a Convolutional layer operations are used to fuse the stitched dual-path features and output the final three-channel image features by adjusting the number of channels; The operator represents the concatenation of features along the channel dimension, connecting two feature tensors to form a new feature tensor; The structural feature tensor represents the geometric structure and basic texture information extracted from the original image, and is obtained by representing it as follows: ; This represents the U-Net encoder section, which consists of multiple convolutional and pooling layers used for downsampling and feature extraction. The lighting rendering feature tensor is obtained by fusing latent lighting variables into structural features through a differentiable rendering module. The acquisition method is represented as follows: ; This represents a differentiable rendering module, calculated as follows: It is used to apply realistic lighting changes to structural features; This represents a channel-level multiplication operator, and the resulting lighting map is broadcast to the structural feature tensor. Each channel is multiplied element-wise to achieve the lighting effect; The representation layer normalization operation is used to stabilize the distribution of features; The weight matrix represents the differentiable rendering module. It is a parameter used in the differentiable rendering module to linearly transform the latent lighting variables and is a trainable parameter. This represents the bias vector of the differentiable rendering module, which is a parameter used in the differentiable rendering module for linearly transforming latent lighting variables, and is a trainable parameter.
[0028] S3. Construct a fine-grained cross-point feature enhancement extraction module. As pixel-level small targets, the rebar intersections have limited distinguishability from the rebar body and background in the original RGB color space. Conventional feature extraction methods are insufficient for the feature discrimination of intersections that are deformed or blurred due to perspective, corrosion, and occlusion in tunnel scenes.
[0029] This invention employs a feature enhancement filter based on multi-scale anisotropic Gaussian convolution kernels to preprocess the enhanced image, generating enhanced feature maps that highlight specific geometric structures at intersections. This amplifies the difference between small targets and the background at the feature level. The specific steps are as follows: 1) Define a multi-scale anisotropic Gaussian convolution kernel group Taking advantage of the directional and specific spatial structure characteristics of rebar intersections, a guideable, anisotropic two-dimensional Gaussian partial derivative convolution kernel is used as a feature enhancement filter, expressed as: ; In the formula, Indicates the use of generating the first The output feature channel, the first A two-dimensional Gaussian partial derivative convolution kernel with a specific orientation angle. and anisotropic scale , As a directional feature detector, it generates a strong response to "edges" perpendicular to that direction and cross-shaped structures by setting a specific orientation and anisotropic scale. This indicates that the convolution kernel is in the horizontal direction, i.e. Local coordinate index along the axis; This indicates that the convolution kernel is in the vertical direction, i.e. Local coordinate index along the axis; Indicates the feature channel index; Indicates the first The principal orientation angle of each filter controls its spatial orientation, enabling it to match the shape of rebar edges or intersections in different directions. Examples of values are shown below. ; Indicates the first Each filter after rotation Direction, i.e., the standard deviation in the principal direction, controls the spatial scale or extension of the filter in that principal direction. For example, a value of 1.0 is used. Indicates the first Each filter after rotation The direction, i.e., the standard deviation in the orthogonal direction, is determined by setting... To achieve anisotropy, the filter can produce different responses to the blocky region at the center of the intersection and the linear region of the steel reinforcement body. For example, a value of 3.0 is used. This represents a mixed second-order partial derivative operator, which makes the kernel function respond most strongly to corner points or cross-shaped structures such as intersection centers, and less strongly to uniform regions or edges in a single direction. Represents the natural exponential function; The horizontal coordinates after rotation are represented as... This is used to make the filter respond to a specific direction; The vertical coordinates after rotation are represented as... Used to make the filter respond to a specific direction, and Together they form the rotated coordinate system; 2) Feature-enhancing convolution operation The enhanced image is subjected to multi-scale convolution operations using multiple two-dimensional Gaussian partial derivative convolution kernels of different scales, and the convolution responses at different scales are weighted and fused to generate an enhanced feature map, represented as follows: ; In the formula, Represents enhanced feature maps It is a three-dimensional tensor with dimensions of . Its spatial dimensions are the same as the input image, and each feature channel enhances the intersection features under a specific combination of orientation and scale. Represents enhanced feature maps In position First The characteristic values of each channel; Indicates the horizontal coordinate index; Indicates the vertical coordinate index; Indicates the feature channel index. Each index corresponds to a set of specific parameters. The filter, Indicates the total number of feature channels. Example value: 16; This indicates the number of scales, i.e., the number of scales used for each feature channel. Convolution is performed using filters of different scales to capture multi-scale information. Example of possible values is shown in 3; Indicates the first Each scale has a weighting coefficient used to balance the importance of features at different scales when fusing multi-scale convolutional responses. This weight can be a fixed value preset for each scale, or it can be a learnable parameter that satisfies... ; This represents the two-dimensional convolution operator.
[0030] In practical implementation, Indicates enhanced image With the Convolutional kernels at various scales In position The convolution response at a given point is obtained by convolving the kernel with each of the three channels of the input image, and then summing the convolution results of the three channels to obtain a single-channel response map. .
[0031] S4. Construct a deformation-sensing intersection recognition network To address the challenges of small targets at steel bar intersections in tunnel scenarios, which are severely affected by perspective distortion and lighting interference, directly applying a general target detector to the enhanced feature map still easily leads to positioning errors and missed detections. This invention employs a deformation-aware intersection recognition network, utilizing the geometric prior information contained in the deformation foreground mask to achieve sub-pixel-level accurate positioning. The specific steps are as follows: 1) Deformation sensing intersection heatmap prediction A deformable foreground mask is used as a geometric guide. Multi-channel features and deformable mask information are fused through weighted convolution to generate a heatmap of intersection points, which adaptively enhances the response in the deformable region. This is represented as: ; In the formula, Indicates the first Horizontal coordinates, number The predicted value of the intersection point in the heat map at the vertical coordinate position is a scalar. A higher value indicates a greater probability that the location is the center of the rebar intersection. The value range is... ; This represents the Sigmoid activation function, used to map a linearly weighted sum to a probability interval. ; Represents the radius of the local convolution kernel, used to define the neighborhood range for calculating the local weighted sum, and is a positive integer, with an example value of 2; Indicates the position relative to the center in the horizontal direction. The offset index, with a value range of 1. arrive ; Indicates the position relative to the center in the vertical direction. The offset index, with a value range of 1. arrive ; Indicates that for the first The two-dimensional convolutional kernel weights for each feature channel are trainable parameters used to extract cross-point feature patterns from the local neighborhood of the enhanced feature map. The weight coefficients representing the contribution of the deformation mask are trainable scalar parameters used to balance the influence of feature map information and deformation mask geometric information on the final heatmap. The deformed foreground mask is represented in the first... Horizontal coordinates, number The value at the vertical coordinate position can be 0 or 1, where 1 represents the foreground steel reinforcement area; The two-dimensional kernel weights represent the convolution of the deformable foreground mask. They are trainable parameters used to extract prior information about geometric deformation from the local neighborhood of the deformable mask.
[0032] 2) Subpixel-level crosspoint offset prediction The predicted intersection locations from heatmaps are limited by a discrete pixel grid, and perspective distortion can cause systematic deviations in local locations. To achieve sub-pixel level positioning accuracy and compensate for the effects of distortion, this invention predicts a two-dimensional offset vector for each pixel location, used to further refine the precise coordinates of the intersections from the peak positions of the heatmap, expressed as: ; In the formula, Indicates the first Horizontal coordinates, number The predicted offset vector at the vertical coordinate position is a two-dimensional vector, represented as... ,in and These represent the horizontal and vertical offsets in pixels, respectively. This represents the hyperbolic tangent activation function, used to restrict the network output to... The interval corresponds to the normalized relative offset value; Indicates the corresponding to the first The weight vector for each feature channel is a two-dimensional weight vector, which is a trainable parameter used to weight the values of each feature channel to synthesize the offset. The weight vector representing the gradient contribution of the deformation mask is a two-dimensional weight vector and a trainable parameter used to adjust the degree of influence of gradient information on the final offset vector. The deformed foreground mask is represented in the first... Horizontal coordinates, number The spatial gradient vector at the vertical coordinate position is a two-dimensional vector, which is approximated by gradient calculation methods such as the Sobel operator.
[0033] S5. Design the total loss function for multi-task joint optimization. To jointly optimize the tasks of heatmap prediction and offset prediction, and to embed geometric consistency constraints in the tunnel scenario, the total loss function is calculated by combining heatmap focusing loss, offset smoothing L1 loss, and deformation consistency loss, and is expressed as: ; In the formula, The total loss function is used to jointly optimize the two tasks of heatmap prediction and offset prediction, and is embedded with geometric consistency constraints in the tunnel scenario. It is the objective function that guides the training and optimization of the entire deformation-sensing intersection recognition network. This represents the weighting coefficient for the heatmap focusing loss, used to adjust the relative importance of the heatmap prediction task in the total loss. An example value is 1.0. This represents the weighting coefficient for offset smoothing L1 loss, used to adjust the relative importance of the sub-pixel offset prediction task in the total loss. An example value is 0.1. The weighting coefficient represents the deformation consistency loss and is used to adjust the strength of the geometric prior constraint in the total loss to prevent overfitting and improve the model's robustness to perspective deformation. An example value is 0.5. The heatmap focusing loss is represented by an improved form, designed to address the extreme pixel imbalance between positive and negative samples at intersections and to enhance attention to difficult-to-classify samples. The calculation method is expressed as follows: ; This represents the offset smoothing L1 loss, calculated only for positive sample pixel positions, used to supervise the accuracy of offset prediction. The calculation method is expressed as follows: ; The deformation consistency loss is represented by the constraint that the gradient of the predicted heatmap should be consistent with the gradient of the deformation mask. Geometric priors are embedded into the learning process, and the calculation method is expressed as follows: ; The heatmap label representing the true intersection point is in the first position. Horizontal coordinates, number The value at the vertical coordinate position is a binary scalar; a value of 1 indicates that the position is the center of the intersection, and a value of 0 indicates the background. Heatmap of network prediction intersections In the Horizontal coordinates, number The value at the vertical coordinate position, i.e. ; This represents the focusing parameter, which is a positive scalar hyperparameter used to dynamically reduce the loss weight of easily classified samples, thereby allowing training to focus more on difficult-to-classify samples. Example values are shown in Figure 2. This represents the total number of pixels, calculated as follows: ; This represents the total number of positive sample pixels in the current image, i.e., satisfying... The number of pixels was obtained through statistics; Indicates an indicator function, when The function value is 1 when the condition is met, and 0 otherwise. Indicates the first Horizontal coordinates, number The true offset vector at the vertical coordinate position is calculated during training based on the true sub-pixel coordinates of that pixel position and the nearest intersection point. That is, for positive sample positions, ,in These are the actual coordinates of the nearest intersection point; This represents the smoothed L1 loss function, used for supervised regression tasks involving offset prediction; Indicates the predicted heatmap In the Horizontal coordinates, number The spatial gradient vector at the vertical coordinate position is approximated by numerical methods such as the finite difference method; This represents the L2 norm.
[0034] In order to train the model proposed in this invention stably and efficiently, a phased progressive training strategy is adopted.
[0035] S6: Execute a phased progressive training strategy. Phase 1 involves pre-training the basic components (offline training). This phase independently trains the multimodal enhancement module. The system employs a conditional image generator and a background generator. Utilizing the original images and their coarse segmentation masks from the constructed dataset, and through adversarial training and other methods, the generator learns to synthesize enhanced images with realistic lighting, texture, and foreground deformation under given masks and random latent variables, without relying on subsequent intersection recognition tasks. This provides high-quality data augmentation capabilities for later stages.
[0036] Phase 2 is for optimizing the feature enhancement filter parameters. In this phase, the pre-trained enhancement module is fixed, and the enhanced image generated by it and the corresponding real cross-point heatmap are used to specifically train the trainable parameters in the multi-scale anisotropic Gaussian convolution kernel group. The goal is to minimize the difference between the simple heatmap directly predicted by the enhanced feature map and the real heatmap, so that the filter group learns to extract the features that best respond to the cross-point structure.
[0037] Phase three involves training the deformation-aware intersection recognition network. In this phase, the parameters of the enhancement modules and feature enhancement filters trained in the first two phases are fixed, and the focus is on training the network parameters of the intersection recognition module. The original image is input into the process, sequentially undergoing enhancement and feature enhancement. Then, a deformation foreground mask is used as a guide to train the network to predict accurate heatmaps and offsets. A loss function is then used for optimization, enabling the network to learn to locate and refine intersection positions in complex backgrounds.
[0038] Phase four involves end-to-end joint fine-tuning. With all components well initialized in previous phases, parameter fixing for all components is removed, and end-to-end joint training is performed with a smaller learning rate. In this phase, data augmentation, feature enhancement, and intersection recognition are synergistically optimized, allowing feature extraction to better adapt to the needs of the recognition task. Simultaneously, the recognition network can fully utilize the enhanced, more feature-rich inputs provided by the front-end modules, further improving overall performance. Phase five focuses on adversarial training to enhance robustness. To improve the model's generalization ability to extreme perturbations and unseen complex situations, adversarial training is introduced after joint fine-tuning. By adding small adversarial noise to the input image and simultaneously optimizing model parameters and adversarial examples during training, the model is forced to maintain stable recognition performance under more challenging inputs, significantly improving its robustness and reliability in the complex and variable environment of real tunnels.
[0039] S7. Implement online identification of intersections of tunnel secondary lining steel mesh. Once the model has been trained in stages, it can be used to automatically identify intersections in new, unprocessed images of tunnel secondary lining steel mesh.
[0040] The recognition process first loads the pre-trained full set of model parameters, including a multimodal enhancement module, a feature enhancement filter bank, and a deformation-aware intersection recognition network. For an input image to be recognized, the system first uses a pre-trained lightweight segmentation network to quickly infer its "rough rebar region segmentation mask"; Then, the original image and the segmentation mask are input into the trained multimodal enhancement module, and combined with randomly sampled lighting and texture latent variables, an "enhanced image" that can simulate a complex tunnel environment and enhance the shape of the foreground steel bars is generated. Then, the enhanced image is convolved using an optimized multi-scale anisotropic Gaussian convolution kernel group to generate an "enhanced feature map" that highlights the geometry of the intersection point; Simultaneously, a random perspective transformation is applied to the coarse segmentation mask to generate a "deformed foreground mask". This mask is used for guidance during the training phase and serves as a geometric prior input during the inference phase. Then, the enhanced feature map and the deformation foreground mask are fed into the trained deformation-aware intersection recognition network. The network propagates forward and outputs two key results: first, an "intersection heatmap", in which the location of local response peaks initially indicates the area where intersections exist; and second, a "sub-pixel offset vector" predicted for each pixel position.
[0041] The post-processing steps of the identification process first involve finding peaks in the predicted heatmap and identifying all local maximum points. The coordinates of these points are the pixel-level positions of the initially detected intersections. Then, for each detected peak position, the offset vector predicted by the network at that position is queried, and the offset is added to the pixel-level coordinates to obtain the refined "sub-pixel-level intersection coordinates", thus achieving high-precision positioning that surpasses the limitations of pixel grid. Finally, the system outputs the precise coordinates of all identified rebar intersections in the original input image, completing the automated identification task of tunnel secondary lining rebar mesh intersections.
[0042] Finally, it should be noted that the above embodiments are merely specific implementations of the present invention, used to illustrate the technical solutions of the present invention, and not to limit it. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed in the present invention, or make equivalent substitutions for some of the technical features; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.
Claims
1. A method for identifying intersections of steel reinforcement mesh in tunnel secondary lining based on machine vision, characterized in that, Includes the following steps: S1. Collect raw image data of the tunnel secondary lining steel mesh, and annotate the raw image data to construct a training dataset; S2. Construct a multimodal data augmentation module based on a coarse segmentation mask: separate the foreground steel reinforcement from the background lining generation, and synthesize the augmented image through a conditional image generator function and a background generator function; S3. Construct a fine-grained intersection feature enhancement extraction module: preprocess the enhanced image to generate an enhanced feature map that can highlight the specific geometric structure of the intersection. S4. Construct a deformation-aware intersection recognition network: Utilize the deformation foreground mask as an implicit geometric prior constraint, guide heat map prediction through weighted fusion, and jointly predict the sub-pixel level offset vector field to achieve sub-pixel level accurate positioning. S5. Design the total loss function for multi-task joint optimization: To jointly optimize the tasks of heatmap prediction and offset prediction, a geometric consistency constraint in the tunnel scenario is introduced. The heatmap focusing loss, offset smoothing L1 loss and deformation consistency loss are weighted and fused to construct the total loss function to drive the iterative update of network parameters. S6. Implement a phased progressive training strategy: decompose the complex system, which includes generation, filtering, and recognition, into five ordered stages: pre-trained generator, optimized filter, trained recognition network, end-to-end fine-tuning, and adversarial training. S7. Implement online identification of intersections of the secondary lining steel mesh in tunnels: Load the trained model parameters and output the set of precise coordinates of all identified steel mesh intersections in the original input image to complete the automated identification task of intersections of the secondary lining steel mesh in tunnels. Specifically, S3 is: Define a multi-scale anisotropic Gaussian convolution kernel group: taking into account the directional and specific spatial structure characteristics of steel bar intersections, a guideable and anisotropic two-dimensional Gaussian partial derivative convolution kernel is used as a feature enhancement filter. Feature enhancement convolution operation: Multi-scale convolution operation is performed on the enhanced image using multiple two-dimensional Gaussian partial derivative convolution kernels of different scales, and the convolution responses of different scales are weighted and fused to generate enhanced feature maps. S4 specifically refers to: Deformation-aware intersection heatmap prediction: A deformation foreground mask is used as a geometric guide. Multi-channel features and deformation mask information are fused through weighted convolution to generate an intersection heatmap, which adaptively enhances the response in the deformation region. Subpixel-level crosspoint offset prediction: Predicts a two-dimensional offset vector for each pixel location, used to further refine the precise coordinates of crosspoints from the peak locations of the heatmap.
2. The method for identifying intersections of tunnel secondary lining steel mesh based on machine vision according to claim 1, characterized in that, The conditional image generator function uses a dual-path structure to extract structural features and rendering lighting features separately, and then fuses them to obtain an enhanced foreground rebar image.
3. The method for identifying intersections of tunnel secondary lining steel mesh based on machine vision according to claim 1, characterized in that, The total loss function in S5 is specifically expressed as follows: In the formula, L total Let λ1 represent the weighting coefficient of the heatmap focusing loss, λ2 represent the weighting coefficient of the offset smoothing L1 loss, and λ3 represent the weighting coefficient of the deformation consistency loss. heat Indicates the focus loss in the heatmap; L off This indicates the offset smoothing L1 loss; L def This indicates the loss of deformation consistency.
4. A method for identifying intersections of tunnel secondary lining steel mesh based on machine vision, as described in claim 1 or 3, characterized in that, S5 also includes an improved form of focus loss to address the problem of extreme pixel imbalance between positive and negative samples at intersections and to enhance attention to samples that are difficult to classify.
5. The method for identifying intersections of tunnel secondary lining steel mesh based on machine vision according to claim 1, characterized in that, S1 also includes data annotation, specifically: Pixel-level semantic segmentation and annotation are performed on the steel reinforcement area in the image to generate a binary mask with the same size as the original image. The steel reinforcement pixels are marked as foreground, and the concrete lining and other background pixels are marked as background. This mask is used for foreground and background separation in the multimodal enhancement module. The intersections of the steel mesh are precisely marked, with a pixel-level point marked at the center of each steel mesh intersection, and a heat map label of the actual intersection is generated based on this.
Citation Information
Patent Citations
Directed target detection method fusing multi-scale context and task consistency focus loss
CN120689765A
Tunnel disease identification model training method and system based on point cloud and image
CN120833543A
