A method for reinforcing bar corrosion thermogram inversion and three-dimensional quantification based on weakly supervised state space model

Abstract

The application discloses a method for reinforcing bar corrosion thermodynamic map inversion and three-dimensional quantification based on a weakly supervised state space model, constructs a parallel high-resolution feature extraction network based on a Mamba mechanism, proposes an inversion positioning method based on weak supervision (image-level label), and establishes a physical quantification process of "thermodynamic map inversion-boundary refining-three-dimensional mapping". The application improves the limit capturing capacity of micro-point corrosion and the anti-interference capacity in a complex environment, realizes physical-level fine fitting of a corrosion boundary, and is more accurate in reinforcing bar corrosion identification.

Description

Technical Field

[0001] This invention relates to the field of civil engineering structural health monitoring and computer vision technology, specifically to a method for inverting and quantifying the thermal map of steel corrosion based on a weakly supervised state space model. Background Technology

[0002] Steel corrosion is a major cause of decreased durability and reduced load-bearing capacity in reinforced concrete (RC) structures. Timely and accurate detection of the condition, extent, and severity of steel corrosion is a crucial prerequisite for structural safety assessments and repair / reinforcement decisions. Currently, existing steel corrosion detection technologies mainly suffer from the following limitations:

[0003] 1. Traditional manual inspection method:

[0004] Method: Relying on the experience of the inspectors, the inspection was conducted through visual inspection, tapping, caliper measurement, and other means.

[0005] Limitations: This method is highly subjective, and the results cannot be standardized; the detection efficiency is low; there are safety risks for detection at high or complex locations; and it can only provide a rough qualitative description, not precise quantitative analysis.

[0006] 2. Electrochemical or physical non-destructive testing methods:

[0007] Methods: such as half-cell potential method, ground-penetrating radar (GPR), ultrasonic method, etc.

[0008] Disadvantages: While these methods achieve non-destructive testing, they typically involve expensive equipment, complex operation, and slow detection speed. More importantly, they mostly provide indirect electrochemical or physical parameters, failing to offer intuitive visual information such as corrosion morphology, spatial distribution, and surface area.

[0009] 3. Detection methods based on two-dimensional images:

[0010] Defect a (Traditional Image Processing): Early methods relied on the HSV color space of rust, texture, edge detection, etc., to set thresholds. Such methods have extremely poor robustness and are easily affected by changes in lighting, shadows, water stains, concrete stains, and other environmental factors, making them almost impractical.

[0011] Defect b (based on 2D deep learning): Techniques have begun to employ deep learning models such as U-Net and HRNet for rust segmentation. However, most of these methods belong to "fully supervised learning," which presents significant challenges for practical application.

[0012] (1) The labeling cost is extremely high: The fully supervised model requires that every tiny rust pit in the massive number of images must be manually labeled pixel by pixel before training. This is a huge and time-consuming project.

[0013] (2) Difficulty in defining the boundary: The corrosion of steel bars is often diffuse and the edges are blurred. It is difficult for people to accurately define the boundary between "rust" and "non-rust", resulting in a large number of subjective errors in the training data.

[0014] Defect c (Perception limitations of traditional network architectures): Existing convolutional neural networks (CNNs) are limited by their local receptive field, making it difficult to capture long-distance texture dependencies (such as longitudinal cracks) that run through the entire steel bar, and they are prone to misidentifying local water stains as rust. While the Transformer architecture has global perception, its computational cost is too high, making it difficult to run in real time on handheld devices.

[0015] Defect d (Inaccurate 2D quantization): Existing image methods calculate pixel area. Since steel bars are cylindrical, their surface curvature in 2D projection causes severe area distortion. Even slight changes in the shooting angle can lead to huge differences in pixel area, resulting in a result that does not reflect the true physical surface area.

[0016] Therefore, there is an urgent need in this field for a novel technical solution that must simultaneously meet all of the following conditions:

[0017] (1) Weak supervision (low cost): Only image-level labels are needed for training, without the need for manual pixel-by-pixel annotation.

[0018] (2) Global perception (anti-interference): It can effectively distinguish between rust and water stains / oil stains by combining long-distance contextual textures.

[0019] (3) Three-dimensional quantization (high precision): It can combine three-dimensional models to calculate the real physical surface area and eliminate projection errors. Summary of the Invention

[0020] To address the aforementioned issues, this invention provides a method for inverting and quantifying the thermogram of steel corrosion based on a weakly supervised state-space model.

[0021] 1. A method for inverting and quantifying the thermal map of steel reinforcement corrosion based on a weakly supervised state-space model, characterized by the following steps:

[0022] Step 1: Construct a dataset of steel rebar images containing various lighting conditions and different degrees of corrosion as the training set:

[0023] Step 2: Construct the WSP-MambaNet network, which includes an AGDP attention pyramid, a Mamba-HRNet parallel backbone, and a MIL-FPN weakly supervised inversion head. The training set is input into the AGDP attention pyramid to obtain a multi-scale spatially weighted feature map that suppresses background noise. This multi-scale spatially weighted feature map is then input into the Mamba-HRNet parallel backbone to obtain a high-resolution parallel feature stream containing global long-range texture dependencies. The high-resolution parallel feature stream is then input into the MIL-FPN weakly supervised inversion head to obtain a rust category activation map and image-level classification prediction probabilities. The loss function of the WSP-MambaNet network is a dynamically adjusted Focal Loss loss function. Training continues until the Focal Loss loss function converges to obtain the trained WSP-MambaNet network.

[0024] Step 3: Input the image of the steel bar to be detected into the trained WSP-MambaNet network to obtain the results of corrosion category activation map inversion and three-dimensional quantization: the results include corrosion degree, corrosion area, corrosion region and processing suggestions.

[0025] 2. The method for inverting and quantizing the heat map of steel corrosion based on a weakly supervised state-space model as described in claim 1, characterized in that the AGDP attention pyramid comprises four levels, each level consisting of a residual module embedded with adaptive channel attention, used to retain key texture features during downsampling. The specific data processing flow is as follows:

[0026] S2.1) Residual Feature Extraction and Downsampling: Input Features First, basic features are extracted through a residual path consisting of two 3×3 convolutional layers. ;

[0027]

[0028] Where BN represents batch normalization, This represents a 3×3 convolution, and ReLU represents the rectified linear activation function. To achieve the pyramid downsampling function, the stride of the 3×3 convolution is set to 2, thereby halving the feature map resolution.

[0029] S2.2) Channel Self-Attention Modeling: In order to capture long-distance dependencies across channels, i.e., the correlation between different rust texture modes in channels, the following model is used: Introduce a channel self-attention mechanism.

[0030] Feature transformation: The channel dimension is compressed to 1×1 through 1×1 convolution. Remodeling after BN and ReLU ,in X represents the feature matrix after dimensionality reduction and reshaping. B represents the number of channels after compression, B represents the batch size, and H represents the input feature map. The height, W represents the input feature map. The width;

[0031]

[0032] Where Reshape represents feature reshaping, BN represents batch normalization, ReLU represents rectified linear activation function, and Conv... 1×1 Represents a 1×1 convolution;

[0033] Channel correlation calculation: The covariance matrix between channels is calculated using matrix multiplication, and then normalized using Softmax to obtain the channel attention map M.

[0034]

[0035] Wherein, the element in the j-th row and i-th column of the attention graph M is This measures the influence of the i-th channel on the j-th channel; Softmax is the activation function; T represents the matrix transpose.

[0036] Weighting and Aggregation: The attention map M is multiplied by the original feature X for weighted aggregation, then reshaped, 1×1 convolution, BN, and ReLU to restore the dimensions, and then compared with the input. By summing the residuals, we obtain the attention-enhanced features. :

[0037]

[0038] Wherein, Transform represents feature transformation and dimension restoration;

[0039] Feature fusion and output: Features enhanced by attention The original input after downsampling and matching dimensions Element-wise summation is performed to obtain the final output multi-scale spatially weighted feature map of the current level. :

[0040]

[0041] This will be used as the input for the next level of the pyramid, with DownSample representing the downsampling.

[0042] 3. The method for inverting and quantizing the heat map of steel corrosion based on a weakly supervised state space model as described in claim 1, characterized in that the Mamba-HRNet parallel backbone is obtained by replacing all the basic residual convolutional blocks of the HRNet network with the visual state space module (VSS Block).

[0043] The data processing flow for the visual state space module is as follows:

[0044] S3.1) Unfold the multi-scale spatially weighted feature map output by the AGDP attention pyramid or the feature map passed inside the parallel backbone of Mamba-HRNet into a one-dimensional sequence along the four directions of upper left, lower right, upper right and lower left, denoted as x;

[0045] S3.2) Full image scanning: The one-dimensional input sequence x is recursively scanned using the discretized state equation, so that each pixel can perceive the texture context of the entire steel bar, and the processed sequence is obtained.

[0046] S3.3) Reconstruction: The processed sequences are merged and restored to 2D feature maps as high-resolution parallel feature streams.

[0047] 4. The method for inverting and quantizing the thermogram of steel corrosion based on a weakly supervised state-space model as described in claim 3, characterized in that, in order to convert the continuous physical signal model into a computer-processable digital signal, a time-scale parameter is used. continuous state parameters Convert to discretized parameters And construct the discretized state equations described in step S3.2:

[0048]

[0049]

[0050]

[0051]

[0052] in, This represents the discretized state transition matrix. Representing time scale parameters The product with the continuous state matrix A, This represents the discretized input projection matrix. Represents the identity matrix. Representing time scale parameters The product of the continuous input matrix B, This represents the hidden state vector at the current time t. Let C represent the input feature vector at the current time t, C represent the output projection matrix, and D represent the direct feedforward matrix.

[0053] 5. The method for inverting and quantifying the thermogram of steel corrosion based on a weakly supervised state-space model as described in claim 1, characterized in that the data processing flow of the MIL-FPN weakly supervised inversion head is as follows:

[0054] The high-resolution parallel feature stream of the Mamba-HRNet parallel backbone, namely the deep semantic features output at 1 / 32 resolution from the Visual State Space Block (VSS Block) in the last layer of Stage 4, is denoted as... The second shallow texture feature at 1 / 8 resolution output by Stage 3 is denoted as... And the first shallow texture feature at 1 / 4 resolution output by Stage 2, denoted as In the future Upsampling step by step and with , Before performing horizontal fusion, an efficient spatial channel attention module (ESCA) is introduced:

[0055] The ESCA data processing flow is as follows: For features... Spatial augmentation is performed to obtain intermediate features. ;

[0056]

[0057] in, This represents the Sigmoid activation function. Indicates average pooling. This indicates max pooling. Represents convolution;

[0058] exist Based on this, channel attention is introduced to obtain attention-enhanced features. ;

[0059]

[0060] in, Indicates global average pooling. This represents one-dimensional convolution;

[0061] The enhanced features are added to the upsampled deep features to obtain the fused features. ;

[0062]

[0063] in, Indicates upsampling;

[0064] Then perform a multi-example inversion:

[0065] Heatmap generation: fusing features The class activation map is mapped using a 1×1 convolution. That is, the original heatmap;

[0066] Global prediction: To predict the probability of an image belonging to a specific type of corrosion, global max pooling is performed. The steps of global max pooling are as follows: For each category channel, the maximum value of all pixels in that channel is selected as the predicted probability for that category, i.e.

[0067] Where i and j represent the row and column indices of the pixel's spatial coordinates; Let represent the pixel value of the c-th category channel in the rust category activation map at coordinates (i, j), and max(i, j) represent the pixel value of the c-th category channel in the rust category activation map at coordinates (i, j). cam Take the maximum value at all spatial locations;

[0068] Boundary Refinement: A fully connected conditional random field, DenseCRF, is introduced. Using the color and texture boundaries of the original RGB image as constraints, the heatmap is spatially regularized to ensure its edges closely conform to the realistic physical contours of steel reinforcement corrosion.

[0069]

[0070] in M represents the original RGB image. refined This represents the refined probability distribution diagram, and CRF represents the optimization of a fully connected conditional random field.

[0071] Then, mask generation is performed on the refined heatmap. Pixel-by-pixel classification is performed to generate pseudo-segmentation masks containing multi-level semantic information: 0-no corrosion, 1-slight corrosion, 2-moderate corrosion, 3-severe corrosion. These masks are directly used for subsequent 3D hierarchical quantization. For each pixel coordinate (x, y) in the image, its probability values ​​in all category channels are compared, and the category with the highest probability is selected as the final label of the pixel.

[0072]

[0073] Where x and y represent the horizontal and vertical coordinates of a pixel. This represents the probability value of the c-th category in the refining heatmap; Represents pixels The final classification labels are: 0 - No rust, 1 - Slight, 2 - Moderate, 3 - Severe.

[0074] 6. The method for inverting and quantizing the heat map of steel corrosion based on a weakly supervised state-space model as described in claim 1, characterized in that the WSP-MambaNet network adopts the Focal Loss loss function to solve the sample imbalance problem, and the loss is as follows:

[0075]

[0076] Where Loss represents Focal Loss; C represents the number of corrosion categories; The image-level true label is 0 or 1, indicating whether the image contains type c corrosion; This represents the image-level probability predicted by the network; that is... , Set it to 2. Indicates in Take the maximum value at all spatial locations (x, y). This represents the pixel value of the c-th category channel in the rust category activation map at coordinates (x, y).

[0077] 7. The method for inverting and quantizing the thermal map of steel corrosion based on a weakly supervised state-space model as described in claim 1, characterized in that the thermal map inversion and 3D quantization steps in step three are as follows:

[0078] S7.1) Set a confidence threshold, truncate the heatmap, and generate a binary pseudo-segmentation mask;

[0079] S7.2): Real-time calculated camera pose and pseudo-segmentation mask;

[0080] S7.3) Back projection: Based on the depth stream and IMU data input from the depth camera, a 3D point cloud map is obtained through Simultaneous Localization and Mapping (SLAM) thread. Traversing the 3D point cloud map For each point P in the array, project it back to the current 2D frame and query its state on the "pseudo-segmentation mask";

[0081] S7.4) Combine the observation results of point P from multiple frames and use the majority voting method to determine the final three-dimensional semantic state of point P;

[0082] S7.5) Generate a semantic point cloud model with rust color markers. And output;

[0083] S7.6) Meshization: For Greedy projection triangulation is performed to generate a continuous 3D mesh. Each triangular facet on the mesh is traversed. If the semantic label of the three vertices of a certain triangular facet is "rust", its physical area is calculated using Heron's formula.

[0084] S7.7) Output Results: Displayed on the screen: Hierarchical quantization calculation: Traverse the 3D mesh and perform face accumulation based on the semantic labels of the mesh vertices; Output Results: Display a structured health diagnosis report on the screen.

[0085] No corrosion: 0.0 cm 2 No processing required;

[0086] Slight rust, i.e., yellow surface rust: XX.X cm² — Recommended treatment: Surface rust removal;

[0087] Moderate rust, i.e., brown layer or slight peeling: YY.Y cm² — Recommended treatment: sandblasting, grinding for deep rust removal;

[0088] Severe corrosion, i.e., pitting or flaking: ZZ.Z cm² — Recommended treatment: structural reinforcement or replacement;

[0089] Overall corrosion rate: WW %

[0090] Transformed into a visual model: an interactive, rotateable 3D color steel bar model;

[0091] The semantic tags include: 0 - no corrosion, 1 - mild, 2 - moderate, and 3 - severe.

[0092] Advantages of this invention:

[0093] 1. A parallel high-resolution feature extraction network (WSP-MambaNet) based on the Mamba mechanism was constructed. Results: This solved the problem of insufficient receptive field in traditional CNNs, which prevented the differentiation between complex backgrounds and rust, and improved the anti-interference capability.

[0094] 2. A weakly supervised (image-level labeling) inversion localization method is proposed. Results: This avoids the high cost of manual pixel-by-pixel annotation required for fully supervised learning, making the model easier to deploy in engineering.

[0095] 3. A physical quantification process of "heatmap inversion - boundary refinement - 3D mapping" was established. Results: It eliminated area calculation errors caused by shooting angle and object curvature, achieving truly non-destructive quantitative detection. Attached Figure Description

[0096] Figure 1 This is an overall flowchart of the method of the present invention.

[0097] Figure 2This is a diagram of the WSP-MambaNet model structure. In the diagram, AGDP is the attention pyramid, with the section labeled AGDP in the header; Mamba-HRNet is the parallel backbone, covering the regions from STAGE 1 to STAGE 3; and MIL-FPN is the weakly supervised inversion head, represented by STAGE 4 and its output on the far right of the diagram.

[0098] The network consists of the following components: Input Image (original RGB image of the steel reinforcement surface); STEM (STEM Sub-network): stem sub-network, the basic feature extraction layer used for initial convolution processing of the input image; AGDP: attention-guided downsampling pyramid; AD: attention downsampling module, used to generate spatial weights and filter background noise; STAGE 1 - STAGE 4: four stages of network processing, demonstrating the multi-scale parallel feature extraction process; VSS (VSSBlock): visual state space module, the core feature extraction unit based on the Mamba mechanism, used to capture long-distance texture dependencies; ESCA: efficient spatial channel attention module, located in the inversion head, used to enhance the rust texture response during feature fusion; CAM: class activation map, i.e., the rust heatmap finally generated by the network inversion; Down Samplings: downsampling operations (downward arrows in the diagram); Up Sampling: upsampling operations (upward arrows in the diagram); Element-wise Addition: element-wise addition; Feature maps from the Multi-Resolution Branches: feature maps from the multi-resolution branches.

[0099] Figure 3 This is a structural diagram of the AGDP attention pyramid. In the diagram, Conv3×3 refers to 3×3 convolution; Norm & ReLU refers to normalized and linearly rectified activation layers, used to accelerate convergence and introduce non-linear features; Channels Self-Attention refers to adaptive channel attention; Element-wise Addition refers to element-wise addition; below... Figure 4 This is a diagram illustrating adaptive channel attention.

[0100] Figure 4This is a diagram of the Channel Self-Attention architecture. In the diagram, Conv1×1 refers to 1×1 convolution; BN & ReLU refer to batch normalization and rectified linear activation functions; Softmax is the activation function; ReShape restores the feature map to the initial dimension; MatMul represents matrix multiplication; Channel Correlation refers to channel correlation calculation; WeightedAggregation refers to weighted aggregation, applying attention weights to the original features; Element-wise Addition refers to element-wise addition.

[0101] Figure 5 This is a diagram of the ESCA efficient spatial channel attention module. The ESCA module consists of two concatenated parts: the Spatial Attention Module and the Channel Attention Module. Max Pool and Avg Pool refer to max pooling and average pooling, respectively; Concat refers to the channel concatenation operation, merging the results of max pooling and average pooling; Conv1×1 refers to a 1×1 convolution; Sigmoid refers to the activation function; Global AvgPool refers to global average pooling; Conv1d k×k refers to a one-dimensional convolutional layer (kernel size k) used to capture local dependencies across channels; and Element-wise Multiplication refers to element-wise multiplication. Detailed Implementation

[0102] The technical solution of the present invention will be specifically described below through specific embodiments and in conjunction with the accompanying drawings.

[0103] Example 1

[0104] A method for inverting and 3D quantizing the thermogram of steel corrosion based on a weakly supervised state-space model:

[0105] 2.1 Data Preparation: Constructing a Weakly Supervised Image-Level Database

[0106] (1) Data acquisition: Use a handheld acquisition device (RealSense D435i) to acquire images of the steel reinforcement surface and construct a dataset containing different lighting and background interference.

[0107] 1. Data Acquisition: Construct a dataset of steel rebar images containing various lighting conditions and different degrees of corrosion.

[0108] 2. Weakly Supervised Annotation: Annotators do not need to perform contour outlining; they only need to assign an image-level category label (Y) to the entire image. Label Definition: The values ​​correspond to 0: uncorroded steel bars; 1: light corrosion (yellow surface rust, mainly located on the surface, without obvious pits or volume loss); 2: moderate corrosion (brown rust layer, with slight peeling, accompanied by measurable roughness changes); 3: severe corrosion (deep black rust, with obvious pits, cross-sectional loss or signs of rust expansion cracks).

[0109] 3. Data processing: Normalize the image size and perform data enhancement operations such as random flipping and illumination jitter.

[0110] 2.2 Core Algorithm

[0111] The WSP-MambaNet designed in this invention contains three core cascaded subsystems: (1) AGDP attention pyramid; (2) Mamba-HRNet parallel backbone; and (3) MIL-FPN weakly supervised inversion head.

[0112] 2.2.1 First Level: Pre-feature Pyramid – Attention-Guided Downsampling Pyramid (AGDP)

[0113] This module contains four levels. The first level inherits the features output by the Stem layer, and each subsequent level generates feature maps with progressively decreasing resolution (1 / 8, 1 / 16, 1 / 32) through stride convolution.

[0114] Internal processing logic: Before passing down each level of the pyramid, embed spatial attention units.

[0115] 1. Residual Feature Extraction and Downsampling: For input features... First, the data passes through a residual path consisting of two 3×3 convolutional layers. To achieve the pyramid downsampling function, the convolution stride is set to 2, resulting in basic features with half the resolution. :

[0116]

[0117] 2. Channel self-attention modeling: In order to capture the correlation between different rust texture patterns in channels, a self-attention mechanism is introduced.

[0118] Correlation calculation: Features Remodeling The covariance between channels is calculated using matrix multiplication, noting that one of the matrices needs to be transposed. The channel attention M is then obtained via Softmax.

[0119]

[0120] 3. Feature Fusion and Output: To address the dimensionality mismatch issue caused by downsampling, the original input... After downsampling (e.g., 1×1 convolution with a stride of 2), the features are added to the attention-enhanced features to obtain the final output.

[0121]

[0122] 2.2.2 Second Level: Core Backbone – Mamba-Enhanced Parallel State Space Network (Parallel VSS-Backbone)

[0123] This invention retains the classic four-way multi-scale parallel architecture of HRNet to maintain 1 / 4 of the high-resolution feature flow. The core innovation lies in replacing all the basic residual convolutional blocks in the original architecture with visual state space modules (VSS blocks).

[0124] • How VSS Block works:

[0125] Traditional CNNs can only extract local features (3×3 window). VSS Block introduces a 2D selective scanning (SS2D) mechanism:

[0126] 1. Serialization: The 2D feature map is unfolded into a one-dimensional sequence along four directions (top left, bottom right, top right, bottom left).

[0127] 2. Full-image scanning: The sequence is recursively scanned using discretized state equations, so that each pixel can perceive the texture context of the entire steel bar.

[0128] 3. Reconstruction: Merge the processed sequences and reconstruct them into 2D feature maps.

[0129] To losslessly process continuous physical crack signals in digital systems, the zero-order hold (ZOH) principle is used for discretization:

[0130]

[0131]

[0132]

[0133]

[0134] 2.2.3 Third Stage: Tail Output – FPN Reconstruction and Weakly Supervised Inversion Head (MIL Head)

[0135] 1. FPN Feature Reconstruction and ESCA Fusion: An inverted Feature Pyramid Network (FPN) is constructed. The deep semantic features (1 / 32) output by the backbone network are upsampled step by step. When laterally fusing with the shallow texture features (1 / 4, 1 / 8), to avoid feature aliasing caused by simple element-wise addition, this invention introduces the efficient Spatial Channel Attention (ESCA) module.

[0136] ESCA working principle: This module performs weighted processing on shallow features before fusion, automatically suppressing the channel response of background noise and enhancing the spatial response of rust texture, ensuring the quality of the fused feature map. It combines high-resolution detail with strong semantic information.

[0137] 2. Multi-Instance Inversion (MIL Inversion):

[0138] Heatmap generation: The 1×1 convolution maps to a Class Activation Map (CAM), which is the original heatmap.

[0139] Global prediction: Perform global max pooling (GMP) on the CAM to predict the probability of which type of corrosion the image belongs to.

[0140] Training Strategy: During the training phase, Focal Loss is used to constrain the network. The model performs backpropagation based solely on the image-level label Y, forcing the network to automatically find the most discriminative rust texture features without any pixel-level annotation.

[0141]

[0142] Inference and Refinement: During inference, the CAM is directly extracted as the initial heatmap. Since the edges of the original CAM are relatively blurry, DenseCRF (fully connected conditional random field) is introduced. The color and texture boundaries of the original RGB image are used as constraints to perform spatial regularization on the heatmap, so that its edges closely follow the real physical contours of the steel rust.

[0143]

[0144] Mask generation: The refined heatmap is classified pixel by pixel to generate a pseudo segmentation mask containing multi-level semantic information (0-no corrosion, 1-light corrosion, 2-moderate corrosion, 3-severe corrosion), which is directly used for subsequent three-dimensional hierarchical quantization.

[0145]

[0146] 2.3 System Hardware Assembly

[0147] Assemble a handheld three-dimensional quantifier for steel bar corrosion.

[0148] 1. Edge computing unit: NVIDIA Jetson AGX Xavier (deploys the WSP-MambaNet algorithm mentioned above).

[0149] 2. Sensing Unit: Intel RealSense D435i (provides RGB color stream + depth stream + IMU data).

[0150] 3. Interaction unit: 5-inch touchscreen.

[0151] 4. Deployment: Convert the trained PyTorch model into an inference engine using TensorRT and deploy it to the Jetson processor.

[0152] 2.4 On-site data collection

[0153] The handheld quantizer is activated, and the operator moves around the target rebar for scanning. The sensor outputs three data streams in real time: (1) RGB image (transmitted to the AI ​​inference thread for texture analysis and rust area identification); (2) Depth map; (3) IMU data (transmitted to the SLAM (Simultaneous Localization and Mapping) thread. Among them, the Depth data is used to convert 2D pixel coordinates into 3D spatial coordinates (i.e., restore the physical scale), and the IMU data is used to assist in calculating the real-time pose (position and attitude) of the camera during the movement process to prevent misalignment of point cloud stitching).

[0154] 2.5 Real-time Inference: Heatmap Inversion and Pseudo-mask Generation

[0155] The processor receives the RGB data stream and starts the WSP-MambaNet inference engine. At this point, the model no longer outputs classification probabilities, but instead performs an "inversion" operation:

[0156] 1. Feature extraction: The image is subjected to AGDP noise reduction and Mamba backbone extraction to generate high-resolution features containing global context.

[0157] 2. Heatmap Generation: Extract the category activation map from the network ends to create a corrosion heatmap. Darker red in the image indicates a higher probability of corrosion.

[0158] 3. Boundary Refinement: To meet the accuracy requirements of physical area quantization, this invention introduces an RGB-guided boundary correction algorithm (such as CRF or guided filtering). Utilizing the significant gradient differences in color and texture between the rusted area and the background in the original image, a pixel affinity matrix is ​​constructed, forcing the blurred heatmap edges to converge towards the true pixel gradient edges (Snap-to-Edge).

[0159] 4. Adaptive segmentation: Set a confidence threshold (e.g., ...) The heatmap is truncated to generate a binary pseudo-segmentation mask.

[0160] 2.6 Fusion of 3D Semantic Models

[0161] Start the semantic mapping module to map the 2D algorithm results to 3D space.

[0162] 1. Input: Camera pose calculated in real time by the SLAM thread using depth and IMU data. and rendezvous point cloud map And the pseudo segmentation mask generated in step 2.5.

[0163] 2. Back projection: Traversing the 3D point cloud map For each point P in the array, project it back to the current 2D frame and query its state (corroded / not corroded) on the "pseudo-segmentation mask".

[0164] 3. Bayesian Update / Voting: Based on the observation results of the point in multiple frames (e.g., the algorithm judges the point as rust in 8 out of 10 frames), the final three-dimensional semantic state of the point is determined by majority voting.

[0165] 4. Output: Generate a semantic point cloud model with rust color markers. .

[0166] 2.7 Quantitative Calculation and Report Output

[0167] Start the quantization report module and perform physical integration.

[0168] 1. Grid-based: For Greedy projection triangulation is performed to generate a continuous 3D mesh.

[0169] 2. Physical Accumulation: Traverse every triangular facet on the mesh. If the semantic label of the facet is "severely corroded," calculate its physical area (unit: cm²) using Heron's formula. 2 ).

[0170] 3. Result Output: Displayed on the screen:

[0171] (1) Hierarchical Quantization Calculation: Traverse the 3D mesh and perform surface accumulation based on the semantic labels of the mesh vertices (0-no corrosion, 1-mild, 2-moderate, 3-severe). Output: Display a structured health diagnosis report on the screen.

[0172] 1. No rust: 0.0cm 2 No processing required.

[0173] 2. Slight rust (yellow surface rust): XX.X cm² — Recommended treatment: Surface rust removal.

[0174] 3. Moderate rust (brown layer / slight peeling): YY.Y cm² — Recommended treatment: sandblasting, grinding for deep rust removal.

[0175] 4. Severe corrosion (pitting / scratching): ZZ.Z cm² — Recommended treatment: structural reinforcement or replacement.

[0176] 5. Overall corrosion rate: WW%.

[0177] (2) Visual model: A 3D colored steel bar model that can be interactively rotated and viewed.

[0178] To fully verify the overall performance of the algorithm proposed in this application in the task of detecting steel corrosion, we selected seven mainstream semantic segmentation models for rigorous horizontal comparison under the same hardware platform and test set.

[0179] Model Recall Prescion F-Score mIoU FPS FLOPS UNet 79.4 80.1 79.7 76.8 32.5 98.4 DeepLabV3+ 87.5 88.1 87.8 86.2 11.8 204.3 BiSeNet V2 75.4 74.2 74.8 73.6 62.4 28.5 SegFormer-B2 88.2 89.0 88.6 87.5 18.2 72.3 Swin-UperNet 88.9 89.5 89.2 88.1 8.5 164.5 PIDNet-L 84.5 85.2 84.8 83.4 45.6 98.2 SegNeXt 89.1 89.8 89.4 88.5 14.3 68.9 Ours 90.5 91.2 90.8 89.4 24.6 158.4

[0180] Existing technologies typically employ the U-Net architecture. As shown in the table above, while U-Net boasts a fast inference speed (32.5 FPS), its limited local perception capabilities due to its convolutional structure prevent it from effectively capturing large-scale texture dependencies, resulting in a mere 76.8% mIoU (mean intersection over union) and less than 80% recall. This means that in practical engineering inspections, existing technologies are highly prone to missing minute early corrosion or misclassifying water stains as corrosion. In contrast, the WSP-MambaNet method proposed in this application captures global long-range dependencies by introducing a Mamba state-space model. Using only image-level weak supervision labels, it achieves an mIoU of 89.4%, a 12.6 percentage point improvement over U-Net; its recall is even higher at 90.5%. This demonstrates the significant technical advantages of this method in identifying minute defects in complex backgrounds, greatly reducing the risk of missed detections. While state-of-the-art TransFormer-based models like Swin-UperNet achieve high accuracy (mIoU 88.1%), their computational demands are extremely high, with an FPS of only 8.5, far below the standard for real-time operation (typically >24 FPS). This means that deploying TransFormer models on handheld devices results in significant screen stuttering and latency, hindering smooth continuous scanning. Although the computational cost of our proposed method (158.4 GFLOPS) is moderate, thanks to the linear computational complexity and parallel scanning mechanism of the Mamba architecture, after TensorRT optimization, the inference speed reaches 24.6 FPS. This not only exceeds the threshold for smooth human vision (24 FPS), but also represents an inference efficiency improvement of 108% and 189% compared to DeepLabV3+ (11.8 FPS) and Swin-UperNet with equivalent accuracy.

[0181] When compared to real-time segmentation networks (such as BiSeNetV2), although its speed is extremely fast (62.4 FPS), it comes at the cost of sacrificing a significant amount of feature extraction capability, causing the mIoU to drop to 73.6%, which fails to meet the requirements of precise engineering quantization. In summary, the WSP-MambaNet proposed in this application is currently the only technical solution in comparative experiments that simultaneously meets the dual standards of "high accuracy (mIoU>89%)" and "real-time operation (FPS>24)". It successfully addresses the industry pain point of complex deep learning models struggling to run smoothly on handheld edge devices while maintaining detection accuracy.

[0182] To further illustrate the engineering significance of the above performance indicators, the following details three key advantages of this model in treating steel reinforcement corrosion:

[0183] (1) Extreme ability to capture minute pitting corrosion: In the early stages of steel corrosion, it often manifests as scattered "pockmarks" or "pinhole-like" pitting corrosion with a diameter of less than 2 mm. Existing technologies (such as U-Net and BiSeNet) lose these minute features in the feature map due to multiple downsampling (32-fold reduction) in the deep layers of the network, resulting in missed detections. The WSP-MambaNet architecture of this application retains 1 / 4 of the high-resolution parallel branches, and combined with Mamba's full-image scanning mechanism, it can keenly capture these minute texture changes. The high recall rate of 90.5% in the experiment proves that this model can detect early defects earlier than existing technologies, significantly improving structural safety.

[0184] (2) Anti-interference capability in complex environments: The construction site environment is complex, and the mud stains, oil stains, and water on the surface of the steel bars are very similar in color to rust (especially brown mud stains). Traditional CNN models, due to their limited receptive field, only focus on local color features, making it difficult to effectively distinguish between interference objects with similar appearances and real rust. They are prone to false positives due to feature confusion, resulting in a high false positive rate in the detection results. The Mamba long-distance dependency mechanism introduced in this model can combine the contextual information of the entire steel bar (e.g., rust is usually distributed longitudinally along the texture, while mud spots are usually randomly splashed) for comprehensive judgment. This global semantic analysis capability enables the model to effectively eliminate the interference of environmental noise and significantly improve the confidence of the detection results.

[0185] (3) Physical-level fine-grained fitting of rust boundaries: Rust is often diffusely distributed with blurred edges. Weakly supervised methods usually only generate a rough "heat map," which cannot accurately delineate the physical contour of rust, resulting in a large error in the final physical area (cm²) calculation. This application designs a "heat map inversion-boundary refinement" mechanism, which uses DenseCRF to introduce the RGB texture gradient of the original image as a constraint, forcing the blurred heat map edges to automatically "adhere" to the real physical rust boundary. This is also the key reason why the model can achieve an mIoU of 89.4%, ensuring the real accuracy of subsequent three-dimensional quantization calculations.

[0186] The above is only one specific implementation method of the present invention, but the design concept of the present invention is not limited thereto. Any non-substantial modifications made to the present invention using this concept shall be considered as infringing the protection scope of the present invention.

Claims

1. A method for inverting and quantifying the thermographic model of steel reinforcement corrosion based on a weakly supervised state-space model, characterized in that, Includes the following steps: Step 1: Construct a dataset of steel rebar images containing various lighting conditions and different degrees of corrosion as the training set: Step 2: Construct the WSP-MambaNet network, which includes an AGDP attention pyramid, a Mamba-HRNet parallel backbone, and a MIL-FPN weakly supervised inversion head. The training set is input into the AGDP attention pyramid to obtain a multi-scale spatially weighted feature map that suppresses background noise. This multi-scale spatially weighted feature map is then input into the Mamba-HRNet parallel backbone to obtain a high-resolution parallel feature stream containing global long-range texture dependencies. The high-resolution parallel feature stream is then input into the MIL-FPN weakly supervised inversion head to obtain a rust category activation map and image-level classification prediction probabilities. The loss function of the WSP-MambaNet network is a dynamically adjusted Focal Loss loss function. Training continues until the Focal Loss loss function converges to obtain the trained WSP-MambaNet network. Step 3: Input the image of the steel bar to be detected into the trained WSP-MambaNet network to obtain the results of corrosion category activation map inversion and three-dimensional quantization: the results include corrosion degree, corrosion area, corrosion region and processing suggestions.

2. The method for inverting and quantifying the thermographic model of steel corrosion based on a weakly supervised state-space model as described in claim 1, characterized in that, The AGDP attention pyramid consists of four levels, each composed of residual modules embedded with adaptive channel attention, used to preserve key texture features during downsampling. The specific data processing flow is as follows: S2.1) Residual Feature Extraction and Downsampling: Input Features First, basic features are extracted through a residual path consisting of two 3×3 convolutional layers. ; Where BN represents batch normalization, This represents a 3×3 convolution, and ReLU represents the rectified linear activation function. To achieve the pyramid downsampling function, the stride of the 3×3 convolution is set to 2, thereby halving the feature map resolution. S2.2) Channel Self-Attention Modeling: In order to capture long-distance dependencies across channels, i.e., the correlation between different rust texture modes in channels, the following model is used: Introduce a channel self-attention mechanism. Feature transformation: The channel dimension is compressed to 1×1 through 1×1 convolution. Remodeling after BN and ReLU ,in X represents the feature matrix after dimensionality reduction and reshaping. B represents the number of channels after compression, B represents the batch size, and H represents the input feature map. The height, W represents the input feature map. The width; Where Reshape represents feature reshaping, BN represents batch normalization, ReLU represents rectified linear activation function, and Conv... 1×1 Represents a 1×1 convolution; Channel correlation calculation: The covariance matrix between channels is calculated using matrix multiplication, and then normalized using Softmax to obtain the channel attention map M. Wherein, the element in the j-th row and i-th column of the attention graph M is This measures the influence of the i-th channel on the j-th channel; Softmax is the activation function; T represents the matrix transpose. Weighting and Aggregation: The attention map M is multiplied by the original feature X for weighted aggregation, then reshaped, 1×1 convolution, BN, and ReLU to restore the dimensions, and then compared with the input. By summing the residuals, we obtain the attention-enhanced features. : Wherein, Transform represents feature transformation and dimension restoration; Feature fusion and output: Features enhanced by attention The original input after downsampling and matching dimensions Element-wise summation is performed to obtain the final output multi-scale spatially weighted feature map of the current level. : This will be used as the input for the next level of the pyramid, with DownSample representing the downsampling.

3. The method for inverting and quantifying the thermogram of steel corrosion based on a weakly supervised state-space model as described in claim 1, characterized in that, The Mamba-HRNet parallel backbone is obtained by replacing all the basic residual convolutional blocks of the HRNet network with the Visual State Space Module (VSS Block). The data processing flow for the visual state space module is as follows: S3.1) Unfold the multi-scale spatially weighted feature map output by the AGDP attention pyramid or the feature map passed inside the parallel backbone of Mamba-HRNet into a one-dimensional sequence along the four directions of upper left, lower right, upper right and lower left, denoted as x; S3.2) Full image scanning: The one-dimensional input sequence x is recursively scanned using the discretized state equation, so that each pixel can perceive the texture context of the entire steel bar, and the processed sequence is obtained. S3.3) Reconstruction: The processed sequences are merged and restored to 2D feature maps as high-resolution parallel feature streams.

4. The method for inverting and quantifying the thermogram of steel corrosion based on a weakly supervised state-space model as described in claim 3, characterized in that, To convert continuous physical signal models into computer-processable digital signals, time-scale parameters are utilized. continuous state parameters Convert to discretized parameters And construct the discretized state equations described in step S3.2: in, This represents the discretized state transition matrix. Representing time scale parameters The product with the continuous state matrix A, This represents the discretized input projection matrix. Represents the identity matrix. Representing time scale parameters The product of the continuous input matrix B, This represents the hidden state vector at the current time t. Let C represent the input feature vector at the current time t, C represent the output projection matrix, and D represent the direct feedforward matrix.

5. The method for inverting and quantifying the thermogram of steel corrosion based on a weakly supervised state-space model as described in claim 1, characterized in that, The data processing flow of the MIL-FPN weakly supervised inversion head is as follows: The high-resolution parallel feature stream of the Mamba-HRNet parallel backbone, namely the deep semantic features output at 1 / 32 resolution from the Visual State Space Block (VSS Block) in the last layer of Stage 4, is denoted as... The second shallow texture feature at 1 / 8 resolution output by Stage 3 is denoted as... And the first shallow texture feature at 1 / 4 resolution output by Stage 2, denoted as In the future Upsampling step by step and with , Before performing horizontal fusion, an efficient spatial channel attention module (ESCA) is introduced: The ESCA data processing flow is as follows: For features... Spatial augmentation is performed to obtain intermediate features. ; in, This represents the Sigmoid activation function. Indicates average pooling. This indicates max pooling. Represents convolution; exist Based on this, channel attention is introduced to obtain attention-enhanced features. ; in, Indicates global average pooling. This represents one-dimensional convolution; The enhanced features are added to the upsampled deep features to obtain the fused features. ; in, Indicates upsampling; Then perform a multi-example inversion: Heatmap generation: fusing features The class activation map is mapped using a 1×1 convolution. That is, the original heatmap; Global prediction: To predict the probability of an image belonging to a specific type of corrosion, global max pooling is performed. The steps of global max pooling are as follows: For each category channel, the maximum value of all pixels in that channel is selected as the predicted probability for that category, i.e. Where i and j represent the row and column indices of the pixel's spatial coordinates; Let represent the pixel value of the c-th category channel in the rust category activation map at coordinates (i, j), and max(i, j) represent the pixel value of the c-th category channel in the rust category activation map at coordinates (i, j). cam Take the maximum value at all spatial locations; Boundary Refinement: A fully connected conditional random field, DenseCRF, is introduced. Using the color and texture boundaries of the original RGB image as constraints, the heatmap is spatially regularized to ensure its edges closely conform to the realistic physical contours of steel reinforcement corrosion. in M represents the original RGB image. refined This represents the refined probability distribution diagram, and CRF represents the optimization of a fully connected conditional random field. Then, mask generation is performed on the refined heatmap. Pixel-by-pixel classification is performed to generate pseudo-segmentation masks containing multi-level semantic information: 0-no corrosion, 1-slight corrosion, 2-moderate corrosion, 3-severe corrosion. These masks are directly used for subsequent 3D hierarchical quantization. For each pixel coordinate (x, y) in the image, its probability values ​​in all category channels are compared, and the category with the highest probability is selected as the final label of the pixel. Where x and y represent the horizontal and vertical coordinates of a pixel. This represents the probability value of the c-th category in the refining heatmap; Represents pixels The final classification labels are: 0 - No rust, 1 - Slight, 2 - Moderate, 3 - Severe.

6. The method for inverting and quantifying the thermogram of steel corrosion based on a weakly supervised state-space model as described in claim 1, characterized in that, The WSP-MambaNet network employs the Focal Loss function to address the imbalance problem, with the loss as follows: Where Loss represents Focal Loss; C represents the number of corrosion categories; The image-level true label is 0 or 1, indicating whether the image contains type c corrosion; This represents the image-level probability predicted by the network; that is... , Set it to 2. Indicates in Take the maximum value at all spatial locations (x, y). This represents the pixel value of the c-th category channel in the rust category activation map at coordinates (x, y).

7. The method for inverting and quantifying the thermogram of steel corrosion based on a weakly supervised state-space model as described in claim 1, characterized in that, The steps for heatmap inversion and 3D quantization in step three are as follows: S7.1) Set a confidence threshold, truncate the heatmap, and generate a binary pseudo-segmentation mask; S7.2): Real-time calculated camera pose and pseudo-segmentation mask; S7.3) Back projection: Based on the depth stream and IMU data input from the depth camera, a 3D point cloud map is obtained through Simultaneous Localization and Mapping (SLAM) thread. Traversing the 3D point cloud map For each point P in the array, project it back to the current 2D frame and query its state on the "pseudo-segmentation mask"; S7.4) Combine the observation results of point P from multiple frames and use the majority voting method to determine the final three-dimensional semantic state of point P; S7.5) Generate a semantic point cloud model with rust color markers. And output; S7.6) Meshization: For Greedy projection triangulation is performed to generate a continuous 3D mesh. Each triangular facet on the mesh is traversed. If the semantic label of the three vertices of a certain triangular facet is "rust", its physical area is calculated using Heron's formula. S7.7) Result Output: Displayed on the screen: Hierarchical quantization calculation: Traverse the 3D mesh and perform surface accumulation according to the semantic labels of the mesh vertices; The results are output, displaying a structured health diagnosis report on the screen: No corrosion: 0.0 cm 2 No processing required; Slight rust, i.e., yellow surface rust: XX.X cm² — Recommended treatment: Surface rust removal; Moderate rust, i.e., brown layer or slight peeling: YY.Y cm² — Recommended treatment: sandblasting, grinding for deep rust removal; Severe corrosion, i.e., pitting or flaking: ZZ.Z cm² — Recommended treatment: structural reinforcement or replacement; Overall corrosion rate: WW % Transformed into a visual model: an interactive, rotateable 3D color steel bar model; The semantic tags include: 0 - no corrosion, 1 - mild, 2 - moderate, and 3 - severe.

Images