Binary classification method for steel corrosion detection with fusion of boundary-aware attention mechanism

Abstract

The present application relates to the technical field of computer vision and deep learning, and discloses a double classification steel structure corrosion detection method fusing boundary perception attention mechanism, which comprises the following steps: obtaining bridge steel structure image data and outputting a reconstructed feature map; a feature decoupling module generates a spatial attention response map and a spatial gradient amplitude matrix based on the reconstructed feature map, separates out a corrosion body feature map and a corrosion edge gradient feature map; the spatial information entropy of the gradient amplitude matrix is calculated to generate a fusion weight, the body and different receptive field features are weighted and aggregated to generate a dynamic pooling feature map, which is fused with the reconstructed feature map and outputted as a candidate corrosion boundary box; corresponding regional features are extracted, and corrosion class and corrosion grade probability distribution are respectively outputted; a joint penalty loss is calculated according to an inter-class prior mutual exclusion matrix to update the network weight, and the matrix is used to perform state combination verification in the detection stage. The present application solves the problem of insufficient gradual edge feature extraction, avoids logical conflicts in evaluation results, and improves detection accuracy.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and deep learning technology, specifically to a dual-classification method for detecting corrosion in steel structures that integrates a boundary-aware attention mechanism. Background Technology

[0002] With the continuous development of transportation infrastructure, corrosion of bridge steel structures during long-term service seriously affects structural safety and service life, making efficient and accurate corrosion detection crucial. Traditional bridge corrosion detection mainly relies on manual visual inspection or manual interpretation of images collected by inspection vehicles. These methods are not only labor-intensive and time-consuming, but also limited by the experience, perspective, and external environmental conditions of the inspectors, resulting in highly subjective results, a high rate of missed detections, and potential safety hazards. Therefore, utilizing computer vision and deep learning technologies to automate bridge corrosion detection has become an inevitable trend to improve inspection efficiency and result stability.

[0003] In practical engineering maintenance decisions, automated inspection often requires dual classification of corrosion defects, that is, simultaneously identifying the specific category of corrosion and assessing its corrosion evolution level. However, existing deep learning detection networks still have significant limitations in handling such dual classification tasks. On the one hand, corrosion defects on steel structure surfaces typically exhibit a gradual transition to normal areas, lacking a clear pixel boundary between the defect and the normal background. Conventional convolutional neural networks treat the feature map as a whole during feature extraction, making it difficult to effectively separate the rust texture from the boundary gradient, resulting in insufficient feature representation ability when dealing with blurred edges. Simultaneously, due to the significant differences in scale and morphology among different corrosion regions, existing feature aggregation modules often employ fixed-size pooling operations, failing to adaptively adjust the receptive field according to the specific distribution patterns of defect boundaries in the input image, thus limiting the model's ability to capture complex and irregular defect features.

[0004] On the other hand, existing multi-task detection networks typically employ independent prediction branches to calculate the probability of defect category and corrosion level separately when performing dual-classification prediction. This purely data-driven independent evaluation mechanism lacks the constraints of engineering physical logic, leading to prediction branches easily outputting classification combinations that contradict physical states in actual operation. For example, it might output a severe corrosion assessment result while classifying a defect as surface rust. This logical conflict in classification deviates significantly from objective physical laws, reducing the reliability of the detection model's output and making it difficult to directly meet the stringent requirements of automated inspection and condition assessment in practical engineering. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a dual-classification steel structure corrosion detection method that integrates a boundary-aware attention mechanism. This method solves the problems of insufficient feature extraction capability when processing gradual edge transitions in existing steel structure corrosion detection methods, and the tendency to generate logical conflicts between corrosion category and corrosion level classification.

[0006] To achieve the above objectives, the present invention provides a dual-classification steel structure corrosion detection method that integrates boundary-aware attention mechanisms, comprising the following steps: The system acquires image data of the bridge steel structure to generate an initial feature map, which is then input into the feature extraction module to output a reconstructed feature map. The feature decoupling module generates a spatial attention response map and a spatial gradient magnitude matrix based on the reconstructed feature map, and separates the rust body feature map and the rust edge gradient feature map. Spatial information entropy is calculated based on the spatial gradient magnitude matrix to generate fusion weights. The rust body feature map is pooled to generate different receptive field features. The rust body feature map and different receptive field features are weighted and concatenated to generate a dynamic pooled feature map. The dynamic pooled feature map and the reconstructed feature map are fused to generate an aggregated feature map and output candidate rust bounding boxes. Based on the candidate rust bounding box, the corresponding rust body region features and rust edge region features are extracted from the rust body feature map and rust edge gradient feature map, respectively, and the rust category probability distribution vector and corrosion level probability distribution vector are output respectively. The network weights are updated by calculating the joint penalty loss based on the preset inter-class prior mutual exclusion matrix, and the state combination verification is performed using the inter-class prior mutual exclusion matrix during the detection phase.

[0007] This method separates ontological features and edge features through a feature decoupling mechanism to extract the rust edge contour; it adjusts the fusion ratio of multi-scale features through spatial information entropy to adapt to rust regions of different sizes; and it uses a matrix structure to construct prior physical mutual exclusion logic to constrain classification combination prediction during network training and inference stages, thereby reducing the false detection rate.

[0008] Furthermore, the steps of acquiring bridge steel structure image data to generate an initial feature map, and inputting it into the feature extraction module to output a reconstructed feature map include: The image data of the bridge steel structure is used as the input image tensor. A spatial-to-depth rearrangement operation is performed on the input image tensor to generate a channel augmentation tensor. A two-dimensional spatial convolution operation is then performed on the channel augmentation tensor to generate an initial feature map. Some convolutional units divide the channels of the initial feature map into a computational channel subset and a reserved channel subset. They perform two-dimensional spatial convolutional feature extraction on the computational channel subset to generate processed sub-feature maps, and perform identity mapping on the reserved channel subset. Perform a cross-channel concatenation operation on the processed sub-feature map and the reserved channel subset after performing identity mapping, and output the reconstructed feature map.

[0009] This step utilizes spatial-to-depth rearrangement to preserve spatial information, and combines channel partitioning and identity mapping mechanisms to reduce the computational cost of spatial convolution while maintaining the original feature representation.

[0010] Furthermore, the steps for generating the spatial attention response map and the spatial gradient magnitude matrix include: the feature decoupling module uses the first branch to extract the dependencies between feature channels, uses the second branch to extract the feature responses within the local spatial receptive field, and performs cross-dimensional feature splicing and cross-channel aggregation on the output features of the two branches to generate a single-channel spatial attention response map. Using the defined difference operator, the partial derivative matrices of the spatial attention response map in the horizontal width direction and the vertical height direction are calculated respectively, generating the horizontal partial derivative matrix and the vertical partial derivative matrix. The sum of squares of the corresponding spatial elements of the horizontal and vertical partial derivative matrices is calculated and the square root is taken to generate the spatial gradient magnitude matrix. This step quantifies the local spatial changes of the feature map by calculating the partial derivative matrices in the horizontal and vertical directions, obtains the edge gradient magnitude values, and provides a calculation basis for feature separation.

[0011] Furthermore, the steps for separating the rust body feature map and the rust edge gradient feature map include: the feature decoupling module normalizes the values ​​in the spatial gradient magnitude matrix to the range of 0 to 1 to generate a normalized spatial gradient magnitude matrix; tensor broadcasting operation along the channel dimension is performed on the spatial attention response map and the normalized spatial gradient magnitude matrix; and masking operation is performed on the reconstructed feature map using the expanded spatial attention response map as a mask matrix to extract the rust body feature map. Using the extended normalized spatial gradient magnitude matrix as a mask matrix, a masking operation is performed on the reconstructed feature map to extract the gradient feature map of the rust edges. This step uses the masking operation to perform spatial filtering on the feature map, aligning spatial attention and gradient magnitude in the channel dimension, thereby separating the independent ontological features and edge features.

[0012] Furthermore, the steps for calculating spatial information entropy and generating fusion weights based on the spatial gradient magnitude matrix include: traversing the normalized spatial gradient magnitude matrix according to spatial coordinates, mapping the gradient magnitude value of each spatial location using an exponential function, calculating the sum of the exponential mapping values ​​of all spatial locations, and dividing the exponential mapping value of each spatial location by the sum of the exponential mapping values ​​to generate a spatial probability mapping map. By traversing the spatial probability map according to spatial coordinates, the probability value at each spatial location is multiplied by the base-2 logarithm of that probability value. The sum of all products is then taken as the negative, yielding the spatial information entropy in scalar form. This step utilizes spatial information entropy to characterize the dispersion of gradient distribution in local regions, providing a numerical basis for the subsequent fusion weight allocation of multi-scale feature pooling.

[0013] Furthermore, the steps of pooling the rust body feature map to generate different receptive field features, and weightedly concatenating and aggregating the rust body feature map with different receptive field features to generate a dynamic pooled feature map include: inputting the spatial information entropy into the multilayer perceptron to obtain four fusion weights corresponding to the original feature branch, the first pooling branch, the second pooling branch, and the third pooling branch. Channel adjustment is performed on the rust body feature map to generate a channel-adjusted feature map, which is then sequentially input into three serially connected max pooling layers. The first pooling feature map, the second pooling feature map, and the third pooling feature map are output as different receptive field features. The four fusion weights are respectively multiplied element-wise by the rust body feature map, the first pooling feature map, the second pooling feature map, and the third pooling feature map. After being concatenated along the channel dimension, the channel fusion is completed to generate a dynamic pooling feature map.

[0014] This step obtains receptive fields of different scales through serial multi-level max pooling, and uses the weights generated by the multilayer perceptron to perform element-wise weighted concatenation to complete multi-scale feature aggregation.

[0015] Furthermore, the steps of extracting corresponding rust body region features and rust edge region features from the rust body feature map and rust edge gradient feature map based on the candidate rust bounding box, and outputting the rust category probability distribution vector and corrosion level probability distribution vector respectively, include: Based on the size of the candidate rust bounding box, select the rust body feature map and rust edge gradient feature map with matching spatial resolution, and use bilinear interpolation to generate the rust body region feature and rust edge region feature corresponding to each candidate rust bounding box. Global average pooling is performed on the features of the rust body region to generate a one-dimensional rust body feature vector, which is then input into a fully connected network layer to generate a rust category probability distribution vector. Global average pooling is performed on the features of the rust edge region to generate a one-dimensional rust edge feature vector, which is then input into an independent fully connected network layer to generate a corrosion level probability distribution vector. This step extracts corresponding region features from the rust body feature map and the rust edge gradient feature map, respectively, based on the classification requirements of rust category and corrosion level, establishing a mapping relationship between feature attributes and the classification task.

[0016] Furthermore, the definition rules for the inter-class prior mutual exclusion matrix include: based on the preset correspondence between the rust categories and corrosion levels of bridge steel structures, an inter-class prior mutual exclusion matrix for logical verification is defined, wherein the number of rows in the inter-class prior mutual exclusion matrix is ​​equal to the total number of rust categories and the number of columns is equal to the total number of corrosion levels. Each matrix element corresponds to a combination of a rust type and a corrosion level. When the corresponding combination meets the preset state correspondence, the matrix element takes the value of 1. When the corresponding combination does not conform to the preset state correspondence, the matrix element takes the value of 0; The inter-class prior mutual exclusion matrix is ​​stored as a static tensor that cannot be trained in the detection network. This step establishes a static inter-class prior mutual exclusion matrix, transforming the correspondence between rust categories and corrosion levels into a numerical form for subsequent logical constraints.

[0017] Furthermore, the step of calculating the joint penalty loss and updating the network weights based on the preset inter-class prior mutual exclusion matrix includes: obtaining training samples with real rust bounding boxes, calculating the cross-union ratio between each candidate rust bounding box and each real rust bounding box, and determining the candidate rust bounding box with the largest cross-union ratio and a cross-union ratio of not less than 0.5 as a positive sample. Multiply each rust category probability in the rust category probability distribution vector corresponding to the positive sample with each corrosion level probability in the corrosion level probability distribution vector to generate a joint probability matrix; traverse the joint probability matrix and the inter-class prior mutual exclusion matrix, and when the matrix element at the corresponding position in the inter-class prior mutual exclusion matrix is ​​0, extract the joint probability value at the same position in the joint probability matrix. The extracted joint probability values ​​are summed to generate a logical conflict penalty loss, which serves as the joint penalty loss. This step, performed during the training phase, calculates the logical conflict penalty loss using an inter-class prior mutual exclusion matrix, reducing the predicted probability of mutually exclusive classification combinations and ensuring the network output conforms to a predefined correspondence.

[0018] Furthermore, the step of performing state combination verification using the inter-class prior mutual exclusion matrix in the detection phase includes: for each candidate rust bounding box, multiplying the probability of each rust category in the rust category probability distribution vector corresponding to the candidate rust bounding box with the probability of each corrosion level in the corrosion level probability distribution vector to generate a joint probability matrix. Multiply the joint probability matrix element by element with the inter-class prior mutual exclusion matrix so that the joint probability corresponding to the combination in the inter-class prior mutual exclusion matrix that has a value of 0 is set to 0. Among the combinations with a value of 1 in the inter-class prior mutual exclusion matrix, the combination of rust category and corrosion level with the highest joint probability is selected as the final classification result of the candidate rust bounding box. This step performs a masking operation on the joint probability matrix using the inter-class prior mutual exclusion matrix during the inference phase, eliminating classification combinations that do not conform to the correspondence, and outputting the detection result.

[0019] This invention provides a dual-classification method for detecting corrosion in steel structures that integrates a boundary-aware attention mechanism. It offers the following advantages: 1. The feature decoupling module of this invention calculates the spatial attention response map and the spatial gradient magnitude matrix, and uses partial derivative operators to quantify spatial feature changes, separating the reconstructed features extracted by the network into a corrosion body feature map and a corrosion edge gradient feature map. This mechanism effectively overcomes the problem of the lack of a clear boundary between the corrosion area of ​​the steel structure and the normal background, providing independent and targeted data support for downstream defect category identification and corrosion evolution level assessment, and improving the accuracy of bi-branch classification prediction.

[0020] 2. This invention utilizes spatial information entropy to scalarly evaluate the distribution pattern of gradient response in image space, and combines it with a multilayer perceptron to dynamically generate fusion weights for different receptive field features. This method enables the detection network to adaptively allocate the fusion ratio of multi-scale feature pooling results based on the concentration or dispersion of defect boundary features in the input image, enhancing the model's spatial adaptability and feature capture capability for rusted regions of different sizes and irregular shapes.

[0021] 3. This invention constructs a non-trainable inter-class prior mutual exclusion matrix, transforming the correspondence rules between rust categories and corrosion levels in engineering practice into numerical constraint tensors within the network. By introducing a logical conflict penalty loss during model training and performing state combination verification during inference, the problem of mutual exclusion of physical state combinations arising from independent prediction by dual-branch networks is effectively avoided, ensuring that the final output defect detection results conform to engineering detection standards and objective physical laws. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of the overall process of the present invention; Figure 2 This is a network structure diagram of an embodiment of the present invention; Figure 3 This is a schematic diagram of the C2f-Faster module structure in the feature extraction module of this embodiment of the invention; Figure 4 This is a schematic diagram of a portion of the convolutional unit structure according to an embodiment of the present invention; Figure 5 This is a schematic diagram of the internal cascaded structure of the two-dimensional spatial convolution module in an embodiment of the present invention; Figure 6 This is a schematic diagram of the dynamic pooling module structure according to an embodiment of the present invention. Detailed Implementation

[0023] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] See attached document Figure 1 and attached Figure 2 This invention provides a dual-classification steel structure corrosion detection method that integrates a boundary-aware attention mechanism, comprising the following steps: S10: Acquire bridge steel structure image data and input it into the image processing module. The image processing module performs spatial-to-depth rearrangement and two-dimensional convolution operations to generate an initial feature map, which is then transmitted to the feature extraction module. The feature extraction module uses some convolution units to divide the channels into a computational channel subset and a reserved channel subset for local feature extraction and cross-channel stitching, outputting reconstructed feature maps at different levels. S20, the reconstructed feature maps of different levels are input into the feature decoupling module. The feature decoupling module generates a spatial attention response map using convolutional branches, performs spatial partial derivative operations on the spatial attention response map to extract the spatial gradient magnitude matrix, and performs a mask dot product operation on the reconstructed feature map by combining the spatial attention response map and the spatial gradient magnitude matrix, thus separating the feature space into the corrosion body feature map and the corrosion edge gradient feature map. S30, the dynamic pooling module receives the spatial gradient magnitude matrix and the corrosion body feature map, performs global normalization on the spatial gradient magnitude matrix to generate a discrete spatial probability distribution matrix, and calculates the spatial information entropy of the discrete spatial probability distribution matrix; based on the spatial information entropy, it generates normalized fusion weights through a multilayer perceptron, and performs weighted splicing and aggregation on the corrosion body feature map and different receptive field features obtained by three-level serial max pooling according to the normalized fusion weights to generate a dynamic pooling feature map. S40, the feature aggregation module receives the deepest dynamic pooling feature map and the reconstructed feature maps of the other layers, performs multi-scale feature fusion and generates an aggregated feature map, and inputs the aggregated feature map into the prediction module. The prediction module extracts the corresponding rust body region features and rust edge region features for each candidate rust bounding box. The category prediction branch outputs the rust category probability distribution vector based on the rust body region features, and the level prediction branch outputs the corrosion level probability distribution vector based on the rust edge region features. The shared localization regression layer outputs the coordinates of the rusted bounding box based on the aggregated feature map, and the target confidence layer outputs the target confidence of the corresponding bounding box based on the aggregated feature map. In the model training phase, the network training module receives the probability distribution vectors of the two branches, the predicted bounding box coordinates, and the target confidence. Based on the internally preset inter-class prior mutual exclusion matrix, it outputs the joint probability distribution of the results and calculates the joint penalty loss. It then calculates the basic detection loss by combining the rust category cross-entropy loss, corrosion level cross-entropy loss, target confidence loss, and bounding box regression loss. Finally, it weights and merges the basic detection loss and the joint penalty loss to obtain the overall loss. With the goal of minimizing the overall loss, it performs gradient backpropagation to update the network weights.

[0025] S101. During the model training phase, images of bridge steel structure corrosion are acquired. The spatial morphology of the samples is changed by rotation, translation, and flipping. The imaging attributes of the samples are changed by adjusting the color and sharpness, thus completing the expansion of training samples. When performing rotation, translation, flipping, scaling, or cropping operations, the bounding box coordinates of the rusted area are updated synchronously according to the same spatial transformation parameters, and invalid annotations located outside the cropped area are deleted. The expanded image is then labeled with the rusted area, rust type, corrosion level, and bounding box coordinates. During the model inference phase, the original inspection image data of the bridge steel structure to be inspected is acquired.

[0026] To eliminate image size differences caused by different inspection equipment viewing angles or shooting distances, and to adapt to the fixed input dimension requirements of the network model, the original inspection image data is scaled and cropped according to the set size to generate a standard dimension input image tensor.

[0027] For the specific implementation of image scaling and cropping operations, those skilled in the art can use bilinear interpolation or center cropping algorithms. The relevant operational specifications are well-known technologies in this field and will not be elaborated here.

[0028] The input image tensor is defined as Its dimensions include height. ,width and number of channels For standard red, green, and blue three-channel color inspection images, the number of channels... The value is 3.

[0029] S102, perform a spatial-to-depth rearrangement operation on the input image tensor to generate a channel augmentation tensor. The spatial-to-depth rearrangement operation extracts and reassembles adjacent pixel data in the spatial dimension into the channel dimension with a fixed step size according to the set spatial downsampling ratio.

[0030] The set spatial downsampling factor is , Select a positive integer greater than 1. For the input image tensor... Periodic pixel sampling is performed based on the spatial row index and column index, and the sampling extraction rules are as follows: Extract row index pairs The modulo result equals And column index pairs The modulo result equals The spatial pixel set. Among them, and All are greater than or equal to 0 and less than An integer representing the height of the input image tensor. With width All can be Divisible, forming The extracted pixel set is stacked to obtain the corresponding number of rearranged sub-tensors. All rearranged sub-tensors are concatenated along the channel dimension to output the complete channel augmentation tensor. .

[0031] In actual calculations, the spatial downsampling factor is set. When the value is 2, the height and width of the channel expansion tensor become half of the original input image tensor, and the number of channels is expanded to 4 times the number of channels in the original input image.

[0032] In one embodiment, the input image size is set to 768×768, and the space-to-depth rearrangement operation forms four rearranged subtensors.

[0033] This spatial pixel rearrangement mechanism reduces the spatial resolution of the image and the number of subsequent computational parameters while preserving the feature distribution information of the underlying steel structure image and reducing the loss of high-frequency edge information.

[0034] Reference Appendix Figure 5 In step S103, a two-dimensional spatial convolution operation is performed on the channel augmentation tensor to generate an initial feature map. The channel augmentation tensor is then processed sequentially through a two-dimensional convolutional computation layer, a batch normalization layer, and a nonlinear activation function layer for forward computation. The local spatial features between the rearranged discrete pixels are then fused through the two-dimensional convolutional computation layer.

[0035] For the details of batch normalization and activation function parameter configuration, those skilled in the art can set them according to conventional deep network architectures. The specific parameter tuning is a well-known technique in the field and will not be elaborated here.

[0036] The kernel size of the current 2D convolutional computation layer is set to 3×3, the stride parameter is 1, the zero-padding parameter is 1, and the number of output feature channels is... The stride parameter is set to 32. The stride parameter of the 1×1 2D convolutional computation layer used for channel mapping in the backbone network is 1, and the zero-padding parameter is 0. Channel augmentation tensor The initial feature map is generated by mapping the data to a new feature space through a two-dimensional convolutional computation layer. The initial feature map height is Width is The number of channels is The initial feature map serves as input data for subsequent backbone network operations, providing basic feature representations.

[0037] See attached document Figure 3 S104, the feature extraction module receives the input feature map passed from the upper-level network. The feature extraction module adopts a C2fFaster structure that introduces the EMA attention mechanism, replaces the bottleneck unit in the original C2f structure with FasterBlock, and sets some convolutional units in FasterBlock.

[0038] Some convolutional units perform a segmentation operation along the channel dimension of the input feature map according to a set channel segmentation ratio, dividing the channels of the input feature map into a subset of computational channels and a subset of retained channels. Setting the channel segmentation ratio to one-quarter means extracting the input features... Figure 4 One-third of the channels are used as the computational channel subset, and the remaining three-quarters of the channels are used as the reserved channel subset.

[0039] The specific data processing logic for channel segmentation is as follows: following the channel index order of the feature map, one-quarter of the preceding continuous channels are truncated and assigned to the calculation channel subset, while the remaining channels are assigned to the reserved channel subset. Through the channel separation operation, the data stream of the input feature map is divided into two independent branches to distinguish the computational logic in subsequent calculations.

[0040] Reference Appendix Figure 4 In step S105, some convolutional units perform two-dimensional spatial convolutional feature extraction on the subset of computational channels and perform identity mapping on the subset of retained channels. For the subset of computational channels, some convolutional units use a two-dimensional convolutional computation layer with a kernel size of 3×3 to perform local spatial feature mapping and generate processed sub-feature maps.

[0041] The cascaded internal components of this two-dimensional convolutional computation layer include batch normalization operations and activation function processing. For the weight initialization and bias settings within the two-dimensional convolutional computation layer, those skilled in the art can use conventional backpropagation algorithms for random configuration and iterative updates; the parameter setting specifications are well-known techniques in this field and will not be elaborated upon here.

[0042] For a subset of channels to be preserved, some convolutional units skip the convolutional computation layer and directly preserve the original values ​​of the feature tensors within that subset of channels and pass them to subsequent network layers. This data transfer logic that does not change the original feature values ​​is defined as an identity mapping.

[0043] S106, some convolutional units perform cross-channel concatenation operations on the processed sub-feature map and the reserved channel subset after identity mapping, and output the reconstructed feature map.

[0044] The backbone network consists of four sequentially connected feature extraction stages. Each feature extraction stage is equipped with a C2fFasterEMA structure. Two-dimensional convolutional computation layers with a kernel size of 3×3, a stride parameter of 2, and a zero-padding parameter of 1 are set between adjacent feature extraction stages to generate four reconstructed feature maps with different spatial scales in sequence.

[0045] The four reconstructed feature maps are processed through steps S201 to S206 respectively to generate corresponding levels of corrosion body feature maps and corrosion edge gradient feature maps. The deepest corrosion body feature map is then input into the dynamic pooling module.

[0046] The processed sub-feature map and the retained channel subset are aligned and stacked along the channel dimension. After the stacking operation, the total number of channels in the reconstructed feature map is consistent with the initial total number of channels in the input feature map.

[0047] This part of the convolution mechanism extracts local features of steel structure images from a subset of computational channels while maintaining the transmission of underlying basic features of the image by using the identity mapping of the retained channel subset, thus reducing feature redundancy and computational parameters caused by conventional full-channel convolution operations.

[0048] S201, the feature decoupling module receives the reconstructed feature map after feature enhancement by the C2fFasterEMA structure, and uses internal parallel convolutional branches to obtain spatial and channel dependencies to generate a spatial attention response map.

[0049] Specifically, the feature decoupling module divides the reconstructed feature map into multiple subgroups along the channel dimension and uses two parallel processing branches to compute the feature subgroups. The first branch uses a 2D convolutional layer with a kernel size of 1×1 to extract the dependencies between feature channels, and the second branch uses a 2D convolutional layer with a kernel size of 3×3 to extract the feature responses within the local spatial receptive field.

[0050] After completing parallel computation, the feature decoupling module performs cross-dimensional feature concatenation on the output features of the two branches, performs cross-channel aggregation using a two-dimensional convolutional computation layer with 1 output channel, and then uses the Sigmoid activation function to map the values ​​to the range of 0 to 1 to generate a single-channel spatial attention response map. The height and width of the spatial attention response map are consistent with the reconstructed feature map.

[0051] From a physical perspective, the pixel values ​​within the spatial attention response map represent the contribution of the corresponding spatial location features to defect identification. A value closer to 1 indicates a higher probability that the location contains a defect; a value closer to 0 indicates that the location tends to be a defect-free, normal background.

[0052] S202, combining the apparent characteristics of surface defects in steel structures, extracts spatial partial derivative data. In actual steel structure engineering environments, the evolution of corrosion defects exhibits a physical phenomenon of spreading from the center to the periphery, resulting in a lack of clear pixel boundaries between the defect edge region and the normal region.

[0053] To extract the feature response of the fuzzy boundary, the feature decoupling module performs spatial partial derivative operations on the spatial attention response map. Using the defined difference operator, the partial derivative matrices of the spatial attention response map in the horizontal width direction and the vertical height direction are calculated respectively, generating the horizontal and vertical partial derivative matrices.

[0054] For the specific form of the difference operator, those skilled in the art can use the conventional Sobel operator to perform convolution calculation and extraction. The weight configuration of its calculation kernel is a well-known technology in this field and will not be elaborated here.

[0055] S203, calculate and generate the spatial gradient magnitude matrix. After obtaining the horizontal and vertical partial derivative matrices, the feature decoupling module calculates the sum of squares of the corresponding spatial position elements and performs a square root operation on the result to generate the spatial gradient magnitude matrix.

[0056] The feature decoupling module normalizes the values ​​in the spatial gradient magnitude matrix to the range of 0 to 1, generating a normalized spatial gradient magnitude matrix.

[0057] Through this gradient magnitude synthesis operation, the spatial gradient magnitude matrix quantifies the spatial rate of change of the attention weights. In the defect center region or normal background region, the spatial distribution of attention weight values ​​is gradual, and the corresponding gradient values ​​are in the lower range; while in the edge evolution region where the defect transitions to the background, the attention weights change rapidly, producing a high gradient numerical response.

[0058] This computational logic, without adding extra network training branches, utilizes mathematical partial derivative operators to achieve the perception and extraction of the gradual rust boundary of the steel structure, providing location guidance data for subsequent feature space decoupling.

[0059] S204, the feature decoupling module receives the reconstructed feature map output from the previous network, as well as the generated spatial attention response map and spatial gradient magnitude matrix. Since the spatial attention response map and spatial gradient magnitude matrix are aggregated across channels during the generation stage, only a single channel dimension is retained.

[0060] To satisfy the dimension alignment condition for subsequent element-wise multiplication operations, the feature decoupling module performs tensor broadcasting along the channel dimension on the spatial attention response map and the spatial gradient magnitude matrix. The single-channel spatial attention response map and the single-channel spatial gradient magnitude matrix are copied and expanded to ensure their channel count matches the number of channels in the multi-channel reconstructed feature map.

[0061] The specific underlying data copying logic of tensor broadcast operations can be implemented by those skilled in the art using conventional deep learning computing frameworks. Its tensor alignment mechanism is a well-known technology in the field and will not be elaborated here.

[0062] S205, using the expanded spatial attention response map as a mask matrix, performs masking operations on the reconstructed feature map to extract the corrosion body feature map. The specific calculation process is as follows: Each pixel feature value in the reconstructed feature map is element-wise multiplied with the weight value of the corresponding spatial coordinates and channel position in the expanded spatial attention response map. Through this masking operation, high-value regions in the spatial attention response map represent areas with a higher probability of containing steel structure defects.

[0063] The corresponding reconstructed feature values ​​are retained proportionally, with low-value areas representing normal steel structure backgrounds. The corresponding feature values ​​are then reduced. This operation, based on the attention weight distribution pattern, filters out feature data representing the core defect area and separates it into an independent feature space to generate a corrosion body feature map.

[0064] S206, using the expanded normalized spatial gradient magnitude matrix as a mask matrix, performs a masking operation on the reconstructed feature map to extract the gradient feature map of the rust edge. The specific calculation process is as follows: Each feature pixel in the reconstructed feature map is element-wise multiplied with the corresponding gradient value in the expanded spatial gradient magnitude matrix. This operation strengthens the feature weights of regions with high gradient values ​​in the reconstructed feature map, while relatively reducing the feature responses of regions with gentle spatial distribution, thus generating a gradient feature map of the rust edge.

[0065] This feature separation mechanism decouples the network data stream at the feature representation level, generating an ontological feature data stream for defect category identification and an edge gradient feature data stream for corrosion evolution level assessment, providing underlying data support for subsequent dual-branch network processing.

[0066] S301, the dynamic pooling module receives the deepest normalized spatial gradient magnitude matrix output by the feature decoupling module and performs spatial probability mapping on the normalized spatial gradient magnitude matrix.

[0067] The dynamic pooling module traverses the normalized spatial gradient magnitude matrix according to the spatial coordinates, maps the gradient magnitude value of each spatial location using an exponential function, calculates the sum of the exponential mapping values ​​of all spatial locations, and divides the exponential mapping value of each spatial location by the sum of the exponential mapping values ​​to generate a spatial probability mapping map.

[0068] The sum of the probability values ​​of all spatial locations in the spatial probability map is 1, and each probability value represents the relative proportion of the boundary gradient response of the corresponding spatial location in the total boundary gradient response.

[0069] S302, the dynamic pooling module calculates spatial information entropy based on the spatial probability map. The dynamic pooling module traverses the spatial probability map according to the spatial coordinates, multiplies the probability value of each spatial location by the logarithm of that probability value to the base 2, sums all the product results and takes the negative, to obtain the spatial information entropy in scalar form.

[0070] When calculating the logarithmic value, a preset positive real offset is added to each probability value to avoid the input value of the logarithmic function being 0. Spatial information entropy is used to characterize the concentration and dispersion of the gradient response of the corrosion boundary in the image space. The more uniform the distribution of the gradient response of the corrosion boundary, the greater the spatial information entropy; when the gradient response of the corrosion boundary is concentrated in a few spatial locations, the spatial information entropy is smaller.

[0071] S303, the dynamic pooling module inputs spatial information entropy into the multilayer perceptron to generate fusion weights corresponding to features in different receptive fields. The multilayer perceptron consists of a first fully connected layer, a ReLU activation function, and a second fully connected layer connected in sequence.

[0072] The first fully connected layer maps the scalar form of spatial information entropy to a preset intermediate feature dimension, and the second fully connected layer outputs a one-dimensional weight vector of length 4. The dynamic pooling module uses the Softmax function to normalize the one-dimensional weight vector, obtaining four fused weights corresponding to the original feature branch, the first pooling branch, the second pooling branch, and the third pooling branch. The sum of the values ​​of the four fused weights is 1.

[0073] See attached document Figure 6 S304, The dynamic pooling module receives the feature map of the deepest rust body and performs channel adjustment on the feature map of the deepest rust body using a two-dimensional convolutional computation layer with a kernel size of 1×1, generating a channel-adjusted feature map.

[0074] The dynamic pooling module sequentially inputs the channel-adjusted feature map into three serially connected max-pooling layers. The kernel size of the three max-pooling layers is 5×5, the stride parameter is 1, and the zero-padding parameter is 2.

[0075] The first max pooling layer receives the channel adjustment feature map and outputs the first pooling feature map; the second max pooling layer receives the first pooling feature map and outputs the second pooling feature map; and the third max pooling layer receives the second pooling feature map and outputs the third pooling feature map.

[0076] The equivalent receptive field sizes corresponding to the first pooling feature map, the second pooling feature map, and the third pooling feature map are 5×5, 9×9, and 13×13, respectively, and each pooling feature map has the same spatial size and number of channels as the channel adjustment feature map.

[0077] S305, the dynamic pooling module multiplies the fusion weights corresponding to the original feature branches element-wise with the channel adjustment feature map, multiplies the fusion weights corresponding to the first pooling branch element-wise with the first pooling feature map, multiplies the fusion weights corresponding to the second pooling branch element-wise with the second pooling feature map, and multiplies the fusion weights corresponding to the third pooling branch element-wise with the third pooling feature map.

[0078] The dynamic pooling module concatenates the weighted channel adjustment feature map, the first pooling feature map, the second pooling feature map, and the third pooling feature map along the channel dimension. The concatenation result is then sequentially input into a two-dimensional convolutional computation layer with a kernel size of 1×1, a batch normalization layer, and an activation function layer to complete the channel fusion between different receptive field features and generate a dynamic pooling feature map.

[0079] S306, the parameters of the fully connected layer, the two-dimensional convolutional computation layer, and the batch normalization parameters in the dynamic pooling module are jointly trained with the overall bridge steel structure corrosion detection network.

[0080] The network training module performs gradient backpropagation based on the overall loss, passes the error gradient corresponding to the overall loss to the dynamic pooling module, and updates the parameters of the multilayer perceptron and the two-dimensional convolutional computation layer according to the error gradient, so that the dynamic pooling module adjusts the fusion ratio of different receptive field features according to the spatial information entropy.

[0081] S401, the feature aggregation module receives four layers of feature maps output by the backbone network, where the deepest feature map is a dynamic pooling feature map, and the other three layers of feature maps are reconstructed feature maps output by the corresponding feature extraction stages.

[0082] Shallow reconstruction feature maps have high spatial resolution and are used to characterize the texture and edge contours of bridge steel structure corrosion; deep feature maps have a large receptive field and are used to characterize the category semantics and spatial distribution of bridge steel structure corrosion.

[0083] The feature aggregation module uses a two-dimensional convolutional computation layer with a kernel size of 1×1, a stride parameter of 1, and a zero-padding parameter of 0 to adjust the number of channels in the feature maps of the four layers, so that the feature maps participating in the splicing in adjacent layers meet the channel alignment requirements.

[0084] S402, the feature aggregation module performs feature fusion from deep to shallow layers, starting from the deepest feature map. The feature aggregation module uses bilinear interpolation to adjust the height and width of the current deep feature map to match the height and width of the adjacent shallow feature map. It then concatenates the upsampled deep feature map with the adjacent shallow feature map along the channel dimension and uses a two-dimensional convolutional layer with a kernel size of 3×3, a stride parameter of 1, and a zero-padding parameter of 1 to perform channel fusion on the concatenated result.

[0085] The feature aggregation module passes the feature to the shallowest layers layer by layer using the same processing method, generating uplink fused feature maps of different levels. Then, starting from the shallowest uplink fused feature map, the feature aggregation module performs feature fusion from shallow to deep layers. It uses a 2D convolutional layer with a kernel size of 3×3, a stride parameter of 2, and a zero-padding parameter of 1 to downsample the current uplink fused feature map. The downsampled result is then concatenated with the adjacent deep uplink fused feature map along the channel dimension. A 2D convolutional layer with a kernel size of 3×3, a stride parameter of 1, and a zero-padding parameter of 1 is used to perform channel fusion on the concatenated result, passing the feature to the deeper layers layer by layer using the same processing method.

[0086] S403, the feature aggregation module generates aggregated feature maps corresponding to multiple detection scales through feature fusion from deep to shallow layers and feature fusion from shallow to deep layers. Each aggregated feature map simultaneously contains shallow corrosion texture information, corrosion edge information, and deep corrosion semantic information.

[0087] The feature aggregation module transmits aggregated feature maps corresponding to different detection scales to the prediction module for corrosion bounding box localization and target confidence prediction. The parameters of the two-dimensional convolutional computation layer in the feature aggregation module are jointly trained with the overall bridge steel structure corrosion detection network and updated based on the backpropagation results of the overall loss.

[0088] S404, the prediction module receives aggregated feature maps, corrosion body feature maps, and corrosion edge gradient feature maps corresponding to different detection scales. The shared localization regression layer outputs the coordinates of candidate corrosion bounding boxes based on each aggregated feature map, and the target confidence layer outputs the target confidence of the corresponding candidate corrosion bounding boxes based on each aggregated feature map.

[0089] The prediction module selects a rust body feature map and a rust edge gradient feature map with a spatial resolution that match the size of the candidate rust bounding box, maps the coordinates of the candidate rust bounding box to the selected feature map, and performs region feature alignment within the spatial region covered by the candidate rust bounding box.

[0090] Region feature alignment employs bilinear interpolation to adjust candidate regions of different spatial sizes to a preset fixed spatial size, generating corrosion body region features and corrosion edge region features corresponding to each candidate corrosion bounding box. The corrosion body region features carry the texture and color information of the candidate corrosion region, while the corrosion edge region features carry the gradient distribution information of the transition from the candidate corrosion region to the normal background.

[0091] S405, the rust category identification branch receives the rust body region features corresponding to each candidate rust bounding box, performs global average pooling on the rust body region features, calculates the average value of each feature channel at all spatial locations within the candidate region, and generates a one-dimensional rust body feature vector.

[0092] The rust category identification branch inputs the one-dimensional rust body feature vector into the fully connected network layer. The number of output nodes of the fully connected network layer is equal to the preset number of rust categories, and each output node corresponds to surface rust, flaky rust, and pitting corrosion, respectively.

[0093] The corrosion category identification branch uses the Softmax function to normalize the node values ​​output by the fully connected network layer, generating a corrosion category probability distribution vector for the corresponding candidate corrosion bounding boxes, so that the sum of the probability values ​​of each corrosion category is 1.

[0094] S406, the corrosion evolution level assessment branch receives the rust edge region features corresponding to each candidate rust bounding box, performs global average pooling on the rust edge region features, calculates the average value of each feature channel at all spatial locations within the candidate region, and generates a one-dimensional rust edge feature vector.

[0095] The corrosion evolution level assessment branch inputs a one-dimensional rust edge feature vector into an independent fully connected network layer. The number of output nodes of the fully connected network layer is equal to the preset number of corrosion levels, and each output node corresponds to light corrosion, moderate corrosion, and severe corrosion, respectively.

[0096] The corrosion evolution level assessment branch uses the Softmax function to normalize the node values ​​output by the fully connected network layer, generating a corrosion level probability distribution vector for the corresponding candidate rust bounding box, so that the sum of the probability values ​​of each corrosion level is 1.

[0097] During the model detection phase, the prediction module deletes candidate corrosion bounding boxes whose target confidence is lower than the preset confidence threshold. It performs non-maximum suppression based on the intersection-union ratio between candidate corrosion bounding boxes, deletes candidate corrosion bounding boxes that overlap with high-confidence candidate corrosion bounding boxes, maps the coordinates of the retained candidate corrosion bounding boxes to the coordinate system of the original inspection image, and outputs the corrosion location, corrosion type, corrosion level, and target confidence corresponding to each candidate corrosion bounding box.

[0098] S501, the verification module defines an inter-class prior mutual exclusion matrix for logical verification based on the preset correspondence between rust categories and corrosion levels of bridge steel structures. The number of rows in the inter-class prior mutual exclusion matrix equals the total number of rust categories, and the number of columns equals the total number of corrosion levels. Each matrix element corresponds to a combination state of one rust category and one corrosion level.

[0099] When the corresponding combination conforms to the preset state correspondence, the matrix element takes the value of 1; when the corresponding combination does not conform to the preset state correspondence, the matrix element takes the value of 0.

[0100] S502, the verification module assigns values ​​to the inter-class prior mutual exclusion matrix according to the preset order of rust categories and corrosion levels. In a specific embodiment, the rows of the inter-class prior mutual exclusion matrix correspond to surface rust, flaky rust, and pitting corrosion in sequence, and the columns correspond to light corrosion, moderate corrosion, and severe corrosion in sequence.

[0101] The matrix element corresponding to surface rust and light corrosion has a value of 1, the matrix element corresponding to surface rust and moderate corrosion has a value of 1, and the matrix element corresponding to surface rust and heavy corrosion has a value of 0. The matrix element corresponding to flaky corrosion and light corrosion has a value of 1, the matrix element corresponding to flaky corrosion and moderate corrosion has a value of 1, and the matrix element corresponding to flaky corrosion and heavy corrosion has a value of 1. The matrix elements corresponding to pitting and mild corrosion have a value of 0, the matrix elements corresponding to pitting and moderate corrosion have a value of 1, and the matrix elements corresponding to pitting and severe corrosion have a value of 1. The inter-class prior mutual exclusion matrix consists of the above nine matrix elements and is stored in the detection network as a non-trainable static tensor.

[0102] In S503, during the model training phase, the backpropagation algorithm does not calculate the error gradient of the inter-class prior mutual exclusion matrix, and the matrix elements in the inter-class prior mutual exclusion matrix remain unchanged.

[0103] During the model detection phase, the verification module multiplies the probability of each corrosion category and the probability of each corrosion level corresponding to each retained candidate corrosion bounding box to generate a joint probability matrix. The joint probability matrix is ​​multiplied element-wise with the inter-class prior mutual exclusion matrix, so that the joint probability corresponding to the combination with a value of 0 in the inter-class prior mutual exclusion matrix is ​​set to 0; among the combinations with a value of 1 in the inter-class prior mutual exclusion matrix, the combination of rust category and corrosion level with the highest joint probability is selected as the final classification result of the candidate rust bounding box.

[0104] S504, the model training module reads the coordinates of the real rust bounding boxes, rust category labels and corrosion level labels labeled in the training samples, and receives the predicted bounding box coordinates output by the shared localization regression layer, the predicted target confidence output by the target confidence layer, the rust category probability distribution vector output by the rust category identification branch and the corrosion level probability distribution vector output by the corrosion evolution level assessment branch.

[0105] The model training module calculates the intersection-union ratio (IUR) between each predicted bounding box and each actual corrosion bounding box. For each actual corrosion bounding box, the predicted bounding box with the largest IUR and an IUR of not less than 0.5 is identified as a positive sample, and a matching relationship is established between the predicted bounding box and the actual corrosion bounding box; predicted bounding boxes that do not establish a matching relationship with actual corrosion bounding boxes are identified as negative samples.

[0106] The model training module calculates the intersection-union ratio regression loss for positive samples based on the coordinates of the matched real rust bounding boxes and the predicted bounding boxes, calculates the rust category cross-entropy loss based on the matched real rust category labels and the rust category probability distribution vector, calculates the corrosion level cross-entropy loss based on the matched real corrosion level labels and the corrosion level probability distribution vector, and sets the target existence flag for positive samples to 1.

[0107] The model training module sets the target presence label for negative samples to 0, and calculates the target confidence loss in the form of binary cross-entropy based on the target presence labels of positive and negative samples and the predicted target confidence. Negative samples are not included in the calculation of rust category cross-entropy loss, corrosion level cross-entropy loss, and bounding box regression loss.

[0108] S505, the model training module calculates the logical conflict penalty loss for positive samples that establish a matching relationship with the actual rust bounding boxes. The model training module multiplies the probability of each rust category and the probability of each corrosion level corresponding to the positive samples pairwise to generate a joint probability matrix.

[0109] The model training module traverses the joint probability matrix and the inter-class prior mutual exclusion matrix. When the matrix element at the corresponding position in the inter-class prior mutual exclusion matrix is ​​0, the joint probability value at the same position in the joint probability matrix is ​​extracted. When the element at the corresponding position in the inter-class prior mutual exclusion matrix is ​​1, the joint probability value at that position is not extracted. The model training module sums all extracted joint probability values ​​to generate a logical conflict penalty loss.

[0110] S506, the model training module performs weighted aggregation of rust category cross-entropy loss, corrosion level cross-entropy loss, target confidence loss, bounding box regression loss, and logical conflict penalty loss.

[0111] The model training module sets preset non-negative weights for the rust category cross-entropy loss, corrosion level cross-entropy loss, target confidence loss, bounding box regression loss, and logical conflict penalty loss. The global joint loss is generated by multiplying each loss by its corresponding non-negative weight and summing the results.

[0112] The weights of each loss remain unchanged during a single model training process and are determined using the validation set corresponding to the training samples.

[0113] S507, the model training module performs gradient backpropagation and network parameter updates based on the global joint loss. The model training module uses the chain rule to propagate the error gradient layer by layer from the corrosion category identification branch, corrosion evolution level assessment branch, target confidence layer and shared localization regression layer to the feature aggregation module, dynamic pooling module, feature decoupling module and feature extraction module.

[0114] The model training module uses a pre-defined optimizer to update the parameters of the 2D convolutional computation layer, multilayer perceptron, fully connected network layer, and batch normalized parameters based on the error gradient. The inter-class prior mutual exclusion matrix is ​​a non-trainable static tensor and does not participate in parameter updates.

[0115] After multiple rounds of training, when the global joint loss reaches the preset convergence condition on the validation set, the network parameter update is stopped and the network parameters are fixed, thus obtaining the bridge steel structure corrosion detection network.

[0116] After the model training is completed, the detection results are evaluated using precision, recall, mean precision, and mean mean precision. When calculating the evaluation index for each corrosion category, true positives, false positives, and false negatives are determined based on the intersection-union ratio between the predicted bounding box and the true corrosion bounding box, the predicted corrosion category, and the target confidence.

[0117] If the intersection-over-union ratio between the predicted bounding box and the actual rust bounding box is not less than the preset evaluation threshold, the predicted rust category is consistent with the actual rust category, and the target confidence is not lower than the preset confidence threshold, then the prediction result is determined as a true case. If the predicted bounding box does not establish a valid match with the corresponding category of the actual corrosion bounding box, or if the predicted corrosion category is incorrect, the prediction result is identified as a false positive; if the actual corrosion bounding box is not matched by the corresponding category of the valid predicted bounding box, the actual corrosion target is identified as a false negative.

[0118] Accuracy is calculated using the following formula: ; In the formula, Precision rate is used to characterize the proportion of correct detection results in the corresponding category of corrosion detection results output by the model; Indicates the number of true instances; Indicates the number of false positives; and The sum represents the total number of corresponding rust detection results output by the model.

[0119] Recall rate is calculated using the following formula: ; In the formula, Recall rate represents the proportion of real-world rust targets of the corresponding category that are correctly detected by the model. Indicates the number of true instances; Indicates the number of false negatives; and The sum represents the total number of real rust targets of the corresponding category in the image sample.

[0120] By adjusting the target confidence threshold, precision and recall were calculated for different target confidence thresholds, and a precision-recall curve was constructed with recall on the x-axis and precision on the y-axis. The mean precision was calculated using the following formula: ; In the formula, The average precision rate representing a single corrosion category; Indicates recall rate; Indicates accuracy; It represents the function of precision as a function of recall; the integral symbol represents the area enclosed by the precision-recall curve within the range of recall values; the lower limit of integration (0) and the upper limit of integration (1) indicate that the range of recall values ​​is from 0 to 1. Let R represent the integral element with recall rate R as the integral variable.

[0121] The mean mean accuracy is obtained by summing the average accuracy rates for all rust categories and taking the average value. The mean mean accuracy is calculated using the following formula: ; In the formula, This represents the mean of the average precision. This indicates the total number of rust categories included in the evaluation; The serial number indicating the type of corrosion. The value ranges from 1 to ; Subscript Indicates the first The average accuracy of each corrosion category; the summation symbol indicates the summation of values ​​from the first to the second. Sum the average precision rates of each corrosion category; denominator Used to calculate the arithmetic mean of the average accuracy across all rust categories.

[0122] Precision, recall, mean precision, and mean mean precision are used only to evaluate the trained bridge steel structure corrosion detection network. They are not part of the global joint loss and do not participate in gradient backpropagation or network parameter updates.

[0123] During the model detection phase, the parameters of the bridge steel structure corrosion detection network are fixed. The inspection images of the bridge steel structure to be detected are processed sequentially from S10 to S40, and the state combination verification of S503 is performed. Loss calculation, gradient backpropagation and network parameter update are not performed. The bounding box coordinates, target confidence, corrosion type and corrosion level of each corrosion area are output.

Claims

1. A dual-classification steel structure corrosion detection method integrating boundary-aware attention mechanism, characterized in that, Includes the following steps: The initial feature map is generated by acquiring image data of the bridge steel structure and then input into the feature extraction module to output a reconstructed feature map. The feature decoupling module generates a spatial attention response map and a spatial gradient magnitude matrix based on the reconstructed feature map, and separates the corrosion body feature map and the corrosion edge gradient feature map. Based on the spatial gradient magnitude matrix, spatial information entropy is calculated to generate fusion weights. The rust body feature map is pooled to generate different receptive field features. The rust body feature map and the different receptive field features are weighted and spliced ​​together to generate a dynamic pooled feature map. The dynamically pooled feature map and the reconstructed feature map are fused to generate an aggregated feature map and candidate rust bounding boxes are output. Based on the candidate rust bounding box, the corresponding rust body region features and rust edge region features are extracted from the rust body feature map and the rust edge gradient feature map, respectively, and the rust category probability distribution vector and corrosion level probability distribution vector are output respectively. The network weights are updated by calculating the joint penalty loss based on the preset inter-class prior mutual exclusion matrix, and the state combination verification is performed using the inter-class prior mutual exclusion matrix during the detection phase.

2. The dual-classification steel structure corrosion detection method based on the fusion of boundary-aware attention mechanism according to claim 1, characterized in that, The steps of acquiring bridge steel structure image data to generate an initial feature map, and inputting it into the feature extraction module to output a reconstructed feature map include: The bridge steel structure image data is used as the input image tensor. A spatial-to-depth rearrangement operation is performed on the input image tensor to generate a channel augmentation tensor. A two-dimensional spatial convolution operation is performed on the channel augmentation tensor to generate the initial feature map. Some convolutional units divide the channels of the initial feature map into a computational channel subset and a reserved channel subset. They perform two-dimensional spatial convolutional feature extraction on the computational channel subset to generate processed sub-feature maps, and perform identity mapping on the reserved channel subset. A cross-channel concatenation operation is performed on the processed sub-feature map and the reserved channel subset that has undergone identity mapping, and the reconstructed feature map is output.

3. The dual-classification steel structure corrosion detection method based on the fusion of boundary-aware attention mechanism according to claim 1, characterized in that, The steps for generating the spatial attention response map and the spatial gradient magnitude matrix include: The feature decoupling module uses the first branch to extract the dependencies between feature channels and the second branch to extract the feature responses within the local spatial receptive field. The output features of the two branches are then combined across dimensions and aggregated across channels to generate the single-channel spatial attention response map. Using the set difference operator, the partial derivative matrices of the spatial attention response map in the horizontal width direction and the vertical height direction are calculated respectively to generate the horizontal direction partial derivative matrix and the vertical direction partial derivative matrix; the sum of squares of the corresponding spatial position elements of the horizontal direction partial derivative matrix and the vertical direction partial derivative matrix are calculated and the square root operation is performed to generate the spatial gradient magnitude matrix.

4. The dual-classification steel structure corrosion detection method based on the fusion of boundary-aware attention mechanism according to claim 1, characterized in that, The step of separating the rust body feature map and the rust edge gradient feature map includes: The feature decoupling module normalizes the values ​​in the spatial gradient magnitude matrix to the range of 0 to 1, generating a normalized spatial gradient magnitude matrix. Perform a tensor broadcast operation along the channel dimension on the spatial attention response map and the normalized spatial gradient magnitude matrix; Using the extended spatial attention response map as a mask matrix, a masking operation is performed on the reconstructed feature map to extract the rust body feature map; Using the extended normalized spatial gradient magnitude matrix as a mask matrix, a masking operation is performed on the reconstructed feature map to extract the gradient feature map of the rust edge.

5. The dual-classification steel structure corrosion detection method based on the fusion of boundary-aware attention mechanism according to claim 4, characterized in that, The steps for calculating spatial information entropy and generating fusion weights based on the spatial gradient magnitude matrix include: The normalized spatial gradient magnitude matrix is ​​traversed according to spatial coordinates. The gradient magnitude value of each spatial location is mapped using an exponential function. The sum of the exponential mapping values ​​of all spatial locations is calculated. The exponential mapping value of each spatial location is divided by the sum of the exponential mapping values ​​to generate a spatial probability mapping map. By traversing the spatial probability mapping according to the spatial coordinates, the probability value of each spatial location is multiplied by the logarithm of that probability value to the base 2. The sum of all the product results is then taken as the opposite, and the spatial information entropy in scalar form is obtained.

6. The dual-classification steel structure corrosion detection method based on the fusion of boundary-aware attention mechanism according to claim 5, characterized in that, The step of performing pooling processing on the rust body feature map to generate different receptive field features, and then weightedly concatenating and aggregating the rust body feature map with the different receptive field features to generate a dynamic pooled feature map includes: The spatial information entropy is input into a multilayer perceptron to obtain four fusion weights corresponding to the original feature branch, the first pooling branch, the second pooling branch, and the third pooling branch; Channel adjustment is performed on the rust body feature map to generate a channel-adjusted feature map, which is then sequentially input into three serially connected max pooling layers. The first pooling feature map, the second pooling feature map, and the third pooling feature map are output as the different receptive field features, respectively. The four fusion weights are respectively multiplied element-wise by the rust body feature map, the first pooling feature map, the second pooling feature map, and the third pooling feature map, and then concatenated along the channel dimension to complete channel fusion and generate the dynamic pooling feature map.

7. The dual-classification steel structure corrosion detection method based on the fusion of boundary-aware attention mechanism according to claim 1, characterized in that, The step of extracting corresponding rust body region features and rust edge region features from the rust body feature map and the rust edge gradient feature map based on the candidate rust bounding box, and outputting the rust category probability distribution vector and corrosion level probability distribution vector respectively, includes: Based on the size of the candidate corrosion bounding box, select the corrosion body feature map and the corrosion edge gradient feature map with a spatial resolution that match it, and use bilinear interpolation to generate the corrosion body region feature and corrosion edge region feature corresponding to each candidate corrosion bounding box. Global average pooling is performed on the features of the corrosion body region to generate a one-dimensional corrosion body feature vector, which is then input into a fully connected network layer to generate the corrosion category probability distribution vector. Global average pooling is performed on the features of the rust edge region to generate a one-dimensional rust edge feature vector, which is then input into an independent fully connected network layer to generate the corrosion level probability distribution vector.

8. The dual-classification steel structure corrosion detection method based on the fusion of boundary-aware attention mechanism according to claim 1, characterized in that, The definition rules for the inter-class prior mutual exclusion matrix include: Based on the preset correspondence between rust categories and corrosion levels of bridge steel structures, a priori mutual exclusion matrix for logical verification is defined, wherein the number of rows in the priori mutual exclusion matrix is ​​equal to the total number of rust categories and the number of columns is equal to the total number of corrosion levels. Each matrix element corresponds to a combination of a rust type and a corrosion level. When the corresponding combination matches the preset state correspondence, the matrix element takes the value 1; when the corresponding combination does not match the preset state correspondence, the matrix element takes the value 0. The inter-class prior mutual exclusion matrix is ​​stored in the detection network as a non-trainable static tensor.

9. The dual-classification steel structure corrosion detection method based on the fusion of boundary-aware attention mechanism according to claim 1, characterized in that, The step of calculating the joint penalty loss and updating the network weights based on the preset inter-class prior mutual exclusion matrix includes: Obtain training samples with real rust bounding boxes, calculate the intersection-union ratio (IUR) between each candidate rust bounding box and each real rust bounding box, and determine the candidate rust bounding box with the largest IUR and an IUR of not less than 0.5 as a positive sample; The probability of each rust category in the probability distribution vector of the rust category corresponding to the positive sample is multiplied pairwise with the probability of each corrosion level in the probability distribution vector of the corrosion level to generate a joint probability matrix; Traverse the joint probability matrix and the inter-class prior mutual exclusion matrix. When the matrix element at the corresponding position in the inter-class prior mutual exclusion matrix is ​​0, extract the joint probability value at the same position in the joint probability matrix. The joint probability values ​​of all extracted values ​​are summed to generate a logical conflict penalty loss as the joint penalty loss.

10. The dual-classification steel structure corrosion detection method based on the fusion of boundary-aware attention mechanism according to claim 1, characterized in that, The steps of performing state combination verification using the inter-class prior mutual exclusion matrix in the detection phase include: For each candidate corrosion bounding box, the probability of each corrosion category in the corrosion category probability distribution vector corresponding to the candidate corrosion bounding box is multiplied pairwise with the probability of each corrosion level in the corrosion level probability distribution vector to generate a joint probability matrix; Multiply the joint probability matrix element by element with the inter-class prior mutual exclusion matrix, so that the joint probability corresponding to the combination with a value of 0 in the inter-class prior mutual exclusion matrix is ​​set to 0; Among the combinations with a value of 1 in the inter-class prior mutual exclusion matrix, the combination of rust category and corrosion level with the highest joint probability is selected as the final classification result of the candidate rust bounding box.

Images