A method and device for detecting cracks in corroded reinforced concrete based on YOLOV11

Abstract

The present invention provides a method and device for detecting cracks in corroded reinforced concrete based on YOLOV11, which belongs to the field of image processing technology. The method comprises: obtaining crack image data of corroded reinforced concrete components and annotating the image data; combining the annotated image data with labels into a data set; constructing a YOLOV11 cross-view comparison model, the model comprising a backbone network, a neck network, and a head network; training the YOLOV11 cross-view comparison model to obtain an optimal model; collecting video data of cracks in the corroded reinforced concrete components to be detected, inputting the data into the DVT_FVS module for image stabilization processing, and then transmitting the video into the YOLOV11 cross-view comparison model for crack detection, and outputting the location, size, and category information of the cracks. The present invention ensures the efficiency and accuracy of crack detection in corroded reinforced concrete.

Description

Technical Field

[0001] The present invention relates to a method and device for detecting cracks in corroded reinforced concrete based on YOLO V11, and belongs to the technical field of image processing. Background Art

[0002] Detecting cracks in corroded reinforced concrete is a core task in ensuring structural safety in the construction industry. It impacts the durability and reliability of a wide range of building infrastructure, including bridges, buildings, dams, and other types of infrastructure. In this field, deep learning technology is becoming a key tool for improving the accuracy and efficiency of crack detection, enabling precise location, quantitative analysis, and prediction of crack development trends in concrete. As the construction industry's requirements for structural health monitoring continue to increase, more cutting-edge technologies are being introduced to the field of modern corroded reinforced concrete crack detection, aiming to overcome the limitations of traditional detection methods and achieve more efficient and intelligent detection.

[0003] Compared to concrete cracking caused by stress, rust-expansion cracking is typically accompanied by reddish-brown rust, cracking along the reinforcement, and wide crack widths, making it easier to distinguish and identify. Currently, numerous traditional methods exist for detecting cracks in corroded reinforced concrete. Threshold segmentation methods use a fixed or adaptive threshold to divide image pixels into crack and non-crack regions. These methods are computationally simple and can quickly segment images with a relatively simple background and distinct crack features. However, the complex surface conditions of concrete in real-world projects are subject to numerous interfering factors. This method is extremely sensitive to the choice of threshold, and improper threshold settings can lead to false or missed crack detections. Edge detection algorithms identify crack edges by detecting sudden changes in grayscale values ​​within an image. While effective for cracks with clear edges, in corroded reinforced concrete, crack edges are often blurred and interfered with by corrosion products, such as rebar corrosion, resulting in discontinuous edge detection results, hindering complete crack identification. Semantic segmentation algorithms, used for crack detection in corroded reinforced concrete, attempt to classify each pixel in the image to accurately delineate crack regions from background regions. Although it can theoretically provide more detailed crack contour information, in practice, due to the diversity of concrete surface materials, changes in lighting conditions, and the irregularity of crack morphology, semantic segmentation models are prone to category confusion, misclassifying non-crack areas as cracks, or vice versa, resulting in limited detection accuracy. Region proposal-based object detection algorithms such as FasterR-CNN generate candidate regions that may contain cracks, and then classify and regress these regions to determine the location and size of the cracks. However, in the task of detecting cracks in corroded reinforced concrete, crack shapes are varied and often intertwined. The candidate regions generated by FasterR-CNN are difficult to accurately cover all cracks, and the computational cost is high, making them inefficient when processing large numbers of images.

[0004] Therefore, how to provide a YOLOV11-based method for detecting cracks in corroded reinforced concrete to improve the accuracy and stability of detection is an urgent problem that technicians in this field need to solve. Summary of the Invention

[0005] The purpose of the present invention is to provide a method and device for detecting cracks in corroded reinforced concrete based on YOLOV11, which solves the problems of poor accuracy and low efficiency in detecting cracks in corroded reinforced concrete under complex concrete surface conditions.

[0006] To achieve the above-mentioned purpose, the present invention is implemented through the following technical solutions:

[0007] A YOLOV11-based method for detecting cracks in corroded reinforced concrete, comprising:

[0008] Acquire crack image data of the corroded reinforced concrete component, and annotate the crack image data to generate corresponding labels;

[0009] Combine the annotated image data and labels into a dataset, which is divided into a training set, a validation set, and a test set;

[0010] Construct a YOLOV11 cross-view comparison model, which includes a backbone network, a neck network, and a head network; the backbone network sequentially includes a VMTV module (cross-view attention network), a C3K2 module, a CSWin_tiny module (cross-window sliding Transformer module), an SPPF (spatial pyramid pooling fast layer) module, and a C2PSA module; the VMTV module includes a view branch module, a cross-view interaction module, a two-stage dynamic fusion module, and an auxiliary module;

[0011] Use the training set to train the YOLOV11 cross-view comparison model, and use the validation set to tune the hyperparameters to obtain the best model;

[0012] The camera collects real-time video data of cracks in the corroded reinforced concrete components to be detected during movement, and inputs the data into the DVT_FVS module (cross-view dynamic video stabilization module) for image stabilization. The video is then passed to the YOLOV11 cross-view comparison model for crack detection, and the location, size, and category of the cracks are output.

[0013] Preferably, the C2PSA module output feature of the YOLOV11 cross-view comparison model backbone network The input neck network is sampled and then concatenated with the features output by the SPPF module of the backbone network. The concatenated features are processed by the C3K2 module to obtain the features. ,feature The features are obtained by concatenating the features output by the C3K2 module of the backbone network after upsampling. , the characteristics Output to the DVT_FVS module of the head network, the features It also processes the features through convolutional layers After splicing, input into C3K2 module to obtain features , the characteristics Output to the DVT_FVS module of the head network, the features It also processes the features through convolutional layers After splicing, input into C3K2 module to obtain features , the characteristics Output to the head network.

[0014] Preferably, the VMTV module comprises:

[0015] View branch module: It uses three parallel multi-view Transformer branches, each of which contains an image cell region embedding layer, a dynamic position encoding layer, four VIT hierarchical Transformer layers (visual Transformer hierarchical Transformer layers), an SE channel attention module, a feedforward network and a normalized residual block and a cross-scale feature pyramid connection layer;

[0016] Cross-view interaction module: includes the intra-view multi-scale self-attention layer, the cross-scale cross-attention layer, and the global semantic alignment layer;

[0017] Two-stage dynamic fusion module: including scale dimension fusion module and view dimension fusion module;

[0018] Auxiliary modules: including hierarchical dynamic routing, cross-view contrast learning module and view Figure 1 Consistency supervision module.

[0019] Preferably, in the view branch module:

[0020] The input image is divided into image cell regions (Patch) of corresponding sizes through the image cell region embedding layer and an embedding vector is generated;

[0021] The dynamic position encoding layer concatenates the learnable sinusoidal code of the embedding vector with the local position deviation matrix and linearly projects it to the feature dimension.

[0022] It then passes through four layers of VIT hierarchical Transformer layers, an SE channel attention module, a feedforward network, and a normalized residual block. The VIT hierarchical Transformer layer includes eight self-attention layers.

[0023] The cross-scale feature pyramid connection layer concatenates the three-scale features output by the normalized residual block to generate multi-scale features of a single view. Specifically, the multi-scale features of a single view are generated by upsampling the low-scale features and concatenating them with the current-scale features.

[0024] Preferably, in the cross-view interaction module:

[0025] The intra-view multi-scale self-attention layer consists of 4 VIT hierarchical Transformer layers, an SE channel attention module, a feedforward network, and a normalized residual block. Each VIT hierarchical Transformer layer contains 8 self-attention layers.

[0026] The cross-scale cross-attention layer includes a bilinear interpolation layer, a convolutional layer, a 12-head cross-attention layer, and a feedforward network;

[0027] The global semantic alignment layer includes a feature splicing layer and three global self-attention layers. The feature splicing layer flattens the multi-scale features of all views into a unified sequence, and then adds view identifier embedding and scale identifier embedding to mark the feature source. The global self-attention layer includes an 8-head self-attention layer, a feedforward network, and a dynamic position encoding layer.

[0028] Preferably, in the two-stage dynamic fusion module:

[0029] The scale dimension fusion module includes a feature splicing layer, a double hidden layer FC weight generator (double hidden layer fully connected weight generator) and a weighted fusion layer;

[0030] The view dimension fusion module serializes the scale fusion features of each view into tokens, and then uses a three-layer VIT-style self-attention layer to complete cross-view feature fusion. The VIT-style self-attention layer includes an 8-head self-attention layer, a feedforward network, layer normalization and residual connections.

[0031] Preferably, in the auxiliary module:

[0032] The hierarchical dynamic routing generates weights for local words through one fully connected layer and one fully connected layer for the global view;

[0033] The cross-view contrastive learning module uses a 2-layer fully connected projection head to calculate the NT-Xent contrastive loss (normalized temperature-scaled cross entropy loss);

[0034] The visual Figure 1 The consistency supervision module generates prediction values ​​through a 3-layer MLP prediction head (multi-layer perceptron prediction head) combined with MSE loss (mean square error loss) and contrast loss.

[0035] Preferably, the DVT_FVS module includes:

[0036] Preprocessing module: Use blind deconvolution motion blur kernel estimation and adaptive non-local means denoising, combined with scale normalization and data enhancement to complete input frame preprocessing;

[0037] Feature fusion module: The optical flow is embedded into a latent representation through a ResNet34_CBAM encoder (residual neural network 34 convolutional block attention module encoder), which is connected to the poses of the real and virtual cameras; the new virtual camera pose is inferred through the BiLSTM+MobileViT block (bidirectional long short-term memory network + mobile visual Transformer block), and a warped mesh is generated using the virtual camera pose; the spatiotemporal feature fusion is then achieved through the 3D convolution module and the Hourglass network (hourglass network) to predict the new virtual camera pose as a quaternion; the input frame is warped based on optical image stabilization and the virtual camera pose to generate a stable frame.

[0038] Preferably, the training process of the DVT_FVS module adopts: a dynamic weight GradNorm strategy (dynamic weight gradient norm strategy) to collaboratively optimize the smoothing loss of the BiLSTM stage (bidirectional long short-term memory network stage), the tracking loss of the Transformer stage, the ArcFace improved angle loss of the Hourglass stage, and the reconstruction loss of the entire process.

[0039] The advantages of the present invention are that it not only enhances the feature learning capability, but also improves the generalization capability of the network through reasonable model structure and hyperparameter tuning, thereby ensuring the efficiency and accuracy of crack detection in corroded reinforced concrete, and providing more reliable technical support for the structural safety assessment of construction projects. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.

[0041] Figure 1 A schematic flow chart of the method provided by the present invention;

[0042] Figure 2 A schematic diagram of the structural composition of the YOLOV11 cross-view comparison model provided by the present invention;

[0043] Figure 3 A schematic diagram of a view branch module provided by the present invention;

[0044] Figure 4 A schematic diagram of the cross-view interaction module provided by the present invention;

[0045] Figure 5Schematic diagram of the two-stage dynamic fusion module provided by the present invention;

[0046] Figure 6 A schematic diagram of the auxiliary module provided by the present invention;

[0047] Figure 7 This is an architectural diagram of the DVT_FVS module provided by the present invention. DETAILED DESCRIPTION

[0048] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0049] Example 1

[0050] like Figure 1 As shown, a YOLOV11-based method for detecting cracks in corroded reinforced concrete includes:

[0051] S1: Acquire crack image data of a corroded reinforced concrete component, annotate the image data, and generate corresponding labels.

[0052] Specifically, image data of cracks in corroded reinforced concrete are obtained, and corresponding labels are assigned to the image data of cracks in corroded reinforced concrete. The method of assigning corresponding labels is as follows: using a high-definition camera to collect data of corroded reinforced concrete under different background environmental conditions, and using a professional labeling tool Labelimg to mark the location and category of the cracks.

[0053] S2: Combine the annotated image data and labels into a dataset, which is divided into a training set, a validation set, and a test set.

[0054] Specifically, the image data of the cracks in the corroded reinforced concrete are combined with the labels to form a data set, and the data set is divided into training set samples, test set samples, and validation set samples in a ratio of 70%, 15%, and 15%. The data is preprocessed by normalization, rotation, flipping, and other operations to improve data quality.

[0055] S3: Construct a YOLOV11 cross-view comparison model, which includes a backbone network, a neck network and a head network; the backbone network includes a VMTV module, a C3K2 module, a CSWin_tiny module, an SPPF module and a C2PSA module in sequence; wherein the VMTV module includes a view branch module, a cross-view interaction module, a two-stage dynamic fusion module and an auxiliary module.

[0056] S4: Use the training set to train the YOLOV11 cross-view comparison model, and tune the hyperparameters on the validation set to obtain the best model.

[0057] Specifically, the training set samples are input into the YOLOV11 cross-view comparison model for training to obtain an initial model, and the validation set samples are used to tune the hyperparameters of the initial model to obtain the optimal model. The Adam optimization algorithm is used to tune hyperparameters such as the learning rate, batch size, and number of training rounds.

[0058] S5: The camera collects video data of cracks in the corroded reinforced concrete component to be detected in real time during movement, inputs the data into the DVT_FVS module for image stabilization, and then passes the video data into the YOLOV11 cross-view comparison model for crack detection, outputting the location, size, and category information of the cracks.

[0059] Specifically, the test set samples are input into the optimal model to obtain the detection results of corroded reinforced concrete cracks. The model outputs information about the location, size, and category of the cracks, and simultaneously calculates evaluation indicators such as accuracy and recall.

[0060] As a refinement of the above embodiment, Figure 2 As shown in FIG, the YOLOV11 cross-view comparison model includes a backbone network (Backbone), a neck network (Neck), and a head network (Head).

[0061] Output features of the C2PSA module of the YOLOV11 cross-view comparison model backbone network The input neck network is sampled and then concatenated with the features output by the SPPF module of the backbone network. The concatenated features are processed by the C3K2 module to obtain the features. ,feature The features are obtained by concatenating the features output by the C3K2 module of the backbone network after upsampling. , the characteristics Output to the DVT_FVS module of the head network, the features It also processes the features through convolutional layers After splicing, input into C3K2 module to obtain features , the characteristics Output to the DVT_FVS module of the head network, the features It also processes the features through convolutional layers After splicing, input into C3K2 module to obtain features , the characteristics Output to the DVT_FVS module of the head network.

[0062] As a refinement of the above embodiment, the VMTV module includes:

[0063] View branch module: It uses three parallel multi-view Transformer branches, each of which contains an image cell region embedding layer, a dynamic position encoding layer, a 4-layer VIT hierarchical Transformer layer, a SE channel attention module, a feedforward network and a normalized residual block and a cross-scale feature pyramid connection layer;

[0064] Cross-view interaction module: includes the intra-view multi-scale self-attention layer, the cross-scale cross-attention layer, and the global semantic alignment layer;

[0065] Two-stage dynamic fusion module: including scale dimension fusion module and view dimension fusion module;

[0066] Auxiliary modules: including hierarchical dynamic routing, cross-view contrast learning module and view Figure 1 Consistency supervision module.

[0067] Specifically, such as Figure 3 As shown, in the view branch module:

[0068] Three parallel multi-view Transformer branches are used in the view branch (with image cell area sizes of 16 / 8 / 4 respectively). Each branch consists of an image cell area embedding layer, a dynamic position encoding layer, a 4-layer VIT hierarchical Transformer layer, a SE channel attention module, a feedforward network, a normalized residual block and a cross-scale feature pyramid connection layer.

[0069] Specifically, the input image is first divided into image cell regions of corresponding sizes by the image cell region embedding layer and an embedding vector is generated; then, through the dynamic position encoding layer, the learnable sinusoidal code of the embedding vector is concatenated with the local position deviation matrix and linearly projected to the feature dimension; then it enters the 4-layer VIT hierarchical Transformer layer, each of which contains 8 self-attention layers in sequence, and then passes through the SE channel attention module (dimensionality reduction and dimension increase are achieved through 2 layers of fully connected layers), the feedforward network (FFN, 64→1024→64) and the normalized residual block; finally, through the cross-scale feature pyramid connection layer, the low-scale features are upsampled and concatenated with the current-scale features to generate multi-scale features of a single view.

[0070] Specifically, such as Figure 4 As shown, in the cross-view interaction module:

[0071] The intra-view multi-scale self-attention layer includes 4 layers of VIT hierarchical Transformer layers, SE channel attention modules, feedforward networks and normalized residual blocks. Each layer of the VIT hierarchical Transformer layer contains 8 self-attention layers. The SE channel attention module is used for global average pooling, dimensionality reduction and dimension increase fully connected layers and activation functions to perform channel weighting on the self-attention output. The feedforward network (FFN) consists of fully connected layers and GELU activation functions (Gaussian error linear unit activation functions). In addition, each layer also contains 2 layers of LayerNorm (layer normalization) and 2 layers of residual connections.

[0072] The cross-scale cross-attention layer includes a bilinear interpolation layer, a convolution layer, a 12-head cross-attention layer and a feedforward network; the cross-scale cross-attention layer unifies the number of channels by a bilinear interpolation layer and a 1×1 convolution layer; the 12-head cross-attention layer is used to split the input features into 12 heads to calculate the attention interaction across views, and the lightweight FFN network is combined to enhance the nonlinear expression, and layer normalization and residual connection are used to fuse the cross-attention output with the input features.

[0073] The global semantic alignment layer includes a feature splicing layer and three global self-attention layers. The feature splicing layer flattens the multi-scale features of all views into a unified sequence, and then adds a view identifier embedding Eview and a scale identifier embedding Escale to mark the feature source. Global semantic integration is performed through three global self-attention layers, each of which contains an eight-head self-attention layer (calculating all interactions between tokens), a feedforward network, and a dynamic position encoding layer.

[0074] Specifically, such as Figure 5 As shown, in the two-stage dynamic fusion module:

[0075] Scale-dimensional fusion includes a feature concatenation layer, a dual-hidden-layer FC weight generator, and a weighted fusion layer. The three-scale features are first concatenated, and channel weights are generated through a dual-hidden-layer FC network. After layer normalization, the multi-scale features are weightedly summed with channel attention. View-dimensional fusion serializes the scale-fused features of each view into word units, and completes the final feature fusion across views through three layers of VIT-style self-attention layers (each layer contains 8-head self-attention, FFN, layer normalization, and residual connections).

[0076] Specifically, such as Figure 6 As shown, in the auxiliary module:

[0077] Hierarchical dynamic routing generates weights for local words through a fully connected layer and a fully connected layer for the global view, guiding the model to focus on key areas; the cross-view contrastive learning module uses a 2-layer fully connected projection head to calculate the NT-Xent contrast loss to enhance the view invariance of features; Figure 1The consistency supervision module generates predictions using a three-layer MLP prediction head (with dropout), combining MSE and contrastive loss to ensure semantic consistency across views. The outputs of each component are ultimately fused with the main pipeline features to optimize model detection performance.

[0078] As a refinement of the above embodiment, Figure 7 As shown, the DVT_FVS module includes:

[0079] Preprocessing module: Use blind deconvolution motion blur kernel estimation and adaptive non-local means denoising, combined with scale normalization and data enhancement to complete input frame preprocessing;

[0080] Feature fusion module: The optical flow is embedded into a latent representation through a ResNet34_CBAM encoder, which is connected to the poses of the real and virtual cameras; the new virtual camera pose is inferred through the BiLSTM+MobileViT block, and a warped mesh is generated using the virtual pose; the spatiotemporal feature fusion is then implemented through a 3D convolution module and an hourglass network to predict the new virtual camera pose as a quaternion; the input frame is warped based on optical image stabilization (OIS) and the virtual camera pose to generate a stabilized frame.

[0081] Specifically, the DVT_FVS module uses sensor data and optical flow to stabilize videos through unsupervised learning. The preprocessing stage employs a motion blur kernel estimation module based on blind deconvolution and adaptive non-local means denoising, combined with scale normalization and data augmentation to preprocess the input frames. The core network comprises a lightweight ResNet34-CBAM encoder (containing four residual blocks, a CBAM attention module, and two fully connected layers) to extract multi-scale features. A four-layer bidirectional BiLSTM captures local temporal dependencies, and a three-layer MobileViT (including one fully connected layer) replaces the Transformer to handle long-range spatiotemporal correlations. The feature fusion stage incorporates optical flow features and a 3D convolution module, combined with an hourglass network to achieve multi-scale spatiotemporal feature fusion. The training process utilizes a dynamic weighted GradNorm strategy to collaboratively optimize the smoothing loss in the BiLSTM stage, the tracking loss in the Transformer stage, the ArcFace improved angle loss in the hourglass stage, and the overall reconstruction loss, ultimately generating stabilized video sequences with global motion compensation.

[0082] It should be noted that the VMTV module utilizes multi-view complementarity: each word unit incorporates the perspectives of different views (occluded areas in the left view are visible in the right view), reducing single-view information loss. Multi-scale hierarchical features: low-resolution features capture global semantics (such as the "rust crack" category), while high-resolution features preserve local details (such as the "crack" texture), forming a coarse-to-fine representation. Dynamic alignment: Cross-view cross-attention and a dynamic weight generator enable the model to adaptively focus on key regions (ROIs in object detection, edges in segmentation), which is more efficient than fixed fusion rules (such as averaging and splicing).

[0083] Example 2

[0084] This embodiment takes 640*640 image data as an example to describe in detail the implementation process of a method for detecting cracks in corroded reinforced concrete based on YOLOV11:

[0085] Patch partitioning and embedding (3 parallel branches);

[0086] Patch size (image cell area size) = 16 branches;

[0087] Number of Patches: =(640 / 16)^2=40*40=1600;

[0088] Embedding dimension: =64(initial channel);

[0089] Input image: ;

[0090] Image cell area embedding (Patch Embed):

[0091] ;

[0092] Patch size=8 branch: ;

[0093] ;

[0094] Patch size=4 branch: ;

[0095] ;

[0096] Learnable sinusoidal encoding:

[0097] ;

[0098] in, represents the sinusoidal position encoding function, and Indicates that the function calculates the position In dimension The positional encoding value on , Indicates that the embedding dimension of the model is 512, Represents a constant.

[0099] generate ,and Add: ;

[0100] Taking the branch with patch size = 16 as an example (feature dimension C = 64, sequence length L = 1600), the layer-by-layer calculation process of the four-layer Transformer layer is detailed (with the first layer as the core, and subsequent layers follow the same structure, with only the input and output differing):

[0101] enter , (batch dimension = 1, sequence length = 1600, channel dimension = 64), multi-head self-attention (8 heads, head dimension =64), total dimension D=8*64=512) Linear transformation generates Q / K / V:

[0102] ;

[0103] in, represents dynamic position encoding, Represents the weight matrix of the query, represents the weight matrix of the key, The weight matrix representing the value, represents the bias vector of the query, Bias vector representing the key A bias vector representing the values.

[0104] ;

[0105] Split into 8 heads (the batch dimension remains unchanged, and the head dimension is added):

[0106] ;

[0107] ;

[0108] ;

[0109] in, A multi-headed tensor representing the query, a multi-headed tensor representing the keys, a long tensor representing values, Indicates a query, Represents a value, Indicates the key, Indicates dimension reshaping, Indicates dimension transposition.

[0110] In the Transformer architecture, implementing multi-head attention requires resplitting and transforming the previously calculated Q (query tensor), K (key tensor), and V (value tensor). This involves splitting the tensor along a certain dimension to form multiple "heads," allowing different heads to capture relationships between input features from different perspectives.

[0111] Calculating attention similarity (including dynamic position deviation Δh∈R1600*1600):

[0112]

[0113] in, and Represents the query and key matrix after multi-head splitting, Attn represents the attention weight of the h-th head (Softmax normalization, dependency distribution between words), Indicates the The head's attention score matrix (dot product similarity, including scale and position bias), represents the scaling factor (the square root of the feature dimension of each head, stabilizing the gradient).

[0114] Weighted aggregate value vectors and concatenate the parts:

[0115] ;

[0116] ;

[0117] in, Indicates the The head's attention output tensor, represents the final output tensor of multi-head attention, Indicates the The head's attention weight matrix, Indicates the The value matrix of the head.

[0118] Residual connection and LayerNorm (after attention):

[0119] ;

[0120] in, Represents the final output tensor of the Self-Attention layer (SA), and LayerNorm represents the layer normalization function, which normalizes the last dimension of the tensor.

[0121] Feedforward network (FFN, hidden dimension = 1024), first layer fully connected (dimensionality increased):

[0122] ;

[0123] in, represents the first layer weight matrix of FFN, represents the bias vector of the first layer of FFN, Represents the output of the self-attention layer.

[0124] The second layer is fully connected (dimensionality reduction):

[0125] ;

[0126] ;

[0127] in, Represents the final output of the feedforward network Represents the output of the first layer of the feedforward network.

[0128] Residual connection and LayerNorm (after FFN):

[0129] ;

[0130] in, Represents the final output feature tensor of the Transformer encoder layer.

[0131] Table 1. Complete computational flow of a 4-layer Transformer

[0132]

[0133] Cross-scale attention (view Figure 1 16 scales → view Figure 2 8 scales, here we take view N=2 as an example)

[0134] Scale alignment: View Figure 2 8 scale features ;

[0135] Bilinear interpolation adjusts the resolution to 40×40 (consistent with the 16-scale): ;

[0136] 1x1 convolution reduces the number of channels to 64: ;

[0137] Cross attention calculation (12 heads, head dimension = 64):

[0138] Query ;

[0139] Key-value ;

[0140] Split into 12 heads ;

[0141] Attention Output ;

[0142] in, represents the normalized exponential function.

[0143] After splicing, CrossAttn (cross attention) ∈ R1*1600*768, and after FFN, the output is FCross (feature cross module) ∈ R1*1600*64;

[0144] Global semantic alignment feature splicing: ;

[0145] Dimension: 1*(1600+6400+25600)*256=1*33600*256;

[0146] 3-layer global self-attention (8 heads, head dimension = 64): Self-attention calculation for each layer:

[0147] ;

[0148] After the output is connected by residual and LayerNorm, the final Vglobal (global semantic alignment feature) ∈ R1*33600*256

[0149] Scale dimension fusion dynamic weight generator input splicing:

[0150] ;

[0151] Double hidden layer FC: ;

[0152] in, represents the intermediate layer activation function in the double hidden layer fully connected network (FC), Represents Softmax output;

[0153] Weighted fusion: ;

[0154] View dimension fusion (N=2 views) view feature sequence: ;

[0155] 3-layer self-attention: calculation per layer ;

[0156] Output ;

[0157] Cross-view contrastive learning projection head ;

[0158] Hierarchical dynamic routing local word weight ;

[0159] Global view weight ;

[0160] Each patch in each view contains global semantic information across views and scales. Low-resolution patches (patchsize=4) encode global context, and high-resolution patches (patchsize=16) retain local details (texture, edges). Cross-view interaction enables each word to fuse complementary information from other views (occluded areas from different perspectives).

[0161] Contrastive learning features Calculate the cross-view contrast loss (NT-Xent) to enhance the view invariance of features.

[0162] See Figure 1 Consistency prediction Generated by 3-layer prediction head for visual Figure 1 Consistency supervision loss (MSE+contrast loss) is used to ensure semantic consistency between different views.

[0163] Object detection (multi-view fusion): The spatial position information of the features is preserved and restored to multi-scale feature maps (1600→40×40, 6400→80×80, 25600→160×160).

[0164] Generate bounding box coordinates and category probabilities through a detection head (such as YOLOV11): .

[0165] Output: target location (x, y, w, h) and category confidence.

[0166] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art will be able to modify the technical solutions described in the aforementioned embodiments or substitute equivalents for some of the technical features. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A method for detecting cracks in corroded reinforced concrete based on YOLOV11, characterized in that: include: Acquire crack image data of the corroded reinforced concrete component, and annotate the crack image data to generate corresponding labels; Combine the annotated image data and labels into a dataset, which is divided into a training set, a validation set, and a test set; Construct a YOLOV11 cross-view comparison model, which includes a backbone network, a neck network, and a head network; the backbone network sequentially includes a VMTV module, a C3K2 module, a CSWin_tiny module, an SPPF module, and a C2PSA module; the VMTV module includes a view branching module, a cross-view interaction module, a two-stage dynamic fusion module, and an auxiliary module; Use the training set to train the YOLOV11 cross-view comparison model, and use the validation set to tune the hyperparameters to obtain the best model; The camera collects video data of cracks in the corroded reinforced concrete component to be detected in real time during movement, inputs it into the DVT_FVS module for image stabilization processing, and then passes the video into the YOLOV11 cross-view comparison model for crack detection, outputting the location, size, and category information of the cracks; Output features of the C2PSA module of the YOLOV11 cross-view comparison model backbone network The input neck network is sampled and then concatenated with the features output by the SPPF module of the backbone network. The concatenated features are processed by the C3K2 module to obtain the features. ,feature The features are obtained by concatenating the features output by the C3K2 module of the backbone network after upsampling. , the characteristics Output to the DVT_FVS module of the head network, the features It also processes the features through convolutional layers After splicing, input into C3K2 module to obtain features , the characteristics Output to the DVT_FVS module of the head network, the features It also processes the features through convolutional layers After splicing, input into C3K2 module to obtain features , the characteristics Output to the head network; The VMTV module includes: View branch module: It uses three parallel multi-view Transformer branches, each of which contains an image cell region embedding layer, a dynamic position encoding layer, a 4-layer VIT hierarchical Transformer layer, a SE channel attention module, a feedforward network and a normalized residual block and a cross-scale feature pyramid connection layer; Cross-view interaction module: includes the intra-view multi-scale self-attention layer, the cross-scale cross-attention layer, and the global semantic alignment layer; Two-stage dynamic fusion module: including scale dimension fusion module and view dimension fusion module; Auxiliary modules: include hierarchical dynamic routing, cross-view contrastive learning module and view consistency supervision module.

2. The method for detecting cracks in corroded reinforced concrete based on YOLOV11 according to claim 1, characterized in that: In the view branch module: The input image is divided into image cell regions of corresponding sizes through the image cell region embedding layer and an embedding vector is generated; The dynamic position encoding layer concatenates the learnable sinusoidal code of the embedding vector with the local position deviation matrix and linearly projects it to the feature dimension. It then passes through four layers of VIT hierarchical Transformer layers, an SE channel attention module, a feedforward network, and a normalized residual block. The VIT hierarchical Transformer layer includes eight self-attention layers. The cross-scale feature pyramid connection layer concatenates the three-scale features output by the normalized residual block to generate multi-scale features of a single view. Specifically, the multi-scale features of a single view are generated by upsampling the low-scale features and concatenating them with the current-scale features.

3. The method for detecting cracks in corroded reinforced concrete based on YOLOV11 according to claim 1, characterized in that: In the cross-view interaction module: The intra-view multi-scale self-attention layer consists of 4 VIT hierarchical Transformer layers, an SE channel attention module, a feedforward network, and a normalized residual block. Each VIT hierarchical Transformer layer contains 8 self-attention layers. The cross-scale cross-attention layer includes a bilinear interpolation layer, a convolutional layer, a 12-head cross-attention layer, and a feedforward network; The global semantic alignment layer includes a feature splicing layer and three global self-attention layers. The feature splicing layer flattens the multi-scale features of all views into a unified sequence, and then adds view identifier embedding and scale identifier embedding to mark the feature source. The global self-attention layer includes an 8-head self-attention layer, a feedforward network, and a dynamic position encoding layer.

4. The method for detecting cracks in corroded reinforced concrete based on YOLOV11 according to claim 1, characterized in that: In the two-stage dynamic fusion module: The scale dimension fusion module includes a feature splicing layer, a double hidden layer FC weight generator and a weighted fusion layer; The view dimension fusion module serializes the scale fusion features of each view into word units, and then uses a three-layer VIT-style self-attention layer to complete cross-view feature fusion. The VIT-style self-attention layer includes an 8-head self-attention layer, a feedforward network, layer normalization and residual connections.

5. The method for detecting cracks in corroded reinforced concrete based on YOLOV11 according to claim 1, characterized in that: In the auxiliary module: The hierarchical dynamic routing generates weights for local words through one fully connected layer and one fully connected layer for the global view; The cross-view contrastive learning module uses a 2-layer fully connected projection head to calculate the NT-Xent contrast loss; The view consistency supervision module generates prediction values ​​through a 3-layer MLP prediction head combined with MSE loss and contrast loss.

6. The method for detecting cracks in corroded reinforced concrete based on YOLO V11 according to claim 1, characterized in that: The DVT_FVS module contains: Preprocessing module: Use blind deconvolution motion blur kernel estimation and adaptive non-local means denoising, combined with scale normalization and data enhancement to complete input frame preprocessing; Feature fusion module: embeds the optical flow into a latent representation through a ResNet34_CBAM encoder, connects it with the poses of the real and virtual cameras; infers the new virtual camera pose through a BiLSTM+MobileViT block, and generates a warped mesh using the virtual camera pose; Then, the spatiotemporal feature fusion is achieved through the 3D convolution module and the hourglass network to predict the new virtual camera pose as a quaternion; the input frame is distorted based on optical image stabilization and the virtual camera pose to generate a stable frame.

7. The method for detecting cracks in corroded reinforced concrete based on YOLOV11 according to claim 6, characterized in that: The training process of the DVT_FVS module adopts: a dynamic weight GradNorm strategy is used to collaboratively optimize the smoothing loss of the BiLSTM stage, the tracking loss of the Transformer stage, the ArcFace improved angle loss of the Hourglass stage, and the reconstruction loss of the entire process.

8. A YOLO V11-based device for detecting cracks in corroded reinforced concrete, comprising a processor and a memory storing program instructions, characterized in that: The processor is configured to execute the corroded reinforced concrete crack detection method based on YOLOV11 as described in any one of claims 1-7 when running the program instructions.

Images