A frequency-decoupled multi-weather scenario crack identification method and system

Abstract

The present invention provides a frequency-decoupled crack identification method and system for multiple weather scenarios. The system denoises a concrete structure image to be identified to obtain a denoised image. The denoised image is downsampled and processed using SharedSTEM to obtain a first feature. The first feature is input into several frequency decoupling modules for processing and classification to identify whether cracks exist in the concrete structure image. The first frequency decoupling module divides the first feature into grids, clusters them, and performs position encoding. High-frequency and low-frequency features of the clustered grids with position encoding information are extracted and fused based on their correlation. The fused feature is then fused with the first feature and input into subsequent frequency decoupling modules as the new first feature. This ensures high-precision crack segmentation under complex weather conditions, providing precise protection for infrastructure maintenance and public transportation safety.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and in particular to a frequency-decoupled multi-weather scenario crack recognition method and system. Background Art

[0002] Surface health monitoring of engineering structures is crucial for infrastructure management. Accurate crack detection and timely repair directly impact the safety and quality of concrete structures. However, crack detection presents significant challenges due to its complex topology, narrow and long directionality, linear subcrack multiplication, and low edge contrast.

[0003] During the feature extraction process, natural images can be decomposed into low spatial frequency components that describe smooth changes and high spatial frequency components that represent rapid changes. In the semantic segmentation task, it is considered to directly decouple these two types of information through frequency filters to extract detailed semantics, and combine prototype learning to retain the semantic information of high-resolution images. Currently, traditional crack detection methods rely on high-quality image datasets, but in practical applications, crack detection often faces many challenges, such as insufficient lighting, low contrast between cracks and background, and image quality degradation caused by bad weather. In addition, noise factors such as engineering surface pollution, light and shadow occlusion, and complex background also significantly affect the detection performance of drones, especially in low-light scenes such as tunnels, subways, and night inspections, where the performance is far inferior to detection under normal lighting conditions.

[0004] To address these issues, image enhancement is often used to improve brightness and contrast before inputting them into a segmentation model. However, poor imaging quality in inclement weather still severely limits crack detection, especially in drone inspections, where improving detection performance with low-quality images remains a challenge.

[0005] In drone-based road health inspections or daily traffic scenarios, images are often affected by multiple weather conditions simultaneously. Common scenarios include sleet, rain in fog, and snow in heavy fog. The random superposition of these multiple degradation factors further deteriorates image quality, greatly challenging the effectiveness of image restoration and processing. Current research focuses on addressing single degradation factors, such as haze, rain, and snow. Although some networks are designed to handle combinations of severe weather conditions, such as rain and snow or rain and fog, they are typically limited to fixed combinations. In contrast, real-world scenarios often involve the random superposition of multiple degradation factors, where two or even three severe weather conditions often lead to severe image degradation. Current methods are relatively ineffective in addressing randomly superimposed crack identification. Summary of the Invention

[0006] The present invention proposes a frequency-decoupled multi-weather scenario crack identification method and system to solve the technical problem that the existing identification methods have poor recognition effect when facing the random superposition of multiple degradation factors.

[0007] To solve the above technical problems, the present invention provides a frequency-decoupled multi-weather scenario crack identification method, comprising the following steps:

[0008] Step S1: De-noising the concrete engineering structure image to be identified to obtain a de-noised image;

[0009] Step S2: downsampling the denoised image and processing it using SharedSTEM to obtain a first feature;

[0010] Step S3: inputting the first feature into a plurality of frequency decoupling modules for processing and then performing classification to identify whether there are cracks in the concrete engineering structure image;

[0011] The first frequency decoupling module divides the first feature into grids, clusters the grids, and performs position encoding; extracts high-frequency features and low-frequency features of the clustered grids with position encoding information, and fuses them according to the correlation between the low-frequency features and the high-frequency features; and fuses the fused features with the first feature and inputs them into the subsequent frequency decoupling module as a new first feature.

[0012] Preferably, in step S1, the concrete engineering structure image to be identified is denoised by a multi-weather integrated restoration module;

[0013] During the training of the multi-weather integrated restoration module, the noise-free crack image is converted into a low-quality crack image through the inverse generative adversarial network AdverseGAN. After the low-quality crack image is denoised through several AIIR modules, the denoising effect is judged for training optimization;

[0014] The AIIR module includes several multi-scale spatial attention modules (MSAs) and a feature pyramid (FFN). The results of different heads are integrated using several MSAs, and after 1×1 convolution stacking, the results are input into the FFN to integrate local and global information to obtain the restored crack image.

[0015] Preferably, the expression of the AIIR module is:

[0016] ;

[0017] ;

[0018] Where, Indicates the preliminary denoising result obtained by concatenating the results of several MSA modules; Cat Represents a splicing operation; Indicates the i Results of the MSA modules; Indicates the iCharacteristics of the group; Represents feature pyramid operation; Represents a 1×1 convolution operation.

[0019] Preferably, each two AIIR modules are processed through a feature mapping network to decouple the image style and the crack style, thereby increasing the distance between the two features.

[0020] Preferably, the low-frequency features are extracted by the following method: dividing the cluster grid into multiple groups, performing adaptive pooling of different sizes on each group, and upsampling to the original size, splicing the upsampling results and extracting the low-frequency features after ReLU activation.

[0021] Preferably, the high-frequency features are extracted by the following method: dividing the cluster grid into multiple groups, simulating the cutoff frequencies of different high-pass filters for each group by using convolution layers with different kernels to form a channel frequency domain information list, and connecting the channel frequency domain information list in the channel dimension through tensor splicing operations to obtain high-frequency features.

[0022] Preferably, the expression for fusion is based on the correlation between low-frequency features and high-frequency features. for:

[0023] ;

[0024] Where, q represents the query vector; Represents high-frequency features; Represents low-frequency features; Frequency components i and frequency components j The correlation coefficient between .

[0025] Preferably, the features after the fusion of high-frequency features and low-frequency features are input into the recurrent neural network DA-FFN based on the dual-stage attention mechanism for processing before subsequent operations are performed.

[0026] Preferably, the method uses a loss function L To train:

[0027] ;

[0028] Where, is the i-th element of the actual label vector; is the i-th element of the probability distribution predicted by the model; N is the number of samples.

[0029] The present invention also provides a frequency-decoupled multi-weather scenario crack identification system, comprising: one or more processors and memories, and one or more programs, wherein the one or more programs are stored in the memories and are configured to be executed by the one or more processors, and the one or more programs include methods for executing the above-mentioned method.

[0030] The beneficial effects of this invention include at least one: It combines diverse weather restoration with spatial frequency guidance, first performing denoising using diverse weather, followed by segmentation processing using a stacked frequency decoupling module. This design effectively decouples high-frequency and low-frequency semantic information, improving the edge accuracy of binary crack segmentation. This addresses the poor recognition accuracy of traditional models in low-resource drone-based inspection scenarios due to limited computing resources. This ensures high-precision crack segmentation in complex weather conditions, providing precise protection for infrastructure maintenance and public transportation safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 Schematic diagram of the model structure of an embodiment of the present invention;

[0032] Figure 2 This is a schematic diagram of a frequency decoupling module according to an embodiment of the present invention;

[0033] Figure 3 A schematic diagram showing the comparison of the method of an embodiment of the present invention with other model results on multiple data sets;

[0034] Figure 4 Schematic diagram of comparison of result data of an embodiment of the present invention with other model results on multiple data sets. DETAILED DESCRIPTION

[0035] The following is a clear and complete description of the technical solutions in the embodiments of the present invention, in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts are within the scope of protection of the present invention.

[0036] Example 1

[0037] The application of lightweight equipment such as drones in road inspection is crucial for structural health monitoring of transportation engineering projects. In crack detection, adverse weather conditions often lead to image quality degradation.

[0038] Therefore, if Figure 1As shown, an embodiment of the present invention provides a frequency decoupled crack recognition method for multiple weather scenarios. First, the collected concrete structure image to be identified is denoised, and then the segmented part is constructed by stacking frequency decoupling modules. The spatial domain frequency is used to directly separate and extract the crack edge and main body information from the original image, ensuring the consistency of the internal crack object while enhancing the edge area supervision. Specifically, the following steps are included:

[0039] Step S1: De-noising the concrete engineering structure image to be identified to obtain a de-noised image.

[0040] Step S2: After downsampling the denoised image, the first feature is obtained by processing it through SharedSTEM.

[0041] Specifically, the denoised concrete structure image is first downsampled multiple times to reduce the feature map to 1 / 8 of its original size. SharedSTEM then processes the downsampled features using two convolutional layers. Each denoised image block is then converted into an embedded feature, and the model initializes a grid point T as the prototype of the crack image, as follows:

[0042] ;

[0043] n represents the 3×3 neighborhood of the cluster center, Represents the local feature vector or data point considered when each cluster center is initialized, for Each point in the grid acts as a local cluster center. Each cluster center contains weighted, initialized semantic information in its corresponding region but lacks a specific semantic description. At this point, the model's understanding of the semantic information in the image is relatively simple and ambiguous. Therefore, we perform multi-layer frequency decoupling and clustering updates on these cluster centers.

[0044] The relative position of the grid T is encoded through the convolutional layer, and the position information is added to the features to retain the spatial information of the image. Our goal is to update each cluster center s in the grid T instead of directly updating the initial features. This reduces the number of learnable parameters compared to the parameterized method of updating the entire feature map.

[0045] In positional encoding, the input grid T is mapped to query Q, key K, and value V respectively through three independent linear transformations, usually fully connected layers or 1x1 convolutions. The query Q, key K, and value V are taken as the first features and passed to subsequent steps.

[0046] Step S3: After the first feature is input into a plurality of frequency decoupling modules for processing, classification is performed to identify whether there are cracks in the concrete engineering structure image.

[0047] The first frequency decoupling module divides the first feature into grids, clusters the grids, and performs position encoding; extracts high-frequency features and low-frequency features of the clustered grids with position encoding information, and fuses them according to the correlation between the low-frequency features and the high-frequency features; and fuses the fused features with the first feature and inputs them into the subsequent frequency decoupling module as a new first feature.

[0048] In this embodiment, the correlation between low-frequency features and high-frequency features is extracted through a frequency domain correlation module; the low-frequency features are extracted through a variable low-frequency domain tuning filter; and the high-frequency features are extracted through a variable high-frequency domain tuning filter.

[0049] Specifically, if Figure 2 As shown in Figure 2, a variable low-frequency domain tuning filter strategy is proposed based on the characteristics that the low-frequency components in the image occupy most of the energy and carry the main semantic information. Its expression is:

[0050] ;

[0051] I represents the interpolation operation, AP () represents adaptive average pooling, Represents the P groups after V is grouped, and low-pass filtering is achieved through adaptive average pooling and grouping: the input grid 𝑇 is divided into multiple groups, and different kernels and strides are controlled to generate dynamic low-pass filters. Adaptive pooling of different sizes is performed respectively, and then upsampled to the original size. Finally, the upsampled results are spliced and activated by ReLU to extract low-frequency information. The cutoff frequency is controlled by adjusting the kernel size and stride.

[0052] The variable high-frequency domain tunable filter module uses the query Q and high-frequency characteristics to perform a matrix element-by-element product operation to modulate all frequency components. Its expression is:

[0053] ;

[0054] in DW () is a deep convolutional layer. First, the input value V is grouped into Q groups and the obtained For each group, we simulate the cutoff frequencies of different high-pass filters by using convolution layers with different kernels to form a list of channel frequency domain information. Subsequently, these frequency domain information are connected into a larger tensor in the channel dimension through tensor concatenation operation , thus globally considering the frequency domain structure of the input value V. The shape rearrangement operation merges The height and width dimensions are adjusted so that Becomes a tensor suitable for weighted addition operations.

[0055] Since different frequency components are distributed in the grid T, this embodiment proposes a frequency domain correlation module to calculate the correlation degree between two different frequency components.

[0056] This module first introduces a fixed-size frequency domain correlation module , used to represent the strength of different frequency components. The original input is decomposed into multi-core signal components through a linear layer, thereby refining the understanding of the spatiotemporal changes of image features.

[0057] The module calculates the key K and value V of the frequency component through the linear layer and normalizes the key K on the frequency component through the Softmax operation. Querying this module can represent the correspondence between different frequency components, and then select important frequency components by querying the frequency domain association module.

[0058] Finally, the outputs of the two variable frequency domain tunable filters are combined to obtain the variable frequency filter output, which is as follows:

[0059] ;

[0060] In this embodiment, the output of the frequency domain correlation module It is regarded as a function transfer so that each component of the grid integrates a similarity coefficient, which is used for the adaptive fusion of variable low-frequency domain tuning filters and variable high-frequency domain tuning filters.

[0061] Query Q and high-frequency information through element-by-element product operation The two methods interact with each other to fine-tune the feature strength at each location and channel. Furthermore, the computationally expensive concatenation method (Concat) is replaced with a low-cost addition method (ADD) to fuse the tuned low- and high-frequency information while maintaining the same number of channels. The resulting features are then fed into the DA-FFN. This mechanism optimizes the correlation between the query and high-frequency features, suppressing high-frequency noise within the object.

[0062] Finally, the first feature processed by SharedSTEM is restored to the original spatial feature F through bilinear interpolation. Finally, a series of linear projection layers are used to fuse the high-frequency and low-frequency fused features with the original spatial features. The updated feature Fi′ contains rich pixel semantic information from the original feature F as well as the prototype semantics collected by the cluster center T(s). This method is beneficial for training convergence and gradient flow.

[0063] By stacking four such frequency decoupling modules and aggregating semantics around cluster centers, the crack segmentation network achieves accurate crack segmentation. The pixel features and updated grid features are input to the classification layer, and the output is used for pixel-level classification.

[0064] Example 2

[0065] Based on Example 1, this embodiment provides a multi-weather integrated restoration module for denoising the concrete engineering structure image to be identified.

[0066] Specifically, given a clean image, an AdverseGAN is first used to generate a corresponding degraded image, thereby constructing a comparison dataset of clean and degraded images. The generated degraded image is then fed into a restoration network to generate a consistent restored image independent of the degradation. Finally, the restored image is fed into a discriminator to determine the type of degradation the restored image underwent prior to restoration. The entire process is designed to be concise and clear, with the testing phase relying solely on the restoration network, eliminating the need for additional inference steps or model modifications, thus ensuring both efficiency and practicality.

[0067] This embodiment uses an inverse generative adversarial network (AdverseGAN) to efficiently generate mixed adverse scenarios to obtain training data for the first stage. Subsequently, a diverse multi-weather integrated restoration module and discriminative learning methods are employed to train the integrated restoration network. Specifically, given a degraded image, a 3×3 convolution is first used to generate shallow feature embeddings. These shallow features are then processed through the four-layer attention-integrated iterative restoration module (AIIR) provided in this embodiment. After four layers of AIIR processing, deep features are obtained.

[0068] In this embodiment, each AIIR consists of several multi-scale spatial attention modules (MSAs) and a feature pyramid (FFN). The MSA extracts long- and short-range contextual information by introducing a self-attention branch in parallel with the convolution branch. The attention branch primarily enhances the model's focus on weather style, while using 3×3 convolution and GeLU activation functions to extract the main image features. The AIIR integrates the results of different heads using multiple MSAs and performs a 1×1 convolution stack. Locally relevant information is key to image restoration. The FFN integrates local and global information to obtain the restored crack image. The formula is as follows:

[0069] ;

[0070] ;

[0071] Where, It represents the preliminary denoising result obtained by concatenating the results of several MSA modules; then the number of channels is restored through FFN and 1×1 convolution to obtain the final denoised image; Cat Represents a splicing operation; Indicates the i Results of the MSA modules; Indicates thei Characteristics of the group; Represents feature pyramid operation; Represents a 1×1 convolution operation.

[0072] In this embodiment, the features extracted by each AIIR module are processed by a feature mapping network. The feature mapping network learns to project the encoded features onto corresponding clean embeddings, locates the reconstruction vector in the codebook, and provides additional auxiliary visual atoms for the restoration process. This decouples the image style and the crack style, thereby widening the gap between the two features.

[0073] Example 3

[0074] This example effectively validates the method of Example 1 on the Crack500, Crack200, and DeepCrack datasets. CRACK500 contains 3,363 crack images at a resolution of 640×360; CRACK200 contains 4,507 crack images at a resolution of 800×600; and DEEPCRACK contains 1,611 crack images at a resolution of 544×384. These datasets demonstrate diverse crack morphologies and various challenging factors.

[0075] like Figure 3 As shown, GhostNetV2 is the ghost segmentation network V2, STDC represents the short-term dense connection network, SeaFormer is the squeeze-enhanced axial transformer, BiSeNetV2 is the bilateral segmentation network V2, SegFormer represents a simple and efficient unified Transformers semantic segmentation framework, AFFormer is the adaptive frequency transformer, TopFormer is the top forming transformer, and PP-LITESEG represents a novel lightweight model based on the Baidu PaddlePaddle platform for real-time semantic segmentation tasks. It can be seen that the method of this embodiment, combined with the discriminant learning strategy, can effectively distinguish various degradation types and significantly improve the model's ability to extract features from low-quality images. In addition, the present invention strengthens the correspondence between frequency domain components and spatial domain pixels, and applies different processing to different frequency components so that the model not only distinguishes between cracks and non-crack elements in the edge area, but also further improves the segmentation effect of the overall crack.

[0076] like Figure 4As shown in the figure, HardNet represents a harmonic densely connected network, BiSeNetV2 represents a bilateral segmentation network V2, STDC represents a short-term densely connected network, SegFormer represents a simple and efficient unified Transformer semantic segmentation framework, GhostNetV2 represents a ghost segmentation network V2, PP-LITESEG represents a novel lightweight model based on the Baidu PaddlePaddle platform for real-time semantic segmentation tasks, DDRNet represents a deep dual-resolution network for real-time accurate semantic segmentation, TopFormer represents a top-forming transformer, SeaFormer represents a squeeze-enhanced axial transformer, and AFFormer represents an adaptive frequency transformer. As can be seen, our model outperforms the next-best SegForemr on the Crack500 dataset, reducing computational cost by 68.0% and parameter count by 28.9%, while increasing mean Interference Over Union (MIOU) by 0.16%. On the DeepCrack dataset, our model achieves the highest mean Interference Over Union (MIOU) and the second-best precision and recall. Compared to the highly accurate TopFormer, our model uses 37.1% fewer parameters. Although our model is not optimized in terms of parameters and FLOPs, results show that, for example, GhostNetV2 and RHACrackNet, which actively prioritize low computational cost and number of parameters but suffer significant losses as a result, our model achieves the best segmentation performance with minimal computational cost.

[0077] The technical features of the above embodiments may be combined in any manner. To simplify the description, not all possible combinations of the technical features in the above embodiments are described. Only preferred embodiments of the present invention are presented. While the description is relatively specific and detailed, it should not be construed as limiting the scope of the present invention. As long as there are no conflicts in the combination of these technical features, they should be considered to be within the scope of this specification.

[0078] It should be noted that, for those skilled in the art, various modifications and improvements can be made without departing from the scope of the present invention, and these modifications and improvements fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.

Claims

1. A frequency-decoupled multi-weather scenario crack identification method, characterized by: The following steps are involved: Step S1: De-noising the concrete engineering structure image to be identified to obtain a de-noised image; Step S2: downsampling the denoised image and processing it using SharedSTEM to obtain a first feature; Step S3: inputting the first feature into a plurality of frequency decoupling modules for processing and then performing classification to identify whether there are cracks in the concrete engineering structure image; The first frequency decoupling module divides the first feature into grids, clusters the grids, and performs position coding; extracts high-frequency features and low-frequency features of the clustered grids with position coding information, and fuses them according to the correlation between the low-frequency features and the high-frequency features; and fuses the fused features with the first feature as a new first feature and inputs them into a subsequent frequency decoupling module; In step S1, the concrete engineering structure image to be identified is denoised by using a multi-weather integrated restoration module; During the training of the multi-weather integrated restoration module, the noise-free crack image is converted into a low-quality crack image through the inverse generative adversarial network AdverseGAN. After the low-quality crack image is denoised through several AIIR modules, the denoising effect is judged for training optimization; The AIIR module includes several multi-scale spatial attention modules (MSAs) and a feature pyramid (FFN). The results of different heads are integrated using several MSAs, and after 1×1 convolution stacking, the results are input into the FFN to integrate local and global information to obtain the restored crack image. The expression of the AIIR module is: ； ； Where, Indicates the preliminary denoising result obtained by concatenating the results of several MSA modules; Cat Represents a splicing operation; Indicates the i Results of the MSA modules; Indicates the i Characteristics of the group; Represents feature pyramid operation; Represents a 1×1 convolution operation; Each two AIIR modules are processed through a feature mapping network to decouple the image style and the crack style, thereby widening the distance between the two features.

2. The frequency-decoupled multi-weather scenario crack identification method according to claim 1 is characterized by: The low-frequency features are extracted by the following method: dividing the cluster grid into multiple groups, performing adaptive pooling of different sizes on each group, upsampling to the original size, concatenating the upsampling results and then performing ReLU activation to extract the low-frequency features.

3. The frequency-decoupled multi-weather scenario crack identification method according to claim 2 is characterized by: The high-frequency features are extracted by the following method: the clustering grid is divided into multiple groups, and for each group, a convolution layer with different kernels is used to simulate the cutoff frequencies of different high-pass filters to form a channel frequency domain information list, and the channel frequency domain information list is connected in the channel dimension through a tensor splicing operation to obtain the high-frequency features.

4. The frequency-decoupled multi-weather scenario crack identification method according to claim 3 is characterized by: The expression for fusion according to the correlation between low-frequency features and high-frequency features for: ； Where, q represents the query vector; Represents high-frequency features; Represents low-frequency features; Frequency components i and frequency components j The correlation coefficient between .

5. The frequency-decoupled multi-weather scenario crack identification method according to claim 1 is characterized in that: The fusion of high-frequency features and low-frequency features is input into the recurrent neural network DA-FFN based on the dual-stage attention mechanism for processing before subsequent operations.

6. The frequency-decoupled multi-weather scenario crack identification method according to claim 1, characterized in that: The method uses the loss function L To train: ； Where, is the i-th element of the actual label vector; is the i-th element of the probability distribution predicted by the model; N is the number of samples.

7. A frequency decoupled multi-weather scenario crack identification system, characterized by: include: One or more processors and memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include instructions for executing the method according to any one of claims 1 to 6.

Images