Mountain area tunnel crack intelligent identification and detection method based on deep learning

Abstract

The invention provides a mountainous area tunnel crack intelligent identification and detection method based on deep learning, and relates to the technical field of deep learning, and the method comprises the steps: collecting multispectral image data, and carrying out the adaptive preprocessing to obtain an enhanced feature map; a double-flow network is combined with a space-channel cascade attention module to extract fusion features; constructing multi-scale feature representation through a feature association graph network; training the network by adopting a joint optimization target; and morphological processing and connectivity analysis are carried out to realize accurate identification and classification of the crack. According to the invention, the detection precision and the anti-interference capability of the tunnel crack in the complex environment are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to deep learning technology, and in particular to a deep learning-based mountain tunnel crack intelligent identification and detection method. BACKGROUND

[0002] Cracks are one of the common diseases of tunnels, which can cause structural strength reduction, water seepage, and safety hazards such as collapse. Traditional tunnel crack detection mainly relies on manual inspection, which is not only labor-intensive and inefficient, but also has a large subjective influence on detection accuracy.

[0003] With the development of computer vision and deep learning technology, automatic crack detection methods based on image processing have been widely applied. Existing technologies mainly include traditional image processing-based methods such as edge detection, threshold segmentation, etc., and deep learning-based methods such as convolutional neural networks, fully convolutional networks, etc. These methods can achieve good crack identification results under ideal conditions.

[0004] However, the mountain tunnel environment is complex, and the existing technology still has obvious deficiencies in practical application: first, the lighting conditions in mountain tunnels are complex and changeable, and a single visible light imaging mode cannot cope with various complex environments, resulting in unstable detection accuracy; second, existing crack detection algorithms mainly focus on spatial features of images, ignoring the fusion of frequency domain information and multi-scale features, and cannot effectively distinguish cracks from background textures; finally, traditional methods lack the ability to extract and analyze the topological structure features of cracks, making it difficult to accurately identify complex cracks, especially small cracks and cross cracks, with low detection accuracy.

[0005] Therefore, it is urgent to develop an intelligent detection method that can adapt to complex environments, fuse multiple information sources, and accurately identify various cracks, to improve the automation level and accuracy of mountain tunnel crack detection. SUMMARY

[0006] The embodiment of the present application provides a deep learning-based mountain tunnel crack intelligent identification and detection method, which can solve the problems in the prior art.

[0007] In a first aspect, the embodiment of the present application provides a deep learning-based mountain tunnel crack intelligent identification and detection method, comprising:

[0008] Collecting multi-spectral image data with labeled information, including visible light image data, near-infrared image data, and thermal infrared image data, the labeled information containing crack position, shape, and type information;

[0009] Adaptive pre-processing the multi-spectral image data, extracting low-frequency and high-frequency features of each spectral data and performing weighted fusion, and performing dynamic range enhancement on the fused image to obtain an enhanced feature map;

[0010] inputting the enhanced feature map into a double-flow network, the double-flow network comprising a spatial feature processing channel for extracting crack morphology and a frequency feature processing channel for extracting crack texture, and introducing a spatial-channel cascade attention module to adaptively weight and fuse the spatial features and the frequency features to obtain fused features;

[0011] based on the fused features, constructing multi-scale feature representations through a feature correlation graph network, performing feature enhancement by combining node importance evaluation and complementary feature extraction modules, and completing feature reconstruction using a reconstruction network with residual connection and progressive fusion;

[0012] training the double-flow network based on a joint optimization objective, the joint optimization objective dynamically adjusting network loss through multi-level morphology feature and energy diffusion consistency constraints;

[0013] performing morphological processing and connectivity analysis on the feature map output by the reconstruction network, extracting topological structure features of the cracks, and matching the topological structure features with the labeled information based on a deep metric learning method to classify the cracks and generate detection results.

[0014] In an optional implementation,

[0015] The step of adaptively pre-processing the multispectral image data, extracting low-frequency and high-frequency features of each spectral data, and performing weighted fusion to obtain an enhanced feature map includes:

[0016] performing multi-scale decomposition on the multispectral image data to obtain low-frequency and high-frequency features;

[0017] inputting the low-frequency and high-frequency features into a deep cascade feature fusion network, the deep cascade feature fusion network comprising a feature complementary enhancement unit, the feature complementary enhancement unit comprising a dilated convolution branch, a deformable convolution branch, and a local attention branch, the dilated convolution branch using multiple dilated convolution layers with different expansion rates to extract multi-scale features; the offset field of the deformable convolution branch is adaptively generated by the input features through an offset generation network, the parameters of the offset generation network being dynamically updated by the feature maps of the dilated convolution branch; the attention weights of the local attention branch are adaptively determined by the crack scale, and are subjected to element-wise multiplication operation with the deformable convolution feature maps; the output features of the three branches are integrated through adaptive weights to obtain enhanced low-frequency features and high-frequency features;

[0018] The enhanced low-frequency feature and high-frequency feature are input into a cross-spectrum feature interaction module, a feature correspondence relationship is established by calculating a similarity matrix between different spectrum features, the similarity matrix is calculated based on channel correlation of a feature map using normalized dot product operation; and different spectrum features are weighted and reorganized based on the similarity matrix to obtain multispectral interaction features.

[0019] The multispectral interaction features are input into a structure-texture decomposition network, the input features are decomposed into a structure feature map and a texture feature map, the structure feature map is enhanced using adaptive piecewise linear mapping, and the texture feature map is processed using an adaptive enhancement function based on texture complexity; and the enhanced structure feature map and texture feature map are fused through an attention mechanism to obtain an enhanced feature map.

[0020] In an optional implementation,

[0021] The enhanced feature map is input into a double-flow network, the double-flow network includes a spatial feature processing path for extracting crack morphology and a frequency feature processing path for extracting crack texture, and a spatial-channel cascading attention module is introduced to adaptively weight and fuse the spatial features and the frequency features to obtain a fusion feature, and the steps include:

[0022] The spatial feature processing path performs parallel processing on the enhanced feature map through multiple convolution layers with different kernel sizes to obtain multi-scale spatial features, the multi-scale spatial features are processed through multiple serial residual blocks and are weighted and fused to obtain spatial features; and the frequency feature processing path processes the enhanced feature map through two-dimensional discrete wavelet transform to obtain approximation coefficients and detail coefficients, the detail coefficients are enhanced through nonlinear mapping and are reconstructed with the approximation coefficients to obtain frequency features.

[0023] The spatial feature is input into a channel attention branch, a channel attention weight is calculated by calculating the correlation between feature channels through global average pooling and a learnable query matrix, the channel attention weight is weighted with the spatial feature to obtain a channel attention feature; and the frequency feature is input into a spatial attention branch, a spatial attention weight is calculated by processing the results of maximum pooling and average pooling through deformable convolution and position encoding, the spatial attention weight is weighted with the frequency feature to obtain a spatial attention feature.

[0024] The channel attention feature and the spatial attention feature are weighted and fused to obtain a fusion feature.

[0025] In an optional implementation,

[0026] The step of weighted fusion of the multi-scale spatial features to obtain spatial features includes:

[0027] A multi-scale feature memory bank is constructed, which includes memory cells corresponding to different convolution kernel sizes, each memory cell including a feature representation vector, a confidence score, and a timestamp, the feature representation vector being obtained by feature extraction on labeled samples and improved K-means clustering, the improved K-means clustering using a weighted combination of intra-class distance sum of squares and logarithm of class sample number as an objective function, and the confidence score being calculated according to a weighted combination of sample class number proportion and feature quality evaluation score;

[0028] The cosine similarity of the multi-scale spatial feature and the corresponding memory cell in the multi-scale feature memory bank is calculated, an adaptive temperature parameter is calculated based on the variance of the cosine similarity, and an initial fusion weight is obtained by performing a softmax operation on the cosine similarity scaled by the adaptive temperature parameter;

[0029] The feature distance and the time interval of the multi-scale spatial feature and the corresponding memory cell are calculated, and when the feature distance is greater than a distance threshold and the time interval is greater than a time threshold, the feature representation vector of the memory cell is updated using a dynamic momentum coefficient based on the feature similarity, and the confidence score of the memory cell is updated based on the feature quality evaluation score;

[0030] The initial fusion weight is multiplied by the confidence score of the memory cell to obtain a final fusion weight, the multi-scale spatial feature is weighted based on the final fusion weight, and a residual connection is constructed with the original feature to obtain a spatial feature.

[0031] In an optional implementation,

[0032] Based on the fusion feature, a multi-scale feature representation is constructed through a feature association graph network, feature enhancement is performed in combination with a node importance evaluation and a complementary feature extraction module, and the steps of feature reconstruction using a residual connection and a progressive fusion reconstruction network include:

[0033] The fusion feature is down-sampled to obtain a multi-scale feature hierarchy, a feature association graph network is constructed, the feature association graph network uses a block diagonal matrix to represent the feature association between adjacent scale levels, initializes the connection weight between levels through a learnable orthogonal transformation matrix, and iteratively optimizes the connection weight according to the local structural similarity of the feature map;

[0034] The importance score of each node in the feature association graph network is calculated, the importance score being determined by the feature response intensity of the node itself and the weighted voting value of adjacent level nodes, the weighted voting value being calculated based on the connection weight between nodes and the feature difference degree;

[0035] The complementary feature extraction module is constructed to sequentially process the multi-scale feature hierarchy, generate a feature selection gating signal based on the importance score, adaptively weight each scale feature, use a deformable convolutional network to spatially adaptively sample the weighted feature to obtain locally enhanced features, and complete and enhance the crack region in the feature map through a conditional generative adversarial network to obtain complementary enhanced features.

[0036] The complementary enhanced features are input into a reconstruction network, feature reconstruction is performed through residual connection and feature progressive fusion, and a loss function combining feature consistency constraint and local structure preservation is used to optimize network parameters.

[0037] In an optional implementation,

[0038] The dual-flow network is trained by constructing a joint optimization target, and the step of dynamically adjusting network loss through multi-level morphological feature and energy diffusion consistency constraint includes:

[0039] Local morphological features of the crack image are extracted, the local morphological features include curvature, directionality and width change rate of the crack, and local feature responses are obtained by mapping the local morphological features through a convolutional network;

[0040] A region association matrix is constructed based on the local feature responses, the region association matrix represents the topological association strength between crack regions, and the region association matrix and the local feature responses are input into a graph convolutional network to extract structural association features of the crack;

[0041] The structural association features are fused with image context information by using an attention mechanism, global semantic features are generated through a feature transformation network, and multi-level feature weights are obtained by inputting the local morphological features, the structural association features and the global semantic features into a weight generation network;

[0042] A feature energy field is calculated by weighted combination of first-order gradient norm and second-order gradient norm of the feature map, an energy-dependent diffusion coefficient is designed based on the feature energy field, the diffusion coefficient is used to construct a feature diffusion equation, energy diffusion is performed on feature maps of different scales, and diffusion consistency loss between adjacent scale feature maps is calculated;

[0043] The multi-level feature weights and the diffusion consistency loss are combined to generate a joint optimization target, and the joint optimization target includes weighted combination of local level loss, structure level loss, global level loss and feature consistency loss.

[0044] In an optional implementation,

[0045] The step of performing morphological processing and connectivity analysis on the feature map output by the reconstruction network, extracting topological structure features of the cracks, and matching the topological structure features with the annotation information based on a deep metric learning method to classify the cracks and generate a detection result includes:

[0046] An adaptive structure element with adjustable direction parameters and scale parameters is constructed, and morphological gradient features are obtained by performing multi-direction dilation operation and multi-scale erosion operation on the feature map output by the reconstruction network using the adaptive structure element, wherein the morphological gradient features include outer boundary features, inner boundary features, and comprehensive gradient features.

[0047] Eight-neighbor connectivity analysis is performed on the morphological gradient features, a hierarchical clustering tree is established, an adaptive segmentation threshold is determined based on the minimum inter-class variance criterion, and a connected domain label map is obtained.

[0048] Distance transformation is performed on the connected domain label map, crack skeleton lines are extracted, branch points and end points on the crack skeleton lines are identified, structure descriptors including direction histograms, length distributions, and width variations are constructed, and topological structure features are obtained.

[0049] The topological structure features and the morphological gradient features are spliced to form a feature vector, the feature vector is mapped to a metric space through a multilayer perceptron, and a feature embedding representation is generated.

[0050] A combined loss function of positive sample pair loss and negative sample pair loss is constructed based on crack type information in the annotation information, the positive sample pair loss is calculated based on the Euclidean distance of the same class sample pair, and the negative sample pair loss is calculated based on the boundary constraint of the different class sample pair; a difficult sample weight is calculated based on the metric distance between sample pairs, the difficult sample weight is adaptively adjusted using a momentum update mechanism, and the difficult sample weight is integrated into the combined loss function.

[0051] The feature embedding representation and global context features are fused, input into a classifier for multi-class classification, uncertainty and confidence are calculated based on the probability distribution of the classification result, and a detection result including crack category and severity is generated based on the position information and morphological information in the annotation information.

[0052] In a second aspect of the embodiment of the present application, an electronic device is provided, comprising:

[0053] a processor;

[0054] a memory for storing processor-executable instructions;

[0055] The processor is configured to invoke the instructions stored in the memory to execute the method described above.

[0056] In a third aspect, the present application provides a computer readable storage medium having computer program instructions stored thereon, which, when executed by a processor, implement the method described above.

[0057] The present application realizes high-precision identification of mountain tunnel cracks through multi-spectral data acquisition and fusion technology, effectively overcomes the limitations of traditional single-spectrum imaging in complex lighting and humidity environments, and significantly improves the accuracy and robustness of crack detection.

[0058] The double-flow network structure and the space-channel cascaded attention module designed by the present application can simultaneously extract and fuse the spatial form and frequency texture features of cracks, solve the problem of blurred crack edges in complex backgrounds, and enable the system to have good recognition ability for cracks of different types and severity.

[0059] The joint optimization target and feature correlation graph network proposed by the present application realizes accurate reconstruction of the topological structure of cracks, greatly improves the integrity and continuity of crack detection through multi-level morphological feature and energy diffusion consistency constraints, and reduces the false detection rate, thereby providing reliable technical support for tunnel safety state evaluation. BRIEF DESCRIPTION OF DRAWINGS

[0060] Figure 1 FIG. 1 is a flowchart of a mountain tunnel crack intelligent identification and detection method based on deep learning according to an embodiment of the present application.

[0061] Figure 2 FIG. 4 is a performance comparison and analysis diagram of the technical solutions. DETAILED DESCRIPTION

[0062] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions of the embodiments of the present application will be described in detail below with reference to the drawings of the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of the present application.

[0063] The technical solutions of the present application will be described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0064] Figure 1 FIG. 1 is a flowchart of a mountain tunnel crack intelligent identification and detection method based on deep learning according to an embodiment of the present application, as shown in the figure, the method comprises: Figure 1

[0065] ​Collecting multi-spectral image data with labeled information, including visible light image data, near-infrared image data and thermal infrared image data, the labeled information containing the position, shape and type information of the cracks;

[0066] Adaptive pre-processing the multi-spectral image data, extracting low-frequency and high-frequency features of each spectral data and performing weighted fusion, performing dynamic range enhancement on the fused image to obtain an enhanced feature map;

[0067] Inputting the enhanced feature map into a double-flow network, the double-flow network containing a spatial feature processing path for extracting crack shape and a frequency feature processing path for extracting crack texture, and introducing a spatial-channel cascade attention module to adaptively weight and fuse the spatial features and the frequency features to obtain a fused feature;

[0068] Based on the fused feature, constructing a multi-scale feature representation through a feature correlation graph network, performing feature enhancement by combining node importance evaluation and complementary feature extraction module, and completing feature reconstruction using a reconstruction network with residual connection and progressive fusion;

[0069] Constructing a joint optimization target to train the double-flow network, the joint optimization target dynamically adjusting the network loss through multi-level shape feature and energy diffusion consistency constraints;

[0070] Performing morphological processing and connectivity analysis on the feature map output by the reconstruction network to extract the topological structure features of the cracks, and matching the topological structure features with the labeled information based on a deep metric learning method, classifying the cracks and generating a detection result.

[0071] In an optional implementation, the step of adaptive pre-processing the multi-spectral image data, extracting low-frequency and high-frequency features of each spectral data and performing weighted fusion, and performing dynamic range enhancement on the fused image to obtain an enhanced feature map includes:

[0072] Performing multi-scale decomposition on the multi-spectral image data to obtain low-frequency and high-frequency features;

[0073] The low-frequency and high-frequency features are input into a deep cascaded feature fusion network, which comprises a feature complementary enhancement unit, the feature complementary enhancement unit comprising a hollow convolution branch, a deformable convolution branch and a local attention branch, the hollow convolution branch adopting multiple hollow convolution layers with different expansion rates to extract multi-scale features; the offset field of the deformable convolution branch is adaptively generated by input features through an offset generation network, and the parameters of the offset generation network are dynamically updated by the feature maps of the hollow convolution branch; the attention weight of the local attention branch is adaptively determined by the crack scale, and is subjected to element-wise multiplication operation with the deformable convolution feature map; the output features of the three branches are integrated through adaptive weights to obtain enhanced low-frequency features and high-frequency features.

[0074] The enhanced low-frequency features and high-frequency features are input into a cross-spectral feature interaction module, and a feature correspondence relationship is established by calculating a similarity matrix between different spectral features, the similarity matrix being calculated based on the channel correlation of the feature map using normalized dot product operation; the different spectral features are weighted and reorganized based on the similarity matrix to obtain multispectral interaction features.

[0075] The multispectral interaction features are input into a structure-texture decomposition network, and the input features are decomposed into a structure feature map and a texture feature map, the structure feature map being enhanced using adaptive piecewise linear mapping, and the texture feature map being processed through an adaptive enhancement function based on texture complexity; the enhanced structure feature map and the texture feature map are fused through an attention mechanism to obtain an enhanced feature map.

[0076] For example, the non-subsampled wavelet transform is used to decompose the input multispectral image data to generate a low-frequency subband and three high-frequency subbands corresponding to the horizontal, vertical and diagonal direction details. For each spectral band, a Haar wavelet basis function is used for two-level decomposition to obtain low-frequency features L and high-frequency features H. For example, for a multispectral image containing infrared, visible light and ultraviolet bands, corresponding low-frequency features LIR, LVIS, LUV and high-frequency features HIR, HVIS, HUV are obtained.

[0077] The deep cascade feature fusion network comprises a feature complementary enhancement unit composed of a hollow convolution branch, a deformable convolution branch and a local attention branch. The hollow convolution branch adopts multiple hollow convolution layers with different expansion rates to extract multi-scale features. Three parallel hollow convolution layers are set, with expansion rates of 2, 4 and 8 respectively, and the convolution kernel size is 3x3, and the output channel number is 64. Through this setting, the receptive field can be expanded without increasing the parameter amount, and the feature information of different scales can be captured. For example, for the input feature map F (size HxWxC), after the hollow convolution with an expansion rate of 2, the feature map F2 is obtained, the size remains HxWx64, but the receptive field is expanded to 5x5; after the hollow convolution with an expansion rate of 4, the feature map F4 is obtained, the receptive field is 9x9; after the hollow convolution with an expansion rate of 8, the feature map F8 is obtained, the receptive field is 17x17. The F2, F4 and F8 are spliced through channels to obtain the output feature Fd of the hollow convolution branch, and the size is HxWx192.

[0078] The offset field of the deformable convolution branch is adaptively generated by the input feature through an offset generation network. The offset generation network is composed of two convolution layers. The first layer uses a 3x3 convolution kernel, and the output channel is 128. The second layer uses a 1x1 convolution kernel, and the output channel is 18 (corresponding to the x and y direction offsets of the 9 sampling positions of the 3x3 convolution kernel). The feature Fd of the hollow convolution branch is input into the offset generation network to obtain the offset field Δp. The deformable convolution is applied to the input feature F, and the sampling position is adjusted according to the offset field Δp to obtain the deformable convolution feature map Fdc, and the size is HxWx64. This design enables the network to adaptively adjust the receptive field shape and better adapt to irregularly shaped targets.

[0079] The attention weight of the local attention branch is adaptively determined by the crack scale. Based on the input feature F, an attention map A is generated through a two-layer convolution network. The first layer uses a 3x3 convolution kernel, and the output channel is 32, using a ReLU activation function. The second layer uses a 1x1 convolution kernel, and the output channel is 1, using a Sigmoid activation function to normalize the value to the 0-1 interval. Then the attention map A and the deformable convolution feature map Fdc are multiplied element by element to obtain the attention weighted feature map Fat, and the size is HxWx64. Through the local attention mechanism, the network can focus on the region with important information and suppress the background interference.

[0080] The outputs of the three branches are fused by adaptive weights, which are generated by a shared fully connected layer that takes the global average pooled feature vector as input and outputs the weights w1, w2, w3 of the three branches, satisfying w1+w2+w3=1. The final enhanced feature F_enhanced = w1xFd+w2xFdc+w3xFat. The above processing is applied to the low-frequency and high-frequency features respectively to obtain the enhanced low-frequency feature L_enhanced and the enhanced high-frequency feature H_enhanced.

[0081] For each pair of spectral features, the feature maps are flattened into matrix form, where the rows correspond to spatial positions and the columns correspond to channel dimensions. The dot product of the two feature matrices is calculated, and row normalization is performed to obtain a similarity matrix S with a size of HWxHW. The similarity matrix S is multiplied by the target feature to obtain the reorganized feature, and the original feature and the reorganized feature are fused through a residual connection to obtain the interaction feature. This operation is performed on the features of all spectral bands to finally obtain the multi-spectral interaction feature F_inter.

[0082] The multi-spectral interaction feature is input into the structure-texture decomposition network to decompose the input feature into a structure feature map F_struct and a texture feature map F_text. The network adopts an encoder-decoder architecture, the encoder extracts multi-scale features through four convolutional layers, the decoder restores the spatial resolution through bilinear upsampling and convolution, and finally outputs the structure feature and the texture feature through two parallel convolutional layers respectively. The structure feature map is enhanced using adaptive piecewise linear mapping. Specifically, according to the gray histogram of the structure feature map, three key points (d_min, d_mid, d_max) are automatically determined, the region with a gray value less than d_min is linearly stretched, the region with a gray value between d_min and d_mid is compressed, the region with a gray value between d_mid and d_max is stretched, and the region with a gray value greater than d_max is compressed to obtain the enhanced structure feature map F_struct_enhanced. The local variance of the texture feature map is calculated as a measure of texture complexity. Then, the enhancement degree is adaptively adjusted according to the texture complexity, a smaller enhancement coefficient is used for regions with high texture complexity, and a larger enhancement coefficient is used for regions with low texture complexity to obtain the enhanced texture feature map F_text_enhanced. The enhanced structure feature map and the enhanced texture feature map are fused through an attention mechanism to obtain an enhanced feature map. The attention weight is generated by a lightweight network that takes the concatenation of the structure feature and the texture feature as input and outputs an attention map of two channels corresponding to the weights of the structure feature and the texture feature. The final enhanced feature map F_final = wsxF_struct_enhanced+wtxF_text_enhanced, where ws and wt are the attention weights.

[0083] The adaptive pre-processing method of the present application realizes effective complementation and synergistic enhancement between different spectral data through a deep cascaded feature fusion network and a cross-spectral feature interaction module. The three-branch structure in the feature complementary enhancement unit can simultaneously capture multi-scale context information, complex morphological features and key region information, while the structure-texture decomposition network can enhance features of different attributes in a targeted manner, so that the system can more accurately recognize and locate various cracks under complex background and variable lighting conditions, greatly improving the feature representation capability and environmental adaptability.

[0084] In an optional embodiment, the enhanced feature map is input into a double-flow network, which includes a spatial feature processing path for extracting crack morphology and a frequency feature processing path for extracting crack texture, and a spatial-channel cascaded attention module is introduced to adaptively weight and fuse the spatial features and frequency features to obtain the fused features, the steps comprising:

[0085] The spatial feature processing path processes the enhanced feature map in parallel through multiple convolution layers with different kernel sizes to obtain multi-scale spatial features, and the multi-scale spatial features are processed through multiple serial residual blocks and weighted fused to obtain spatial features; the frequency feature processing path processes the enhanced feature map through two-dimensional discrete wavelet transform to obtain approximation coefficients and detail coefficients, and the detail coefficients are enhanced after nonlinear mapping and reconstructed with the approximation coefficients to obtain frequency features;

[0086] The spatial features are input into a channel attention branch, the correlation between feature channels is calculated through global average pooling and a learnable query matrix to obtain channel attention weights, and the channel attention weights are weighted with the spatial features to obtain channel attention features; the frequency features are input into a spatial attention branch, the results of maximum pooling and average pooling are processed through deformable convolution and position encoding to obtain spatial attention weights, and the spatial attention weights are weighted with the frequency features to obtain spatial attention features;

[0087] The channel attention features and the spatial attention features are weighted and fused to obtain the fused features.

[0088] For example, the spatial feature processing path uses three different sizes of convolution kernels: 3x3, 5x5, and 7x7, and the output channel number of each convolution layer is 64. These different sizes of convolution kernels can capture crack morphology information at different scales. For example, the 3x3 convolution kernel is suitable for extracting fine crack features, while the 7x7 convolution kernel is conducive to extracting the overall morphology of wider cracks. For an input of a 256x256x3 enhanced feature map, each convolution layer outputs a 256x256x64 feature map. These multi-scale features are further processed by four serial residual blocks, each of which contains two 3x3 convolution layers and a skip connection. The use of residual blocks can alleviate the gradient vanishing problem in deep network training and maintain effective transmission of features. After processing by the residual blocks, 1x1 convolution is used to reduce the channel dimension of the features at different scales, and the weighted sum is obtained through the weight coefficients to obtain the final spatial features with a size of 256x256x64.

[0089] The frequency feature processing path uses Haar wavelet as the basis function to perform one-level wavelet decomposition on the input 256x256x3 feature map to obtain four subbands: approximate coefficients LL (128x128x3), horizontal detail coefficients LH (128x128x3), vertical detail coefficients HL (128x128x3), and diagonal detail coefficients HH (128x128x3). The approximate coefficients contain low-frequency information of the image, while the detail coefficients contain high-frequency information such as edges and textures. The three types of detail coefficients are respectively enhanced by using a 3x3 convolution layer with a LeakyReLU activation function (negative slope of 0.2) for nonlinear mapping, and the output channel number of each convolution layer is 16. The enhanced detail coefficients are reconstructed together with the approximate coefficients by inverse wavelet transform to obtain frequency features with a size of 256x256x48. This frequency domain processing method can effectively capture the texture features of cracks, especially the edge and texture changes of fine cracks.

[0090] The spatial-channel cascaded attention module is composed of a channel attention branch and a spatial attention branch. The channel attention branch processes the spatial features, first performs global average pooling on the 256x256x64 spatial features to obtain a 1x1x64 vector, then performs matrix multiplication with a 64x64 learnable query matrix to calculate the correlation between feature channels, and obtains the channel attention weight. After the weight is normalized to the 0-1 interval by the Sigmoid function, it is multiplied with the original spatial features channel by channel to obtain the channel attention features. For example, for the channels corresponding to the main crack distribution area, the weight may be 0.85, while the weight of the channels corresponding to the background area may be 0.3, which can highlight the crack features.

[0091] The spatial attention branch processes the frequency feature, and maximum pooling and average pooling are performed on the frequency feature of 256x256x48 to obtain two feature maps of 256x256x1. The two feature maps are processed by 3x3 deformable convolution, which can adaptively adjust the receptive field shape according to the crack morphology. The features after convolution are added element by element with a learnable 256x256 position encoding matrix to further integrate the position information. Finally, a 256x256x1 spatial attention weight is obtained through 1x1 convolution and a Sigmoid function, and the weight is multiplied element by element with the original frequency feature to obtain the spatial attention feature. For example, the spatial attention weight of the crack area may be 0.9, and the weight of the non-crack area may be 0.2, so as to enhance the feature expression of the crack area.

[0092] Finally, the channel attention feature and the spatial attention feature are fused. First, 1x1 convolution is used to adjust the two features to the same number of channels (such as 128), and then weighted summation is performed through learnable fusion weights (such as 0.6 and 0.4) to obtain the final fusion feature with a size of 256x256x128. This adaptive weighted fusion method can flexibly adjust the importance of spatial features and frequency features according to the characteristics of cracks in different scenes, and improve the detection ability of the model for various types of cracks.

[0093] The present application effectively solves the problem of insufficient single feature representation by processing spatial domain and frequency domain information in parallel. The spatial feature processing path can accurately capture the morphology and position information of the crack, while the frequency feature processing path can extract the texture and detail features of the crack. The spatial-channel cascaded attention module realizes the adaptive fusion of the two features, so that the system can pay attention to both local details and global structures at the same time, greatly improving the recognition ability of different types of cracks, especially the detection accuracy of cracks in fuzzy boundaries and complex backgrounds.

[0094] In an optional implementation, the step of obtaining the spatial feature by weighting and fusing the multi-scale spatial features comprises:

[0095] A multi-scale feature memory bank is constructed, the multi-scale feature memory bank comprising memory cells corresponding to different convolution kernel sizes, each memory cell comprising a feature representation vector, a confidence score and a time stamp, the feature representation vector being obtained by feature extraction on labeled samples and improved K-means clustering, the improved K-means clustering using a weighted combination of intra-class distance sum of squares and logarithm term of class sample number as an objective function, and the confidence score being calculated according to a weighted combination of sample category proportion and feature quality evaluation score;

[0096] The cosine similarity of the multi-scale spatial feature and the corresponding memory cell in the multi-scale feature memory bank is calculated, an adaptive temperature parameter is calculated based on the variance of the cosine similarity, and an initial fusion weight is obtained by scaling the cosine similarity by the adaptive temperature parameter and performing a softmax operation;

[0097] The feature distance and the time interval of the multi-scale spatial feature and the corresponding memory cell are calculated, and when the feature distance is greater than a distance threshold and the time interval is greater than a time threshold, the feature representation vector of the memory cell is updated using a dynamic momentum coefficient based on the feature similarity, and the confidence score of the memory cell is updated based on the feature quality evaluation score;

[0098] The initial fusion weight is multiplied by the confidence score of the memory cell to obtain a final fusion weight, the multi-scale spatial feature is weighted based on the final fusion weight, and a spatial feature is obtained by connecting the residual of the original feature.

[0099] For example, the multi-scale feature memory bank construction process first creates a corresponding memory cell set for each convolution kernel size (3x3, 5x5, and 7x7). Each memory cell contains three key components: a feature representation vector, a confidence score, and a timestamp. The feature representation vector is a 256-dimensional vector that represents the typical feature pattern of a crack. The confidence score is a floating-point number between 0 and 1, indicating the reliability of the memory cell. The timestamp is an integer value that records the time step of the last update of the memory cell.

[0100] For the initialization of the feature representation vector, first randomly select 10,000 image blocks labeled with cracks from the training data set, and obtain the feature vectors of these image blocks through the feature extraction network corresponding to the convolution kernel size. For example, for the memory cell corresponding to the 3x3 convolution kernel, use the feature extraction network with a 3x3 convolution kernel to process the image block and obtain a 256-dimensional feature vector. Then, the improved K-means clustering is performed on these feature vectors, and the number of clusters is set to 20. The improved K-means clustering uses the weighted combination of the within-class distance sum of squares and the logarithm of the number of class samples as the objective function, where the within-class distance sum of squares is represented as the sum of the squared Euclidean distances of each sample to its class center, and the logarithm of the number of class samples is the logarithmic value of the number of samples in each class. The weight of the within-class distance sum of squares in the objective function is 0.7, and the weight of the logarithm of the number of class samples is 0.3. This setting can avoid the clustering results produced by the traditional K-means algorithm, such as the situation where some cluster centers contain only a small number of samples.

[0101] After clustering, each cluster center is taken as a memory cell's feature representation vector. The confidence score is calculated according to two factors: the proportion of the number of samples in this cluster to the total number of samples (weight 0.4) and the feature quality evaluation score (weight 0.6). The feature quality evaluation score is obtained by calculating the average similarity of the samples in the cluster to the cluster center. For example, if a certain cluster contains 800 samples (accounting for 8%), and the average cosine similarity of the samples in the cluster to the center is 0.85, then the confidence score of this memory cell is 0.4x0.08+0.6x0.85=0.542. The timestamp is initialized to 0, representing the time step when the memory cell is created.

[0102] When processing the input multi-scale spatial features, the cosine similarity of these features to the corresponding memory cells in the memory bank is calculated. For example, the feature map (256x256x64) generated from the 3x3 convolution kernel is first reduced to a 1x1x64 vector through global average pooling, and then transformed into a 256-dimensional vector through linear projection. This 256-dimensional vector is calculated with the cosine similarity of the feature representation vectors of the 20 memory cells in the 3x3 convolution kernel corresponding memory bank, obtaining 20 similarity values.

[0103] The variance of the similarity values is calculated, and if the variance is less than 0.05 (indicating that the similarity distribution is relatively uniform), the temperature parameter is set to 2.0; if the variance is greater than 0.2 (indicating that some memory cells are significantly more similar), the temperature parameter is set to 0.5; otherwise, the temperature parameter is calculated by linear interpolation, for example, if the variance is 0.1, the temperature parameter is 1.5. The smaller the temperature parameter, the steeper the probability distribution output by the softmax function, and the more likely it is to select a small number of memory cells with the highest similarity; the larger the temperature parameter, the flatter the probability distribution, and more memory cells will obtain a relatively balanced weight.

[0104] After dividing the cosine similarity by the temperature parameter, the initial fusion weight is converted through the softmax function. For example, if the similarity values are [0.7, 0.65, 0.8, 0.5,...], and the temperature parameter is 1.0, then the initial fusion weight after softmax processing may be [0.23, 0.22, 0.25, 0.18,...].

[0105] For the dynamic updating mechanism of the memory unit, the system calculates the Euclidean distance between the current feature and the feature representation vector of the memory unit, as well as the difference between the current time step and the memory unit timestamp. If the feature distance is greater than the preset distance threshold (such as 0.4) and the time interval is greater than the time threshold (such as 100 steps), the updating process is triggered. The updating process uses a dynamic momentum coefficient, which is adaptively calculated based on the feature similarity. Specifically, the momentum coefficient a is calculated by the formula a = 0.9 x (1 - similarity). For example, if the feature similarity is 0.7, the momentum coefficient is 0.9 x (1 - 0.7) = 0.27.

[0106] The feature representation vector of the memory unit is updated to a weighted combination of the current value and the new feature, with weights of (1 - a) and a respectively. For example, if the old feature representation vector is [0.2, 0.3, 0.4,...], the new feature is [0.3, 0.2, 0.5,...], and the momentum coefficient a is 0.27, then the updated feature representation vector is [(1 - 0.27) x 0.2 + 0.27 x 0.3, (1 - 0.27) x 0.3 + 0.27 x 0.2, (1 - 0.27) x 0.4 + 0.27 x 0.5,...] = [0.227, 0.273, 0.427,...].

[0107] At the same time, the confidence score of the memory unit is also updated. The updating method is a weighted average of the old confidence score and the new feature quality evaluation score, and the weight also uses the momentum coefficient. For example, if the old confidence score is 0.7, the new feature quality evaluation score is 0.8, and the momentum coefficient is 0.27, then the updated confidence score is (1 - 0.27) x 0.7 + 0.27 x 0.8 = 0.727. The timestamp of the memory unit is updated to the current time step, indicating the time of the last update.

[0108] Finally, the calculation of the final fusion weight is to multiply the initial fusion weight by the confidence score of the memory unit. For example, if the initial fusion weight is [0.23, 0.22, 0.25, 0.18,...], and the confidence score of the corresponding memory unit is [0.6, 0.7, 0.8, 0.5,...], then the final fusion weight is [0.23 x 0.6, 0.22 x 0.7, 0.25 x 0.8, 0.18 x 0.5,...] = [0.138, 0.154, 0.2, 0.09,...]. The final fusion weight is normalized to ensure that the sum of all weights is 1.

[0109] The final fusion weight is used to weight the multi-scale spatial features. For example, the weighted result of the 3x3 convolution kernel generated feature map is the weighted sum of all memory cell feature representation vectors. This weighted result is transformed by back projection to restore the same dimension (256x256x64) as the original feature map. The same processing is performed on the feature maps generated by 5x5 and 7x7 convolution kernels to obtain three weighted feature maps. The three feature maps are spliced through the channel dimension to form a 256x256x192 feature map. Finally, the residual connection is constructed with the original feature map (i.e. the 256x256x64 feature map obtained by 1x1 convolution after splicing of the multi-scale features), and the final spatial feature is obtained through 1x1 convolution to reduce the dimension to 256x256x64.

[0110] In practical applications, the size of the memory bank can be adjusted according to the application scenario and computing resources. For example, for crack detection in complex scenarios, the number of memory cells corresponding to each convolution kernel can be increased to 50 to capture more diverse feature patterns; while for simple scenarios, 20 memory cells are sufficient. Similarly, the dimension of the feature representation vector can also be adjusted, which can be reduced to 128 in the case of limited computing resources, and increased to 512 in the case of pursuing high precision.

[0111] The prior art mainly realizes feature fusion through feature pyramid network, multi-scale convolution, etc. However, these methods often use fixed weights or simple adaptive mechanisms, lacking the ability to remember historical feature patterns. Especially in the field of crack detection, traditional feature fusion methods have difficulty in dealing with feature extraction problems under conditions such as illumination changes and complex surface textures. The multi-scale feature memory bank mechanism of the present invention innovatively introduces long-term memory and dynamic updating ability, enabling the network to accumulate historical knowledge and adapt to new sample features. Through improved K-means clustering and time interval-based dynamic updating strategy, this method can maintain the representativeness and timeliness of feature representation. The introduction of adaptive temperature parameter solves the problem of uneven distribution of feature similarity, while the dynamic momentum coefficient balances stability and adaptability. This memory-enhanced feature fusion mechanism significantly improves the system's ability to recognize similar crack patterns and generalize new cracks.

[0112] In an optional implementation, based on the fusion features, the steps of constructing multi-scale feature representation through a feature association graph network, enhancing features by combining node importance evaluation and complementary feature extraction module, and completing feature reconstruction by using a residual connection and a progressive fusion reconstruction network include:

[0113] The fusion features are obtained through down-sampling of different scales to obtain a multi-scale feature hierarchy, and a feature correlation graph network is constructed, wherein the feature correlation graph network uses a block diagonal matrix to represent the feature correlation between adjacent scale levels, initializes the connection weight between levels through a learnable orthogonal transformation matrix, and iteratively optimizes the connection weight according to the local structural similarity of the feature map;

[0114] The importance score of each node in the feature correlation graph network is calculated, which is determined by the feature response strength of the node itself and the weighted voting value of the adjacent level nodes, and the weighted voting value is calculated based on the connection weight between nodes and the feature difference degree;

[0115] A complementary feature extraction module is constructed to sequentially process the multi-scale feature hierarchy, a feature selection gate signal is generated based on the importance score, each scale feature is adaptively weighted, a deformable convolution network is used to spatially adaptively sample the weighted feature to obtain a locally enhanced feature, a conditional generative adversarial network is used to complete and enhance the crack area in the feature map, and a complementary enhanced feature is obtained;

[0116] The complementary enhanced feature is input into a reconstruction network, feature reconstruction is performed through residual connection and feature progressive fusion, and a loss function combining feature consistency constraint and local structure preservation is used to optimize the network parameters.

[0117] For example, the input fusion feature F first applies a 3x3 convolution kernel and a down-sampling operation with a step size of 2 to obtain the first-level feature F1, and the feature channel number is 64; the same parameters are applied to F1 to obtain the second-level feature F2, and the feature channel number is 128; the third-level feature F3 and the fourth-level feature F4 are obtained in the same way, and the feature channel numbers are 256 and 512 respectively. In this way, a multi-scale feature hierarchy set {F0, F1, F2, F3, F4} from high resolution to low resolution is formed, wherein F0 is the original fusion feature.

[0118] Based on the above multi-scale feature hierarchy, a feature correlation graph network is constructed. The network uses a block diagonal matrix W to represent the correlation between adjacent feature levels. For adjacent feature levels Fi and Fi+1, the correlation matrix Wi,i+1 is initialized as a learnable orthogonal transformation matrix Q, satisfying QTQ (the matrix product of the transpose matrix QT and the matrix Q itself) = I, where I is the identity matrix. The initialization uses the Glorot uniform distribution with a parameter range of [-0.1, 0.1]. During training, the correlation matrix is iteratively optimized based on the local structural similarity of the feature map. For each 3x3 region in the feature map, the structural similarity of the center pixel and the surrounding 8 pixels is calculated, and the higher the similarity value, the greater the connection weight. When the similarity value is greater than 0.75, the weight increases by 0.1; when the similarity value is less than 0.25, the weight decreases by 0.1, ensuring that the weight value is within the range [0, 1].

[0119] After the feature correlation graph network is constructed, the importance score of each node in the network is calculated. For node p in the i-th layer feature map, its importance score S(p) consists of two parts: the feature response intensity of the node itself and the weighted voting value from the nodes of the adjacent level. The feature response intensity is calculated by applying a 3x3 Gaussian filter to the feature map and taking the local maximum value, with a value range of [0, 1]. The weighted voting value V(p) is calculated based on the connection weight between nodes and the feature difference: when the feature difference between the nodes from the upper and lower layers and the current node is less than the threshold value 0.3, the voting value increases; when the difference is greater than 0.7, the voting value decreases. Finally, the importance score S(p) of node p is the weighted sum of the feature response intensity and the normalized weighted voting value, with a weight ratio of 7:3, and the score range is [0, 1].

[0120] The complementary feature extraction module generates a feature selection gating signal G based on the importance score. For regions with a score higher than 0.6, the gating value is set to 1; for regions with a score lower than 0.2, the gating value is set to 0.3; the gating value of the intermediate region is linearly distributed in proportion. The gating signal G is multiplied element-wise with the original feature map to achieve adaptive weighting of the features.

[0121] For the weighted features, a deformable convolution network is used for spatial adaptive sampling. A 5x5 deformable convolution kernel is used, with an offset range controlled within [-2, 2] pixels, and the number of sampling points is 25. For the detected crack regions (continuous regions with importance scores lower than 0.4), feature completion and enhancement are performed through a conditional generative adversarial network. The generator uses a U-Net architecture, including 4 layers of encoders and 4 layers of decoders, with channel numbers of 64, 128, 256, and 512 for the encoders, and symmetric channel numbers for the decoders. The discriminator uses a 5-layer PatchGAN structure with a receptive field size of 70x70 pixels. The training of the generative network uses a batch size of 16, a learning rate of 0.0002, and Adam optimizer parameters of β1 = 0.5 and β2 = 0.999.

[0122] After generating the complementary enhanced features, they are input into the reconstruction network, which uses a residual connection and feature progressive fusion architecture. For each scale level feature Fi, two 3x3 convolution layers (with unchanged channel numbers) are used for feature extraction, and then the original feature Fi is added through residual connection to obtain the enhanced feature Fi'. Starting from the lowest resolution F4', the feature is upsampled to the resolution of F3' through transposed convolution (4x4 convolution kernel, step size 2), and is fused with F3' through channel concatenation and 1x1 convolution to adjust the channel number. In this way, the features are progressively upsampled and fused with the corresponding level of enhanced features, and finally the reconstruction feature with the same resolution as the original fusion feature F0 is obtained.

[0123] During network training, a loss function combining feature consistency constraint and local structure preservation is used. The feature consistency constraint is realized by calculating the pixel-level L1 distance between the reconstructed feature and the original fusion feature, with a weight coefficient of 0.7; the local structure preservation constraint is realized by calculating the local gradient direction consistency of the feature map, and the difference in gradient direction within a 3x3 sliding window is penalized, with a weight coefficient of 0.3. The entire network is trained using the Adam optimizer with an initial learning rate of 0.001, which is decayed to 0.1 every 30 epochs, and a total of 100 epochs are trained.

[0124] The prior art mainly uses methods such as feature pyramid, skip connection and attention mechanism to process multi-scale features, these methods usually pass feature information in one direction, and lack explicit modeling of the relationship between different scale features. The feature correlation graph network proposed by the present application constructs an explicit correlation structure between multi-scale features, and introduces a node importance evaluation and complementary feature extraction mechanism. A block diagonal matrix is used to represent the feature correlation between adjacent scale levels, and a learnable orthogonal transformation matrix is used to initialize the connection weight to ensure the stability of information transmission; a node importance evaluation mechanism is designed, which considers the node's own feature response and adjacent level voting information, so that the network can adaptively identify key feature regions; a complementary feature extraction module is constructed, which realizes spatial adaptive sampling through deformable convolution, and innovatively introduces a conditional generative adversarial network to complete and enhance the crack region. This multi-scale collaborative and feature complementary processing method significantly improves the detection capability and anti-interference capability of the system for cracks of different sizes and morphologies.

[0125] In an optional implementation, the step of constructing a joint optimization target to train the double-flow network, the joint optimization target dynamically adjusting network loss through multi-level morphological feature and energy diffusion consistency constraint comprises:

[0126] Extracting local morphological features of the crack image, the local morphological features including curvature, directionality and width change rate of the crack, mapping the local morphological features through a convolutional network to obtain local feature responses;

[0127] Constructing a region correlation matrix based on the local feature responses, the region correlation matrix representing the topological correlation strength between crack regions, inputting the region correlation matrix and the local feature responses into a graph convolutional network to extract structural correlation features of the crack;

[0128] Fusing the structural correlation features and image context information through an attention mechanism, generating global semantic features through a feature transformation network, inputting the local morphological features, the structural correlation features and the global semantic features into a weight generation network to obtain multi-level feature weights;

[0129] Calculating a feature energy field through a weighted combination of the first-order gradient norm and the second-order gradient norm of the feature map, designing an energy-dependent diffusion coefficient based on the feature energy field; constructing a feature diffusion equation using the diffusion coefficient to perform energy diffusion on feature maps of different scales to calculate diffusion consistency loss between adjacent scale feature maps;

[0130] Combining the multi-level feature weights and the diffusion consistency loss to generate a joint optimization target, the joint optimization target including a weighted combination of local level loss, structural level loss, global level loss and feature consistency loss.

[0131] Exemplarily, when extracting the local morphological features of the crack image, a set of Gaussian difference filters is used to perform multi-scale filtering on the input crack image to obtain response maps at different scales. The curvature feature is calculated for each response map, and the curvature feature is specifically represented by calculating the rate of change of the direction of the image gradient to represent the bending degree of the crack. The directional feature is represented by calculating the main direction of the image gradient and its consistency, and the gradient direction histogram can be used to count the distribution of the gradient direction in the local region. The width change rate feature is obtained by calculating the change of the brightness distribution of the cross section of the crack, and the local maximum suppression is used to find the crack edge points, and the distance between adjacent edge points is the crack width. These local morphological features are organized into a feature tensor and input into a feature mapping network composed of 8 convolution layers, the convolution kernel size is 3x3, the channel number increases from 64 to 256, and down-sampling is performed once after every two layers to obtain a local feature response map.

[0132] When constructing the region association matrix based on the local feature response, the feature map is divided into N×N blocks, and a feature vector is extracted for each block. The feature similarity between the blocks is calculated, and the cosine similarity is used to measure the similarity. The block pair with a similarity value greater than a preset threshold of 0.7 is considered to be associated. The association strength is determined according to the similarity value and the spatial distance between the blocks. The closer the distance and the higher the similarity, the greater the association strength of the block pair. The constructed region association matrix has a size of (N×N)×(N×N), representing the topological association strength between any two blocks. The association matrix and the local feature response are input into a graph convolution network, which includes 3 graph convolution layers, and the output channel number of each layer is 128, 256, and 512, respectively, for extracting structure association features reflecting the structural relationship between crack regions.

[0133] When the attention mechanism is used to fuse the structure association features and the image context information, two mechanisms of channel attention and spatial attention are used. The channel attention extracts channel statistical features through global average pooling and maximum pooling, and generates channel weights through two fully connected layers (the number of neurons is 1 / 16 and the number of feature channels, respectively). The spatial attention generates a spatial weight map through average pooling and maximum pooling along the channel dimension, and then through a 7x7 convolution layer. The weighted features are input into a feature transformation network, which is composed of 5 residual blocks, each of which includes two 3x3 convolution layers and an identity mapping branch, to generate global semantic features containing high-level semantic information.

[0134] The weight generation network receives three kinds of feature inputs: local morphological features, structural correlation features and global semantic features. The network consists of three parallel branches, each branch contains two 1x1 convolutional layers with an output channel number of 1, generating the weight map of the corresponding feature level. The final weight is normalized by the softmax function, ensuring that the sum of the weights of the three levels is 1. During the training process, the local level weight is higher when the crack morphology in a certain area is obvious; the structural level weight increases when the crack continuity is strong; the global level weight dominates when the background is complex or the crack is fuzzy.

[0135] The calculation of the feature energy field is based on the gradient information of the feature map. The first-order gradient norm calculates the gradient of the feature map in the horizontal and vertical directions by the Sobel operator, and then calculates the gradient amplitude. The second-order gradient norm calculates the second-order derivative of the feature map by the Laplacian operator. The feature energy field is the weighted sum of the first-order gradient norm and the second-order gradient norm, with weights of 0.7 and 0.3 respectively. An energy-dependent diffusion coefficient is designed based on the feature energy field. The diffusion coefficient is inversely proportional to the energy value, specifically exp(-energy value / control parameter), and the control parameter is set to 0.15. The feature diffusion equation is constructed. At each diffusion time step, the value of each point on the feature map is updated to the combination of its own value and the weighted sum of the surrounding points, and the weight is determined by the diffusion coefficient. The feature maps of different scales (such as 1 / 2, 1 / 4, 1 / 8 original resolution) in the network are iterated for 10 times, and the diffusion consistency loss between adjacent scale feature maps is calculated, that is, the mean square error of the up-sampled low-resolution feature map and the high-resolution feature map diffusion result.

[0136] The construction of the joint optimization target integrates multi-level feature weights and diffusion consistency constraints to form a comprehensive network training loss function. The multi-level feature weights are dynamically generated by the weight generation network based on features at different abstraction levels (local morphological features, structural correlation features, and global semantic features), reflecting the importance of each level of features to the current sample. These weights are directly used to adjust the corresponding level loss: the local level loss (binary cross-entropy loss) focuses on accurate identification of the local morphology of the crack; the structural level loss (combination of Dice loss and edge perception loss) focuses on the overall shape and edge accuracy of the crack; the global level loss (weighted cross-entropy loss) ensures correct segmentation of the crack area in complex background, emphasizing the importance of crack pixels through a weight ratio of 5:1. At the same time, the diffusion consistency loss ensures the continuity of feature representation at multiple scales by calculating the consistency of different scale feature maps (1 / 2, 1 / 4, 1 / 8 original resolution) after energy diffusion, which is crucial for detecting cracks of different sizes. The final joint optimization target combines the four losses in proportion: the weights of the three level losses are dynamically generated by the network (reflecting the characteristics of the current sample), while the feature consistency loss has a fixed weight of 0.2 (ensuring multi-scale consistency). This optimization strategy combining hierarchical and dynamic weights enables the network to adapt to different types and complexity of crack scenes, significantly improving detection accuracy and robustness.

[0137] The prior art mainly uses a single or simply combined loss function, and although the constraints based on physical models have been applied in some fields, they are rarely closely combined with the loss function design of deep learning. These limitations result in the existing methods being difficult to balance local details and global structure when dealing with morphologically diverse and complex background cracks in mountainous tunnels. The joint optimization target of the present invention innovatively integrates multi-level morphological features and energy diffusion consistency constraints, establishing a complete feature representation system from the local morphology of the crack to the global semantics. The introduction of feature energy field and diffusion equation organically combines physical models with deep learning, enabling the network to maintain feature consistency and continuity at different scales. The design of the multi-level loss function ensures that the network is effectively supervised at different abstraction levels, significantly improving the generalization ability and robustness of the model, especially the detection performance in non-standard morphological cracks and complex environments.

[0138] Figure 2For technical solution performance comparison and analysis diagram, from specific data, the "invention" shows overwhelming advantages in all evaluation dimensions. In terms of "average detection accuracy", the "invention" reaches 94.5%, which is 9.5 percentage points higher than 85.0% of the "U-Net foundation", showing higher overall accuracy. In terms of "complex background robustness", the performance gap is most significant, and the "invention" achieves a high score of 91.8%, far exceeding 76.5% of the "U-Net foundation", indicating that it can still maintain stable and reliable detection capability in the face of complex interference environment. In addition, for "morphology variable crack recall rate" and "local detail accuracy rate", two indicators measuring the model's ability to handle complex situations and fine details, the "invention" reaches 92.3% and 93.7% respectively, also significantly leading the "U-Net foundation" of 78.9% and 82.1%.

[0139] In summary, the chart powerfully proves that the "invention" using joint optimization target has achieved comprehensive and significant improvement in the overall performance of crack detection, environmental adaptability, capture ability of irregular targets, and detail description accuracy compared to the prior art.

[0140] In an optional implementation, the step of performing morphological processing and connectivity analysis on the feature map output by the reconstruction network, extracting the topological structure features of the cracks, and matching the topological structure features with the annotation information based on a deep metric learning method to classify the cracks and generate the detection results includes:

[0141] An adaptive structure element with adjustable direction parameters and scale parameters is constructed, and the adaptive structure element is used to perform multi-direction dilation operation and multi-scale erosion operation on the feature map output by the reconstruction network to obtain morphological gradient features, including outer boundary features, inner boundary features, and comprehensive gradient features;

[0142] Eight-neighbor connectivity analysis is performed on the morphological gradient features, a hierarchical clustering tree is established, an adaptive segmentation threshold is determined based on the minimum inter-class variance criterion, and a connected domain label map is obtained;

[0143] Distance transformation is performed on the connected domain label map, the crack skeleton line is extracted, the branch points and end points on the crack skeleton line are identified, the structure descriptors including direction histogram, length distribution, and width variation are constructed, and the topological structure features are obtained;

[0144] The topological structure features and the morphological gradient features are spliced to form a feature vector, the feature vector is mapped to a metric space through a multi-layer perception machine to generate a feature embedding representation;

[0145] A combination loss function of a positive sample pair loss and a negative sample pair loss is constructed based on the crack type information in the annotation information, the positive sample pair loss is calculated based on the Euclidean distance of the same category sample pair, and the negative sample pair loss is calculated based on the boundary constraint of the different category sample pair; a hard sample weight is calculated based on the metric distance between the sample pairs, the momentum update mechanism is used to adaptively adjust the hard sample weight, and the hard sample weight is integrated into the combination loss function;

[0146] The feature embedding representation is fused with global context features, input into a classifier for multi-class classification, the uncertainty and confidence are calculated based on the probability distribution of the classification result, and the detection result containing the crack category and severity is generated according to the position information and morphological information in the annotation information.

[0147] For example, an adaptive structure element with adjustable direction parameters and scale parameters is constructed, the adaptive structure element includes a linear structure element and a disc structure element, the direction angle of the linear structure element can be adjusted in the range of 0° to 180° with a step of 15°, and the length can be valued between 3 and 15 pixels; the radius of the disc structure element can be adjusted between 2 and 8 pixels. The feature maps output by the reconstruction network are subjected to multi-directional dilation operation and multi-scale erosion operation using these adaptive structure elements, the multi-directional dilation operation adopts the linear structure element to dilate in four main directions of 0°, 45°, 90° and 135° to obtain the dilated feature map; the multi-scale erosion operation adopts the disc structure element with radii of 2, 4 and 6 pixels to erode to obtain the eroded feature map. The outer boundary feature is calculated by the difference between the dilated feature map and the original feature map; the inner boundary feature is calculated by the difference between the original feature map and the eroded feature map; and the comprehensive gradient feature is calculated by the difference between the dilated feature map and the eroded feature map.

[0148] The morphological gradient feature is subjected to gray value statistics to construct a pixel intensity histogram, and a bottom-up hierarchical clustering method is used to group adjacent pixel points according to the gray value similarity. In the clustering process, the initial number of categories is set to 256 (corresponding to the gray level), and similar categories are gradually merged until the number of categories is reduced to a preset threshold (such as 20 categories). An adaptive segmentation threshold is determined based on the minimum inter-class variance criterion, the overall variance is decomposed into intra-class variance and inter-class variance, and the threshold that maximizes the inter-class variance is selected as the optimal segmentation point. For the pixel point with an intensity of 127 in the crack image, if the variance with the surrounding area exceeds a preset threshold of 25, it is divided into an independent connected domain. Through a connected domain labeling algorithm, each independent connected region is assigned a unique identifier, and finally a connected domain label map is obtained.

[0149] The distance transform of the connected domain label map is performed, and a distance field is generated by calculating the shortest distance from each point in the connected domain to the boundary. For the connected domain with a label value of 1, the Euclidean distance from each pixel point in the interior to the boundary is calculated to obtain a distance transform map. A non-maximum suppression algorithm is applied to the distance transform map to retain local maximum points and form an initial skeleton line. Through an iterative thinning algorithm, non-skeleton points are deleted, and finally a crack skeleton line with a width of 1 pixel is obtained. Branch points and end points on the crack skeleton line are identified, the branch point is defined as a skeleton point connected to three or more pixel points in the neighborhood, and the end point is defined as a skeleton point connected to only one pixel point in the neighborhood. The path from the end point to the branch point or from the branch point to the branch point is calculated to extract the segmentation information of the crack. A structural descriptor including a direction histogram, a length distribution, and a width variation is constructed to obtain the topological structure features. The direction histogram statistics the direction distribution of each skeleton line segment in the range of 0° to 180°, the length distribution records the length statistical information of each segment, and the width variation is measured by corresponding the skeleton line to the original connected domain.

[0150] The topological structure features include 128-dimensional direction histogram features, 64-dimensional length distribution features, and 32-dimensional width variation features; the morphological gradient features include 64-dimensional outer boundary features, 64-dimensional inner boundary features, and 128-dimensional comprehensive gradient features. After splicing, a 480-dimensional feature vector is obtained. The feature vector is mapped to a metric space through a multilayer perceptron, the multilayer perceptron includes three fully connected layers with node numbers of 256, 128, and 64, respectively, and is followed by batch normalization and ReLU activation function at each layer, and finally outputs a 64-dimensional feature embedding representation.

[0151] The combination loss function of the positive sample pair loss and the negative sample pair loss is constructed based on the crack type information in the annotation information. This step is the core of deep metric learning, and the purpose is to train the model to learn a feature space that can cluster the same type of cracks and separate different types of cracks, thereby achieving high-precision crack classification. For each batch of training samples, a sample pair set is constructed, including positive sample pairs (same category samples) and negative sample pairs (different category samples). The positive sample pair loss is calculated by calculating the Euclidean distance between the feature embeddings of the same category sample pairs (such as two longitudinal crack samples), and compared with the boundary value 1.0. The smaller the distance, the lower the loss value, which encourages the network to map the same category samples to similar positions. The negative sample pair loss is calculated by calculating the Euclidean distance between the feature embeddings of different category sample pairs (such as longitudinal crack and transverse crack samples), which is required to be greater than the boundary value 5.0. The larger the distance, the lower the loss value, which encourages the network to clearly separate different category samples. The difficult sample weight mechanism solves the sample imbalance problem in training, and identifies and emphasizes those samples that the network is difficult to correctly classify. For positive sample pairs, samples with a Euclidean distance greater than 0.8 are considered as difficult-to-distinguish similar samples, and the weight is increased to twice the original; for negative sample pairs, samples with a Euclidean distance less than 2.0 are considered as easy-to-confuse samples, and the weight is increased to three times the original. The momentum update mechanism is used to adaptively adjust the difficult sample weight, avoiding the weight from fluctuating sharply. Specifically, the current weight is multiplied by 0.9 plus the newly calculated weight multiplied by 0.1. Finally, the difficult sample weight is directly multiplied by the loss value of the corresponding sample pair, integrated into the combination loss function, so that the network pays more attention to difficult-to-classify samples and improves the model's discrimination ability.

[0152] The feature embedding representation is fused with global context features to combine local structure information and global environment information, and to enhance the accuracy of classification. The global context features are extracted from the original feature map through a spatial pyramid pooling module, which includes four scale pooling layers (1x1, 2x2, 3x3 and 6x6) to capture context information of different receptive fields. The fusion method uses feature splicing followed by 1x1 convolution to adjust the channel number and obtain a 128-dimensional enhanced feature vector. This vector is input into a three-layer fully connected network for classification, with hidden layer node numbers of 64 and 32, respectively, and an output layer corresponding to the number of crack categories (usually 4-8 categories, such as horizontal cracks, vertical cracks, network cracks, etc.). The classifier uses a Softmax activation function to obtain the probability distribution of each category, and the highest probability value is used as the confidence index, and the entropy value of all category probabilities is used as the uncertainty measure. At the same time, the system evaluates the severity of the cracks according to features such as skeleton line length, average width and texture complexity, combined with preset severity evaluation criteria (such as slight: width <1mm; moderate: width 1-3mm; severe: width >3mm). The final detection report contains key information such as crack position coordinates, crack category and confidence, severity rating, total length and average width, and according to the confidence level (high: >0.85; medium: 0.6-0.85; low: <0.6), low-confidence detection results are automatically labeled as "to be manually confirmed", providing a comprehensive and reliable basis for subsequent maintenance decisions.

[0153] The morphological processing and connectivity analysis method of the present application realizes accurate extraction of crack topological structure through adaptive structure elements and hierarchical clustering trees. The deep metric learning framework innovatively fuses topological features with visual features, and optimizes feature space distribution through positive and negative sample pair loss and difficult sample mining strategy. In particular, the multi-class classification method combined with uncertainty and confidence enables the system to provide reliable crack classification results and risk assessment. This method combining structure feature analysis and deep learning greatly improves the accuracy of crack classification and the reliability of detection reports, providing a solid basis for subsequent repair decisions.

[0154] In a second aspect, the embodiment of the present application provides an electronic device, comprising:

[0155] a processor;

[0156] a memory for storing processor-executable instructions;

[0157] The processor is configured to invoke the instructions stored in the memory to execute the method described above.

[0158] In a third aspect, the embodiment of the present application provides a computer-readable storage medium having computer program instructions stored thereon, wherein the computer program instructions are executed by a processor to implement the method described above.

[0159] The present application can be a method, an apparatus, a system, and / or a computer program product. The computer program product can include a computer readable storage medium (or media) having computer readable program instructions thereon for performing various aspects of the present application.

[0160] Finally, it should be noted that the above-mentioned embodiments are merely used to illustrate the technical solutions of the present application, rather than limit the present application; even though the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can be modified, or some or all of the technical features thereof can be substituted with equivalent replacements; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for intelligent identification and detection of cracks in mountain tunnels based on deep learning, characterized in that, include: Collect multispectral image data with annotation information, the annotation information including the location, shape and type of cracks; Adaptive preprocessing is performed on the multispectral image data to extract low-frequency and high-frequency features of each spectral data and perform weighted fusion. Dynamic range enhancement is then applied to the fused image to obtain an enhanced feature map. The enhanced feature map is input into a two-stream network, which includes a spatial feature processing path for extracting crack morphology and a frequency feature processing path for extracting crack texture. A spatial-channel cascaded attention module is introduced to adaptively weight and fuse the spatial and frequency features to obtain fused features. Based on the fusion features, a multi-scale feature representation is constructed through a feature association graph network, and feature enhancement is performed by combining node importance evaluation and complementary feature extraction modules. Feature reconstruction is completed by using a reconstruction network with residual connection and progressive fusion. The dual-stream network is trained by constructing a joint optimization objective, which dynamically adjusts the network loss through multi-level morphological features and energy diffusion consistency constraints. Morphological processing and connectivity analysis are performed on the feature map output by the reconstructed network to extract the topological features of the cracks. The topological features are then matched with the annotation information based on a deep metric learning method to classify the cracks and generate detection results.

2. The method according to claim 1, characterized in that, The steps of adaptive preprocessing of the multispectral image data, extracting low-frequency and high-frequency features from each spectral data and performing weighted fusion, and then performing dynamic range enhancement on the fused image to obtain an enhanced feature map include: The multispectral image data is decomposed into low-frequency and high-frequency features by multi-scale decomposition. Low-frequency and high-frequency features are input into a deep concatenated feature fusion network. This network includes a feature complementarity enhancement unit, which comprises a dilated convolution branch, a deformable convolution branch, and a local attention branch. The dilated convolution branch extracts multi-scale features using multiple dilated convolutional layers with different dilation rates. The offset field of the deformable convolution branch is adaptively generated from the input features through an offset generation network. The attention weights of the local attention branch are adaptively determined by the crack scale and are element-wise multiplied with the deformable convolutional feature map. The output features of the three branches are integrated using adaptive weights to obtain the enhanced low-frequency and high-frequency features. The enhanced low-frequency and high-frequency features are input into the cross-spectral feature interaction module, and the feature correspondence is established by calculating the similarity matrix between different spectral features; based on the similarity matrix, the different spectral features are weighted and recombined to obtain multispectral interactive features; The multispectral interactive features are input into a structure-texture decomposition network, which decomposes the input features into a structure feature map and a texture feature map. The structure feature map is enhanced by an adaptive piecewise linear mapping, and the texture feature map is processed by an adaptive enhancement function based on texture complexity. The enhanced structure feature map and texture feature map are then fused through an attention mechanism to obtain an enhanced feature map.

3. The method according to claim 1, characterized in that, The enhanced feature map is input into a two-stream network, which includes a spatial feature processing path for extracting crack morphology and a frequency feature processing path for extracting crack texture. A spatial-channel cascaded attention module is introduced to adaptively weight and fuse the spatial and frequency features to obtain the fused features. The steps include: The spatial feature processing pathway processes the enhanced feature map in parallel using multiple convolutional layers with different kernel sizes to obtain multi-scale spatial features. The multi-scale spatial features are then processed by multiple cascaded residual blocks and weighted to obtain spatial features. The frequency feature processing pathway processes the enhanced feature map using two-dimensional discrete wavelet transform to obtain approximation coefficients and detail coefficients. The detail coefficients are then enhanced by nonlinear mapping and reconstructed with the approximation coefficients to obtain frequency features. The spatial features are input into the channel attention branch, and the correlation between feature channels is calculated through global average pooling and a learnable query matrix to obtain channel attention weights. The channel attention weights are weighted with the spatial features to obtain channel attention features. The frequency features are input into the spatial attention branch, and the results of max pooling and average pooling are processed through deformable convolution and positional encoding to obtain spatial attention weights. The spatial attention weights are weighted with the frequency features to obtain spatial attention features. The channel attention features and the spatial attention features are weighted and fused to obtain the fused features.

4. The method according to claim 3, characterized in that, The steps for obtaining spatial features through weighted fusion of multi-scale spatial features include: A multi-scale feature memory is constructed, comprising memory units corresponding to different convolutional kernel sizes. Each memory unit includes a feature representation vector and a confidence score. The feature representation vector is obtained by extracting features from labeled samples and performing improved K-means clustering. The improved K-means clustering uses a weighted combination of the sum of squared intra-class distances and the logarithm of the number of samples in each class as the objective function. The confidence score is calculated based on a weighted average of the proportion of sample categories and the feature quality evaluation score. Calculate the cosine similarity between the multi-scale spatial features and the corresponding memory units in the multi-scale feature memory bank, calculate the adaptive temperature parameter based on the variance of the cosine similarity, and obtain the initial fusion weight by scaling the cosine similarity with the adaptive temperature parameter and then performing a softmax operation. Calculate the feature distance and time interval between the multi-scale spatial features and the corresponding memory units. When the feature distance is greater than a distance threshold and the time interval is greater than a time threshold, update the feature representation vector of the memory unit using a dynamic momentum coefficient based on feature similarity, and update the confidence score of the memory unit based on the feature quality assessment score. The initial fusion weights are multiplied by the confidence scores of the memory units to obtain the final fusion weights. The multi-scale spatial features are then weighted based on the final fusion weights and residual connections are constructed with the original features to obtain the spatial features.

5. The method according to claim 1, characterized in that, Based on the fused features, a multi-scale feature representation is constructed through a feature association graph network. Feature enhancement is performed by combining node importance evaluation and complementary feature extraction modules. The feature reconstruction is completed using a reconstruction network with residual connections and progressive fusion. The steps include: The fused features are downsampled at different scales to obtain multi-scale feature levels, and a feature association graph network is constructed. The feature association graph network uses a block diagonal matrix to represent the feature association between adjacent scale levels. The connection weights between levels are initialized by a learnable orthogonal transformation matrix, and the connection weights are iteratively optimized according to the local structural similarity of the feature maps. The importance score of each node in the feature association graph network is calculated. The importance score is determined by the feature response strength of the node itself and the weighted voting value of the adjacent level nodes. The weighted voting value is calculated based on the connection weight and feature difference between the nodes. A complementary feature extraction module is constructed to process the multi-scale feature hierarchy sequentially, generate a feature selection gating signal based on the importance score, adaptively weight the features at each scale, use a deformable convolutional network to perform spatial adaptive sampling on the weighted features to obtain local enhanced features, and use a conditional generative adversarial network to fill in and enhance the crack region in the feature map to obtain complementary enhanced features. The complementary enhancement features are input into the reconstruction network, and feature reconstruction is performed through residual connections and feature progressive fusion. The network parameters are optimized using a loss function that combines feature consistency constraints and local structure preservation.

6. The method according to claim 1, characterized in that, The steps of constructing a joint optimization objective to train the dual-stream network, wherein the joint optimization objective dynamically adjusts the network loss through multi-level morphological features and energy diffusion consistency constraints, include: Local morphological features of the crack image are extracted, including the crack curvature, directionality, and width change rate. The local morphological features are then mapped using a convolutional network to obtain local feature responses. A regional correlation matrix is ​​constructed based on the local feature response. The regional correlation matrix represents the topological correlation strength between crack regions. The regional correlation matrix and the local feature response are input into a graph convolutional network to extract the structural correlation features of the cracks. An attention mechanism is used to fuse the structural association features with image context information. A feature transformation network is used to generate global semantic features. The local morphological features, the structural association features, and the global semantic features are input into a weight generation network to obtain multi-level feature weights. The feature energy field is calculated by weighted combination of the first-order gradient norm and the second-order gradient norm of the feature map. Based on the feature energy field, an energy-dependent diffusion coefficient is designed. The feature diffusion equation is constructed using the diffusion coefficient, and energy diffusion is performed on feature maps of different scales to calculate the diffusion consistency loss between feature maps of adjacent scales. The multi-level feature weights are combined with the diffusion consistency loss to generate a joint optimization objective, which includes a weighted combination of local level loss, structural level loss, global level loss and feature consistency loss.

7. The method according to claim 1, characterized in that, The steps of performing morphological processing and connectivity analysis on the feature map output by the reconstructed network, extracting the topological features of the cracks, and matching the topological features with the annotation information based on a deep metric learning method to classify the cracks and generate detection results include: An adaptive structural element with adjustable orientation and scale parameters is constructed. The adaptive structural element is used to perform multi-directional dilation and multi-scale erosion operations on the feature map output by the reconstructed network to obtain morphological gradient features, which include outer boundary features, inner boundary features and comprehensive gradient features. Eight-neighbor connectivity analysis is performed on the morphological gradient features to establish a hierarchical clustering tree. An adaptive segmentation threshold is determined based on the inter-class variance minimization criterion to obtain a connected component labeling map. Distance transformation is performed on the connected component marker graph to extract the crack skeleton line, the branch points and endpoints on the crack skeleton line are identified, and a structural descriptor including orientation histogram, length distribution and width variation is constructed to obtain topological features; The topological features and the morphological gradient features are concatenated to form a feature vector, which is then mapped to a metric space using a multilayer perceptron to generate a feature embedding representation. Based on the crack type information in the annotation information, a combined loss function of positive sample pair loss and negative sample pair loss is constructed; the weight of hard samples is calculated based on the metric distance between sample pairs, and the weight of hard samples is adaptively adjusted using a momentum update mechanism, and the weight of hard samples is integrated into the combined loss function; The embedded feature representation is fused with global context features and input into a classifier for multi-class classification. The uncertainty and confidence are calculated based on the probability distribution of the classification results. Detection results containing crack category and severity are generated based on the location and morphological information in the annotation information.

8. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the method according to any one of claims 1 to 7.

9. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1 to 7.

Images