Intelligent recognition and detection method for mountain tunnel cracks based on deep learning

Abstract

The application provides a mountain tunnel crack intelligent identification and detection method based on deep learning, relates to the technical field of deep learning, and comprises the following steps: acquiring multi-spectral image data, and obtaining an enhanced feature map through self-adaptive preprocessing; extracting and fusing features by using a double-flow network combined with a space-channel cascaded attention module; constructing multi-scale feature representation through a feature correlation graph network; training the network by using a joint optimization target; and performing morphological processing and connectivity analysis to realize accurate identification and classification of cracks. The application improves the detection precision and anti-interference capability of tunnel cracks in complex environments.

Description

Technical Field

[0001] This invention relates to deep learning technology, and more particularly to a method for intelligent identification and detection of cracks in mountain tunnels based on deep learning. Background Technology

[0002] Cracks are a common type of tunnel defect, which can lead to safety hazards such as reduced structural strength, water seepage, and collapse. Traditional tunnel crack detection mainly relies on manual inspection, which is not only labor-intensive and inefficient, but also its accuracy is greatly affected by subjective factors.

[0003] With the development of computer vision and deep learning technologies, automatic crack detection methods based on image processing have been widely applied. Existing technologies mainly include methods based on traditional image processing, such as edge detection and threshold segmentation; and methods based on deep learning, such as convolutional neural networks and fully convolutional networks. These methods can achieve good crack recognition results under ideal conditions.

[0004] However, mountain tunnels present complex environments, and existing technologies still have significant shortcomings in practical applications: First, the lighting conditions in mountain tunnels are complex and variable, and a single visible light imaging mode is insufficient to cope with various complex environments, resulting in unstable detection accuracy; second, existing crack detection algorithms mainly focus on the spatial features of images, ignoring frequency domain information and the fusion of multi-scale features, and cannot effectively distinguish cracks from background textures; finally, traditional methods lack the ability to extract and analyze the topological features of cracks, making it difficult to accurately identify cracks with complex shapes, especially with low detection accuracy for micro-cracks and intersecting cracks.

[0005] Therefore, there is an urgent need to develop an intelligent detection method that can adapt to complex environments, integrate multiple information sources, and accurately identify various types of cracks, so as to improve the automation level and accuracy of crack detection in mountain tunnels. Summary of the Invention

[0006] This invention provides a deep learning-based intelligent identification and detection method for cracks in mountain tunnels, which can solve the problems in the prior art.

[0007] A first aspect of this invention provides a deep learning-based intelligent identification and detection method for cracks in mountain tunnels, comprising:

[0008] Collect multispectral image data with annotation information, including visible light image data, near-infrared image data and thermal infrared image data, wherein the annotation information includes the location, shape and type of cracks;

[0009] Adaptive preprocessing is performed on the multispectral image data to extract low-frequency and high-frequency features of each spectral data and perform weighted fusion. Dynamic range enhancement is then applied to the fused image to obtain an enhanced feature map.

[0010] The enhanced feature map is input into a two-stream network, which includes a spatial feature processing path for extracting crack morphology and a frequency feature processing path for extracting crack texture. A spatial-channel cascaded attention module is introduced to adaptively weight and fuse the spatial and frequency features to obtain fused features.

[0011] Based on the fusion features, a multi-scale feature representation is constructed through a feature association graph network, and feature enhancement is performed by combining node importance evaluation and complementary feature extraction modules. Feature reconstruction is completed by using a reconstruction network with residual connection and progressive fusion.

[0012] The dual-stream network is trained by constructing a joint optimization objective, which dynamically adjusts the network loss through multi-level morphological features and energy diffusion consistency constraints.

[0013] Morphological processing and connectivity analysis are performed on the feature map output by the reconstructed network to extract the topological features of the cracks. The topological features are then matched with the annotation information based on a deep metric learning method to classify the cracks and generate detection results.

[0014] In one alternative implementation,

[0015] The steps of adaptive preprocessing of the multispectral image data, extracting low-frequency and high-frequency features from each spectral data and performing weighted fusion, and then performing dynamic range enhancement on the fused image to obtain an enhanced feature map include:

[0016] The multispectral image data is decomposed into low-frequency and high-frequency features by multi-scale decomposition.

[0017] Low-frequency and high-frequency features are input into a deep concatenated feature fusion network. This network includes a feature complementarity enhancement unit, which comprises a dilated convolution branch, a deformable convolution branch, and a local attention branch. The dilated convolution branch extracts multi-scale features using multiple dilated convolutional layers with different dilation rates. The offset field of the deformable convolution branch is adaptively generated from the input features through an offset generation network, whose parameters are dynamically updated by the feature map of the dilated convolution branch. The attention weights of the local attention branch are adaptively determined by the crack scale and multiplied element-wise with the deformable convolution feature map. The output features of the three branches are integrated using adaptive weights to obtain the enhanced low-frequency and high-frequency features.

[0018] The enhanced low-frequency and high-frequency features are input into the cross-spectral feature interaction module. Feature correspondence is established by calculating the similarity matrix between different spectral features. The similarity matrix is ​​calculated using normalized dot product operation based on the channel correlation of the feature map. Based on the similarity matrix, the different spectral features are weighted and recombined to obtain multispectral interactive features.

[0019] The multispectral interactive features are input into a structure-texture decomposition network, which decomposes the input features into a structure feature map and a texture feature map. The structure feature map is enhanced by an adaptive piecewise linear mapping, and the texture feature map is processed by an adaptive enhancement function based on texture complexity. The enhanced structure feature map and texture feature map are then fused through an attention mechanism to obtain an enhanced feature map.

[0020] In one alternative implementation,

[0021] The enhanced feature map is input into a two-stream network, which includes a spatial feature processing path for extracting crack morphology and a frequency feature processing path for extracting crack texture. A spatial-channel cascaded attention module is introduced to adaptively weight and fuse the spatial and frequency features to obtain the fused features. The steps include:

[0022] The spatial feature processing pathway processes the enhanced feature map in parallel using multiple convolutional layers with different kernel sizes to obtain multi-scale spatial features. The multi-scale spatial features are then processed by multiple cascaded residual blocks and weighted to obtain spatial features. The frequency feature processing pathway processes the enhanced feature map using two-dimensional discrete wavelet transform to obtain approximation coefficients and detail coefficients. The detail coefficients are then enhanced by nonlinear mapping and reconstructed with the approximation coefficients to obtain frequency features.

[0023] The spatial features are input into the channel attention branch, and the correlation between feature channels is calculated through global average pooling and a learnable query matrix to obtain channel attention weights. The channel attention weights are weighted with the spatial features to obtain channel attention features. The frequency features are input into the spatial attention branch, and the results of max pooling and average pooling are processed through deformable convolution and positional encoding to obtain spatial attention weights. The spatial attention weights are weighted with the frequency features to obtain spatial attention features.

[0024] The channel attention features and the spatial attention features are weighted and fused to obtain the fused features.

[0025] In one alternative implementation,

[0026] The steps for obtaining spatial features through weighted fusion of multi-scale spatial features include:

[0027] A multi-scale feature memory is constructed, comprising memory units corresponding to different convolutional kernel sizes. Each memory unit includes a feature representation vector, a confidence score, and a timestamp. The feature representation vector is obtained by extracting features from labeled samples and performing improved K-means clustering. The improved K-means clustering uses a weighted combination of the sum of squared intra-class distances and the logarithm of the number of samples in each class as the objective function. The confidence score is calculated as a weighted average of the proportion of sample categories and the feature quality evaluation score.

[0028] Calculate the cosine similarity between the multi-scale spatial features and the corresponding memory units in the multi-scale feature memory bank, calculate the adaptive temperature parameter based on the variance of the cosine similarity, and obtain the initial fusion weight by scaling the cosine similarity with the adaptive temperature parameter and then performing a softmax operation.

[0029] Calculate the feature distance and time interval between the multi-scale spatial features and the corresponding memory units. When the feature distance is greater than a distance threshold and the time interval is greater than a time threshold, update the feature representation vector of the memory unit using a dynamic momentum coefficient based on feature similarity, and update the confidence score of the memory unit based on the feature quality assessment score.

[0030] The initial fusion weights are multiplied by the confidence scores of the memory units to obtain the final fusion weights. The multi-scale spatial features are then weighted based on the final fusion weights and residual connections are constructed with the original features to obtain the spatial features.

[0031] In one alternative implementation,

[0032] Based on the fused features, a multi-scale feature representation is constructed through a feature association graph network. Feature enhancement is performed by combining node importance evaluation and complementary feature extraction modules. The feature reconstruction is completed using a reconstruction network with residual connections and progressive fusion. The steps include:

[0033] The fused features are downsampled at different scales to obtain multi-scale feature levels, and a feature association graph network is constructed. The feature association graph network uses a block diagonal matrix to represent the feature association between adjacent scale levels. The connection weights between levels are initialized by a learnable orthogonal transformation matrix, and the connection weights are iteratively optimized according to the local structural similarity of the feature maps.

[0034] The importance score of each node in the feature association graph network is calculated. The importance score is determined by the feature response strength of the node itself and the weighted voting value of the adjacent level nodes. The weighted voting value is calculated based on the connection weight and feature difference between the nodes.

[0035] A complementary feature extraction module is constructed to process the multi-scale feature hierarchy sequentially, generate a feature selection gating signal based on the importance score, adaptively weight the features at each scale, use a deformable convolutional network to perform spatial adaptive sampling on the weighted features to obtain local enhanced features, and use a conditional generative adversarial network to fill in and enhance the crack region in the feature map to obtain complementary enhanced features.

[0036] The complementary enhancement features are input into the reconstruction network, and feature reconstruction is performed through residual connections and feature progressive fusion. The network parameters are optimized using a loss function that combines feature consistency constraints and local structure preservation.

[0037] In one alternative implementation,

[0038] The steps of constructing a joint optimization objective to train the dual-stream network, wherein the joint optimization objective dynamically adjusts the network loss through multi-level morphological features and energy diffusion consistency constraints, include:

[0039] Local morphological features of the crack image are extracted, including the crack curvature, directionality, and width change rate. The local morphological features are then mapped using a convolutional network to obtain local feature responses.

[0040] A regional correlation matrix is ​​constructed based on the local feature response. The regional correlation matrix represents the topological correlation strength between crack regions. The regional correlation matrix and the local feature response are input into a graph convolutional network to extract the structural correlation features of the cracks.

[0041] An attention mechanism is used to fuse the structural association features with image context information. A feature transformation network is used to generate global semantic features. The local morphological features, the structural association features, and the global semantic features are input into a weight generation network to obtain multi-level feature weights.

[0042] The feature energy field is calculated by weighted combination of the first-order gradient norm and the second-order gradient norm of the feature map. Based on the feature energy field, an energy-dependent diffusion coefficient is designed. The feature diffusion equation is constructed using the diffusion coefficient, and energy diffusion is performed on feature maps of different scales to calculate the diffusion consistency loss between feature maps of adjacent scales.

[0043] The multi-level feature weights are combined with the diffusion consistency loss to generate a joint optimization objective, which includes a weighted combination of local level loss, structural level loss, global level loss and feature consistency loss.

[0044] In one alternative implementation,

[0045] The steps of performing morphological processing and connectivity analysis on the feature map output by the reconstructed network, extracting the topological features of the cracks, and matching the topological features with the annotation information based on a deep metric learning method to classify the cracks and generate detection results include:

[0046] An adaptive structural element with adjustable orientation and scale parameters is constructed. The adaptive structural element is used to perform multi-directional dilation and multi-scale erosion operations on the feature map output by the reconstructed network to obtain morphological gradient features, which include outer boundary features, inner boundary features and comprehensive gradient features.

[0047] Eight-neighbor connectivity analysis is performed on the morphological gradient features to establish a hierarchical clustering tree. An adaptive segmentation threshold is determined based on the inter-class variance minimization criterion to obtain a connected component labeling map.

[0048] Distance transformation is performed on the connected component marker graph to extract the crack skeleton line, the branch points and endpoints on the crack skeleton line are identified, and a structural descriptor including orientation histogram, length distribution and width variation is constructed to obtain topological features;

[0049] The topological features and the morphological gradient features are concatenated to form a feature vector, which is then mapped to a metric space using a multilayer perceptron to generate a feature embedding representation.

[0050] Based on the crack type information in the annotation information, a combined loss function of positive sample pair loss and negative sample pair loss is constructed. The positive sample pair loss is calculated based on the Euclidean distance of sample pairs of the same category, and the negative sample pair loss is calculated based on the boundary constraints of sample pairs of different categories. The hard sample weight is calculated based on the metric distance between sample pairs, and the hard sample weight is adaptively adjusted using a momentum update mechanism. The hard sample weight is then integrated into the combined loss function.

[0051] The embedded feature representation is fused with global context features and input into a classifier for multi-class classification. The uncertainty and confidence are calculated based on the probability distribution of the classification results. Detection results containing crack category and severity are generated based on the location and morphological information in the annotation information.

[0052] A second aspect of the present invention provides an electronic device, comprising:

[0053] processor;

[0054] Memory used to store processor-executable instructions;

[0055] The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0056] A third aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0057] This invention achieves high-precision identification of tunnel cracks in mountainous areas through multispectral data acquisition and fusion technology, effectively overcoming the limitations of traditional single-spectral imaging in complex lighting and humidity environments, and significantly improving the accuracy and robustness of crack detection.

[0058] The dual-stream network structure and spatial-channel cascaded attention module designed in this invention can simultaneously extract and fuse the spatial morphology and frequency texture features of cracks, solving the problem of blurred crack edges in fine cracks and complex backgrounds, and enabling the system to have good recognition capabilities for cracks of different types and severity.

[0059] The joint optimization objective and feature association graph network proposed in this invention achieves accurate reconstruction of crack topology. Through multi-level morphological features and energy diffusion consistency constraints, it significantly improves the integrity and continuity of crack detection, reduces the false detection rate, and provides reliable technical support for tunnel safety status assessment. Attached Figure Description

[0060] Figure 1 This is a flowchart illustrating the intelligent identification and detection method for cracks in mountain tunnels based on deep learning, as described in an embodiment of the present invention.

[0061] Figure 2 This is a diagram illustrating the performance comparison and analysis of the technical solutions. Detailed Implementation

[0062] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0063] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0064] Figure 1 This is a flowchart illustrating the intelligent identification and detection method for cracks in mountain tunnels based on deep learning, as described in an embodiment of the present invention. Figure 1 As shown, the method includes:

[0065] Collect multispectral image data with annotation information, including visible light image data, near-infrared image data and thermal infrared image data, wherein the annotation information includes the location, shape and type of cracks;

[0066] Adaptive preprocessing is performed on the multispectral image data to extract low-frequency and high-frequency features of each spectral data and perform weighted fusion. Dynamic range enhancement is then applied to the fused image to obtain an enhanced feature map.

[0067] The enhanced feature map is input into a two-stream network, which includes a spatial feature processing path for extracting crack morphology and a frequency feature processing path for extracting crack texture. A spatial-channel cascaded attention module is introduced to adaptively weight and fuse the spatial and frequency features to obtain fused features.

[0068] Based on the fusion features, a multi-scale feature representation is constructed through a feature association graph network, and feature enhancement is performed by combining node importance evaluation and complementary feature extraction modules. Feature reconstruction is completed by using a reconstruction network with residual connection and progressive fusion.

[0069] The dual-stream network is trained by constructing a joint optimization objective, which dynamically adjusts the network loss through multi-level morphological features and energy diffusion consistency constraints.

[0070] Morphological processing and connectivity analysis are performed on the feature map output by the reconstructed network to extract the topological features of the cracks. The topological features are then matched with the annotation information based on a deep metric learning method to classify the cracks and generate detection results.

[0071] In one optional implementation, the steps of adaptively preprocessing the multispectral image data, extracting low-frequency and high-frequency features from each spectral data and performing weighted fusion, and then performing dynamic range enhancement on the fused image to obtain an enhanced feature map include:

[0072] The multispectral image data is decomposed into low-frequency and high-frequency features by multi-scale decomposition.

[0073] Low-frequency and high-frequency features are input into a deep concatenated feature fusion network. This network includes a feature complementarity enhancement unit, which comprises a dilated convolution branch, a deformable convolution branch, and a local attention branch. The dilated convolution branch extracts multi-scale features using multiple dilated convolutional layers with different dilation rates. The offset field of the deformable convolution branch is adaptively generated from the input features through an offset generation network, whose parameters are dynamically updated by the feature map of the dilated convolution branch. The attention weights of the local attention branch are adaptively determined by the crack scale and multiplied element-wise with the deformable convolution feature map. The output features of the three branches are integrated using adaptive weights to obtain the enhanced low-frequency and high-frequency features.

[0074] The enhanced low-frequency and high-frequency features are input into the cross-spectral feature interaction module. Feature correspondence is established by calculating the similarity matrix between different spectral features. The similarity matrix is ​​calculated using normalized dot product operation based on the channel correlation of the feature map. Based on the similarity matrix, the different spectral features are weighted and recombined to obtain multispectral interactive features.

[0075] The multispectral interactive features are input into a structure-texture decomposition network, which decomposes the input features into a structure feature map and a texture feature map. The structure feature map is enhanced by an adaptive piecewise linear mapping, and the texture feature map is processed by an adaptive enhancement function based on texture complexity. The enhanced structure feature map and texture feature map are then fused through an attention mechanism to obtain an enhanced feature map.

[0076] For example, non-subsampling wavelet transform is used to decompose the input multispectral image data, generating one low-frequency sub-band and three high-frequency sub-bands, corresponding to detail information in the horizontal, vertical, and diagonal directions, respectively. For each spectral band, a second-level decomposition is performed using Haar wavelet basis functions to obtain low-frequency features L and high-frequency features H. For instance, for a multispectral image containing infrared, visible, and ultraviolet bands, the corresponding low-frequency features LIR, LVIS, and LUV and high-frequency features HIR, HVIS, and HUV are obtained, respectively.

[0077] The deep cascaded feature fusion network includes a feature complementarity enhancement unit, which consists of a dilated convolution branch, a deformable convolution branch, and a local attention branch. The dilated convolution branch uses multiple dilated convolutional layers with different dilation rates to extract multi-scale features. Three parallel dilated convolutional layers are used, with dilation rates of 2, 4, and 8, a kernel size of 3×3, and 64 output channels for each. This setup expands the receptive field without increasing the number of parameters, capturing feature information at different scales. For example, for an input feature map F (size H×W×C), after a dilated convolution with a dilation rate of 2, feature map F2 is obtained, maintaining the size H×W×64 but expanding the receptive field to 5×5; after a dilated convolution with a dilation rate of 4, feature map F4 is obtained, with a receptive field of 9×9; and after a dilated convolution with a dilation rate of 8, feature map F8 is obtained, with a receptive field of 17×17. Concatenating F2, F4, and F8 through channels yields the output feature Fd of the dilated convolution branch, with a size of H×W×192.

[0078] The offset field of the deformable convolution branch is adaptively generated by an offset generation network from the input features. This network consists of two convolutional layers: the first layer uses a 3×3 convolutional kernel with 128 output channels, and the second layer uses a 1×1 convolutional kernel with 18 output channels (corresponding to the x and y offsets of the 9 sampling positions of the 3×3 convolutional kernel). The features Fd from the dilated convolution branch are input into the offset generation network to obtain the offset field Δp. A deformable convolution is then applied to the input features F, with the sampling positions adjusted according to the offset field Δp, resulting in a deformable convolutional feature map Fdc with dimensions H×W×64. This design allows the network to adaptively adjust the receptive field shape, better adapting to irregularly shaped targets.

[0079] The attention weights of the local attention branch are adaptively determined by the crack scale. Based on the input feature F, an attention map A is generated through a two-layer convolutional network. The first layer uses a 3×3 convolutional kernel with 32 output channels and a ReLU activation function. The second layer uses a 1×1 convolutional kernel with 1 output channel and a Sigmoid activation function to normalize the values ​​to the 0-1 range. Then, the attention map A is element-wise multiplied with the deformable convolutional feature map Fdc to obtain the attention-weighted feature map Fat, with dimensions H×W×64. Through the local attention mechanism, the network can focus on regions with important information and suppress background interference.

[0080] The output features of the three branches are processed through adaptive weights, which are generated by a shared fully connected layer. This layer takes the feature vector after global average pooling as input and outputs the weight values ​​w1, w2, and w3 for the three branches, satisfying w1 + w2 + w3 = 1. The final enhanced feature F_enhanced = w1 × Fd + w2 × Fdc + w3 × Fat. Applying the above processing to the low-frequency and high-frequency features respectively, we obtain the enhanced low-frequency feature L_enhanced and the high-frequency feature H_enhanced.

[0081] For each pair of spectral features, the feature map is flattened into a matrix, where rows correspond to spatial locations and columns correspond to channel dimensions. The dot product of the two feature matrices is calculated and row normalized to obtain a similarity matrix S of size HW×HW. The similarity matrix S is multiplied by the target feature to obtain the reconstructed feature. The original feature and the reconstructed feature are then fused using a residual join to obtain the interactive feature. This process is repeated for features across all spectral bands to finally obtain the multispectral interactive feature F_inter.

[0082] Multispectral interactive features are input into a structure-texture decomposition network (SMR), which decomposes the input features into a structural feature map (F_struct) and a texture feature map (F_text). This network employs an encoder-decoder architecture. The encoder extracts multi-scale features through four convolutional layers, while the decoder restores spatial resolution through bilinear upsampling and convolution. Finally, two parallel convolutional layers output the structural and texture features respectively. The structural feature map is enhanced using an adaptive piecewise linear mapping. Specifically, based on the gray-level histogram of the structural feature map, three key points (d_min, d_mid, d_max) are automatically determined. Regions with gray values ​​less than d_min are linearly stretched, regions with gray values ​​between d_min and d_mid are compressed, regions with gray values ​​between d_mid and d_max are stretched, and regions with gray values ​​greater than d_max are compressed, resulting in the enhanced structural feature map F_struct_enhanced. The local variance of the texture feature map is calculated as a measure of texture complexity. Then, the enhancement level is adaptively adjusted based on texture complexity, with smaller enhancement coefficients for regions of high texture complexity and larger enhancement coefficients for regions of low texture complexity, resulting in the enhanced texture feature map F_text_enhanced. The enhanced structural feature map and texture feature map are then fused using an attention mechanism to obtain the enhanced feature map. The attention weights are generated by a lightweight network that takes the concatenation of structural and texture features as input and outputs two-channel attention maps, corresponding to the weights of the structural and texture features. The final enhanced feature map F_final = ws × F_struct_enhanced + wt × F_text_enhanced, where ws and wt are the attention weights.

[0083] The adaptive preprocessing method of this invention achieves effective complementarity and synergistic enhancement between different spectral data through a deep cascaded feature fusion network and a cross-spectral feature interaction module. The three-branch structure in the feature complementarity enhancement unit can simultaneously capture multi-scale contextual information, complex morphological features, and key region information, while the structure-texture decomposition network can specifically enhance features of different attributes. This enables the system to more accurately identify and locate various cracks under complex backgrounds and varying lighting conditions, significantly improving feature representation capabilities and environmental adaptability.

[0084] In one optional implementation, the enhanced feature map is input into a two-stream network, which includes a spatial feature processing path for extracting crack morphology and a frequency feature processing path for extracting crack texture. A spatial-channel cascaded attention module is introduced to adaptively weight and fuse the spatial and frequency features to obtain the fused features. The steps include:

[0085] The spatial feature processing pathway processes the enhanced feature map in parallel using multiple convolutional layers with different kernel sizes to obtain multi-scale spatial features. The multi-scale spatial features are then processed by multiple cascaded residual blocks and weighted to obtain spatial features. The frequency feature processing pathway processes the enhanced feature map using two-dimensional discrete wavelet transform to obtain approximation coefficients and detail coefficients. The detail coefficients are then enhanced by nonlinear mapping and reconstructed with the approximation coefficients to obtain frequency features.

[0086] The spatial features are input into the channel attention branch, and the correlation between feature channels is calculated through global average pooling and a learnable query matrix to obtain channel attention weights. The channel attention weights are weighted with the spatial features to obtain channel attention features. The frequency features are input into the spatial attention branch, and the results of max pooling and average pooling are processed through deformable convolution and positional encoding to obtain spatial attention weights. The spatial attention weights are weighted with the frequency features to obtain spatial attention features.

[0087] The channel attention features and the spatial attention features are weighted and fused to obtain the fused features.

[0088] For example, the spatial feature processing pathway uses three convolutional kernels of different sizes: 3×3, 5×5, and 7×7, with each convolutional layer having 64 output channels. These different sized kernels can capture crack morphology information at different scales. For instance, a 3×3 kernel is suitable for extracting features of small cracks, while a 7×7 kernel is better for extracting the overall morphology of wider cracks. For an input 256×256×3 enhanced feature map, each convolutional layer outputs a 256×256×64 feature map. These multi-scale features are further processed through four cascaded residual blocks, each containing two 3×3 convolutional layers and one skip connection. The use of residual blocks can alleviate the gradient vanishing problem in deep network training and maintain effective feature propagation. After residual block processing, 1×1 convolutions are used to reduce the channel dimensionality of the features at different scales, and a weighted sum is performed using weight coefficients to obtain the final spatial features with a size of 256×256×64.

[0089] The frequency feature processing pathway uses Haar wavelets as the basis function to perform first-level wavelet decomposition on the input 256×256×3 feature map, resulting in four sub-bands: approximation coefficients LL (128×128×3), horizontal detail coefficients LH (128×128×3), vertical detail coefficients HL (128×128×3), and diagonal detail coefficients HH (128×128×3). The approximation coefficients contain low-frequency information of the image, while the detail coefficients contain high-frequency information such as edges and textures. These three detail coefficients are enhanced using nonlinear mapping with 3×3 convolutional layers with a LeakyReLU activation function (negative slope of 0.2), each convolutional layer having 16 output channels. The enhanced detail coefficients, together with the approximation coefficients, are reconstructed using inverse wavelet transform to obtain the frequency features, with a size of 256×256×48. This frequency domain processing method effectively captures the texture features of cracks, especially being more sensitive to the edges and texture variations of subtle cracks.

[0090] The spatial-channel cascaded attention module consists of a channel attention branch and a spatial attention branch. The channel attention branch processes spatial features. First, it performs global average pooling on the 256×256×64 spatial features to obtain a 1×1×64 vector. Then, it performs matrix multiplication with a 64×64 learnable query matrix to calculate the correlation between feature channels, obtaining the channel attention weights. These weights are normalized to the 0-1 range using the sigmoid function and then multiplied channel-by-channel with the original spatial features to obtain the channel attention features. For example, the weight of the channel corresponding to the main crack distribution area might be 0.85, while the weight of the channel corresponding to the background area might be 0.3, thus highlighting the crack features.

[0091] The spatial attention branch processes frequency features, performing max pooling and average pooling on the 256×256×48 frequency features to obtain two 256×256×1 feature maps. These two feature maps are then processed by a 3×3 deformable convolution, which adaptively adjusts the receptive field shape according to the crack morphology. The convolved features are then element-wise added to a learnable 256×256 position encoding matrix to further incorporate positional information. Finally, a 1×1 convolution and a sigmoid function are used to obtain 256×256×1 spatial attention weights. These weights are then element-wise multiplied with the original frequency features to obtain the spatial attention features. For example, the spatial attention weight for the crack region might be 0.9, while the weight for non-crack regions might be 0.2, thus enhancing the feature representation of the crack region.

[0092] Finally, the channel attention features and spatial attention features are fused. First, a 1×1 convolution is used to adjust the two features to the same number of channels (e.g., 128). Then, a weighted sum is performed using learnable fusion weights (e.g., 0.6 and 0.4) to obtain the final fused feature with a size of 256×256×128. This adaptive weighted fusion method can flexibly adjust the importance of spatial and frequency features according to the characteristics of cracks in different scenarios, improving the model's ability to detect various types of cracks.

[0093] This invention effectively addresses the problem of insufficient representation by a single feature by processing spatial and frequency domain information in parallel. The spatial feature processing pathway accurately captures the morphology and location information of cracks, while the frequency feature processing pathway extracts the texture and detail features of cracks. The spatial-channel cascaded attention module achieves adaptive fusion of the two types of features, enabling the system to simultaneously focus on local details and global structure, significantly improving the ability to identify different types of cracks, especially the detection accuracy of cracks in blurred boundaries and complex backgrounds.

[0094] In one optional implementation, the step of obtaining spatial features through multi-scale spatial feature weighted fusion includes:

[0095] A multi-scale feature memory is constructed, comprising memory units corresponding to different convolutional kernel sizes. Each memory unit includes a feature representation vector, a confidence score, and a timestamp. The feature representation vector is obtained by extracting features from labeled samples and performing improved K-means clustering. The improved K-means clustering uses a weighted combination of the sum of squared intra-class distances and the logarithm of the number of samples in each class as the objective function. The confidence score is calculated as a weighted average of the proportion of sample categories and the feature quality evaluation score.

[0096] Calculate the cosine similarity between the multi-scale spatial features and the corresponding memory units in the multi-scale feature memory bank, calculate the adaptive temperature parameter based on the variance of the cosine similarity, and obtain the initial fusion weight by scaling the cosine similarity with the adaptive temperature parameter and then performing a softmax operation.

[0097] Calculate the feature distance and time interval between the multi-scale spatial features and the corresponding memory units. When the feature distance is greater than a distance threshold and the time interval is greater than a time threshold, update the feature representation vector of the memory unit using a dynamic momentum coefficient based on feature similarity, and update the confidence score of the memory unit based on the feature quality assessment score.

[0098] The initial fusion weights are multiplied by the confidence scores of the memory units to obtain the final fusion weights. The multi-scale spatial features are then weighted based on the final fusion weights and residual connections are constructed with the original features to obtain the spatial features.

[0099] For example, the multi-scale feature memory construction process first creates a corresponding set of memory cells for each convolutional kernel size (3×3, 5×5, and 7×7). Each memory cell contains three key components: a feature representation vector, a confidence score, and a timestamp. The feature representation vector is a 256-dimensional vector used to characterize the typical feature patterns of the cracks. The confidence score is a floating-point number between 0 and 1, representing the reliability of the memory cell. The timestamp is an integer value that records the time step of the most recent update of the memory cell.

[0100] For initializing the feature representation vectors, 10,000 image patches labeled with cracks are randomly selected from the training dataset. These image patches are then processed by a feature extraction network with a corresponding convolutional kernel size to obtain feature vectors. For example, for a memory unit corresponding to a 3×3 convolutional kernel, a feature extraction network with a 3×3 convolutional kernel is used to process the image patch, resulting in a 256-dimensional feature vector. Then, improved K-means clustering is performed on these feature vectors, with the number of clusters set to 20. The improved K-means clustering uses a weighted combination of the sum of squared intra-cluster distances and the logarithm of the number of samples in each class as the objective function. The sum of squared intra-cluster distances represents the sum of the squared Euclidean distances from each sample to the center of its respective class, and the logarithm of the number of samples in each class is the logarithm of the number of samples in each class. The weight of the sum of squared intra-cluster distances in the objective function is 0.7, and the weight of the logarithm of the number of samples in each class is 0.3. This setting avoids the severely imbalanced clustering results produced by the traditional K-means algorithm, such as some cluster centers containing only a small number of samples.

[0101] After clustering, each cluster center serves as a feature representation vector for a memory unit. The confidence score is calculated based on two factors: the proportion of samples in the cluster to the total number of samples (weight 0.4) and the feature quality assessment score (weight 0.6). The feature quality assessment score is obtained by calculating the average similarity between samples within the cluster and the cluster centers. For example, if a cluster contains 800 samples (8%) and the average cosine similarity between samples within the cluster and the centers is 0.85, then the confidence score for this memory unit is 0.4 × 0.08 + 0.6 × 0.85 = 0.542. The timestamp is initialized to 0, representing the time step at which the memory unit was created.

[0102] When processing multi-scale spatial features of the input, the cosine similarity between these features and the corresponding memory units in the memory bank is calculated. For example, the feature map (256×256×64) generated from a 3×3 convolutional kernel is first reduced to a 1×1×64 vector through global average pooling, and then transformed into a 256-dimensional vector through linear projection. The cosine similarity between this 256-dimensional vector and the feature representation vectors of the 20 memory units in the memory bank corresponding to the 3×3 convolutional kernel is calculated, resulting in 20 similarity values.

[0103] The variance of the similarity values ​​is calculated. If the variance is less than 0.05 (indicating a relatively uniform similarity distribution), the temperature parameter is set to 2.0; if the variance is greater than 0.2 (indicating that some memory cells are significantly more similar), the temperature parameter is set to 0.5; otherwise, the temperature parameter is calculated using linear interpolation, for example, when the variance is 0.1, the temperature parameter is 1.5. The smaller the temperature parameter, the steeper the probability distribution output by the softmax function, which tends to select the few memory cells with the highest similarity; the larger the temperature parameter, the flatter the probability distribution, and more memory cells will receive relatively balanced weights.

[0104] After dividing the cosine similarity by the temperature parameter, it is converted into initial fusion weights using the softmax function. For example, if the similarity value is [0.7, 0.65, 0.8, 0.5, ...] and the temperature parameter is 1.0, then the initial fusion weights after softmax processing may be [0.23, 0.22, 0.25, 0.18, ...].

[0105] For the dynamic update mechanism of the memory unit, the system calculates the Euclidean distance between the current feature and the feature representation vector of the memory unit, as well as the difference between the current time step and the timestamp of the memory unit. If the feature distance is greater than a preset distance threshold (e.g., 0.4) and the time interval is greater than a time threshold (e.g., 100 steps), the update process is triggered. The update process uses a dynamic momentum coefficient, which is adaptively calculated based on feature similarity. Specifically, the momentum coefficient α is calculated using the formula α = 0.9 × (1 - similarity). For example, if the feature similarity is 0.7, then the momentum coefficient is 0.9 × (1 - 0.7) = 0.27.

[0106] The feature representation vector of the memory unit is updated to a weighted combination of the current value and the new feature, with weights of (1-α) and α, respectively. For example, if the old feature representation vector is [0.2, 0.3, 0.4, ...], the new feature is [0.3, 0.2, 0.5, ...], and the momentum coefficient α is 0.27, then the updated feature representation vector is [(1-0.27)×0.2+0.27×0.3, (1-0.27)×0.3+0.27×0.2, (1-0.27)×0.4+0.27×0.5, ...]=[0.227, 0.273, 0.427, ...].

[0107] Simultaneously, the confidence score of the memory cell is also updated. The update method is a weighted average of the old confidence score and the new feature quality assessment score, with the weights also using the momentum coefficient. For example, if the old confidence score is 0.7, the new feature quality assessment score is 0.8, and the momentum coefficient is 0.27, then the updated confidence score is (1-0.27)×0.7+0.27×0.8=0.727. The timestamp of the memory cell is updated to the current time step, indicating the time of the most recent update.

[0108] The final fusion weight is calculated by multiplying the initial fusion weight by the confidence score of the memory cell. For example, if the initial fusion weight is [0.23, 0.22, 0.25, 0.18, ...], and the corresponding confidence score of the memory cell is [0.6, 0.7, 0.8, 0.5, ...], then the final fusion weight is [0.23×0.6, 0.22×0.7, 0.25×0.8, 0.18×0.5, ...] = [0.138, 0.154, 0.2, 0.09, ...]. The final fusion weight is normalized to ensure that the sum of all weights is 1.

[0109] The multi-scale spatial features are weighted using the final fusion weights. For example, for a feature map generated by a 3×3 convolution kernel, the weighted result with 20 memory units is the weighted sum of the feature representation vectors of all memory units. This weighted result is then transformed by backprojection to restore it to the same dimension as the original feature map (256×256×64). The same processing is applied to the feature maps generated by 5×5 and 7×7 convolution kernels, resulting in three weighted feature maps. These three feature maps are concatenated along the channel dimension to form a 256×256×192 feature map. Finally, a 1×1 convolution is used to reduce the dimension to 256×256×64, and a residual connection is constructed with the original feature map (i.e., the 256×256×64 feature map obtained by concatenating multi-scale features and then performing a 1×1 convolution) to obtain the final spatial features.

[0110] In practical applications, the size of the memory can be adjusted according to the application scenario and computing resources. For example, for crack detection in complex scenarios, the number of memory units corresponding to each convolutional kernel can be increased to 50 to capture more diverse feature patterns; while for simple scenarios, 20 memory units are sufficient. Similarly, the dimension of the feature representation vector can also be adjusted, which can be reduced to 128 dimensions when computing resources are limited, while it can be increased to 512 dimensions when high accuracy is required.

[0111] Existing technologies mainly achieve feature fusion through feature pyramid networks and multi-scale convolutions, but these methods often employ fixed weights or simple adaptive mechanisms, lacking the ability to memorize historical feature patterns. Especially in the field of crack detection, traditional feature fusion methods struggle to handle feature extraction under conditions of varying illumination and complex surface textures. This invention's multi-scale feature memory mechanism innovatively introduces long-term memory and dynamic update capabilities, enabling the network to accumulate historical knowledge and adapt to new sample features. Through improved K-means clustering and a time-interval-based dynamic update strategy, this method maintains the representativeness and timeliness of feature representations. The introduction of an adaptive temperature parameter addresses the problem of uneven feature similarity distribution, while the dynamic momentum coefficient balances stability and adaptability. This memory-enhanced feature fusion mechanism significantly improves the system's ability to identify similar crack patterns and its generalization ability to novel cracks.

[0112] In one optional implementation, based on the fused features, a multi-scale feature representation is constructed through a feature association graph network, and feature enhancement is performed by combining node importance assessment and complementary feature extraction modules. The feature reconstruction is completed using a reconstruction network with residual connections and progressive fusion, including the following steps:

[0113] The fused features are downsampled at different scales to obtain multi-scale feature levels, and a feature association graph network is constructed. The feature association graph network uses a block diagonal matrix to represent the feature association between adjacent scale levels. The connection weights between levels are initialized by a learnable orthogonal transformation matrix, and the connection weights are iteratively optimized according to the local structural similarity of the feature maps.

[0114] The importance score of each node in the feature association graph network is calculated. The importance score is determined by the feature response strength of the node itself and the weighted voting value of the adjacent level nodes. The weighted voting value is calculated based on the connection weight and feature difference between the nodes.

[0115] A complementary feature extraction module is constructed to process the multi-scale feature hierarchy sequentially, generate a feature selection gating signal based on the importance score, adaptively weight the features at each scale, use a deformable convolutional network to perform spatial adaptive sampling on the weighted features to obtain local enhanced features, and use a conditional generative adversarial network to fill in and enhance the crack region in the feature map to obtain complementary enhanced features.

[0116] The complementary enhancement features are input into the reconstruction network, and feature reconstruction is performed through residual connections and feature progressive fusion. The network parameters are optimized using a loss function that combines feature consistency constraints and local structure preservation.

[0117] For example, the input fused feature F is first downsampled using a 3×3 convolution kernel with a stride of 2 to obtain the first-level feature F1, with 64 feature channels; then, the same downsampling operation is applied to F1 to obtain the second-level feature F2, with 128 feature channels; and so on to obtain the third-level feature F3 and the fourth-level feature F4, with 256 and 512 feature channels, respectively. This forms a multi-scale feature hierarchy set {F0, F1, F2, F3, F4} from high resolution to low resolution, where F0 is the original fused feature.

[0118] Based on the aforementioned multi-scale feature hierarchy, a feature association graph network is constructed. This network uses a block diagonal matrix W to represent the association relationships between adjacent feature levels. For adjacent feature levels Fi and Fi+1, the association matrix Wi,i+1 is initialized as a learnable orthogonal transformation matrix Q, satisfying QTQ (the matrix product of the transpose of matrix Q and matrix Q itself) = I, where I is the identity matrix. Initialization uses a Glorot uniform distribution with parameters ranging from [-0.1, 0.1]. During training, the association matrix is ​​iteratively optimized based on the local structural similarity of the feature map. For each 3×3 region in the feature map, the structural similarity between the center pixel and its 8 surrounding pixels is calculated; the higher the similarity value, the greater the corresponding connection weight. When the similarity value is greater than 0.75, the weight increases by 0.1; when the similarity value is less than 0.25, the weight decreases by 0.1, ensuring the weight values ​​remain within the range [0, 1].

[0119] After the feature association graph network is constructed, the importance score of each node in the network is calculated. For node p in the i-th layer feature graph, its importance score S(p) consists of two parts: the node's own feature response strength and the weighted voting value from adjacent layer nodes. The feature response strength is calculated by applying a 3×3 Gaussian filter to the feature graph and taking the local maximum value, with a value range of [0, 1]. The weighted voting value V(p) is calculated based on the connection weights between nodes and the feature difference: when the feature difference between the node from the upper or lower layer and the current node is less than the threshold of 0.3, the voting value increases; when the difference is greater than 0.7, the voting value decreases. Finally, the importance score S(p) of node p is the weighted sum of the feature response strength and the normalized weighted voting value, with a weight ratio of 7:3, and the score range is [0, 1].

[0120] The complementary feature extraction module generates a feature selection gating signal G based on importance scores. For regions with scores higher than 0.6, the gating value is set to 1; for regions with scores lower than 0.2, the gating value is set to 0.3; and the gating values ​​for intermediate regions are linearly distributed proportionally. The gating signal G is multiplied element-wise with the original feature map to achieve adaptive weighting of the features.

[0121] For the weighted features, a deformable convolutional network is used for spatial adaptive sampling, employing a 5×5 deformable convolutional kernel with an offset range controlled within [-2, 2] pixels and a sampling number of 25. For detected crack regions (continuous regions with importance scores below 0.4), a conditional generative adversarial network is used for feature completion and enhancement. The generator adopts a U-Net architecture, containing a 4-layer encoder and a 4-layer decoder, with encoder channels of 64, 128, 256, and 512 respectively, and the decoder channel count is symmetrical to that of the encoder. The discriminator uses a 5-layer PatchGAN structure with a receptive field size of 70×70 pixels. The generator network is trained using a batch size of 16, a learning rate of 0.0002, and the Adam optimizer parameters β1 = 0.5 and β2 = 0.999.

[0122] After complementary enhancement features are generated, they are input into the reconstruction network, which employs an architecture of residual connections and progressive feature fusion. For each scale level feature Fi, features are first extracted through two 3×3 convolutional layers (keeping the number of channels constant), and then added to the original feature Fi via residual connections to obtain the enhanced feature Fi'. Starting from the lowest resolution F4', the features are upsampled to the resolution of F3' through transposed convolutions (4×4 convolutional kernels, stride of 2), and then fused with F3'. The fusion method involves concatenating channels and then adjusting the number of channels through a 1×1 convolution. This process is repeated step by step, upsampling at each level and fusing with the enhanced features of the corresponding level, ultimately obtaining a reconstructed feature with the same resolution as the original fused feature F0.

[0123] During network training, a loss function combining feature consistency constraints and local structure preservation is employed. Feature consistency constraints are implemented by calculating the pixel-level L1 distance between the reconstructed features and the original fused features, with a weight coefficient of 0.7. Local structure preservation constraints are implemented by calculating the consistency of local gradient directions in the feature maps, penalizing gradient direction differences within a 3×3 sliding window, with a weight coefficient of 0.3. The entire network is trained using the Adam optimizer with an initial learning rate of 0.001, decaying to 0.1 every 30 epochs, for a total of 100 epochs.

[0124] Existing technologies mainly employ feature pyramids, skip connections, and attention mechanisms to handle multi-scale features. These methods typically transmit feature information unidirectionally and lack explicit modeling of the relationships between features at different scales. This invention proposes a feature association graph network that constructs an explicit association structure between multi-scale features and introduces node importance evaluation and complementary feature extraction mechanisms. A block diagonal matrix is ​​used to represent the feature associations between adjacent scale levels, and connection weights are initialized using a learnable orthogonal transformation matrix to ensure the stability of information transmission. A node importance evaluation mechanism is designed, comprehensively considering the node's own feature response and voting information from adjacent levels, enabling the network to adaptively identify key feature regions. A complementary feature extraction module is constructed, achieving spatial adaptive sampling through deformable convolution, and innovatively introducing a conditional generative adversarial network to complete and enhance crack regions. This multi-scale collaborative and feature complementary processing approach significantly improves the system's detection capability and anti-interference ability for cracks of different sizes and shapes.

[0125] In one optional implementation, the step of constructing a joint optimization objective to train the two-stream network, wherein the joint optimization objective dynamically adjusts the network loss through multi-level morphological features and energy diffusion consistency constraints, includes:

[0126] Local morphological features of the crack image are extracted, including the crack curvature, directionality, and width change rate. The local morphological features are then mapped using a convolutional network to obtain local feature responses.

[0127] A regional correlation matrix is ​​constructed based on the local feature response. The regional correlation matrix represents the topological correlation strength between crack regions. The regional correlation matrix and the local feature response are input into a graph convolutional network to extract the structural correlation features of the cracks.

[0128] An attention mechanism is used to fuse the structural association features with image context information. A feature transformation network is used to generate global semantic features. The local morphological features, the structural association features, and the global semantic features are input into a weight generation network to obtain multi-level feature weights.

[0129] The feature energy field is calculated by weighted combination of the first-order gradient norm and the second-order gradient norm of the feature map. Based on the feature energy field, an energy-dependent diffusion coefficient is designed. The feature diffusion equation is constructed using the diffusion coefficient, and energy diffusion is performed on feature maps of different scales to calculate the diffusion consistency loss between feature maps of adjacent scales.

[0130] The multi-level feature weights are combined with the diffusion consistency loss to generate a joint optimization objective, which includes a weighted combination of local level loss, structural level loss, global level loss and feature consistency loss.

[0131] For example, when extracting local morphological features from a crack image, a Gaussian difference filter bank is used to perform multi-scale filtering on the input crack image to obtain response maps at different scales. Curvature features are calculated for each response map, specifically by calculating the rate of change of the image gradient direction to characterize the degree of crack curvature. Directional features are represented by calculating the principal direction and consistency of the image gradient; a gradient direction histogram can be used to statistically analyze the distribution of gradient directions within a local region. Width variation rate features are obtained by calculating the change in brightness distribution across the crack cross-section; local maximum suppression is used to identify crack edge points, and the distance between adjacent edge points represents the crack width. These local morphological features are organized into feature tensors and input into a feature mapping network consisting of eight convolutional layers with a kernel size of 3×3 and the number of channels increasing from 64 to 256. Downsampling is performed after every two layers to obtain local feature response maps.

[0132] When constructing the regional association matrix based on local feature responses, the feature map is divided into N×N blocks, and a feature vector is extracted from each block. The feature similarity between blocks is calculated using cosine similarity; blocks with a similarity value greater than a preset threshold of 0.7 are considered associated. The association strength is determined by both the similarity value and the spatial distance between blocks; the closer the blocks are and the higher their similarity, the stronger the association. The constructed regional association matrix has a size of (N×N)×(N×N), representing the topological association strength between any two blocks. This association matrix, along with the local feature responses, is input into a graph convolutional network containing three graph convolutional layers with 128, 256, and 512 output channels per layer, respectively, to extract structural association features reflecting the structural relationships between crack regions.

[0133] When fusing structurally related features with image contextual information using an attention mechanism, two mechanisms are employed: channel attention and spatial attention. Channel attention extracts channel statistical features through global average pooling and max pooling, and generates channel weights after passing through two fully connected layers (with the number of neurons being 1 / 16 and the number of feature channels, respectively). Spatial attention generates a spatial weight map through average pooling and max pooling along the channel dimension, followed by a 7×7 convolutional layer. The weighted features are then input into a feature transformation network, which consists of five residual blocks. Each residual block contains two 3×3 convolutional layers and an identity mapping branch, generating global semantic features that include high-level semantic information.

[0134] The weight generation network receives three types of feature inputs: local morphological features, structural correlation features, and global semantic features. The network consists of three parallel branches, each containing two 1×1 convolutional layers with one output channel, generating weight maps for the corresponding feature level. The final weights are normalized using a softmax function to ensure that the sum of the weights at the three levels is 1. During training, local layer weights are higher when the crack morphology is obvious; structural layer weights increase when the crack continuity is strong; and global layer weights dominate when the background is complex or the crack is blurred.

[0135] The feature energy field is calculated based on the gradient information of the feature map. The first-order gradient norm is calculated using the Sobel operator to calculate the gradients of the feature map in the horizontal and vertical directions, and then the gradient magnitude is calculated. The second-order gradient norm is calculated using the Laplacian operator to calculate the second derivative of the feature map. The feature energy field is a weighted sum of the first-order and second-order gradient norms, with weights of 0.7 and 0.3, respectively. An energy-dependent diffusion coefficient is designed based on the feature energy field. The diffusion coefficient is inversely proportional to the energy value, specifically exp(-energy value / control parameter), with the control parameter set to 0.15. A feature diffusion equation is constructed. Within each diffusion time step, the value of each point on the feature map is updated to a combination of its own value and the weighted sum of the values ​​of its surrounding points, with the weights determined by the diffusion coefficient. Ten iterations of diffusion are performed on feature maps at different scales (e.g., 1 / 2, 1 / 4, 1 / 8 of the original resolution) in the network. The diffusion consistency loss between adjacent scale feature maps is calculated, which is the mean square error between the diffusion results of the upsampled low-resolution feature map and the high-resolution feature map.

[0136] The joint optimization objective integrates multi-level feature weights and diffusion consistency constraints to form a comprehensive network training loss function. The multi-level feature weights are dynamically generated by the weight generation network based on features at different abstraction levels (local morphological features, structural correlation features, and global semantic features), reflecting the importance of each level of features to the current sample. These weights are directly used to adjust the corresponding level losses: the local level loss (binary cross-entropy loss) focuses on the accurate identification of the local crack morphology; the structural level loss (a combination of Dice loss and edge perception loss) focuses on the overall shape and edge accuracy of the crack; and the global level loss (weighted cross-entropy loss) ensures correct segmentation of crack regions in complex backgrounds, emphasizing the importance of crack pixels through a 5:1 weight ratio. Simultaneously, the diffusion consistency loss ensures the coherence of feature representations across multiple scales by calculating the consistency of feature maps at different scales (1 / 2, 1 / 4, and 1 / 8 of the original resolution) after energy diffusion, which is crucial for detecting cracks of different sizes. The final joint optimization objective combines the four losses proportionally: the weights of the three hierarchical losses are dynamically generated by the network (reflecting the characteristics of the current sample), while the feature consistency loss has a fixed weight of 0.2 (ensuring multi-scale consistency). This optimization strategy, which combines hierarchical structure with dynamic weights, enables the network to adapt to crack scenarios of different types and complexities, significantly improving detection accuracy and robustness.

[0137] Existing technologies primarily employ single or simple combinations of loss functions. While constraints based on physical models have some applications in certain fields, they are rarely closely integrated with the design of loss functions in deep learning. These limitations make it difficult for existing methods to simultaneously consider local details and global structure when dealing with cracks in mountainous tunnels with varied morphologies and complex backgrounds. This invention innovatively integrates multi-level morphological features and energy diffusion consistency constraints into a joint optimization objective, establishing a complete feature representation system from the local morphology of the crack to its global semantics. The introduction of feature energy fields and diffusion equations organically combines physical models with deep learning, enabling the network to maintain feature consistency and coherence at different scales. The design of multi-level loss functions ensures effective supervision of the network at different levels of abstraction, significantly improving the model's generalization ability and robustness, particularly its detection performance for non-standard morphological cracks and complex environments.

[0138] Figure 2This diagram illustrates the performance comparison of the technical solutions. Based on the specific data, the "Invention" demonstrates an overwhelming advantage across all evaluation dimensions. In terms of "average detection accuracy," the "Invention" achieves 94.5%, a 9.5 percentage point improvement over the "U-Net-based" model's 85.0%, indicating higher overall accuracy. The most significant performance difference lies in "robustness against complex backgrounds," where the "Invention" achieves a high score of 91.8%, far exceeding the "U-Net-based" model's 76.5%, demonstrating its stable and reliable detection capabilities even in complex interference environments. Furthermore, for the two metrics measuring the model's ability to handle complex situations and refine details—"recall rate of morphologically variable cracks" and "local detail accuracy"—the "Invention" achieves 92.3% and 93.7% respectively, again significantly outperforming the "U-Net-based" model's 78.9% and 82.1%.

[0139] In summary, the chart powerfully demonstrates that the "invention" employing joint optimization objectives achieves comprehensive and significant improvements over existing technologies in terms of overall crack detection performance, environmental adaptability, ability to capture irregular targets, and detail characterization accuracy.

[0140] In one optional implementation, the steps of performing morphological processing and connectivity analysis on the feature map output by the reconstructed network to extract the topological features of the cracks, and matching the topological features with the annotation information based on a deep metric learning method to classify the cracks and generate detection results include:

[0141] An adaptive structural element with adjustable orientation and scale parameters is constructed. The adaptive structural element is used to perform multi-directional dilation and multi-scale erosion operations on the feature map output by the reconstructed network to obtain morphological gradient features, which include outer boundary features, inner boundary features and comprehensive gradient features.

[0142] Eight-neighbor connectivity analysis is performed on the morphological gradient features to establish a hierarchical clustering tree. An adaptive segmentation threshold is determined based on the inter-class variance minimization criterion to obtain a connected component labeling map.

[0143] Distance transformation is performed on the connected component marker graph to extract the crack skeleton line, the branch points and endpoints on the crack skeleton line are identified, and a structural descriptor including orientation histogram, length distribution and width variation is constructed to obtain topological features;

[0144] The topological features and the morphological gradient features are concatenated to form a feature vector, which is then mapped to a metric space using a multilayer perceptron to generate a feature embedding representation.

[0145] Based on the crack type information in the annotation information, a combined loss function of positive sample pair loss and negative sample pair loss is constructed. The positive sample pair loss is calculated based on the Euclidean distance of sample pairs of the same category, and the negative sample pair loss is calculated based on the boundary constraints of sample pairs of different categories. The hard sample weight is calculated based on the metric distance between sample pairs, and the hard sample weight is adaptively adjusted using a momentum update mechanism. The hard sample weight is then integrated into the combined loss function.

[0146] The embedded feature representation is fused with global context features and input into a classifier for multi-class classification. The uncertainty and confidence are calculated based on the probability distribution of the classification results. Detection results containing crack category and severity are generated based on the location and morphological information in the annotation information.

[0147] For example, adaptive structural elements with adjustable orientation and scale parameters are constructed. These adaptive structural elements include linear and disk structural elements. The orientation angle of the linear structural elements can be adjusted in 15° steps within the range of 0° to 180°, and the length can be between 3 and 15 pixels. The radius of the disk structural elements can be adjusted between 2 and 8 pixels. These adaptive structural elements are used to perform multi-directional dilation and multi-scale erosion operations on the feature maps output by the reconstructed network. The multi-directional dilation operation uses the linear structural elements to dilate in four main directions: 0°, 45°, 90°, and 135°, resulting in dilated feature maps. The multi-scale erosion operation uses disk structural elements with radii of 2, 4, and 6 pixels, respectively, to erode the feature maps, resulting in eroded feature maps. The outer boundary features are calculated by the difference between the dilated and original feature maps; the inner boundary features are calculated by the difference between the original and eroded feature maps; and the comprehensive gradient features are calculated by the difference between the dilated and eroded feature maps.

[0148] Morphological gradient features are statistically analyzed for grayscale values ​​to construct pixel intensity histograms. A bottom-up hierarchical clustering method is used to group adjacent pixels according to grayscale similarity. During clustering, the initial number of categories is set to 256 (corresponding to grayscale levels), and categories with high similarity are gradually merged until the number of categories is reduced to a preset threshold (e.g., 20 categories). An adaptive segmentation threshold is determined based on the criterion of minimizing inter-class variance. The overall variance is decomposed into intra-class variance and inter-class variance, and the threshold that maximizes the inter-class variance is selected as the optimal segmentation point. For a pixel with an intensity of 127 in the crack image, if its variance with the surrounding area exceeds the preset threshold of 25, it is classified as an independent connected component. A connected component labeling algorithm is used to assign a unique identifier to each independent connected component, ultimately obtaining a connected component labeling map.

[0149] A distance transformation is performed on the connected component labeled graph. A distance field is generated by calculating the shortest distance from each point within the connected component to the boundary. For connected components with a label value of 1, the Euclidean distance from each pixel within the component to the boundary is calculated, resulting in a distance transformation graph. A non-maximum suppression algorithm is applied to the distance transformation graph to retain local maxima points, forming an initial skeleton line. Through an iterative refinement algorithm, non-skeleton points are removed, ultimately yielding a crack skeleton line with a width of 1 pixel. Branch points and endpoints on the crack skeleton line are identified. A branch point is defined as a skeleton point connecting 3 or more pixels in its neighborhood, and an endpoint is defined as a skeleton point connecting only 1 pixel in its neighborhood. Crack segmentation information is extracted by calculating paths from endpoints to branch points or from branch points to branch points. A structural descriptor including an orientation histogram, length distribution, and width variation is constructed to obtain topological features. The orientation histogram statistically analyzes the orientation distribution of each skeleton line segment within the range of 0° to 180°. The length distribution records the length statistics of each segment. The width variation is measured by mapping the skeleton line to the original connected component and measuring the width variation perpendicular to the skeleton line direction.

[0150] The topological features include 128-dimensional orientation histogram features, 64-dimensional length distribution features, and 32-dimensional width variation features; the morphological gradient features include 64-dimensional outer boundary features, 64-dimensional inner boundary features, and 128-dimensional comprehensive gradient features. These are concatenated to obtain a 480-dimensional feature vector. A multilayer perceptron (MLP) is used to map the feature vector to the metric space. The MLP contains three fully connected layers with 256, 128, and 64 nodes respectively. Each layer is followed by batch normalization and a ReLU activation function, ultimately outputting a 64-dimensional feature embedding representation.

[0151] The core of deep metric learning is constructing a combined loss function based on crack type information from the annotation information, combining positive and negative sample pair losses. This step aims to train the model to learn a feature space that can cluster cracks of the same type and separate cracks of different types, thereby achieving high-precision crack classification. For each batch of training samples, a set of sample pairs is constructed, including positive sample pairs (samples of the same class) and negative sample pairs (samples of different classes). The positive sample pair loss calculates the Euclidean distance between the feature embeddings of sample pairs of the same class (e.g., two longitudinal crack samples) and compares it with a boundary value of 1.0. The smaller the distance, the lower the loss value, prompting the network to map samples of the same class to nearby positions. The negative sample pair loss calculates the Euclidean distance between the feature embeddings of sample pairs of different classes (e.g., longitudinal crack and transverse crack samples), requiring it to be greater than a boundary value of 5.0. The larger the distance, the lower the loss value, prompting the network to explicitly separate samples of different classes. The hard sample weight mechanism addresses the sample imbalance problem during training by identifying and emphasizing samples that the network struggles to classify correctly. For positive sample pairs, samples with an Euclidean distance greater than 0.8 are considered difficult-to-distinguish similar samples, and their weights are doubled. For negative sample pairs, samples with an Euclidean distance less than 2.0 are considered easily confused samples, and their weights are tripled. A momentum update mechanism is used to adaptively adjust the weights of difficult samples to avoid drastic weight fluctuations; specifically, the current weight is multiplied by 0.9, and the newly calculated weight is multiplied by 0.1. Finally, the weights of difficult samples are directly multiplied by the loss value of the corresponding sample pair and integrated into the combined loss function, making the network pay more attention to difficult samples and improving the model's discriminative ability.

[0152] The fusion of feature embedding representations with global contextual features aims to combine local structural information and global environmental information to enhance classification accuracy. Global contextual features are extracted from the original feature map using a spatial pyramid pooling module, containing pooling layers of four scales (1×1, 2×2, 3×3, and 6×6) to capture contextual information from different receptive fields. The fusion method involves concatenating features and then adjusting the number of channels using a 1×1 convolution to obtain a 128-dimensional enhanced feature vector. This vector is input into a three-layer fully connected network for classification, with 64 and 32 hidden layer nodes respectively. The output layer corresponds to the number of crack categories (typically 4-8 categories, such as transverse cracks, longitudinal cracks, and mesh cracks). The classifier uses the Softmax activation function to obtain the probability distribution of each category, with the highest probability value used as the confidence index and the entropy value of all category probabilities used as the uncertainty measure. Simultaneously, the system evaluates the severity of cracks based on features such as skeleton line length, average width, and texture complexity, combined with preset severity evaluation criteria (e.g., slight: width <1mm; moderate: width 1-3mm; severe: width >3mm). The final inspection report includes key information such as crack location coordinates, crack type and its confidence level, severity rating, total length and average width. Based on the confidence level classification (high: >0.85; medium: 0.6-0.85; low: <0.6), low-confidence inspection results are automatically marked as "requires manual confirmation", providing a comprehensive and reliable basis for subsequent maintenance decisions.

[0153] The morphological processing and connectivity analysis methods of this invention achieve accurate extraction of crack topology through adaptive structuring elements and hierarchical clustering trees. The deep metric learning framework innovatively fuses topological and visual features, and optimizes the feature space distribution through positive and negative sample loss and hard sample mining strategies. In particular, the combination of uncertainty and confidence-based multi-class classification methods enables the system to provide reliable crack classification results and risk assessments. This approach, combining structural feature analysis and deep learning, significantly improves the accuracy of crack classification and the reliability of detection reports, providing a solid basis for subsequent maintenance decisions.

[0154] A second aspect of the present invention provides an electronic device, comprising:

[0155] processor;

[0156] Memory used to store processor-executable instructions;

[0157] The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0158] A third aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0159] This invention can be a method, apparatus, system, and / or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of the invention.

[0160] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A deep learning-based intelligent recognition and detection method for mountain tunnel cracks, characterized in that, The method comprises the following steps: Collecting multi-spectral image data with annotation information, which contains the position, shape and type information of the cracks; Adaptive pre-processing the multi-spectral image data, extracting the low-frequency and high-frequency features of each spectral data and performing weighted fusion, and performing dynamic range enhancement on the fused image to obtain an enhanced feature map; Input the enhanced feature map into a double-flow network, which contains a spatial feature processing path for extracting crack morphology and a frequency feature processing path for extracting crack texture, and introduces a spatial-channel cascade attention module to adaptively weight the spatial and frequency features, and obtains a fused feature; Based on the fused feature, a multi-scale feature representation is constructed through a feature correlation graph network, a feature enhancement is performed by combining node importance evaluation and complementary feature extraction module, and a feature reconstruction is completed by using a reconstruction network with residual connection and progressive fusion, specifically including: obtaining multi-scale feature levels by down-sampling the fused feature through different scales, constructing a feature correlation graph network, the feature correlation graph network uses a block diagonal matrix to represent the feature correlation between adjacent scale levels, initializes the connection weight between levels through a learnable orthogonal transformation matrix, and iteratively optimizes the connection weight according to the local structural similarity of the feature map; Calculate the importance score of each node in the feature correlation graph network, the importance score is determined by the feature response strength of the node itself and the weighted voting value of the adjacent level nodes, and the weighted voting value is calculated based on the connection weight and the feature difference between nodes; Construct a complementary feature extraction module to sequentially process the multi-scale feature levels, generate a feature selection gating signal based on the importance score, adaptively weight each scale feature, use a deformable convolution network to spatially adaptively sample the weighted feature to obtain a locally enhanced feature, and use a conditional generative adversarial network to complete and enhance the crack area in the feature map to obtain a complementary enhanced feature; Input the complementary enhanced feature into the reconstruction network, perform feature reconstruction through residual connection and feature progressive fusion, and optimize the network parameters by using a loss function combining feature consistency constraint and local structure preservation; Construct a joint optimization target to train the double-flow network, the joint optimization target dynamically adjusts the network loss by multi-level morphological feature and energy diffusion consistency constraint, specifically including: extracting the local morphological features of the crack image, the local morphological features include the curvature, directionality and width change rate of the crack, and mapping the local morphological features through a convolution network to obtain a local feature response; Based on the local feature response, a region correlation matrix is constructed, which represents the topological correlation strength between crack regions, and the region correlation matrix and the local feature response are input into a graph convolution network to extract the structural correlation features of the crack; Fuse the structural correlation features with the image context information by using an attention mechanism, generate a global semantic feature through a feature transformation network, input the local morphological features, the structural correlation features and the global semantic features into a weight generation network to obtain multi-level feature weights; The feature energy field is calculated by a weighted combination of the first-order gradient norm and the second-order gradient norm of the feature map, and an energy-dependent diffusion coefficient is designed based on the feature energy field; a feature diffusion equation is constructed using the diffusion coefficient to perform energy diffusion on feature maps of different scales and calculate the diffusion consistency loss between adjacent scale feature maps; The multi-level feature weight and the diffusion consistency loss are combined to generate a joint optimization target, which includes a weighted combination of local level loss, structure level loss, global level loss and feature consistency loss; The feature map output by the reconstruction network is subjected to morphological processing and connectivity analysis to extract the topological structure features of the cracks, and the topological structure features are matched with the label information based on a deep metric learning method to classify the cracks and generate a detection result.

2. The method of claim 1, wherein, The steps of adaptively pre-processing the multi-spectral image data, extracting low-frequency and high-frequency features of each spectral data, and performing weighted fusion, and dynamically enhancing the range of the fused image to obtain an enhanced feature map include: Multi-scale decomposition is performed on the multi-spectral image data to obtain low-frequency and high-frequency features; The low-frequency and high-frequency features are input into a deep cascaded feature fusion network, which includes a feature complementary enhancement unit, the feature complementary enhancement unit includes a dilated convolution branch, a deformable convolution branch and a local attention branch, the dilated convolution branch uses multiple dilated convolution layers with different expansion rates to extract multi-scale features; the offset field of the deformable convolution branch is adaptively generated by the input features through an offset generation network; the attention weight of the local attention branch is adaptively determined according to the crack scale, and is multiplied element by element with the deformable convolution feature map; the output features of the three branches are integrated through adaptive weights to obtain enhanced low-frequency and high-frequency features; The enhanced low-frequency and high-frequency features are input into a cross-spectral feature interaction module to establish a feature correspondence relationship by calculating a similarity matrix between different spectral features; the different spectral features are weighted and reorganized based on the similarity matrix to obtain multi-spectral interaction features; The multi-spectral interaction features are input into a structure-texture decomposition network to decompose the input features into structure feature maps and texture feature maps, the structure feature maps are enhanced using adaptive piecewise linear mapping, and the texture feature maps are processed using an adaptive enhancement function based on texture complexity; the enhanced structure feature maps and texture feature maps are fused through an attention mechanism to obtain enhanced feature maps.

3. The method of claim 1, wherein, The enhanced feature maps are input into a dual-flow network, the dual-flow network includes a spatial feature processing path for extracting crack morphology and a frequency feature processing path for extracting crack texture, and a spatial-channel cascaded attention module is introduced to adaptively weight and fuse the spatial features and frequency features to obtain fused features. The spatial feature processing path processes the enhanced feature map through convolution layers with different kernel sizes in parallel to obtain multi-scale spatial features, and the multi-scale spatial features are processed through a plurality of serial residual blocks and weighted fusion to obtain spatial features; the frequency feature processing path processes the enhanced feature map through two-dimensional discrete wavelet transform to obtain approximation coefficients and detail coefficients, and the detail coefficients are enhanced through nonlinear mapping and reconstructed with the approximation coefficients to obtain frequency features; The spatial feature input channel attention branch is input into the channel attention branch, the correlation between feature channels is calculated through global average pooling and a learnable query matrix to obtain channel attention weights, and the channel attention weights are weighted with the spatial features to obtain channel attention features; the frequency features are input into the spatial attention branch, and the spatial attention weights are obtained through the maximum pooling and average pooling results, deformable convolution and position encoding processing, and the spatial attention weights are weighted with the frequency features to obtain spatial attention features; The channel attention features and the spatial attention features are weighted and fused to obtain fusion features.

4. The method of claim 3, wherein, The step of weighting and fusing the multi-scale spatial features to obtain spatial features comprises: A multi-scale feature memory bank is constructed, the multi-scale feature memory bank includes memory units corresponding to different convolution kernel sizes, each memory unit includes a feature representation vector and a confidence score, the feature representation vector is obtained by feature extraction on labeled samples and improved K-means clustering, the improved K-means clustering adopts a weighted combination of intra-class distance sum of squares and logarithmic term of class sample number as a target function, and the confidence score is calculated according to the weighted combination of sample category proportion and feature quality evaluation score; The cosine similarity between the multi-scale spatial features and the corresponding memory units in the multi-scale feature memory bank is calculated, the adaptive temperature parameter is calculated based on the variance of the cosine similarity, the cosine similarity is scaled by the adaptive temperature parameter, and the initial fusion weight is obtained by softmax operation; The feature distance and the time interval between the multi-scale spatial features and the corresponding memory units are calculated, when the feature distance is greater than the distance threshold and the time interval is greater than the time threshold, the feature representation vector of the memory unit is updated using a dynamic momentum coefficient based on feature similarity, and the confidence score of the memory unit is updated based on the feature quality evaluation score; The initial fusion weight is multiplied by the confidence score of the memory unit to obtain the final fusion weight, the multi-scale spatial features are weighted based on the final fusion weight, and the spatial features are obtained by constructing a residual connection with the original features.

5. The method of claim 1, wherein, The steps of performing morphological processing and connectivity analysis on the feature map output by the reconstruction network, extracting the topological structure features of the cracks, and matching the topological structure features with the labeled information based on deep metric learning method, classifying the cracks and generating detection results comprise: A self-adaptive structuring element with adjustable direction parameters and scale parameters is constructed, and a morphological gradient feature is obtained by performing multi-direction dilation operation and multi-scale erosion operation on the feature map output by the reconstruction network using the self-adaptive structuring element. The morphological gradient feature includes an outer boundary feature, an inner boundary feature, and a comprehensive gradient feature. An eight-neighbor connectivity analysis is performed on the morphological gradient feature, a hierarchical clustering tree is established, an adaptive segmentation threshold is determined based on the minimum inter-class variance criterion, and a connected domain label map is obtained. A distance transform is performed on the connected domain label map, a crack skeleton line is extracted, branch points and end points on the crack skeleton line are identified, a structural descriptor including a direction histogram, a length distribution, and a width variation is constructed, and a topological structure feature is obtained. The topological structure feature and the morphological gradient feature are spliced to form a feature vector, the feature vector is mapped to a metric space by a multilayer perceptron, and a feature embedding representation is generated. A combined loss function of positive sample pair loss and negative sample pair loss is constructed based on the crack type information in the annotation information, a difficult sample weight is calculated based on the metric distance between sample pairs, the difficult sample weight is adaptively adjusted using a momentum update mechanism, and the difficult sample weight is integrated into the combined loss function. The feature embedding representation and the global context feature are fused, input into a classifier for multi-class classification, uncertainty and confidence are calculated based on the probability distribution of the classification result, and a detection result including crack category and severity is generated based on the position information and morphological information in the annotation information.

6. An electronic device, comprising: comprise: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to invoke the instructions stored in the memory to perform the method of any one of claims 1 to 5.

7. A computer-readable storage medium having stored thereon computer program instructions, wherein, The computer program instructions, when executed by the processor, implement the method of any one of claims 1 to 5.

Images