High-precision crack detection method based on multi-scale feature fusion and direction distribution

Abstract

The invention discloses a high-precision crack detection method based on multi-scale feature fusion and direction distribution, and the method comprises the steps: firstly, constructing a multi-scale feature fusion module, and aims at efficiently integrating inherent but not fully utilized multi-level features in a UNet encoder, therefore, the comprehensive perception capability of the model on the target from local details to global context is enhanced. And then, aiming at the problem that a serial structure is easily interfered by noise in a complex scene, a parallel attention unit is built in a fusion module, and can evaluate feature importance in channel and spatial dimensions in parallel and dynamically screen and strengthen key information, so that the signal-to-noise ratio of output features is remarkably improved. Through the curve geometric prior introduced by the direction distribution module, the model has the ability of enhancing the crack trend signal in a targeted manner, and the inherent problem of low signal-to-noise ratio of a tiny target in a feature space is effectively overcome.

Description

Technical Field

[0001] This invention relates to the field of crack detection technology, specifically a high-precision crack detection method based on multi-scale feature fusion and orientation allocation. Background Technology

[0002] In current crack detection research, image segmentation-based methods (such as UNet and YOLO) dominate. These methods, with their powerful feature learning capabilities, have achieved high recognition accuracy on standard crack datasets. However, their ability to detect minute cracks in images is quite limited. Because general models tend to capture macroscopic contour features, subtle crack morphology is often ignored or lost during segmentation, leading to unsatisfactory performance in practical detection tasks.

[0003] Current technology addresses this problem in the following ways:

[0004] Multi-scale feature fusion and parallel attention: Addressing the issue that UNet possesses excellent multi-scale features but fails to utilize them efficiently, this method integrates feature information scattered across various layers through a feature fusion strategy to enhance the model's panoramic perception of the target. Furthermore, addressing the inherent limitations of existing sequential structures in complex scenes—namely, the difficulty in effectively suppressing irrelevant interference—this method incorporates parallel attention within the fusion module. This allows for the parallel selection and enhancement of important features, significantly improving the signal-to-noise ratio of the fused features.

[0005] Orientation Assignment Module: To address the issues of weak response to microcrack features and susceptibility to background interference in existing models, we designed an orientation assignment module. This module constructs a direction-sensitive filter based on prior knowledge of the crack curve to amplify the realistic crack signal and suppress noise. This design effectively improves the accuracy of microcrack localization and identification, enhancing the model's robustness.

[0006] Existing technologies have the following shortcomings:

[0007] Insufficient robustness in detecting minute cracks: Existing image segmentation models perform well on controlled experimental datasets, but their generalization ability is limited. When applied to highly complex and variable real-world environments, these models are susceptible to multiple interference factors such as lighting, weather, occlusion, and viewpoint, leading to a significant degradation in recognition performance. This phenomenon is particularly prominent when detecting minute cracks with low signal-to-noise ratios, severely limiting the reliability and practicality of the detection system.

[0008] Ignoring the visual structural priors of cracks: Most existing methods rely solely on data annotation for end-to-end supervised learning, failing to effectively utilize the inherent visual characteristics of cracks themselves (such as strong spatial continuity in morphology). This limitation makes it difficult for the model to learn essential features with strong generalization capabilities, resulting in unstable recognition accuracy when facing new scenes. In particular, when predicting continuous long cracks, it is prone to producing fragmented or incomplete erroneous segmentation results. Summary of the Invention

[0009] The purpose of this invention is to provide a high-precision crack detection method based on multi-scale feature fusion and orientation assignment. The steps of this high-precision crack detection method based on multi-scale feature fusion and orientation assignment are as follows:

[0010] Step 1: Image preprocessing: The input is an RGB image, and the output is a black and white binary image, corresponding to the binary classification problem of background and crack. Before building the network, the crack image needs to be collected and labeled at the pixel level. When loading the dataset, image enhancement and preprocessing are performed, namely image scaling, random rotation, random brightness, and normalization to mean.

[0011] Step 2: Construct the overall multi-scale feature fusion network architecture: Establish the UNet model architecture, and add a combination of feature fusion module (FFM), feature enhancement module (FEM) and decoder (Dec) after layers enc-2, enc-3, and enc4. The last layer feature of the encoder, enc-5, is fed into the non-local context module (NLB) and then upsampled by 2 times before being fed into the FEM after layer enc-4. At the same time, the original feature of enc-5 is upsampled by 16 times for calculating the structural loss. The FFM after enc-4 receives the low-scale features of layer enc-5, performs preliminary fusion of the features and feeds them into the subsequent FEM. The FEM also receives features from the NLB after layer enc-5 and features upsampled by 2 times, performs feature enhancement and feeds them into Dec to obtain the fused features of that layer. This process is repeated to calculate the output of the Dec module of each layer, and finally inputs it into the Dec module after enc-1 to obtain the final fused features and the predicted mask. The final fused features are then subjected to orientation assignment (OA) and the orientation loss is calculated.

[0012] Step 3: Feature Fusion Module (FFM): The Feature Fusion Module (FFM) accepts two inputs, namely high-scale features and low-scale features, and is used to perform preliminary fusion of features at the two scales.

[0013] Step 4: Feature Enhancement Module (FEM): The Feature Enhancement Module (FEM) also accepts high-scale features and low-scale features. First, it directly adds and fuses the features of the two scales. Then, it sends the result to the parallel attention module for feature filtering. The filtered results are multiplied and fused with the original features. Finally, the features of the two scales are concatenated together as the output fused enhanced feature.

[0014] Step 5: Parallel Attention Module: The parallel attention module mainly divides the original features into global and local features and then fuses them to output attention.

[0015] Step 6: Direction Assignment Module: First, using the following curve to support the difference of the Gaussian function, a set of multi-scale curve structure templates with fixed parameters is generated:

[0016]

[0017] in, , Represents pixel coordinates in the image domain. , , Let be the variances and parameters of the two-dimensional Gaussian distribution in the horizontal and vertical directions, respectively. Used to control the curvature of the Gaussian support domain;

[0018] After determining the structural template, a multi-class cross-entropy loss function is used to evaluate the difference between the predicted radiation pattern pd and the reference radiation pattern td. The calculation formula is as follows:

[0019]

[0020] in, This indicates the orientation category of the crack pixel, while and They represent the corresponding direction vectors. and The Middle The values ​​of each element are determined by the fact that the crack pixels occupy a very small portion of the image, referring to the orientation pattern. Background labels have been omitted to facilitate modeling of curve patterns across all regions.

[0021] Preferably, the image is enhanced and preprocessed to a range of mean [0.485, 0.456, 0.406] and standard deviation [0.229, 0.224, 0.225], and the labels are converted, with pixel 0 converted to label 0 and pixel 255 converted to label 1.

[0022] Preferably, in step 3, specifically, the high-scale features will go through a computation chain of deconvolution, average pooling, and convolution, while the low-scale feature part focuses on minimizing feature omissions by using a single convolutional layer and multiplying it twice with the output of the high-scale feature computation chain to fuse the features.

[0023] Preferably, in step 5, specifically, global features are obtained through average pooling, and the global and local features are extracted and added together, and the final feature attention is obtained by applying the Sigmoid activation function.

[0024] Compared with the prior art, the beneficial effects of the present invention are:

[0025] 1. Significantly improved detection capability for minute cracks: By introducing curve geometric priors through the direction assignment module, the model possesses the ability to specifically enhance crack orientation signals, effectively overcoming the inherent challenge of low signal-to-noise ratio for small targets in the feature space. Combining multi-scale feature fusion and parallel attention mechanism for dynamic filtering of key information, the system can accurately separate subtle crack features from complex backgrounds, thereby achieving high-sensitivity detection of minute cracks.

[0026] 2. Enhancing Model Robustness in Complex Scenes: The parallel attention mechanism, by simultaneously evaluating channel and spatial dimensions, achieves collaborative screening of key features, significantly improving the signal-to-noise ratio of the output features. Simultaneously, the directional filter actively suppresses irrelevant noise, further reducing the impact of complex environmental factors such as illumination variations and texture interference. This dual optimization enables the model to maintain stable and reliable detection performance when facing diverse interference in real-world scenes.

[0027] 3. Optimize feature utilization efficiency and model generalization ability: The traditional feature utilization model has been revolutionized, upgrading multi-scale features from simple connections to an integrated "fusion-filtering" mechanism, significantly improving feature utilization efficiency. The introduction of the direction assignment module transforms data-driven passive learning into geometrically prior-guided active feature enhancement. This shift reduces the model's dependence on large amounts of labeled data, strengthens its adaptability to unseen scenes, and thus significantly improves the model's generalization performance. Attached Figure Description

[0028] Figure 1 This is a diagram of the overall architecture of the present invention;

[0029] Figure 2 This is a diagram of the NLB module of the present invention;

[0030] Figure 3 This is a diagram of the FFM module of the present invention;

[0031] Figure 4 This is a diagram of the FEM module of the present invention;

[0032] Figure 5 This is a diagram of the parallel attention module of the present invention. Detailed Implementation

[0033] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0034] Example 1

[0035] A high-precision crack detection method based on multi-scale feature fusion and orientation assignment is characterized by the following steps:

[0036] Step 1: Image Preprocessing: The input is an RGB image, and the output is a black and white binary image, corresponding to a binary classification problem of background and crack. Before building the network, the crack image needs to be collected and labeled at the pixel level. When loading the dataset, image enhancement and preprocessing are performed, namely image scaling, random rotation, random brightness, normalization to the mean [0.485, 0.456, 0.406] and standard deviation [0.229, 0.224, 0.225] range, and the labels are transformed, pixel 0 is transformed into label 0, and pixel 255 is transformed into label 1.

[0037] Step 2: Construct the overall multi-scale feature fusion network architecture: Establish the UNet model architecture, adding a combination of Feature Fusion Module (FFM), Feature Enhancement Module (FEM), and Decoder (Dec) after layers enc-2, enc-3, and enc-4. The features from the last layer of the encoder, enc-5, are fed into the Non-Local Context Module (NLB) and then upsampled by 2x before being fed into the FEM after layer enc-4. Simultaneously, the original features of enc-5 are upsampled by 16x for calculating the structural loss. The FFM after enc-4 receives the low-scale features from layer enc-5, performs preliminary fusion, and then feeds them into the subsequent FEM. The FEM also receives features from the NLB after layer enc-5 and the upsampled features, performs feature enhancement, and then feeds them into Dec to obtain the fused features for that layer. This process is repeated for each layer, calculating the output of the Dec module for each layer. Finally, the output is fed into the Dec module after enc-1 to obtain the final fused features and the predicted mask. Orientation assignment (OA) is performed on the final fused features, and the orientation loss is calculated. The overall architecture is as follows: Figure 1 As shown, the architecture of NLB is as follows: Figure 2 As shown.

[0038] Step 3: Feature Fusion Module (FFM): The Feature Fusion Module (FFM) accepts two inputs: high-scale features and low-scale features. It performs an initial fusion of features from the two scales. Specifically, the high-scale features undergo a computation chain of deconvolution, average pooling, and convolution. The low-scale features, focusing on minimizing feature loss, are processed using a single convolutional layer, which is then multiplied twice by the output of the high-scale feature computation chain to fuse the features. The module architecture is as follows: Figure 3 As shown.

[0039] Step 4: Feature Enhancement Module (FEM): The Feature Enhancement Module (FEM) also accepts high-scale and low-scale features. First, it directly adds and fuses the features from both scales. Then, the result is fed into a parallel attention module for feature filtering. The filtered results are multiplied and fused with the original features. Finally, the features from both scales are concatenated as the output fused enhanced feature. The module architecture is as follows: Figure 4 As shown.

[0040] Step 5: Parallel Attention Module: The parallel attention module mainly divides the original features into global and local features, then fuses them to output attention. Specifically, it obtains global features through average pooling, extracts features from both global and local features, adds them together, and applies a sigmoid activation function to obtain the final feature attention. The module architecture is as follows: Figure 5 As shown.

[0041] Step 6: Direction Assignment Module: First, using the following curve to support the difference of the Gaussian function, a set of multi-scale curve structure templates with fixed parameters is generated:

[0042]

[0043] in, , Represents pixel coordinates in the image domain. , , Let be the variances and parameters of the two-dimensional Gaussian distribution in the horizontal and vertical directions, respectively. Used to control the curvature of the Gaussian support domain;

[0044] After determining the structural template, a multi-class cross-entropy loss function is used to evaluate the difference between the predicted radiation pattern pd and the reference radiation pattern td. The calculation formula is as follows:

[0045]

[0046] in, This indicates the orientation category of the crack pixel, while and They represent the corresponding direction vectors. and The Middle The values ​​of each element are determined by the fact that the crack pixels occupy a very small portion of the image, referring to the orientation pattern. Background labels have been omitted to facilitate modeling of curve patterns across all regions.

[0047] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A high-precision crack detection method based on multi-scale feature fusion and orientation assignment, characterized in that: The steps of this high-precision crack detection method based on multi-scale feature fusion and orientation assignment are as follows: Step 1: Image preprocessing: The input is an RGB image, and the output is a black and white binary image, corresponding to the binary classification problem of background and crack. Before building the network, the crack image needs to be collected and labeled at the pixel level. When loading the dataset, image enhancement and preprocessing are performed, namely image scaling, random rotation, random brightness, and normalization to mean. Step 2: Construct the overall multi-scale feature fusion network architecture: Establish the UNet model architecture, and add a combination of feature fusion module, feature enhancement module and decoder after layers enc-2, enc-3 and enc4. The last layer feature enc-5 of the encoder is sent to the non-local context module and then upsampled by 2 times before being sent to the FEM after layer enc-4. At the same time, the original feature of enc-5 is upsampled by 16 times for calculating the structural loss. The FFM after enc-4 receives the low-scale feature of layer enc-5, performs preliminary fusion of the feature and sends it to the subsequent FEM. The FEM also receives the NLB and upsampled features from layer enc-5, performs feature enhancement and sends it to Dec to obtain the fused feature of this layer. The output of the Dec module of each layer is calculated in turn, and finally input into the Dec module after enc-1 to obtain the final fused feature and the predicted mask. The final fused feature is oriented and the directional loss is calculated. Step 3 Feature Fusion Module: The Feature Fusion Module (FFM) accepts two inputs, high-scale features and low-scale features, and is used to initially fuse the features of the two scales. Step 4 Feature Enhancement Module: The Feature Enhancement Module (FEM) also accepts high-scale and low-scale features. First, it directly adds and fuses the features of the two scales. Then, it sends the result to the parallel attention module for feature filtering. The filtered results are multiplied and fused with the original features. Finally, the features of the two scales are concatenated together as the output fused enhanced feature. Step 5: Parallel Attention Module: The parallel attention module mainly divides the original features into global and local features and then fuses them to output attention. Step 6: Direction Assignment Module: First, using the following curve to support the difference of the Gaussian function, a set of multi-scale curve structure templates with fixed parameters is generated: in, , Represents pixel coordinates in the image domain. , , Let be the variances and parameters of the two-dimensional Gaussian distribution in the horizontal and vertical directions, respectively. Used to control the curvature of the Gaussian support domain; After determining the structural template, a multi-class cross-entropy loss function is used to evaluate the difference between the predicted radiation pattern pd and the reference radiation pattern td. The calculation formula is as follows: in, This indicates the orientation category of the crack pixel, while and They represent the corresponding direction vectors. and The Middle The values ​​of each element are determined by the fact that the crack pixels occupy a very small portion of the image, referring to the orientation pattern. Background labels have been omitted to facilitate modeling of curve patterns across all regions.

2. The high-precision crack detection method based on multi-scale feature fusion and orientation allocation according to claim 1, characterized in that: Image enhancement and preprocessing are performed to bring the mean [0.485, 0.456, 0.406] and standard deviation [0.229, 0.224, 0.225] within the range, and the labels are transformed, with pixel 0 being transformed into label 0 and pixel 255 being transformed into label 1.

3. The high-precision crack detection method based on multi-scale feature fusion and orientation allocation according to claim 1, characterized in that: In step 3, specifically, the high-scale features will go through a computation chain of deconvolution, average pooling, and convolution, while the low-scale feature part focuses on minimizing feature omissions by using a convolutional layer and multiplying it twice with the output of the high-scale feature computation chain to fuse the features.

4. The high-precision crack detection method based on multi-scale feature fusion and orientation allocation according to claim 1, characterized in that: In step 5, specifically, global features are obtained through average pooling, and the global and local features are extracted and added together. The final feature attention is obtained by applying the Sigmoid activation function.

Images