Complex environment pavement crack detection method fusing high and low frequency characteristics

Abstract

The invention discloses a complex environment pavement crack detection method fusing high and low frequency characteristics. According to the method, effective separation and complementary enhancement of high-frequency information and low-frequency information are realized by using the frequency collaborative enhancement module and adopting a decomposition type feature processing mechanism aiming at the obtained to-be-detected road surface image data, so that the technical problems of low detection precision and reliability in the prior art are effectively solved; the technical effect of accurately detecting the pavement cracks in the complex environment is achieved.

Description

Technical Field

[0001] This invention relates to the field of pavement crack detection technology, and in particular to a method for detecting pavement cracks in complex environments that integrates high and low frequency characteristics. Background Technology

[0002] Cracks are a common problem in road maintenance, and their number and width gradually increase over time. Severe cracks not only shorten the service life of roads and increase road maintenance costs, but also pose safety hazards and affect passenger comfort. Therefore, it is crucial to regularly inspect road cracks, assess the extent of damage, and repair them promptly.

[0003] Existing detection methods generally focus on modeling the spatial structure, lacking in-depth research on the frequency domain characteristics of crack images. Furthermore, in complex environments, such as those with shadows, lane markings, slippery surfaces, and low-light conditions at night, the contrast between the crack and the background is significantly reduced, and image noise is significantly increased. This makes it difficult for the model to stably extract the complete crack region, leading to missed detections and false detections, severely impacting the accuracy and reliability of the detection. Summary of the Invention

[0004] This invention provides a method for detecting road surface cracks in complex environments by integrating high and low frequency features, which solves the technical problems of low detection accuracy and reliability in the prior art and achieves the technical effect of accurate detection of road surface cracks in complex environments.

[0005] This invention provides a method for detecting pavement cracks in complex environments by integrating high- and low-frequency features, comprising:

[0006] Step 1: Image of the road surface to be inspected As input, a strided convolution is performed to downsample and adjust the number of channels, followed by batch normalization and non-linear activation function operations, and then another downsampling and normalization to obtain the initial scale feature map. ;in, The initial input height for the image. The initial input width of the image. This represents the initial number of input channels for the image.

[0007] Step 2: Convert the initial scale feature map obtained in Step 1 into a single image. As input, four stages of processing are performed sequentially. Each stage includes downsampling and frequency-co-enhanced residual building blocks, resulting in feature maps at four different scales. ;in, Feature map of the first stage The number of channels, For the second stage feature map The number of channels, For the third stage feature map The number of channels, For the fourth stage feature map The number of channels;

[0008] Step 3: Combine the four feature maps obtained in Step 2 at different scales In As input, the data is fused by convolution to achieve channel and scale alignment, then fed into a lightweight context aggregation unit for multi-branch semantic fusion. Finally, a class probability map is output via convolution and upsampled to the original image size. ;in, The number of channels in the probability graph;

[0009] Step 4: Calculate the probability graph obtained in Step 3. The input is thresholded, and the output is the road crack segmentation result. .

[0010] Specifically, the frequency-coordinated enhancement residual building block in step 2 performs the following sub-steps sequentially on the input features:

[0011] Step 2.1: Convert the initial scale feature map obtained in Step 1 into a single image. The input is batch normalized to obtain the output. ;

[0012] Step 2.2: Take the output of Step 2.1 As input, the frequency-coordinated enhancement module produces the output. ;

[0013] Step 2.3: Convert the output of Step 2.2 Compared with the initial scale feature map obtained in step 1 Add to get the output ;

[0014] Step 2.4: Convert the output of Step 2.3 As input, the output is obtained after batch normalization. ;

[0015] Step 2.5: Convert the output of Step 2.4 As input, the signal is processed through a locally enhanced feedforward neural network to obtain the output. ;

[0016] Step 2.6: Convert the output of Step 2.5 The output of step 2.3 The output is obtained by adding features. Thus, the feature map is obtained. ;

[0017] Step 2.7: Convert the output of Step 2.6 As input to step 2.1, execute step 2.1. Step 2.6 yields And so on, until we obtain... and .

[0018] Specifically, the frequency co-enhancement module in step 2.2 performs the following sub-steps sequentially on the input features:

[0019] Step A1: Take the output of step 2.1 As input, low-frequency extraction and high-frequency residual decomposition are performed; the low-frequency residual is obtained through average pooling operation, i.e. Then, the low-frequency residual is upsampled and aligned to the size, i.e., the low-frequency residual. Subsequently, the high-frequency residuals were obtained through a single subtraction. ;

[0020] Step A2: The high-frequency residual obtained in step A1 The high-frequency enhancement branch is used to enhance edge and fine texture information to obtain the output. The low-frequency residual Input low-frequency enhancement branch to improve contour and region consistency to obtain output , will the Upsampling to With consistent spatial dimensions, the enhancement results of the two branches are stitched together along the channel dimension to obtain the desired result. ;

[0021] Step A3: Convert the output of step A2 The input is fed into the attention fusion unit to adaptively allocate weights for high-frequency and low-frequency signals and map them back to the preset number of channels to obtain the output. ;

[0022] Step A4: Take the output of step A3 The output of step 2.1 The output of the frequency co-enhancement module is obtained by adding the components together. .

[0023] Specifically, the high-frequency enhancement branch in step A2 performs the following sub-steps on the input features sequentially:

[0024] Step B1: The high-frequency residual obtained in step A1 The input features are fed into the three convolutional pairs in parallel to obtain... , and ;

[0025] Step B2: Convert the three outputs , and Element-by-element addition yields ;

[0026] Step B3: Obtain the result from step B2 The input is used for pointwise convolution and normalized activation to obtain... ;

[0027] Step B4: Obtain the result from step B3 The high-frequency residual obtained in step A1 Intermediate features are formed by adding elements one by one;

[0028] Step B5: Apply pointwise convolution, nonlinear activation, batch normalization, pointwise convolution, nonlinear activation, and batch normalization sequentially to the intermediate features obtained in Step B4, and output the result. .

[0029] Specifically, all convolutions in the high-frequency enhancement branch are strip-shaped depthwise convolutions to capture crack features at different scales.

[0030] Specifically, the low-frequency enhancement branch in step A2 performs the following sub-steps on the input features sequentially:

[0031] Step C1: Process the low-frequency residual obtained in step A1 Perform pointwise convolution and average pooling sequentially to form at least three layers of low-frequency features, and obtain the following outputs: , , and ;

[0032] Step C2: Output from step C1 , and High-frequency components are extracted through average pooling and differencing operations. Subsequently, these high-frequency features are processed by convolution, normalization, and a sigmoid activation function to enhance effective details. Finally, they are fused back to the original input via residual connections. This mechanism compensates for local high-frequency information by applying it to low-frequency features at different levels, while preserving global structural information, resulting in… , and ;

[0033] Step C3: Obtain from each layer , and Upsampled to the same spatial size and convolved with shallow pointwise features from the input path obtained by splicing in the channel dimension ;

[0034] Step C4: The splicing result obtained in step C3 Applying pointwise convolution, the output yields the result of the low-frequency enhancement branch. .

[0035] Specifically, the local enhancement feedforward neural network in step 2.5 performs the following sub-steps sequentially on the input features:

[0036] Step D1: Output of step 2.4 First, pointwise convolution is applied to complete channel fusion, and then channel-wise convolution is applied for spatial filtering to explicitly enhance the response to crack discontinuities and sharp high-frequency changes, thus obtaining the output. ;

[0037] Step D2: The output of step D1 By sequentially applying a nonlinear activation function and a random dropout operation, the output is obtained. ;

[0038] Step D3: The output of step D2 Large-kernel depthwise convolution is used to obtain long-range complex texture context, followed by channel fusion using pointwise convolution to obtain... ;

[0039] Step D4: The results obtained in step D3 By applying random dropout, the output of the locally enhanced feedforward neural network is obtained. .

[0040] One or more technical solutions provided in this invention have at least the following technical effects or advantages:

[0041] 1. Based on the acquired road surface image data to be detected, the frequency collaborative enhancement module adopts a decomposition feature processing mechanism to achieve effective separation and complementary enhancement of high-frequency and low-frequency information. This effectively solves the technical problems of low detection accuracy and reliability in existing technologies and achieves the technical effect of accurate detection of road surface cracks in complex environments.

[0042] 2. By utilizing a locally enhanced feedforward neural network, the correlation between pixels is improved and the network's sensitivity to local details is enhanced. By fusing large kernel convolution and depthwise separable convolution, the receptive field is effectively expanded and the contextual relationship between features is enhanced, thereby achieving accurate detection of road surface cracks in complex environments.

[0043] 3. By using average pooling to separate high- and low-frequency features, the accuracy of crack detection in complex road surfaces is improved.

[0044] 4. The high-frequency enhancement branch is used to process the high-frequency components, and the low-frequency enhancement branch is used to extract low-frequency information through hierarchical downsampling, which further improves the accuracy of crack detection in complex road surfaces.

[0045] Therefore, this invention has high accuracy in crack detection of high-resolution road images affected by factors such as lighting and climate in complex environments, which helps in road damage assessment and repair guidance. Attached Figure Description

[0046] Figure 1 This is an overall flowchart of the method for detecting road surface cracks in complex environments that integrates high and low frequency features provided in this embodiment of the invention;

[0047] Figure 2 This is a network structure diagram of the frequency co-enhancement module FCEM in the complex environment pavement crack detection method that integrates high and low frequency features provided in the embodiments of the present invention;

[0048] Figure 3 This is a network structure diagram of the high-frequency enhancement branch HFE in the complex environment pavement crack detection method that integrates high and low frequency features provided in the embodiments of the present invention;

[0049] Figure 4 This is a network structure diagram of the low-frequency enhancement branch LFE in the complex environment pavement crack detection method that integrates high and low frequency features provided in the embodiments of the present invention;

[0050] Figure 5 This is a network structure diagram of the locally enhanced feedforward neural network LE-FFN in the complex environment pavement crack detection method that integrates high and low frequency features provided in the embodiments of the present invention;

[0051] Figure 6 This is a comparison chart of the detection effect of the complex environment road surface crack detection method that integrates high and low frequency features provided in the embodiments of the present invention with that of the existing technology. Detailed Implementation

[0052] This invention provides a method for detecting pavement cracks in complex environments by integrating high and low frequency features. This solves the technical problems of low accuracy and reliability in existing technologies and achieves the technical effect of accurate detection of pavement cracks in complex environments.

[0053] The technical solutions in the embodiments of the present invention are designed to achieve the above-mentioned technical effects, and the overall concept is as follows:

[0054] The present invention provides a method for detecting pavement cracks in complex environments by integrating high and low frequency features, comprising:

[0055] Step 1: Image Input and Initialization Processing. Acquire the image of the road surface to be detected. After performing one downsampling and channel adjustment via strided convolution, batch normalization and pass-through (BN) are then executed sequentially. To obtain the initial scale features .

[0056] Step 2: Multi-stage feature extraction (convolutional encoder). Four stages are executed sequentially, each including "downsampling". Frequency Co-enhanced Residual Building Block (FCERB)”, respectively, yielded:

[0057]

[0058] Step 3: Multi-scale feature fusion (decoding). Only features from the last three stages are fused. First through Convolution completes channel alignment and... Upsampling to Same spatial scale; then input lightweight context aggregation unit for multi-branch semantic fusion; finally... The convolution outputs a probability map and upsamples it to the original image size, resulting in... .

[0059] Step 4: Output. For the probability graph... Thresholding is performed to output the road crack segmentation results. .

[0060] The FCERB (Frequency Co-enhanced Residual Building Block) includes, in sequence: BN FCEM Residual 1 BN LE-FFN Residual 2.

[0061] Residual 1: The output of FCEM is added to the input of the building block;

[0062] Residual 2: The output of LE-FFN is added to the intermediate features of residual 1 as the block output.

[0063] The FCEM (Frequency Collaborative Enhancement Module) module includes the following sub-steps:

[0064] A1 sub-step: Low-frequency extraction and high-frequency residual decomposition: Perform average pooling on the input features to obtain low-frequency representations and upsample and align them; obtain high-frequency residuals by a single subtraction method (subtract the aligned low frequencies from the input).

[0065] A2 Sub-step: Branch Enhancement: High-frequency residuals are fed into HFE to enhance edges / fine textures; low-frequency characterizations are fed into LFE to enhance contours / region consistency, and the LFE output is upsampled to the same scale as HFE.

[0066] A3 Sub-step: Attention Fusion: Concatenate the two results in the channel dimension, input CA-Fuse to adaptively assign high / low frequency weights and map them back to the preset number of channels.

[0067] A4 Sub-step: Residual Back-Add: Add the fusion result to the FCEM input to obtain the FCEM output.

[0068] The HFE (High-Frequency Enhancement Branch) includes the following sub-steps:

[0069] B1 sub-step: Multi-branch asymmetric convolution group: Three-way parallel convolution pairs ( Multi-scale high-frequency textures are extracted using strip-shaped / directional large kernels;

[0070] B2 sub-step: branching and merging: element-wise addition of the three outputs;

[0071] B3 sub-step: pointwise convolution + GELU + BN;

[0072] B4 Sub-step: Residual addition: Add to the HFE input to form intermediate features;

[0073] B5 Sub-step: Channel dilation and bottleneck compression: Pointwise convolution Pointwise convolution Output the HFE result.

[0074] The above convolution is preferably implemented using strip-shaped depth convolution or an equivalent lightweight implementation to capture crack features at different scales.

[0075] The LFE (Low-Frequency Enhancement Branch) includes the following sub-steps:

[0076] C1 sub-step: Generation of low frequencies step by step: Execute "Pointwise Conv(dim)" sequentially. dim / 2) Avg-Pool" forms a low-frequency pyramid of no less than three layers;

[0077] C2 sub-step: High-Frequency Refinement Extractor (HFRE): Average pooling and differencing are performed on the low-frequency features of each layer to extract high-frequency components at the corresponding scale. The high-frequency components are then processed... After convolution, normalization, and sigmoid noise suppression gain, the low-frequency features of the layer are injected back through residual connections to achieve layer-by-layer refinement.

[0078] C3 Sub-step: Scale Alignment and Stitching: Upsample the outputs of each HFRE layer to a uniform scale and stitch them together with the shallow Pointwise Conv from the input path. Features are spliced ​​in the channel dimension;

[0079] C4 Sub-step: Channel Recovery: Restore the channel to dim using Pointwise Conv and output the LFE result.

[0080] LE-FFN (Local-Enhancement Feed-Forward Network) includes the following sub-steps:

[0081] D1 sub-step: Inverse Depthwise Separable Convolution (IDSC): First, pointwise convolution (PW) is applied to complete channel mixing / dilation, and then channelwise convolution (DW) is applied to perform spatial filtering, so as to explicitly enhance the response to crack discontinuities and sharp high-frequency changes.

[0082] D2 sub-step: GELU and random discard;

[0083] D3 sub-step: Large-Kernel Depthwise Separable Convolution (LDSC): DW with large kernel or dilated large kernel is used to obtain long-distance and complex texture context, and then PW is used to fuse channels to improve the accuracy of edge and microstructure depiction.

[0084] D4 sub-step: Random discard: As the output of LE-FFN, it is used for the second residual of the building block.

[0085] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0086] like Figure 1 As shown in the embodiment of the present invention, the method for detecting pavement cracks in complex environments by integrating high and low frequency features includes:

[0087] Step 1: Image Input and Initialization Processing. Input the image of the road surface to be detected. As input, a strided convolution is performed to achieve downsampling and channel number adjustment, followed by batch normalization (BN) and Gaussian Error Linear Unit (GELU) operations, and then another downsampling and normalization to obtain the initial scale feature map. ;in, The initial input height for the image. The initial input width of the image. This represents the initial number of input channels for the image.

[0088] Step 2: Multi-stage feature extraction. The initial scale feature map obtained in Step 1... As input, four stages of processing are performed sequentially. Each stage includes downsampling and a Frequency-Collaborative Enhancement Residual Block (FCERB), resulting in feature maps at four different scales. ;in, Feature map of the first stage The number of channels (64), For the second stage feature map The number of channels (128), For the third stage feature map The number of channels (256), For the fourth stage feature map The number of channels (512);

[0089] Specifically, within the frequency-co-enhanced residual building block in step 2, the following sub-steps are performed sequentially on the input features:

[0090] Step 2.1: Convert the initial scale feature map obtained in Step 1 into a single image. The input is batch normalized to obtain the output. ;

[0091] Step 2.2: Take the output of Step 2.1 As input, it passes through the Frequency Collaborative Enhancement Module (FCEM) to obtain the output. ;

[0092] In this embodiment, the frequency co-enhancement module in step 2.2 performs the following sub-steps sequentially on the input features:

[0093] Step A1: Take the output of step 2.1 As input, low-frequency extraction and high-frequency residual decomposition are performed; the low-frequency residual is obtained through average pooling operation, i.e. Then, the low-frequency residual is upsampled and aligned to the size, i.e., the low-frequency residual. Subsequently, the high-frequency residuals were obtained through a single subtraction. ;

[0094] Step A2: Branch Enhancement. The high-frequency residuals obtained in Step A1... The high-frequency enhancement branch (HFE) is used to enhance edge and fine texture information to obtain the output. ; low-frequency residual The input low-frequency enhancement branch (LFE) is used to enhance contour and region consistency to obtain the output. ,Will Upsampling to With consistent spatial dimensions, the enhancement results of the two branches are stitched together along the channel dimension to obtain the desired result. ;

[0095] Furthermore, within the high-frequency enhancement branch in step A2, the following sub-steps are performed sequentially on the input features:

[0096] Step B1: Multi-branch asymmetric convolution group. The high-frequency residuals obtained in step A1... The input features are fed into three convolutional pairs in parallel, respectively. Convolution followed by convolution, Convolution followed by Follow Convolution yields , and ;

[0097] Step B2: Branch Merge. Merge the three outputs. , and Element-by-element addition yields ;

[0098] Step B3: Obtain the result from step B2 The input is used for pointwise convolution and normalized activation to obtain... ;

[0099] Step B4: Obtain the result from step B3 The high-frequency residual obtained in step A1 Intermediate features are formed by adding elements one by one;

[0100] Step B5: Channel Expansion and Bottleneck Compression. Apply pointwise convolutions sequentially to the intermediate features obtained in Step B4 (channel dimensions increased from...). Expansion to Nonlinear activation, batch normalization, pointwise convolution (channel dimension is changed from nonlinear activation, batch normalization, pointwise convolution) Compress to Nonlinear activation and batch normalization are applied to obtain the output. .

[0101] To capture crack features at different scales, all convolutions in the high-frequency enhancement branch are strip-shaped depthwise convolutions.

[0102] The low-frequency enhancement branch in step A2 performs the following sub-steps on the input features in sequence:

[0103] Step C1: Take the contents of step A1 Low-frequency generation is performed step by step. The low-frequency residuals obtained in step A1 are then processed. Perform pointwise convolution sequentially (reducing the channel dimension from...) Adjusted to Using average pooling (AvgPool), at least three layers of low-frequency features are formed, resulting in outputs of respectively , , and ;

[0104] Step C2: High-Frequency Refinement Extractor (HFRE). This involves refining the output of step C1... , and High-frequency components are extracted through average pooling and differencing operations. Subsequently, these high-frequency features are processed by 1×1 convolution, normalization, and a sigmoid activation function to enhance effective details. Finally, they are fused back to the original input via residual connections. This mechanism compensates for local high-frequency information by applying it to low-frequency features at different levels, while preserving global structural information, resulting in… , and ;

[0105] Step C3: Scale alignment and splicing. The results obtained from each layer... , and Upsampled to the same spatial size and convolved with shallow pointwise features from the input path obtained by splicing in the channel dimension ;

[0106] Step C4: Channel Restoration. This involves restoring the stitched result obtained in step C3. Applying pointwise convolution (restoring the channel dimension to dim) yields the result of the low-frequency enhancement branch. .

[0107] Step A3: Convert the output of step A2 The input is fed into the Channel Attention-Fuse (CA-Fuse) unit to adaptively allocate weights for high-frequency and low-frequency signals and map them back to the preset number of channels to obtain the output. ;

[0108] Step A4: Take the output of step A3 The output of step 2.1 The summation yields the output of the frequency co-enhancement module. .

[0109] Step 2.3: First residual calculation, using the output of Step 2.2 Compared with the initial scale feature map obtained in step 1 Add to get the output ;

[0110] Step 2.4: Convert the output of Step 2.3 As input, the data is processed by batch normalization (BN) to obtain the output. ;

[0111] Step 2.5: Convert the output of Step 2.4 As input, the signal is processed through a Local-Enhancement Feed-Forward Network (LE-FNN) to obtain the output. ;

[0112] In this embodiment, the local augmentation feedforward neural network in step 2.5 performs the following sub-steps sequentially on the input features:

[0113] Step D1: Inverse Depthwise Separable Convolution (IDSC) stage. This is the output of step 2.4. First, pointwise convolution is applied to complete channel fusion, and then channel-wise convolution is applied for spatial filtering to explicitly enhance the response to crack discontinuities and sharp high-frequency changes, thus obtaining the output. ;

[0114] Step D2: GELU and random discard. The output of step D1... By sequentially applying a nonlinear activation function and a random dropout operation, the output is obtained. ;

[0115] Step D3: Large Kernel Depthwise Separable Convolution (LDSC). This involves applying the output of Step D2... Large-kernel depthwise convolution is employed to obtain long-range complex texture context, followed by pointwise convolution for channel fusion, thereby improving the depiction accuracy of crack edges and microstructures. ;

[0116] Step D4: Randomly discard. The results obtained in step D3... Applying random dropout, the output of the locally enhanced feedforward neural network is obtained. .

[0117] Step 2.6: Second residual calculation, using the output of step 2.5 The output of step 2.3 The output is obtained by adding features. Thus, the feature map is obtained. ;

[0118] Step 2.7: Convert the output of Step 2.6 As input to step 2.1, execute step 2.1. Step 2.6 yields And so on, until we obtain... and .

[0119] Step 3: Multi-scale feature fusion. The four feature maps obtained in Step 2 at different scales are fused together. In As input for fusion, through After convolution completes channel and scale alignment, the input is fed into a lightweight context aggregation unit for multi-branch semantic fusion, and finally... The convolution outputs a class probability map and upsamples it to the original image size. ;in, The number of channels in the probability graph is equal to the number of categories (2);

[0120] Step 4: Calculate the probability graph obtained in Step 3. The input is thresholded, and the output is the road crack segmentation result. .

[0121] The method for detecting road surface cracks in complex environments by integrating high and low frequency features, provided by the present invention, will be described in further detail below with reference to the accompanying drawings. Specifically, the method includes the following specific steps:

[0122] (1) Obtain the road crack dataset

[0123] Download the following road crack datasets from the internet: Sun520 (sunny day), Rain365 (rainy day), and BJN260 (nighttime). All datasets were acquired using high-resolution images (3968×2240 resolution). The Sun520 dataset consists of 520 images taken in sunny weather, covering various factors that can affect crack detection accuracy, such as shadows, lane markings, and stains, making it more difficult to distinguish cracks from the background. The Rain365 dataset contains 365 daytime images after rainfall, further subdivided into three subsets: completely wet, partially wet, and partially dry. The BJN260 dataset consists of 260 images taken at night in Beijing, covering different types of light sources, lighting intensities, and complex road surface conditions.

[0124] This invention performs data augmentation on the training set, including 180-degree rotation, horizontal flipping, and adding zero-mean Gaussian noise with a standard deviation of 0.01, expanding the training set size to four times its original size. For the specific task of road crack detection, images are scaled to 2048 x 1152 pixels, and then randomly cropped to 768 x 768 pixels for both training and testing. Each dataset is randomly allocated as the training and testing set in an 8:2 ratio.

[0125] (2) Frequency Co-enhanced Residual Building Block (FCERB)

[0126] The FCERB, as the basic unit of the encoder, has the following structure: BN FCEM (Residual 1) BN LE-FFN (Residual 2).

[0127] Among them, residual 1: the output of FCEM is added element by element to the input of the building block to obtain intermediate features;

[0128] Residual 2: The output of LE-FFN is added to the aforementioned intermediate features to serve as the output of the building block. This dual residual design improves the stability of gradient propagation while preserving the original semantics.

[0129] (3) Frequency Co-enhancement Module (FCEM)

[0130] FCEM, as a core module of FCERB, such as Figure 2 As shown, it includes the following steps:

[0131] Step A1: Low-frequency extraction and high-frequency residual decomposition: Perform average pooling on the input features to obtain low-frequency representations. Then, upsample and align to the spatial size of the input to obtain L; at the only subtraction node, subtract L from the input to obtain the high-frequency residual H.

[0132] Step A2: Branch Enhancement:

[0133] High-frequency branched HFE: Multi-branch asymmetric convolution enhancement of H to highlight edges and fine cracks;

[0134] Low-frequency branch LFE: for A low-frequency pyramid is constructed and refined at multiple levels to enhance the consistency of the outline and region; the output of LFE is upsampled to the same spatial size as HFE.

[0135] Step A3: Attention Fusion (CA-Fuse): The enhancement results of the two branches are concatenated in the channel dimension. The input channel attention fusion unit adaptively assigns high / low frequency weights and maps them back to the preset number of channels.

[0136] Step A4: Residual back-addition: Add the fusion result to the FCEM input element by element to obtain the FCEM output.

[0137] (4) High-frequency enhanced branch HFE

[0138] The structure of HFE is as follows Figure 3 As shown, it includes the following steps:

[0139] Step B1: Multi-branch asymmetric convolution group: Parallel three-way convolution pairs, respectively (Available for depth separability).

[0140] Step B2: Branches merge, and the results from the three paths are added element by element.

[0141] Step B3: Pointwise Convolution and Activation Normalization GELU BN.

[0142] Step B4: Residual Summation: Summing the output of step B3 with the input of HFE to obtain intermediate features.

[0143] Step B5: Channel Expansion – Bottleneck Compression: Pointwise Conv GELU Pointwise Conv GELU+BN outputs the HFE result.

[0144] (5) Low-frequency enhancement branch LFE

[0145] The structure of LFE is as follows Figure 4 As shown, it includes the following steps:

[0146] C1 step: Generation of low frequencies step by step. Execute "Pointwise Conv" sequentially along the input path. ) Avg-Pool" forms The low-frequency pyramid of the layer;

[0147] Step C2: High-Frequency Refinement (HFRE). For each layer's low-frequency features, calculate the difference between it and the low-pass filter of the same layer to obtain the high-frequency components. The high-frequency components are then processed... Convolution, normalization, and Sigmoid are used to obtain a gated map; the low-frequency features of this layer are back-injected as residuals to achieve edge and detail compensation at this scale.

[0148] Step C3: Scale Alignment and Channel Concatenation. Upsample the outputs of each HFRE layer to a uniform size and concatenate them with the shallow Pointwise Conv from the input path. Features are spliced ​​in the channel dimension;

[0149] C4 step: Channel restoration. Pointwise Conv restores the channels to dim and outputs the LFE result.

[0150] (6) Locally Enhanced Feedforward Neural Network (LE-FFN)

[0151] The structure of LE-FFN is as follows Figure 5 As shown, it includes the following steps:

[0152] D1 Step: Inverse Depthwise Separable Convolution (IDSC). Pointwise Convolution (PW) The sequence of channel-wise convolution (DW) is to first perform functional channel mixing / expansion, and then perform spatial filtering to enhance the response to discontinuous cracks and sharp high-frequency changes.

[0153] D2 steps: GELU and Dropout;

[0154] Step D3: Deeply Separable Convolution with Large Kernel (LDSC). A large kernel or dilated large kernel DW (e.g., 7 / 11 / 21, dilation 1-3) is used to obtain long-range and complex texture context, followed by PW to complete channel fusion;

[0155] Step D4: Randomly drop the data. The output is used as the second residual input for FCERB. The drop rate at both Dropout points can be set to... .

[0156] (7) Multi-scale feature fusion and decoding

[0157] Fusion only First use Convolution unifies the three features into the number of channels. And Upsampling to Consistent space dimensions; then input lightweight context aggregation units (such as the LightHamburger head), in local branches (DWConv) ), large core branch ( Lightweight fusion is performed between low-frequency pooling branches; ultimately... Convolution output The channel probability map is upsampled to the original image size to obtain accurate segmentation results of road cracks.

[0158] (8) Training process

[0159] Loss function: weighted sum of binary cross-entropy (BCE) and Dice loss.

[0160] Optimization and scheduling: AdamW (initial learning rate 6e-4, weight decay 0.05).

[0161] Data augmentation: random scaling ), random flipping, etc.

[0162] (9) Reasoning and Post-processing

[0163] The input is scaled to the resolution supported by the network by either the long or short side, and then subjected to sliding window (stride 512) or full-graph inference to obtain a probability map P. P is then thresholded to obtain a binary crack mask.

[0164] (10) Software and hardware environment and deployment

[0165] Training environment: PyTorch / AdamW, GPU is NVIDIA RTX series. Inference can be implemented on TensorRT (FP16) or OpenVINO (INT8).

[0166] Example

[0167] The detection effectiveness of this invention can be further illustrated by the following experiments:

[0168] In terms of experimental hardware configuration: the computing platform is equipped with an NVIDIA GeForce RTX4090 graphics card with 24GB of video memory.

[0169] Performance comparison with other road crack detection algorithms on road crack datasets: Sun520 (sunny day), Rain365 (rainy day), and BJN260 (nighttime):

[0170] In challenging rainy and nighttime scenarios, FE-SegNeX (the method of this invention) utilizes a high-low frequency collaborative enhancement mechanism to achieve a balanced optimization between precision and recall, thereby significantly improving overall performance metrics. Although FE-SegNeXt's precision value is not the highest among all models, this is because the model is designed to prioritize improving recall to ensure the extraction of as many crack features as possible and accurately, rather than indicating that its precision is inherently low.

[0171] Table 1. Comparison of evaluation metrics between the two methods on the SUN520 dataset.

[0172]

[0173] Table 2 Comparison of evaluation metrics for the two methods on the Rain365 dataset

[0174]

[0175] Table 3 Comparison of evaluation metrics for the two methods on the BJN260 dataset

[0176]

[0177] To provide a more intuitive comparison of detection results, this invention visualizes the detection results of various algorithms, such as... Figure 6 As shown in the figure, FE-SegNeXt excels in capturing detailed cracks, especially faint or blurred cracks in the image, further validating its robustness in complex environments. The road crack detection algorithm designed in this invention is suitable for crack detection under complex lighting conditions, wet road surfaces, and noisy environments. It not only maintains high precision but also achieves a more ideal balance between recall and precision.

[0178] In summary, the method for detecting pavement cracks in complex environments that integrates high and low frequency features provided in this embodiment of the invention is suitable for detecting pavement cracks in complex environments.

[0179] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0180] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0181] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0182] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0183] Any aspects of this invention not described in detail in the embodiments are well-known techniques to those skilled in the art. Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this invention and not to limit it. Although this invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of this invention without departing from the spirit and scope of this invention, and all such modifications and substitutions should be covered within the scope of the claims of this invention.

Claims

1. A method for detecting pavement cracks in complex environments by integrating high and low frequency features, characterized in that, include: Step 1: Image of the road surface to be inspected As input, a strided convolution is performed to downsample and adjust the number of channels, followed by batch normalization and non-linear activation function operations, and then another downsampling and normalization to obtain the initial scale feature map. ;in, The initial input height for the image. The initial input width of the image. This represents the initial number of input channels for the image. Step 2: Convert the initial scale feature map obtained in Step 1 into a single image. As input, four stages of processing are performed sequentially. Each stage includes downsampling and frequency-co-enhanced residual building blocks, resulting in feature maps at four different scales. ;in, Feature map of the first stage The number of channels, For the second stage feature map The number of channels, For the third stage feature map The number of channels, For the fourth stage feature map The number of channels; Step 3: Combine the four feature maps obtained in Step 2 at different scales In As input, the data is fused by convolution to achieve channel and scale alignment, then fed into a lightweight context aggregation unit for multi-branch semantic fusion. Finally, a class probability map is output via convolution and upsampled to the original image size. ;in, The number of channels in the probability graph; Step 4: Calculate the probability graph obtained in Step 3. The input is thresholded, and the output is the road crack segmentation result. .

2. The method for detecting pavement cracks in complex environments by integrating high and low frequency features as described in claim 1, characterized in that, The frequency-coordinated enhancement residual building block in step 2 performs the following sub-steps sequentially on the input features: Step 2.1: Convert the initial scale feature map obtained in Step 1 into a single image. The input is batch normalized to obtain the output. ; Step 2.2: Take the output of Step 2.1 As input, the frequency-coordinated enhancement module produces the output. ; Step 2.3: Convert the output of Step 2.2 Compared with the initial scale feature map obtained in step 1 Add to get the output ; Step 2.4: Convert the output of Step 2.3 As input, the output is obtained after batch normalization. ; Step 2.5: Convert the output of Step 2.4 As input, the signal is processed through a locally enhanced feedforward neural network to obtain the output. ; Step 2.6: Convert the output of Step 2.5 The output of step 2.3 The output is obtained by adding features. Thus, the feature map is obtained. ; Step 2.7: Convert the output of Step 2.6 As input to step 2.1, execute step 2.

1. Step 2.6 yields And so on, until we obtain... and .

3. The method for detecting pavement cracks in complex environments by integrating high and low frequency features as described in claim 2, characterized in that, The frequency co-enhancement module in step 2.2 performs the following sub-steps sequentially on the input features: Step A1: Take the output of step 2.1 As input, low-frequency extraction and high-frequency residual decomposition are performed; the low-frequency residual is obtained through average pooling operation, i.e. Then, the low-frequency residual is upsampled and aligned to the size, i.e., the low-frequency residual. Subsequently, the high-frequency residuals were obtained through a single subtraction. ; Step A2: The high-frequency residual obtained in step A1 The high-frequency enhancement branch is used to enhance edge and fine texture information to obtain the output. The low-frequency residual Input low-frequency enhancement branch to improve contour and region consistency to obtain output , will the Upsampling to With consistent spatial dimensions, the enhancement results of the two branches are stitched together along the channel dimension to obtain the desired result. ; Step A3: Convert the output of step A2 The input is fed into the attention fusion unit to adaptively allocate weights for high-frequency and low-frequency signals and map them back to the preset number of channels to obtain the output. ; Step A4: Take the output of step A3 The output of step 2.1 The output of the frequency co-enhancement module is obtained by adding the components together. .

4. The method for detecting pavement cracks in complex environments by integrating high and low frequency features as described in claim 3, characterized in that, The high-frequency enhancement branch in step A2 performs the following sub-steps sequentially on the input features: Step B1: The high-frequency residual obtained in step A1 The input features are fed into the three convolutional pairs in parallel to obtain... , and ; Step B2: Convert the three outputs , and Element-by-element addition yields ; Step B3: Obtain the result from step B2 The input is used for pointwise convolution and normalized activation to obtain... ; Step B4: Obtain the result from step B3 The high-frequency residual obtained in step A1 Intermediate features are formed by adding elements one by one; Step B5: Apply pointwise convolution, nonlinear activation, batch normalization, pointwise convolution, nonlinear activation, and batch normalization sequentially to the intermediate features obtained in Step B4, and output the result. .

5. The method for detecting pavement cracks in complex environments by integrating high and low frequency features as described in claim 4, characterized in that, All convolutions in the high-frequency enhancement branch are strip-shaped depthwise convolutions to capture crack features at different scales.

6. The method for detecting pavement cracks in complex environments by integrating high and low frequency features as described in claim 3, characterized in that, The low-frequency enhancement branch in step A2 performs the following sub-steps on the input features sequentially: Step C1: Process the low-frequency residual obtained in step A1 Perform pointwise convolution and average pooling sequentially to form at least three layers of low-frequency features, and obtain the following outputs: , , and ; Step C2: Output from step C1 , and High-frequency components are extracted through average pooling and differencing operations. Subsequently, these high-frequency features are processed by convolution, normalization, and a sigmoid activation function to enhance effective details. Finally, they are fused back to the original input via residual connections. This mechanism compensates for local high-frequency information by applying it to low-frequency features at different levels, while preserving global structural information, resulting in… , and ; Step C3: Obtain from each layer , and Upsampled to the same spatial size and convolved with shallow pointwise features from the input path obtained by splicing in the channel dimension ; Step C4: The splicing result obtained in step C3 Applying pointwise convolution, the output yields the result of the low-frequency enhancement branch. .

7. The method for detecting pavement cracks in complex environments by integrating high and low frequency features as described in claim 2, characterized in that, The local augmentation feedforward neural network in step 2.5 performs the following sub-steps sequentially on the input features: Step D1: Output of step 2.4 First, pointwise convolution is applied to complete channel fusion, and then channel-wise convolution is applied for spatial filtering to explicitly enhance the response to crack discontinuities and sharp high-frequency changes, thus obtaining the output. ; Step D2: The output of step D1 By sequentially applying a nonlinear activation function and a random dropout operation, the output is obtained. ; Step D3: The output of step D2 Large-kernel depthwise convolution is used to obtain long-range complex texture context, followed by channel fusion using pointwise convolution to obtain... ; Step D4: The results obtained in step D3 By applying random dropout, the output of the locally enhanced feedforward neural network is obtained. .

Images