Crack segmentation method based on conditional modulation and hierarchical dual-path network

Abstract

The invention discloses a crack segmentation method based on condition debugging and a hierarchical dual-path network, and relates to the technical field of computer vision, and the method comprises the steps: 1, collecting and marking a crack image, and generating a training data set; 2, constructing a condition debugging and hierarchical dual-path network model, and training by using the training data set to obtain a crack segmentation model; the conditional debugging and hierarchical dual-path network model comprises a ResNet50 branch network, a CMT branch network and a double-flow cascade segmentation header network, the CMT branch network comprises a conditional modulation gating convolution module, and the double-flow cascade segmentation header network comprises a hierarchical adaptive feature modulation module; and 3, acquiring a to-be-detected image, and inputting the to-be-detected image into the crack segmentation model to obtain a crack segmentation result. According to the method, global context and local feature extraction are effectively combined, and the most advanced accuracy and robustness are achieved in various crack scenes.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and more specifically to a crack segmentation method based on conditional debugging and hierarchical dual-path networks. Background Technology

[0002] The integrity of civil infrastructure such as bridges, highways, and buildings is constantly threatened by environmental stressors, including high traffic volume, extreme weather, and long-term material fatigue. These forces can lead to crack formation. If these cracks are not detected and addressed in a timely manner, they can gradually damage the structure, ultimately affecting its safety and service life. Early crack detection is crucial to preventing structural failures that could result in significant economic losses and serious safety risks. Given the scale of the infrastructure and the material properties involved (such as concrete, steel, and masonry), the development of automated crack detection systems has become both necessary and urgent.

[0003] In recent years, advancements in deep learning technology have significantly propelled the development of crack detection in civil infrastructure, with convolutional neural networks (CNNs) becoming a fundamental tool for local feature extraction and segmentation. However, as the complexity of crack patterns increases, the performance of CNNs declines significantly. They struggle to capture the global contextual cues needed to understand crack continuity and the overall structural background, performing poorly when dealing with discontinuous, sparsely distributed, or large-area cracks with fluctuating intensity. This often leads to fragmented segmentation maps, potentially resulting in the omission or misclassification of crucial crack structures. Furthermore, CNN-based methods frequently underperform in noisy or low-contrast imaging conditions common in real-world infrastructure inspections.

[0004] While Transformer-based architectures have emerged as a promising alternative, effectively capturing long-range dependencies and global context through self-attention mechanisms, they also face a number of challenges, particularly in fine-grained crack segmentation tasks. Transformers often struggle to preserve subtle local details crucial for accurately depicting cracks. This limitation stems in part from their inherent block-based processing and the global nature of their self-attention mechanisms, which can sometimes lead to over-smoothing of local features, resulting in a loss of accurate spatial resolution.

[0005] Therefore, how to achieve robust and accurate crack segmentation in complex real-world infrastructure is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] In view of this, the present invention provides a crack segmentation method based on conditional tuning and hierarchical dual-path network. The conditional tuning and hierarchical dual-path network CMHA-Net uses three key components: conditional modulation gated convolution, hierarchical adaptive feature modulation, and dual-stream cascaded segmentation head. It adopts a dual-branch parallel structure for feature extraction and uses the dual-stream cascaded segmentation head to complete the final crack segmentation. It effectively combines global context and local feature extraction and achieves state-of-the-art accuracy and robustness in a variety of crack scenarios.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] A crack segmentation method based on conditional debugging and hierarchical dual-path network includes the following steps:

[0009] Step 1: Collect and label crack images to generate a training dataset;

[0010] Step 2: Construct a conditional tuning and hierarchical dual-path network model and train it using the training dataset to obtain a crack segmentation model; the conditional tuning and hierarchical dual-path network model includes a ResNet50 branch network, a CMT branch network and a two-stream cascaded segmentation head network, wherein the CMT branch network includes a conditional modulation gated convolution module and the two-stream cascaded segmentation head network includes a hierarchical adaptive feature modulation module.

[0011] Step 3: Acquire the image to be detected and input the image to be detected into the crack segmentation model to obtain the crack segmentation result.

[0012] Preferably, in the conditional debugging and hierarchical dual-path network model, the ResNet50 branch network and the CMT branch network are parallel structures. The ResNet50 branch network outputs multi-scale depth features based on the input crack image, and the CMT branch network outputs multi-scale refined features based on the input crack image. The multi-scale depth features and multi-scale refined features are input to the dual-stream cascaded segmentation head network to output crack segmentation results.

[0013] Preferably, the ResNet50 branch network includes 50 stacked residual modules to extract multi-scale depth features, mainly focusing on capturing local details, such as small crack edges and texture information, and finally generating Feature 1 as a multi-scale depth feature.

[0014] Preferably, the residual module includes a point convolutional layer, a ReLU activation function, a 3×3 convolutional layer, another ReLU activation function, another point convolutional layer, and a residual connection layer connected in sequence; the enhanced features output by the first point convolutional layer and the enhanced features output by the second point convolutional layer are residually connected in the residual connection layer.

[0015] Preferably, the CMT branch network comprises a four-layer stacked encoder that integrates local details in the crack image to obtain global features as multi-scale refined features. Each encoder layer includes a Patch Embed module, a Conditional Modulation Gated Convolutional module (CMGC), a Transformer component, and a Layer Normalization layer connected in sequence. The Patch Embed module performs spatial downsampling and channel expansion to generate multi-scale tensors with gradually decreasing resolution, resulting in richer semantic representation. The Conditional Modulation Gated Convolutional module adaptively adjusts the convolutional features using global context vectors extracted from the multi-scale tensors, enhancing the model's ability to handle complex visual patterns and obtaining deep convolutional outputs. The Transformer component captures the long-distance dependencies of the deep convolutional outputs to obtain semantic features. The Layer Normalization layer improves the stability of training by normalizing the semantic features. The Layer Normalization layer of the last encoder layer outputs multi-scale refined features.

[0016] Preferably, the conditional modulation gated convolution module includes a global average pooling layer, a conditional modulation convolution (CMC) unit, and an additive layer; the global average pooling layer extracts the global context vector c from the multi-scale tensor output by the corresponding layer patch embedding module. i The condition vector CF is formed by flattening the condition vector C and serving as the condition signal for all convolutional blocks in the conditional modulation convolutional unit. The conditional modulation convolutional unit obtains convolutional features based on the condition signal (i.e., the flattened condition vector CF) and the multi-scale tensor output by the patch embedding module. The learned low-rank transformation is injected into the condition signal (i.e., the flattened condition vector CF) to enhance the convolutional features. The summation layer performs element-wise addition and fusion on the enhanced convolutional features output by the conditional modulation convolutional unit and the multi-scale tensor output by the patch embedding module to obtain the depth convolutional output.

[0017] Preferably, the Conditional Modulation Convolutional Unit (CMC) comprises two parallel CMC branches, in which the conditional vector C is reused to ensure consistency of conditions. The upper CMC branch comprises two stacked CMC blocks, each using a 3×3 kernel, with the multi-scale tensor output from the patch embedding module input to the first CMC block. The lower CMC branch comprises two stacked lightweight CMC blocks, each using a 1×1 kernel, with the multi-scale tensor output from the patch embedding module input to the first lightweight CMC block. The convolutional features output from the upper CMC branch and the convolutional features output from the first lightweight CMC block of the lower CMC branch are fused element-wise by addition before being input to the second lightweight CMC block of the lower CMC branch. The flattened conditional vector C is input to each CMC block in the two CMC branches, and each CMC block is followed by a normalization (GN) layer and a ReLU activation function to stabilize the training process and enhance nonlinear effects. The CMC block includes a depthwise separable convolutional layer and a low-rank linear transform (LR) layer. Linear and View layers.

[0018] Preferably, the multi-scale tensor output by the patch embedding module is typically of shape [B, N, C], where N = H × W, B represents the batch size, H represents the high resolution of the image, W represents the wide resolution of the image, and C represents the number of channels in the image. Before being input to the conditional modulation gated convolution module, the output is resized to the shape [B, C, H, W]. This step restores the spatial structure required for processing context-aware features. After processing by the conditional modulation gated convolution module, the feature map is again resized to the shape [B, N, C] for use in subsequent stages. The conditional modulation gated convolution module first extracts the tensor X from each scale of the input multi-scale tensor through global average pooling. in ∈R B×C×H×W Extract the global context vector c i :

[0019]

[0020] The global context vectors corresponding to tensors of all scales are arranged and combined into a condition vector CF∈R. B×C This vector, after being flattened, serves as the conditional signal for all convolutional blocks in the conditional modulation gated convolution module.

[0021] Preferably, the low-rank linear transform layer reduces parameter costs while maintaining expressive power, and generates modulation weights W based on the condition vector. C And changing the tensor shape in the view layer will modulate the weights W. C ∈R B×C′ Remodeling into R B×C′×1×1 The modulation weights are applied to the convolution output of the depth-separable convolutional layer through element-wise multiplication.

[0022] Preferably, the CMC block performs a convolution conditional operation on the input, specifically as follows:

[0023] YCMC=Conv(X)⊙Reshape(W C );

[0024] W C =W2(W1CF), W1∈R r×C W2∈R C×r , r << C;

[0025] Where YCMC represents the convolutional features output by the CMC block; Conv represents the convolution operation; X represents the input of the CMC block; Reshape represents the operation of changing the shape of the tensor; W C denoted as modulation weights; W1 and W2 represent the weight matrices obtained during training, respectively; CF represents the flattened conditional vector; r represents the rank in the low-rank matrix.

[0026] Preferably, the calculation of the additive layer is expressed as follows:

[0027] X out =ReLU(GN(Convcond(X) fused ,CF)))+X in

[0028] Among them, X out Indicates the output of a depthwise convolution; X fused This represents the convolutional feature obtained by fusing the convolutional features output by the first CMC block of the lower CMC branch with the convolutional features output by the upper CMC branch through element-wise addition; GN represents normalization; X in represents the multi-scale tensor output by the patch embedding module; ReLU represents the activation function; CF represents the flattened conditional vector; Convcond represents the convolution conditional operation.

[0029] Preferably, the dual-stream cascaded segmentation head network includes a fusion module (Concat), a pixel attention fusion (PAF) module, an upsampling module (Upsample), a fusion module (Concat), a hierarchical adaptive feature modulation module (HAFM), and a point convolutional layer (Point Conv) stacked sequentially. After the upsampling module and the second fusion module, a linear layer and a group normalization layer are set sequentially. The linear layer performs feature mapping, and the group normalization layer divides the channels into groups and normalizes them individually within each group, making the training process more stable and efficient.

[0030] The first fusion module performs a fusion operation on multi-scale deep features and multi-scale refined features at each level. Each level is connected along the channel dimension to obtain initial fused features.

[0031] The pixel attention fusion module adaptively learns pixel-level attention weights to guide the further fusion of multi-scale deep features and multi-scale refined features at each layer, enhancing important features while suppressing redundant information to obtain enhanced fused features. This achieves more discriminative feature fusion and effectively enhances the representational power of the fused features. The pixel attention fusion module consists of a convolutional layer, a multiplicative fusion layer, an upsampling layer, an attention activation layer, and an attention fusion layer connected in sequence. Multi-scale deep features and multi-scale refined features are reduced in number by channel convolution operations in the convolutional layer and then multiplied element-wise in the multiplicative fusion layer to obtain pixel fused features. The pixel fused features are then upsampled in the upsampling layer to increase the spatial size of the feature map and ensure feature alignment. In the attention activation layer, a pixel-level attention weight map is generated by the sigmoid activation function. Finally, the attention fusion layer performs pixel-wise weighted fusion of multi-scale deep features and multi-scale refined features based on the attention weight map to obtain enhanced fused features.

[0032] The upsampling module applies a uniform upsampling operation to the deep fusion feature obtained by adding and fusing the enhanced fusion feature and the initial fusion feature, and rescales the image to a predefined common spatial resolution to achieve multi-scale alignment.

[0033] The second fusion module connects the aligned enhanced fusion features along the channel dimension to form a unified multi-scale representation and obtain multi-scale fusion features;

[0034] The hierarchical adaptive feature modulation module effectively refines spatial details and integrates hierarchical contextual cues to obtain a unified feature representation by performing multi-branch spatial decomposition, adaptive scale-specific processing, and context-aware gating on multi-scale fused features.

[0035] Point convolutional layers perform final transformation and channel compression on the unified feature representation through convolution operations to obtain the crack segmentation result.

[0036] Preferably, the hierarchical adaptive feature modulation module includes a segmentation layer, an adaptive max pooling layer (AMP), a low-rank convolutional layer (LRC), a transpose convolutional layer (Transpose Conv), a fusion module (Concat), a conditional modulation gated convolutional module, an attention filter layer, and a product fusion layer;

[0037] The segmentation layer fuses multi-scale features X∈R B×C×H×W Divide into n groups uniformly along the channel dimension Each set of features corresponds to a specific processing branch, and each branch focuses on a different spatial scale while maintaining a balance in computational load;

[0038] The adaptive max-pooling layer applies adaptive max-pooling to the features of all branches, adjusting the spatial resolution to... Obtain the adjusted features;

[0039] The low-rank convolutional layer performs a 1×1 convolution operation on the adjusted features to reduce the number of channels, achieve low-rank approximation, effectively filter redundant information, and obtain n sets of convolutional filtered features. Then, depthwise separable convolution operation is performed on the n sets of convolutional filtered features to simplify the number of erosion-preserving parameters in spatial feature extraction and obtain n sets of depthwise separable convolutional features. Finally, pointwise convolution operation is performed on the n sets of depthwise separable convolutional features to restore the output channels and obtain n sets of low-rank convolutional features.

[0040] The transposed convolutional layer upsamples n sets of low-rank convolutional features to the original spatial resolution. And seamlessly merge them to obtain multi-scale convolutional features.

[0041] The fusion module uses channel axes to connect multi-scale convolutional features. Perform fusion to obtain the fusion tensor F cat ;

[0042] Conditional modulation gated convolution module for fused tensor F cat The process is performed to obtain a depthwise convolutional fusion output;

[0043] Attention filter layer for multi-scale fusion features X∈R B×C×H×W Perform a 3×3 attention convolution operation and a sigmoid activation function operation to generate an attention map W∈R. B×C×H×W ;

[0044] The product fusion layer performs element-wise multiplication fusion of the deep convolution fusion output and the attention map to obtain a unified feature representation.

[0045] Preferably, in step 2, a comprehensive loss function is used to train the model, with Dice loss L. Dice and binary cross-entropy loss L BCE The weighted summation yields the comprehensive loss function, expressed as:

[0046]

[0047] L=α·L Dice +β·L BCE

[0048] in, p represents the predicted probability of the j-th sample. j represents the true label of the j-th sample; M represents the number of samples; ∈ represents the smoothing term, which takes a small value to prevent division by zero; α and β represent the weighting coefficients, with the ratio set to 1:5.

[0049] As can be seen from the above technical solutions, compared with the prior art, the present invention discloses a crack segmentation method based on conditional debugging and hierarchical dual-path network. The construction of the conditional debugging and hierarchical dual-path network model (CMHA-Net) effectively integrates local and global information flows and significantly improves the accuracy and robustness of segmentation by means of dynamic feature adjustment and background noise suppression. CMHA-Net combines Conditional Modulation Gated Convolution (CMGC), Hierarchical Adaptive Feature Modulation (HAFM), and Two-Stream Concatenated Segmentation Head (DSCS) techniques to achieve superior performance. CMGC dynamically adjusts convolutional features through globally extracted conditional variables, balancing local details with the global background. This helps achieve a balance between high-level contextual understanding and fine-grained crack edge segmentation. By introducing modulation, branch gating, and residual connections, it not only improves the efficiency of feature extraction but also suppresses noise, thereby improving segmentation accuracy. HAFM adaptively integrates multi-level features through partitioning, multi-scale processing, and gated attention mechanisms, further optimizing feature representation. This enables the model to capture subtle and fragmented cracks while effectively suppressing background noise. DSCS performs multi-level feature fusion, upsampling, and multi-scale alignment between Transformer and CNN branches, ensuring that the global context is preserved while maintaining key local details, thus improving segmentation continuity and overall accuracy.

[0050] CMHA-Net demonstrates superior accuracy and robustness across various crack scenarios, providing excellent generalization capabilities and high-quality fine-grained segmentation. The network model of this invention not only exhibits strong generalization capabilities but also excels in fine-grained segmentation in critical civilian infrastructure applications, while maintaining the efficiency required for practical deployment in industrial and embedded environments. This provides a more accurate and reliable solution for crack segmentation in critical civilian infrastructure. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0052] Figure 1 This is a schematic diagram of the overall architecture of CMHA-Net provided by the present invention;

[0053] Figure 2 A schematic diagram of the hierarchical coding process of the CMT branch network provided by the present invention;

[0054] Figure 3 This is a schematic diagram of the CMGC module structure provided by the present invention;

[0055] Figure 4 This is a schematic diagram of the HAFM module structure provided by the present invention;

[0056] Figure 5 This is a schematic diagram of the visual experimental results provided by the present invention. Detailed Implementation

[0057] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0058] This invention discloses a crack segmentation method based on conditional debugging and a hierarchical dual-path network, comprising the following steps:

[0059] S1: Collect crack images and annotate them to generate a training dataset;

[0060] S2: Construct a conditional debugging and hierarchical dual-path network model, and train it using the training dataset to obtain a crack segmentation model; such as Figure 1 As shown, the conditional tuning and hierarchical dual-path network model includes a ResNet50 branch network, a CMT branch network, and a two-stream cascaded segmentation head network. The CMT branch network includes a conditional modulation gated convolution module, and the two-stream cascaded segmentation head network includes a hierarchical adaptive feature modulation module.

[0061] S3: Acquire the image to be detected, input the image to be detected into the crack segmentation model, and obtain the crack segmentation result.

[0062] Furthermore, in the conditional debugging and hierarchical dual-path network model, the ResNet50 branch network and the CMT branch network are parallel structures. The ResNet50 branch network outputs multi-scale depth features based on the input crack image, and the CMT branch network outputs multi-scale refined features based on the input crack image. The multi-scale depth features and multi-scale refined features are input to the dual-stream cascaded segmentation head network to output the crack segmentation results.

[0063] Furthermore, such as Figure 1As shown, the ResNet50 branch network consists of 50 stacked residual modules that extract multi-scale depth features, focusing on capturing local details such as small crack edges and texture information, and finally generating Feature 1 as a multi-scale depth feature.

[0064] Furthermore, the residual module includes a point convolutional layer, a ReLU activation function, a 3×3 convolutional layer, a ReLU activation function, a point convolutional layer, and a residual connection layer connected in sequence; the enhanced features output by the first point convolutional layer and the enhanced features output by the second point convolutional layer are residually connected in the residual connection layer.

[0065] Furthermore, such as Figure 1 and 2 As shown, the CMT branch network consists of four stacked encoder layers that integrate local details from crack images to obtain global features as multi-scale refined features. Each encoder layer includes a Patch Embed module, a Conditional Modulation Gated Convolutional module (CMGC), a Transformer component, and a Layer Normalization layer connected in sequence. The Patch Embed module performs spatial downsampling and channel expansion to generate multi-scale tensors with gradually decreasing resolution, resulting in richer semantic representations. The Conditional Modulation Gated Convolutional module adaptively adjusts the convolutional features using global context vectors extracted from the multi-scale tensors, enhancing the model's ability to handle complex visual patterns and obtaining deep convolutional outputs. The Transformer component captures long-distance dependencies in the deep convolutional outputs to obtain semantic features. The Layer Normalization layer improves training stability by normalizing the semantic features. The Layer Normalization layer of the last encoder layer outputs multi-scale refined features. The CMT branch network consists of four stages and employs a multi-stage embedding strategy. Each embedding layer is implemented through stride convolution. This approach maintains the spatial structure while enabling the model to progressively reduce resolution and extract abstract semantic features in a hierarchical design similar to that in ResNet convolutional neural networks, generating Feature 2 as a multi-scale refined feature.

[0066] Furthermore, the output of the patch embedding module is typically of shape [B, N, C], where N = H × W, B represents the batch size, H represents the high resolution of the image, W represents the wide resolution of the image, and C represents the number of channels in the image. Before being input into the conditional modulation gated convolution module, the output is reshaped to [B, C, H, W], a step that restores the spatial structure required for processing context-aware features. After processing by the conditional modulation gated convolution module, the feature map is again reshaped to [B, N, C] for use in subsequent stages. The conditional modulation gated convolution module first extracts each scale of the tensor X from the input multi-scale tensor through global average pooling. in ∈R B×C×H×W Extract the global context vector c i :

[0067]

[0068] The global context vectors corresponding to tensors of all scales are arranged and combined into a condition vector CF∈R. B×C This vector, after being flattened, serves as the conditional signal for all convolutional blocks in the conditional modulation gated convolution module.

[0069] Furthermore, such as Figure 3 As shown, the conditional modulation gated convolution module includes a global average pooling layer, a conditional modulation convolution (CMC) unit, and a summation layer; the global average pooling layer extracts the global context vector c from the multi-scale tensor output by the corresponding layer patch embedding module. i The condition vector CF is formed and flattened to serve as the condition signal for all convolutional blocks in the conditional modulation convolutional unit. The conditional modulation convolutional unit obtains the convolutional output based on the flattened conditional vector CF and the tensor output by the patch embedding module. The convolutional features are enhanced by injecting the learned low-rank transform into the conditional vector. The summation layer performs element-wise addition and fusion on the enhanced convolutional features output by the conditional modulation convolutional unit and the multi-scale tensor output by the patch embedding module to obtain the depth convolutional output.

[0070] Furthermore, the Conditional Modulation Convolutional Unit (CMC) comprises two parallel CMC branches, in which the conditional vector is reused to ensure consistency of conditions. The upper CMC branch consists of two stacked CMC blocks, each using a 3×3 kernel, with the multi-scale tensor output from the patch embedding module input to the first CMC block. The lower CMC branch consists of two stacked lightweight CMC blocks, each using a 1×1 kernel, with the multi-scale tensor output from the patch embedding module input to the first lightweight CMC block. The outputs of the upper CMC branch and the first CMC block of the lower CMC branch are fused element-wise by addition before being input to the second lightweight CMC block of the lower CMC branch. The flattened conditional vector C is input to each CMC block in both CMC branches. Each CMC block is followed by a normalization (GN) layer and a ReLU activation function to stabilize the training process and enhance nonlinear effects. The CMC block includes a depthwise separable convolutional layer, a low-rank linear transformation layer (LRLinear), and a view layer.

[0071] Furthermore, the low-rank linear transform layer reduces parameter costs while maintaining expressive power, generating modulation weights W based on the conditional vector. C And changing the tensor shape in the view layer will modulate the weights W. C ∈R B×C′ Remodeling into R B×C′×1×1 The modulation weights are applied to the convolution output of the depth-separable convolutional layer through element-wise multiplication.

[0072] Furthermore, the CMC block performs convolution conditional operations on the input, specifically as follows:

[0073] YCMC=Conv(X)⊙Reshape(W C );

[0074] W C =W2(W1CF), W1∈R r×C W2∈R C×r , r << C;

[0075] Where YCMC represents the convolutional features output by the CMC block; Conv represents the convolution operation; X represents the input of the CMC block; Reshape represents the operation of changing the shape of the tensor; W C denoted as modulation weights; W1 and W2 represent the weight matrices obtained during training, respectively; CF represents the flattened conditional vector; r represents the rank in the low-rank matrix.

[0076] Furthermore, the calculation of the additive layer is expressed as follows:

[0077] X out =ReLU(GN(Convcond(X)fused ,CF)))+X in

[0078] Among them, X out Indicates the output of a depthwise convolution; X fused This represents the convolutional feature obtained by fusing the convolutional features output by the first CMC block of the lower CMC branch with the convolutional features output by the upper CMC branch through element-wise addition; GN represents normalization; X in represents the multi-scale tensor output by the patch embedding module; ReLU represents the activation function; CF represents the flattened conditional vector; Convcond represents the convolution conditional operation.

[0079] Furthermore, the dual-stream cascaded segmentation head network (DSCS module) includes a fusion module (Concat), a pixel attention fusion (PAF) module, an upsampling module (Upsample), a fusion module (Concat), a hierarchical adaptive feature modulation module (HAFM), and a point convolutional layer (Point Conv) stacked sequentially. After the upsampling module and the second fusion module, a linear layer and a group normalization layer are set sequentially. The linear layer performs feature mapping, and the group normalization layer divides the channels into groups and normalizes them individually within each group, making the training process more stable and efficient.

[0080] The first fusion module performs a fusion operation on multi-scale deep features and multi-scale refined features at each level. Each level is connected along the channel dimension to obtain initial fused features.

[0081] The pixel attention fusion module adaptively learns pixel-level attention weights to guide the further fusion of multi-scale deep features and multi-scale refined features at each layer, enhancing important features while suppressing redundant information to obtain enhanced fused features. This achieves more discriminative feature fusion and effectively enhances the representational power of the fused features. The pixel attention fusion module consists of a convolutional layer, a multiplicative fusion layer, an upsampling layer, an attention activation layer, and an attention fusion layer connected in sequence. Multi-scale deep features and multi-scale refined features are reduced in number by channel convolution operations in the convolutional layer and then multiplied element-wise in the multiplicative fusion layer to obtain pixel fused features. The pixel fused features are then upsampled in the upsampling layer to increase the spatial size of the feature map and ensure feature alignment. In the attention activation layer, a pixel-level attention weight map is generated by the sigmoid activation function. Finally, the attention fusion layer performs pixel-wise weighted fusion of multi-scale deep features and multi-scale refined features based on the attention weight map to obtain enhanced fused features.

[0082] The upsampling module applies a uniform upsampling operation to the deep fusion feature obtained by adding and fusing the enhanced fusion feature and the initial fusion feature, and rescales the image to a predefined common spatial resolution to achieve multi-scale alignment.

[0083] The second fusion module connects the aligned enhanced fusion features along the channel dimension to form a unified multi-scale representation and obtain multi-scale fusion features;

[0084] The hierarchical adaptive feature modulation module effectively refines spatial details and integrates hierarchical contextual cues to obtain a unified feature representation by performing multi-branch spatial decomposition, adaptive scale-specific processing, and context-aware gating on multi-scale fused features.

[0085] Point convolutional layers perform final transformation and channel compression on the unified feature representation through convolution operations to obtain the crack segmentation result.

[0086] The dual-stream cascaded segmentation head network makes full use of the feature information extracted from different network depths and adopts a multi-level feature cascaded fusion strategy for fusion.

[0087] The HAFM module enables hierarchical perception and spatially adaptive feature refinement. In this module, the input feature map is decomposed into multiple branches of specific scales, enabling the network to extract and enhance spatial context at different resolutions. These features are then efficiently integrated through conditional modulation and attention-based gating techniques, effectively capturing subtle details and global dependencies in the image, and ultimately generating a unified and information-rich representation.

[0088] Furthermore, the first fusion module fuses multi-scale deep features F trans (i) and multi-scale refining feature F cnn (i), i = 1...L, where L represents the total number of feature levels. The DSCS module performs a fusion operation at each level i to form initial fused features, which can be simplified as F fuse (i);

[0089] The computation of the PAF module at level i is represented as follows:

[0090] A(i)=S(U(i)(C reduction (F cnn (i))☉C reduction (F trans (i))))

[0091] F fuse_enhanced (i)=(1-A(i))☉F cnn (i)+A(i)☉F trans (i)

[0092] Among them, Creduction ⊙ represents a 1×1 channel convolution operation; U(i) represents an element-wise multiplication operation; S(·) represents an upsampling operation; and S(·) represents an attention activation operation using the sigmoid activation function. Represents a pixel-level attention weight map; F fuse_enhanced (i) represents the enhanced fusion feature of the i-th level.

[0093] Furthermore, such as Figure 4 As shown, the hierarchical adaptive feature modulation module includes a segmentation layer, an adaptive max pooling layer (AMP), a low-rank convolutional layer (LRC), a transpose convolutional layer (Transpose Conv), a fusion module (Concat), a conditional modulation gated convolutional module, an attention filter layer, and a product fusion layer.

[0094] The segmentation layer fuses multi-scale features X∈R B×C×H×W Divide into n groups uniformly along the channel dimension Each set of features corresponds to a specific processing branch, and each branch focuses on a different spatial scale while maintaining a balance in computational load;

[0095] The adaptive max-pooling layer applies adaptive max-pooling to the features of all branches, adjusting the spatial resolution to... Obtain the adjusted features;

[0096] The low-rank convolutional layer performs a 1×1 convolution operation on the adjusted features to reduce the number of channels, achieve low-rank approximation, effectively filter redundant information, and obtain n sets of convolutional filtered features. Then, depthwise separable convolution operation is performed on the n sets of convolutional filtered features to simplify the number of erosion-preserving parameters in spatial feature extraction, and obtain n sets of depthwise separable convolutional features. Finally, the output channels are restored from the n sets of depthwise separable convolutional features to obtain n sets of low-rank convolutional features.

[0097] The transposed convolutional layer upsamples n sets of low-rank convolutional features to the original spatial resolution. And seamlessly merge them to obtain multi-scale convolutional features.

[0098] The fusion module uses channel axes to connect multi-scale convolutional features. Perform fusion to obtain the fusion tensor F cat ;

[0099] Conditional modulation gated convolution module for fused tensor F cat The process is performed to obtain a depthwise convolutional fusion output;

[0100] Attention filter layer for multi-scale fusion features X∈R B×C×H×WPerform a 3×3 attention convolution operation and a sigmoid activation function operation to generate an attention map W∈R. B×C×H×W ;

[0101] The product fusion layer performs element-wise multiplication fusion of the deep convolutional fusion output and the attention map to obtain a unified feature representation. The HAFM module introduces a structured multi-scale processing flow, utilizing channel-level scale decomposition, adaptive pooling, LRC-based local enhancement, global conditional modulation, and attention filtering to form a unified representation.

[0102] Furthermore, an adaptive max-pooling operation is applied to each branch i to adjust the spatial resolution to... The expression is:

[0103]

[0104] in, The adjusted features corresponding to the i-th set of features; This indicates a downsampling operation, which downsamples the feature X of each branch and adjusts the spatial size of the feature.

[0105] Furthermore, the calculation expression for LRC is as follows:

[0106] LRC(X) = PWConv 1×1 (DWConv k×k (Conv 1×1 (X)))

[0107] in, Represents the features of the input; Conv 1×1 This represents a 1×1 convolution operation, which is a reduced layer for the initial channels; DWConv k×k PWConv represents a k×k depth separable convolution operation. 1×1 represents a 1×1 pointwise convolution operation; LRC(X) represents a low-rank convolution feature.

[0108] Furthermore, the transposed convolutional layer resamples the features of the branch with reduced resolution (i>1) back to the original spatial resolution. This process ensures that all outputs maintain a consistent spatial scale and achieves seamless fusion; the expression is:

[0109]

[0110] Where T represents a convolutional layer with a kernel size of 4 and a stride of 2; This represents the i-th group of multi-scale convolutional features.

[0111] Furthermore, the calculation expression for the fusion module is as follows:

[0112]

[0113] Where Concat represents the concatenation operation; F cat ∈R B×C×H×W .

[0114] Furthermore, the calculation expression for the attention filter layer is as follows:

[0115]

[0116] Among them, C attn σ represents attention convolution; σ represents the sigmoid activation function.

[0117] Furthermore, the calculation expression for the product fusion layer is as follows:

[0118] Y HAFM =CMGC(F cat )☉W

[0119] Among them, Y HAFM represents a unified feature representation; CMGC() represents the output of a deep convolutional fusion; ⊙ represents element-wise multiplication.

[0120] Furthermore, in S2, a comprehensive loss function is used to train the model, with Dice loss L. Dice and binary cross-entropy loss L BCE The weighted summation yields the comprehensive loss function, expressed as:

[0121]

[0122]

[0123] L=α·L Dice +β·L BCE

[0124] in, p represents the predicted probability of the j-th sample. j represents the true label of the j-th sample; M represents the number of samples; ∈ represents the smoothing term, which has a small value and is used to prevent division by zero; α and β represent the weighting coefficients, with the ratio set to 1:5.

[0125] The comprehensive loss function aims to address the inherent class imbalance problem in crack images and optimize the model's segmentation performance, improving its accuracy in crack detection and overall segmentation. The Dice loss function, based on region overlap, is inherently robust to class imbalance, making it particularly suitable for small object detection tasks and effectively improving the model's sensitivity to crack edges. The binary cross-entropy loss evaluates the accuracy of pixel-level predictions, effectively improving the model's classification accuracy at the pixel level, thus enhancing the overall segmentation performance. Especially in crack segmentation, it finely optimizes the classification result for each pixel. By adjusting the weight coefficients α and β, the contributions of Dice loss and binary cross-entropy loss in model training can be flexibly balanced. A reasonable weight allocation strategy prioritizes improving pixel-level accuracy while effectively mitigating class imbalance, thereby optimizing crack segmentation performance.

[0126] On the other hand, in one specific embodiment, the performance of the conditional tuning and hierarchical dual-path network model in the method of the present invention was experimentally tested on four public datasets: SteelCrack, Crack896, Chinese Chemical Society, and CrackR. These datasets vary in resolution, crack features, and scene complexity, providing a comprehensive benchmark for crack segmentation performance. SteelCrack and Crack896 contain small and sparse cracks, which challenges the model's ability to detect small-scale features in background-dominated environments. The Chinese Chemical Society dataset contains high-resolution images and binary alpha maps as ground truth to evaluate the model's performance in identifying cracks on different structural surfaces. CrackR contains images taken under controlled conditions from different distances (30-100 cm), ensuring multi-angle perspectives for crack detection. These datasets cover a variety of materials, including steel, concrete, and masonry, enabling a comprehensive evaluation of the model's ability to segment regular and irregular cracks in complex and noisy environments. Each dataset was divided into training, validation, and test sets in a 7:1:2 ratio to ensure a comprehensive and balanced evaluation of the model's performance.

[0127] (1) Experimental setup

[0128] CMHA-Net was developed using PyTorch v2.1.2 and trained on an Ubuntu 20.04.6LTS system with an Intel Xeon Platinum 8336C CPU. Eight Nvidia GeForce RTX 4090 graphics cards (each with 24GB of VRAM) were used. The AdamW optimizer was employed during training with an initial learning rate of 0.0005, which was then adjusted using the StepLR scheduler. A weight decay of 0.01 was set to prevent overfitting. The model was trained for 50 epochs with a batch size of 1, and the final test results were based on the performance on the validation set, selecting the optimal checkpoint.

[0129] (2) Experimental comparison

[0130] The performance of CMHA-Net was evaluated by comparing it with seven state-of-the-art methods, including both CNN-based and Transformer-based methods: RIND, UCTNet, SFIAN, CTCS, Crackmer, DTrCNet, and SimCrack models.

[0131] Model performance was evaluated using six key metrics: precision (P), recall (R), F1 score, optimal dataset size (ODS), optimal image size (OIS), and mean intersection-union ratio (mIoU).

[0132] 1) The SteelCrack dataset mainly contains small, sparsely distributed cracks on steel surfaces, accompanied by complex background noise interference. Experimental results are shown in Table 1. Although state-of-the-art methods such as DTrCNet and SimCrack perform well, traditional CNN-based methods, such as UCTNet and Crackmer, still show limited performance on this dataset, with ODS and OIS scores below 0.75. In contrast, Transformer-based methods, such as SFIAN, perform better in ODS and OIS, reaching approximately 0.80, indicating a significant advantage of Transformers in capturing global contextual information and suppressing background noise. However, CMHA-Net significantly outperforms all baseline methods on the SteelCrack dataset, especially achieving the best results in ODS, OIS, F1, and mIoU metrics. In the challenging SteelCrack dataset with complex background noise, the CMGC module effectively helps the model distinguish cracks from background noise, reducing false detections and improving sensitivity to fine crack edges, thereby enhancing segmentation accuracy. Furthermore, the DSCS module integrates features from Transformer and CNN branches, further enhancing the synergy between global context and local details, ensuring that CMHA-Net achieves superior segmentation performance on the SteelCrack dataset.

[0133] Table 1. Experimental results in the SteelCrack dataset.

[0134]

[0135] 2) The CrackR dataset mainly contains images of concrete cracks. Due to varying shooting distances, the scale and shape of the cracks in these images differ. Experimental results are shown in Table 2. Traditional convolutional neural network-based methods, such as UCTNet and CTCS, perform poorly on this dataset, with ODS scores below 0.35. This indicates limitations in handling cracks of different scales and detecting cracks from a distance. Although methods like Crackmer and DTrCNet have made improvements, they still haven't fully realized the potential of this dataset. Transformer-based methods, such as SFIAN, achieved relatively good results in ODS and OIS metrics, but still lag behind the best-performing methods. In contrast, CMHA-Net demonstrates a significant advantage on the CrackR dataset, significantly outperforming other methods in both ODS and OIS metrics. The multi-scale feature fusion strategy of the HAFM module enables CMHA-Net to effectively capture crack details at different scales in the CrackR dataset. Whether it's wide cracks in close-up images or fine cracks at long distances, the HAFM module dynamically adjusts feature extraction and fusion, significantly improving the model's robustness and segmentation accuracy. In addition, the DSCS module adopts a multi-level cascade fusion strategy to integrate features from different network depths, enabling the model to understand image information more comprehensively and effectively cope with the challenges brought by scale changes in the CrackR dataset.

[0136] Table 2 shows the experimental results in the CrackR dataset.

[0137]

[0138] 3) The Crack896 dataset also contains concrete crack images, but it features greater diversity in crack morphology and more complex background conditions. Experimental results are shown in Table 3. Traditional CNN-based methods, such as UCTNet and CTCS, still perform poorly, with ODS scores below 0.71. Although methods like SFIAN and Crackmer have improved, newer methods such as DTrCNet and SimCrack perform better, with ODS scores approaching 0.80. CMHA-Net maintains its leading performance on the Crack896 dataset, achieving the best results in ODS, OIS, and F1 metrics, further demonstrating the effectiveness of its architecture. The multi-scale feature fusion capability of the HAFM module plays a crucial role in handling the diverse crack morphologies in Crack896, enabling the model to capture crack features of different scales and shapes. Furthermore, the conditional modulation mechanism in the CMGC module combined with the dual-branch fusion strategy of the DSCS module allows CMHA-Net to maintain excellent segmentation accuracy even in the complex backgrounds and highly variable crack patterns of the Crack896 dataset.

[0139] Table 3 shows the experimental results on the Crack896 dataset.

[0140]

[0141] 4) The Chinese Chemical Society dataset contains high-resolution images of concrete cracks and their corresponding binarized ground truth α masks. Experimental results are shown in Table 4. Traditional CNN-based methods, such as CTCS and UCTNet, still perform poorly on this dataset, with ODS scores below 0.45. Although methods like SFIAN and SimCrack show only minor improvements, Crackmer and DTrCNet perform relatively better, with ODS scores approaching 0.70. CMHA-Net again achieves the best performance on the Chinese Chemical Society dataset, significantly outperforming all other methods in metrics such as ODS, OIS, F1, and mIoU. Accurately capturing subtle crack features is crucial in high-resolution datasets like the Chinese Chemical Society dataset.

[0142] Table 4. Experimental results from the Chinese Chemical Society dataset.

[0143]

[0144] The above experiments compare the visualization results obtained by the model on different datasets, as shown below. Figure 5 As shown, the critical crack area is marked in red, while segmentation errors from other methods are marked with yellow boxes.

[0145] 5) Comparison of complexity

[0146] To further evaluate the balance between segmentation performance and computational efficiency, a comprehensive comparison of CMHA-Net's performance with several representative methods in terms of FLOPs, number of parameters, and inference latency was conducted, as detailed in Table 5. All models were evaluated using 512×512 resolution input images from the Crack896 dataset. Among the compared methods, CMHA-Net is not the most lightweight in terms of computational cost and number of parameters; however, it achieves a better performance balance. CMHA-Net has 14.96M parameters and 73.65G of FLOPs, and its segmentation performance (F1 = 0.8250, mIoU = 0.8411) significantly outperforms most methods of comparable or higher complexity.

[0147] Table 5. Results of FLOPs, Number of Parameters, Inference Latency, and Key Performance Indicators

[0148]

[0149] Compared to DTrCNet, while the latter achieved the fastest inference speed (21.28 ms), its F1 score was lower (0.7810). CMHA-Net improved accuracy by 4.4 percentage points while maintaining a low latency of 31.25 ms. In contrast, RIND had significantly poorer segmentation accuracy (F1 = 0.5874), the highest computational cost (695.77 G FLOPs), and the slowest inference speed (90.91 ms), highlighting the inefficiency of unoptimized heavy-duty models.

[0150] These results demonstrate that CMHA-Net's architecture not only enhances feature representation but also ensures model compactness and fast inference. This makes CMHA-Net well-suited for real-time deployment in industrial and embedded scenarios where accuracy and efficiency are paramount.

[0151] In summary, CMHA-Net was evaluated using four publicly available datasets covering a variety of scenarios and material structures. Experimental results show that the model of this invention surpasses current state-of-the-art methods and is able to generate high-quality crack segmentation maps.

[0152] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0153] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for crack segmentation based on conditional debugging and hierarchical dual-path networks, characterized in that, The method comprises the following steps: Step 1: collecting crack images and labeling to generate a training data set; Step 2: constructing a conditional debugging and hierarchical dual-path network model and training the model using the training data set to obtain a crack segmentation model; the conditional debugging and hierarchical dual-path network model comprises a ResNet50 branch network, a CMT branch network and a dual-flow cascaded segmentation head network, wherein the CMT branch network comprises a conditional modulation gated convolution module, and the dual-flow cascaded segmentation head network comprises a hierarchical adaptive feature modulation module; Step 3: collecting an image to be detected, inputting the image to be detected into the crack segmentation model and obtaining a crack segmentation result.

2. The method of claim 1, wherein, The ResNet50 branch network and the CMT branch network in the conditional debugging and hierarchical dual-path network model are in parallel structure, the ResNet50 branch network outputs multi-scale deep features according to the input crack image, and the CMT branch network outputs multi-scale refined features according to the input crack image; the multi-scale deep features and the multi-scale refined features are input into the dual-flow cascaded segmentation head network to output a crack segmentation result.

3. The method of claim 2, wherein, The ResNet50 branch network comprises 50 stacked residual modules for extracting multi-scale deep features.

4. The method of claim 3, wherein, The residual module comprises a point convolution layer, an activation function ReLU, a 3*3 convolution layer, an activation function ReLU, a point convolution layer and a residual connection layer connected in sequence; the enhanced features output by the first point convolution layer and the enhanced features output by the second point convolution layer are connected in residual connection in the residual connection layer.

5. The method of claim 2, wherein, The CMT branch network comprises 4 layers of stacked encoders for integrating local details in the crack image to obtain global features as multi-scale refined features; each layer of the encoder comprises a patch embedding module, a conditional modulation gated convolution module, a Transformer component and a layer normalization layer connected in sequence; the patch embedding module generates multi-scale tensors gradually reduced in resolution; the conditional modulation gated convolution module adaptively adjusts the convolution features using the global context vector extracted from the multi-scale tensors to obtain deep convolution output; the Transformer component captures long-distance dependency of the deep convolution output to obtain semantic features; the layer normalization layer improves the stability of training by performing layer normalization on the semantic features; and the layer normalization layer of the last layer of the encoder outputs the multi-scale refined features.

6. The method of claim 5, wherein, The conditional modulation gated convolution module comprises a global average pooling layer, a conditional modulation convolution unit and a summation layer; the global average pooling layer extracts a global context vector from the multi-scale tensor output by the patch embedding module of the corresponding layer to form a condition vector CF, and the condition vector is flattened to be used as a condition signal of all convolution blocks in the conditional modulation convolution unit; The conditional modulation convolution unit obtains convolution features according to the condition vector CF and the multi-scale tensor output by the patch embedding module, and enhances the convolution features by injecting the condition vector through a learned low-rank transformation; the summation layer performs element-level addition fusion on the enhanced convolution features output by the conditional modulation convolution unit and the multi-scale tensor output by the patch embedding module to obtain deep convolution output.

7. The method of claim 6, wherein, The conditional modulation convolution unit includes two CMC branches designed in parallel, and the condition vector is reused in each branch to ensure consistency of the condition; the upper CMC branch includes two stacked CMC blocks, each CMC block uses a 3*3 kernel, and the multi-scale tensor output by the patch embedding module is input into the first CMC block; the lower CMC branch includes two stacked lightweight CMC blocks, each lightweight CMC block uses a 1*1 kernel, and the multi-scale tensor output by the patch embedding module is input into the first lightweight CMC block, the convolutional features output by the upper CMC branch and the first lightweight CMC block of the lower CMC branch are fused by element-level addition and then input into the second lightweight CMC block of the lower CMC branch; the condition vector is input into each CMC block in the two CMC branches, and each CMC block is sequentially connected with a normalization layer and an activation function ReLU; the CMC block includes a depth separable convolution layer, a low-rank linear transformation layer and a view layer.

8. The method of claim 7, wherein, The CMC block performs a convolutional conditional operation on the input, and the specific calculation is represented as: YCMC = Conv(X) Θ Reshape(W C ); W C = W2(W1CF), W1∈R r×C , W2∈R C×r , r<<C; where YCMC represents the convolutional features output by the CMC block; Conv represents a convolution operation, X represents the input of the CMC block; Reshape represents a tensor shape changing operation; W C represents a modulation weight; W1 and W2 represent weight matrices obtained through training respectively; CF represents a flattened condition vector; r represents the rank in a low-rank matrix; C represents the number of channels of an image. The calculation of the addition layer is represented as: X out = ReLU(GN(Convcond(X fused , CF)) + X in wherein X out represents the deep convolutional output; X fused represents the convolutional feature obtained by element-wise addition of the convolutional feature of the first CMC block output of the lower CMC branch and the convolutional feature output by the upper CMC branch; GN represents normalization; X in represents the multi-scale tensor output by the patch embedding module; ReLU represents an activation function; CF represents the flattened conditional vector; Convcond represents a convolutional conditional operation.

9. The method of claim 2, wherein, The dual-flow cascaded segmentation head network includes a fusion module, a pixel attention fusion module, an upsampling module, a fusion module, a hierarchical adaptive feature modulation module and a point convolution layer stacked in sequence; a linear layer and a group normalization layer are further arranged in sequence after the upsampling module and the second fusion module, the linear layer realizes feature mapping, and the group normalization layer divides the channels into groups and separately normalizes each group; The first fusion module performs a fusion operation on the multi-scale deep features and the multi-scale refined features at each level, and each level is connected along the channel dimension to obtain initial fusion features; The pixel attention fusion module adaptively learns the pixel-level attention weight, guides the multi-scale deep features and the multi-scale refined features to further fuse at each level, and obtains enhanced fusion features; The upsampling module applies uniform upsampling operation to the depth fusion features obtained by adding and fusing the enhanced fusion features and the initial fusion features, and realizes multi-scale alignment; The second fusion module connects the aligned enhanced fusion features along the channel dimension to obtain multi-scale fusion features; The hierarchical adaptive feature modulation module obtains a unified feature representation through multi-branch spatial decomposition, adaptive scale-specific processing and context-aware gating processing on the multi-scale fusion features; The point convolution layer converts and channel-compresses the unified feature representation through convolution operation to obtain a crack segmentation result.

10. The method of claim 9, wherein, The hierarchical adaptive feature modulation module includes a segmentation layer, an adaptive max-pooling layer, a low-rank convolution layer, a transpose convolution layer, a fusion module, a conditional modulation gating convolution module, an attention filtering layer and a product fusion layer; The segmentation layer uniformly divides the multi-scale fusion features along the channel dimension into n groups, and each group of features corresponds to a specific processing branch; The adaptive max-pooling layer applies an adaptive max-pooling operation to the features of all branches to obtain adjusted features; The low-rank convolution layer performs a 1×1 convolution operation on the adjusted feature to obtain n groups of convolution filtering features, performs a depth separable convolution operation on the n groups of convolution filtering features to obtain n groups of depth separable convolution features, and finally performs a point-by-point convolution operation on the n groups of depth separable convolution features to obtain n groups of low-rank convolution features; The transpose convolution layer up-samples the n groups of low-rank convolution features to the original spatial resolution and performs seamless fusion to obtain multi-scale convolution features; The fusion module fuses the multi-scale convolution features through a channel axis to obtain a fusion tensor; The conditional modulation gated convolution module processes the fusion tensor to obtain a deep convolution fusion output; The attention filtering layer performs a 3×3 attention convolution operation on the multi-scale fusion features and an activation operation of a sigmoid activation function to generate an attention map; The product fusion layer performs element-by-element multiplication fusion on the deep convolution fusion output and the attention map to obtain a unified feature representation.

Images