An asphalt concrete image segmentation system and method based on a gating fusion mechanism

Abstract

The present application relates to a kind of based on gating fusion mechanism's asphalt concrete image segmentation system and method, its system includes: double-branch image acquisition module;Heterogeneous double-branch feature extraction module;Channel space alignment module;Multiple attention module;Gating fusion module;Lightweight high-resolution decoder module.The present application is based on double-branch respectively extracts material local texture feature and structure global semantic feature, realizes cross-modal channel-space dimension matching by dynamic feature alignment mechanism, combined with multiple attention collaborative establishment bidirectional cross attention interaction, then utilize the gating weight dynamic fusion bidirectional cross attention feature, restore details by progressive up-sampling;The present application can effectively suppress modal suppression phenomenon, improve the precision of multi-phase material interface segmentation, while guaranteeing video memory efficiency, realize the pixel-level semantic segmentation of material microstructure, suitable for high-precision image analysis of non-homogeneous composite materials such as poured asphalt concrete.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image processing, in particular to a bituminous concrete image segmentation system and method based on a gating fusion mechanism. BACKGROUND

[0002] The poured bituminous concrete is a non-homogeneous composite material composed of aggregate, bituminous mortar, grouting material and residual voids, and its mechanical properties are highly dependent on the spatial distribution of the mesostructure. The meso-mechanical method based on finite elements provides a new research method for revealing the complex mechanical behavior of poured bituminous concrete. Image processing is the starting point of all research, and the meso-parameters obtained by high-precision image processing are crucial for accurate research conclusions. However, commercial FEA software lacks a dedicated image processing module for complex multi-phase poured bituminous concrete, which restricts the efficiency of technology transfer. Real geometry reconstruction technology based on image segmentation has gradually become the mainstream.

[0003] The existing traditional segmentation method (such as UNet, DeepLab series) is based on a single feature extraction network, which is difficult to balance local details and global context information, and lacks sensitivity to the boundaries of micro-multiphase structures of composite materials.

[0004] The existing single-modal deep learning model is difficult to balance local texture and global structure features, and the traditional multi-modal fusion method ignores the anisotropy characteristics of materials, resulting in poor semantic consistency of multi-scale structures.

[0005] The existing conventional attention mechanism (Transformer) has low computational efficiency in feature interaction and lacks differentiated processing for different modal features, resulting in modal suppression when facing a double-branch network, which leads to the loss of structure edge details.

[0006] In addition, when processing high-resolution images, the existing model has high memory occupation, making it difficult to balance efficiency and accuracy. SUMMARY

[0007] The present application provides a bituminous concrete image segmentation system and method based on a gating fusion mechanism to solve at least one of the above technical problems.

[0008] The technical solution of the present application to solve the above technical problems is as follows: a bituminous concrete image segmentation system based on a gating fusion mechanism, comprising:

[0009] A double-branch image acquisition module for region labeling and edge detection processing of the original image to be segmented to obtain material branch images and structure branch images;

[0010] a heterogeneous dual-branch feature extraction module, configured to locally induce bias on the material branch image to extract local material texture, and globally model the structure branch image to extract global interface structure, thereby obtaining a dual-modal feature image based on heterogeneous complementarity of spatial details and global semantics;

[0011] a channel space alignment module, configured to perform channel adjustment and spatial size matching on the dual-modal feature image;

[0012] a multiple attention module, configured to perform self-attention and cross-attention processing on the dual-modal feature image after channel space alignment, to establish long-range dependency between the dual-modal feature image, thereby obtaining a dual-path cross-modal associated feature image;

[0013] a gated fusion module, configured to perform dynamic weighted fusion on the dual-path cross-modal associated feature image by using an adaptive gated fusion mechanism, so that the fusion weight is biased towards the material branch in the boundary region and biased towards the structure branch in the internal region, thereby obtaining a fusion feature image;

[0014] a lightweight high-resolution decoder module, configured to perform multi-level progressive up-sampling operation on the fusion feature image to generate a pixel-level segmentation result image.

[0015] On the basis of the above technical solutions, the application can also be improved as follows.

[0016] Further, the heterogeneous dual-branch feature extraction module comprises:

[0017] a deep residual network, configured to locally induce bias on the material branch image to extract local material texture, thereby obtaining a material feature image;

[0018] a visual Transformer network, configured to globally model the structure branch image to extract global interface structure, thereby obtaining a structure feature image;

[0019] The material feature image and the structure feature image constitute a dual-modal feature image based on heterogeneous complementarity of spatial details and global semantics.

[0020] Further, the deep residual network comprises:

[0021] a convolution layer, configured to extract local material texture from the structure branch image by using its own local induction bias, thereby obtaining an initial material feature image;

[0022] a normalization layer, configured to perform normalization processing on the initial material feature image, thereby obtaining a normalized material feature image;

[0023] a max-pooling layer configured to perform spatial size halving on the normalized material feature image to obtain a pooled material feature image;

[0024] a plurality of cascaded bottleneck layers configured to perform step-by-step expanded receptive field processing on the pooled material feature image to obtain the material feature image.

[0025] Further, the visual Transformer network comprises:

[0026] a patching layer configured to convert a pixel space of the structure branch image into a Token sequence;

[0027] a plurality of cascaded SwinTransformer layers configured to perform progressive learning from local features to abstract global semantics on the Token sequence to obtain the structure feature image.

[0028] Further, the channel-space alignment module comprises:

[0029] a two-path feature projection unit configured to perform convolutional channel adjustment on the dual-modal feature image;

[0030] a dynamic spatial interpolation unit configured to perform feature map size matching on the channel-adjusted dual-modal feature image using a bilinear interpolation method.

[0031] Further, the dual-modal feature image based on spatial details and global semantic heterogeneous complementarity comprises a material feature image and a structure feature image;

[0032] The dual-path cross-modal associated feature image comprises a structure-material associated feature image and a material-structure associated feature image;

[0033] The multiple attention modules comprise:

[0034] a material self-attention layer configured to establish spatial association within the material feature by a query-key value mechanism to obtain a material self-attention image;

[0035] a material position encoding layer configured to perform position encoding on the material self-attention image to generate a material self-attention encoding image;

[0036] a material normalization layer configured to perform normalization processing on the material self-attention encoding image to obtain a material self-attention normalized image;

[0037] a structure self-attention layer configured to capture global dependencies of structure features in the structure feature image by a multi-head attention mechanism to obtain a structure self-attention image;

[0038] a structure position encoding layer, configured to perform position encoding on the structure self-attention image to generate a structure self-attention encoded image;

[0039] a structure normalization layer, configured to perform normalization processing on the structure self-attention encoded image to obtain a structure self-attention normalized image;

[0040] a structure-material cross-attention layer, configured to process relationships between the structure self-attention normalized image and the material self-attention normalized image in different semantic subspaces in a structure-material direction using linear projection weights, to inject local material features into global structure features, and further to obtain a structure-material correlation feature image;

[0041] a material-structure cross-attention layer, configured to process relationships between the material self-attention normalized image and the structure self-attention normalized image in different semantic subspaces in a material-structure direction using linear projection weights, to enhance the complement of local material features to global structure features, and further to obtain a material-structure correlation feature image.

[0042] Further, the structure-material cross-attention layer and the material-structure cross-attention layer have the same structure, and both include:

[0043] a feature projection sub-layer, configured to map the structure self-attention normalized image and the material self-attention normalized image to a query space and a key-value space respectively, to generate a feature projection matrix; in the structure-material direction, the query comes from the structure self-attention normalized image, and the key and the value come from the material self-attention normalized image; in the material-structure direction, the query comes from the material self-attention normalized image, and the key and the value come from the structure self-attention normalized image;

[0044] a dynamic attention weight generator, configured to generate an attention region through feature projection matrix multiplication and softmax normalization processing;

[0045] a feature reconstruction sub-layer, configured to perform weighted fusion of the attention weight and an original feature in the attention region, and configure a proportion factor, to obtain the structure-material correlation feature image or the material-structure correlation feature image.

[0046] Further, the gating fusion module includes:

[0047] a feature concatenation layer, configured to concatenate the structure-material correlation feature image and the material-structure correlation feature image along a channel dimension to obtain a concatenated image;

[0048] A gated weight generator layer is used to learn spatial weights based on the stitched image and reduce the number of channels to 1 using convolution to generate adaptive gated weights.

[0049] An adaptive feature mixer layer is used to dynamically weight and fuse the structure-material association feature image and the material-structure association feature image according to the adaptive gating weights, generating a pixel-level segmentation result image; wherein, the formula for dynamic weighted fusion is:

[0050] ;

[0051] This represents the pixel-level segmentation result image. This represents the material-structure association feature image. This represents the structure-material correlation feature image. This represents the adaptive gating weight.

[0052] Furthermore, the lightweight high-resolution decoder module includes multiple cascaded feature upsampling units, each of which includes a convolutional layer, a batch normalization layer, and an activation function layer connected in sequence; wherein, the convolutional layer in the first-level feature upsampling unit is a 3×3 convolutional layer, and the convolutional layers in the other feature upsampling units are transposed convolutional layers.

[0053] Based on the aforementioned asphalt concrete image segmentation system based on a gated fusion mechanism, this invention also provides an asphalt concrete image segmentation method based on a gated fusion mechanism.

[0054] A gated fusion-based asphalt concrete image segmentation method, applied to the gated fusion-based asphalt concrete image segmentation system described above, includes:

[0055] The original image to be segmented is processed by region labeling and edge detection to obtain corresponding material branch images and structure branch images;

[0056] The material branch image is locally inductively biased to extract local material texture, and the structural branch image is globally modeled to extract global interface structure, thereby obtaining a bimodal feature image based on spatial details and global semantic heterogeneous complementarity.

[0057] Channel adjustment and spatial size matching are performed on the dual-modal feature image;

[0058] Self-attention and cross-attention processing are performed on the channel-space aligned bimodal feature images to establish long-range dependencies between the bimodal feature images, thereby obtaining bimodal cross-modal associated feature images;

[0059] An adaptive gating fusion mechanism is used to dynamically weight and fuse the dual-path cross-modal correlation feature images, so that the fusion weights are biased towards material branches in the boundary region and towards structural branches in the internal region, thereby obtaining the fused feature image;

[0060] The fused feature image is subjected to multi-level progressive upsampling operations to generate a pixel-level segmentation result image.

[0061] The beneficial effects of this invention are as follows: This invention provides an image segmentation system and method for asphalt concrete based on a gated fusion mechanism. First, it extracts local texture features and global semantic features of the material based on a dual-branch approach. Then, it achieves cross-modal channel-spatial dimension matching through a dynamic feature alignment mechanism. Next, it establishes bidirectional cross-attention interaction by combining multiple attention collaboration. Then, it dynamically fuses bidirectional cross-attention features using gated weights. Finally, it restores details through progressive upsampling. This system and method can effectively suppress modal suppression, improve the segmentation accuracy of multiphase material interfaces, and achieve pixel-level semantic segmentation of the material's microstructure while ensuring memory efficiency. It is suitable for high-precision image analysis of heterogeneous composite materials such as poured asphalt concrete. Attached Figure Description

[0062] Figure 1 This is a structural block diagram of an asphalt concrete image segmentation system based on a gated fusion mechanism according to the present invention.

[0063] Figure 2 Here is a block diagram of the deep residual network;

[0064] Figure 3 Here is a block diagram of the structure of the visual Transformer network;

[0065] Figure 4 Here is a block diagram of the multi-attention module;

[0066] Figure 5 This is a block diagram of the lightweight high-resolution decoder module.

[0067] Figure 6 This is a flowchart of an image segmentation method for asphalt concrete based on a gated fusion mechanism according to the present invention. Detailed Implementation

[0068] The principles and features of the present invention are described below with reference to the accompanying drawings. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention.

[0069] like Figure 1 As shown, an asphalt concrete image segmentation system based on a gated fusion mechanism includes:

[0070] The dual-branch image acquisition module is used to perform region labeling and edge detection processing on the original image to be segmented, so as to obtain material branch images and structure branch images respectively.

[0071] The heterogeneous dual-branch feature extraction module is used to perform local inductive bias on the material branch image to extract local material texture, and to perform global modeling on the structural branch image to extract global interface structure, thereby obtaining a dual-modal feature image based on spatial details and global semantic heterogeneous complementarity.

[0072] The channel spatial alignment module is used to perform channel adjustment and spatial size matching on the dual-modal feature image;

[0073] A multi-attention module is used to perform self-attention and cross-attention processing on the channel-space aligned bimodal feature images to establish long-range dependencies between the bimodal feature images, thereby obtaining bimodal cross-modal associated feature images.

[0074] The gated fusion module is used to dynamically weight and fuse the dual-path cross-modal correlation feature images using an adaptive gated fusion mechanism, so that the fusion weights are biased towards material branches in the boundary region and towards structural branches in the internal region, thereby obtaining a fused feature image.

[0075] A lightweight high-resolution decoder module is used to perform multi-level progressive upsampling operations on the fused feature image to generate a pixel-level segmentation result image.

[0076] The following is a detailed introduction to each module:

[0077] In some embodiments, the dual-branch image acquisition module includes:

[0078] An annotation unit is used to annotate the regions of the image to be segmented to obtain the material branch image;

[0079] An edge detection unit is used to perform interface edge detection on the image to be segmented to obtain the structural branch image.

[0080] Specifically, this embodiment takes poured asphalt concrete as an example to perform image segmentation on high-resolution poured asphalt concrete. First, preparatory work is carried out by preparing a large batch of poured asphalt concrete samples. After curing, these samples are cut and captured using digital equipment to obtain high-resolution original images. Annotation software is used to annotate the regions of the high-resolution poured asphalt concrete original images to obtain material branch images; an improved edge detection algorithm is used to perform interface edge detection on the high-resolution poured asphalt concrete original images to obtain structural branch images. Training and validation sets are divided according to requirements. Data augmentation operations are performed on the training set, including random horizontal flipping, vertical flipping, rotation, color transformation, random cropping, scaling, translation, and shearing, to increase the diversity of training samples and enhance the robustness of the model.

[0081] In some embodiments, the heterogeneous dual-branch feature extraction module includes:

[0082] A deep residual network is used to perform local inductive bias on the material branch image to extract local material texture and obtain a material feature image.

[0083] A visual Transformer network is used to globally model the structural branch image to extract the global interface structure and obtain a structural feature image.

[0084] The material feature image and the structural feature image constitute a bimodal feature image based on spatial detail and global semantic heterogeneity complementarity.

[0085] Specifically, this invention utilizes the local inductive bias of residual networks and the global modeling capability of visual Transformers to capture features from local material textures and global interface structures, respectively, forming complementary spatial details and global semantic heterogeneous features, thereby improving the model's ability to jointly model local and global information and balancing performance and computational cost.

[0086] In some embodiments, such as Figure 2 As shown, the deep residual network includes:

[0087] Convolutional layers are used to extract local material textures from the structural branch image using their own local inductive bias, to obtain an initial material feature image;

[0088] The normalization layer normalizes the initial material feature image to obtain a normalized material feature image.

[0089] The max pooling layer halves the spatial size of the normalized material feature image to obtain the pooled material feature image;

[0090] Multiple cascaded bottleneck layers are used to progressively expand the receptive field of the pooled material feature image to obtain the material feature image.

[0091] Specifically, the material branch image is input into a deep residual network, and material texture features are extracted through the inductive bias of convolution; then max pooling is performed to halve the spatial size; and then the receptive field is gradually expanded through multiple bottleneck layers to output shallow features. The detailed processing formula is as follows:

[0092] ;

[0093] ;

[0094] ;

[0095] ;

[0096] ;

[0097] in, This indicates that a 7×7 convolution pair is used for the input. (i.e., the structural branch image) is convolved to obtain the initial material feature image. ; Represents the initial material feature image Normalization is performed to obtain (i.e., normalized material feature image); Indicates the use of activation function on Introducing a nonlinear transformation, we obtain ; Indicates to Perform max pooling operation to obtain (i.e., pooling material feature image); This represents multiple cascaded bottleneck layers (four bottleneck layers are set in this embodiment). The residuals are calculated sequentially to gradually expand the receptive field, using different amounts of residuals at different stages.

[0098] In some embodiments, such as Figure 3 As shown, the visual Transformer network includes:

[0099] A block layer is used to convert the pixel space of the structured branch image into a token sequence;

[0100] Multiple cascaded SwinTransformer layers are used to progressively learn the token sequence from local features to abstract global semantics to obtain the structural feature image.

[0101] Specifically, in the structural branch, the structural branch image is converted into a token sequence through a block layer, embedding initial features to achieve efficient modeling of single-channel image feature data in the reconstruction input layer. Then, features are progressively extracted through multiple cascaded SwinTransformer layers (four SwinTransformer layers in this embodiment), achieving multi-scale structural understanding through progressive learning from local features to abstract global semantics, thereby outputting multi-stage features, i.e., the structural feature image. Each SwinTransformer layer contains a Swin sub-layer and an image merging sub-layer. During the progressive learning process from local features to abstract global semantics in multiple cascaded SwinTransformer layers, a hierarchical shift window-based attention mechanism is adopted, with configurable embedding dimension parameters and the number of attention heads. The basic configuration includes a stage-wise attention head setting of [3, 6, 12, 24]. Furthermore, the formula for calculating the window attention in the k-th stage is as follows:

[0102] , ;

[0103] in, Indicates a query. Indicates key, Represents value, This represents the input features of the k-th stage SwinTransformer layer. This represents a trainable query weight matrix used to project input features onto the query space. This represents a trainable key weight matrix used to project input features onto the key space. This represents a trainable value weight matrix used to project input features onto the value space. This represents the window attention in the k-th stage. This represents the relative position offset matrix within the window. This represents the scaling factor for the k-th stage.

[0104] In some embodiments, the channel space alignment module includes:

[0105] A dual-path feature projection unit is used to perform convolution channel adjustment on the dual-modal feature image;

[0106] The dynamic spatial interpolation unit is used to perform feature map size matching on the channel-adjusted bimodal feature image using bilinear interpolation.

[0107] Specifically, when adjusting the convolution channels in a dual-modal feature image, a 1×1 convolution channel adjustment is performed.

[0108] In some embodiments, such as Figure 4As shown, the multi-attention module includes:

[0109] The material self-attention layer is used to establish spatial relationships within the material features of the material feature image through a query-key mechanism, thereby obtaining a material self-attention image.

[0110] A material location encoding layer is used to encode the location of the material self-attention image to generate a material self-attention encoded image.

[0111] The material normalization layer is used to normalize the material self-attention encoding image to obtain a material self-attention normalized image.

[0112] A structural self-attention layer is used to capture the global dependencies of structural features in the structural feature image through a multi-head attention mechanism, thereby obtaining a structural self-attention image;

[0113] A structural position encoding layer is used to perform position encoding on the structural self-attention image to generate a structural self-attention encoded image.

[0114] A structure normalization layer is used to normalize the structure self-attention encoded image to obtain a structure self-attention normalized image.

[0115] A structure-material cross-attention layer is used to process the relationship between the structure self-attention normalized image and the material self-attention normalized image in different semantic subspaces using linear projection weights in the structure-material direction, so as to inject local material features into the global structural features and thus obtain the structure-material associated feature image.

[0116] The material-structure cross-attention layer is used to process the relationship between the material self-attention normalized image and the structure self-attention normalized image in different semantic subspaces using linear projection weights in the material-structure direction, so as to enhance the supplementation of local material features to global structural features, and thus obtain the material-structure associated feature image.

[0117] Specifically, the multi-attention mechanism in this invention is used to effectively integrate complementary information from material and structural branches. Multi-attention includes material branch self-attention, structural branch self-attention, and cross-modal cross-attention (cross-modal cross-attention includes structure-material cross-attention and material-structure cross-attention). After the structure-material cross-attention and material-structure cross-attention processes features in parallel, an enhanced feature representation is generated through the adaptive gating fusion mechanism of the subsequent gating fusion module.

[0118] The material branch self-attention and the structure branch self-attention use the same architecture but have different independent parameters, processing the features of their respective branches. In self-attention (CA), given the input feature map, global average pooling is performed first, then two fully connected layers are used to compress the number of channels to C / 8 before restoring them, as shown in the following formula:

[0119] ;

[0120] in, Indicates self-attention, This represents the output feature map. This indicates global splicing pooling. This represents the Sigmoid activation function. and Let represent the weight matrix, where Used for dimensionality reduction fully connected layers Used to restore the number of channels. This is the ReLU activation function.

[0121] In some embodiments, the structure-material cross-attention layer has the same structure as the material-structure cross-attention layer, both comprising:

[0122] A feature projection sublayer is used to map the structure self-attention normalized image and the material self-attention normalized image to the query space and the key-value space, respectively, to generate a feature projection matrix; wherein, in the structure-material direction, the query comes from the structure self-attention normalized image, and the key and value come from the material self-attention normalized image; in the material-structure direction, the query comes from the material self-attention normalized image, and the key and value come from the structure self-attention normalized image;

[0123] The dynamic attention weight generator generates the region of interest through feature projection matrix multiplication and softmax normalization; the specific form of feature projection matrix multiplication is the right multiplication of the input feature matrix by the projection weight matrix.

[0124] A feature reconstruction sublayer is used to weight and fuse the attention weights with the original features in the region of interest, and configure a scaling factor to obtain the structure-material association feature image or the material-structure association feature image.

[0125] Specifically, in the material→structure and structure→material directions, independent linear projection weights are used to process the relationships between different semantic subspaces in parallel. In the material→structure direction, the query (Q) comes from the features of the material branch, while the key (K) and value (V) come from the features of the structure branch, enhancing the supplementation of structural information by material details. In the structure→material direction, the query (Q) comes from the features of the structure branch, while the key (K) and value (V) come from the features of the material branch, injecting local features into the global structure. Local material features can correct misjudgments of global semantic information, while global structural features can constrain the rationality of local material classification, achieving bidirectional complementarity between local material details and global semantic information, forcing equal interaction between the two modalities, and ensuring balanced information fusion. This effectively solves the problem of feature space offset in the two branches and establishes cross-modal associations. The formula is as follows:

[0126] ;

[0127] ;

[0128] ;

[0129] in, Indicates cross attention. Represents the original dimension of the input features. Indicates the number of attention heads. Indicates the scaling factor. This indicates that normalization is performed to generate a weight distribution. Indicates material branches, Indicates structural branches, This represents the material-structure association feature image. This represents the structure-material associated feature image.

[0130] In some embodiments, the gating fusion module includes:

[0131] The feature stitching layer is used to stitch the structure-material association feature image and the material-structure association feature image along the channel dimension to obtain a stitched image;

[0132] A gated weight generator layer is used to learn spatial weights based on the stitched image and reduce the number of channels to 1 using convolution to generate adaptive gated weights.

[0133] An adaptive feature mixer layer is used to dynamically weight and fuse the structure-material association feature image and the material-structure association feature image according to the adaptive gating weights, generating a pixel-level segmentation result image; wherein, the formula for dynamic weighted fusion is:

[0134] ;

[0135] This represents the pixel-level segmentation result image. This represents the material-structure association feature image. This represents the structure-material correlation feature image. This represents the adaptive gating weight.

[0136] Specifically, the gating fusion module dynamically generates gating weights instead of fixed weights, improving model flexibility. First, channel compression is performed by concatenating the cross-attention results along the channel dimension. A 3×3 convolution is used to compress the channels, reducing computation, and then an activation function is used to introduce non-linearity. Next, spatial weight learning is performed, further reducing the number of channels to 1 using a 1×1 convolution to generate spatially sensitive weights. Then, an activation function is used for normalization, mapping the weights to [0,1]. Finally, the two feature streams are weighted and summed according to the weights. This ensures that in material boundary regions, gating favors material branches, improving edge segmentation accuracy; in the interior regions of materials, gating favors structural branches, ensuring class consistency. The adaptive gating weight formula is as follows:

[0137] ;

[0138] in, This represents the Sigmoid activation function; Represents a 1×1 convolution; Indicates a splicing operation; This represents the result of cross-attention between two paths; Indicates the number of channels. Indicates dual-channel characteristics; This indicates the number of pixels or spatial size of the feature map along the Y-axis. This represents the number of pixels or spatial size of the feature map along the X-axis; each image It's just a passage, one The feature map is open Stacked together, Carrying feature type information, and Spatial location information that shares common characteristics.

[0139] This invention uses gating weights to assign higher weights to material branches at object boundaries. →1) To preserve interface details, assign higher weights to structural branches within the object. →0) to maintain semantic consistency. In addition, the gated fusion module uses only one 3×3 convolution and one 1×1 convolution to replace the fully connected layer, effectively reducing the number of model parameters while maintaining high performance.

[0140] In some embodiments, such as Figure 5 As shown, the lightweight high-resolution decoder module includes multiple cascaded feature upsampling units. Each feature upsampling unit includes a convolutional layer, a batch normalization layer, and an activation function layer connected in sequence. The convolutional layer in the first feature upsampling unit is a 3×3 convolutional layer, and the convolutional layers in the other feature upsampling units are transposed convolutional layers.

[0141] Specifically, in the lightweight high-resolution decoder module, progressive upsampling is employed, consisting of multiple cascaded feature upsampling units, to restore the feature map to the input resolution and generate a high-resolution output. The formula is as follows:

[0142] ;

[0143] in, Indicates the first Second upsampling result, This represents the activation function. Indicates batch normalization. This represents transposed convolution. Indicates the first Second upsampling result.

[0144] The lightweight high-resolution decoder module employs a progressive resolution recovery path, restoring the feature map to the input resolution through at least five 2x upsampling operations. The final output layer contains a category projection layer with a 1×1 convolutional kernel, outputting the segmented image. The asphalt concrete image segmentation system based on the gated fusion mechanism of this invention is further evaluated below using qualitative evaluation metrics.

[0145] Based on the preparation process of grouted asphalt concrete, it can be inferred that during the hot mixing process, the aggregate is completely surrounded by asphalt grout, and then grout is injected into the gaps, forming a transitional form of "aggregate-asphalt grout-aggregate" or "aggregate-asphalt grout-grout-asphalt grout-aggregate". Therefore, in addition to commonly used evaluation indicators such as mean intersection-over-union ratio (mIoU) and F1 score, further consideration should be given to aggregate independence, asphalt grout encapsulation integrity, and edge sharpness.

[0146] This embodiment designs an Aggregate Separation Index (ASI) to quantify the degree of adhesion between aggregates, as shown in the following formula. This represents the number of aggregate pairs that adhere to each other after segmentation, determined by calculating the contact pixels or overlapping boundaries between aggregate regions. The theoretical independent quantity of all aggregates in the image is obtained through manual annotation.

[0147] ;

[0148] In addition, an Asphalt Coating Integrity (ACI) index is designed to measure whether the asphalt layer is continuous and closed, and the formula is as follows. Representing the The effective perimeter of the asphalt wrapping around the aggregate (closed and without breaks). Representing the The actual perimeter of each aggregate. This indicates the total number of aggregates.

[0149] ;

[0150] Finally, an edge sharpness (ES) index was designed to quantify the sharpness of the aggregate-asphalt boundary, as shown in the following formula. Wherein, Represents boundary pixels The gradient intensity at a given point is obtained by calculating using the Sobel operator. Represents the total number of boundary pixels.

[0151] ;

[0152] This embodiment uses four semantic segmentation schemes from baseline models (UNet and DeepLabv3+) and advanced methods (TransUNet and Swin-UNet) as benchmark schemes. All performance-related benchmark schemes were trained on a self-built asphalt concrete injection dataset, and the qualitative evaluation results are shown in Table 1 below.

[0153] Table 1 Qualitative evaluation results of each model

[0154]

[0155] The significant improvements in mIoU (83.7%) and F1 (89.2%) demonstrate the advantages of the proposed method in pixel-level classification and class balance prediction. The improved aggregate independence index (93.1%) indicates its stronger ability to separate cohesive aggregates. The inclusion integrity (94.1%) verifies its accurate modeling of the closed continuity of asphalt mastic. The breakthrough performance in edge sharpness (0.912) demonstrates the advantage of bimodal features in segmenting regions with blurred boundaries. These results show that the proposed method is significantly superior to existing solutions.

[0156] Based on the aforementioned asphalt concrete image segmentation system based on a gated fusion mechanism, this invention also provides an asphalt concrete image segmentation method based on a gated fusion mechanism.

[0157] like Figure 6 As shown, an asphalt concrete image segmentation method based on a gated fusion mechanism is applied to the asphalt concrete image segmentation system based on the gated fusion mechanism described above, comprising:

[0158] The original image to be segmented is processed by region labeling and edge detection to obtain corresponding material branch images and structure branch images;

[0159] The material branch image is locally inductively biased to extract local material texture, and the structural branch image is globally modeled to extract global interface structure, thereby obtaining a bimodal feature image based on spatial details and global semantic heterogeneous complementarity.

[0160] Channel adjustment and spatial size matching are performed on the dual-modal feature image;

[0161] Self-attention and cross-attention processing are performed on the channel-space aligned bimodal feature images to establish long-range dependencies between the bimodal feature images, thereby obtaining bimodal cross-modal associated feature images;

[0162] An adaptive gating fusion mechanism is used to dynamically weight and fuse the dual-path cross-modal correlation feature images, so that the fusion weights are biased towards material branches in the boundary region and towards structural branches in the internal region, thereby obtaining the fused feature image;

[0163] The fused feature image is subjected to multi-level progressive upsampling operations to generate a pixel-level segmentation result image.

[0164] The steps of the asphalt concrete image segmentation method based on the gating fusion mechanism of the present invention are described in the specific functions of each module of the asphalt concrete image segmentation system based on the gating fusion mechanism of the present invention, and will not be repeated here.

[0165] This invention discloses an image segmentation system and method for asphalt concrete based on a gated fusion mechanism. First, it extracts local texture features and global semantic features of the material through a dual-branch approach. Then, it achieves cross-modal channel-spatial dimension matching through a dynamic feature alignment mechanism. Next, it establishes bidirectional cross-attention interaction by combining multiple attention mechanisms. Then, it dynamically fuses the bidirectional cross-attention features using gated weights. Finally, it restores details through progressive upsampling. This system and method can effectively suppress modal suppression, improve the segmentation accuracy of multiphase material interfaces, and achieve pixel-level semantic segmentation of the material's microstructure while ensuring memory efficiency. It is suitable for high-precision image analysis of heterogeneous composite materials such as poured asphalt concrete.

[0166] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An asphalt concrete image segmentation system based on a gated fusion mechanism, characterized in that, include: The dual-branch image acquisition module is used to perform region labeling and edge detection processing on the original image to be segmented, so as to obtain material branch images and structure branch images respectively. The heterogeneous dual-branch feature extraction module is used to perform local inductive bias on the material branch image to extract local material texture, and to perform global modeling on the structural branch image to extract global interface structure, thereby obtaining a dual-modal feature image based on spatial details and global semantic heterogeneous complementarity. The channel spatial alignment module is used to perform channel adjustment and spatial size matching on the dual-modal feature image; A multi-attention module is used to perform self-attention and cross-attention processing on the channel-space aligned bimodal feature images to establish long-range dependencies between the bimodal feature images, thereby obtaining bimodal cross-modal associated feature images. The gated fusion module is used to dynamically weight and fuse the dual-path cross-modal correlation feature images using an adaptive gated fusion mechanism, so that the fusion weights are biased towards material branches in the boundary region and towards structural branches in the internal region, thereby obtaining a fused feature image. A lightweight high-resolution decoder module is used to perform multi-level progressive upsampling operations on the fused feature image to generate a pixel-level segmentation result image.

2. The asphalt concrete image segmentation system based on gated fusion mechanism according to claim 1, characterized in that, The heterogeneous dual-branch feature extraction module includes: A deep residual network is used to perform local inductive bias on the material branch image to extract local material texture and obtain a material feature image. A visual Transformer network is used to globally model the structural branch image to extract the global interface structure and obtain a structural feature image. The material feature image and the structural feature image constitute a bimodal feature image based on spatial detail and global semantic heterogeneity complementarity.

3. The asphalt concrete image segmentation system based on gated fusion mechanism according to claim 2, characterized in that, The deep residual network includes: Convolutional layers are used to extract local material textures from the structural branch image using their own local inductive bias, to obtain an initial material feature image; The normalization layer normalizes the initial material feature image to obtain a normalized material feature image. The max pooling layer halves the spatial size of the normalized material feature image to obtain the pooled material feature image; Multiple cascaded bottleneck layers are used to progressively expand the receptive field of the pooled material feature image to obtain the material feature image.

4. The asphalt concrete image segmentation system based on gated fusion mechanism according to claim 2, characterized in that, The visual Transformer network includes: A block layer is used to convert the pixel space of the structured branch image into a token sequence; Multiple cascaded SwinTransformer layers are used to progressively learn the token sequence from local features to abstract global semantics to obtain the structural feature image.

5. The asphalt concrete image segmentation system based on gated fusion mechanism according to claim 1, characterized in that, The channel space alignment module includes: A dual-path feature projection unit is used to perform convolution channel adjustment on the dual-modal feature image; The dynamic spatial interpolation unit is used to perform feature map size matching on the channel-adjusted bimodal feature image using bilinear interpolation.

6. The asphalt concrete image segmentation system based on gated fusion mechanism according to claim 1, characterized in that, Dual-modal feature images based on spatial detail and global semantic heterogeneity complementarity include material feature images and structural feature images; The dual-path cross-modal correlation feature image includes a structure-material correlation feature image and a material-structure correlation feature image; The multi-attention module includes: The material self-attention layer is used to establish spatial relationships within the material features of the material feature image through a query-key mechanism, thereby obtaining a material self-attention image. A material location encoding layer is used to encode the location of the material self-attention image to generate a material self-attention encoded image. The material normalization layer is used to normalize the material self-attention encoding image to obtain a material self-attention normalized image. A structural self-attention layer is used to capture the global dependencies of structural features in the structural feature image through a multi-head attention mechanism, thereby obtaining a structural self-attention image; A structural position encoding layer is used to perform position encoding on the structural self-attention image to generate a structural self-attention encoded image. A structure normalization layer is used to normalize the structure self-attention encoded image to obtain a structure self-attention normalized image. A structure-material cross-attention layer is used to process the relationship between the structure self-attention normalized image and the material self-attention normalized image in different semantic subspaces using linear projection weights in the structure-material direction, so as to inject local material features into the global structural features and thus obtain the structure-material associated feature image. The material-structure cross-attention layer is used to process the relationship between the material self-attention normalized image and the structure self-attention normalized image in different semantic subspaces using linear projection weights in the material-structure direction, so as to enhance the supplementation of local material features to global structural features, and thus obtain the material-structure associated feature image.

7. The asphalt concrete image segmentation system based on gated fusion mechanism according to claim 6, characterized in that, The structure-material cross-attention layer has the same structure as the material-structure cross-attention layer, both including: A feature projection sublayer is used to map the structure self-attention normalized image and the material self-attention normalized image to the query space and the key-value space, respectively, to generate a feature projection matrix; wherein, in the structure-material direction, the query comes from the structure self-attention normalized image, and the key and value come from the material self-attention normalized image; in the material-structure direction, the query comes from the material self-attention normalized image, and the key and value come from the structure self-attention normalized image; The dynamic attention weight generator generates regions of interest through feature projection matrix multiplication and softmax normalization. A feature reconstruction sublayer is used to weight and fuse the attention weights with the original features in the region of interest, and configure a scaling factor to obtain the structure-material association feature image or the material-structure association feature image.

8. The asphalt concrete image segmentation system based on gated fusion mechanism according to claim 1, characterized in that, The dual-path cross-modal correlation feature image includes a structure-material correlation feature image and a material-structure correlation feature image; The gating fusion module includes: The feature stitching layer is used to stitch the structure-material association feature image and the material-structure association feature image along the channel dimension to obtain a stitched image; A gated weight generator layer is used to learn spatial weights based on the stitched image and reduce the number of channels to 1 using convolution to generate adaptive gated weights. An adaptive feature mixer layer is used to dynamically weight and fuse the structure-material association feature image and the material-structure association feature image according to the adaptive gating weights, generating a pixel-level segmentation result image; wherein, the formula for dynamic weighted fusion is: ; This represents the pixel-level segmentation result image. This represents the material-structure association feature image. This represents the structure-material correlation feature image. This represents the adaptive gating weight.

9. The asphalt concrete image segmentation system based on gated fusion mechanism according to claim 1, characterized in that, The lightweight high-resolution decoder module includes multiple cascaded feature upsampling units. Each feature upsampling unit includes a convolutional layer, a batch normalization layer, and an activation function layer connected in sequence. The convolutional layer in the first feature upsampling unit is a 3×3 convolutional layer, and the convolutional layers in the other feature upsampling units are transposed convolutional layers.

10. A method for segmenting asphalt concrete images based on a gated fusion mechanism, characterized in that, The asphalt concrete image segmentation system based on a gated fusion mechanism as described in any one of claims 1 to 9 includes: The original image to be segmented is processed by region labeling and edge detection to obtain corresponding material branch images and structure branch images; The material branch image is locally inductively biased to extract local material texture, and the structural branch image is globally modeled to extract global interface structure, thereby obtaining a bimodal feature image based on spatial details and global semantic heterogeneous complementarity. Channel adjustment and spatial size matching are performed on the dual-modal feature image; Self-attention and cross-attention processing are performed on the channel-space aligned bimodal feature images to establish long-range dependencies between the bimodal feature images, thereby obtaining bimodal cross-modal associated feature images; An adaptive gating fusion mechanism is used to dynamically weight and fuse the dual-path cross-modal correlation feature images, so that the fusion weights are biased towards material branches in the boundary region and towards structural branches in the internal region, thereby obtaining the fused feature image; The fused feature image is subjected to multi-level progressive upsampling operations to generate a pixel-level segmentation result image.

Images