Asphalt concrete image segmentation system and method based on gating fusion mechanism

Abstract

The invention relates to an asphalt concrete image segmentation system and method based on a gating fusion mechanism. The system comprises a double-branch image acquisition module; a heterogeneous double-branch feature extraction module; a channel space alignment module; a multiple attention module; a gating fusion module; the invention relates to a lightweight high-resolution decoder module. According to the method, local texture features of a material and global semantic features of a structure are extracted based on double branches, cross-modal channel-space dimension matching is realized through a dynamic feature alignment mechanism, bidirectional cross attention interaction is established in combination with multiple attention collaboration, and then bidirectional cross attention features are dynamically fused by using gating weight, so that the cross attention interaction efficiency is improved. Details are recovered through progressive up-sampling; according to the method, the modal suppression phenomenon can be effectively inhibited, the multi-phase material interface segmentation precision is improved, the pixel-level semantic segmentation of the material microstructure is realized while the video memory efficiency is ensured, and the method is suitable for high-precision image analysis of heterogeneous composite materials such as poured asphalt concrete.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to an asphalt concrete image segmentation system and method based on a gated fusion mechanism. Background Art

[0002] Asphalt concrete is a heterogeneous composite material composed of aggregate, asphalt mortar, grouting material, and residual voids. Its mechanical properties are highly dependent on the spatial distribution of its mesostructure. Finite element-based micromechanics methods provide a new research approach to revealing the complex mechanical behavior of asphalt concrete. Image processing is the starting point of all research, and mesoscopic parameters obtained through high-precision image processing are crucial for accurate research conclusions. However, commercial FEA software lacks dedicated image processing modules for complex multiphase asphalt concrete, hindering the efficient transfer of technology. Real geometry reconstruction technology based on image segmentation is gradually becoming mainstream.

[0003] Existing traditional segmentation methods (such as UNet and DeepLab series) are based on a single feature extraction network, which makes it difficult to take into account both local details and global contextual information, and are insufficiently sensitive to the boundaries of the microscopic multiphase structure of composite materials.

[0004] Existing unimodal deep learning models find it difficult to balance local texture and global structural features, while traditional multimodal fusion methods ignore the anisotropic characteristics of materials, resulting in poor semantic consistency of multi-scale structures.

[0005] The existing conventional attention mechanism (Transformer) has low computational efficiency in feature interaction and lacks differentiated processing for different modal features. When facing a two-branch network, there is a modal suppression phenomenon, resulting in the loss of structural edge details.

[0006] In addition, when processing high-resolution images, existing models have excessive video memory usage, making it difficult to strike a balance between efficiency and accuracy. Summary of the Invention

[0007] The present invention provides an asphalt concrete image segmentation system and method based on a gated fusion mechanism to solve at least one of the above technical problems.

[0008] The present invention solves the above technical problems with the following technical solutions: an asphalt concrete image segmentation system based on a gated fusion mechanism, comprising: The dual-branch image acquisition module is used to perform region annotation and edge detection processing on the original image to be segmented, so as to obtain the material branch image and the structure branch image respectively; a heterogeneous dual-branch feature extraction module for performing local inductive bias on the material branch image to extract local material texture, and performing global modeling on the structure branch image to extract global interface structure, thereby obtaining a dual-modal feature image based on heterogeneous complementarity of spatial details and global semantics; A channel space alignment module, used for performing channel adjustment and spatial size matching on the bimodal feature image; A multi-attention module is used to perform self-attention and cross-attention processing on the bimodal feature images after channel space alignment to establish long-range dependencies between the bimodal feature images and thereby obtain a dual-path cross-modal correlation feature image; a gated fusion module for dynamically weighting the dual-path cross-modal correlation feature images using an adaptive gated fusion mechanism, so that the fusion weights are biased toward the material branch in the boundary area and toward the structure branch in the internal area, thereby obtaining a fused feature image; The lightweight high-resolution decoder module is used to perform a multi-level progressive upsampling operation on the fused feature image to generate a pixel-level segmentation result image.

[0009] On the basis of the above technical solution, the present invention can also be improved as follows.

[0010] Furthermore, the heterogeneous dual-branch feature extraction module includes: A deep residual network is used to perform local inductive bias on the material branch image to extract local material texture and obtain a material feature image; a visual Transformer network, configured to globally model the structural branch image to extract the global interface structure and obtain a structural feature image; The material feature image and the structural feature image constitute a dual-modal feature image based on heterogeneous complementarity of spatial details and global semantics.

[0011] Furthermore, the deep residual network includes: A convolutional layer, configured to extract local material texture from the structure branch image using its own local inductive bias to obtain an initial material feature image; a normalization layer, performing normalization processing on the initial material characteristic image to obtain a normalized material characteristic image; a maximum pooling layer, which reduces the spatial size of the normalized material feature image by half to obtain a pooled material feature image; Multiple cascaded bottleneck layers are used to gradually expand the receptive field of the pooled material feature image to obtain the material feature image.

[0012] Furthermore, the visual Transformer network includes: A block layer, used to convert the pixel space of the structure branch image into a Token sequence; Multiple cascaded SwinTransformer layers are used to progressively learn the Token sequence from local features to abstract global semantics to obtain the structural feature image.

[0013] Furthermore, the channel space alignment module includes: A dual-path feature projection unit, configured to perform convolution channel adjustment on the dual-modal feature image; The dynamic spatial interpolation unit is used to perform feature map size matching on the bimodal feature image after channel adjustment using a bilinear interpolation method.

[0014] Furthermore, the bimodal feature images based on the heterogeneous complementarity of spatial details and global semantics include material feature images and structural feature images; The dual-path cross-modality correlation feature image includes a structure-material correlation feature image and a material-structure correlation feature image; The multiple attention module includes: A material self-attention layer is used to establish spatial correlations within the material features of the material feature image through a query-key value mechanism to obtain a material self-attention image; A material position encoding layer, configured to perform position encoding on the material self-attention image to generate a material self-attention encoded image; A material normalization layer, configured to normalize the material self-attention encoded image to obtain a material self-attention normalized image; A structural self-attention layer is used to capture the global dependency of structural features in the structural feature image through a multi-head attention mechanism to obtain a structural self-attention image; A structure position encoding layer, configured to perform position encoding on the structure self-attention image to generate a structure self-attention encoded image; A structure normalization layer, configured to normalize the structure self-attention encoded image to obtain a structure self-attention normalized image; a structure-material cross attention layer, configured to process the relationship between the structure self-attention normalized image and the material self-attention normalized image in different semantic subspaces using linear projection weights in the structure-material direction, so as to inject local material features into the global structure features, thereby obtaining the structure-material association feature image; The material-structure cross attention layer is used to process the relationship between the material self-attention normalized image and the structure self-attention normalized image in different semantic subspaces using linear projection weights in the material-structure direction, so as to enhance the complementation of local material features to global structural features, and thereby obtain the material-structure association feature image.

[0015] Furthermore, the structure-material cross attention layer has the same structure as the material-structure cross attention layer, and both include: A feature projection sublayer is used to map the structure self-attention normalized image and the material self-attention normalized image to a query space and a key-value space, respectively, to generate a feature projection matrix; wherein, in the structure-material direction, the query comes from the structure self-attention normalized image, and the key and value come from the material self-attention normalized image; in the material-structure direction, the query comes from the material self-attention normalized image, and the key and value come from the structure self-attention normalized image; Dynamic attention weight generator, which generates attention regions through feature projection matrix multiplication and softmax normalization; The feature reconstruction sublayer is used to perform weighted fusion of the attention weight and the original feature in the focus area, and configure the scaling factor to obtain the structure-material association feature image or the material-structure association feature image.

[0016] Furthermore, the gated fusion module includes: A feature stitching layer, configured to stitch the structure-material association feature image and the material-structure association feature image along a channel dimension to obtain a stitched image; a gating weight generator layer for learning spatial weights based on the stitched image and using convolution to reduce the number of channels to 1 to generate adaptive gating weights; The adaptive feature mixer layer is used to perform dynamic weighted fusion on the structure-material association feature image and the material-structure association feature image according to the adaptive gating weight to generate a pixel-level segmentation result image; wherein the formula for dynamic weighted fusion is: ; represents the pixel-level segmentation result image, represents the material-structure association characteristic image, represents the structure-material association characteristic image, represents the adaptive gating weight.

[0017] Furthermore, the lightweight high-resolution decoder module includes multiple cascaded feature upsampling units, each of which includes a convolutional layer, a batch normalization layer, and an activation function layer connected in sequence; wherein, the convolutional layer in the first-level feature upsampling unit is a 3×3 convolutional layer, and the convolutional layers in other feature upsampling units except the first-level feature upsampling unit are transposed convolutional layers.

[0018] On the basis of the above-mentioned asphalt concrete image segmentation system based on the gated fusion mechanism, the present invention also provides an asphalt concrete image segmentation method based on the gated fusion mechanism.

[0019] A method for asphalt concrete image segmentation based on a gated fusion mechanism, applied to the asphalt concrete image segmentation system based on a gated fusion mechanism as described above, comprises: Performing region labeling and edge detection on the original image to be segmented to obtain material branch images and structure branch images respectively; Performing local inductive bias on the material branch image to extract local material texture, and performing global modeling on the structure branch image to extract global interface structure, thereby obtaining a bimodal feature image based on heterogeneous complementarity of spatial details and global semantics; performing channel adjustment and spatial size matching on the dual-modal feature image; Performing self-attention and cross-attention processing on the bimodal feature images after channel space alignment to establish a long-range dependency relationship between the bimodal feature images, thereby obtaining a dual-path cross-modal correlation feature image; Adopting an adaptive gated fusion mechanism to dynamically weight the dual-path cross-modal correlation feature images so that the fusion weights are biased toward the material branch in the boundary area and toward the structure branch in the internal area, thereby obtaining a fused feature image; A multi-level progressive upsampling operation is performed on the fused feature image to generate a pixel-level segmentation result image.

[0020] The beneficial effects of the present invention are: the present invention provides an asphalt concrete image segmentation system and method based on a gated fusion mechanism, which first extracts the local texture features of the material and the global semantic features of the structure based on dual branches, and then realizes cross-modal channel-space dimension matching through a dynamic feature alignment mechanism, and then combines multiple attentions to establish a two-way cross-attention interaction, and then uses gated weights to dynamically fuse the two-way cross-attention features, and finally restores the details through progressive upsampling; this system and method can effectively suppress the modal suppression phenomenon, improve the segmentation accuracy of the multi-phase material interface, and realize pixel-level semantic segmentation of the material microstructure while ensuring the memory efficiency, and is suitable for high-precision image analysis of heterogeneous composite materials such as poured asphalt concrete. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 This is a structural block diagram of an asphalt concrete image segmentation system based on a gated fusion mechanism according to the present invention; Figure 2 It is the structural block diagram of the deep residual network; Figure 3 This is the structural diagram of the visual Transformer network; Figure 4 This is the structural diagram of the multiple attention module; Figure 5 This is the structural block diagram of the lightweight high-resolution decoder module.

[0022] Figure 6 This is a flow chart of an asphalt concrete image segmentation method based on a gated fusion mechanism according to the present invention. DETAILED DESCRIPTION

[0023] The principles and features of the present invention are described below with reference to the accompanying drawings. The examples given are only used to explain the present invention and are not used to limit the scope of the present invention.

[0024] like Figure 1 As shown, an asphalt concrete image segmentation system based on a gated fusion mechanism includes: The dual-branch image acquisition module is used to perform region annotation and edge detection processing on the original image to be segmented, so as to obtain the material branch image and the structure branch image respectively; a heterogeneous dual-branch feature extraction module for performing local inductive bias on the material branch image to extract local material texture, and performing global modeling on the structure branch image to extract global interface structure, thereby obtaining a dual-modal feature image based on heterogeneous complementarity of spatial details and global semantics; A channel space alignment module, used for performing channel adjustment and spatial size matching on the bimodal feature image; A multi-attention module is used to perform self-attention and cross-attention processing on the bimodal feature images after channel space alignment to establish long-range dependencies between the bimodal feature images and thereby obtain a dual-path cross-modal correlation feature image; a gated fusion module for dynamically weighting the dual-path cross-modal correlation feature images using an adaptive gated fusion mechanism, so that the fusion weights are biased toward the material branch in the boundary area and toward the structure branch in the internal area, thereby obtaining a fused feature image; The lightweight high-resolution decoder module is used to perform a multi-level progressive upsampling operation on the fused feature image to generate a pixel-level segmentation result image.

[0025] The following is a detailed introduction to each module: In some embodiments, the dual-branch image acquisition module includes: a labeling unit, configured to label regions of the image to be segmented to obtain the material branch image; The edge detection unit is used to perform interface edge detection on the image to be segmented to obtain the structural branch image.

[0026] Specifically, this embodiment takes poured asphalt concrete as an example to perform image segmentation on high-resolution poured asphalt concrete. First, preparatory work is carried out to prepare poured asphalt concrete samples in large quantities, cut them after curing, and use digital equipment to collect them to obtain high-resolution original images. Annotation software is used to perform regional annotation on the high-resolution poured asphalt concrete original image to obtain a material branch image; an improved edge detection algorithm is used to perform interface edge detection on the high-resolution poured asphalt concrete original image to obtain a structural branch image. The training set and validation set are divided according to needs, and data enhancement operations are performed on the training set, including random horizontal flipping, vertical flipping, rotation, color conversion, random cropping, scaling, translation, and shearing, to increase the diversity of training samples and enhance the robustness of the model.

[0027] In some embodiments, the heterogeneous dual-branch feature extraction module includes: A deep residual network is used to perform local inductive bias on the material branch image to extract local material texture and obtain a material feature image; a visual Transformer network, configured to globally model the structural branch image to extract the global interface structure and obtain a structural feature image; The material feature image and the structural feature image constitute a dual-modal feature image based on heterogeneous complementarity of spatial details and global semantics.

[0028] Specifically, the present invention utilizes the local inductive bias of the residual network and the global modeling capability of the visual Transformer to capture features from the local material texture and the global interface structure respectively, forming a complementary combination of spatial details and global semantic heterogeneous features, improving the model's ability to jointly model local and global information, and balancing performance and computational complexity.

[0029] In some embodiments, as Figure 2 As shown, the deep residual network includes: A convolutional layer, configured to extract local material texture from the structure branch image using its own local inductive bias to obtain an initial material feature image; a normalization layer, performing normalization processing on the initial material characteristic image to obtain a normalized material characteristic image; a maximum pooling layer, which reduces the spatial size of the normalized material feature image by half to obtain a pooled material feature image; Multiple cascaded bottleneck layers are used to gradually expand the receptive field of the pooled material feature image to obtain the material feature image.

[0030] Specifically, the material branch image is input into the deep residual network, and the material texture features are extracted through the inductive bias of convolution. Then, maximum pooling is performed to halve the spatial size. Then, the receptive field is gradually expanded through multiple bottleneck layers to output shallow features. The detailed processing formula is as follows: ; ; ; ; ; in, Indicates using a 7×7 convolution on the input (i.e., structural branch image) is convolved to obtain the initial material characteristic image ; Represents the initial material characteristic image Normalize it and get (i.e. normalized material characteristic image); Indicates the use of activation function Introducing nonlinear transformation, we get ; Express Perform the maximum pooling operation and get (i.e. pooled material feature images); Indicates multiple cascaded bottleneck layers (four bottleneck layers are set in this embodiment) The residual calculation is performed sequentially to gradually expand the receptive field, and different numbers of residuals are used in different stages.

[0031] In some embodiments, as Figure 3 As shown, the visual Transformer network includes: A block layer, used to convert the pixel space of the structure branch image into a Token sequence; Multiple cascaded SwinTransformer layers are used to progressively learn the Token sequence from local features to abstract global semantics to obtain the structural feature image.

[0032] Specifically, in the structure branch, the pixel space of the structure branch image is converted into a Token sequence through a block layer, and the initial features are embedded to achieve efficient modeling of the single-channel image feature data by the reconstructed input layer. Afterwards, features are gradually extracted through multiple cascaded SwinTransformer layers (four SwinTransformer layers are set in this embodiment), and multi-scale structural understanding is achieved through progressive learning from local features to abstract global semantics, thereby outputting multi-stage features, that is, outputting the structural feature image. Each SwinTransformer layer includes a Swin sublayer and an image merging sublayer; in the process of progressive learning from local features to abstract global semantics in multiple cascaded SwinTransformer layers, an attention mechanism based on a hierarchical shift window is adopted, and the embedding dimension parameters and the number of attention heads can be configured, where the basic configuration includes the stage-by-stage attention head number settings of [3, 6, 12, 24]; in addition, the window attention calculation formula for the kth stage is as follows: , ; in, Indicates a query, Indicates the key, Represents a value, represents the input features of the k-th stage SwinTransformer layer, represents a trainable query weight matrix used to project input features into the query space, represents a trainable key weight matrix used to project input features into the key space, represents a trainable value weight matrix used to project input features into the value space, represents the window attention of the k-th stage, Represents the relative position bias matrix within the window, represents the scaling factor for the kth stage.

[0033] In some embodiments, the channel space alignment module includes: A dual-path feature projection unit, configured to perform convolution channel adjustment on the dual-modal feature image; The dynamic spatial interpolation unit is used to perform feature map size matching on the bimodal feature image after channel adjustment using a bilinear interpolation method.

[0034] Specifically, when the convolution channel adjustment is performed on the bimodal feature image, a 1×1 convolution channel adjustment is performed.

[0035] In some embodiments, as Figure 4 As shown, the multiple attention module includes: A material self-attention layer is used to establish spatial correlations within the material features of the material feature image through a query-key value mechanism to obtain a material self-attention image; A material position encoding layer, configured to perform position encoding on the material self-attention image to generate a material self-attention encoded image; A material normalization layer, configured to normalize the material self-attention encoded image to obtain a material self-attention normalized image; A structural self-attention layer is used to capture the global dependency of structural features in the structural feature image through a multi-head attention mechanism to obtain a structural self-attention image; A structure position encoding layer, configured to perform position encoding on the structure self-attention image to generate a structure self-attention encoded image; A structure normalization layer, configured to normalize the structure self-attention encoded image to obtain a structure self-attention normalized image; a structure-material cross attention layer, configured to process the relationship between the structure self-attention normalized image and the material self-attention normalized image in different semantic subspaces using linear projection weights in the structure-material direction, so as to inject local material features into the global structure features, thereby obtaining the structure-material association feature image; The material-structure cross attention layer is used to process the relationship between the material self-attention normalized image and the structure self-attention normalized image in different semantic subspaces using linear projection weights in the material-structure direction, so as to enhance the complementation of local material features to global structural features, and thereby obtain the material-structure association feature image.

[0036] Specifically, the multi-attention mechanism in this invention is used to effectively integrate the complementary information of the material branch and the structure branch. This multi-attention mechanism comprises material branch self-attention, structure branch self-attention, and cross-modal attention (cross-modal attention comprises structure-material cross-attention and material-structure cross-attention). After the structure-material cross-attention and material-structure cross-attention process features in parallel, an enhanced feature representation is generated through the adaptive gated fusion mechanism of the subsequent gated fusion module.

[0037] The material branch self-attention and the structure branch self-attention use the same architecture but different independent parameters to process the features of their respective branches. In self-attention (CA), given an input feature map, global average pooling is first performed, and then two fully connected layers are used to compress the number of channels to C / 8 and then restore it. The formula is as follows: ; in, represents self-attention, represents the output feature map, represents global splicing pooling, represents the Sigmoid activation function, and Denotes the weight matrix, where For dimensionality reduction fully connected layer, Used to restore the number of channels, is the ReLU activation function.

[0038] In some embodiments, the structure-material cross attention layer has the same structure as the material-structure cross attention layer, and both include: A feature projection sublayer is used to map the structure self-attention normalized image and the material self-attention normalized image to a query space and a key-value space, respectively, to generate a feature projection matrix; wherein, in the structure-material direction, the query comes from the structure self-attention normalized image, and the key and value come from the material self-attention normalized image; in the material-structure direction, the query comes from the material self-attention normalized image, and the key and value come from the structure self-attention normalized image; The dynamic attention weight generator generates the attention region through feature projection matrix multiplication and softmax normalization. The specific form of feature projection matrix multiplication is to multiply the input feature matrix by the projection weight matrix. The feature reconstruction sublayer is used to perform weighted fusion of the attention weight and the original feature in the focus area, and configure the scaling factor to obtain the structure-material association feature image or the material-structure association feature image.

[0039] Specifically, in the two directions of material → structure and structure → material, independent linear projection weights are used to process the relationship between different semantic subspaces in parallel. In the material → structure direction, the query (Q) comes from the features of the material branch, and the key (K) and value (V) come from the features of the structure branch, which enhances the supplement of material details to structural information; in the structure → material direction, the query (Q) comes from the features of the structure branch, and the key (K) and value (V) come from the features of the material branch, injecting local features into the global structure. Local material features can correct the misjudgment of global semantic information, and global structural features can constrain the rationality of local material classification, realizing the two-way complementarity of local material details and global semantic information, forcing the two modalities to interact equally and ensure balanced information fusion. It can effectively solve the problem of dual-branch feature space offset and establish cross-modal association. The formula is as follows: ; ; ; in, Indicates cross attention, represents the original dimension of the input features, represents the number of attention heads, represents the scaling factor, Indicates normalization and generates weight distribution. Indicates the material branch, Indicates a structural branch, represents the material-structure association characteristic image, Represents the structure-material association characteristic image.

[0040] In some embodiments, the gated fusion module includes: A feature stitching layer, configured to stitch the structure-material association feature image and the material-structure association feature image along a channel dimension to obtain a stitched image; a gating weight generator layer for learning spatial weights based on the stitched image and using convolution to reduce the number of channels to 1 to generate adaptive gating weights; The adaptive feature mixer layer is used to perform dynamic weighted fusion on the structure-material association feature image and the material-structure association feature image according to the adaptive gating weight to generate a pixel-level segmentation result image; wherein the formula for dynamic weighted fusion is: ; represents the pixel-level segmentation result image, represents the material-structure association characteristic image, represents the structure-material association characteristic image, represents the adaptive gating weight.

[0041] Specifically, the gated fusion module replaces fixed weights by dynamically generating gated weights to improve model flexibility. First, channel compression is performed, and the two-way cross-attention results are spliced ​​along the channel dimension. A 3×3 convolution is used to compress the channel to reduce the amount of calculation, and then an activation function is used to introduce nonlinearity; then spatial weight learning is performed, and a 1×1 convolution is used to further reduce the number of channels to 1 to generate spatially sensitive weights; then the activation function is used for normalization to map the weights to [0,1]; finally, the two-way features are weighted and summed according to the weights. In the material boundary area, the gate is biased towards the material branch to improve the edge segmentation accuracy; in the internal area of ​​the material, the gate is biased towards the structure branch to ensure category consistency. Among them, the adaptive gate weight formula is: ; in, Represents the Sigmoid activation function; Represents 1×1 convolution; Represents a splicing operation; Indicates the result of two-way cross attention; Indicates the number of channels, Represents dual-channel features; Indicates the number of pixels or spatial size of the feature map in the Y-axis direction; Indicates the number of pixels or spatial size of the feature map in the X-axis direction; each It's a channel, a The feature map is open Stacked together, Carrying feature type information, and Together they carry the spatial location information of features.

[0042] The present invention uses gated weights to assign higher weights to material branches at the object boundary ( →1) to preserve interface details and give higher weight to structural branches inside the object ( →0) to maintain semantic consistency. In addition, the gated fusion module uses only one 3×3 convolution layer and one 1×1 convolution layer to replace the fully connected layer. The gated fusion module effectively reduces the number of model parameters while maintaining high performance.

[0043] In some embodiments, as Figure 5 As shown, the lightweight high-resolution decoder module includes multiple cascaded feature upsampling units, each of which includes a convolutional layer, a batch normalization layer, and an activation function layer connected in sequence; wherein, the convolutional layer in the first-level feature upsampling unit is a 3×3 convolutional layer, and the convolutional layers in the feature upsampling units other than the first-level feature upsampling unit are transposed convolutional layers.

[0044] Specifically, in the lightweight high-resolution decoder module, progressive upsampling is used, which consists of multiple cascaded feature upsampling units to restore the feature map to the input resolution and generate a high-resolution output. The formula is as follows: ; in, Indicates the The upsampling result is represents the activation function, represents batch normalization, represents transposed convolution, Indicates the Upsampling result.

[0045] The lightweight high-resolution decoder module uses a progressive resolution restoration path, restoring the feature map to the input resolution through at least five 2x upsampling operations. Finally, the output layer, a class projection layer containing a 1×1 convolution kernel, outputs the segmented image. The following further evaluates the proposed asphalt concrete image segmentation system based on the gated fusion mechanism using qualitative evaluation metrics.

[0046] The preparation process for poured asphalt concrete suggests that during the hot mixing process, the aggregate is completely encapsulated by the asphalt mortar. Grouting is then poured into the gaps, forming a transitional structure of "aggregate-asphalt mortar-aggregate" or "aggregate-asphalt mortar-grouting material-asphalt mortar-aggregate." Therefore, in addition to commonly used evaluation metrics such as mean intersection over union (mIoU) and F1 score, further considerations should be given to aggregate independence, asphalt mortar encapsulation integrity, and edge clarity.

[0047] In this embodiment, the Aggregate Separation Index (ASI) is designed to quantify the degree of adhesion between aggregates. The formula is as follows. represents the number of aggregate pairs that are adhered after segmentation, which is determined by counting the contact pixels or overlapping boundaries between aggregate regions; Represents the theoretical independent number of all aggregates in the image, obtained through manual annotation.

[0048] ; In addition, the Asphalt Coating Integrity (ACI) is designed to measure whether the asphalt layer is continuously closed. The formula is as follows. Representative The effective perimeter of the asphalt wrapping around the aggregate (closed and without breaks); Representative The actual perimeter of the aggregate. Indicates the total number of aggregates.

[0049] ; Finally, the edge sharpness index (ES) is designed to quantify the sharpness of the aggregate-asphalt boundary. The formula is as follows: Represents boundary pixels The gradient intensity at is calculated by the Sobel operator; Represents the total number of boundary pixels.

[0050] ; This example uses four semantic segmentation schemes from baseline models (UNet and DeepLabv3+) and advanced methods (TransUNet and Swin-UNet) as benchmarks. All performance-related benchmarks were trained on a self-built asphalt concrete dataset. The qualitative evaluation results are shown in Table 1 below.

[0051] Table 1 Qualitative evaluation results of each model Significant improvements in mIoU (83.7%) and F1 (89.2%) demonstrate the advantages of our method in pixel-level classification and class-balanced prediction. An improvement in the aggregate independence index (93.1%) demonstrates its enhanced ability to separate adhered aggregates. The package integrity (94.1%) verifies its accurate modeling of the closure continuity of asphalt mortar. The breakthrough in edge clarity (0.912) demonstrates the superiority of bimodal features in segmenting regions with fuzzy boundaries. These results demonstrate that our method significantly outperforms existing solutions.

[0052] On the basis of the above-mentioned asphalt concrete image segmentation system based on the gated fusion mechanism, the present invention also provides an asphalt concrete image segmentation method based on the gated fusion mechanism.

[0053] like Figure 6 As shown, a method for asphalt concrete image segmentation based on a gated fusion mechanism is applied to the asphalt concrete image segmentation system based on a gated fusion mechanism as described above, comprising: Performing region labeling and edge detection on the original image to be segmented to obtain material branch images and structure branch images respectively; Performing local inductive bias on the material branch image to extract local material texture, and performing global modeling on the structure branch image to extract global interface structure, thereby obtaining a bimodal feature image based on heterogeneous complementarity of spatial details and global semantics; performing channel adjustment and spatial size matching on the dual-modal feature image; Performing self-attention and cross-attention processing on the bimodal feature images after channel space alignment to establish a long-range dependency relationship between the bimodal feature images, thereby obtaining a dual-path cross-modal correlation feature image; Adopting an adaptive gated fusion mechanism to dynamically weight the dual-path cross-modal correlation feature images so that the fusion weights are biased toward the material branch in the boundary area and toward the structure branch in the internal area, thereby obtaining a fused feature image; A multi-level progressive upsampling operation is performed on the fused feature image to generate a pixel-level segmentation result image.

[0054] The steps of the asphalt concrete image segmentation method based on the gated fusion mechanism of the present invention refer to the specific functions of the modules of the asphalt concrete image segmentation system based on the gated fusion mechanism of the present invention, which will not be repeated here.

[0055] The present invention provides an asphalt concrete image segmentation system and method based on a gated fusion mechanism. First, the local texture features of the material and the global semantic features of the structure are extracted respectively based on dual branches, and then cross-modal channel-space dimension matching is achieved through a dynamic feature alignment mechanism. Then, two-way cross-attention interaction is established in combination with multiple attentions. Then, the two-way cross-attention features are dynamically fused using gated weights, and finally, details are restored through progressive upsampling. The system and method can effectively suppress the modal suppression phenomenon, improve the segmentation accuracy of the multi-phase material interface, and achieve pixel-level semantic segmentation of the material microstructure while ensuring video memory efficiency. It is suitable for high-precision image analysis of heterogeneous composite materials such as poured asphalt concrete.

[0056] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. An asphalt concrete image segmentation system based on gated fusion mechanism, characterized by: include: The dual-branch image acquisition module is used to perform region annotation and edge detection processing on the original image to be segmented, so as to obtain the material branch image and the structure branch image respectively; a heterogeneous dual-branch feature extraction module for performing local inductive bias on the material branch image to extract local material texture, and performing global modeling on the structure branch image to extract global interface structure, thereby obtaining a dual-modal feature image based on heterogeneous complementarity of spatial details and global semantics; A channel space alignment module, used for performing channel adjustment and spatial size matching on the bimodal feature image; A multi-attention module is used to perform self-attention and cross-attention processing on the bimodal feature images after channel space alignment to establish long-range dependencies between the bimodal feature images and thereby obtain a dual-path cross-modal correlation feature image; a gated fusion module for dynamically weighting the dual-path cross-modal correlation feature images using an adaptive gated fusion mechanism, so that the fusion weights are biased toward the material branch in the boundary area and toward the structure branch in the internal area, thereby obtaining a fused feature image; The lightweight high-resolution decoder module is used to perform a multi-level progressive upsampling operation on the fused feature image to generate a pixel-level segmentation result image.

2. The asphalt concrete image segmentation system based on gated fusion mechanism according to claim 1 is characterized in that: The heterogeneous dual-branch feature extraction module includes: A deep residual network is used to perform local inductive bias on the material branch image to extract local material texture and obtain a material feature image; a visual Transformer network, configured to globally model the structural branch image to extract the global interface structure and obtain a structural feature image; The material feature image and the structural feature image constitute a dual-modal feature image based on heterogeneous complementarity of spatial details and global semantics.

3. The asphalt concrete image segmentation system based on gated fusion mechanism according to claim 2 is characterized in that: The deep residual network includes: A convolutional layer, configured to extract local material texture from the structure branch image using its own local inductive bias to obtain an initial material feature image; a normalization layer, performing normalization processing on the initial material characteristic image to obtain a normalized material characteristic image; a maximum pooling layer, which reduces the spatial size of the normalized material feature image by half to obtain a pooled material feature image; Multiple cascaded bottleneck layers are used to gradually expand the receptive field of the pooled material feature image to obtain the material feature image.

4. The asphalt concrete image segmentation system based on gated fusion mechanism according to claim 2 is characterized in that: The visual Transformer network includes: A block layer, used to convert the pixel space of the structure branch image into a Token sequence; Multiple cascaded SwinTransformer layers are used to progressively learn the Token sequence from local features to abstract global semantics to obtain the structural feature image.

5. The asphalt concrete image segmentation system based on gated fusion mechanism according to claim 1 is characterized in that: The channel space alignment module includes: A dual-path feature projection unit, configured to perform convolution channel adjustment on the dual-modal feature image; The dynamic spatial interpolation unit is used to perform feature map size matching on the bimodal feature image after channel adjustment using a bilinear interpolation method.

6. The asphalt concrete image segmentation system based on gated fusion mechanism according to claim 1 is characterized in that: The dual-modal feature images based on the heterogeneous complementarity of spatial details and global semantics include material feature images and structural feature images; The dual-path cross-modality correlation feature image includes a structure-material correlation feature image and a material-structure correlation feature image; The multiple attention module includes: A material self-attention layer is used to establish spatial correlations within the material features of the material feature image through a query-key value mechanism to obtain a material self-attention image; A material position encoding layer, configured to perform position encoding on the material self-attention image to generate a material self-attention encoded image; A material normalization layer, configured to normalize the material self-attention encoded image to obtain a material self-attention normalized image; A structural self-attention layer is used to capture the global dependency of structural features in the structural feature image through a multi-head attention mechanism to obtain a structural self-attention image; A structure position encoding layer, configured to perform position encoding on the structure self-attention image to generate a structure self-attention encoded image; A structure normalization layer, configured to normalize the structure self-attention encoded image to obtain a structure self-attention normalized image; a structure-material cross attention layer, configured to process the relationship between the structure self-attention normalized image and the material self-attention normalized image in different semantic subspaces using linear projection weights in the structure-material direction, so as to inject local material features into the global structure features, thereby obtaining the structure-material association feature image; The material-structure cross attention layer is used to process the relationship between the material self-attention normalized image and the structure self-attention normalized image in different semantic subspaces using linear projection weights in the material-structure direction, so as to enhance the complementation of local material features to global structural features, and thereby obtain the material-structure association feature image.

7. The asphalt concrete image segmentation system based on gated fusion mechanism according to claim 6 is characterized in that: The structure-material cross attention layer has the same structure as the material-structure cross attention layer, both including: A feature projection sublayer is used to map the structure self-attention normalized image and the material self-attention normalized image to a query space and a key-value space, respectively, to generate a feature projection matrix; wherein, in the structure-material direction, the query comes from the structure self-attention normalized image, and the key and value come from the material self-attention normalized image; in the material-structure direction, the query comes from the material self-attention normalized image, and the key and value come from the structure self-attention normalized image; Dynamic attention weight generator, which generates attention regions through feature projection matrix multiplication and softmax normalization; The feature reconstruction sublayer is used to perform weighted fusion of the attention weight and the original feature in the focus area, and configure the scaling factor to obtain the structure-material association feature image or the material-structure association feature image.

8. The asphalt concrete image segmentation system based on gated fusion mechanism according to claim 1 is characterized in that: The gated fusion module includes: A feature stitching layer, configured to stitch the structure-material association feature image and the material-structure association feature image along a channel dimension to obtain a stitched image; a gating weight generator layer for learning spatial weights based on the stitched image and using convolution to reduce the number of channels to 1 to generate adaptive gating weights; The adaptive feature mixer layer is used to perform dynamic weighted fusion on the structure-material association feature image and the material-structure association feature image according to the adaptive gating weight to generate a pixel-level segmentation result image; wherein the formula for dynamic weighted fusion is: ; represents the pixel-level segmentation result image, represents the material-structure association characteristic image, represents the structure-material association characteristic image, represents the adaptive gating weight.

9. The asphalt concrete image segmentation system based on gated fusion mechanism according to claim 1, characterized in that: The lightweight high-resolution decoder module includes multiple cascaded feature upsampling units, each of which includes a convolutional layer, a batch normalization layer, and an activation function layer connected in sequence; wherein the convolutional layer in the first-level feature upsampling unit is a 3×3 convolutional layer, and the convolutional layers in other feature upsampling units except the first-level feature upsampling unit are transposed convolutional layers.

10. A method for asphalt concrete image segmentation based on gated fusion mechanism, characterized in that: The asphalt concrete image segmentation system based on the gated fusion mechanism as claimed in any one of claims 1 to 9 comprises: Performing region labeling and edge detection on the original image to be segmented to obtain material branch images and structure branch images respectively; Performing local inductive bias on the material branch image to extract local material texture, and performing global modeling on the structure branch image to extract global interface structure, thereby obtaining a bimodal feature image based on heterogeneous complementarity of spatial details and global semantics; performing channel adjustment and spatial size matching on the dual-modal feature image; Performing self-attention and cross-attention processing on the bimodal feature images after channel space alignment to establish a long-range dependency relationship between the bimodal feature images, thereby obtaining a dual-path cross-modal correlation feature image; Adopting an adaptive gated fusion mechanism to dynamically weight the dual-path cross-modal correlation feature images so that the fusion weights are biased toward the material branch in the boundary area and toward the structure branch in the internal area, thereby obtaining a fused feature image; A multi-level progressive upsampling operation is performed on the fused feature image to generate a pixel-level segmentation result image.

Images