Navigation semantic segmentation method based on hierarchical multi-scale transformer

Abstract

The application discloses a navigation semantic segmentation method based on a layered multi-scale Transformer, relates to the technical field of navigation semantic segmentation, and comprises the following steps: S1, an input image is acquired, and multi-level feature maps are extracted; S2, the highest level feature map in the multi-level feature maps is input into a layered multi-scale window Transformer module for processing, so that multi-scale semantic aggregation features are obtained; S3, the features of all levels in the multi-level feature maps are gradually fused with deep semantic features and shallow spatial detail features of two adjacent levels, so that multi-level refined features are obtained; S4, the multi-scale semantic aggregation features and the multi-level refined features are up-sampled to a unified size, are cascaded and fused in a channel dimension, and are finally fused; and S5, the finally fused features are input into a classification layer, so that a pixel-level semantic segmentation result is obtained.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of navigation semantic segmentation, and particularly relates to a navigation semantic segmentation method based on hierarchical multi-scale Transformer. BACKGROUND

[0002] Semantic segmentation is a pixel-level image understanding task, which plays a key role in intelligent driving systems. Its accuracy directly affects the perception accuracy of vehicles on the surrounding environment (such as roads, vehicles, pedestrians, traffic signs, etc.), thereby determining the safety of subsequent planning and decision-making. Convolutional Neural Networks (CNN), especially the encoder-decoder architecture represented by Fully Convolutional Networks (FCN), has become the mainstream method for semantic segmentation. Subsequent research has improved the segmentation performance of CNN models by expanding the receptive field, introducing a spatial pyramid to capture multi-scale features, using dense connections, and designing various attention mechanisms. However, the inherent local receptive field of CNN limits its ability to effectively model the global context and long-distance dependencies of the image, which is particularly important for understanding the spatial layout and mutual relationship between objects in complex driving scenarios. In addition, some improvement methods introduce complex modules, leading to increased computational complexity, which is not conducive to deployment on resource-constrained vehicle platforms.

[0003] In recent years, the Transformer architecture has brought new opportunities for semantic segmentation due to its self-attention mechanism that can effectively capture long-range dependencies between image regions and establish a global context model. Vision Transformer (ViT) and its variants, such as SETR (Segmentation Transformer), Swin Transformer, and SegFormer, have proven their potential in semantic segmentation tasks. Among them, Swin Transformer enhances local feature modeling and improves computational efficiency by introducing windowed self-attention and hierarchical design. However, existing Transformer-based segmentation methods still have room for optimization in decoder design and multi-scale feature processing. The decoders of many models are relatively simple and fail to fully utilize the rich hierarchical features produced by the Transformer encoder. The "semantic gap" between different levels of features makes simple feature concatenation and fusion suboptimal, affecting segmentation accuracy, especially the precise positioning of object boundaries.

[0004] Furthermore, existing Transformer models face challenges when processing targets with drastic size changes in driving scenarios. Most models use fixed-size windows or uniform processing for all scales in the decoder or self-attention module, which limits the model's ability to flexibly capture and represent different size targets (from close-range large trucks to far-distance pedestrians or traffic signs). The receptive field of each feature channel is relatively fixed, hindering the model from fully exploiting the potential of multi-scale features, which is crucial for robust understanding of the driving environment. SUMMARY

[0005] To address the above technical problems, the present application provides a navigation semantic segmentation method based on hierarchical multi-scale Transformer, which enhances the model's adaptive representation ability of multi-scale features and optimizes the feature fusion strategy to improve the semantic segmentation performance in complex driving scenarios.

[0006] To solve the above technical problems, the technical solution provided by the present application is:

[0007] A navigation semantic segmentation method based on hierarchical multi-scale Transformer, comprising the following steps:

[0008] Step S1: Obtain the input image and extract multi-level feature maps;

[0009] Step S2: Input the highest-level feature map in the multi-level feature map into the hierarchical multi-scale window Transformer module for processing to obtain multi-scale semantic aggregation features;

[0010] Step S3: The feature maps of all levels in the multi-level feature map are fused level by level with the deep semantic features and shallow spatial detail features of two adjacent levels to obtain multi-level refined features.

[0011] Step S4: Upsample the multi-scale semantic aggregation features and multi-level refinement features to a uniform size, and perform cascade fusion in the channel dimension, and then perform final feature fusion.

[0012] Step S5: Input the final fused features into the classification layer to obtain pixel-level semantic segmentation results.

[0013] A further improvement to the above technical solution is as follows:

[0014] Preferably, in step S2, the specific steps for obtaining the multi-scale semantic aggregation features are as follows:

[0015] S2-1, capturing multi-scale features from the highest-level feature map of a multi-level feature map;

[0016] Let the input features be The multi-scale features are:

[0017] ;

[0018] ;

[0019] ;

[0020] in, Indicates the first One branch, ; For multiple heads, For the number of windows, This represents the adjusted input feature map, whose structure is designed to accommodate subsequent multi-scale splitting and processing. Represents the feature space set, Indicates the height of the input feature map. This indicates the width of the input feature map. Represents the feature dimension. This represents the merged feature map after processing by a multi-scale window multi-head self-attention mechanism. This indicates multi-head self-attention, and its output feature map is transformed back into... , and the output of each branch is ;

[0021] S2-2, integration stage: integrating features of different window branches;

[0022] performing full connection layer operation, activation layer processing, then generating channel-level statistical information through global average pooling, and then generating a global feature vector through another full connection layer and activation layer processing ;

[0023] S2-3, selection stage: deriving a respective relevance vector for each branch according to the integrated information ; , processing the weighted fused features and restoring the channel size;

[0024] S2-4, obtaining multi-scale semantic aggregation features;

[0025] The output of the entire hierarchical multi-scale window Transformer module is , and the expression is as follows:

[0026] , , which represents the main path feature, is the feature after the first full connection layer processing of ;

[0027] , which represents the feature after adaptive weighted fusion, is a correction increment to the main path feature.

[0028] Preferably, in the step S3, the deep semantic feature linear mapping is defined as ; the shallow spatial detail feature linear mapping is defined as and , and the process of obtaining multi-level refined features is as follows:

[0029] S3-1, performing global average pooling and convolution operation on , to generate weights corresponding to the deep feature map ;

[0030] S3-2, using channel attention mechanism to element-wise multiply the weights and the deep feature map , to obtain weighted features ;

[0031] S3-3, performing max-pooling operation on to down-sample, and increasing the channel dimension of the shallow feature map or performing feature transformation, so that it is consistent with the deep feature map​ the number of channels remains consistent, obtaining a processed feature map ;

[0032] S3-4, element-wise addition of the weighted feature and the processed feature to obtain a refined feature output ; refine each level feature in the remaining levels.

[0033] Preferably, the hierarchical multi-scale window Transformer module is an HMW-Former module, which includes a hierarchical multi-scale window multi-head self-attention module, and the multi-head self-attention module includes a plurality of multi-head self-attention sub-modules for extracting multi-scale features; and a multi-scale window adaptive module for adaptive fusion of information.

[0034] Preferably, in step S3, the feature maps of all levels are input into a refined feature pyramid module for processing to obtain multi-level refined features; the refined feature pyramid module gradually fuses deep semantic features and shallow spatial detail features of adjacent levels from bottom to top through a plurality of refined feature multi-head self-attention units.

[0035] Preferably, in step S4, the multi-scale semantic aggregation features output by the HMW-Former module of step S2 and the multi-level refined features output by the refined feature pyramid module of step S3 are up-sampled to have uniform sizes, then fused in the channel dimension, and finally fused through a multi-layer perception layer to combine the refined features of each stage with the deep multi-scale features.

[0036] The hierarchical multi-scale Transformer-based navigation semantic segmentation method provided by the present application has the following advantages compared with the prior art:

[0037] (1) The hierarchical multi-scale Transformer-based navigation semantic segmentation method of the present application proposes a hierarchical multi-scale window Transformer (HMW-Former) module, which significantly enhances the model's representation ability for multi-scale targets in driving scenes through multi-scale window multi-head self-attention sub-modules that process different granularity information in parallel and multi-scale window adaptive sub-modules that dynamically fuse hierarchical information.

[0038] (2) The navigation semantic segmentation method based on the hierarchical multi-scale Transformer of the application, the refined feature pyramid (RFP) module designed effectively fuses the deep semantic features and shallow spatial detail features output by the Transformer backbone network, overcomes the semantic differences between feature levels, improves the effectiveness of feature fusion, and thus improves the segmentation accuracy, especially for small objects and boundary regions.

[0039] (3) The navigation semantic segmentation method based on the hierarchical multi-scale Transformer of the application, the novel Transformer semantic segmentation network based on HMW-Former and RFP is constructed, which organically combines the two core modules proposed, and can achieve competitive comprehensive performance in complex scene perception, especially in intelligent driving related scenes. BRIEF DESCRIPTION OF DRAWINGS

[0040] Figure 1 It is a schematic diagram of the overall network structure when the application is applied.

[0041] Figure 2 It is a simplified schematic diagram of the hierarchical multi-scale window Transformer module when the application is applied (taking two window scales as an example).

[0042] Figure 3 It is a detailed structure schematic diagram of the hierarchical multi-scale window Transformer module when the application is applied.

[0043] Figure 4 It is a structure schematic diagram of the refined feature MSA module when the application is applied. DETAILED DESCRIPTION

[0044] The specific embodiments of the application will be described in detail below. It should be understood that the specific embodiments described herein are only used to illustrate and explain the application, and are not used to limit the application.

[0045] The navigation semantic segmentation method based on the hierarchical multi-scale Transformer of the application is mainly used for an automatic driving navigation semantic segmentation network. Based on the navigation semantic segmentation method of the application, the overall structure of the automatic driving navigation semantic segmentation network includes an encoder and a decoder. The encoder backbone network adopts the MiT-B3 (full name is MixTransformer-B3, B3 refers to the third size of the backbone network) structure of Segformer. The decoder fully utilizes the four hierarchical output features of the encoder. The navigation semantic segmentation method of the application has the following process in the decoder:

[0046] First, the fourth layer output feature (the highest level, for example The channel dimension is reduced by a Multilayer Perceptron (MLP), for example, the feature channel dimension of the output is 240, and then a Hierarchical Multi-scale Window Transformer (HMW-Former) module is used for multi-scale feature extraction.

[0047] Meanwhile, the output features of the first to fourth layers of the encoder are refined by a Refinement Feature Pyramid (RFP) module, and the deep semantic features and the shallow spatial detail features of two adjacent layers are fused step by step from bottom to top.

[0048] Finally, the output of the HMW-Former module and the features after the fusion of the RFP module at each level (usually after up-sampling to the same size) are fused by a Multilayer Perceptron (MLP), and then subsequent up-sampling and pixel classification are performed. A 3x3 convolution can be added in the feature fusion module to enhance local position information.

[0049] The HMW-Former module includes two main parts: a hierarchical multi-scale window multi-head self-attention module, which includes multiple multi-head self-attention sub-modules for extracting multi-scale features; and a multi-scale window adaptive module for adaptive fusion of the information.

[0050] Furthermore, the navigation semantic segmentation method based on the hierarchical multi-scale Transformer includes the following steps:

[0051] Step S1, an input image is obtained, and a multi-level feature map of the input image is extracted by using a preset backbone network.

[0052] The backbone network can adopt the MiT-B3 structure of Segformer. For a given image with a resolution of , the image is cut into 4x4 image patches. Then the image patches are passed into the backbone network to generate multi-level features with resolutions of 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the original image. These features provide high-resolution fine-grained features and low-resolution semantic features, respectively. The channel numbers are 64, 128, 320, and 512, respectively.

[0053] Step S2, the highest level feature map in the multi-level feature map is input into a hierarchical multi-scale window Transformer module for processing to obtain multi-scale semantic aggregation features.

[0054] The Transformer module adopts the HMW-Former module. The HMW-Former is a Transformer module specially designed for efficiently solving multi-scale problems in computer vision, especially semantic segmentation. Its core lies in the "hierarchical structure-embedded multi-scale window processing".

[0055] The HMW-Former module captures context information of different granularities in parallel within a single module by grouping the "heads" in the multi-head self-attention mechanism and assigning different attention window sizes to different groups. At the same time, a branch weight adaptive adjustment strategy is introduced, enabling the model to dynamically fuse information from different scale windows according to the input content. As shown in Figure 2 , assuming the input is a 16x16 size feature map. It is divided into two branches, each using a window size of 8x8 and 4x4 sliding window operation, and input into the multi-head self-attention model. Finally, each scale of features obtains an adaptive weight through the adaptive model, realizing channel dimension fusion.

[0056] In this embodiment, the specific steps for obtaining multi-scale semantic aggregation features are as follows:

[0057] S2-1, capture multi-scale features from the highest level feature map of the multi-level feature map.

[0058] As shown in Figure 3 , in this embodiment, the number of heads is , the number of windows is , each window is a branch, and the multi-head features are evenly divided into groups, each group contains heads, and different window scales are used for multi-head self-attention operation.

[0059] Let the input feature be , the multi-scale feature is obtained by the following formula:

[0060] ;

[0061] ;

[0062] ;

[0063] wherein, represents the th branch, ; represents the adjusted input feature map, which is structured to adapt to the subsequent multi-scale splitting and processing, represents the feature space set, represents the height of the input feature map, This indicates the width of the input feature map. Represents the feature dimension. This represents the merged feature map after processing by a multi-scale window multi-head self-attention mechanism. This represents multi-head self-attention, where the output feature map is transformed back into... The spatial dimension, the output of each branch is Reshape indicates adjusting the shape of the data; Split indicates splitting the data into blocks; Concat indicates concatenating the data along the channel dimension. The window size is... In the Each window in each branch expands to × A one-dimensional sequence of length is used as the input to the multi-head attention window model. In this embodiment, the outputs of each branch are concatenated along the channel dimension to form the output of the entire multi-scale window multi-head attention model. .

[0064] Let the number of heads be h=6, and the number of windows be n=3 (i.e., three branches). The window scales are respectively... ;

[0065] The multi-head feature was evenly divided into three groups, each containing The multi-head self-attention operation was performed using different window sizes (2x2, 4x4, 8x8).

[0066] S2-2, Integration Stage: Integrating the features of three different window branches.

[0067] First of all Perform the first fully connected layer The operation then passes through the first activation layer. (e.g., GELU, an activation function called Gaussian error linear unit), then global average pooling is used. Channel-level statistics are generated to embed global information; then, they pass through a second fully connected layer. Second activation layer Generate a global feature vector .

[0068] The calculation process is as follows:

[0069] ;

[0070] ;

[0071] ;

[0072] wherein, represents the main path feature, which is processed by the feature after the first fully connected layer processing, represents the global feature vector, which contains the comprehensive information of all scale branches, and is used for subsequent calculation of the fusion weight of each branch. represents the channel dimension of the global feature vector.

[0073] S2-3, selection stage: according to the integrated information derives a respective correlation vector for each branch , processes the weighted fused feature and restores the channel size.

[0074] The formula is as follows:

[0075] ;

[0076] ;

[0077] ;

[0078] wherein, , represents the fully connected layer, represents the fully connected layer of the i-th branch, exp represents the exponential processing, and and respectively represent the first linear layer and the second linear layer; is the output of each branch in the multi-scale window multi-head self-attention module.

[0079] S2-4, obtain the multi-scale semantic aggregation feature.

[0080] The output of the entire hierarchical multi-scale window Transformer module is , and its expression is as follows: , represents the main path feature, which is processed by the feature after the first fully connected layer processing;

[0081] represents the feature after adaptive weighted fusion, which is a correction increment to the main path feature.

[0082] Step S3, input the remaining lower level feature maps to the refined feature pyramid (RFP) module for processing to obtain multi-level refined features.

[0083] The RFP module fuses the deep semantic features and the shallow spatial detail features of adjacent layers from bottom to top through multiple carefully designed refinement feature multi-head self-attention (MSA) units, effectively bridges the gap between the high-level semantic features and the low-level spatial detail features, and improves the accuracy of the final segmentation map.

[0084] The refinement feature pyramid (RFP) module is used to combine the shallow local information and the deep global information. The refinement feature pyramid (RFP) is four inputs and three outputs. As shown in Figure 1 , the RFP module is composed of three refinement feature MSAs, which process the first, second and third layer outputs of the encoder respectively. Each refinement feature MSA module maintains a scale branch for fusing the current level features and the features from the deeper (or higher level), and its structure is as shown in Figure 4 . Each refinement feature MSA submodule (2 inputs, 1 output) fuses the features of two adjacent inputs, one is deep and the other is shallow. In two adjacent levels, deep and shallow are the description of the relative relationship, shallow refers to the earlier stage or initial level of the network; deep refers to the subsequent stage or higher level of the network, and the specific level is determined by the hierarchical output of the backbone network transformer block. Figure 1 In the embodiment, the hierarchical features output by the backbone network become deeper from the first layer to the fourth layer, and the feature size (width and height) gradually decreases, while the number of channels of the features gradually increases.

[0085] The linear mapping of the deep feature in the refinement feature MSA submodule is defined as ; the two linear mappings of the shallow feature are defined as and .

[0086] In the embodiment, the specific process of obtaining multi-level refinement features is as follows:

[0087] S3-1, the is subjected to global average pooling GAP and convolution operation (convolution kernel is 1) to generate the weight corresponding to the deep feature map .

[0088] S3-2, the weight is multiplied element by element with the deep feature map by using the channel attention mechanism to obtain the weighted feature .

[0089] S3-3, the is subjected to max-pooling Max-pool operation for down-sampling, and a The convolution layer (convolution kernel 3) increases the channel dimension of the shallow feature map or performs feature transformation, so that the channel number of the shallow feature map is consistent with that of the deep feature map , and a processed feature map is obtained.

[0090] S3-4, the weighted feature is element-wise added to the processed feature , and a refined feature output is obtained through a convolution layer .

[0091] The expression is as follows:

[0092] ;

[0093] ;

[0094] ;

[0095] ;

[0096] In the formula, represents a convolution operation with a convolution kernel of 1, represents a convolution operation with a convolution kernel of 3, Max-pool represents max pooling, represents matrix multiplication, represents element-wise addition.

[0097] The output of the RFP module is the result of the first three layers of the encoder feature after refinement.

[0098] Step S4, the multi-scale semantic aggregation features output by the HMW-Former module in step S2 and the multi-level refined features output by the RFP module in step S3 are upsampled to unify their sizes (for example, to 1 / 4 resolution of the original image), and then are fused in the channel dimension, and finally a multi-layer perception (MLP) layer is used for the final feature fusion, so that the refined features of each stage are effectively combined with the deep multi-scale features.

[0099] The fourth layer feature of the encoder is processed by the HMW-Former module to obtain multi-scale semantic aggregation features . The first to third layer features of the encoder are processed by the refined feature MSA unit of the RFP module in turn, and .

[0100] The four feature maps , where ^ represents upsampled to a unified size, such as ) Concatenate in the channel dimension, and then reduce and fuse through a multi-layer perception (MLP) layer.

[0101] ;

[0102] ;

[0103] wherein, represents the final fused feature, represents the total number of channels of the concatenated feature, C represents the number of feature channels output by the MLP, represents the number of categories of the image, M is the final segmentation result image, and Linear represents linear processing.

[0104] Step S5, the final fused feature after final fusion is input to a classification layer (such as a linear layer followed by Softmax, a digital processing), to obtain a category prediction of each pixel, so as to complete semantic segmentation.

[0105] Experimental verification:

[0106] The present application is verified by experiments on Cityscapes, PASCAL VOC2012 and ADE20K data sets.

[0107] Implementation details: two NVIDIA GeForce RTX 3090 graphics cards are used for testing. The model uses pre-trained MIT-B3 backbone network parameters. Random scaling (0.5-2.0 times) and horizontal flip data augmentation are applied. The AdamW optimizer is used in the optimization process, with an initial learning rate of 6e-5, combined with the "poly" learning rate scheduling. For different data sets, the training crop size is set to 512x512 for ADE20K and PASCAL VOC2012, and 768x768 for Cityscapes. The model is trained for 160K iterations on the ADE20K and Cityscapes data sets, with a batch size of 8 for ADE20K / VOC and 4 for Cityscapes. When evaluating, the short side of the image is scaled to the corresponding training size while maintaining the original aspect ratio.

[0108] As shown in Table 1, the experimental results show that the method of the present application improves the performance accuracy by 2.7% on the ADE20K data set compared with the benchmark Segformer using the same MiT-B3 backbone network.

[0109] On the Cityscapes and PASCAL VOC2012 data sets, the performance accuracy is improved by 1.7% and 1.3% respectively compared with the benchmark network Segformer, while maintaining similar parameter quantity and computational quantity.

[0110] Table 1. Performance (mloU), computation (FLOPs), and computation parameter (Params) comprehensive comparison on three datasets

[0111]

[0112] The above embodiments are only the preferred embodiments of the present application, and do not limit the present application in any form. Although the present application has been disclosed with the preferred embodiments as above, it is not intended to limit the present application. Therefore, any simple modification, equivalent change and modification made to the above embodiments without departing from the technical solution of the present application, according to the technical essence of the present application, shall fall within the scope of protection of the technical solution of the present application.

Claims

1. A hierarchical multi-scale Transformer-based navigation semantic segmentation method, characterized in that, The method comprises the following steps: Step S1, acquiring an input image and extracting a multi-level feature map; Step S2, inputting the highest level feature map in the multi-level feature map into a hierarchical multi-scale window Transformer module for processing to obtain multi-scale semantic aggregation features; Step S3, gradually fusing deep semantic features and shallow spatial detail features of two adjacent levels of features in all levels of the multi-level feature map to obtain multi-level refined features; Step S4, upsampling the multi-scale semantic aggregation features and the multi-level refined features to a unified size, performing cascade fusion in the channel dimension, and then performing final feature fusion; Step S5, inputting the finally fused features into a classification layer to obtain a pixel-level semantic segmentation result; In the step S2, the specific steps for obtaining the multi-scale semantic aggregation features are as follows: S2-1, capturing multi-scale features from the highest level feature map of the multi-level feature map; Let The input features are The multi-scale features are: ； ； ； wherein, represents the branch, ; is the number of heads, is the number of windows, represents the adjusted input feature map, the structure of which is adapted to the subsequent multi-scale splitting and processing, represents the feature space set, represents the height of the input feature map, represents the width of the input feature map, represents the feature dimension, represents the merged feature map after the multi-scale window multi-head self-attention mechanism processing, represents the multi-head self-attention, the output feature map of which is reconverted into the spatial dimension of , and the output of each branch is ; S2-2, integration stage: integrating features of different window branches; To perform a fully connected layer operation, an activation layer processing, and then generate channel-level statistical information through global average pooling, and then generate a global feature vector through another fully connected layer and activation layer processing ; S2-3, selection stage: based on integrated information deriving respective correlation vectors for each branch processing the weighted fused features and restoring the channel dimension S2-4, obtaining multi-scale semantic aggregation features; The output of the entire hierarchical multi-scale window Transformer module is , and the expression is as follows: , , represents the main path feature, which is the feature after the first full connection layer processing by ; , represents the feature after adaptive weighted fusion, which is a correction increment to the main path feature; The hierarchical multi-scale window Transformer module is an HMW-Former module, which comprises a hierarchical multi-scale window multi-head self-attention module, the multi-head self-attention module comprises a plurality of multi-head self-attention sub-modules for extracting multi-scale features, and a multi-scale window adaptive module for adaptive fusion of information.

2. The hierarchical multi-scale Transformer-based navigation semantic segmentation method according to claim 1, characterized in that, The deep semantic feature linear mapping in the step S3 is defined as The shallow spatial detail feature linear mapping is respectively defined as and The process of obtaining the multi-level refined features is as follows: S3-1, to perform global average pooling and convolution operation, generate deep feature map corresponding weights ; S3-2, adopt channel attention mechanism to weight with deep feature map element-wise multiplication, get weighted feature ; S3-3, to perform a max-pooling operation to down-sample and increase the channel dimension of the shallow feature map or perform a feature transformation so that the channel number of the shallow feature map is consistent with that of the deep feature map , to obtain a processed feature map ; S3-4, the weighted features with the processed features are element-wise added to obtain refined feature outputs ; Each level of feature in the remaining levels is refined.

3. The hierarchical multi-scale Transformer-based navigation semantic segmentation method of claim 1, wherein, In the step S3, all levels of features are input into a refined feature pyramid module to obtain multi-level refined features; the refined feature pyramid module gradually fuses deep semantic features and shallow spatial detail features of adjacent levels from bottom to top through a plurality of refined feature multi-head self-attention units.

4. The hierarchical multi-scale Transformer-based navigation semantic segmentation method of claim 3, wherein, In the step S4, the multi-scale semantic aggregation features output by the HMW-Former module of step S2 and the multi-level refined features output by the refined feature pyramid module of step S3 are subjected to an upsampling operation to make their sizes uniform, then cascade fusion is performed in the channel dimension, and finally a multi-layer perceptron layer is used for final feature fusion to combine the refined features of each stage with the deep multi-scale features.

Images