Domain generalization semantic segmentation enhancement method based on efficient multi-scale and simple attention module

Abstract

The invention discloses a field generalization semantic segmentation enhancement method based on an efficient multi-scale and simple attention module, and belongs to the field of computer vision. According to the invention, the model is based on a TQDM (Text Query-Driven Mask Transfer) framework, and an EMA (Efficient Multi-Scale Attention Module) and a SimAM (Simple Parameter-free Attention Module) are fused, so that the adaptability and the robustness of a semantic segmentation task on a plurality of domains are improved. According to the invention, by introducing the EMA module, multi-scale feature aggregation and cross-space information interaction are realized, so that the understanding ability of the model to a complex scene is enhanced; meanwhile, in combination with a SimAM module, feature expression is optimized under the condition of not increasing extra parameters, and the precision of small target segmentation and target boundary detection is improved. The method can be widely applied to computer vision tasks related to cross-domain semantic segmentation, such as automatic driving, intelligent monitoring and medical image analysis.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision, and specifically provides a domain-generalized semantic segmentation enhancement method based on efficient multi-scale and simple attention modules. Background Art

[0002] Machine learning is widely used in fields such as facial recognition, autonomous driving, and healthcare. However, traditional machine learning methods often assume that the distribution of training and test data is consistent. In practice, this assumption often fails, leading to domain shift and affecting the model's ability to generalize to unseen domains.

[0003] In computer vision tasks, semantic segmentation requires semantic classification of each pixel in an image and is crucial in fields such as autonomous driving and robotic vision. However, due to factors such as weather changes, lighting conditions, and material diversity, the generalization ability of semantic segmentation models across different domains is limited. This problem is known as the domain generalization problem.

[0004] To alleviate the performance degradation in domain generalization semantic segmentation tasks, researchers have proposed a variety of optimization strategies. For example, CrossNorm and SelfNorm techniques improve generalization capabilities through feature decoupling, while memory mechanisms enhance the domain adaptability of models by storing representative features. In addition, the IBN-Net model, which combines instance normalization (IN) with batch normalization (BN), and RobustNet, which uses instance-selective whitening, have both improved cross-domain robustness to a certain extent. However, these methods fail to fully utilize semantic information, and in particular, they still suffer from insufficient generalization in complex environments.

[0005] In recent years, vision-language models (VLMs), with their cross-modal feature extraction capabilities, have provided new solutions to the domain-generalization semantic segmentation problem. For example, Rein improves generalization through efficient fine-tuning, GR-MPL enhances model adaptability through meta-learning and gradient regulation, and Hybrid Domain Generalization (HDG) improves cross-domain robustness by combining cross-modal learning strategies. However, these methods still lack the ability to fully utilize language information.

[0006] To address these issues, the Textual Query-driven Mask Transformer (TQDM) model is a semantic segmentation method directly driven by text. TQDM uses text embeddings from visual language models (such as CLIP and EVA02-CLIP) as query vectors to guide the generation of segmentation masks, thereby improving cross-domain generalization capabilities. Compared to traditional semantic segmentation methods that rely on visual features for pixel classification, TQDM enhances global semantic understanding through textual information and demonstrates strong robustness in cross-domain environments.

[0007] Although TQDM demonstrates good generalization capabilities in domain generalization semantic segmentation tasks, it still has certain limitations. First, in scenes with low lighting, occlusion, or high background noise, TQDM struggles to effectively extract key visual features, affecting segmentation accuracy. Second, when dealing with small targets and object boundaries, its fine-grained feature modeling capabilities are limited and susceptible to background interference, resulting in inaccurate segmentation of edge regions. Furthermore, because TQDM relies on text queries for semantic segmentation, when there is a significant difference in the distribution of the visual domain and the text domain, the model's cross-domain adaptability decreases, thus affecting generalization performance. Summary of the Invention

[0008] This paper aims to address the above-mentioned problems in the prior art. It proposes a semantic segmentation model for cross-modal fusion, ES-TQDM. This model is used to perform a domain-generalized semantic segmentation enhancement method based on an efficient multi-scale and simple attention module.

[0009] The technical solution adopted by the present invention is: a domain-generalized semantic segmentation enhancement method based on efficient multi-scale and simple attention modules, comprising the following steps:

[0010] Adopting a Transformer-based encoder-decoder structure to achieve cross-modal feature fusion of image and text features;

[0011] The encoder uses CLIP-ViT-B-16 as the backbone network and combines it with the Simple Parameter-free Attention Module (SimAM) module to extract features from the input image, obtain image features, and then input them into the cross-modal feature fusion module;

[0012] The input text is preprocessed to generate text features, which are then input into the cross-modal feature fusion module;

[0013] The cross-modal feature fusion module performs cross-modal fusion of image features and text features, including: deconstructing image features into global features and visual embeddings, using an efficient multi-scale attention module (EMA) to perform cross-space learning on the visual embeddings, and then using SimAM to perform pixel-level attention calculations, splicing the global features to obtain the final image features; the text features are passed through a text encoder to generate text embeddings; and the final image features are cross-modally aligned with the text embeddings.

[0014] Finally, the decoder calculates the semantic mask through the Transformer and outputs the final pixel-level semantic segmentation result.

[0015] Furthermore, the encoder uses CLIP-ViT-B-16 as the backbone network, and the specific steps of the processing include:

[0016] The input image is first divided into patches of fixed size through the Patch Embedding layer and mapped to a high-dimensional feature space;

[0017] Using the Vision Transformer (ViT) structure, the global correlation between different patches is modeled through the Multi-Head Self-Attention (MHSA) mechanism to capture long-distance dependency information.

[0018] In the hierarchical calculation process of ViT, features from different Transformer layers are fused to gradually extract high-level semantic information and enhance visual representation capabilities;

[0019] Finally, the visual features calculated by multiple layers of Transformer are extracted to obtain image features.

[0020] Furthermore, the combined SimAM module measures the relative importance of neuron t in the channel by calculating its energy. To this end, SimAM defines an energy function e t :

[0021]

[0022] in, and Represents t and x respectively i The linear transformation applied, t represents the target neuron in a channel of the input feature X, and x i Represents other neurons in the same channel, index i traverses the entire spatial dimension, and the total number of neurons in the channel is M = H × W, H and W represent the height and width of the feature map respectively, wt and b t are weights and biases respectively, and y t and y o is a binary label:

[0023] SimAM learns the optimal weights and biases by minimizing the energy function to separate the target neuron from other neurons as much as possible;

[0024] Among them, the analytical solution of neuron weights and biases is:

[0025]

[0026] in represents the mean value of neurons in the channel, represents the variance of neurons within the channel, and λ is the regularization term;

[0027] SimAM further evaluates the importance of each neuron by calculating the minimum energy and adjusts the attention weight accordingly:

[0028]

[0029] in represents the sample mean of all M neurons under this channel, Represents the sample variance of all M neurons under this channel.

[0030] Finally, in order to effectively incorporate the calculated importance information into the input features, SimAM uses the Sigmoid function to normalize the attention weights and applies it to each pixel of the input feature map:

[0031]

[0032] Where E is the energy value in all channels and spatial dimensions, Denotes the normalized output feature map, and X denotes the original feature map. By weighting pixels one by one, SimAM adaptively adjusts the attention weights at multiple scales.

[0033] Furthermore, the text preprocessing of the input text includes tokenizing the input category name text, converting each category name into a discrete text representation, and the tokenization operation decomposes the category name into vocabulary units or subwords, which are mapped to representations in high-dimensional space through pre-trained word vectors or embedding layers.

[0034] Furthermore, the text encoder generates the text embedding through a trainable context generation mechanism, combined with the tokenized representation of each category.

[0035] Furthermore, the EMA module performs cross-space learning on visual embeddings, including grouping the input feature map by channel dimension to generate multiple sub-features, each of which is processed by a 1×1 convolution branch and a 3×3 convolution branch. The specific operation is expressed as:

[0036]

[0037] where z c represents the global pooling feature of the cth channel, x c Represents the input feature of the c-th channel, i represents the "row" index of the spatial domain, that is, the position of the feature map in the height direction, and j represents the "column" index of the spatial domain, that is, the position of the feature map in the width direction.

[0038] The outputs of the 1×1 and 3×3 branches are fused and the final pixel-level attention map is generated through the Sigmoid activation function.

[0039] Furthermore, the SimAM module continues to perform pixel-level attention calculations on visual embeddings based on the EMA module, including modeling a parameter-free energy function to evaluate the relative importance of each neuron, thereby highlighting the features of the target area and reducing the interference of redundant information.

[0040] Furthermore, the decoder calculates the semantic mask through Transformer and outputs the final pixel-level semantic segmentation result, including: the decoder processes the final image features through the Transformer structure, calculates the semantic category of each pixel, generates a preliminary semantic mask, and through a fixed matching strategy, the decoder finally outputs the processed pixel-level semantic segmentation result, provides the semantic category label corresponding to each pixel, and completes the semantic segmentation task of the image; during the decoding process, text embedding is used as auxiliary information to ensure the semantic alignment between visual features and text descriptions.

[0041] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and runnable on the processor, characterized in that when the processor executes the computer program, the above-mentioned domain generalization semantic segmentation enhancement method based on efficient multi-scale and simple attention modules is implemented.

[0042] The advantages and beneficial effects of the present invention are as follows:

[0043] 1. Based on the TQDM model, this paper proposes the ES-TQDM semantic segmentation model, introduces the SimAM lightweight attention mechanism and the EMA multi-scale attention module, so that the model can show better generalization ability in cross-domain semantic segmentation tasks, especially maintaining high segmentation accuracy under different scenarios and data distributions.

[0044] 2. SimAM optimizes feature extraction through a spatial attention mechanism, particularly when processing small objects and complex boundaries, effectively improving the accuracy and robustness of fine-grained semantic segmentation. SimAM's parameter-free design also reduces the computational burden, making training on large datasets more efficient and ensuring the model retains good feature capture capabilities while maintaining computational efficiency.

[0045] EMA enhances the model's adaptability across diverse data domains through multi-scale information aggregation and a smooth feature update mechanism. Its parallel computing and cross-spatial learning strategy enable the model to stably learn both global and local features during training, reducing sudden fluctuations and improving cross-domain generalization. Especially in unknown target domains or complex environments, EMA enables the model to more robustly model visual features at different scales, thereby improving semantic segmentation accuracy.

[0046] ES-TQDM further optimizes the text-driven semantic segmentation mechanism, enabling a tighter fusion of visual and text features, reducing generalization errors caused by cross-modal information mismatch. This improvement enhances the model's adaptability across multiple visual domains, enabling it to more effectively utilize text query information and guide more accurate semantic segmentation. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 This is a flowchart of a preferred embodiment of the present invention providing a domain generalization semantic segmentation enhancement method based on an efficient multi-scale and simple attention module. DETAILED DESCRIPTION

[0048] The following will describe the technical solutions in the embodiments of the present invention in detail with reference to the accompanying drawings. The described embodiments are only a part of the embodiments of the present invention.

[0049] like Figure 1 As shown in the figure, ES-TQDM: A domain-generalized semantic segmentation enhancement method based on efficient multi-scale and simple attention modules, including:

[0050] S1 adopts a Transformer-based encoder-decoder structure to achieve cross-modal fusion of image and text features.

[0051] In S2, the encoder uses CLIP-ViT-B-16 as the backbone. It first maps the input image to a high-dimensional vector sequence through Patch Embedding, then uses multi-layer ViT multi-head self-attention to capture long-range dependencies, and fuses features of different depths between key layers to progressively enhance semantic representation. The final output high-semantic visual features are fed into the subsequent cross-modal fusion and decoding modules.

[0052] S3, the encoder combines with the SimAM module to extract features, enhance the expression of global and local features, and improve the model's perception of key information. Specifically, it includes:

[0053] SimAM solves the problem of existing attention mechanisms relying on explicit parameter learning and extra calculations by modeling a parameter-free energy function. SimAM measures the relative importance of neurons in a channel by calculating their energy. To this end, SimAM defines an energy function e t :

[0054]

[0055] in, and Represents t and x respectively i The linear transformation applied, t represents the target neuron in a channel of the input feature X, and x i Represents other neurons in the same channel, index i traverses the entire spatial dimension, and the total number of neurons in the channel is M = H × W, H and W represent the height and width of the feature map respectively, w t and b t are weights and biases respectively, and y t and y o is a binary label.

[0056] SimAM minimizes the energy function and learns the optimal weights and biases to separate the target neuron from other neurons as much as possible, thereby enhancing the expressiveness of features. Specifically, SimAM calculates weights using the mean and variance of each neuron. The optimized attention weights can highlight important features and suppress redundant information, improving the robustness of the model.

[0057] Among them, the analytical solution of neuron weights and biases is:

[0058]

[0059] in represents the mean value of neurons in the channel, represents the variance of neurons within the channel, and λ is the regularization term.

[0060] SimAM further evaluates the importance of each neuron by calculating the minimum energy and adjusts the attention weight accordingly:

[0061]

[0062] in represents the sample mean of all M neurons under this channel, Represents the sample variance of all M neurons under this channel. This formula shows that when the value of the target neuron t deviates significantly from the channel mean, its energy value e * t A lower value means that the neuron is more prominent in visual processing and has higher importance. On the contrary, if the value of the neuron is close to the channel mean, the energy value e * t A higher value indicates that the neuron contributes less to the overall information.

[0063] Finally, in order to effectively incorporate the calculated importance information into the input features, SimAM uses the Sigmoid function to normalize the attention weights and applies it to each pixel of the input feature map:

[0064]

[0065] Where E is the energy value in all channels and spatial dimensions, represents the normalized output feature map, and X represents the original feature map. Through pixel-by-pixel weighting, SimAM adaptively adjusts attention weights at multiple scales, thereby strengthening the response of important features and suppressing the influence of unimportant features. Through SimAM's enhanced feature representation, the encoder can more accurately capture global and local semantic information, especially small objects and object boundaries, thereby effectively improving the accuracy and robustness of fine-grained semantic segmentation.

[0066] S4, the semantic segmentation decoder is based on the TQDM structure. It introduces text embeddings during the decoding process to enhance semantic consistency and improves the generalization ability of the model by fusing multi-scale features. In the decoder, the trained text embeddings are used to guide the image features to ensure the consistency between the semantic information of the image and the text description. At the same time, a multi-scale feature fusion mechanism is adopted to combine visual features from different scales to help the model better adapt to diverse scenes and improve its ability to capture details. By integrating cross-scale information, the model's ability to recognize objects of different sizes is further enhanced, thereby improving the accuracy and robustness of semantic segmentation. Finally, after the fusion of text embeddings and multi-scale features, high-quality semantic segmentation results are generated, ensuring the excellent performance of the model in various complex scenes.

[0067] S5, text embeddings are generated using learnable context. Category names are first tokenized, then combined with the trainable context to generate the final text features. Cross-modal alignment is achieved in the Transformer decoder stage. First, the input category names are tokenized, converting each category name into a discrete text representation. Tokenization decomposes the category names into lexical units or subwords, which are then mapped to a high-dimensional representation using pre-trained word vectors or embedding layers.

[0068] Next, a trainable context generation mechanism is used to combine the tokenized representation of each category to generate the final text features. This context information is generated through learnable parameters (such as the weights of the embedding layer) and can provide semantically complementary information for each category.

[0069] During the Transformer decoder stage, text embeddings are cross-modally aligned with image features. Specifically, through the fusion of text and image features, the semantic information expressed in the text description is matched with the visual information in the image, thereby enhancing the recognition and location of semantic regions in the image. Through this process, the resulting text features not only provide additional semantic guidance for the image but also effectively enhance the semantic consistency of the model when processing different categories, improving the effectiveness of cross-modal fusion.

[0070] In the feature enhancement stage, S6 uses the EMA module to perform cross-space learning on visual embeddings, enabling the model to maintain stable feature expression capabilities at different scales. It also combines the SimAM module to perform pixel-level attention calculations on visual embeddings to reduce redundant information and enhance the responsiveness of the target area. Specifically, the following steps are involved:

[0071] First, the EMA module is used for cross-space learning. The EMA module adopts a parallel substructure design, combining 1×1 convolution, 3×3 convolution and feature grouping mechanism to effectively model local and global information. First, EMA groups the input feature map by channel dimension to generate multiple sub-features, improving the model's ability to capture fine-grained features at different scales. Each sub-feature is processed by a 1×1 convolution branch and a 3×3 convolution branch respectively. The specific operation is expressed as:

[0072]

[0073] where z c represents the global pooling feature of the cth channel, x crepresents the input feature of the cth channel, i represents the "row" index in the spatial domain, that is, the position of the feature map in the height direction, and j represents the "column" index in the spatial domain, that is, the position of the feature map in the width direction. This mechanism ensures that EMA can aggregate information at different spatial scales, not only enhancing the model's ability to model long-range dependencies, but also more accurately embedding spatial location information, ultimately generating efficient and accurate pixel-level attention maps.

[0074] Finally, EMA fuses the outputs of the 1×1 and 3×3 branches and generates the final pixel-level attention map using a sigmoid activation function. This module integrates global and local information at multiple scales, improving the model's ability to model multi-scale visual features.

[0075] Building on this foundation, we further introduced the SimAM module to enhance the pixel-level saliency of the visual embedding features output by EMA. This module performs pixel-level attention calculations on the input visual embedding. SimAM uses a parameter-free energy function model to assess the relative importance of each neuron, thereby highlighting the features of the target region and reducing the interference of redundant information. This process enhances the responsiveness of the target region and effectively improves the model's performance when handling small objects and boundary details.

[0076] S7, finally, the decoder calculates the semantic mask through Transformer, and combines it with a fixed matching strategy to ensure the consistency of the predicted categories, and outputs the final pixel-level semantic segmentation result.

[0077] The decoder processes the visual features output by the encoder through the Transformer structure, calculates the semantic category of each pixel, and generates a preliminary semantic mask. A fixed matching strategy is used to ensure that the category predictions generated by the decoder are consistent with the actual semantic information in the input image, thereby ensuring the consistency and accuracy of the model's predictions in different scenarios. The decoder ultimately outputs the processed pixel-level semantic segmentation results, providing the semantic category label corresponding to each pixel, completing the semantic segmentation task of the image. At the same time, during the decoding process, text embedding is used as auxiliary information to ensure semantic alignment between visual features and text descriptions, further improving the effect of cross-modal fusion. These steps ensure the accuracy of semantic segmentation, and the fixed matching strategy and cross-modal alignment improve the model's adaptability to different scenarios.

[0078] An electronic device comprises a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps of the above-mentioned ES-TQDM: domain generalization semantic segmentation enhancement method based on efficient multi-scale and simple attention module are implemented.

[0079] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions.

[0080] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0081] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0082] The above embodiments should be understood as merely illustrating the present invention and not as limiting the scope of protection of the present invention. After reading the contents of the present invention, technicians may make various changes or modifications to the present invention, and these equivalent changes and modifications also fall within the scope defined by the claims of the present invention.

Claims

1. A domain-generalized semantic segmentation enhancement method based on efficient multi-scale and simple attention modules, characterized by: The following steps are involved: Adopting a Transformer-based encoder-decoder structure to achieve cross-modal feature fusion of image and text features; The encoder uses CLIP-ViT-B-16 as the backbone network and combines it with the SimAM module to extract features from the input image, obtain image features, and then input them into the cross-modal feature fusion module; The input text is preprocessed to generate text features, which are then input into the cross-modal feature fusion module; The cross-modal feature fusion module performs cross-modal fusion of image features and text features, including: deconstructing image features into global features and visual embeddings, using the EMA module to perform cross-space learning on the visual embeddings, and then using SimAM to perform pixel-level attention calculations, splicing the global features to obtain the final image features; the text features are passed through a text encoder to generate text embeddings; and the final image features are cross-modally aligned with the text embeddings. Finally, the decoder calculates the semantic mask through the Transformer and outputs the final pixel-level semantic segmentation result.

2. The domain-generalized semantic segmentation enhancement method based on efficient multi-scale and simple attention modules according to claim 1, characterized in that: The encoder uses CLIP-ViT-B-16 as the backbone network, and the specific processing steps include: The input image is first divided into patches of fixed size through the Patch Embedding layer and mapped to a high-dimensional feature space; Using the Vision Transformer (ViT) structure, the global relationship between different patches is modeled through the Multi-Head Self-Attention (MHSA) mechanism; In the hierarchical calculation process of ViT, the features of different Transformer layers are fused; Finally, the visual features calculated by multiple layers of Transformer are extracted to obtain image features.

3. The domain-generalized semantic segmentation enhancement method based on efficient multi-scale and simple attention modules according to claim 1 or 2, characterized in that: The combined SimAM module measures the relative importance of neuron t in the channel by calculating its energy. To this end, SimAM defines an energy function e t : in, and Represents t and x respectively i The linear transformation applied, t represents the target neuron in a channel of the input feature X, and x i Represents other neurons in the same channel, index i traverses the entire spatial dimension, and the total number of neurons in the channel is M = H × W, H and W represent the height and width of the feature map respectively, w t and b t are weights and biases respectively, and y t and y o is a binary label: SimAM learns the optimal weights and biases by minimizing the energy function to separate the target neuron from other neurons as much as possible; Among them, the analytical solution of neuron weights and biases is: in represents the mean value of neurons in the channel, represents the variance of neurons within the channel, and λ is the regularization term; SimAM further evaluates the importance of each neuron by calculating the minimum energy and adjusts the attention weight accordingly: in represents the sample mean of all M neurons under this channel, Represents the sample variance of all M neurons under this channel; Finally, in order to effectively incorporate the calculated importance information into the input features, SimAM uses the Sigmoid function to normalize the attention weights and applies it to each pixel of the input feature map: Where E is the energy value in all channels and spatial dimensions, Denotes the normalized output feature map, and X denotes the original feature map. Through pixel-by-pixel weighting, SimAM adaptively adjusts the attention weights at multiple scales.

4. The domain-generalized semantic segmentation enhancement method based on efficient multi-scale and simple attention modules according to claim 1, characterized in that: The text preprocessing of the input text includes tokenizing the input category name text, converting each category name into a discrete text representation, and the tokenization operation decomposes the category name into vocabulary units or subwords, which are mapped to representations in a high-dimensional space through pre-trained word vectors or embedding layers.

5. The domain-generalized semantic segmentation enhancement method based on efficient multi-scale and simple attention modules according to claim 1 or 4, characterized in that: The text encoder generates the text embedding through a trainable context generation mechanism, combined with the tokenized representation of each category.

6. The domain-generalized semantic segmentation enhancement method based on efficient multi-scale and simple attention modules according to claim 1, characterized in that: The EMA module performs cross-space learning on visual embeddings. EMA groups the input feature maps by channel dimension to generate multiple sub-features. Each sub-feature is processed by a 1×1 convolution branch and a 3×3 convolution branch. The specific operation is expressed as follows: where z c represents the global pooling feature of the cth channel, x c Represents the input feature of the c-th channel, i represents the "row" index of the spatial domain, that is, the position of the feature map in the height direction, j represents the "column" index of the spatial domain, that is, the position of the feature map in the width direction; The outputs of the 1×1 and 3×3 branches are fused and the final pixel-level attention map is generated through the Sigmoid activation function.

7. The domain-generalized semantic segmentation enhancement method based on efficient multi-scale and simple attention modules according to claim 1 or 6, characterized in that: The use of SimAM to further perform pixel-level attention calculation on visual embedding based on the EMA module includes modeling a parameter-free energy function to evaluate the relative importance of each neuron, thereby highlighting the characteristics of the target area and reducing the interference of redundant information.

8. The domain-generalized semantic segmentation enhancement method based on efficient multi-scale and simple attention modules according to claim 1, characterized in that: The decoder calculates the semantic mask through the Transformer and outputs the final pixel-level semantic segmentation result, including: the decoder processes the final image features through the Transformer structure, calculates the semantic category of each pixel, generates a preliminary semantic mask, and finally outputs the processed pixel-level semantic segmentation result through a fixed matching strategy, providing the semantic category label corresponding to each pixel, completing the semantic segmentation task of the image; during the decoding process, text embedding is used as auxiliary information to ensure the semantic alignment between visual features and text descriptions.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, it implements the domain generalization semantic segmentation enhancement method based on efficient multi-scale and simple attention module as described in any one of claims 1 to 8.

Images