A semantic segmentation method and system based on multi-scale semantic enhancement

Abstract

This invention belongs to the field of semantic segmentation technology, proposing a semantic segmentation method and system based on multi-scale semantic enhancement. The method includes: constructing a multi-scale semantic segmentation model; acquiring an input image and inputting it into a backbone network for feature extraction, obtaining a shallow feature map containing spatial structure information and a deep feature map containing high-level semantic information; inputting the deep feature map into a multi-scale semantic enhancement module for multi-scale modeling enhancement, obtaining deep enhanced features; inputting the deep enhanced features into a multi-scale context aggregation module for global semantic integration, obtaining integrated features; inputting the shallow feature map and integrated features into a feature fusion module for feature fusion, obtaining fused features; and inputting the fused features into a classification output module for classification processing and outputting pixel-level semantic segmentation results. This invention solves the problems of insufficient multi-scale modeling capability, inadequate representation of structural information, and difficulty in balancing accuracy and efficiency in existing semantic segmentation methods.

Description

Technical Field

[0001] This invention belongs to the field of semantic segmentation technology, and in particular relates to a semantic segmentation method and system based on multi-scale semantic enhancement. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] Semantic segmentation, as a pixel-level dense prediction task, has significant application value in high-precision vision scenarios such as autonomous driving, remote sensing image analysis, and medical image processing. In these applications, the system not only requires high segmentation accuracy but also needs to meet the deployment requirements of real-time inference and low computational resource consumption.

[0004] Existing semantic segmentation methods primarily rely on convolutional neural networks for feature extraction. While convolutional structures offer high computational efficiency and good local modeling capabilities, their receptive field is limited by a fixed kernel structure and preset dilation rate, making it difficult to dynamically adjust based on image content. This results in insufficient modeling capabilities in scenarios with multiple scale targets, especially in scenes with small targets, complex boundaries, or significant scale variations. Fixed receptive field mechanisms often lead to insufficient semantic representation or loss of spatial details. In real-time semantic segmentation tasks, lightweight backbone networks are typically employed to balance speed and accuracy.

[0005] However, existing lightweight methods often neglect detailed modeling of boundary regions and small target regions during network compression and structural simplification, leading to the loss of structural information and affecting the spatial continuity and edge accuracy of the segmentation results. Meanwhile, most real-time models lack explicit geometric or structural prior modeling mechanisms, resulting in insufficient ability to express local spatial relationships. Furthermore, while introducing global modeling structures such as Transformers can enhance semantic consistency and long-distance dependency modeling capabilities, it typically significantly increases the model parameter scale and computational complexity, reduces inference speed, and fails to meet the demands of high-resolution real-time applications. Summary of the Invention

[0006] To overcome the shortcomings of the prior art, this invention provides a semantic segmentation method and system based on multi-scale semantic enhancement, which solves the problems of insufficient multi-scale modeling capability, insufficient expression of structural information, and difficulty in balancing accuracy and efficiency in existing semantic segmentation methods.

[0007] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions: The first aspect of this invention provides a semantic segmentation method based on multi-scale semantic enhancement, comprising: Construct a multi-scale semantic segmentation model, including a backbone network and a multi-scale semantic enhancement module, a multi-scale context aggregation module, a feature fusion module, and a classification output module that are introduced into the backbone network; The input image is acquired and fed into the backbone network for feature extraction, resulting in a shallow feature map containing spatial structure information and a deep feature map containing high-level semantic information. The deep feature map is input into the multi-scale semantic enhancement module for multi-scale modeling and enhancement to obtain deep enhanced features; The deep enhancement features are input into the multi-scale context aggregation module for global semantic integration to obtain integrated features; The shallow feature map and the integrated feature are input into the feature fusion module for feature fusion to obtain the fused feature; The fused features are input into the classification output module for classification processing and the pixel-level semantic segmentation results are output.

[0008] As a further technical solution, the specific process of constructing a multi-scale semantic segmentation model is as follows: Based on the FENet network, parallel local feature extraction branches and global semantic modeling branches are constructed during the training phase. The local feature extraction branches are used as the backbone network, and the semantic expressive power of the backbone network is enhanced through feature alignment constraints. During the inference phase, only the backbone network is retained, and the input image is sequentially processed through feature extraction, multi-scale modeling enhancement, global semantic integration, feature fusion, classification, and output steps to obtain the semantic segmentation result.

[0009] As a further technical solution, the deep feature map is input into the multi-scale semantic enhancement module for multi-scale modeling enhancement, specifically including: The multi-scale semantic enhancement module includes an adaptive attention-dilated convolutional unit, ConvAttention, MLP, and an edge-guided refinement module. In this process, multi-scale feature extraction is performed on the deep feature map by an adaptive attention-dilated convolutional unit to obtain multi-scale enhanced features; The edge-guided thinning module enhances the local geometric structure information of the deep feature map to obtain structurally enhanced features. The multi-scale enhancement features are fused with the structural enhancement features to obtain the deep enhancement features.

[0010] As a further technical solution, the multi-scale feature extraction of the deep feature map using adaptive attention-dilated convolutional units specifically includes: The adaptive attention-dilated convolutional unit includes a channel attention enhancement unit and a learnable dilated convolutional branch structure; Channel attention enhancement is applied to the deep feature map to obtain channel-weighted enhanced features; The channel enhancement features are respectively input into multiple parallel dilated convolution branches with learnable dilation rates to obtain multiple dilated convolution branch outputs; The outputs of the multiple dilated convolution branches are weighted and fused to output the multi-scale enhanced features.

[0011] As a further technical solution, the enhancement of local geometric structure information of the deep feature map through the edge-guided thinning unit specifically includes: Each spatial location in the deep feature map is taken as a vertex to obtain vertex features, and the neighborhood features of each vertex in a preset direction are extracted. Edge features are constructed based on vertex features and neighborhood features. The edge features are aggregated to obtain aggregated edge features; The vertex features and aggregated edge features are updated and modulated to obtain modulated features; The modulated features are fused with the deep feature map to output the structure-enhanced features.

[0012] As a further technical solution, the vertex features and aggregated edge features are updated and modulated, specifically including: The aggregated edge features are concatenated with the vertex features to obtain the updated vertex features; The updated vertex features are concatenated with the edge features in each direction to obtain the updated edge features in each direction; The updated features in each direction are compressed and fused to obtain structural modulation features; The structure modulation is achieved by multiplying the structure modulation feature element-wise with the updated vertex feature to obtain the modulated feature.

[0013] As a further technical solution, feature fusion is performed between the shallow feature map and the integrated feature input feature fusion module, specifically including: The shallow feature map and the integrated feature are initially fused together, and the fused feature is then refined by convolution to become the fused feature.

[0014] A second aspect of the present invention provides a semantic segmentation system based on multi-scale semantic enhancement, comprising: The model building module is configured to: build a multi-scale semantic segmentation model, including a backbone network and a multi-scale semantic enhancement module, a multi-scale context aggregation module, a feature fusion module, and a classification output module that are introduced into the backbone network; The feature extraction module is configured to: acquire an input image and input it into the backbone network for feature extraction, thereby obtaining a shallow feature map containing spatial structure information and a deep feature map containing high-level semantic information; The feature enhancement module is configured to input the deep feature map into the multi-scale semantic enhancement module for multi-scale modeling enhancement to obtain deep enhanced features; The multi-scale context aggregation module is configured to: input the deep enhancement features into the multi-scale context aggregation module for global semantic integration to obtain integrated features; The feature fusion module is configured to: input the shallow feature map and the integrated feature into the feature fusion module to perform feature fusion to obtain fused features; The classification output module is configured to: input the fused features into the classification output module for classification processing and output pixel-level semantic segmentation results.

[0015] A third aspect of the present invention provides a computer-readable storage medium having a program stored thereon that, when executed by a processor, implements the steps of a semantic segmentation method based on multi-scale semantic enhancement as described in the first aspect of the present invention.

[0016] A fourth aspect of the present invention provides an electronic device including a memory, a processor, and a program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of a semantic segmentation method based on multi-scale semantic enhancement as described in the first aspect of the present invention.

[0017] The above one or more technical solutions have the following beneficial effects: This invention introduces a multi-scale semantic enhancement module into the backbone network, enabling adaptive feature modeling of targets at different scales. Compared to traditional fixed receptive field convolutional structures, this effectively improves the model's ability to express multi-scale targets, thereby enhancing semantic segmentation accuracy in complex scenes. A multi-scale context aggregation module performs global semantic integration of deep features, enhancing the network's ability to model long-distance dependencies while maintaining low computational complexity. This allows the model to obtain more complete global semantic information, improving the segmentation consistency and integrity of large-scale target regions. A feature fusion module effectively integrates shallow spatial structure information with deep semantic information, preserving rich spatial detail while maintaining high-level semantic expressiveness, thus improving segmentation accuracy in boundary regions and small target regions. The overall network structure maintains high segmentation accuracy while keeping computationally low and inference speed high, demonstrating significant application value in real-time vision applications such as autonomous driving, achieving an effective balance between accuracy and efficiency in semantic segmentation tasks.

[0018] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0019] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0020] Figure 1 This is a flowchart of the semantic segmentation method in the first embodiment; Figure 2 This is a diagram of the overall network structure constructed in the first embodiment; Figure 3 This is a structural diagram of the adaptive attention dilated convolution module in the first embodiment; Figure 4 This is a structural diagram of the edge guidance refinement module in the first embodiment; Figure 5 This is a schematic diagram of the visualization results on the Cityscapes dataset in the first embodiment. Detailed Implementation

[0021] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0022] It should be noted that the terminology used herein is for the purpose of describing particular implementations only and is not intended to limit the exemplary implementations of the present invention.

[0023] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0024] The overall concept proposed in this invention is as follows: First, while existing real-time semantic segmentation methods based on convolutional neural networks (CNNs) offer advantages in inference efficiency, their fixed receptive fields and lack of effective modeling of long-distance dependencies make them difficult to achieve adaptive semantic representation in scenarios with multiple scale targets, particularly in small target regions and complex boundary regions where segmentation accuracy is insufficient. Second, existing methods typically rely on self-attention or Transformer structures to improve global semantic modeling capabilities, but these structures have high computational complexity and large parameter scales, making them unsuitable for real-time inference under high-resolution input conditions, resulting in a trade-off between accuracy and efficiency.

[0025] Therefore, how to improve the model's adaptive modeling ability for multi-scale targets and enhance the structural representation ability of boundary and small target regions without increasing the computational overhead of the inference stage, while maintaining the overall lightweight and efficient nature of the network, has become a key technical problem that urgently needs to be solved in the field of real-time semantic segmentation.

[0026] This invention constructs a multi-scale adaptive feature enhancement mechanism to dynamically adjust the receptive field of targets at different scales, and strengthens the expression of local structural information by introducing a boundary structure refinement mechanism, thereby effectively improving the model's segmentation accuracy in boundary and fine-grained regions. By constructing a training-inference decoupled dual-branch feature extraction structure, an auxiliary semantic enhancement mechanism is introduced during the training phase to enhance the expressive power of the backbone network, while only the lightweight backbone structure is retained during the inference phase. This achieves a synergistic improvement in semantic expressive power and real-time performance without increasing the computational complexity of inference.

[0027] To achieve the above objectives, the implementation methods will be explained in detail through the following embodiments.

[0028] Example 1 like Figure 1 As shown, this embodiment discloses a semantic segmentation method based on multi-scale semantic enhancement, including: S1: Construct a multi-scale semantic segmentation model, including a backbone network and a multi-scale semantic enhancement module, a multi-scale context aggregation module, a feature fusion module, and a classification output module that are introduced into the backbone network; S2: Acquire the input image and input it into the backbone network for feature extraction to obtain a shallow feature map containing spatial structure information and a deep feature map containing high-level semantic information; S3: Input the deep feature map into the multi-scale semantic enhancement module for multi-scale modeling and enhancement to obtain deep enhanced features; S4: Input the deep enhancement features into the multi-scale context aggregation module for global semantic integration to obtain integrated features; S5: Input the shallow feature map and the integrated feature into the feature fusion module to obtain the fused feature; S6: The fused features are input to the classification output module for classification processing and the pixel-level semantic segmentation results are output.

[0029] As a further technical solution, in step S1, based on the FENet network, parallel local feature extraction branches and global semantic modeling branches are constructed during the training phase. The local feature extraction branches are used as the backbone network, and the semantic expression capability of the backbone network is enhanced through feature alignment constraints. During the inference phase, only the backbone network is retained, and the input image is sequentially processed through feature extraction, multi-scale modeling enhancement, global semantic integration, feature fusion, classification, and output steps to obtain the semantic segmentation result.

[0030] Specifically, to balance segmentation accuracy and inference speed in semantic segmentation tasks, the overall network structure is constructed as follows: Figure 2 As shown, a Transformer-CNN dual-branch structure is constructed during the training phase. The CNN branch serves as the backbone network to extract local spatial features, while the Transformer branch is used to model global semantic information. A feature alignment module is used to constrain the semantic consistency of the outputs of the two branches, thereby enhancing the expressive power of the backbone network. During the inference phase, only the lightweight CNN branch is retained for forward computation to balance segmentation accuracy and inference efficiency.

[0031] In step S2, the input image first enters the backbone network for feature extraction. In the shallow layer stage, the image is downsampled using a 3×3 convolutional module with a stride of 2 to extract high-resolution spatial structure information, providing guidance for subsequent detail restoration. In the middle layer stage, residual structural units are used to extract semantic features, improving semantic expressive power while controlling computational overhead. In the deep layer stage, an enhanced multi-scale semantic augmentation module (MSSEB) is introduced to strengthen high-level feature modeling capabilities.

[0032] In step S3, the multi-scale semantic enhancement module internally integrates the adaptive attention dilated convolutional unit (AADC) ConvAttention, MLP, and edge-guided refinement module (EGRM).

[0033] Among them, the adaptive attention-dilated convolutional unit combines channel attention mechanism with learnable dilation rate to achieve dynamic adjustment of convolutional receptive field, thereby enhancing the network's ability to model targets of different scales; the edge-guided refinement module introduces geometric structure constraint information by modeling the structural relationship between local neighborhoods in the feature map, and generates structure-enhanced features to improve the expressive ability of boundary regions and small target regions.

[0034] As a further technical solution, multi-scale feature extraction is performed on the deep feature map by adaptive attention dilated convolutional units to obtain multi-scale enhanced features; local geometric structure information enhancement is performed on the deep feature map by edge-guided thinning modules to obtain structural enhanced features; and the multi-scale enhanced features and structural enhanced features are fused to obtain deep enhanced features.

[0035] Specifically, to address the problem of limited multi-scale modeling capabilities caused by the fixed dilation rate and lack of channel selection mechanism in traditional multi-scale dilated convolution, this embodiment provides an adaptive attention dilated convolution module to achieve flexible and efficient multi-scale feature representation under lightweight constraints.

[0036] As a further technical feature, such as Figure 3As shown, the adaptive attention dilated convolution module includes a channel attention enhancement unit and a learnable dilated convolution branch structure. It performs channel attention enhancement on the deep feature map to obtain channel-weighted channel-enhanced features. The channel-enhanced features are then input into multiple parallel dilated convolution branches with learnable dilation rates to obtain multiple dilated convolution branch outputs. The outputs of the multiple dilated convolution branches are then weighted and fused to output multi-scale enhanced features.

[0037] Specifically, firstly, let the deep feature map output from the backbone network and input to the multi-scale semantic enhancement module be... Where C represents the number of channels in the feature map, and H and W represent the height and width of the feature map, respectively. Channel attention modeling is performed on the input feature map to highlight key information channels and suppress redundant channels. Channel information is extracted through global average pooling and global max pooling, respectively, to obtain: (1) (2) in, and This indicates a fully connected mapping layer. This represents the channel description vector obtained through global average pooling. This represents the channel description vector obtained through global max pooling. and The parameters of the corresponding fully connected mapping layer are represented by: AvgPool(·) represents the global average pooling operation; MaxPool(·) represents the global max pooling operation; and ReLU(·) represents the linear rectified activation function.

[0038] The two types of channel information are added together and then activated using a sigmoid function to obtain channel attention weights; subsequently, the input features are channel-weighted. This indicates an element-wise multiplication operation by channel: (3) (4) Where σ(·) represents the Sigmoid activation function; This represents the generated channel attention weight vector; This indicates an element-wise multiplication operation by channel; This represents the channel-enhanced features after channel attention weighting.

[0039] After obtaining the channel enhancement features, they are input into three parallel 3×3 dilated convolution branches, each branch corresponding to a learnable dilation rate. To ensure stability during training, the expansion rate is limited to a preset effective range: (5) No. The output of each dilated convolution branch is: , (6) in, (·) denotes the interval restriction function, used to restrict the expansion rate to [ , Within the range of ]; A learnable parameter representing the expansion rate. and These represent the minimum and maximum values ​​of the expansion rate, respectively; RPReLU(·) represents the parameterized linear rectified activation function with a learnable offset parameter; Indicates the first The feature map output by each dilated convolution branch.

[0040] During training, the dilation rate parameter is adaptively adjusted based on the image content. When the input contains large-scale structural regions, the dilation rate is increased to expand the effective receptive field; when the input contains small targets or boundary regions, the dilation rate is decreased to avoid over-smoothing, thus achieving adaptive modeling of features at different scales. The outputs of the three branches are weighted and fused using learnable weights to obtain the final output features: (7) in, It is a learnable scalar parameter used to control the contribution ratio of features at each scale.

[0041] By combining the channel attention enhancement mechanism and the adaptive expansion rate learning mechanism, this embodiment significantly improves the multi-scale feature modeling capability while maintaining low computational complexity and taking into account both local detailed features and the ability to express large-scale contextual information.

[0042] As a further technical feature, each spatial location in the deep feature map is taken as a vertex to obtain vertex features, and the neighborhood features of each vertex in a preset direction are extracted. Edge features are constructed based on vertex features and neighborhood features. The edge features are aggregated to obtain aggregated edge features. The vertex features and aggregated edge features are updated and modulated to obtain modulated features. The modulated features are fused with the deep feature map to output structure-enhanced features.

[0043] As a further technical feature, the aggregated edge features are concatenated with the vertex features to obtain updated vertex features; the updated vertex features are then concatenated with edge features in each direction to obtain updated edge features in each direction; the updated edge features in each direction are then compressed and fused to obtain... ;pass The updated vertex features are multiplied element-wise to achieve structural modulation, resulting in modulated features. This represents the structural modulation features obtained by compressing and fusing the updated features from each direction, used to characterize the position. Local structural relationship information.

[0044] Specifically, generally speaking, after obtaining high-level semantic features, feature extraction networks typically perform prediction or upsampling recovery directly through convolutional fusion. However, while this method of directly utilizing convolutional operations for feature enhancement is simple to implement, it lacks explicit modeling of local structural relationships, especially in boundary regions and small target regions, resulting in insufficient feature representation capabilities and thus affecting the model's ability to characterize detailed information. Therefore, as... Figure 4 As shown, this embodiment proposes an edge-guided refinement module to enhance the feature representation capabilities of boundary regions and small target regions. The edge-guided refinement module enhances the input features with fine-grained structure by constructing a vertex-edge interaction mechanism and introducing a structure modulation strategy.

[0045] First, the vertices and edges are constructed. Given the input deep feature map... , to each spatial location Consider it as a vertex, its corresponding feature is represented as For each vertex, select its neighborhood features in the four directions of up, down, left, and right, and represent them as follows: Based on the local interaction relationships between vertices and their neighbors, the edge features in the four directions are defined as follows: (8) in, This represents element-wise multiplication. Indicates position Its first Edge features constructed between neighborhoods in each direction are used to characterize the local interaction relationships between the current vertex and its neighboring vertices.

[0046] Subsequently, the edge features in the four directions are aggregated to obtain a stable local structure representation. After obtaining the aggregated edge features, they are concatenated with the original vertex features, and the vertex representation is updated. The specific process can be represented by the following formula: (9) (10) in, This represents the aggregated edge feature obtained by aggregating edge features from four directions, used to comprehensively represent the position. The local structural relationships of the surrounding neighborhood; This represents the updated vertex features obtained by fusing the original vertex features with the aggregated edge features, which are used to enhance the ability to express the local structure of vertices.

[0047] To further enhance the edge feature representation capability, an edge update and modulation mechanism is introduced. First, vertex features are concatenated with edge features in the corresponding directions to update the edge features. Then, the updated edge features in the four directions are compressed and fused. The specific process can be represented by the following formula: (11) (12) in, , , , Both represent edge feature aggregation functions, composed of Linear + BatchNorm + ReLU; Indicates the first The updated edge feature representations in each direction are used to further enhance the structural interaction between vertices and their neighborhoods; This represents the structural modulation features obtained by compressing and fusing the updated features from four directions, used to comprehensively characterize the position. Local geometric structure information; symbol " The colon indicates a splicing operation. Finally, structure modulation is achieved through element-wise multiplication: (13) in, This indicates that the structure enhancement features obtained through the structure modulation mechanism are used to strengthen the boundary information and local structural information in the vertex features.

[0048] To enhance structural expressiveness while preserving original semantic information, the modulated features... Input features The features are concatenated and then fused using a 1×1 convolution to obtain the final output features: (14) The edge-guided refinement module achieves explicit modeling of local geometric structure information by constructing vertex-edge interaction relationships, edge feature aggregation, and structural modulation mechanisms. Because this module employs a progressive enhancement approach, vertex update results participate in subsequent edge update and modulation processes, thus forming a progressive structural refinement.

[0049] Finally, the deep enhancement features are input into the multi-scale context aggregation module. The feature map is modeled using multi-scale receptive fields through context information extraction branches at different scales. The global semantic information is integrated through feature fusion operations, thereby obtaining integrated features with stronger semantic expressive power.

[0050] Subsequently, the integrated features are fused with the high-resolution spatial features output from the shallow stage of the backbone network to achieve complementary enhancement of high-level semantic information and spatial detail information.

[0051] After feature fusion, the fused features are further optimized through a convolutional thinning module to reduce feature noise and enhance semantic consistency.

[0052] The refined feature map is classified and predicted by a pixel-by-pixel classification layer, and a semantic label map corresponding to the size of the input image is output, thus obtaining the final semantic segmentation result.

[0053] In this embodiment, the proposed method was trained and evaluated on the Cityscapes and ADE20K datasets, and its effectiveness was verified through comparative experiments with various state-of-the-art methods.

[0054] In the experiments, all datasets were trained and tested according to standard partitioning. mIoU was used as the evaluation metric for semantic segmentation accuracy, and FPS was used as the evaluation metric for model inference speed, to comprehensively measure the model's performance in balancing accuracy and efficiency. The experimental results are shown in Tables 1 and 2. Excellent segmentation performance was achieved on both datasets, achieving a higher mIoU metric while maintaining high inference speed. Compared to existing state-of-the-art methods, this demonstrates a superior overall performance in terms of accuracy and speed, validating the effectiveness of the proposed structure in multi-scale modeling, boundary refinement, and feature representation enhancement.

[0055] Table 1. Comparison with other state-of-the-art methods on the Cityscapes dataset.

[0056] Table 2 Comparison with other state-of-the-art methods on the ADE20K dataset

[0057] like Figure 5As shown, to further verify the technical effectiveness of this invention in multi-scale feature representation and boundary refinement modeling, the segmentation results of the model on the Cityscapes dataset are visually compared with existing methods. In the figure, the first column is the original image, the second column is the ground truth label, the third column is SCTNet-B-75, and the fourth column is FENet-B-75. In the scenarios shown in the first, second, and fifth rows, this embodiment can accurately identify small-scale targets such as pedestrians, traffic signs, and utility poles at scene corners, while maintaining the integrity of the target structure and reducing missegmentation and omission of small target regions, demonstrating its technical advantages in fine-grained structure modeling. In the scenarios shown in the third and fourth rows, for large-scale target regions such as buses and roads, it shows more stable prediction results in terms of region coverage integrity and boundary continuity, accurately covering the target region and clearly defining its boundaries, reducing the problems of internal region fragmentation and category confusion.

[0058] This embodiment demonstrates stable and consistent segmentation performance across key semantic categories in autonomous driving-related scenarios, further validating the technical effectiveness in multi-scale feature representation and boundary refinement modeling.

[0059] Example 2 This embodiment discloses a semantic segmentation system based on multi-scale semantic enhancement, including: The model building module is configured to: build a multi-scale semantic segmentation model, including a backbone network and a multi-scale semantic enhancement module, a multi-scale context aggregation module, a feature fusion module, and a classification output module that are introduced into the backbone network; The feature extraction module is configured to: acquire an input image and input it into the backbone network for feature extraction, thereby obtaining a shallow feature map containing spatial structure information and a deep feature map containing high-level semantic information; The feature enhancement module is configured to input the deep feature map into the multi-scale semantic enhancement module for multi-scale modeling enhancement to obtain deep enhanced features; The multi-scale context aggregation module is configured to: input the deep enhancement features into the multi-scale context aggregation module for global semantic integration to obtain integrated features; The feature fusion module is configured to: input the shallow feature map and the integrated feature into the feature fusion module to perform feature fusion to obtain fused features; The classification output module is configured to: input the fused features into the classification output module for classification processing and output pixel-level semantic segmentation results.

[0060] Example 3 The purpose of this embodiment is to provide a computer-readable storage medium.

[0061] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of a semantic segmentation method based on multi-scale semantic enhancement as described in Embodiment 1 of this disclosure.

[0062] Example 4 The purpose of this embodiment is to provide an electronic device.

[0063] An electronic device includes a memory, a processor, and a program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps in a semantic segmentation method based on multi-scale semantic enhancement as described in Embodiment 1 of this disclosure.

[0064] The steps and methods involved in the apparatuses of Embodiments 2, 3, and 4 above correspond to those in Embodiment 1. For specific implementation details, please refer to the relevant description section of Embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood as including any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and enabling the processor to perform any of the methods in this invention.

[0065] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computer devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computer device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. The present invention is not limited to any particular combination of hardware and software.

[0066] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A semantic segmentation method based on multi-scale semantic enhancement, characterized in that, include: Construct a multi-scale semantic segmentation model, including a backbone network and a multi-scale semantic enhancement module, a multi-scale context aggregation module, a feature fusion module, and a classification output module that are introduced into the backbone network; The input image is acquired and fed into the backbone network for feature extraction, resulting in a shallow feature map containing spatial structure information and a deep feature map containing high-level semantic information. The deep feature map is input into the multi-scale semantic enhancement module for multi-scale modeling and enhancement to obtain deep enhanced features; The deep enhancement features are input into the multi-scale context aggregation module for global semantic integration to obtain integrated features; The shallow feature map and the integrated feature are input into the feature fusion module for feature fusion to obtain the fused feature; The fused features are input into the classification output module for classification processing and the pixel-level semantic segmentation results are output.

2. The semantic segmentation method based on multi-scale semantic enhancement as described in claim 1, characterized in that, The specific process of constructing a multi-scale semantic segmentation model is as follows: Based on the FENet network, parallel local feature extraction branches and global semantic modeling branches are constructed during the training phase. The local feature extraction branches are used as the backbone network, and the semantic expressive power of the backbone network is enhanced through feature alignment constraints. During the inference phase, only the backbone network is retained, and the input image is sequentially processed through feature extraction, multi-scale modeling enhancement, global semantic integration, feature fusion, classification, and output steps to obtain the semantic segmentation result.

3. The semantic segmentation method based on multi-scale semantic enhancement as described in claim 1, characterized in that, The deep feature map is input into the multi-scale semantic enhancement module for multi-scale modeling and enhancement, specifically including: The multi-scale semantic enhancement module includes an adaptive attention-dilated convolutional unit, ConvAttention, MLP, and an edge-guided refinement module. Specifically, multi-scale feature extraction is performed on the deep feature map using an adaptive attention-dilated convolutional unit to obtain multi-scale enhanced features; and local geometric structure information enhancement is performed on the deep feature map using an edge-guided thinning module to obtain structural enhanced features. The multi-scale enhancement features are fused with the structural enhancement features to obtain the deep enhancement features.

4. The semantic segmentation method based on multi-scale semantic enhancement as described in claim 3, characterized in that, The step of performing multi-scale feature extraction on the deep feature map using adaptive attention-dilated convolutional units specifically includes: The adaptive attention-dilated convolutional unit includes a channel attention enhancement unit and a learnable dilated convolutional branch structure; Channel attention enhancement is applied to the deep feature map to obtain channel-weighted enhanced features; The channel enhancement features are respectively input into multiple parallel dilated convolution branches with learnable dilation rates to obtain multiple dilated convolution branch outputs; The outputs of the multiple dilated convolution branches are weighted and fused to output the multi-scale enhanced features.

5. The semantic segmentation method based on multi-scale semantic enhancement as described in claim 3, characterized in that, The enhancement of local geometric structure information in the deep feature map through the edge-guided thinning unit specifically includes: Each spatial location in the deep feature map is taken as a vertex to obtain vertex features, and the neighborhood features of each vertex in a preset direction are extracted. Edge features are constructed based on vertex features and neighborhood features. The edge features are aggregated to obtain aggregated edge features; The vertex features and aggregated edge features are updated and modulated to obtain modulated features; The modulated features are fused with the deep feature map to output the structure-enhanced features.

6. The semantic segmentation method based on multi-scale semantic enhancement as described in claim 5, characterized in that, The vertex features and aggregated edge features are updated and modulated, specifically including: The aggregated edge features are concatenated with the vertex features to obtain the updated vertex features; The updated vertex features are concatenated with the edge features in each direction to obtain the updated edge features in each direction; The updated features in each direction are compressed and fused to obtain structural modulation features; The structure modulation is achieved by multiplying the structure modulation feature element-wise with the updated vertex feature to obtain the modulated feature.

7. The semantic segmentation method based on multi-scale semantic enhancement as described in claim 1, characterized in that, The shallow feature map and the integrated feature input feature fusion module are fused to form a feature fusion module, specifically including: The shallow feature map and the integrated feature are initially fused together, and the fused feature is then refined by convolution to become the fused feature.

8. A semantic segmentation system based on multi-scale semantic enhancement, characterized in that, include: The model building module is configured to: build a multi-scale semantic segmentation model, including a backbone network and a multi-scale semantic enhancement module, a multi-scale context aggregation module, a feature fusion module, and a classification output module that are introduced into the backbone network; The feature extraction module is configured to: acquire an input image and input it into the backbone network for feature extraction, thereby obtaining a shallow feature map containing spatial structure information and a deep feature map containing high-level semantic information; The feature enhancement module is configured to input the deep feature map into the multi-scale semantic enhancement module for multi-scale modeling enhancement to obtain deep enhanced features; The multi-scale context aggregation module is configured to: input the deep enhancement features into the multi-scale context aggregation module for global semantic integration to obtain integrated features; The feature fusion module is configured to: input the shallow feature map and the integrated feature into the feature fusion module to perform feature fusion to obtain fused features; The classification output module is configured to: input the fused features into the classification output module for classification processing and output pixel-level semantic segmentation results.

9. A computer-readable storage medium having a program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps in the semantic segmentation method based on multi-scale semantic enhancement as described in any one of claims 1-7.

10. An electronic device comprising a memory, a processor, and a program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the semantic segmentation method based on multi-scale semantic enhancement as described in any one of claims 1-7.

Images