Complex environment image semantic segmentation method based on fine-grained enhancement network

Abstract

This invention discloses a semantic segmentation method for complex environment images based on a fine-grained augmentation network. The method acquires an environment image to be detected and inputs it into a pre-trained fine-grained augmentation network model to obtain the semantic segmentation result of the environment image. The method utilizes an optimized fine-grained augmentation network model, which deeply extracts and fuses global and local feature information at multiple scales through the backbone network layer, enhancing the model's feature extraction capability. Then, by leveraging upsampling and detail enhancement processing in the neck network layer, the model's detail recovery capability is enhanced while effectively reducing image noise. Finally, the semantic segmentation result of the environment image is obtained through a detection head, achieving accurate semantic segmentation for complex environment images.

Description

Technical Field

[0001] This invention relates to the fields of deep learning technology and image processing technology, specifically to a semantic segmentation method for complex environment images based on fine-grained enhancement networks. Background Technology

[0002] In complex environments such as mountains and hills, high-precision semantic segmentation technology enables autonomous unmanned systems to more effectively identify terrain features and various objects within the environment, thereby constructing detailed and accurate environmental models. In these environments, land cover types often exhibit high spatial heterogeneity and fine-grained variations, and the spatial distribution of different land cover types is also diverse, making semantic segmentation a particularly complex task. Therefore, researching accurate semantic segmentation methods is of great significance for improving the intelligence level and application breadth of autonomous unmanned systems.

[0003] Deep learning-based semantic segmentation methods are generally divided into two-stage and one-stage methods, which differ significantly in processing flow and application scenarios. Two-stage methods, such as Mask R-CNN, the DeepLab series, and HRNet, first generate candidate regions and then perform fine-grained pixel-level classification on these regions. This method typically performs well in segmentation accuracy, but the models are complex and difficult to meet real-time requirements. In contrast, one-stage methods, such as the YOLO series, SAM (Segment Anything Model), and BiSeNet, directly complete object detection and segmentation tasks in a unified network. They do not rely on candidate region generation, have faster inference speeds, and lower computational costs, making them more suitable for semantic segmentation scenarios with high real-time requirements, such as autonomous driving and robot vision.

[0004] In recent years, single-stage semantic segmentation methods, represented by YOLO, have been widely used in fields such as autonomous driving, monitoring, and industrial inspection. The ASF-YOLO model proposed by Kang et al. enhances the network's multi-scale information extraction capability by introducing SSFF and TFE modules, and improves segmentation performance by focusing on useful channels and spatial locations. Asifa et al. effectively improved the accuracy of YOLOv5 in vehicle detection and tracking by combining segmentation techniques, SURF feature extraction, feature matching, and Kalman filtering. Yuan et al. proposed an adaptive multi-scale AM ​​YOLO algorithm, which fully integrates feature information and contextual details at different scales by introducing the multi-scale adaptive feature enhancement module MAEM and the bidirectional feature pyramid network RBiFPN. Bhavana et al.'s POT-YOLO method uses a contrast-stretched adaptive Gaussian star filter (CAGF) for image preprocessing and combines it with a Sobel edge detector to achieve high-precision pit segmentation.

[0005] The aforementioned scholars have achieved efficient semantic segmentation in various scenarios using the single-stage YOLO algorithm. However, the existing methods still lack fine-grained recognition capabilities in complex environments such as mountains and hills, leaving room for further improvement in semantic segmentation accuracy. Summary of the Invention

[0006] To address the shortcomings of the existing technologies, this invention provides a semantic segmentation method for complex environment images based on fine-grained enhancement networks, which improves the semantic segmentation accuracy for complex environments such as mountains and hills.

[0007] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0008] A semantic segmentation method for complex environment images based on fine-grained augmentation networks is proposed. The method acquires the environment image to be detected, inputs it into a pre-trained fine-grained augmentation network model, and obtains the semantic segmentation result of the environment image to be detected.

[0009] The fine-grained enhancement network model includes a backbone network layer, a neck network layer, and a detection head. The backbone network layer is used to extract global and local features and perform multi-scale fusion processing on the environmental image input to the feature reconstructing network to obtain a multi-scale feature map. The neck network layer is used to upsample and enhance the details of the multi-scale feature map to obtain an enhanced feature map. The detection head is used to obtain the semantic segmentation result of the environmental image to be detected based on the enhanced feature map, which serves as the output of the fine-grained enhancement network model.

[0010] Specifically, in the fine-grained augmentation network model:

[0011] The backbone network layer includes sequentially connected convolutional layers, four convolutional fusion units, and one spatial pyramid pooling module; each convolutional fusion unit includes cascaded convolutional layers and a C2f-SSA module.

[0012] The neck network layer includes two feature dynamic upsampling units and two convolutional attention enhancement units connected in sequence; wherein, the feature dynamic upsampling unit includes a Dysample-ConvBN upsampling module, a connection layer and a C2f-SSA module connected in sequence; the convolutional attention enhancement unit includes a convolutional layer, a connection layer and a C2f-SSA module connected in sequence.

[0013] The detection head is used to obtain the semantic segmentation result of the environmental image based on the output of the neck network layer;

[0014] Specifically, the input of the fine-grained enhancement network model serves as the input to the backbone network layer. Within the backbone network layer, the outputs of the second and third convolutional fusion units also serve as the inputs to the connection layers of the second and first dynamic feature upsampling units in the neck network layer, respectively. In addition to serving as the input to the neck network layer, the output of the spatial pyramid pooling module in the backbone network layer also serves as the input to the connection layer of the second convolutional attention enhancement unit in the neck network layer. Furthermore, the output of the first dynamic feature upsampling unit in the neck network layer also serves as the input to the connection layer of the first convolutional attention enhancement unit. The outputs of the second dynamic feature upsampling unit and the two convolutional attention enhancement units are all output to the detection head.

[0015] Specifically, the C2f-SSA module includes a cascaded first convolutional layer, a split module, multiple bottleneck modules, a connection layer, a second convolutional layer, and an SSA hybrid attention module. The two split outputs of the split module serve as the inputs to the first fast convolutional module and the connection layer, respectively. The outputs of each fast convolutional module also serve as the inputs to the connection layer. The output of the SSA hybrid attention module serves as the output of the C2f-SSA module.

[0016] Specifically, the processing procedure for each bottleneck module is as follows: after performing two convolutional processes on the input feature map of the bottleneck module, the feature map is then superimposed with the input feature map of the bottleneck module to obtain the output feature map of the bottleneck module.

[0017] Specifically, the SSA hybrid attention module includes cascaded SE attention modules and SA attention modules;

[0018] The processing procedure of the SE attention module is as follows:

[0019] First, the input feature map of the SE attention module is processed by a convolutional layer. Then, the convolution result is processed by global average pooling and fully connected mapping in sequence. The fully connected mapping result is multiplied by the previous convolution result to obtain the output feature map.

[0020] The processing procedure of the SA attention module is as follows:

[0021] First, channel grouping is performed on the output feature map of the SE attention module to obtain several channel group feature maps;

[0022] Then, for each channel group feature map, it is further divided into two sub-feature maps for channel attention and spatial attention operations respectively. In the channel attention operation branch, the sub-feature map is processed by average pooling, fully connected mapping, and sigmoid activation function in sequence, and then multiplied with the sub-feature map entering the branch to obtain the channel attention feature map. In the spatial attention operation branch, the sub-feature map is processed by group normalization, fully connected mapping, and sigmoid activation function in sequence, and then multiplied with the sub-feature map entering the branch to obtain the spatial attention feature map. Finally, the channel attention feature map and the spatial attention feature map are merged by element-wise multiplication to obtain the attention feature sub-map of the channel group.

[0023] Finally, the attention feature sub-maps of each channel group are rearranged and stitched together to obtain the attention feature map as the output.

[0024] Specifically, the spatial pyramid pooling module includes a cascaded first convolutional layer, three two-dimensional max pooling layers, a connecting layer, and a second convolutional layer. The outputs of the first convolutional layer and the three two-dimensional max pooling layers are also used as inputs to the connecting layer, and the output of the second convolutional layer is used as the output of the spatial pyramid pooling module.

[0025] Specifically, the Dysample-ConvBN upsampling module includes a cascaded Dysample upsampling module and a ConvBN module;

[0026] The processing procedure of the Dysample upsampling module is as follows: first, a sampling set is created from the input feature map of the Dysample upsampling module through a sampling point generator; then, the input feature map is resampled using the sampling point positions in the sampling set through a grid sampling function to obtain upsampled features.

[0027] The ConvBN module includes a two-dimensional convolutional layer, a two-dimensional batch normalization layer, and a SiLU activation function layer connected in sequence.

[0028] The ConvBN module is used to further upsample the result X output by the Dysample upsampling module. g Local feature extraction is performed to obtain the upsampled features output by the Dysample-ConvBN upsampling module. Figure X ′.

[0029] Specifically, the fine-grained enhancement network model is trained in the following manner:

[0030] S101: Prepare an environmental sample image dataset, which is divided into a training image dataset and a test image dataset. The environmental sample images in the environmental sample image dataset have all been pre-segmented and labeled with semantic segmentation location and category.

[0031] S102: Input the training image dataset into the fine-grained augmentation network model for training, and optimize the parameters of the fine-grained augmentation network model with the goal of minimizing the loss function until the fine-grained augmentation network model converges, thus obtaining the trained fine-grained augmentation network model.

[0032] S103: Test the fine-grained augmentation network model using the test image dataset to confirm the semantic segmentation performance of the trained fine-grained augmentation network model; if the semantic segmentation performance meets the requirements, end the training of the fine-grained augmentation network model; otherwise, return to step S102.

[0033] Specifically, in training the fine-grained augmentation network model, the localization loss uses the SIoU loss function, and the classification loss uses the binary cross-entropy (BCE) loss function.

[0034] Specifically, the SIoU loss function is calculated using the following formula:

[0035]

[0036] Among them, Loss SIoU SIoU loss represents the intersection-union ratio loss between the semantic segmentation prediction box and the semantic segmentation target box; Δ represents the distance loss between the semantic segmentation prediction box and the semantic segmentation target box; and Ω represents the shape loss between the semantic segmentation prediction box and the semantic segmentation target box.

[0037] The shape loss Ω is:

[0038]

[0039] Where w and h are the pixel width and pixel height of the semantic segmentation prediction box, respectively; w gt h gt , where are the pixel width and pixel height of the semantic segmentation target bounding box, respectively; e is the natural constant; θ is the shape attention coefficient;

[0040] The distance loss Δ is:

[0041]

[0042] in, These are the x-coordinate and y-coordinate of the center point of the semantic segmentation prediction bounding box, respectively. These are the x-coordinate and y-coordinate of the center point of the semantic segmentation bounding box, respectively; s w sh Λ represents the pixel width and pixel height of the minimum bounding rectangles of the semantic segmentation prediction box and the semantic segmentation target box, respectively; Λ represents the angular loss between the semantic segmentation prediction box and the semantic segmentation target box.

[0043]

[0044] Where, σ c c is the pixel distance between the center points of the semantic segmentation target bounding box and the semantic segmentation prediction bounding box. h This represents the difference in pixel y-coordinates between the semantic segmentation target bounding box and the semantic segmentation prediction bounding box.

[0045] Compared with the prior art, the present invention has the following technical effects:

[0046] 1. This invention provides a semantic segmentation method for complex environment images based on fine-grained augmentation networks. It utilizes an optimized fine-grained augmentation network model to extract and fuse global and local feature information at a deep level through the backbone network layer, thereby enhancing the model's feature extraction capability. Then, by leveraging upsampling and detail enhancement processing in the neck network layer, the model's detail recovery capability is enhanced while effectively reducing image noise. Finally, the semantic segmentation result of the environment image is obtained through the detection head, achieving accurate semantic segmentation for complex environment images.

[0047] 2. In the fine-grained enhancement network model of the present invention, the C2f module is combined with the SSA hybrid attention module to design a C2f-SSA module, which is introduced into the backbone network layer and the neck network layer. By arranging and calibrating the channels of the feature map, the global feature information and local feature information are extracted and fused at multiple scales in a deep level, capturing segmentation categories of different sizes and enhancing the feature extraction capability of the model.

[0048] 3. In the fine-grained enhancement network model of the present invention, a lightweight Dysample-ConvBN upsampling module is constructed in the neck network layer. It precisely controls the point sampling operation by dynamically adjusting the upsampling scale factor, and combines upsampling and convolution operations to enhance the model's detail recovery capability, effectively reduce image noise, and save computational resources. Attached Figure Description

[0049] To make the objectives, technical solutions, and advantages of the invention clearer, the invention will now be described in further detail with reference to the accompanying drawings, wherein:

[0050] Figure 1 This is a schematic diagram of the architecture of the fine-grained enhancement network model used in the method of the present invention;

[0051] Figure 2 This is a schematic diagram of the C2f-SSA module.

[0052] Figure 3 This is a structural diagram of the bottleneck module Bottleneck in the C2f-SSA module;

[0053] Figure 4 This is a schematic diagram of the SSA hybrid attention module;

[0054] Figure 5 This is a flowchart of the SE attention module's processing flow.

[0055] Figure 6 This is a schematic diagram of the ConvBN module.

[0056] Figure 7 This is a schematic diagram of the Dysample-ConvBN module.

[0057] Figure 8 A schematic diagram of the static range factor and dynamic range factor control process in the Dysample upsampling module;

[0058] Figure 9 This is a schematic diagram of the SPPF spatial pyramid pooling module.

[0059] Figure 10 This is a comparison chart of the semantic segmentation results of different models on an environmental image in the MT dataset in the embodiments;

[0060] Figure 11 This is a comparison chart of the semantic segmentation results of different models on another environmental image in the MT dataset in the example.

[0061] Figure 12 The image shown is of a physical model of the intelligent following robot that uses the model proposed in this invention in the embodiments.

[0062] Figure 13 The image shown is a semantic segmentation result of the environmental image of the intelligent following robot using the model proposed in this invention in the embodiment. Detailed Implementation

[0063] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but only to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0064] The present invention will now be described in further detail with reference to the accompanying drawings.

[0065] This invention discloses a semantic segmentation method for complex environment images based on a fine-grained augmentation network. This method acquires the environment image to be detected, inputs it into a pre-trained fine-grained augmentation network model, and obtains the semantic segmentation result of the environment image. The fine-grained augmentation network model used in this invention is obtained by optimizing and improving the YOLOv8 network as the basic model framework. Figure 1 As shown, it includes a backbone network layer, a neck network layer, and a detection head. The backbone network layer is used to extract global and local features and perform multi-scale fusion processing on the environmental image input to the feature reconstructing network to obtain a multi-scale feature map. The neck network layer is used to upsample and enhance the details of the multi-scale feature map to obtain an enhanced feature map. The detection head is used to obtain the semantic segmentation result of the environmental image to be detected based on the enhanced feature map, which serves as the output of the fine-grained enhancement network model.

[0066] This invention presents a semantic segmentation method for complex environment images based on fine-grained augmentation networks. Utilizing an optimized fine-grained augmentation network model, the backbone network layer deeply extracts and fuses global and local feature information at multiple scales, enhancing the model's feature extraction capabilities. Then, by employing upsampling and detail enhancement processing in the neck network layer, the model's detail recovery capabilities are improved while effectively reducing image noise. Finally, the semantic segmentation results of the environment image are obtained through a detection head, achieving accurate semantic segmentation for complex environment images.

[0067] The following section provides a more detailed description of the semantic segmentation method for complex environment images based on fine-grained augmentation networks and the fine-grained augmentation network model used in this invention.

[0068] 1 Fine-grained augmentation network model

[0069] Existing models lack fine-grained recognition capabilities in complex environments such as mountains and hills, resulting in room for further improvement in semantic segmentation accuracy. Therefore, this invention proposes a semantic segmentation method based on FENet, employing a pre-trained fine-grained augmentation network model for semantic segmentation of environmental images. The fine-grained augmentation network model used in this invention is obtained by optimizing and improving the YOLOv8 network as its basic model architecture, such as... Figure 1 As shown, it includes a backbone network layer, a neck network layer, and a detection head.

[0070] The backbone network layer consists of sequentially connected convolutional layers, four convolutional fusion units, and one spatial pyramid pooling module; each convolutional fusion unit includes cascaded convolutional layers and a C2f-SSA module.

[0071] The neck network layer consists of two feature dynamic upsampling units and two convolutional attention enhancement units connected in sequence. The feature dynamic upsampling unit consists of a Dysample-ConvBN upsampling module, a connection layer, and a C2f-SSA module connected in sequence. The convolutional attention enhancement unit consists of a convolutional layer, a connection layer, and a C2f-SSA module connected in sequence.

[0072] The detection head is used to obtain semantic segmentation results of the environment image based on the output of the neck network layer.

[0073] Among them, such as Figure 1 As shown, the input of the fine-grained enhancement network model serves as the input of the backbone network layer. Within the backbone network layer, the outputs of the second and third convolutional fusion units also serve as the inputs to the connection layers of the second and first dynamic feature upsampling units in the neck network layer, respectively. In addition to serving as the input of the spatial pyramid pooling module in the backbone network layer to the neck network layer, the output of the spatial pyramid pooling module also serves as the input to the connection layer of the second convolutional attention enhancement unit in the neck network layer. Furthermore, the output of the first dynamic feature upsampling unit in the neck network layer also serves as the input to the connection layer of the first convolutional attention enhancement unit. The outputs of the second dynamic feature upsampling unit and the two convolutional attention enhancement units are all output to the detection head.

[0074] The fine-grained augmentation network model processes the following steps: First, the model is input with a plateau and mountain image. After processing by the Conv module in the Backbone section, it enters the C2f-SSA module to rearrange the number of channels and fuse features at multiple levels, thereby capturing segmentation categories of different sizes and improving the model's feature extraction capability. Then, the SPPF module further fuses multi-scale feature information. Next, the feature map enters the Neck layer, where the Dysample-ConvBN module upsamples the feature map. The upsampling scaling factor controls point sampling to improve the model's detail recovery capability. Then, it is refined layer by layer through the Contact, C2f-SSA, and Conv modules. Finally, in the Head section, the SIoU loss function is used to optimize bounding box regression, improving semantic segmentation accuracy. The final output is the segmentation category and its corresponding confidence score in complex environments such as mountains and hills.

[0075] The fine-grained enhancement network model used in this invention, based on the YOLOv8 network, primarily involves the following optimizations and improvements: First, a C2f-SSA (C2f-SE-ShuffleAttention) module is designed by combining the C2f module with the SSA hybrid attention module. This module is introduced into both the backbone and neck layers. By arranging and calibrating the channels of the feature maps, it deeply extracts and fuses global and local feature information at multiple scales, capturing segmentation categories of different sizes and enhancing the model's feature extraction capabilities. Second, a lightweight Dysample-ConvBN upsampling module (Dynamic Upsmaple) is constructed in the neck layer. Conv-BN precisely controls point sampling operations through dynamically adjusted upsampling scaling factors. Combined with upsampling and convolution operations, it enhances the model's detail recovery capabilities, effectively reduces image noise, and saves computational resources. Furthermore, in training the fine-grained enhancement network model, this invention replaces the original CIoU loss function with the SIoU loss function to further optimize the localization and regression performance of semantic segmentation bounding boxes, which will be discussed later. Ultimately, this achieves more accurate semantic segmentation for images in complex environments.

[0076] Next, we will introduce in detail the main optimizations and improvements in the fine-grained augmentation network model.

[0077] 1.1C2f-SSA Module

[0078] To better extract global and local features from environmental images and fuse them at multiple scales, thereby improving the model's feature extraction capabilities, this invention combines the SE (Squeeze and Excitation) attention mechanism with the SA (Shuffle Attention) attention mechanism to form the SSA (SE and Shuffle Attention) hybrid attention structure. This structure is then connected and integrated into the tail of the C2f module to construct the C2f-SSA (C2f-SE-Shuffle Attention) module.

[0079] The overall module structure of the C2f-SSA module is as follows: Figure 2 As shown, the architecture includes a cascaded first convolutional layer, a splitting module, multiple bottleneck modules, a connection layer, a second convolutional layer, and an SSA hybrid attention module. The two splitting outputs of the splitting module serve as inputs to the first fast convolutional module and the connection layer, respectively. The outputs of each fast convolutional module also serve as inputs to the connection layer. The output of the SSA hybrid attention module serves as the output of the C2f-SSA module. Each bottleneck module performs a second-level convolutional concatenation process, such as... Figure 3 As shown, the processing procedure is as follows: after performing two convolutions on the input feature map of the bottleneck module, it is then superimposed with the input feature map of the bottleneck module to obtain the output feature map of the bottleneck module.

[0080] The processing flow of the C2f-SSA module is as follows: First, the input feature map of the C2f-SSA module passes through a 1×1 first convolutional layer to double the number of channels, thereby enhancing the model's feature representation capability. Then, the convolutional feature map is split into two parts by the Split module. One part is directly passed to the Contact module, while the other part continues to be processed. The further processed feature map passes through multiple bottleneck modules to extract global features and deeper local feature information. The feature maps output by each bottleneck module enter the Contact module, where they are fused and concatenated with the convolutional feature map directly passed from the first convolutional layer. The fused feature map passes through a second convolutional layer for channel compression to reduce computational complexity. Finally, the feature map passes through the SSA hybrid attention module to achieve multi-scale feature fusion of global and local features, ultimately generating an optimized output feature map.

[0081] The SSA hybrid attention module consists of cascaded SE and SA attention modules, combining the SE (Squeeze and Excitation) and SA (Shuffle Attention) attention mechanisms. Its design incorporates group convolution, spatial attention, channel attention, and ShuffleNetV2 principles, thus increasing the network's ability to perceive key information while maintaining low computational cost. Its network structure diagram is shown below. Figure 4 As shown.

[0082] The purpose of the SE (Squeeze and Excitation) attention module is to assign different weights to different locations in the image from the perspective of the channel domain using a weight matrix, enabling the network to acquire more important feature information. Its processing is as follows: Figure 5 As shown: First, the input feature map of the SE attention module is processed by a convolutional layer. Then, the convolution result is processed by global average pooling and fully connected mapping in sequence. The fully connected mapping result is multiplied by the previous convolution result to obtain the output feature map.

[0083] Specifically, the input features to the SE attention module are of size C'×W'×H'. Figure X First, it will go through a convolution operation F. tr Generate a feature map U of size C×W×H; then perform global average pooling F on the feature map U. sq This generates a 1×1×C vector, allowing each channel to be represented by a numerical value; then, by adjusting the fully connected layer F... ex The weights W are mapped to generate suitable weights; finally, the generated suitable weights are used to multiply the feature map U and assign weights F. scale To obtain the final output features Figure X Its size is consistent with the feature map U.

[0084] The SA (Shuffle Attention) attention module further processes the output feature map of the SE attention module, such as... Figure 4 The processing procedure in the SA attention module is as follows:

[0085] First, channel grouping is performed on the output feature map of the SE attention module to obtain several channel group feature maps;

[0086] Then, for each channel group feature map, it is further divided into two sub-feature maps for channel attention and spatial attention operations respectively. In the channel attention operation branch, the sub-feature map is processed by average pooling, fully connected mapping, and sigmoid activation function in sequence, and then multiplied with the sub-feature map entering the branch to obtain the channel attention feature map. In the spatial attention operation branch, the sub-feature map is processed by group normalization, fully connected mapping, and sigmoid activation function in sequence, and then multiplied with the sub-feature map entering the branch to obtain the spatial attention feature map. Finally, the channel attention feature map and the spatial attention feature map are merged by element-wise multiplication to obtain the attention feature sub-map of the channel group.

[0087] Finally, the attention feature sub-maps of each channel group are rearranged and stitched together to obtain the attention feature map as the output.

[0088] Specifically, in the processing of the SSA hybrid attention module, such as Figure 4 As shown, first input features Figure X After introducing the SE attention module, the system proceeds to the SA attention module for grouping operations, dividing the C channels into g groups. Each channel group feature map contains C / g channels. Next, each channel group feature map is further divided into two sub-feature maps, each containing C / 2g channels. These two sub-feature maps are then processed by an average pooling layer (F...). gp The processing is done by the normalization layer (GN) and then each is processed by a fully connected layer (F). c After mapping, weights are generated by the Sigmoid activation function and then multiplied by the sub-feature maps of their respective branches to obtain channel attention feature maps and spatial attention feature maps, respectively. Subsequently, the channel attention feature maps and spatial attention feature maps output by these two branches are merged by element-wise multiplication to mix features from different groups to increase information flow across channels, resulting in the attention feature sub-map of that channel group. Finally, the attention feature sub-maps of all channel groups are merged, and the channel shuffle operator is used to reorder the channel groups to enable information communication between different sub-features, generating the final output attention feature map S.

[0089] This design allows the network to more effectively utilize information from different channels, enhancing the expressiveness of features, which is particularly important when handling visual tasks with complex backgrounds and variable objects. The global and local attention mechanisms of the SSA hybrid attention module provide broader contextual information, helping the model generalize better to new or unseen data and improving its stability and reliability in practical applications. Although it adds some computational steps, its optimized design (such as the relatively low computational cost of channel shuffle) does not significantly increase the overall computational burden of the model, allowing it to maintain good performance even in resource-constrained environments.

[0090] During feature extraction, the complex types and distribution of terrain features in mountainous and hilly environments can negatively impact segmentation accuracy. To further enhance the model's segmentation capabilities in complex outdoor scenes, this invention integrates SSA (Special Attention Module) into the output of the C2f module, designing a C2f-SSA module. This helps the model better focus on important regions and features in the image, forming a strategy to strengthen feature processing capabilities. This integration method improves network accuracy while maintaining efficiency in mountainous scene segmentation. In the design of the C2f-SSA module, the SSA hybrid attention module is applied to the output feature map, enabling recalibration and optimization of feature map channels at both global and local levels, thereby achieving higher-level information fusion within the deep network structure. In summary, the introduction of the C2f-SSA module not only adds powerful feature processing capabilities but also optimizes the overall model performance and efficiency, providing strong technical support for handling highly complex visual tasks.

[0091] 1.2 Dysample-ConvBN Upsampling Module

[0092] To enhance the model's detail recovery capabilities and achieve real-time semantic segmentation of environmental images, a Dysample-ConvBN upsampling module is constructed by combining the Conv module with the Dysample upsampling structure. The designed Dysample-ConvBN module, as a lightweight dynamic upsampling module, defines upsampling from a point-sampling perspective. It precisely controls the point-sampling operation through a dynamically adjusted upsampling scaling factor. This enhances the model's detail recovery capabilities, effectively reduces image noise, and saves computational resources, enabling faster and more accurate segmentation of different object categories in complex environmental images such as mountains and hills.

[0093] Because the Dysample upsampling module primarily relies on pixel-level interpolation and offsetting, and does not incorporate additional feature extraction or optimization steps when processing image details, the output image may still retain noise or artifacts. This module introduces a ConvBN module at the output stage of dynamic upsampling for further convolutional normalization to enhance image quality. The structure of the ConvBN module is as follows: Figure 6 As shown, it includes a two-dimensional convolutional layer, a two-dimensional batch normalization layer, and a SiLU activation function layer connected in sequence. The architecture of the Dysample-ConvBN upsampling module is as follows: Figure 7 As shown, it includes a cascaded Dysample upsampling module and a ConvBN module. The ConvBN module extracts local features from the image output after upsampling by the Dysample upsampling module, which helps the model learn complex patterns and further improves the ability to restore details and the denoising effect.

[0094] like Figure 7 As shown, the main process of the Dysample-ConvBN upsampling module is as follows:

[0095] The Dysample upsampling module first processes the input features Figure X A sampling set δ is created using a sampling point generator, and then the sampling point positions in the sampling set are used to input features via the grid sampling function grid_Sample. Figure X Resampling is performed to obtain the upsampled feature X. g .

[0096] In the Dysample upsampling module, given an upsampling scaling factor s and input features of size C×H×W... Figure X The sampling point generator processes the input features. Figure X Perform bilinear interpolation using an input channel of C and an output channel of 2gs. 2 A linear layer, generating a size of 2gs 2 The offset O of ×H×W is controlled by a static or dynamic range factor. Then, the offset O is reshaped to 2g×sH×sW through pixel transformation. The offset O is then added to the original sampling grid G ​​to obtain the sampling set δ. Finally, the grid sampling function grid_Sample uses the sampling point positions in the sampling set δ to process the input features. Figure X Resampling generates an upsampled result X of size C×sH×sW. g The processing expression of the Dysample upsampling module can be represented as:

[0097] X g =grid_sample(X,δ).

[0098] The control process of static range factor and dynamic range factor in the Dysample upsampling module is as follows: Figure 8 As shown.

[0099] Using a static range factor to control the offset avoids the influence of s 2 The positional relationship between adjacent points results in an uneven distribution of initial sampling locations, leading to the problem of unnecessary points participating in upsampling. For example... Figure 8 As shown in (a), the initial position is changed to bilinear initialization by using DySample upsampling, and the offset is multiplied by a correction factor of 0.25, thereby avoiding output artifacts caused by stepwise transmission errors and satisfying the theoretical boundary conditions for overlap and non-overlap. The offset can then be expressed by the following formula:

[0100] O = 0.25linear(X).

[0101] DySample upsampling generates a dynamic range factor point-by-point by linearly projecting input features, such as... Figure 8 As shown in (b), by introducing the sigmoid function and a static factor of 0.5, the dynamic scope is controlled within the range of [0, 0.5], further enhancing the flexibility of the offset. The offset can then be expressed by the following formula:

[0102] O=0.5sigmoid(linear1(X))·linear2(X).

[0103] The point sampling set δ is the sum of the offset O and the original sampling grid G, as shown in the following equation:

[0104] δ=O+G.

[0105] Compared to CARAFE, which reconstructs input features by generating content-aware upsampling kernels, DySample upsampling performs upsampling by generating sampling point locations. It collects one point at each location and divides it into s... 2 For each sampling point, as long as s 2 If multiple sampling points can be dynamically segmented, then the entire upsampling process only needs to collect one point. According to kernel-based sampling standards, DySample upsampling uses a 2×2 bilinear kernel, while CARAFE upsampling uses a 5×5 bilinear kernel, meaning CARAFE upsampling has a significantly higher kernel count than DySample. Furthermore, the kernel weights in CARAFE upsampling are learned and require K×K channel feature maps, while the kernel weights in DySample upsampling are based on x and y positions, requiring only two channel feature maps. Therefore, CARAFE upsampling has a more complex internal structure and slower inference speed, while DySample upsampling is more efficient.

[0106] Finally, the ConvBN module is used to further upsample the result X output by the Dysample upsampling module. g Local feature extraction is performed to obtain the upsampled features output by the Dysample-ConvBN upsampling module. Figure X ′.

[0107] 1.3 SPPF Spatial Pyramid Pooling Module

[0108] The structure of the SPPF spatial pyramid pooling module is as follows: Figure 9 As shown, it includes a cascaded first convolutional layer, three two-dimensional max pooling layers, a connection layer, and a second convolutional layer. The outputs of the first convolutional layer and the three two-dimensional max pooling layers are also used as inputs to the connection layer, and the output of the second convolutional layer is used as the output of the spatial pyramid pooling module.

[0109] The SPPF (Spatial Pyramid Pooling Fast) module acquires multi-scale feature information through pooling operations at different scales. This multi-scale feature representation helps the model better capture the shape, size, and positional information of the target. Compared to the traditional SPP (Spatial Pyramid Pooling), which applies pooling kernels of different sizes in parallel to achieve multi-scale feature extraction, the SPPF module reduces computation by applying pooling kernels of the same size serially, effectively improving the model's running speed while maintaining feature extraction capabilities.

[0110] 2. Training of Fine-Grained Augmentation Network Model

[0111] In this invention, the fine-grained augmentation network model is trained in the following manner:

[0112] S101: Prepare an environmental sample image dataset, which is divided into a training image dataset and a test image dataset. The environmental sample images in the environmental sample image dataset have all been pre-segmented and labeled with semantic segmentation location and category.

[0113] S102: Input the training image dataset into the fine-grained augmentation network model for training, and optimize the parameters of the fine-grained augmentation network model with the goal of minimizing the loss function until the fine-grained augmentation network model converges, thus obtaining the trained fine-grained augmentation network model.

[0114] S103: Test the fine-grained augmentation network model using the test image dataset to confirm the semantic segmentation performance of the trained fine-grained augmentation network model; if the semantic segmentation performance meets the requirements, end the training of the fine-grained augmentation network model; otherwise, return to step S102.

[0115] Among the loss functions used in the training process, SIoU loss function is selected for localization loss and binary cross-entropy (BCE) loss function is selected for classification loss, which helps to balance the convergence and regression speed and training accuracy of model training.

[0116] In semantic segmentation tasks, the loss function is a crucial parameter in deep learning regression, serving as a measure of the accuracy of the expected results. Its primary function is to help optimize the model, enabling it to accurately locate the target and improve the accuracy of semantic segmentation. The loss functions used in YOLOv8 network models for image semantic segmentation applications include localization loss and classification loss. Localization loss describes the positional deviation between the model's predicted semantic segmentation bounding box and the labeled semantic segmentation target box in the sample image. Classification loss describes the classification deviation between the model's predicted object category and the labeled semantic segmentation target object category in the sample image. During training, the model adjusts its parameters by minimizing the loss value to make the semantic segmentation prediction result closer to the labeled target result in the sample image. In YOLOv8 network models, the CIoU loss function is typically chosen for localization loss, and the binary cross-entropy (BCE) loss function is typically chosen for classification loss.

[0117] In the training of the fine-grained augmented network model of this invention, the binary cross-entropy (BCE) loss function is used for classification loss, but the SIoU loss function is used instead of the CIoU loss for localization loss. While the CIoU loss introduces aspect ratio regression based on the DIoU loss, the introduced aspect ratio has certain irrationalities and fails to consider the regression direction in each iteration, leading to slower convergence and lower efficiency in model training. Therefore, this invention chooses the SIoU loss function instead of the CIoU loss function for semantic segmentation and localization to avoid the aforementioned problems.

[0118] SIoU loss is a loss function used for object detection and semantic segmentation tasks, consisting of three parts: angle cost, distance cost, and shape cost. Specifically, the expression for the SIoU loss function is:

[0119]

[0120] Among them, Loss SIoU SIoU represents the loss; IoU represents the intersection-union ratio loss between the semantic segmentation prediction box and the semantic segmentation target box; Δ is the distance loss between the semantic segmentation prediction box and the semantic segmentation target box; Ω is the shape loss between the semantic segmentation prediction box and the semantic segmentation target box. At the same time, the distance loss Δ is also related to the angle loss Λ.

[0121] Specifically, the angle loss Λ is:

[0122]

[0123] In the formula, These are the x-coordinate and y-coordinate of the center point of the semantic segmentation prediction bounding box, respectively. These are the x-coordinate and y-coordinate of the center point of the semantic segmentation bounding box, respectively; σ c c is the pixel distance between the center points of the semantic segmentation target bounding box and the semantic segmentation prediction bounding box. h This represents the difference in pixel y-coordinates between the semantic segmentation target bounding box and the semantic segmentation prediction bounding box.

[0124] The distance loss Δ is:

[0125]

[0126] In the formula, s w s h These are the pixel width and pixel height of the minimum bounding rectangles of the semantic segmentation prediction bounding box and the semantic segmentation target bounding box, respectively. If we let It can be seen that when α approaches 0, the distance loss is small, and when α equals π / 4, the distance loss reaches its maximum.

[0127] The shape loss Ω is:

[0128]

[0129] Where w and h are the pixel width and pixel height of the semantic segmentation prediction box, respectively; w gt h gt Here, θ represents the pixel width and pixel height of the semantic segmentation bounding box, respectively; θ is the shape attention coefficient. The shape attention coefficient θ is typically set between 2 and 6. A smaller value indicates that the SIoU loss function prioritizes shape loss. If the shape attention coefficient θ is set to 1, the SIoU loss function will prioritize shape loss, thus reducing its focus on distance and angle losses. Therefore, a shape attention coefficient θ greater than 1 is generally preferred. Research analysis shows that setting the shape attention coefficient θ to 4 is the optimal value for best results.

[0130] 3 Examples

[0131] In this embodiment, a dataset is used to test and verify the method of the present invention and other methods to better demonstrate the technical advantages of the method of the present invention. This will be described in detail below.

[0132] 3.1 Experimental Dataset

[0133] To verify the performance of the fine-grained enhancement network model proposed in this invention as a semantic segmentation neural network model in complex environmental image semantic segmentation scenarios, this experiment used a self-built mountain dataset (MT). This dataset consists of 2854 images taken by a Realsense D455 camera at a test site in Chongqing from different angles, lighting conditions, and distances. To enhance the diversity of the dataset, image enhancement operations such as random rotation, random cropping, and random sharpening were further performed, ultimately expanding the dataset to 6204 images. Labelme was used for labeling, covering six categories. The dataset was divided into training, test, and validation sets in an 8:1:1 ratio, resulting in a training set of 4963 images, a test set of 621 images, and a validation set of 620 images. This dataset contains diverse road types and natural environments, making it applicable to the varied conditions in mountainous scenes and helping to evaluate the algorithm's performance in complex environments such as mountains and hills. Using this dataset, the experiment can preliminarily evaluate the performance of the proposed semantic segmentation neural network model in complex and diverse environments.

[0134] 3.2 Experimental Environment

[0135] The experimental platform configuration is shown in Table 1. In training the semantic segmentation neural network model, all models were trained without pre-trained weight files. The initial learning rate (lr0) was 0.01, the number of early stopping epochs (patience) was 50, the batch size was 30, the number of worker threads (works) was 8, the input image size (imgsz) was (640, 640), the momentum was 0.937, the weight decay was 0.0005, the number of warmup epochs (warmup_epochs) was 3.0, the initial momentum (warmup_momentum) of the warmup phase was 0.8, the initial bias learning rate (warmup_bias_lr) of the warmup phase was 0.1, and the training epochs were 300.

[0136] Table 1. Experimental Platform Configuration Table

[0137]

[0138] 3.3 Results Analysis

[0139] 3.3.1 Quantitative Analysis

[0140] (1) Comparative Experiment

[0141] To quantitatively analyze the semantic segmentation performance of the fine-grained augmentation network model proposed in this invention, comparative experiments were conducted on the MT dataset using YOLOv5n-seg, YOLOv7-seg, YOLOv8n-seg, YOLOv8s-seg, and YOLOv9t-seg models with the proposed model, while maintaining consistency in all experimental parameters. Precision, F1 score, mAP50, mAP50-95, parameters, and GFLOPs were used to quantitatively evaluate all models. The experimental results are shown in Table 2.

[0142] Table 2 Comparison of experimental results

[0143]

[0144] As shown in Table 2, the model proposed in this invention outperforms the YOLOv5n-seg and YOLOv9t-seg models. Specifically, the model proposed in this invention, while being more lightweight and efficient, significantly outperforms the YOLOv5n-seg and YOLOv9t-seg models in terms of Precision, F1 score, mAP50, and mAP50-95. Specifically, Precision is improved by 4.51% and 1.65%, F1 score by 1.3% and 0.7%, mAP50 by 0.96% and 0.43%, and mAP50-95 by 2.51% and 6.75%, respectively. This significant performance improvement indicates that the model proposed in this invention has superior capabilities in feature extraction and segmentation accuracy for mountainous data.

[0145] Although the original YOLOv8n-seg model has slightly fewer parameters and GFLOPs than the model proposed in this invention, its relatively simple original structure results in significantly inferior segmentation performance. Compared to YOLOv8n-seg, the precision, F1 score, mAP50, and mAP50-95 of the proposed model are improved by 1.11%, 1.04%, 2.17%, and 5.26%, respectively. This further highlights the superiority of the proposed model in terms of architecture design and training strategy, proving that the proposed model can effectively improve detection accuracy and segmentation quality while maintaining a lightweight design, thereby achieving more accurate segmentation.

[0146] Furthermore, despite significant improvements in parameters and GFLOPs, the YOLOv8s-seg model achieves overall performance comparable to the model proposed in this invention. Specifically, YOLOv8s-seg slightly outperforms the model proposed in this invention in F1 and mAP50-95, but its precision and mAP50 are inferior. This indicates that although YOLOv8s-seg has higher complexity, it does not significantly surpass the model proposed in this invention in specific metrics.

[0147] Compared to the model proposed in this invention, the YOLOv7-seg model significantly increases parameters by 34,000,000 and GFLOPs by 130, respectively. However, its F1 score, mAP50, and mAP50-95 only improve by 3.98%, 3.96%, and 3.88%, respectively, and its precision is still inferior to the model proposed in this invention. Overall, the model proposed in this invention not only maintains high-precision segmentation while keeping reasonable computational overhead and parameter count, but also surpasses more complex network models in several key performance indicators, making it well-suited for mountain segmentation scenarios.

[0148] Therefore, the model proposed in this invention precisely controls the point sampling operation by dynamically adjusting the upsampling scale factor through the Dysmaple-conv module, which enhances the model's detail recovery capability and effectively reduces image noise; and improves the model's feature extraction capability by realizing multi-level fusion of global and local features through the C2f-SSA module.

[0149] (2) Ablation test

[0150] To further verify the effectiveness of the improvements made to the proposed model and enhance its robustness, an ablation experiment was designed for verification. Using YOLOv8n-seg as the baseline model, the training epochs were all 300 rounds. The Dysmple-conv module, C2f-SSA module, and SIoU loss function were sequentially stacked step-by-step. The improved model was then trained and tested, and the ablation experiment results are shown in Table 3.

[0151] Table 3. Results of ablation experiments

[0152]

[0153] As shown in Table 3, under the same experimental conditions, each iteration of the baseline model improved the evaluation metrics Precision, F1 score, mAP50, and mAP50-95 to varying degrees, proving the effectiveness of the model proposed in this invention.

[0154] Experiment 1 used YOLOv8n-seg as the baseline model. Experiment 2 designed the Dysample-ConvBN module, which aims to improve the model's detail recovery capability through dynamic upsampling. Compared with Experiment 1, Experiment 2 improved the precision, F1 score, mAP50, and mAP50-95 by 0.11%, 0.23%, 1.05%, and 1%, respectively, indicating the module's contribution to improving model accuracy. Experiment 3 further added the C2f-SSA module based on Experiment 2, enhancing the model's global and local performance. The feature extraction capability increased the model's precision, mAP50, and mAP50-95 by 0.37%, 0.2%, and 2.69%, respectively. Experiment ④ presents the experimental results of the model proposed in this invention. Based on the superposition of the Dysmaple-conv and C2f-SSA modules, the SIoU loss function was further introduced to more accurately locate the target. Compared with Experiment ①, these four evaluation indicators improved by 1.11%, 1.04%, 2.17%, and 5.26%, respectively, further demonstrating the effectiveness of the method proposed in this invention.

[0155] Experiment ② designed the Dysample-ConvBN module, which dynamically adjusts the upsampling ratio factor from the perspective of point sampling to precisely control the point sampling operation and enhance the model's detail recovery ability. The convolution operation at the output end learns complex features, further improving the image clarity and detail, while smoothing the input image, thereby improving the overall performance of the model. Experiment ③, based on the use of lightweight dynamic upsampling, superimposed the C2f-SSA module. Under the action of the SE attention mechanism and the Shuffle Attention mechanism, the network's ability to perceive key information is improved, enabling it to integrate a wider range of contextual information and make full use of the correlation information between channels. This design effectively recalibrates and optimizes the channels of the feature map at the global and local levels, thereby achieving higher-level information fusion in deep networks and significantly improving the model's accuracy. Experiment ④, based on the previous work, introduced the SIoU loss function with a reasonable aspect ratio, which accelerated the model's convergence speed and improved the model's convergence efficiency. Multi-scale targets existing in complex environments such as mountains and hills can also be better measured by the multi-scale factor introduced by SIoU. Therefore, the model proposed in this invention fully considers the segmentation characteristics of the network in complex mountainous environments through the Dysample-ConvBN module, C2f-SSA module, and SIoU loss function. This allows for better extraction and transmission of detailed and semantic information within the model, significantly improving its performance in detail recovery, feature extraction, and convergence efficiency. This improvement not only enhances segmentation accuracy but also meets the application needs of practical engineering.

[0156] 3.3.2 Comparative Analysis of Semantic Segmentation Results

[0157] To verify the excellent semantic segmentation performance of the proposed model, semantic segmentation was compared with that of YOLOv5n-seg, YOLOv8n-seg, and YOLOv9t-seg on the MT dataset. Two environmental images from the dataset were selected, and the semantic segmentation results of each model are shown below. Figure 10 and Figure 11 As shown. In Figure 10 and Figure 11 In the diagram, sub-image (a) shows the YOLOv5n-SEG segmentation result, sub-image (b) shows the YOLOv8n-SEG segmentation result, sub-image (c) shows the YOLOv9t-SEG segmentation result, and sub-image (d) shows the segmentation result of the model proposed in this invention. In sub-image (a), the colors of the mask and detection boxes are dark blue, pink, white, blue, cyan, and green, corresponding to the six categories: sky, vegetation, road, obstacle, sign, and pedestrian. In sub-images (b), (c), and (d), the colors of the mask and detection boxes are yellow, green, brown, red, orange, and pink, also corresponding to the six categories: sky, vegetation, road, obstacle, sign, and pedestrian. The numbers next to the category labels indicate the confidence level of the prediction result. Figure 10 and Figure 11 As can be seen, the model proposed in this invention can effectively segment various categories such as sky, vegetation, and roads in complex scenes. Compared with YOLOv5n-seg, YOLOv8n-seg, and YOLOv9t-seg, the model proposed in this invention shows higher confidence in the segmentation of the above six categories, thus achieving better segmentation results.

[0158] Specifically, the performance of YOLOv5n-seg is somewhat unstable, with significant fluctuations in detection results for some key categories such as "sky" and "obstacles." This indicates insufficient robustness of the model when handling mountainous scenes, and its adaptability to different data needs improvement. The performance of YOLOv8n-seg and YOLOv9t-seg also fluctuates, particularly in the segmentation of the "sky" category, reflecting their limitations in feature extraction and model generalization capabilities, making them relatively weak in handling complex and varied mountainous scenes. In contrast, the model proposed in this invention shows more stable detection results in key categories such as "vegetation" and "roads," with significantly improved accuracy, indicating that the proposed model can accurately segment and identify these important mountainous scene elements. Furthermore, the detection values ​​for the "roads" category repeatedly reach high levels, demonstrating its superior ability in mountainous road segmentation. Moreover, the model proposed in this invention exhibits more balanced performance across categories, avoiding extreme errors in any single category and providing reliable detection results across multiple key categories. In summary, the semantic segmentation neural network model proposed in this invention exhibits higher stability and accuracy when processing mountainous scenes, and can more effectively segment and identify key scene elements. Its balanced overall performance enables it to provide reliable detection results in multiple key categories, demonstrating its wide adaptability and superior performance in practical applications.

[0159] 4. Model Application

[0160] The above experiments show that the model proposed in this invention meets the requirements of practical engineering applications. Therefore, the model proposed in this invention is applied to a four-wheel drive following robot, and the physical prototype of this intelligent following robot is shown below. Figure 12 As shown, it used an NVIDIA Jetson Xavier NX as the experimental device, with Ubuntu 18.04 as the operating system, PyTorch as the deep learning framework, and Python version 3.8.0 as the operating system.

[0161] After the intelligent following robot collects environmental images in a mountainous environment, the semantic segmentation results obtained using the model proposed in this invention are as follows: Figure 13 As shown. Figure 13 In the diagram, the mask and detection boxes are colored yellow, green, brown, red, orange, and pink, corresponding to the six categories: sky, vegetation, road, obstacles, signs, and pedestrians, respectively. The numbers next to the category labels indicate the confidence level of the prediction result. Figure 13As can be seen, the model proposed in this invention exhibits excellent segmentation performance in four-wheel drive following robot applications. The model achieves high confidence levels when processing pixels from different categories, including sky, vegetation, roads, landmarks, and obstacles. Specifically, the segmentation confidence level for "sky" is as high as 0.98, indicating that the model can almost flawlessly identify this background category, providing stable visual information for the robot. The recognition confidence level for "vegetation" is 0.91, ensuring that the robot can accurately determine vegetation distribution in natural environments, thereby making reasonable obstacle avoidance decisions. The segmentation confidence level for "road" is 0.97, a high confidence level meaning that the robot can clearly identify the driving path, providing a solid foundation for stable navigation. Although the confidence levels for "landmarks" and "obstacles" are slightly lower, at 0.59 and 0.52 respectively, these are still within an acceptable range, and with further training and optimization, higher accuracy is expected. Therefore, the model proposed in this invention demonstrates good generalization performance and can be effectively applied to intelligent following robot systems.

[0162] 5. Overview

[0163] This invention presents a semantic segmentation method for complex environment images based on fine-grained augmentation networks. Utilizing an optimized fine-grained augmentation network model, the backbone network layer deeply extracts and fuses global and local feature information at multiple scales, enhancing the model's feature extraction capabilities. Then, by employing upsampling and detail enhancement processing in the neck network layer, the model's detail recovery capabilities are improved while effectively reducing image noise. Finally, the semantic segmentation results of the environment image are obtained through a detection head, achieving accurate semantic segmentation for complex environment images. The fine-grained enhancement network model proposed in this invention combines the C2f module with the SSA hybrid attention module to design a C2f-SSA module, which is introduced into both the backbone and neck layers. By arranging and calibrating the channels of the feature maps, it deeply extracts and fuses global and local feature information at multiple scales, capturing segmentation categories of different sizes and enhancing the model's feature extraction capabilities. Secondly, in the neck layer, a lightweight Dysample-ConvBN upsampling module is constructed. It precisely controls the point sampling operation through dynamically adjusted upsampling scaling factors and combines upsampling and convolution operations to enhance the model's detail recovery capabilities, effectively reduce image noise, and save computational resources. Furthermore, in the training of the fine-grained enhancement network model, this invention replaces the original CIoU loss function with the SIoU loss function to further optimize the localization and regression performance of semantic segmentation bounding boxes. Finally, it achieves more accurate semantic segmentation for complex environment images.

[0164] Experimental results show that the proposed model achieves accuracy, F1 score, mAP50, and mAP50-95 of 92.7%, 87.05%, 88.66%, and 66.59% respectively on the MT dataset, representing improvements of 1.11%, 1.04%, 2.17%, and 5.26% compared to the baseline model YOLOv8n-seg, thus meeting practical engineering needs. Furthermore, the proposed model has been successfully applied to an intelligent following robot, demonstrating its effectiveness in real-world applications.

[0165] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described with reference to preferred embodiments, those skilled in the art should understand that various changes in form and detail can be made without departing from the spirit and scope of the invention as defined in the appended claims.

Claims

1. A semantic segmentation method for complex environment images based on fine-grained enhancement networks, characterized in that, The environment image to be detected is acquired and input into a pre-trained fine-grained augmentation network model to obtain the semantic segmentation result of the environment image to be detected. The fine-grained enhancement network model includes a backbone network layer, a neck network layer, and a detection head. The backbone network layer is used to extract global and local features and perform multi-scale fusion processing on the environmental image input to the feature reconstruction network to obtain a multi-scale feature map. The neck network layer is used to upsample and enhance the details of the multi-scale feature map to obtain an enhanced feature map. The detection head is used to obtain the semantic segmentation result of the environmental image to be detected based on the enhanced feature map, which serves as the output of the fine-grained enhancement network model. In the fine-grained augmentation network model: The backbone network layer includes sequentially connected convolutional layers, four convolutional fusion units, and one spatial pyramid pooling module; each convolutional fusion unit includes cascaded convolutional layers and a C2f-SSA module. The neck network layer includes two feature dynamic upsampling units and two convolutional attention enhancement units connected in sequence; wherein, the feature dynamic upsampling unit includes a Dysample-ConvBN upsampling module, a connection layer and a C2f-SSA module connected in sequence; the convolutional attention enhancement unit includes a convolutional layer, a connection layer and a C2f-SSA module connected in sequence. The detection head is used to obtain the semantic segmentation result of the environmental image based on the output of the neck network layer; Specifically, the input of the fine-grained enhancement network model serves as the input to the backbone network layer. Within the backbone network layer, the outputs of the second and third convolutional fusion units also serve as the inputs to the connection layers of the second and first dynamic feature upsampling units in the neck network layer, respectively. In addition to serving as the input to the neck network layer, the output of the spatial pyramid pooling module in the backbone network layer also serves as the input to the connection layer of the second convolutional attention enhancement unit in the neck network layer. Furthermore, the output of the first dynamic feature upsampling unit in the neck network layer also serves as the input to the connection layer of the first convolutional attention enhancement unit. The outputs of the second dynamic feature upsampling unit and the two convolutional attention enhancement units are all output to the detection head. The C2f-SSA module includes a cascaded first convolutional layer, a splitting module, multiple bottleneck modules, a connection layer, a second convolutional layer, and an SSA hybrid attention module; the SSA hybrid attention module includes a cascaded SE attention module and an SA attention module.

2. The semantic segmentation method for complex environment images based on fine-grained enhancement networks according to claim 1, characterized in that, The processing procedure for each bottleneck module is as follows: after performing two convolutional processes on the input feature map of the bottleneck module, the feature map is then superimposed with the input feature map of the bottleneck module to obtain the output feature map of the bottleneck module.

3. The semantic segmentation method for complex environment images based on fine-grained enhancement networks according to claim 1, characterized in that, The processing procedure of the SE attention module is as follows: First, the input feature map of the SE attention module is processed by a convolutional layer. Then, the convolution result is processed by global average pooling and fully connected mapping in sequence. The fully connected mapping result is multiplied by the previous convolution result to obtain the output feature map. The processing procedure of the SA attention module is as follows: First, channel grouping is performed on the output feature map of the SE attention module to obtain several channel group feature maps; Then, for each channel group feature map, it is further divided into two sub-feature maps for channel attention and spatial attention operations respectively. In the channel attention operation branch, the sub-feature map is processed by average pooling, fully connected mapping, and sigmoid activation function in sequence, and then multiplied with the sub-feature map entering the branch to obtain the channel attention feature map. In the spatial attention operation branch, the sub-feature map is processed by group normalization, fully connected mapping, and sigmoid activation function in sequence, and then multiplied with the sub-feature map entering the branch to obtain the spatial attention feature map. Finally, the channel attention feature map and the spatial attention feature map are merged by element-wise multiplication to obtain the attention feature sub-map of the channel group. Finally, the attention feature sub-maps of each channel group are rearranged and stitched together to obtain the attention feature map as the output.

4. The semantic segmentation method for complex environment images based on fine-grained enhancement networks according to claim 1, characterized in that, The spatial pyramid pooling module includes a cascaded first convolutional layer, three two-dimensional max pooling layers, a connecting layer, and a second convolutional layer. The outputs of the first convolutional layer and the three two-dimensional max pooling layers are also used as inputs to the connecting layer, and the output of the second convolutional layer is used as the output of the spatial pyramid pooling module.

5. The semantic segmentation method for complex environment images based on fine-grained enhancement networks according to claim 1, characterized in that, The Dysample-ConvBN upsampling module includes a cascaded Dysample upsampling module and a ConvBN module; The processing procedure of the Dysample upsampling module is as follows: first, a sampling set is created from the input feature map of the Dysample upsampling module through a sampling point generator; then, the input feature map is resampled using the sampling point positions in the sampling set through a grid sampling function to obtain upsampled features. The ConvBN module includes a two-dimensional convolutional layer, a two-dimensional batch normalization layer, and a SiLU activation function layer connected in sequence. The ConvBN module is used to further upsample the output of the Dysample upsampling module. Local feature extraction is performed to obtain the upsampled feature map, which is the output of the Dysample-ConvBN upsampling module. .

6. The semantic segmentation method for complex environment images based on fine-grained enhancement networks according to claim 1, characterized in that, The fine-grained augmentation network model is trained in the following manner: S101: Prepare an environmental sample image dataset, which is divided into a training image dataset and a test image dataset. The environmental sample images in the environmental sample image dataset have all been pre-segmented and labeled with semantic segmentation location and category. S102: Input the training image dataset into the fine-grained augmentation network model for training, and optimize the parameters of the fine-grained augmentation network model with the goal of minimizing the loss function until the fine-grained augmentation network model converges, thus obtaining the trained fine-grained augmentation network model. S103: Test the fine-grained augmentation network model using a test image dataset to confirm the semantic segmentation performance of the trained fine-grained augmentation network model; if the semantic segmentation performance meets the requirements, then end the training of the fine-grained augmentation network model. Otherwise, return to step S102.

7. The semantic segmentation method for complex environment images based on fine-grained enhancement networks according to claim 6, characterized in that, In training the fine-grained augmentation network model, the SIoU loss function is selected for localization loss, and the binary cross-entropy (BCE) loss function is selected for classification loss.

8. The semantic segmentation method for complex environment images based on fine-grained enhancement networks according to claim 7, characterized in that, The SIoU loss function is calculated using the following formula: ； in, Indicates SIoU loss; The intersection-union loss between the semantic segmentation predicted bounding box and the semantic segmentation target bounding box is represented. The distance loss is calculated between the semantic segmentation predicted bounding box and the semantic segmentation target bounding box. The shape loss is used for the semantic segmentation predicted bounding box and the semantic segmentation target bounding box; The shape loss for: ； ， ； in, , These are the pixel width and pixel height of the semantic segmentation prediction bounding box, respectively; , These are the pixel width and pixel height of the semantic segmentation target bounding box, respectively. It is a natural constant; Shape attention coefficient; The distance loss for: ； ， ， ； in, , These are the x-coordinate and y-coordinate of the center point of the semantic segmentation prediction bounding box, respectively. , These are the pixel x-coordinates and pixel y-coordinates of the center point of the semantic segmentation bounding box, respectively. , These are the pixel width and pixel height of the minimum bounding rectangles of the semantic segmentation prediction bounding box and the semantic segmentation target bounding box, respectively. The angular loss between the semantic segmentation predicted bounding box and the semantic segmentation target bounding box: ； ， ； in, This is the pixel distance between the center points of the semantic segmentation target bounding box and the semantic segmentation prediction bounding box; This represents the difference in pixel y-coordinates between the semantic segmentation target bounding box and the semantic segmentation prediction bounding box.

Images