Remote sensing image ground object segmentation method and system based on YOLOv8-seg-GCHA, and electronic equipment

Abstract

The invention provides a remote sensing image ground object segmentation method and system based on YOLOv8-seg-GCHA, and electronic equipment. The method comprises the following steps: acquiring a remote sensing image, performing ground feature labeling on the acquired image, and generating a ground feature segmentation data set; the method comprises the following steps: constructing a YOLOv8-seg-GCHA network structure; according to the YOLOv8-seg-GCHA network structure, a GCHA module and a normalized depth separable convolution DSCBS module are introduced into YOLOv8-seg, and the GCHA module and the normalized depth separable convolution DSCBS module are introduced into the YOLOv8-seg. Wherein the GCHA module is used for identifying global information and local details of an image; using the surface feature segmentation data set to train the YOLOv8-seg-GCHA network structure, and obtaining a surface feature segmentation model; and inputting a remote sensing image of a target area into the ground object segmentation model to obtain a ground object segmentation result of the target area. By applying the technical scheme provided by the invention, the features can be decoded and classified more accurately, the capability of analyzing the boundary and internal details of an actual ground object by a network is improved, the segmentation precision is improved, and meanwhile, the loss of computing resources is reduced.

Description

Technical Field

[0001] The present invention relates to the field of image processing and vector map updating, and in particular to a remote sensing image ground object segmentation method, system and electronic equipment based on YOLOv8-seg-GCHA. Background Art

[0002] Remote sensing image object segmentation analyzes high-resolution remote sensing images to identify and segment different types of objects in the image. Traditional segmentation methods include threshold segmentation, region growing, edge segmentation, etc. Deep learning-based segmentation methods mainly use U-Net, YOLOv8-seg and other networks for segmentation.

[0003] The threshold segmentation method uses the grayscale information of the image for segmentation. Since it ignores the spatial relationship between pixels, it is not effective for images with complex grayscale value distribution or a lot of overlap, and it is difficult to find a suitable threshold to divide different objects in high-resolution remote sensing images. The region growing method selects typical feature pixels as seed points and merges adjacent pixels with similar attributes (such as color and texture) to the seed points into the seed area, but inappropriate seed point selection may lead to segmentation errors and is too random. The edge segmentation method divides the image into different regions based on the edge (where features such as brightness or color in the image mutate), but the edge detection algorithm is sensitive to noise and cannot accurately locate the real boundary when the edge is blurred or the contrast between the edge and the background is not high. The segmentation effect is not ideal when there is a feature mutation within a single instance. The U-Net network is suitable for images with few object categories and simple background information (such as medical images), while there are many types of small target objects in remote sensing images and complex background information, resulting in low segmentation accuracy and efficiency.

[0004] YOLOv8-seg is a network developed on the basis of YOLO specifically for image segmentation tasks. It performs outstandingly in the case of multiple targets and complex backgrounds, and provides five networks of different sizes (x, l, m, s, n), with decreasing computational complexity and accuracy. YOLOv8-seg still faces many challenges when processing complex remote sensing images: the segmentation results of remote sensing images are easily disturbed by complex backgrounds and noise, such as lighting, clouds, shadows, scattering, etc.; when processing single instances with inconsistent features such as color or texture, it is difficult for the training model to capture enough global information and local details, resulting in the inability to accurately segment the complete instance, and even erroneously segmenting a single object into multiple parts. Therefore, the model needs to better understand the spatial information of the image and the changes within the object to improve segmentation accuracy. Summary of the invention

[0005] In order to solve the problem that the existing deep learning-based segmentation method is easily disturbed and difficult to capture sufficient global information and local details, resulting in low segmentation accuracy, the present invention proposes a remote sensing image ground object segmentation method, system and electronic equipment based on YOLOv8-seg-GCHA.

[0006] The present invention provides a remote sensing image ground object segmentation method based on YOLOv8-seg-GCHA, comprising:

[0007] Step 1: Obtain remote sensing images, annotate the acquired images, and generate object segmentation datasets;

[0008] Step 2: construct a YOLOv8-seg-GCHA network structure; the YOLOv8-seg-GCHA network structure introduces a GCHA module and a normalized depth separable convolution DSCBS module into YOLOv8-seg; wherein the GCHA module is used to identify global information and local details of the image;

[0009] Step 3: Use the object segmentation dataset to train the YOLOv8-seg-GCHA network structure to obtain a object segmentation model;

[0010] Step 4: Input the remote sensing image of the target area into the ground object segmentation model to obtain the ground object segmentation result of the target area.

[0011] Further, the GCHA module in step 2 includes a global context module GCM and a hybrid attention module HAM;

[0012] The feature map output by the global context module GCM is directly used as the input of the hybrid attention module HAM, and the final feature map is obtained by processing by the hybrid attention module HAM;

[0013] Wherein, the global context module GCM integrates the global information of the image;

[0014] The hybrid attention module HAM specifically includes: a self-attention unit, a spatial attention unit and a channel attention unit; the self-attention unit is used to provide global information to the channel attention unit and the spatial attention unit respectively, the channel attention unit is used to process the global information to generate a channel attention weight map, and the spatial attention unit is used to process the global information to generate a spatial attention map.

[0015] Furthermore, the spatial attention unit is specifically used to restore the output of the self-attention unit to the original spatial dimension, perform maximum pooling and average pooling respectively, concatenate the two pooling results obtained in the channel dimension, and generate a spatial attention map through convolution and Sigmoid activation function;

[0016] The channel attention unit is specifically used to restore the output of the self-attention unit to the original spatial dimension, use adaptive average pooling to compress the spatial information of each channel to a single value, reduce the number of channels through convolution, use ReLU activation function to increase nonlinearity, and then use convolution to restore the number of channels. Finally, a weight factor for each channel is generated by a Sigmoid function to generate a channel attention weight map;

[0017] Correspondingly, the final feature map is obtained by processing with the hybrid attention module HAM, specifically including expanding the channel attention weight map and the spatial attention map to the original dimension and performing element-by-element multiplication operation with the output of the self-attention to form the final feature map.

[0018] Furthermore, the introduction of the GCHA module in step 2 specifically includes: introducing a GCHA module after the SPPF module of the YOLOv8-seg network structure, and introducing a GCHA module before each of the three segmentation heads.

[0019] Furthermore, the introduction of the normalized depthwise separable convolutional DSCBS module in step 2 specifically includes replacing the CBS convolutional module after the first C2F module of the backbone network with the normalized depthwise separable convolutional DSCBS module.

[0020] Furthermore, the normalized depthwise separable convolution DSCBS module specifically includes a depthwise separable convolution DSC module unit, a two-dimensional batch normalization BatchNorm2d unit and a SiLU function unit which are connected in sequence.

[0021] Furthermore, the loss function L is used in step 3 total The YOLOv8-seg-GCHA network structure is trained; wherein the loss function L total By the classification loss L cls , bounding box regression loss L bbox and segmentation loss L seg The calculation formula is as follows:

[0022] L total =λ cls L cls +λ bbox L bbox +λ seg L seg

[0023] where λ cls ,λ bbox and λ seg is a hyperparameter.

[0024] Furthermore, step 4 also includes correcting the object segmentation result, specifically including removing the prediction box through NMS, generating an instance segmentation mask using the prototype mask and the mask coefficient, and post-processing the instance segmentation mask through threshold processing and morphological operations to obtain the final segmentation result.

[0025] On the other hand, the present invention provides a remote sensing image ground object segmentation system based on YOLOv8-seg-GCHA, comprising:

[0026] The data acquisition module is used to acquire remote sensing images, annotate the acquired images with ground objects, and generate ground object segmentation data sets;

[0027] A network structure building module, used to build a YOLOv8-seg-GCHA network structure; the YOLOv8-seg-GCHA network structure introduces a GCHA module and a normalized depth separable convolution DSCBS module into YOLOv8-seg; wherein the GCHA module is used to identify global information and local details of an image;

[0028] A network structure training module, used to train the YOLOv8-seg-GCHA network structure using the land object segmentation data set to obtain a land object segmentation model;

[0029] The target data processing module is used to input the remote sensing image of the target area into the ground object segmentation model to obtain the ground object segmentation result of the target area.

[0030] On the other hand, the present invention provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method described in any one of the remote sensing image object segmentation methods based on YOLOv8-seg-GCHA is implemented.

[0031] Beneficial effects of the present invention:

[0032] The GCHA module proposed in the present invention can more effectively extract image features globally and locally by combining the global context module GCM and the hybrid attention module HAM. GCM uses the global information of the image to more accurately locate and identify various objects, which is conducive to distinguishing similar objects and accurately segmenting objects in complex scenes. HAM combines self-attention, channel attention and spatial attention. Self-attention captures long-distance dependencies, while spatial and channel attention are sensitive to local features and can effectively cope with color and texture changes inside objects. The combination of the two can more effectively capture global information and local details in the image.

[0033] The present invention replaces some traditional convolutions with depthwise separable convolutions, which reduces the number of parameters and computational complexity, improves computational efficiency, and enables YOLOv8-seg-GCHA to run effectively even in resource-limited environments. It helps prevent overfitting when there are few training samples.

[0034] The GCM of the present invention assists in the precise positioning and identification of objects by integrating the global information of the image. The HAM is more sensitive to the key features in the image, which enhances the accuracy and robustness of the model in processing color and texture changes, while improving the ability to resist noise, so that the network maintains a stable segmentation effect in a changing environment.

[0035] The present invention provides network versions of different sizes (x, l, m, s, n), so that the model can be flexibly deployed in different hardware and application scenarios to meet diverse applications ranging from high precision to low latency requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 A flow chart of a remote sensing image object segmentation method based on YOLOv8-seg-GCHA provided in an embodiment of the present invention;

[0037] Figure 2 The overall flow chart provided for the embodiment of the present invention

[0038] Figure 3 A structural diagram of the GCHA provided in an embodiment of the present invention;

[0039] Figure 4 A YOLOv8-seg-GCHA network structure diagram provided for an embodiment of the present invention;

[0040] Figure 5 A DSCBS structure diagram provided for an embodiment of the present invention;

[0041] Figure 6 The YOLOv8-seg network segmentation result provided by the embodiment of the present invention;

[0042] Figure 7 The YOLOv8-seg-GCHA network segmentation result provided by the embodiment of the present invention;

[0043] Figure 8 A schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution in the embodiment of the present invention will be clearly described below in conjunction with the drawings in the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] The present invention is further explained below with reference to the accompanying drawings and specific embodiments:

[0046] like Figure 1 As shown, the embodiment of the present invention provides a remote sensing image ground object segmentation method based on YOLOv8-seg-GCHA, comprising the following steps:

[0047] S101: Acquire a remote sensing image, annotate the acquired image with ground objects, and generate a ground object segmentation data set;

[0048] Specifically, ArcGIS software is used to annotate land features such as water systems, roads, buildings, cultivated land and forest land in remote sensing images to form a high-quality land feature segmentation dataset.

[0049] S102: construct a YOLOv8-seg-GCHA network structure; the YOLOv8-seg-GCHA network structure introduces a GCHA module and a normalized depth separable convolution DSCBS module into YOLOv8-seg; wherein the GCHA module is used to identify global information and local details of the image;

[0050] Specifically, YOLOv8-seg is the latest model for instance segmentation in the YOLO series, which adds pixel-level segmentation capabilities to the target based on target detection. YOLOv8-seg has 5 pre-trained models of different sizes, namely n, s, m, l and x, among which YOLOv8n-seg is the smallest model with fast speed, while YOLOv8x-seg is the most accurate model but relatively slow. The embodiments of the present invention are mainly explained based on YOLOv8l-seg.

[0051] Specifically, the GCHA module proposed in the embodiment of the present invention can more effectively extract image features globally and locally by combining the global context module GCM and the hybrid attention module HAM. GCM uses the global information of the image to more accurately locate and identify various objects, which is conducive to distinguishing similar objects and accurately segmenting objects in complex scenes. HAM combines self-attention, channel attention and spatial attention. Self-attention captures long-distance dependencies, while spatial and channel attention are sensitive to local features, which enhances the accuracy and robustness of the model when dealing with color and texture changes, while improving the ability to resist noise, so that the network maintains a stable segmentation effect in a changing environment. The combination of the two can more effectively capture global information and local details in the image.

[0052] Specifically, the embodiment of the present invention replaces part of the traditional convolution with the normalized depthwise separable convolution DSCBS module, which reduces the number of parameters and computational complexity, improves computational efficiency, and enables YOLOv8-seg-GCHA to run effectively even in resource-limited environments. It helps prevent overfitting when there are few training samples.

[0053] S103: Using the object segmentation data set to train the YOLOv8-seg-GCHA network structure to obtain an object segmentation model;

[0054] Specifically, the YOLOv8-seg-GCHA network structure is trained using a loss function; wherein the loss function L taotal By the classification loss L cls , bounding box regression loss L bbox and segmentation loss L seg The calculation formula is as follows:

[0055] L total =λ cls L cls +λ bbox L bbox +λ seg L seg

[0056] Among them, λ cls ,λ bbox and λ seg It is a hyperparameter used to adjust the proportion of each part of the loss in the total loss;

[0057] Specifically, the classification loss L cls Use BCE (binary cross entropy) as the classification loss, judge "whether it is this category" for each category, and output the confidence.

[0058]

[0059] where y i is the one-hot encoding of the category label, p i is the class probability predicted by the model.

[0060] Specifically, the bounding box regression loss L bbox By L CIoU and L DFL It consists of two parts:

[0061] L bbox =L CIoU +L DFL

[0062] L CIoU The aspect ratio criterion is added to the intersection-over-union loss, L DFL It is the form of cross entropy.

[0063] Specifically, the segmentation loss L seg The cross entropy loss is calculated based on the predicted mask mask of each instance and the corresponding true mask gt_mask, and normalized by the true bounding box area of ​​the corresponding instance:

[0064]

[0065] Where f is the binary cross entropy loss and area is the area of ​​the corresponding true box.

[0066] Specifically, during the training phase of the YOLOv8-seg-GCHA network, a variety of data augmentation strategies are used to enhance the generalization ability of the model, and appropriate regularization and optimization algorithms are applied to adjust the network parameters until the network converges to ensure the efficiency and stability of the model.

[0067] Specifically, during the training phase of the YOLOv8-seg-GCHA network, the embodiment of the present invention provides the scale parameters (such as Figure 4 ), the scale parameters can be adjusted according to the actual situation to optimize the model performance to meet different performance and resource requirements.

[0068] S104: Inputting the remote sensing image of the target area into the ground object segmentation model to obtain a ground object segmentation result of the target area.

[0069] Specifically, the trained model is used to infer and correct the remote sensing image, and the inference result is further processed to obtain the final segmentation result (such as Figure 2 First, redundant prediction boxes are removed by NMS, and then the prototype mask and mask coefficients are used to generate the final instance segmentation mask. Finally, the segmentation mask is post-processed by threshold processing and morphological operations to obtain a more accurate segmentation result.

[0070] Specifically, the performance of YOLOv8-seg and YOLOv8-seg-GCHA networks is compared when processing the same remote sensing image. Figure 6 and Figure 7 As shown, it can be observed that YOLOv8-seg-GCHA exhibits higher accuracy and stronger adaptability when processing instances with complex color and texture changes, which shows that the YOLOv8-seg-GCHA network has significant advantages in segmenting complex remote sensing images and can provide more accurate and detailed instance segmentation effects.

[0071] Based on the above embodiments, Figure 3 As shown, an embodiment of the present invention provides a method for constructing a GCHA module, wherein the GCHA module includes a global context module GCM and a hybrid attention module HAM;

[0072] The feature map output by the global context module GCM is directly used as the input of the hybrid attention module HAM, and the final feature map is obtained by processing by the hybrid attention module HAM;

[0073] Wherein, the global context module GCM integrates the global information of the image;

[0074] The hybrid attention module HAM specifically includes: a self-attention unit, a spatial attention unit and a channel attention unit; the self-attention unit is used to provide global information to the channel attention unit and the spatial attention unit respectively, the channel attention unit is used to process the global information to generate a channel attention weight map, and the spatial attention unit is used to process the global information to generate a spatial attention map.

[0075] Specifically, the global context module (GCM) is a mechanism used in deep learning to enhance the model's ability to capture global information. It extracts global context features by aggregating information from the entire image or feature map, and then feeds these features back to each local location, thereby enhancing the model's ability to learn global dependencies.

[0076] Specifically, the self-attention part in the HAM of the embodiment of the present invention adopts the self-attention mechanism in the standard Transformer. Assuming that the size of the input feature map is (B, C, H, W), where B is the batch size, C is the number of channels, H and W are the height and width of the feature map, the input feature map is converted to the format of (B, H×W, C), and the query matrix Q, key matrix K and value matrix V are generated respectively through the learnable weight matrices WQ, WK, and WV, and the self-attention of the input features is calculated to obtain an output feature map of size (B, H×W, C). This output feature map captures the long-distance dependencies between all positions in the input, providing rich global information for the channel attention and spatial attention modules.

[0077] The spatial attention unit is specifically used to restore the output of the self-attention unit to the original spatial dimension, perform maximum pooling and average pooling respectively, concatenate the two pooling results obtained in the channel dimension, and generate a spatial attention map through convolution and Sigmoid activation function;

[0078] Specifically, the following is the processing of spatial attention. The output of self-attention is restored to the original spatial dimensions (B, C, H, W), and maximum pooling and average pooling are performed respectively to highlight the spatial features. The two pooling results are concatenated in the channel dimension (B, 2×C, H, W), and then a 7×7 convolution kernel and Sigmoid activation function are used to generate a spatial attention map (B, 1, H, W), which assigns an importance weight to each spatial position.

[0079] The channel attention is specifically used to restore the output of the self-attention unit to the original spatial dimension, use adaptive average pooling to compress the spatial information of each channel to a single value, reduce the number of channels through convolution, use ReLU activation function to increase nonlinearity, and then use convolution to restore the number of channels. Finally, the weight factor of each channel is generated by the Sigmoid function to generate a channel attention weight map;

[0080] Specifically, adaptive average pooling is a pooling operation in deep learning that divides the input feature map into multiple small blocks, then calculates the average value for each small block, and finally uses these average values ​​as the values ​​of the corresponding positions of the output feature map. In this way, regardless of the size of the input feature map, the size of the output feature map is fixed, thereby retaining the feature distribution of the input feature map without losing too much information, and being able to convert an input feature map of any size into an output feature map of a fixed size.

[0081] Specifically, the following is the processing of channel attention. The output of self-attention is restored to the original spatial dimensions (B, C, H, W), and the spatial information of each channel is compressed to a single value (B, C, 1, 1) through adaptive average pooling, emphasizing the importance of each channel. The number of channels is reduced through a 1×1 convolution layer (B, C / / reduce, 1, 1), where reduce is a preset hyperparameter, where " / / " indicates a rounding down operation. By adjusting the value of reduce, the number of parameters and the amount of calculation in the model can be reduced, and the complexity and computational cost of the model can be controlled. The ReLU activation function is used to increase nonlinearity, and the number of channels (B, C, 1, 1) is restored through a 1×1 convolution. Finally, the weight factor (B, C, 1, 1) of each channel is generated through the Sigmoid function.

[0082] Correspondingly, the final feature map is obtained by processing with the hybrid attention module HAM, specifically including expanding the channel attention weight map and the spatial attention map to the original dimension and performing element-by-element multiplication operation with the output of the self-attention to form the final feature map.

[0083] Specifically, the channel attention weight map and the spatial attention map are expanded to the original dimensions (B, C, H, W), and element-wise multiplication is performed with the output of the self-attention (B, C, H, W) to form the final feature map (B, C, H, W). The output of the self-attention layer is used as the input of the channel attention and spatial attention layers to ensure that the attention mechanism refines the adjustment of important features based on global dependencies. The features of each channel and each spatial position are weighted according to their importance. The final output feature map combines the adjustment results of self-attention, channel attention, and spatial attention, enhancing the model's responsiveness to important features. The number of heads in the self-attention layer is an adjustable hyperparameter (for example, it can be set to 8), depending on the actual task requirements and computing resources.

[0084] Specifically, the GCHA module consists of a global context module GCM and a hybrid attention module HAM. GCM assists in the precise positioning and identification of objects by integrating the global information of the image. It can still effectively identify and segment the target objects under noise interference, improving the adaptability and accuracy of the model in complex environments. HAM combines self-attention, channel attention and spatial attention. The self-attention layer generates query, key and value matrices through self-learning weight matrices, strengthening the correlation between features; the channel and spatial attention layers further refine these features and form a comprehensive feature map through element-by-element multiplication operations, which enhances the accuracy and robustness of the model in processing color and texture changes, and improves the ability to resist noise interference. By dynamically adjusting the importance of features and improving the network's response to each part of the image, it can effectively handle color and texture changes in complex scenes such as remote sensing images, and improve the accuracy of segmentation and detection.

[0085] Based on the above embodiments, the present invention provides a method for introducing a GCHA module.

[0086] The introducing of the GCHA module specifically includes introducing a GCHA module after the SPPF module of the YOLOv8-seg network structure, and introducing a GCHA module before each of the three segmentation heads.

[0087] Specifically, the YOLOv8-seg-GCHA network structure is as follows Figure 4 As shown, a GCHA module is introduced after the SPPF module and before the three segmentation heads. Figure 4 The SPPF module extracts and merges multi-scale spatial information. The GCHA module is introduced afterwards to integrate global information and optimize feature expression before further feature processing. The GCHA module is introduced before the final segmentation to help the model segment the boundary more accurately.

[0088] Specifically, the GCHA module is integrated into the backbone network after the SPPF module and before the three segmentation heads. After the multi-scale feature fusion, the information is further refined and the feature expression is optimized before the segmentation head processing. This unique integration method can decode and classify features more accurately, improve the network's ability to parse the boundaries and internal details of actual objects, and improve segmentation accuracy. By integrating and optimizing global and local information from the image, the GCHA module can better solve the problem of a single instance being segmented into different instances due to inconsistent color or texture in the object segmentation task.

[0089] Based on the above embodiments, Figure 5 As shown, an embodiment of the present invention provides a method for constructing a normalized depth-separable convolution DSCBS module.

[0090] The normalized depthwise separable convolution DSCBS module specifically includes a depthwise separable convolution DSC module unit, a two-dimensional batch normalization BatchNorm2d unit and a SiLU function unit connected in sequence.

[0091] Specifically, the feature map output by the previous module is input into the DSC convolution module to generate a new output feature map, the new output feature map is normalized using BatchNorm2d, and each element of the normalized feature map is nonlinearly transformed using the SiLU function to form the final output and enter the next module.

[0092] Specifically, the DSC convolution module is divided into channel-by-channel convolution and point-by-point convolution. Channel-by-channel convolution refers to the use of a separate convolution kernel for each input channel (or feature map) to perform a convolution operation. Each input channel is convolved with only one convolution kernel, and there is no cross-channel calculation. It only focuses on feature extraction in the spatial dimension, without involving the channel dimension. Point-by-point convolution means that after depth convolution, point-by-point convolution performs a convolution operation on all channels of the previous layer through a 1x1 convolution kernel to extract channel features. The main function of point-by-point convolution is to recombine and adjust channel information to generate the final feature map.

[0093] Based on the above embodiments, Figure 4 As shown, an embodiment of the present invention provides a method for introducing a normalized depth-separable convolution DSCBS module.

[0094] The introduction of the normalized depthwise separable convolutional DSCBS module specifically includes replacing the CBS convolutional module after the first C2F module of the backbone network with the normalized depthwise separable convolutional DSCBS module.

[0095] Specifically, in order to optimize computational efficiency, this solution introduces a normalized depthwise separable convolution (DSCBS) module into the backbone of the YOLOv8-seg-GCHA network, decomposing the traditional convolution into channel-by-channel convolution and 1×1 point-by-point convolution. The C2F module aims to improve feature expression capabilities through cross-stage partial connection and full connection fusion. The CBS convolution module after the first C2F module in the backbone network is replaced with a normalized depthwise separable convolution (DSCBS) module ( Figure 4 The layer numbered 3 in ), further optimizes the feature fusion process and reduces the computational complexity. The structure of DSCBS is as follows Figure 5 As shown. Taking YOLOv8-seg-GCHA(l) as an example, when the batch size is 1, Figure 4The input feature map size of the layer numbered 3 is 160×160×128, the output is 80×80×256, and the convolution kernel size is 3×3. If the standard convolution parameter is convolution kernel W×convolution kernel H×input channel number×output channel number, that is, 3×3×128×256=294912, the depthwise separable convolution decomposes the standard convolution into channel-by-channel convolution and 1×1 point-by-point convolution. The channel-by-channel convolution parameter is convolution kernel W×convolution kernel H×input channel number, and the point-by-point convolution parameter is 1×1×input channel number×output channel number. Therefore, under the same circumstances, the parameter amount of depthwise separable convolution is 3×3×128+1×1×128×256=33920, which significantly reduces the number of parameters and computational costs, improves the running efficiency of the model, and maintains good segmentation accuracy.

[0096] It can be understood that on the basis of YOLOv8-seg-GCHA, the CBS convolution module after the first C2F module of the backbone network is replaced with a normalized depth-separable convolution DSCBS module, which significantly reduces the number of parameters and improves the operation efficiency. This invention reduces the consumption of computing resources while maintaining the segmentation accuracy of the model, so that it can be deployed on devices with limited computing resources and even meet the needs of real-time analysis.

[0097] The embodiment of the present invention provides a remote sensing image object segmentation system based on YOLOv8-seg-GCHA, which specifically includes:

[0098] The data acquisition module is used to acquire remote sensing images, annotate the acquired images with ground objects, and generate ground object segmentation data sets;

[0099] A network structure building module, used to build a YOLOv8-seg-GCHA network structure; the YOLOv8-seg-GCHA network structure introduces a GCHA module and a normalized depth separable convolution DSCBS module into YOLOv8-seg; wherein the GCHA module is used to identify global information and local details of an image;

[0100] A network structure training module, used to train the YOLOv8-seg-GCHA network structure using the land object segmentation data set to obtain a land object segmentation model;

[0101] The target data processing module is used to input the remote sensing image of the target area into the ground object segmentation model to obtain the ground object segmentation result of the target area.

[0102] like Figure 8As shown, an embodiment of the present invention provides an electronic device for remote sensing image object segmentation based on YOLOv8-seg-GCHA, and the electronic device may include: a processor (processor) 501, a communication interface (Communications Interface) 502, a memory (memory) 503 and a communication bus 504, wherein the processor 501, the communication interface 502, and the memory 503 communicate with each other through the communication bus 504. The processor 501 can call the logic instructions in the memory 503 to execute the method provided in the above embodiment, for example, including:

[0103] S101: Acquire a remote sensing image, annotate the acquired image with objects, and generate an object segmentation dataset; S102: Construct a YOLOv8-seg-GCHA network structure; the YOLOv8-seg-GCHA network structure introduces a GCHA module and a normalized depth-separable convolution DSCBS module into YOLOv8-seg; wherein the GCHA module is used to identify global information and local details of the image; S103: Use the object segmentation dataset to train the YOLOv8-seg-GCHA network structure to obtain an object segmentation model; S104: Input the remote sensing image of the target area into the object segmentation model to obtain an object segmentation result of the target area.

[0104] In addition, when the logic instructions in the above-mentioned memory 403 are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0105] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A remote sensing image object segmentation method based on YOLOv8-seg-GCHA, characterized in that: include: Step 1: Obtain remote sensing images, annotate the acquired images, and generate object segmentation datasets; Step 2: construct a YOLOv8-seg-GCHA network structure; the YOLOv8-seg-GCHA network structure introduces a GCHA module and a normalized depth separable convolution DSCBS module into YOLOv8-seg; wherein the GCHA module is used to identify global information and local details of the image; Step 3: Use the object segmentation dataset to train the YOLOv8-seg-GCHA network structure to obtain a object segmentation model; Step 4: Input the remote sensing image of the target area into the ground object segmentation model to obtain the ground object segmentation result of the target area.

2. The remote sensing image object segmentation method based on YOLOv8-seg-GCHA according to claim 1, characterized in that: The GCHA module in step 2 includes a global context module GCM and a hybrid attention module HAM; the feature map output by the global context module GCM is directly used as the input of the hybrid attention module HAM, and the final feature map is obtained by processing by the hybrid attention module HAM; Wherein, the global context module GCM integrates the global information of the image; The hybrid attention module HAM specifically includes: a self-attention unit, a spatial attention unit and a channel attention unit; the self-attention unit is used to provide global information to the channel attention unit and the spatial attention unit respectively, the channel attention unit is used to process the global information to generate a channel attention weight map, and the spatial attention unit is used to process the global information to generate a spatial attention map.

3. The remote sensing image object segmentation method based on YOLOv8-seg-GCHA according to claim 2, characterized in that: The spatial attention unit is specifically used to restore the output of the self-attention unit to the original spatial dimension, perform maximum pooling and average pooling respectively, concatenate the two pooling results obtained in the channel dimension, and generate a spatial attention map through convolution and Sigmoid activation function; The channel attention unit is specifically used to restore the output of the self-attention unit to the original spatial dimension, use adaptive average pooling to compress the spatial information of each channel to a single value, reduce the number of channels through convolution, use ReLU activation function to increase nonlinearity, and then use convolution to restore the number of channels. Finally, a weight factor for each channel is generated by a Sigmoid function to generate a channel attention weight map; Correspondingly, the final feature map is obtained by processing with the hybrid attention module HAM, specifically including expanding the channel attention weight map and the spatial attention map to the original dimension and performing element-by-element multiplication operation with the output of the self-attention to form the final feature map.

4. The remote sensing image object segmentation method based on YOLOv8-seg-GCHA according to claim 1, characterized in that: The introduction of the GCHA module described in step 2 specifically includes: introducing a GCHA module after the SPPF module of the YOLOv8-seg network structure, and introducing a GCHA module before each of the three segmentation heads.

5. The remote sensing image object segmentation method based on YOLOv8-seg-GCHA according to claim 1, characterized in that: The introduction of the normalized depth-wise separable convolutional DSCBS module described in step 2 specifically includes replacing the CBS convolutional module after the first C2F module of the backbone network with the normalized depth-wise separable convolutional DSCBS module.

6. The remote sensing image object segmentation method based on YOLOv8-seg-GCHA according to claim 1, characterized in that: The normalized depthwise separable convolution DSCBS module specifically includes a depthwise separable convolution DSC module unit, a two-dimensional batch normalization BatchNorm2d unit and a SiLU function unit connected in sequence.

7. The remote sensing image object segmentation method based on YOLOv8-seg-GCHA according to claim 1, characterized in that: The loss function L is used in step 3 total The YOLOv8-seg-GCHA network structure is trained; wherein the loss function L total By the classification loss L cls , bounding box regression loss L bbox and segmentation loss L seg The calculation formula is as follows: L total =λ cls L cls +λ bbox L bbox +λ seg L seg where λ cls ,λ bbox and λ seg is a hyperparameter.

8. The remote sensing image object segmentation method based on YOLOv8-seg-GCHA according to claim 1, characterized in that: Step 4 also includes correcting the ground object segmentation result, specifically including removing the prediction box through NMS, generating an instance segmentation mask using the prototype mask and the mask coefficient, and post-processing the instance segmentation mask through threshold processing and morphological operations to obtain the final segmentation result.

9. The remote sensing image segmentation system based on YOLOv8-seg-GCHA is characterized by: include: The data acquisition module is used to acquire remote sensing images, annotate the acquired images with ground objects, and generate ground object segmentation data sets; A network structure building module, used to build a YOLOv8-seg-GCHA network structure; the YOLOv8-seg-GCHA network structure introduces a GCHA module and a normalized depth separable convolution DSCBS module into YOLOv8-seg; wherein the GCHA module is used to identify global information and local details of an image; A network structure training module, used to train the YOLOv8-seg-GCHA network structure using the land object segmentation data set to obtain a land object segmentation model; The target data processing module is used to input the remote sensing image of the target area into the ground object segmentation model to obtain the ground object segmentation result of the target area.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method according to any one of claims 1 to 8 is implemented.

Images