A remote sensing image segmentation method and system based on multi-scale decentralized attention

Abstract

The present invention relates to a remote sensing image segmentation method and system based on multi-scale decentralized attention. The method comprises: step S1, obtaining remote sensing images of a target object at different scales, and performing feature extraction on each of the remote sensing images to generate multiple first feature maps of different high and low resolutions; step S2, randomly selecting a portion of the remote sensing images, capturing the spatial dependency between any two positions of the remote sensing images corresponding to the first feature map, and obtaining a second feature map; step S3, fusing the corresponding position information of each first feature map with the corresponding second feature map to obtain a third feature map, and performing target object segmentation based on the first feature map, the second feature map, and the third feature map. By inputting feature maps of different scales during the model training phase and then fusing the different feature maps, the problem of object category confusion caused by the different scales of object categories at different resolutions is solved, thereby improving the accuracy of category classification.

Description

Technical Field

[0001] The embodiments of the present invention relate to the technical field of remote sensing image semantic segmentation, and in particular to a remote sensing image segmentation method and system based on multi-scale decentralized attention. Background Art

[0002] In recent years, with the rapid development of remote sensing acquisition technology, the application of remote sensing images has become more and more extensive. Remote sensing images contain a large amount of ground object information. Due to the diversity and complexity of different ground object information, the semantic segmentation of remote sensing images is relatively difficult.

[0003] Currently, semantic segmentation of remote sensing images primarily relies on traditional image segmentation algorithms and deep learning-based image segmentation. Traditional algorithms suffer from low segmentation accuracy and efficiency. Deep learning-based image segmentation leverages the powerful feature learning capabilities of convolutional neural networks to address the segmentation problem of remote sensing images. Objects in remote sensing images at different ground resolutions exhibit varying scales and features. Convolutional neural network models fed with a single-scale feature map are prone to confusion between object categories and can lose detailed information when segmenting multi-category remote sensing images. Summary of the Invention

[0004] The present invention aims to solve the technical problems existing in the prior art and provides a remote sensing image segmentation method and system based on multi-scale decentralized attention.

[0005] To solve the above technical problems, in a first aspect, an embodiment of the present invention provides a remote sensing image segmentation method based on multi-scale decentralized attention, comprising:

[0006] Step S1: acquiring remote sensing images of the target object at different scales, and performing feature extraction on each of the remote sensing images to generate a plurality of first feature maps of different high and low resolutions;

[0007] Step S2: randomly selecting a portion of the remote sensing images, capturing the spatial dependency between any two positions of the portion of the remote sensing images corresponding to the first feature map, and obtaining a second feature map;

[0008] Step S3: perform corresponding position information fusion processing on each of the first feature maps and the corresponding second feature map to obtain a third feature map, and perform target object segmentation based on the first feature map, the second feature map and the third feature map.

[0009] Preferably, in step S1, after generating a plurality of feature maps of different high and low resolutions, the method further comprises:

[0010] The feature map is subjected to feature enhancement processing based on the OCRNet model to enhance the contextual semantic information of the feature map.

[0011] Preferably, the step S1 specifically includes:

[0012] Step S11: acquiring a remote sensing image containing a target object, performing convolution, batch normalization, and activation function processing on the remote sensing image, and scaling the remote sensing image into remote sensing images of different scales, where the remote sensing images of different scales include at least the first-scale remote sensing image and the second-scale remote sensing image;

[0013] Step S12: performing feature extraction on the remote sensing image based on the hrnetV2 backbone feature extraction model to obtain a plurality of first feature maps of different resolutions.

[0014] Preferably, the step S2 specifically includes:

[0015] performing feature enhancement processing on first feature maps corresponding to the first-scale remote sensing image and the second-scale remote sensing image;

[0016] Position attention with different weight coefficients is used to capture the spatial dependency between any two positions of the first feature map corresponding to the first-scale remote sensing image, thereby obtaining a second feature map.

[0017] Preferably, in step S3, performing target segmentation based on the first feature map, the second feature map, and the third feature map specifically includes:

[0018] The first feature map, the second feature map, and the third feature map are input into a pre-trained segmentation model to predict the pixel category of each feature map at the original size of the remote sensing image, and output the segmented image of each target object in the remote sensing image.

[0019] In a second aspect, an embodiment of the present invention provides a remote sensing image segmentation system based on multi-scale decentralized attention, comprising:

[0020] Scale feature map module, used to obtain remote sensing images of different scales of the target object;

[0021] A backbone feature extraction module performs feature extraction on each of the remote sensing images to generate a plurality of first feature maps of different high and low resolutions;

[0022] A decentralized attention module randomly selects a portion of the remote sensing image, captures the spatial dependency between any two positions of the portion of the remote sensing image corresponding to the first feature map, and obtains a second feature map;

[0023] a feature fusion module, fusing corresponding position information of each of the first feature maps with the corresponding second feature map to obtain a third feature map;

[0024] A post-processing output module performs target object segmentation based on the first feature map, the second feature map, and the third feature map.

[0025] As a preference, it also includes:

[0026] The feature enhancement module performs feature enhancement processing on the feature map based on the OCRNet model to enhance the contextual semantic information of the feature map.

[0027] Preferably, the remote sensing images of different scales include at least the first-scale remote sensing image and the second-scale remote sensing image;

[0028] The feature fusion module is specifically configured to use position attention with different weight coefficients to capture the spatial dependency between any two positions of the first feature map corresponding to the first-scale remote sensing image to obtain a second feature map;

[0029] The post-processing output module is used to predict the pixel categories of each feature map at the original size of the remote sensing image based on the third feature map after the first-scale remote sensing image feature fusion, the first-scale remote sensing image, the first feature map corresponding to the second-scale remote sensing image, and the second feature map corresponding to the first-scale remote sensing image, and output the segmented image of each target object in the remote sensing image.

[0030] In a third aspect, an embodiment of the present invention provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and runnable on the processor. When the processor executes the program, the steps of the remote sensing image segmentation method based on multi-scale decentralized attention as described in the embodiment of the first aspect of the present invention are implemented.

[0031] In a fourth aspect, an embodiment of the present invention provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the remote sensing image segmentation method based on multi-scale decentralized attention as described in the embodiment of the first aspect of the present invention.

[0032] The beneficial effects of the present invention are as follows: compared with traditional image segmentation algorithms, by inputting feature maps of different scales during the model training phase and then fusing the different feature maps, the problem of confusion between object categories caused by different scales of object categories at different resolutions is solved, and the classification accuracy is improved; the segmentation accuracy is higher and the efficiency is higher; compared with the convolutional neural network model with a single-scale feature map as input, the remote sensing image segmentation method based on multi-scale decentralized attention can effectively reduce the confusion between different object categories and improve the segmentation accuracy; it can pay more attention to the expression of detailed information between object categories, thereby improving the segmentation accuracy BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1This is a flow chart of a remote sensing image segmentation method based on multi-scale decentralized attention according to an embodiment of the present invention;

[0034] Figure 2 This is a flowchart of a remote sensing image segmentation method based on multi-scale decentralized attention according to an embodiment of the present invention;

[0035] Figure 3 Schematic diagram of the remote sensing image to be segmented;

[0036] Figure 4 Schematic diagram of a remote sensing segmented image segmented according to a method according to an embodiment of the present invention;

[0037] Figure 5 is a schematic diagram of an electronic device according to an embodiment of the present invention;

[0038] Figure 6 Schematic diagram of a computer-readable storage medium according to an embodiment of the present invention. DETAILED DESCRIPTION

[0039] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0040] In the embodiments of the present application, the term "and / or" is merely a description of the association relationship between associated objects, indicating that three relationships may exist. For example, A and / or B may represent three situations: A exists alone, A and B exist at the same time, and B exists alone.

[0041] The terms "first" and "second" in the embodiments of the present application are only used for descriptive purposes and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of the present application, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusions. For example, a system, product or device comprising a series of components or units is not limited to the listed components or units, but may optionally also include components or units that are not listed, or may optionally also include other components or units that are inherent to these products or devices. In the description of the present application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0042] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0043] Figures 1 to 2 The present invention provides a remote sensing image segmentation method based on multi-scale decentralized attention, including:

[0044] Step S1: acquiring remote sensing images of the target object at different scales, and performing feature extraction on each of the remote sensing images to generate a plurality of first feature maps of different high and low resolutions;

[0045] Step S11: Acquire a remote sensing image containing a target object, perform convolution, batch normalization, and activation function processing on the remote sensing image, and scale the remote sensing image into remote sensing images of different scales, where the remote sensing images of different scales include at least the first-scale remote sensing image and the second-scale remote sensing image; for example, scale the original image to a scale of 0.5 to generate a first-scale remote sensing image, and scale the original image to a scale of 1 to generate a second-scale remote sensing image; the first-scale remote sensing image and the second-scale remote sensing image are obtained after convolution, batch normalization, and activation function activation.

[0046] Step S12: Extract features from the remote sensing image based on the HRNetV2 backbone feature extraction model to obtain a plurality of first feature maps of different resolutions. The features of HRNetV2 are mainly derived from its parallel feature extraction part, and high-resolution, medium-resolution and low-resolution features are always maintained during the feature extraction process.

[0047] The feature map is enhanced based on an OCRNet model to enhance the contextual semantic information of the feature map. The OCRNet model integrates contextual information by correlating the current pixel position with the contextual pixel to obtain an enhanced pixel representation.

[0048] Step S2: randomly selecting a portion of the remote sensing image, capturing the spatial dependency between any two positions of the portion of the remote sensing image corresponding to the first feature map, and obtaining a second feature map that pays more attention to edge details;

[0049] like Figure 2 As shown in , in this embodiment, dual-line processing is performed on the original remote sensing image, and feature enhancement processing is performed on the first feature maps corresponding to the first-scale remote sensing image and the second-scale remote sensing image;

[0050] Position attention with different weight coefficients is used to capture the spatial dependency between any two positions of the first feature map corresponding to the first-scale remote sensing image, thereby obtaining a second feature map.

[0051] Step S3: perform corresponding position information fusion processing on each of the first feature maps and the corresponding second feature map to obtain a third feature map, and perform target object segmentation based on the first feature map, the second feature map and the third feature map.

[0052] The first feature map, the second feature map and the third feature map are input into a pre-trained segmentation model to predict the pixel category of each feature map at the original size of the remote sensing image, and output the segmented image of each target object in the remote sensing image. Figure 3 、 Figure 4 As shown in , it is a segmentation effect image of the remote sensing image segmentation method based on multi-scale decentralized attention according to an embodiment of the present invention, wherein Figure 3 is the remote sensing image to be segmented, Figure 4 This is a remote sensing segmentation image segmented based on this method. Different grayscale colors in the image represent different types of land objects.

[0053] An embodiment of the present invention further provides a remote sensing image segmentation system based on multi-scale decentralized attention, which is based on the remote sensing image segmentation method based on multi-scale decentralized attention in the above embodiment, including:

[0054] Scale feature map module, used to obtain remote sensing images of different scales of the target object;

[0055] A backbone feature extraction module performs feature extraction on each of the remote sensing images to generate a plurality of first feature maps of different high and low resolutions;

[0056] A decentralized attention module randomly selects a portion of the remote sensing image, captures the spatial dependency between any two positions of the portion of the remote sensing image corresponding to the first feature map, and obtains a second feature map;

[0057] a feature fusion module, fusing corresponding position information of each of the first feature maps with the corresponding second feature map to obtain a third feature map;

[0058] A post-processing output module performs target object segmentation based on the first feature map, the second feature map, and the third feature map.

[0059] As a preference, it also includes:

[0060] The feature enhancement module performs feature enhancement processing on the feature map based on the OCRNet model to enhance the contextual semantic information of the feature map.

[0061] Preferably, the remote sensing images of different scales include at least the first-scale remote sensing image and the second-scale remote sensing image;

[0062] The feature fusion module is specifically configured to use position attention with different weight coefficients to capture the spatial dependency between any two positions of the first feature map corresponding to the first-scale remote sensing image to obtain a second feature map;

[0063] The post-processing output module is used to predict the pixel categories of each feature map at the original size of the remote sensing image based on the third feature map after the first-scale remote sensing image feature fusion, the first-scale remote sensing image, the first feature map corresponding to the second-scale remote sensing image, and the second feature map corresponding to the first-scale remote sensing image, and output the segmented image of each target object in the remote sensing image.

[0064] See also Figure 5 , Figure 5 Schematic diagram of an embodiment of an electronic device provided by an embodiment of the present invention. Figure 5 As shown, an embodiment of the present invention provides an electronic device 500, including a memory 510, a processor 520, and a computer program 511 stored in the memory 520 and executable on the processor 520. When the processor 520 executes the computer program 511, the following steps are implemented:

[0065] Step S1: acquiring remote sensing images of the target object at different scales, and performing feature extraction on each of the remote sensing images to generate a plurality of first feature maps of different high and low resolutions;

[0066] Step S2: randomly selecting a portion of the remote sensing images, capturing the spatial dependency between any two positions of the portion of the remote sensing images corresponding to the first feature map, and obtaining a second feature map;

[0067] Step S3: perform corresponding position information fusion processing on each of the first feature maps and the corresponding second feature map to obtain a third feature map, and perform target object segmentation based on the first feature map, the second feature map and the third feature map.

[0068] See also Figure 6 , Figure 6 Schematic diagram of an embodiment of a computer-readable storage medium provided by an embodiment of the present invention. Figure 6 As shown, this embodiment provides a computer-readable storage medium 600 on which a computer program 611 is stored. When the computer program 611 is executed by a processor, the following steps are implemented:

[0069] Step S1: acquiring remote sensing images of the target object at different scales, and performing feature extraction on each of the remote sensing images to generate a plurality of first feature maps of different high and low resolutions;

[0070] Step S2: randomly selecting a portion of the remote sensing images, capturing the spatial dependency between any two positions of the portion of the remote sensing images corresponding to the first feature map, and obtaining a second feature map;

[0071] Step S3: perform corresponding position information fusion processing on each of the first feature maps and the corresponding second feature map to obtain a third feature map, and perform target object segmentation based on the first feature map, the second feature map and the third feature map.

[0072] It should be noted that, in the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0073] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0074] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0075] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0076] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0077] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0078] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. A remote sensing image segmentation method based on multi-scale decentralized attention, characterized in that: include: Step S1: acquiring remote sensing images of the target object at different scales, and performing feature extraction on each of the remote sensing images to generate a plurality of first feature maps of different high and low resolutions; Specifically include: Step S11: acquiring a remote sensing image containing a target object, performing convolution, batch normalization, and activation function processing on the remote sensing image, and scaling the remote sensing image into remote sensing images of different scales, where the remote sensing images of different scales include at least a first-scale remote sensing image and a second-scale remote sensing image; Step S12: performing feature extraction on the remote sensing image based on the hrnetV2 backbone feature extraction model to obtain a plurality of first feature maps with different resolutions; Step S2: randomly selecting a portion of the remote sensing image, capturing the spatial dependency between any two positions of the portion of the remote sensing image corresponding to the first feature map, and obtaining a second feature map; specifically comprising: performing feature enhancement processing on first feature maps corresponding to the first-scale remote sensing image and the second-scale remote sensing image; Using position attention with different weight coefficients to capture the spatial dependency between any two positions of the first feature map corresponding to the first-scale remote sensing image, to obtain a second feature map; Step S3, performing corresponding position information fusion processing on each of the first feature maps and the corresponding second feature map to obtain a third feature map, and performing target object segmentation based on the first feature map, the second feature map, and the third feature map; comprising: Based on the third feature map after feature fusion of the first-scale remote sensing image, the first-scale remote sensing image, the first feature map corresponding to the second-scale remote sensing image, and the second feature map corresponding to the first-scale remote sensing image, the pixel category of each feature map at the original size of the remote sensing image is predicted, and the segmented image of each target object in the remote sensing image is output.

2. The remote sensing image segmentation method based on multi-scale decentralized attention according to claim 1, characterized in that: In the step S1, after generating a plurality of feature maps of different high and low resolutions, the process further includes: The feature map is subjected to feature enhancement processing based on the OCRNet model to enhance the contextual semantic information of the feature map.

3. The remote sensing image segmentation method based on multi-scale decentralized attention according to claim 1, characterized in that: In step S3, segmenting the target object based on the first feature map, the second feature map, and the third feature map specifically includes: The first feature map, the second feature map, and the third feature map are input into a pre-trained segmentation model to predict the pixel category of each feature map at the original size of the remote sensing image, and output the segmented image of each target object in the remote sensing image.

4. A remote sensing image segmentation system based on multi-scale decentralized attention, characterized in that: include: A scale feature map module is used to obtain remote sensing images of different scales of the target object, wherein the remote sensing images of different scales include at least a first-scale remote sensing image and a second-scale remote sensing image; A backbone feature extraction module performs feature extraction on each of the remote sensing images to generate a plurality of first feature maps of different high and low resolutions; A decentralized attention module randomly selects a portion of the remote sensing image, captures the spatial dependency between any two positions of the portion of the remote sensing image corresponding to the first feature map, and obtains a second feature map; a feature fusion module, performing corresponding position information fusion processing on each of the first feature maps and the corresponding second feature map to obtain a third feature map; specifically, using position attention with different weight coefficients to capture the spatial dependency between any two positions of the first feature map corresponding to the first-scale remote sensing image to obtain a second feature map; The post-processing output module performs target object segmentation based on the first feature map, the second feature map, and the third feature map; specifically, based on the third feature map after feature fusion of the first-scale remote sensing image, the first-scale remote sensing image, the first feature map corresponding to the second-scale remote sensing image, and the second feature map corresponding to the first-scale remote sensing image, predicts the pixel category of each feature map at the original size of the remote sensing image, and outputs the segmented image of each target object in the remote sensing image.

5. The remote sensing image segmentation system based on multi-scale decentralized attention according to claim 4, characterized in that: Also includes: The feature enhancement module performs feature enhancement processing on the feature map based on the OCRNet model to enhance the contextual semantic information of the feature map.

6. An electronic device, characterized in that: include: Memory for storing computer software programs; A processor is used to read and execute the computer software program, thereby implementing the remote sensing image segmentation method based on multi-scale decentralized attention as described in any one of claims 1-3.

7. A non-transitory computer-readable storage medium, characterized in that The storage medium stores a computer software program for implementing the remote sensing image segmentation method based on multi-scale decentralized attention as described in any one of claims 1 to 3.

Images