Semantic segmentation method and device for high-resolution remote sensing image, equipment and medium

Abstract

The application discloses a high-resolution remote sensing image semantic segmentation method and device, equipment and medium, including: using an encoder and decoder architecture to establish a remote sensing image semantic segmentation model; the encoder in the model includes two encoding branches, which are respectively used for processing radar image data and optical image data; collecting remote sensing image data corresponding to a preset area, constructing a training set according to the remote sensing image data, and using the training set to iteratively train the remote sensing image semantic segmentation model; obtaining target remote sensing image data corresponding to a target area, inputting the target remote sensing image data into the trained remote sensing image semantic segmentation model, and outputting corresponding semantic segmentation results through the remote sensing image semantic segmentation model. The technical scheme of the embodiment of the application can fuse image information of radar and optical images of two different modalities, and improve the accuracy of the remote sensing image semantic segmentation result.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to semantic segmentation methods, apparatus, devices, and media for high-resolution remote sensing images. Background Technology

[0002] With the development of space sensing technology and the increasing variety of remote sensing sensors, multi-source remote sensing data, including optical, synthetic aperture radar (SAR), and lidar, has driven the transformation of semantic segmentation tasks from single-modality to multi-modal fusion. Multi-modal remote sensing image semantic information fusion technology, by integrating the complementary characteristics of remote sensing data from multiple platforms, sensors, and spectral bands, can achieve more accurate semantic segmentation. In the field of remote sensing monitoring, commonly used imaging remote sensing data mainly consists of optical images and SAR images; therefore, effectively fusing the complementary characteristics of different data is particularly important.

[0003] In existing technologies, static fusion strategies are mainly adopted, such as multispectral fusion networks and RGB thermal infrared fusion networks. These methods mainly use simple fusion methods such as summation or cascading to fuse multimodal features, which may not be able to accurately capture important information between different modalities, resulting in low reliability of semantic segmentation results of remote sensing images. Summary of the Invention

[0004] This invention provides a method, apparatus, device, and medium for semantic segmentation of high-resolution remote sensing images. It can fuse information from two different modalities, radar and optical, to compensate for the shortcomings of a single remote sensing sensor, solve the problems of complementary utilization of heterogeneous data and noise suppression, and improve the accuracy of semantic segmentation results of remote sensing images.

[0005] According to one aspect of the present invention, a semantic segmentation method for high-resolution remote sensing images is provided, the method comprising:

[0006] A semantic segmentation model for remote sensing images is established using an encoder and decoder architecture. The encoder in the model includes two coding branches, which are used to process radar image data and optical image data, respectively.

[0007] Collect remote sensing image data corresponding to a preset area, construct a training set based on the remote sensing image data, and use the training set to iteratively train the remote sensing image semantic segmentation model;

[0008] Obtain the target remote sensing image data corresponding to the target area, input the target remote sensing image data into the trained remote sensing image semantic segmentation model, and output the corresponding semantic segmentation result through the remote sensing image semantic segmentation model.

[0009] Optionally, in the remote sensing image semantic segmentation model, each coding branch consists of five feature extraction layers, and a multimodal channel spatial attention module is connected after each feature extraction layer. A hollow spatial pyramid pooling module is also connected after the fourth and fifth feature extraction layers, respectively.

[0010] Each feature extraction layer includes a convolutional network, a normalization layer, an activation function layer, and a max pooling layer.

[0011] Optionally, in the remote sensing image semantic segmentation model, the decoder includes a compressed excitation attention module and a convolutional layer;

[0012] The compressed excitation attention module is used to fuse the final output results of each feature extraction layer in the encoder.

[0013] The convolutional layer is used to perform convolution processing on the fusion result and to perform image restoration using deconvolution operation. Then, based on the restored image, an activation function is used to output the semantic segmentation result.

[0014] Optionally, the multimodal channel spatial attention module includes three convolutional layers, a global average pooling layer, a multilayer perceptron, a weighted fusion module, and a spatial attention mechanism module;

[0015] The first convolutional layer is used to fuse radar image features and optical image features; the global average pooling layer is used to extract channel features from the fusion result.

[0016] The multilayer perceptron is used to determine cross-channel attention weight values ​​based on channel features; the weighted fusion module is used to perform weighted fusion of radar image features and optical image features based on the attention weight values.

[0017] The spatial attention mechanism module is used to generate two single-channel feature maps based on the weighted fusion result; the second convolutional layer is deployed within the spatial attention mechanism module and is used to generate a spatial attention map based on the two single-channel feature maps.

[0018] The third convolutional layer is used to output the feature processing results based on the spatial attention map.

[0019] Optionally, a training set is constructed based on the remote sensing image data, including:

[0020] The remote sensing image data is processed using a data augmentation algorithm, and the processed image data is semantically labeled to obtain the training set corresponding to the remote sensing image semantic segmentation model.

[0021] Optionally, the remote sensing image semantic segmentation model is iteratively trained using the training set, including:

[0022] Training samples are acquired in batches from the training set and used as current samples.

[0023] The current sample is input into the remote sensing image semantic segmentation model, and the model parameters are adjusted based on the model output.

[0024] Determine whether the performance of the remote sensing image semantic segmentation model meets the preset requirements;

[0025] If not, return to the operation of obtaining training samples in batches from the training set as the current samples until the performance of the model meets the preset requirements.

[0026] According to another aspect of the present invention, a semantic segmentation apparatus for high-resolution remote sensing images is provided, the apparatus comprising:

[0027] The model building module is used to establish a semantic segmentation model for remote sensing images using an encoder and decoder architecture; the encoder in the model includes two coding branches, which are used to process radar image data and optical image data respectively;

[0028] The model training module is used to collect remote sensing image data corresponding to a preset area, construct a training set based on the remote sensing image data, and use the training set to iteratively train the remote sensing image semantic segmentation model.

[0029] The model application module is used to acquire target remote sensing image data corresponding to the target area, input the target remote sensing image data into the trained remote sensing image semantic segmentation model, and output the corresponding semantic segmentation result through the remote sensing image semantic segmentation model.

[0030] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0031] At least one processor; and

[0032] A memory communicatively connected to the at least one processor; wherein,

[0033] The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the semantic segmentation method for high-resolution remote sensing images according to any embodiment of the present invention.

[0034] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the semantic segmentation method for high-resolution remote sensing images according to any embodiment of the present invention.

[0035] According to another aspect of the present invention, a computer program product is provided, the computer program product comprising a computer program that, when executed by a processor, implements the semantic segmentation method for high-resolution remote sensing images according to any embodiment of the present invention.

[0036] The technical solution provided by this invention establishes a remote sensing image semantic segmentation model using an encoder and decoder architecture, collects remote sensing image data corresponding to a preset region, constructs a training set based on the remote sensing image data, and uses the training set to iteratively train the remote sensing image semantic segmentation model to obtain target remote sensing image data corresponding to the target region. The target remote sensing image data is then input into the trained remote sensing image semantic segmentation model, and the model outputs the corresponding semantic segmentation result. This technical means can fuse information from radar and optical modalities, compensate for the shortcomings of a single remote sensing sensor, solve the problems of complementary utilization of heterogeneous data and noise suppression, and improve the accuracy of remote sensing image semantic segmentation results.

[0037] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1a This is a flowchart of a semantic segmentation method for high-resolution remote sensing images provided according to an embodiment of the present invention;

[0040] Figure 1b This is a schematic diagram of the structure of a remote sensing image semantic segmentation model provided in an embodiment of the present invention;

[0041] Figure 2a This is a flowchart of another semantic segmentation method for high-resolution remote sensing images provided by an embodiment of the present invention;

[0042] Figure 2b This is a schematic diagram of the structure of a multimodal channel spatial attention module in a model provided by an embodiment of the present invention;

[0043] Figure 2c This is a schematic diagram illustrating the effect of a semantic segmentation method for high-resolution remote sensing images provided by an embodiment of the present invention;

[0044] Figure 3 This is a schematic diagram of the structure of a semantic segmentation device for high-resolution remote sensing images provided in an embodiment of the present invention;

[0045] Figure 4 This is a schematic diagram of the structure of an electronic device that implements the semantic segmentation method for high-resolution remote sensing images according to embodiments of the present invention. Detailed Implementation

[0046] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0047] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0048] Figure 1a This is a flowchart illustrating a semantic segmentation method for high-resolution remote sensing images provided in an embodiment of the present invention. This embodiment is applicable to the automatic semantic segmentation of high-resolution remote sensing images. The method can be executed by a semantic segmentation device for high-resolution remote sensing images, which can be implemented in hardware and / or software and configured in an electronic device, such as... Figure 1a As shown, the method includes:

[0049] Step 110: Establish a remote sensing image semantic segmentation model using an encoder and decoder architecture; the encoder in the model includes two coding branches, which are used to process radar image data and optical image data respectively.

[0050] In one embodiment of this example, Figure 1bThis diagram illustrates the structure of a radar optical attention network (SAR-OPT Attention Network, SOANet) model provided for this embodiment. The SOANet model is also the remote sensing image semantic segmentation model in this invention. Figure 1b As shown, each encoding branch in this model consists of five feature extraction layers. Each feature extraction layer is followed by a Multimodal Convolutional Block Attention Module (MCBAM). The fourth and fifth feature extraction layers are each followed by an Atrous Spatial Pyramid Pooling (ASPP) module. Each feature extraction layer contains a convolutional network, a normalization layer, an activation function layer, and a max-pooling layer.

[0051] like Figure 1b As shown, the decoder includes a squeeze and excitation (SE) attention module and convolutional layers. The SE attention module is used to fuse the final outputs of each feature extraction layer in the encoder. The convolutional layers are used to perform convolution processing on the fused result, and then perform deconvolution operations to restore the image. Finally, an activation function is applied to the restored image to output the semantic segmentation result.

[0052] In one specific embodiment, the SOANet model can progressively extract deep semantic features through hierarchical downsampling. By introducing an MCBAM module after each feature extraction layer, a channel-space dual-dimensional attention mechanism can be used to achieve cross-modal feature interaction, enabling the model to autonomously strengthen complementary features between the two modalities. Subsequently, by introducing an ASPP module after the fourth and fifth feature extraction layers, the contextual information of the fourth and fifth feature extraction layers can be captured in parallel, enhancing the model's ability to recognize complex scenes.

[0053] In this embodiment, since the ASPP module is used to expand the receptive field of the convolutional kernel and integrate multi-scale semantic information, its application location must match the semantic level of the features. In the shallow layers of the network, the feature maps have high resolution and are mainly low-level details, such as edges and texture information. If the ASPP module is introduced at this stage, on the one hand, it will lead to a significant increase in computational complexity, and on the other hand, the shallow features lack sufficient semantic abstraction, making it difficult for ASPP's multi-scale fusion to fully exert its advantages. It may even ignore the fine expression of its local details due to premature focus on the global context. Therefore, in the SOAnet model, the ASPP module is embedded in the fourth and fifth layers at the end of the network to achieve multi-scale information complementarity through cross-layer feature interaction. In the shallow layers, only the Convolutional Block Attention Module (CBAM) is used to extract local details, thereby achieving multi-modal and multi-scale feature fusion from shallow to deep layers.

[0054] Furthermore, the decoder in the SOANet model employs an SE attention module to dynamically calibrate the weights of the multimodal features output from each feature extraction layer. Through an autonomous enhancement mechanism for feature channels, the network focuses on the discriminative features after cross-modal fusion. Subsequently, two convolutional layers are used to further process the fused features, and deconvolution operations are employed to progressively restore the image size. Finally, a sigmoid activation function is used to predict the probability value of each category to obtain the final segmentation result.

[0055] Step 120: Collect remote sensing image data corresponding to the preset area, construct a training set based on the remote sensing image data, and use the training set to iteratively train the remote sensing image semantic segmentation model.

[0056] In this embodiment, remote sensing image data corresponding to a preset area (e.g., a water pumping irrigation area) can be collected. The remote sensing image data may include optical image data and radar image data.

[0057] The optical image data involves four channels: RGB and near-infrared. The satellite's optical imaging system is a panchromatic multispectral camera with spatial resolutions of 2m and 8m, respectively. After multispectral and panchromatic fusion, the resolution is reduced to 2m. The radar image data uses a fully polarimetric stripe 1 imaging mode with a spatial resolution of 8m. To maintain consistent resolution between the radar and optical image data, bilinear interpolation can be used to resample the radar image data to 2m, and image registration methods can be employed to achieve geometric correspondence between the two.

[0058] After constructing a training set based on the aforementioned remote sensing image data, the SOANet model can be iteratively trained using this training set. Specifically, during training, the number of iterations and the initial learning rate can be set, weight decay techniques are used for regularization, the Adam adaptive learning rate gradient descent algorithm is selected as the optimizer, and the cross-entropy loss function is used.

[0059] Step 130: Obtain the target remote sensing image data corresponding to the target area, input the target remote sensing image data into the trained remote sensing image semantic segmentation model, and output the corresponding semantic segmentation result through the remote sensing image semantic segmentation model.

[0060] In this step, the target remote sensing image data can be remote sensing image data to be semantically segmented.

[0061] The technical solution provided by this invention establishes a remote sensing image semantic segmentation model using an encoder and decoder architecture, collects remote sensing image data corresponding to a preset region, constructs a training set based on the remote sensing image data, and uses the training set to iteratively train the remote sensing image semantic segmentation model to obtain target remote sensing image data corresponding to the target region. The target remote sensing image data is then input into the trained remote sensing image semantic segmentation model, and the model outputs the corresponding semantic segmentation result. This technical means can fuse information from radar and optical modalities, compensate for the shortcomings of a single remote sensing sensor, solve the problems of complementary utilization of heterogeneous data and noise suppression, and improve the accuracy of remote sensing image semantic segmentation results.

[0062] Figure 2a A flowchart of another semantic segmentation method for high-resolution remote sensing images provided in an embodiment of the present invention is shown below. Figure 2a As shown, the method includes:

[0063] Step 210: Establish a semantic segmentation model for remote sensing images using an encoder and decoder architecture.

[0064] In one embodiment of this example, Figure 2b This is a schematic diagram of the structure of an MCBAM module in an encoder, as shown below. Figure 2bAs shown, the MCBAM module includes three convolutional layers, a global average pooling layer, a multi-layer perceptron (MLP), a weighted fusion module, and a spatial attention mechanism module. Specifically, the first convolutional layer fuses radar image features and optical image features; the global average pooling layer extracts channel features from the fusion result; the MLP determines cross-channel attention weights based on the channel features; the weighted fusion module performs weighted fusion of radar image features and optical image features based on the attention weights; the spatial attention mechanism (SAM) module generates two single-channel feature maps based on the weighted fusion result; the second convolutional layer, deployed within the spatial attention mechanism module, generates a spatial attention map based on the two single-channel feature maps; and the third convolutional layer outputs the feature processing result based on the spatial attention map.

[0065] In a specific embodiment, the MCBAM module first performs preliminary feature fusion on radar image data and optical image data through a 1*1 convolutional layer, then uses a global average pooling layer to extract channel features from the fusion result, and uses MLP to calculate cross-channel attention weight values.

[0066] The attention weights can be divided into two parts, corresponding to optical image features and radar image features respectively. The weighted fusion module can weight and fuse the radar image features and optical image features according to the attention weights, enabling the model to automatically learn the relative importance between the two inputs and achieve a reasonable combination of data from different modalities. Afterwards, the SAM module can calculate the mean and maximum values ​​of the weighted fused features along the channel dimension, generating two single-channel feature maps. Then, a 7*7 convolutional layer is used to generate a spatial attention map based on the single-channel feature maps, further weighting the features in the spatial dimension to highlight important regions. Finally, the MCBAM module uses a 1*1 convolutional layer to fuse the features adjusted for channel and spatial attention to obtain the final output.

[0067] The advantage of this setup is that the MCBAM module can improve the feature representation capability of multimodal images by controlling the effectiveness of intermodal information flow, thereby suppressing noise and unnecessary information interference.

[0068] Step 220: Collect remote sensing image data corresponding to the preset area, process the remote sensing image data using a data augmentation algorithm, and perform semantic annotation on the processed image data to obtain the training set corresponding to the remote sensing image semantic segmentation model.

[0069] In this embodiment, data augmentation algorithms (such as random rotation, random flipping, random cropping, and random color perturbation) can be used to process the remote sensing image data. Among them, random color perturbation simulates different lighting and environmental conditions when different remote sensing images are acquired by changing the brightness, color temperature, and hue of the optical image.

[0070] The advantage of this setup is that it helps the model learn more generalized features, thus enabling it to better handle images in various situations in practical applications.

[0071] After performing data augmentation on remote sensing image data, targets in the image data can be labeled. Taking the remote sensing image data corresponding to a certain irrigation area as an example, semantic segmentation of rice crops can be performed to label the location information, irrigation information, etc. of rice crop sample points in the image data.

[0072] Step 230: Obtain training samples in batches from the training set as current samples.

[0073] Step 240: Input the current sample into the remote sensing image semantic segmentation model, and adjust the model parameters according to the model output results.

[0074] Step 250: Determine whether the performance of the remote sensing image semantic segmentation model meets the preset requirements. If yes, determine that the model training is complete. If no, return to step 230 to perform the operation of obtaining training samples in batches from the training set as the current samples until the performance of the model meets the preset requirements.

[0075] In this step, the performance of the remote sensing image semantic segmentation model can be evaluated using a test set based on preset performance metrics.

[0076] Step 260: Obtain the target remote sensing image data corresponding to the target area, input the target remote sensing image data into the trained remote sensing image semantic segmentation model, and output the corresponding semantic segmentation result through the remote sensing image semantic segmentation model.

[0077] The technical solution provided by this invention establishes a remote sensing image semantic segmentation model using an encoder and decoder architecture, collects remote sensing image data corresponding to a preset region, processes the remote sensing image data using a data augmentation algorithm, and performs semantic annotation on the processed image data to obtain a training set corresponding to the remote sensing image semantic segmentation model. Training samples are acquired in batches from the training set as current samples, and these current samples are input into the remote sensing image semantic segmentation model. Model parameters are adjusted based on the model output results. It is determined whether the performance of the remote sensing image semantic segmentation model meets preset requirements. If yes, the model training is considered complete; otherwise, the process returns to acquiring training samples in batches from the training set as current samples until the model performance meets the preset requirements. This method acquires target remote sensing image data corresponding to the target region, inputs the target remote sensing image data into the trained remote sensing image semantic segmentation model, and outputs corresponding semantic segmentation results through the remote sensing image semantic segmentation model. This technique can fuse information from radar and optical modalities, compensate for the shortcomings of a single remote sensing sensor, solve the problems of complementary utilization of heterogeneous data and noise suppression, and improve the accuracy of remote sensing image semantic segmentation results.

[0078] Based on the above embodiments, Figure 2c This can be a schematic diagram illustrating the effect of a semantic segmentation method for high-resolution remote sensing images in this embodiment, such as... Figure 2c As shown, assuming the target area is an irrigation area and the rice planting area is the target segmentation object, the semantic segmentation method provided in this embodiment can effectively distinguish the rice planting area within the cultivated land, achieving a better semantic segmentation effect.

[0079] Figure 3 This is a schematic diagram of a semantic segmentation device for high-resolution remote sensing images provided in an embodiment of the present invention. The device is configured in an electronic device, such as... Figure 3 As shown, the device includes: a model building module 310, a model training module 320, and a model application module 330.

[0080] The model building module 310 is used to establish a remote sensing image semantic segmentation model using an encoder and decoder architecture; the encoder in the model includes two coding branches, which are used to process radar image data and optical image data respectively;

[0081] The model training module 320 is used to collect remote sensing image data corresponding to a preset area, construct a training set based on the remote sensing image data, and use the training set to iteratively train the remote sensing image semantic segmentation model.

[0082] The model application module 330 is used to acquire target remote sensing image data corresponding to the target area, input the target remote sensing image data into the trained remote sensing image semantic segmentation model, and output the corresponding semantic segmentation result through the remote sensing image semantic segmentation model.

[0083] The technical solution provided by this invention establishes a remote sensing image semantic segmentation model using an encoder and decoder architecture, collects remote sensing image data corresponding to a preset region, constructs a training set based on the remote sensing image data, and uses the training set to iteratively train the remote sensing image semantic segmentation model to obtain target remote sensing image data corresponding to the target region. The target remote sensing image data is then input into the trained remote sensing image semantic segmentation model, and the model outputs the corresponding semantic segmentation result. This technical means can fuse information from radar and optical modalities, compensate for the shortcomings of a single remote sensing sensor, solve the problems of complementary utilization of heterogeneous data and noise suppression, and improve the accuracy of remote sensing image semantic segmentation results.

[0084] Based on the above embodiments, in the remote sensing image semantic segmentation model, each coding branch consists of five-level feature extraction layers, and a multimodal channel spatial attention module is connected after each feature extraction layer. A hollow spatial pyramid pooling module is also connected after the fourth and fifth feature extraction layers, respectively.

[0085] Each feature extraction layer includes a convolutional network, a normalization layer, an activation function layer, and a max pooling layer.

[0086] In the remote sensing image semantic segmentation model, the decoder includes a compressed excitation attention module and a convolutional layer; the compressed excitation attention module is used to fuse the final output results of each level of feature extraction layers in the encoder; the convolutional layer is used to perform convolution processing on the fused result, and to perform image restoration by deconvolution operation, and then to output the semantic segmentation result by applying an activation function based on the restored image.

[0087] The multimodal channel spatial attention module includes three convolutional layers, a global average pooling layer, a multilayer perceptron, a weighted fusion module, and a spatial attention mechanism module.

[0088] The first convolutional layer is used to fuse radar image features and optical image features; the global average pooling layer is used to extract channel features from the fusion result.

[0089] The multilayer perceptron is used to determine cross-channel attention weight values ​​based on channel features; the weighted fusion module is used to perform weighted fusion of radar image features and optical image features based on the attention weight values.

[0090] The spatial attention mechanism module is used to generate two single-channel feature maps based on the weighted fusion result; the second convolutional layer is deployed within the spatial attention mechanism module and is used to generate a spatial attention map based on the two single-channel feature maps.

[0091] The third convolutional layer is used to output the feature processing results based on the spatial attention map.

[0092] Model training module 320 includes:

[0093] The data augmentation unit is used to process the remote sensing image data using a data augmentation algorithm and to perform semantic annotation on the processed image data to obtain the training set corresponding to the remote sensing image semantic segmentation model.

[0094] An iterative training unit is used to acquire training samples in batches from the training set as current samples; input the current samples into the remote sensing image semantic segmentation model and adjust the model parameters according to the model output results; determine whether the performance of the remote sensing image semantic segmentation model meets the preset requirements; if not, return to the operation of acquiring training samples in batches from the training set as current samples until the performance of the model meets the preset requirements.

[0095] The above-described apparatus can execute the methods provided in all the foregoing embodiments of the present invention, and has the corresponding functional modules and beneficial effects for executing the above methods. Technical details not described in detail in the embodiments of the present invention can be found in the methods provided in all the foregoing embodiments of the present invention.

[0096] Figure 4 A schematic diagram of an electronic device 10 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0097] like Figure 4As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded into the RAM 13 from the storage unit 18. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0098] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0099] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processing (DSP) processors, and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as semantic segmentation methods for high-resolution remote sensing images.

[0100] In some embodiments, the semantic segmentation method for high-resolution remote sensing images may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the semantic segmentation method for high-resolution remote sensing images described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the semantic segmentation method for high-resolution remote sensing images by any other suitable means (e.g., by means of firmware).

[0101] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-chips (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0102] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0103] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, RAM, ROM, erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0104] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a cathode ray tube (CRT) or a liquid crystal display (LCD)) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0105] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0106] A computing system can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system. It addresses the shortcomings of traditional physical hosts and Virtual Private Servers (VPS) in terms of management difficulty and weak business scalability.

[0107] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0108] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A semantic segmentation method for high-resolution remote sensing images, characterized in that, The method includes: A semantic segmentation model for remote sensing images is established using an encoder and decoder architecture. The encoder in the model includes two coding branches, which are used to process radar image data and optical image data, respectively. Collect remote sensing image data corresponding to a preset area, construct a training set based on the remote sensing image data, and use the training set to iteratively train the remote sensing image semantic segmentation model; Acquire the target remote sensing image data corresponding to the target area, input the target remote sensing image data into the trained remote sensing image semantic segmentation model, and output the corresponding semantic segmentation result through the remote sensing image semantic segmentation model; In the remote sensing image semantic segmentation model, each coding branch consists of five feature extraction layers. Each feature extraction layer is followed by a multimodal channel spatial attention module. The multimodal channel spatial attention modules after the fourth and fifth feature extraction layers are respectively followed by an additional hollow spatial pyramid pooling module. Each feature extraction layer includes a convolutional network, a normalization layer, an activation function layer, and a max pooling layer. The multimodal channel spatial attention module includes three convolutional layers, a global average pooling layer, a multilayer perceptron, a weighted fusion module, and a spatial attention mechanism module. The first convolutional layer is used to fuse radar image features and optical image features; the global average pooling layer is used to extract channel features from the fusion result. The multilayer perceptron is used to determine cross-channel attention weight values ​​based on channel features; the weighted fusion module is used to perform weighted fusion of radar image features and optical image features based on the attention weight values. The spatial attention mechanism module is used to generate two single-channel feature maps based on the weighted fusion result; the second convolutional layer is deployed within the spatial attention mechanism module to generate a spatial attention map based on the two single-channel feature maps. The third convolutional layer is used to output the feature processing results based on the spatial attention map; The attention weight value comprises two parts that correspond one-to-one with optical image features and radar image features, respectively.

2. The method according to claim 1, characterized in that, In the remote sensing image semantic segmentation model, the decoder includes a compressed excitation attention module and a convolutional layer; The compressed excitation attention module is used to fuse the final output results of each feature extraction layer in the encoder. The convolutional layer is used to perform convolution processing on the fusion result and to perform image restoration using deconvolution operation. Then, based on the restored image, an activation function is used to output the semantic segmentation result.

3. The method according to claim 1, characterized in that, A training set is constructed based on the remote sensing image data, including: The remote sensing image data is processed using a data augmentation algorithm, and the processed image data is semantically labeled to obtain the training set corresponding to the remote sensing image semantic segmentation model.

4. The method according to claim 1, characterized in that, Iterative training of the remote sensing image semantic segmentation model using the training set includes: Training samples are acquired in batches from the training set and used as current samples. The current sample is input into the remote sensing image semantic segmentation model, and the model parameters are adjusted based on the model output. Determine whether the performance of the remote sensing image semantic segmentation model meets the preset requirements; If not, return to the operation of obtaining training samples in batches from the training set as the current samples until the performance of the model meets the preset requirements.

5. A semantic segmentation device for high-resolution remote sensing images, characterized in that, The device includes: The model building module is used to establish a semantic segmentation model for remote sensing images using an encoder and decoder architecture; the encoder in the model includes two coding branches, which are used to process radar image data and optical image data respectively; The model training module is used to collect remote sensing image data corresponding to a preset area, construct a training set based on the remote sensing image data, and use the training set to iteratively train the remote sensing image semantic segmentation model. The model application module is used to acquire target remote sensing image data corresponding to the target area, input the target remote sensing image data into the trained remote sensing image semantic segmentation model, and output the corresponding semantic segmentation result through the remote sensing image semantic segmentation model. In the remote sensing image semantic segmentation model, each coding branch consists of five feature extraction layers. Each feature extraction layer is followed by a multimodal channel spatial attention module. The multimodal channel spatial attention modules after the fourth and fifth feature extraction layers are respectively followed by an additional hollow spatial pyramid pooling module. Each feature extraction layer includes a convolutional network, a normalization layer, an activation function layer, and a max pooling layer. The multimodal channel spatial attention module includes three convolutional layers, a global average pooling layer, a multilayer perceptron, a weighted fusion module, and a spatial attention mechanism module. The first convolutional layer is used to fuse radar image features and optical image features; the global average pooling layer is used to extract channel features from the fusion result. The multilayer perceptron is used to determine cross-channel attention weight values ​​based on channel features; the weighted fusion module is used to perform weighted fusion of radar image features and optical image features based on the attention weight values. The spatial attention mechanism module is used to generate two single-channel feature maps based on the weighted fusion result; the second convolutional layer is deployed within the spatial attention mechanism module to generate a spatial attention map based on the two single-channel feature maps. The third convolutional layer is used to output the feature processing results based on the spatial attention map; The attention weight value comprises two parts that correspond one-to-one with optical image features and radar image features, respectively.

6. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, which enables the at least one processor to perform the semantic segmentation method for high-resolution remote sensing images according to any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the semantic segmentation method for the high-resolution remote sensing image according to any one of claims 1-4.

8. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements a semantic segmentation method for high-resolution remote sensing images according to any one of claims 1-4.

Images