Complex landform remote sensing semantic segmentation method and device, medium and product

Abstract

The invention discloses a complex landform remote sensing semantic segmentation method and device, a medium and a product, and relates to the field of remote sensing image processing, and the method comprises the steps: carrying out the feature extraction of a target remote sensing image through an encoder, and obtaining a plurality of first features of different levels; performing feature enhancement on the plurality of first features through an expert feature enhancer to obtain a plurality of second features of different levels; the expert feature intensifier performs modeling based on axial semantics and extracts semantic features in the transverse direction and the longitudinal direction respectively; performing layer-by-layer fusion on the plurality of second features according to a hierarchical sequence from deep to shallow through a sub-decoder to obtain a target feature; the sub-decoder comprises a parallel feature expert compensator and a cross sparse attention module, the feature expert compensator is used for generating intra-class spatial information compensation features according to the input of the sub-decoder, and the cross sparse attention module is used for generating key area enhancement features according to the input of the sub-decoder; and generating a segmentation result of the target remote sensing image through the segmentation head according to the target features.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of remote sensing image processing, in particular to a complex landform remote sensing semantic segmentation method, device, medium and product. BACKGROUND

[0002] Remote sensing semantic segmentation of complex landforms (such as karst landforms, landslide areas, and post-earthquake areas) is a difficult problem in the field of remote sensing image interpretation. Complex landforms exhibit large intra-class feature differences in elevation, scale, spectral reflectance, and texture, resulting in poor modeling ability of existing remote sensing semantic segmentation networks for semantic consistency of the same class. Secondly, some landform classes are highly similar to the background in spectral features or spatial distribution, making it difficult for traditional remote sensing semantic segmentation networks to effectively distinguish between landform classes and the background. Finally, complex landform regions often lack clear boundaries, and their morphology is irregular and spans a large range, and may even evolve over time, further exacerbating boundary recognition errors and inaccurate region division.

[0003] In summary, the existing remote sensing semantic segmentation method has the problem of low segmentation accuracy due to the poor modeling ability of the segmentation network for semantic consistency of the same class and the poor recognition ability of the foreground boundary. SUMMARY

[0004] The purpose of the present application is to provide a complex landform remote sensing semantic segmentation method, device, medium and product, which solves the problem of low segmentation accuracy of the existing remote sensing semantic segmentation method.

[0005] To achieve the above purpose, the present application provides the following solutions: In a first aspect, the present application provides a complex landform remote sensing semantic segmentation method, comprising: extracting features of a target remote sensing image through an encoder to obtain a feature pyramid, the feature pyramid comprising a plurality of first features at different levels; enhancing the features of the plurality of first features through an expert feature enhancer to obtain a plurality of second features at different levels; wherein the expert feature enhancer is based on axial semantic modeling and extracts semantic features in horizontal and vertical directions, and the second features are generated based on the fusion results of the two semantic features; The target feature is obtained by layer-by-layer fusion of the second features in a deep-to-shallow hierarchical order through a sub-decoder; the sub-decoder comprises a feature expert compensator and a cross-sparse attention module in parallel, the feature expert compensator is configured to generate an intra-class spatial information compensation feature according to an input of the sub-decoder, the cross-sparse attention module is configured to generate a key region enhancement feature according to the input of the sub-decoder, and an output of the sub-decoder is generated according to a fusion result of the intra-class spatial information compensation feature and the key region enhancement feature; The segmentation result of the target remote sensing image is generated according to the target feature by the segmentation head.

[0006] In a second aspect, the present application provides a computer device, comprising: a memory, a processor to store a computer program on the memory and run the computer program on the processor, and the processor executes the computer program to implement the steps of the complex terrain remote sensing semantic segmentation method in any one of the above embodiments.

[0007] In a third aspect, the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps of the complex terrain remote sensing semantic segmentation method in any one of the above embodiments.

[0008] In a fourth aspect, the present application provides a computer program product, which comprises a computer program, and the computer program is executed by a processor to implement the steps of the complex terrain remote sensing semantic segmentation method in any one of the above embodiments.

[0009] According to the specific embodiments provided by the present application, the following technical effects are disclosed: The application provides a complex topography remote sensing semantic segmentation method, device, medium and product. First, the encoder performs basic feature extraction on the target remote sensing image to output first features of different levels. Then, a plurality of expert feature enhancers perform feature enhancement on the plurality of first features respectively to output corresponding second features. The expert feature enhancer is based on axial semantic modeling and extracts semantic features in the horizontal and vertical directions respectively, can realize efficient long-distance dependency modeling, and improve the expression ability of the model to irregular morphological and large spatial span topography categories. Second, the sub-decoder performs layer-by-layer fusion on the plurality of second features in the order from deep to shallow to finally output target features. The layer-by-layer fusion can integrate feature information of different levels while considering global information and local information. Further, the feature expert compensator in the sub-decoder is used to generate intra-class spatial information compensation features according to the input of the sub-decoder, which is conducive to enhancing the model's recognition of important information within the topography category, thereby obtaining more complete information within the topography category and improving the model's modeling ability for semantic consistency of the same category. In addition, the cross-sparse attention module in the sub-decoder is used to generate key region enhancement features according to the input of the sub-decoder. Compared with the complete attention module, the cross-sparse attention module not only enables the model to pay more attention to the key region, but also eliminates background interference features, thereby improving the model's ability to distinguish the topography category from the background. Through the cooperation of the above network modules, the segmentation result generated based on the target features has high segmentation accuracy. Therefore, the steps 210 to 240 solve the problem of low segmentation accuracy of the existing remote sensing semantic segmentation method. BRIEF DESCRIPTION OF DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0011] Figure 1 A flowchart of a complex topography remote sensing semantic segmentation method in an embodiment of the present application; Figure 2 An architecture diagram of a semantic segmentation network model provided by an embodiment of the present application; Figure 3 An architecture diagram of a sub-decoder provided by an embodiment of the present application; Figure 4 An architecture diagram of an expert feature enhancer provided by an embodiment of the present application; Figure 5 An architecture diagram of a feature expert compensator provided by an embodiment of the present application; Figure 6 FIG. 1 is a schematic diagram of a cross-sparse attention module according to an embodiment of the present application. Figure 7 FIG. 2 is a schematic diagram of a computer device according to an embodiment of the present application. DETAILED DESCRIPTION

[0012] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0013] The above-mentioned purposes, features and advantages of the present application will be more apparent and understandable. The present application will be described in further detail below with reference to the drawings and specific embodiments.

[0014] The complex terrain remote sensing semantic segmentation method provided by the embodiments of the present application can be applied to a terminal and a server. The terminal communicates with the server through a network. The complex terrain remote sensing semantic segmentation method can be realized by cooperation of the terminal and the server. For example, the terminal can send a target remote sensing image to the server. After receiving the target remote sensing image, the server can feed back the obtained segmentation result to the terminal. In addition, in some embodiments, the complex terrain remote sensing semantic segmentation method can also be realized by the server or the terminal alone.

[0015] The terminal can be, but is not limited to, various desktop computers, notebook computers, smart phones, tablet computers. The server can be implemented by an independent server or a server cluster composed of multiple servers, and can also be a cloud server.

[0016] In an exemplary embodiment, as shown in FIG. 1, a complex terrain remote sensing semantic segmentation method is provided, which is executed by a computer device, specifically, can be executed by a terminal or a server, etc. computer device alone, or can be executed by a terminal and a server together. In the embodiments of the present application, the method is applied to the terminal as an example, which includes the following steps 210 to 240. Figure 2

[0017] Step 210: performing feature extraction on a target remote sensing image by an encoder to obtain a feature pyramid, the feature pyramid including a plurality of first features of different levels.

[0018] ​At step 220, the first features are enhanced by expert feature enhancers respectively to obtain second features of different levels; wherein the expert feature enhancers are based on axial semantic modeling and extract semantic features in horizontal and vertical directions respectively, and the second features are generated according to fusion results of the two semantic features.

[0019] At step 230, the second features are fused by a sub-decoder in a layer-by-layer order from deep to shallow to obtain target features; wherein the sub-decoder includes a feature expert compensator and a cross-sparse attention module in parallel, the feature expert compensator is used to generate intra-class spatial information compensation features according to inputs of the sub-decoder, the cross-sparse attention module is used to generate key region enhancement features according to the inputs of the sub-decoder, and the outputs of the sub-decoder are generated according to fusion results of the intra-class spatial information compensation features and the key region enhancement features.

[0020] At step 240, a segmentation result of the target remote sensing image is generated by a segmentation head according to the target features.

[0021] In the embodiment, the layer-by-layer fusion of the second features by the sub-decoder in the order from deep to shallow specifically includes: firstly, the second features of the deepest level are fused into the second features of the second deepest level by a sub-decoder, then the second features of the second deepest level are fused into the second features of the third deepest level by a sub-decoder, and so on, and finally, the second features of the second shallowest level are fused into the second features of the shallowest level by a sub-decoder.

[0022] In essence, the innovation of the above steps 210 to 240 lies in that a new semantic segmentation network model (LENet) is adopted to realize semantic segmentation of the target remote sensing image. As shown in Figure 2 , the new semantic segmentation network model includes an encoder and a decoder, the encoder is composed of a backbone network, and the decoder includes multiple expert feature enhancers (EEBlock) and multiple sub-decoders (Decoder Block). As shown in Figure 3 , the sub-decoder includes a feature expert compensator (FEC) and a cross-sparse attention module (CSA) in parallel. After obtaining the target remote sensing image, it is input into the semantic segmentation network model, and the segmentation result can be obtained through the semantic segmentation network model.

[0023] Firstly, the encoder performs basic feature extraction on the target remote sensing image to output first features of different levels; then, a plurality of expert feature enhancers respectively perform feature enhancement on the plurality of first features to output corresponding second features; wherein, the expert feature enhancers are based on axial semantic modeling and extract semantic features in the horizontal and vertical directions respectively, which can realize efficient long-distance dependency modeling and improve the expression ability of the model for irregular morphological categories with large spatial span; secondly, the target feature is finally output by layer-by-layer fusion of a plurality of second features in the order from deep to shallow by a sub-decoder, and the layer-by-layer fusion can integrate feature information of different levels while taking into account global information and local information. Further, the feature expert compensator in the sub-decoder is used to generate intra-class spatial information compensation features according to the input of the sub-decoder, which is conducive to enhancing the model's recognition of important information within the landform category, thereby obtaining more complete information within the landform category and improving the model's ability to model semantic consistency of the same category. In addition, the cross-sparse attention module in the sub-decoder is used to generate key region enhancement features according to the input of the sub-decoder. Compared with the complete attention module, the cross-sparse attention module not only allows the model to pay more attention to the key region, but also eliminates background interference features, thereby improving the model's ability to distinguish between landform categories and backgrounds. Through the cooperation of the above network modules, the segmentation result generated based on the target feature has high segmentation accuracy. Therefore, the steps 210 to 240 solve the problem of low segmentation accuracy of existing remote sensing semantic segmentation methods.

[0024] For example, referring to Figure 2 , the number of levels of the first features and the second features is 4. In this example, the encoder includes 4 Swin Block modules stacked in sequence, the output of a previous Swin Block module is input into a next Swin Block module after being down-sampled, and the first features output by the 4 Swin Block modules stacked in sequence are defined as S1, S2, S3 and S4 respectively. The sizes of the 4 first features are one quarter, one eighth, one sixteenth and one thirty-second of the target remote sensing image in sequence. Correspondingly, the number of expert feature enhancers is also 4, and the second features output by the expert feature enhancers are defined as F1, F2, F3 and F4 respectively. Then, in the process of layer-by-layer fusion, F4 is first fused into F3 by a sub-decoder to obtain F`3, then F`3 after being up-sampled is fused into F2 by a sub-decoder to obtain F`2, and finally F`2 after being up-sampled is fused into F1 to obtain the target feature.

[0025] It should be noted that the above is only an example of the number of levels of the first features and the second features, which can be enhanced or reduced according to actual conditions, such as reducing to 3 or increasing to 5.

[0026] For the expert feature enhancer, referring to Figure 4 , exemplarily, it first processes the input first feature S i through a depthwise separable convolution DWConv to obtain three-dimensional semantic features F l , F v and F h respectively. Among them, F v and F h are transverse semantic features and longitudinal semantic features respectively, the transverse semantic features are obtained through vertical feature extraction, and the longitudinal semantic features are obtained through horizontal feature extraction. Then, the three-dimensional semantic features are fused and sequentially processed through 1×1 convolution, batch normalization BN and ReLU activation (corresponding to Conv1×1, BN and ReLU in Figure 4 F i respectively) to obtain the second feature

[0027] Referring to Figure 3 , exemplarily, the intra-class spatial information compensation feature and the key region enhancement feature are fused through a cosine similarity mechanism. Specifically, different levels of second features F i and F` i+1 are simultaneously input into the feature expert compensator and the cross sparse attention module (in the deepest level sub-decoder, different levels of second features are F i and F i+1 ). The outputs of the feature expert compensator and the cross sparse attention module are fused through the cosine similarity mechanism, the fusion result is first multiplied by the output of the feature expert compensator and then added to the output of the cross sparse attention module to form a first intermediate feature, the first intermediate feature is sequentially subjected to ReLU activation, average pooling, 1×1 convolution, ReLU activation, 1×1 convolution and Sigmoid activation (corresponding to ReLU, AvgPool, Conv1x1, ReLU, Conv1x1 and Sigmoid in Figure 3 F` i respectively) to form a second intermediate feature, and the second intermediate feature is multiplied by the first intermediate feature to form the second feature

[0028] ​​In this embodiment, refer to Figure 5 The Feature Expert Compensator (FAC) comprises channel branching and spatial branching. Through a parallel mechanism that separates channel weights from spatial weights, the FAC guides the model to focus on the more semantically discriminative feature channels and compensates for spatial information in hard-to-learn regions of the spatial features.

[0029] For example, generating intra-class spatial information compensation features based on the input of the sub-decoder specifically includes: (1) The input of the sub-decoder contains feature 1 and feature 2 (e.g., second features at different levels). F i and F ` i+1 The first fusion feature F is obtained by fusing the features together. F .

[0030] (2) For the first fusion feature F F After performing adaptive average pooling and adaptive max pooling respectively, the two pooling results are concatenated to obtain the channel weight C. A .

[0031] (3) For the first fusion feature F F Perform maximum and average value calculations separately, and then concatenate the two results to obtain the concatenated feature F. S F S =Concat(Mean(F F ),Max(F F )).

[0032] (4) For splicing feature F S Perform convolution to generate a spatial offset Off, Off = Conv(F S ), for splicing feature F S Deformable convolution DConv with spatial offset Off generates intra-class spatial feature compensation weights F A FA=DConv(F S ,Off).

[0033] (5) Based on the channel weight C A Intra-class spatial feature compensation weight F A Generate intra-class spatial information compensation features using features 1 and 2. This step specifically includes: sequentially combining features 1 and 2 from the input of the sub-decoder with the channel weights C. A Intra-class spatial feature compensation weight F A Multiply the results to obtain two multiplication results; add the two multiplication results together to obtain the intra-class spatial information compensation feature.

[0034] Through the above mechanism, the feature expert compensator can balance the channel relationship after multi-scale feature fusion and reduce the problem of intra-class information loss in spatial dimension, thereby obtaining more accurate intra-class feature recognition.

[0035] It should be noted that the intra-class spatial feature compensation weight F is generated. A The deformable convolution DConv differs from traditional convolutions in that it learns spatial offsets from spatial information features through an additional convolutional layer. This convolutional layer has 2×K input channels (where K is the kernel size; for example, K is 9 in a 3×3 convolution). The offset at each position includes both horizontal and vertical directions, and the offsets are optimized through backpropagation during training, allowing the convolutional kernel to adaptively adjust its sampling position based on the input features. Traditional convolutions, for a given input feature map... x Its output feature map y of p The formula for calculating position 0 is as follows: In the formula, w Represents the convolution kernel weights. x For input features in a local neighborhood ( p n The weighted sum of the terms allows convolution to focus on detailed information within the neighborhood, but it cannot weight sample each region within that neighborhood. Deformable convolution, on the other hand, introduces a sampling position offset Δ into the convolution process. p n Δ p n Generated based on spatial offset Off, its sampling position becomes p 0+ p n +Δ p n This solves the problem of traditional convolution failing to focus on specific regions within the neighborhood. Therefore, deformable convolution, for a given input feature map... x Its output feature map y of p The formula for calculating position 0 is as follows: The cross-sparse attention module used in this embodiment differs from traditional cross-attention modules, which rely on the complete attention mask map for computation. Instead, it introduces multi-sparse rate masks and weights them during computation, retaining masks with higher weights (i.e., those containing category information) while including irrelevant background noise in masks with lower weights. This reduces computational complexity and suppresses interference from irrelevant features. This design effectively weakens noise interference in the attention weights, enhances the model's selective learning of semantic information, and thus improves the ability to discriminate complex terrain regions.

[0036] With reference to Figure 6 In this embodiment, the key region enhancement feature is generated according to the input of the sub-decoder, specifically including: (1) multiplying the query and the key to obtain a complete attention map and adding relative position encoding to the complete attention map.

[0037] (2) screening the elements in the complete attention map through a plurality of mask matrices of different sparsity rates respectively to obtain a plurality of sparse attention maps of different sparsity rates.

[0038] (3) multiplying a plurality of sparse attention maps with values respectively and then performing weighted multiplication fusion, and generating a key region enhancement feature based on the fusion result.

[0039] Among them, the query is generated according to one of the second features in the input of the sub-decoder, and the key and the value are generated according to another second feature in the input of the sub-decoder.

[0040] Exemplarily, there are four mask matrices of different sparsity rates. Feature 1 and feature 2 in the input of the sub-decoder are respectively integrated in the channel through normalization BN and standard convolution module (Conv-BN-ReLU), and then mapped through a convolution layer with a size of 1×1 to generate query Q , key K and value V as inputs of the cross sparse attention module. Among them, feature 1 generates Q , feature 2 generates K and V . The complete attention map is obtained by multiplying Q with K , and relative position encoding is added to construct the spatial relationship between the features, and then the elements of the complete attention map are multiplied with V using four sparsity rate mask matrices, and finally the output feature is obtained by weighted multiplication fusion of the attention maps of each sparsity rate, and the output feature is normalized and nonlinearly processed by the standard convolution group to obtain the key region enhancement feature, which enhances the expression ability of the related features. In this example, the cross sparse attention module can be defined as follows: Among them, Q represents the query token generated by feature 1, K , V represent the key token and the value token generated by feature 2, B represents the relative position encoding, T i represents the i-th sparsity rate matrix, ω i represents the attention weight corresponding to the i-th sparsity rate matrix.

[0041] As above, it is a complete example of the complex terrain remote sensing semantic segmentation method in this embodiment. The effectiveness of the method is verified by a test example as follows.

[0042] This test example evaluates the semantic segmentation network model (LENet) used by the complex terrain remote sensing semantic segmentation method in a karst data set and a landslide data set. Karst landform development is often related to faults, folds and other structures, and its distribution has obvious discontinuity and irregularity, with complex terrain features. Landslides often occur in areas with weak geological conditions or steep terrain, and their distribution is patchy, fragmented and uneven. Moreover, multiple and repeated compound landslides may occur in the same area, and the landform pattern shows strong dynamic evolution characteristics, also with complex terrain features.

[0043] This test example is extended on the basis of the karst data set PKLD. The extended PKLD contains 17720 karst landform images of 256x256, of which the proportions of the training set, test set and validation set are 6:2:2, i.e. 10632 training set images, 3544 validation set images and 3544 test set images. This area is located in Gongjue County, Changdu City, Tibet Autonomous Region, China, and the surface karst landform is mainly composed of limestone peaks and limestone debris accumulation. Considering the wide distribution of karst landforms, the data set selects different mountain backgrounds, including karst in bare mountains, karst covered with vegetation and karst in obvious stone forest areas.

[0044] This test example uses the landslide public data set GVLM. The GVLM data set contains a large number of landslides in different countries and regions of Asia, Africa, North America, South America, Europe and Oceania, which can well meet the segmentation task of complex landslide terrain under different background conditions. This test example selects 2690 images of 256x256 size from this data set and uses random seeds to divide them into training set, test set and validation set in the ratio of 6:2:2 to prove the generalization ability of the segmentation model in the complex landslide terrain segmentation task.

[0045] The experiments of this test example are implemented under the PyTorch framework and run on a workstation equipped with an NVIDIA GeForce RTX5080 GPU (16 GB video memory). The input image size is set to 256x256 pixels, the batch size is 16, and after multiple experiments, the training round is selected as 150 rounds to ensure that the segmentation model does not overfit or underfit. The optimizer uses AdamW, and the initial learning rate is set to 0.00006. In the training phase, random flipping and photometric distortion data augmentation strategies are used to improve the generalization ability of the segmentation model.

[0046] The technical method implementation process is as follows: (1) Data preparation and preprocessing: In this test example, both of the two different complex terrain data sets are cropped to 512x512 size, and the training, testing and validation set ratio is 6:2:2.

[0047] (2) Use the pre-trained encoder Swin Transformer-tiny to extract the deep semantic information of the original data, and input it into the decoder for further processing.

[0048] (3) Input the multi-scale features input in the encoder into the decoder stage, and first enter the expert feature enhancer for axial semantic information enhancement. The enhanced multi-scale features are sequentially input into the sub-decoder from deep to shallow.

[0049] (4) In the sub-decoder, the input multi-scale features are input into the cross-sparse attention module and the feature expert compensator in parallel, respectively, to remove background noise and focus on important areas within the class.

[0050] (5) Perform per-pixel class prediction on the final features processed by the decoder to achieve pixel-level segmentation.

[0051] In order to prove that the complex terrain remote sensing semantic segmentation method in the above embodiment has effect on the semantic segmentation task of remote sensing images containing highland karst terrain and landslide disaster and other complex terrains, this test example compares a plurality of advanced semantic segmentation models in the past five years.

[0052] Comprehensive comparison of various advanced semantic segmentation models, the comparison results are shown in Table 1.

[0053] Table 1 Comparison of segmentation effects of different semantic segmentation models

[0054] As can be seen from Table 1, the semantic segmentation network model (LENet) used in this embodiment achieves excellent results in the comprehensive indicators F1, IoU (intersection over union) and mIoU (mean intersection over union), and has excellent segmentation efficiency, which is sufficient to prove the progress of Lenet in the complex terrain segmentation task.

[0055] LENet obtains more complete intra-class information through the feature expert compensator (FEC) for the identification of intra-class key information in the fusion stage, so that the F1 and IoU values are significantly improved. Compared with the model Swin Transformer (UperNet decoder), the FLOPS (floating point operations per second) of LENet is reduced by 46.541G, and the total FLOPS is only about 22.4% of Swin Transformer. At the same time, the Params (total number of weights and biases determined by learning) is reduced by 10.619M, and the total Params is about 82% of it. Among them, the expert feature enhancer (EEBlock) using strip convolution to simulate attention mechanism significantly reduces the computational complexity of long-distance feature modeling, and the cross-sparse attention (CSA) using sparse matrix suppresses the input of irrelevant background noise in the multi-scale feature fusion stage, realizing more efficient segmentation of complex terrain remote sensing images.

[0056] In an exemplary embodiment, a computer device, which can be a server or a terminal, is provided, and an internal structure diagram thereof can be as shown in Figure 7 The computer device includes a processor, a memory, an input / output interface (I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used for target remote sensing images and their segmentation results. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals through network connection. The computer program is executed by the processor to implement the complex terrain remote sensing semantic segmentation method.

[0057] Those skilled in the art can understand that Figure 7 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0058] In an exemplary embodiment, a computer device is also provided, including a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps in each of the above method embodiments.

[0059] In an example embodiment, a computer readable storage medium storing a computer program is provided, the computer program, when executed by a processor, implements the steps of any of the above method embodiments.

[0060] In an example embodiment, a computer program product is provided, comprising a computer program, the computer program, when executed by a processor, implements the steps of any of the above method embodiments.

[0061] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant regulations.

[0062] It can be understood by those skilled in the art that all or part of the processes in the above-mentioned embodiments can be completed by a computer program instructing related hardware, and the computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. Any reference to memory, database or other medium used in the embodiments provided by the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (Read-Only Memory, ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetoresistive memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric memory (Ferroelectric Random Access Memory, FRAM), phase change memory (Phase Change Memory, PCM), graphene memory, etc. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, etc. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), etc.

[0063] The database involved in each of the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a blockchain, and the like, without being limited thereto. The processor involved in each of the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, and the like, without being limited thereto.

[0064] The technical features of the above embodiments can be combined in any manner. To make the description concise, all possible combinations of the technical features in the above embodiments are not described, but it should be considered that any combination of the technical features is within the scope of the present disclosure, as long as there is no contradiction.

[0065] The principles and implementation manners of the present application are described by using specific examples herein, and the above embodiments are only used to help understand the method of the present application and its core idea. Meanwhile, for those skilled in the art, the specific implementation manners and application ranges can be changed according to the idea of the present application. In summary, the content of the present description should not be understood as a limitation of the present application.

Claims

1. A complex terrain remote sensing semantic segmentation method, characterized in that, The method comprises the following steps: feature extraction is performed on a target remote sensing image by an encoder to obtain a feature pyramid, the feature pyramid comprising a plurality of first features at different levels; feature enhancement is performed on the first features by an expert feature enhancer to obtain a plurality of second features at different levels, wherein the expert feature enhancer is based on axial semantic modeling and extracts semantic features in the horizontal and vertical directions respectively, and the second features are generated according to the fusion results of the two semantic features; a target feature is obtained by layer-by-layer fusion of the second features in the order from deep to shallow by a sub-decoder, wherein the sub-decoder comprises a feature expert compensator and a cross-sparse attention module in parallel, the feature expert compensator is used to generate intra-class spatial information compensation features according to the input of the sub-decoder, the cross-sparse attention module is used to generate key region enhancement features according to the input of the sub-decoder, and the output of the sub-decoder is generated according to the fusion results of the intra-class spatial information compensation features and the key region enhancement features; a segmentation result of the target remote sensing image is generated by a segmentation head according to the target feature.

2. The complex terrain remote sensing semantic segmentation method according to claim 1, characterized in that, The number of levels of the first features and the second features is 4.

3. The complex terrain remote sensing semantic segmentation method according to claim 1, characterized in that, The intra-class spatial information compensation features and the key region enhancement features are fused by a cosine similarity mechanism.

4. The complex terrain remote sensing semantic segmentation method according to claim 1, characterized in that, The feature expert compensator comprises a channel branch and a spatial branch. The intra-class spatial information compensation features are generated according to the input of the sub-decoder, specifically comprising: the input of the sub-decoder is fused to obtain first fusion features; the first fusion features are respectively subjected to adaptive average pooling and adaptive maximum pooling, and the two pooling results are spliced to obtain channel weights; the first fusion features are respectively subjected to maximum value calculation and average value calculation, and the two calculation results are spliced to obtain spliced features; the spliced features are subjected to convolution to generate spatial offsets, and the spliced features and the spatial offsets are subjected to deformable convolution to generate intra-class spatial feature compensation weights; the intra-class spatial information compensation features are generated according to the channel weights, the intra-class spatial feature compensation weights and the input of the sub-decoder.

5. The method of claim 4, wherein, The intra-class spatial information compensation features are generated according to the channel weights, the intra-class spatial feature compensation weights and the input of the sub-decoder, specifically comprising: two second features in the input of the sub-decoder are multiplied with the channel weights and the intra-class spatial feature compensation weights respectively in sequence to obtain two multiplication results; the two multiplication results are added to obtain the intra-class spatial information compensation features.

6. The method of claim 1, wherein, The key region enhancement features are generated according to the input of the sub-decoder, specifically comprising: query and key are multiplied to obtain a complete attention map, and relative position encoding is added to the complete attention map; elements in the complete attention map are screened by a plurality of mask matrices with different sparsity rates to obtain a plurality of sparse attention maps with different sparsity rates; the plurality of sparse attention maps are multiplied with values respectively, and then weighted multiplication fusion is performed, and the key region enhancement features are generated based on the fusion results; The query is generated according to one of the second features in the input of the sub-decoder, and the key and value are generated according to another of the second features in the input of the sub-decoder.

7. The method of claim 6, wherein the complex terrain remote sensing semantic segmentation is performed by a convolutional neural network. The mask matrices with different sparseness rates have four in common.

8. A computer device comprising: Memory, a processor, and a computer program stored on the memory and capable of running on the processor, characterized in that the processor executes the computer program to implement the complex terrain remote sensing semantic segmentation method of any one of claims 1-7.

9. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the complex terrain remote sensing semantic segmentation method of any one of claims 1-7.

10. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the complex terrain remote sensing semantic segmentation method of any one of claims 1-7. The computer program is executed by the processor to implement the complex terrain remote sensing semantic segmentation method of any one of claims 1-7.

Images