Semantic segmentation method for optical remote sensing image

Abstract

The invention provides a semantic segmentation method for an optical remote sensing image, and relates to the technical field of image processing, and the method comprises the steps: carrying out the down-sampling processing of a to-be-processed optical remote sensing image of a target pixel size through a down-sampling module, obtaining a first feature map, and obtaining a first normalized feature; performing linear attention processing on the first normalized feature by adopting a linear attention module, and normalizing the target global feature by adopting a second normalization layer to obtain a second normalized feature; performing feature mapping by adopting a multi-layer perceptron to obtain a second feature map with the same pixel size as the first feature map; performing up-sampling processing on the second feature map by adopting an up-sampling module to obtain a target feature map of a target pixel size; and performing feature segmentation on the target feature map by using a feature segmentation module to obtain a semantic segmentation map of the optical remote sensing image. The semantic segmentation calculation accuracy is improved, the calculation complexity is reduced, and the requirement for calculation resources is lowered.

Description

Technical Field

[0001] The present application relates to the technical field of image processing, and in particular to a semantic segmentation method for optical remote sensing images. Background Art

[0002] Semantic segmentation of optical remote sensing images is a crucial task in the field of remote sensing and computer vision. Its goal is to assign a corresponding category label to each pixel and generate a fine pixel-level segmentation map. This technology is of great value in many application fields such as urban planning, environmental monitoring, and disaster assessment. However, since remote sensing images often contain complex ground structures, diverse scale changes, and rich detail information, achieving high-precision segmentation faces huge challenges.

[0003] At present, deep learning technology is mainly used to segment optical remote sensing images. Convolution operations can capture local spatial relationships in images, such as edges, corners, and textures, which are crucial information in image segmentation. Through multiple layers of convolution and pooling, the network can gradually abstract more complex features, thereby identifying objects in images at different scales.

[0004] However, deep learning technology has limited performance in capturing the global spatial information of images and processing long-distance dependencies, especially in high-resolution remote sensing image scenarios, where computational efficiency issues are particularly prominent. It is impossible to guarantee image processing accuracy and reduce computational complexity while processing high-resolution remote sensing images. Summary of the invention

[0005] The purpose of this application is to provide a semantic segmentation method for optical remote sensing images in order to address the deficiencies in the above-mentioned prior art, thereby improving the accuracy of semantic segmentation calculations, reducing calculation complexity, and lowering the demand for computing resources.

[0006] To achieve the above purpose, the technical solution adopted in the embodiment of the present application is as follows:

[0007] In a first aspect, an embodiment of the present application provides a semantic segmentation method for an optical remote sensing image, the method comprising:

[0008] Applied to an image segmentation network model, the image segmentation network model includes: a feature extraction module and a feature segmentation module, the feature extraction module includes: a downsampling module, a linear attention network and an upsampling module, the linear attention network includes: a first normalization layer, a linear attention module, a second normalization layer and a multilayer perceptron, the method includes:

[0009] The downsampling module is used to downsample the optical remote sensing image of the target pixel size to be processed to obtain a first feature map;

[0010] Using the first normalization layer to normalize the first feature map to obtain a first normalized feature;

[0011] Using the linear attention module to perform linear attention processing on the first normalized feature to obtain a target global feature with context information;

[0012] Normalizing the target global feature using the second normalization layer to obtain a second normalized feature;

[0013] Performing feature mapping on the second normalized features using the multilayer perceptron to obtain a second feature map having the same pixel size as the first feature map;

[0014] The upsampling module is used to perform upsampling processing on the second feature map to obtain a target feature map of the target pixel size;

[0015] The feature segmentation module is used to perform feature segmentation on the target feature map to obtain a semantic segmentation map of the optical remote sensing image.

[0016] Optionally, the linear attention module includes: a linear processing layer, a first aggregation layer, a second aggregation layer, a third aggregation layer, a fourth aggregation layer, a first attention calculation layer, and a second attention calculation layer; the linear attention module is used to perform linear processing on the first normalized feature to obtain a target global feature with context information, including:

[0017] Using the linear processing layer to perform linear processing on the first normalized feature to obtain a first linear query vector, a first linear key vector, and a first linear value vector corresponding to the first linear key vector;

[0018] Using the first aggregation layer to perform aggregation calculation on a first global feature corresponding to the first linear query vector and the first linear key vector to obtain a first aggregated feature with context information;

[0019] Using the first attention calculation layer, perform attention calculation on the first aggregated features to obtain a first attention matrix;

[0020] Using the second aggregation layer to perform aggregation calculation on the first attention matrix and the first linear value vector to obtain a second global feature;

[0021] Using the third aggregation layer to perform aggregation calculation on the activation function feature of the first linear query vector and the first global feature to obtain a second aggregate feature;

[0022] Using the second attention calculation layer, performing attention calculation on the second aggregated features to obtain a second attention matrix;

[0023] The fourth aggregation layer is used to perform aggregation calculation on the second attention matrix and the first global feature to obtain the target global feature.

[0024] Optionally, the downsampling module comprises: a plurality of convolutional layers, the plurality of convolutional layers respectively corresponding to a plurality of pixel sizes;

[0025] The downsampling module is used to process the optical remote sensing image of the target pixel size to obtain a first feature map, including:

[0026] Using the multiple convolutional layers to sequentially perform convolution processing on the optical remote sensing images to obtain first feature maps of the multiple pixel sizes;

[0027] The using the first normalization layer to perform normalization processing on the first feature map to obtain a first normalized feature includes:

[0028] The first normalization layer is used to normalize the first feature map of the minimum pixel size to obtain the first normalized feature.

[0029] Optionally, the upsampling module includes: a plurality of deconvolution layers; the plurality of deconvolution layers correspond to the plurality of pixel sizes respectively; the upsampling process of the second feature map by the upsampling module to obtain the target feature map of the target pixel size includes:

[0030] The second feature map is sequentially deconvolved using the multiple deconvolution layers to obtain third feature maps of the multiple pixel sizes, and the third feature map of the maximum pixel size is determined as the target feature map.

[0031] Optionally, the up-sampling module further includes: a plurality of gating modules, the plurality of gating modules respectively corresponding to the plurality of pixel sizes, and the method further includes:

[0032] Using each gating module, performing feature fusion on the first feature map and the second feature map of the corresponding pixel size of each gating module to obtain a fused feature map of the corresponding pixel size;

[0033] The step of sequentially performing deconvolution processing on the second feature map using the multiple deconvolution layers to obtain third feature maps of the multiple pixel sizes, and determining the third feature map of the maximum pixel size as the target feature map includes:

[0034] Each deconvolution layer is used to perform deconvolution processing on the fused feature map of the corresponding pixel size to obtain a third feature map of the next pixel size.

[0035] Optionally, each of the gating modules includes: a first mixing unit, a second mixing unit, a weight function unit, and a connection function unit; using each of the gating modules to perform feature fusion on the first feature map and the second feature map of the corresponding pixel size of each of the gating modules to obtain the fused feature map of the corresponding pixel size includes:

[0036] Using the weight function unit to perform weight calculation on the first feature map and the second feature map of the corresponding pixel size to obtain a first weight parameter;

[0037] Using the first mixing unit, performing weighted calculation according to the first weight parameter and the first feature map of the corresponding pixel size to obtain a first weighted feature map;

[0038] Using the second mixing unit, performing weighted calculation according to the second weight parameter corresponding to the first weight parameter and the second feature map of the corresponding pixel size, to obtain a second weighted feature map; wherein the sum of the first weight parameter and the second weight parameter is 1;

[0039] The first weighted feature map and the second weighted feature map are concatenated using the connection function unit to obtain a fused feature map of the corresponding pixel size.

[0040] Optionally, each of the gating modules includes: a first convolution unit, a second convolution unit, a first mixing unit, a second mixing unit, a weight function unit and an addition unit;

[0041] The method of using each gating module to perform feature fusion on the first feature map and the second feature map of the corresponding pixel size of each gating module to obtain the fused feature map of the corresponding pixel size includes:

[0042] Using the first convolution unit and the second convolution unit to perform convolution processing on the first feature map and the second feature map of the corresponding pixel size respectively, to obtain a first mapping feature and a second mapping feature;

[0043] Using the weight function unit to perform weight calculation on the first mapping feature and the second mapping feature to obtain a first weight parameter;

[0044] Using the first mixing unit, performing weighted calculation according to the first weight parameter and the first mapping feature to obtain a first weighted feature map;

[0045] Using the second mixing unit, performing weighted calculation according to a second weight parameter corresponding to the first weight parameter and the second mapping feature to obtain a second weighted feature map; wherein the sum of the first weight parameter and the second weight parameter is 1;

[0046] The adding unit is used to perform addition processing on the first weighted feature map and the second weighted feature map to obtain the fused feature map of the corresponding pixel size.

[0047] Optionally, each of the gating modules includes: a first convolution unit, a second convolution unit, a first mixing unit, a second mixing unit, a weight function unit and a connection function unit;

[0048] The method of using each gating module to perform feature fusion on the first feature map and the second feature map of the corresponding pixel size of each gating module to obtain the fused feature map of the corresponding pixel size includes:

[0049] Using the first convolution unit and the second convolution unit to perform convolution processing on the first feature map and the second feature map of the corresponding pixel size respectively, to obtain a first mapping feature and a second mapping feature;

[0050] Using the weight function unit to perform weight calculation on the first mapping feature and the second mapping feature to obtain a first weight parameter;

[0051] Using the first mixing unit, performing weighted calculation according to the first weight parameter and the first feature map of the corresponding pixel size to obtain a first weighted feature map;

[0052] Using the first mixing unit, performing weighted calculation according to a second weight parameter corresponding to the first weight parameter and the second feature map of the corresponding pixel size, to obtain a second weighted feature map; wherein the sum of the first weight parameter and the second weight parameter is 1;

[0053] The first weighted feature map and the second weighted feature map are concatenated using the connection function unit to obtain a fused feature map of the corresponding pixel size.

[0054] Optionally, the feature segmentation module includes: a feature mapping unit and a generation unit;

[0055] The step of using the feature segmentation module to perform feature segmentation on the target feature map to obtain a semantic segmentation map of the optical remote sensing image includes:

[0056] The feature mapping unit is used to perform a convolution operation on the target feature map according to a preset number of categories to obtain a new feature map, wherein the features of each pixel in the new feature map have a corresponding category label and the probability of the corresponding category;

[0057] The generating unit is used to generate the semantic segmentation map according to the new feature map.

[0058] Optionally, before the downsampling module is used to downsample the optical remote sensing image of the target pixel size to be processed to obtain the first feature map, the method further includes:

[0059] The original remote sensing image is cropped to obtain an optical remote sensing image of the target pixel size.

[0060] In a second aspect, another embodiment of the present application provides a semantic segmentation device for optical remote sensing images, which is applied to an image segmentation network model, wherein the image segmentation network model includes: a feature extraction module and a feature segmentation module, wherein the feature extraction module includes: a downsampling module, a linear attention network, and an upsampling module, wherein the linear attention network includes: a first normalization layer, a linear attention module, a second normalization layer, and a multilayer perceptron, wherein the device includes:

[0061] A first sampling module, configured to use the downsampling module to perform downsampling processing on the optical remote sensing image of the target pixel size to be processed, so as to obtain a first feature map;

[0062] A first processing module, configured to perform normalization processing on the first feature map by using the first normalization layer to obtain a first normalized feature;

[0063] A second processing module, configured to perform linear attention processing on the first normalized feature using the linear attention module to obtain a target global feature with context information;

[0064] A third processing module, configured to normalize the target global feature by using the second normalization layer to obtain a second normalized feature;

[0065] A mapping module, configured to perform feature mapping on the second normalized feature using the multilayer perceptron to obtain a second feature map having the same pixel size as the first feature map;

[0066] A second sampling module, configured to perform upsampling processing on the second feature map by using the upsampling module to obtain a target feature map of the target pixel size;

[0067] A segmentation module is used to perform feature segmentation on the target feature map using the feature segmentation module to obtain a semantic segmentation map of the optical remote sensing image.

[0068] In the third aspect, another embodiment of the present application provides a computer device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor communicates with the memory through the bus, and the processor executes the machine-readable instructions to perform the steps of any method described in the first aspect above.

[0069] In a fourth aspect, another embodiment of the present application provides a storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of any method described in the first aspect are executed.

[0070] The beneficial effects of this application are:

[0071] The present application provides a semantic segmentation method for optical remote sensing images, which is applied to an image segmentation network model, wherein a downsampling module is used to downsample an optical remote sensing image of a target pixel size to be processed, and a first feature map is obtained. A first normalization layer is used to normalize the first feature map, and a first normalized feature is obtained. A linear attention module is used to perform linear attention processing on the first normalized feature, and a target global feature with context information is obtained. A second normalization layer is used to normalize the target global feature, and a second normalized feature is obtained. A multi-layer perceptron is used to perform feature filming on the second normalized feature, and a second feature map with the same pixel size as the first feature map is obtained. An upsampling module is used to upsample the second feature map, and a target feature map of the target pixel size is obtained. A feature segmentation module is used to perform feature segmentation on the target feature map, and a semantic segmentation map of the optical remote sensing image is obtained. The present application includes a linear attention module that performs linear processing on the first normalized feature, which can effectively process long-distance dependencies in optical remote sensing images, and uses complex spatial relationships with multi-scale semantic segmentation tasks, thereby improving calculation accuracy, reducing calculation complexity, and reducing the demand for calculation resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0073] Figure 1 A schematic diagram of the structure of the first image segmentation network model provided in an embodiment of the present application;

[0074] Figure 2 A flowchart of a semantic segmentation method for optical remote sensing images provided in an embodiment of the present application;

[0075] Figure 3 A schematic diagram of the structure of an image segmentation network model provided in an embodiment of the present application;

[0076] Figure 4 A schematic diagram of a process for determining global features of a target in a semantic segmentation method of an optical remote sensing image provided in an embodiment of the present application;

[0077] Figure 5 A schematic diagram of the structure of a downsampling module of an image segmentation network model provided in an embodiment of the present application;

[0078] Figure 6 A schematic diagram of a process for determining a first normalized feature in a semantic segmentation method for an optical remote sensing image provided in an embodiment of the present application;

[0079] Figure 7 A schematic diagram of the structure of an upsampling module in an image segmentation network model provided in an embodiment of the present application;

[0080] Figure 8 A schematic diagram of the structure of an upsampling module in another image segmentation network model provided in an embodiment of the present application;

[0081] Fig. 9 A schematic diagram of a process for determining a third feature map in a semantic segmentation method for a remote sensing image provided in an embodiment of the present application;

[0082] Fig.10 A schematic diagram of the structure of a gating module in a first image segmentation network model provided in an embodiment of the present application;

[0083] Fig.11 A schematic diagram of a process for determining a fusion feature map in a semantic segmentation method for an optical remote sensing image provided in an embodiment of the present application;

[0084] Fig.12 A schematic diagram of the structure of a gating module in a second image segmentation network model provided in an embodiment of the present application;

[0085] Fig.13 A schematic diagram of a process for determining a fusion feature map in a second method for semantic segmentation of optical remote sensing images provided in an embodiment of the present application;

[0086] Fig.14 A schematic diagram of the structure of a gating module in a third image segmentation network model provided in an embodiment of the present application;

[0087] Fig.15 A schematic diagram of a process for determining a fused feature map in a third method for semantic segmentation of optical remote sensing images provided in an embodiment of the present application;

[0088] Fig.16 A schematic diagram of the structure of a feature segmentation module in an image segmentation network model provided in an embodiment of the present application;

[0089] Fig.17 A schematic diagram of a process for generating a semantic segmentation map in a semantic segmentation method for an optical remote sensing image provided in an embodiment of the present application;

[0090] Fig.18 A schematic diagram of a semantic segmentation device for optical remote sensing images provided in an embodiment of the present application;

[0091] Fig.19 A schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0092] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of explanation and description and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn in real proportion. The flowchart used in this application shows the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowchart can be implemented out of sequence, and the steps without logical context can be reversed in order or implemented simultaneously. In addition, those skilled in the art can add one or more other operations to the flowchart under the guidance of the content of the present application, or remove one or more operations from the flowchart.

[0093] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0094] It should be noted that the term "comprising" will be used in the embodiments of the present application to indicate the existence of the features declared thereafter, but does not exclude the addition of other features.

[0095] To clearly describe the semantic segmentation method of optical remote sensing images provided in the embodiment of the present application, the semantic segmentation method of optical remote sensing images is described below in conjunction with multiple figures. The semantic segmentation method of optical remote sensing images in the present application is applied to an image segmentation network model, which may be a convolutional neural network model. Figure 1 A schematic diagram of the structure of the first image segmentation network model provided in the embodiment of the present application is shown in FIG. Figure 1As shown, the image segmentation network model includes: a feature extraction module 100 and a feature segmentation module 200. The feature extraction module 100 includes: a downsampling module 101, a linear attention network 102 and an upsampling module 103. The linear attention network 102 includes: a first normalization layer 1021, a linear attention module 1022, a second normalization layer 1023 and a multilayer perceptron 1024. Among them, the feature extraction module 100 is used to extract low-level features and high-level features in the optical remote sensing image, and the feature segmentation module 200 is used to determine the label of each pixel in the optical remote sensing image. The downsampling module 101 is used to convert the image from a high-resolution image to a low-resolution image. The linear attention network 102 is used to extract the target global features of the image. The upsampling module 103 is used to convert the image from a low-resolution image to a high-resolution image. The first normalization layer 1021 is used to normalize the pixel values ​​in the image. The linear attention module 1022 is used to extract the global features of the image with context information. The second normalization layer 1023 is used to normalize the global features of the extracted image with context information. The multi-layer perceptron 1024 is used to learn data features and perform feature transformation.

[0096] Figure 2 A flowchart of a semantic segmentation method for optical remote sensing images provided in an embodiment of the present application is shown in FIG. Figure 2 As shown, the method includes:

[0097] Step 201: Use a downsampling module to downsample the optical remote sensing image of the target pixel size to obtain a first feature map.

[0098] The downsampling module is used to convert the optical remote sensing image of the target pixel size into a low-resolution optical remote sensing image. The pixel size of the first feature map is smaller than the target pixel size. The number of first feature maps in the present application can be multiple, and the embodiment of the present application does not limit this. When there are multiple first feature maps, the pixel sizes of the multiple first feature maps are different and are all smaller than the target pixel size.

[0099] Optionally, a downsampling module is used to downsample the optical remote sensing image of the target pixel size to be processed, so that the optical remote sensing image of the target pixel size becomes a low-resolution optical remote sensing image, namely, the first feature map.

[0100] Step 202: Use a first normalization layer to normalize the first feature map to obtain a first normalized feature.

[0101] The first normalization layer is used to normalize each pixel value in the first feature map.

[0102] Optionally, the first feature map is normalized using formula (1).

[0103]

[0104] Among them, x (i) is the pixel value of the i-th pixel in the first feature map, is the normalized value of the i-th pixel, μ B is the mean value of the pixel values ​​in the first feature map, is the variance of the pixel values ​​in the first feature map, γ is the training scaling parameter of the first normalization layer, and β is the training translation parameter of the first normalization layer.

[0105] Step 203: Use a linear attention module to perform linear attention processing on the first normalized feature to obtain a target global feature with context information.

[0106] Among them, the linear attention module is used to extract global features with contextual information in the image. The contextual information refers to the relationship between the pixels of the optical remote sensing image of the target pixel size to be processed and the association between each first normalized feature and the entire scene. The target global feature is the feature of the optical remote sensing image of the target pixel size to be processed, and the target global feature can represent the properties of the entire optical remote sensing image of the target pixel size to be processed.

[0107] Optionally, a linear attention module is used to perform linear attention processing on the first normalized feature, and the first normalized feature is interacted with the context to obtain a target global feature with context information.

[0108] Step 204: normalize the target global features using a second normalization layer to obtain second normalized features.

[0109] Among them, the second normalization layer is used to normalize the target global features.

[0110] Optionally, a second normalization layer is used to normalize the target global feature to obtain a second normalized feature, or a second normalization layer is used to normalize the target global feature and the first feature map to obtain a second normalized feature.

[0111] Step 205: Use a multilayer perceptron to perform feature mapping on the second normalized feature to obtain a second feature map with the same pixel size as the first feature map.

[0112] Among them, the multi-layer perceptron is used to learn data features and perform feature transformation.

[0113] Optionally, a multilayer perceptron is used to perform feature mapping on the second normalized feature, and the second normalized feature is compressed and converted to obtain a second feature map with the same pixel size as the first feature map, so that the second feature map can be used for subsequent convolution operations and feature extraction.

[0114] Step 206: Use an upsampling module to upsample the second feature map to obtain a target pixel size.

[0115] Feature map.

[0116] Among them, the upsampling module is used to convert the low-resolution image into a high-resolution image.

[0117] Optionally, an upsampling module is used to perform upsampling processing on the second feature map to obtain a target feature map of a target pixel size. The target pixel size is the target pixel size of the optical remote sensing image with the target pixel size to be processed.

[0118] Step 207: Use a feature segmentation module to perform feature segmentation on the target feature map to obtain a semantic segmentation map of the optical remote sensing image.

[0119] The feature segmentation module is used to determine the analogy of each pixel in the optical remote sensing image. The semantic segmentation map of the optical remote sensing image is an image that classifies and labels each pixel in the optical remote sensing image according to the ground object category to which it belongs.

[0120] Optionally, a feature segmentation module is used to perform feature segmentation on the target feature map, so that semantic information can be effectively extracted from the optical remote sensing image, thereby obtaining a semantic segmentation map of the optical remote sensing image based on the semantic information.

[0121] The present application provides a semantic segmentation method for optical remote sensing images, which is applied to an image segmentation network model, wherein a downsampling module is used to downsample an optical remote sensing image of a target pixel size to be processed, and a first feature map is obtained. A first normalization layer is used to normalize the first feature map, and a first normalized feature is obtained. A linear attention module is used to perform linear attention processing on the first normalized feature, and a target global feature with context information is obtained. A second normalization layer is used to normalize the target global feature, and a second normalized feature is obtained. A multi-layer perceptron is used to perform feature filming on the second normalized feature, and a second feature map with the same pixel size as the first feature map is obtained. An upsampling module is used to upsample the second feature map, and a target feature map of the target pixel size is obtained. A feature segmentation module is used to perform feature segmentation on the target feature map, and a semantic segmentation map of the optical remote sensing image is obtained. The present application includes a linear attention module that performs linear processing on the first normalized feature, which can effectively process long-distance dependencies in remote sensing images, and uses complex spatial relationships with multi-scale semantic segmentation tasks, thereby improving calculation accuracy, reducing calculation complexity, and reducing the demand for calculation resources.

[0122] Based on the above embodiments, the linear attention module includes: a linear processing layer, a first aggregation layer, a second aggregation layer, a third aggregation layer, a fourth aggregation layer, a first attention calculation layer and a second attention calculation layer. Figure 3 A schematic diagram of the structure of an image segmentation network model provided in an embodiment of the present application is shown in FIG. Figure 3 As shown, the linear attention module 1022 includes: a linear processing layer 10221, a first aggregation layer 10222, a second aggregation layer 10223, a third aggregation layer 10224, a fourth aggregation layer 10225, a first attention calculation layer 10226, and a second attention calculation layer 10227. Among them, the linear processing layer 10221 is used to perform a linear transformation on the first normalized feature and map the first normalized feature to the linear attention module. The first aggregation layer 10222, the second aggregation layer 10223, the third aggregation layer 10224, and the fourth aggregation layer 10225 are all used for aggregation calculation. The first attention calculation layer 10226 and the second attention calculation layer 10227 are used for attention calculation.

[0123] To this end, the present application also provides a process for determining the global features of a target in a semantic segmentation method of an optical remote sensing image. Figure 4 A flow chart of determining a target global feature in a semantic segmentation method of an optical remote sensing image provided in an embodiment of the present application, such as using a linear attention module in the above step 203 to linearly process the first normalized feature to obtain a target global feature with context information, including:

[0124] Step 401: Use a linear processing layer to perform linear processing on a first normalized feature to obtain a first linear query vector, a first linear key vector, and a first linear value vector corresponding to the first linear key vector.

[0125] The first linear query vector is Query (Q), which is used to extract relevant information in the linear processing layer and represents the context information currently required to be queried. The first linear key vector is Key (K), which is used to match the first linear query vector to determine the relevance of each input feature to the current query. The first linear value vector is Value (V), which contains the actual information of the input feature and is used to update the actual output of the linear processing layer.

[0126] Step 402: Use a first aggregation layer to perform aggregation calculation on a first global feature and a first linear key vector corresponding to a first linear query vector to obtain a first aggregate feature with context information.

[0127] Among them, the first global feature corresponding to the first linear query vector is a new feature vector generated based on the first linear query vector, the first global feature includes context information, the number of first global features can be flexibly set, and the embodiment of the present application does not limit this. The generation strategy can be to apply an additional weight matrix to the first linear query vector for linear transformation, or to apply a nonlinear function to the first linear query vector, etc., and the embodiment of the present application does not limit this. Aggregation calculation is mainly used to calculate the dot product of the first global feature and the first linear key vector.

[0128] Optionally, before the first aggregation layer is used to perform aggregation calculation on the first global feature and the first linear key vector corresponding to the first linear query vector, the first linear key vector needs to be optimized by a nonlinear function.

[0129] Optionally, a first global feature corresponding to the first linear query vector is generated through the first linear query vector, the first linear key vector is optimized through a nonlinear function, and the first global feature and the optimized first key vector are aggregated using a first aggregation layer to obtain a first aggregated feature with context information.

[0130] Step 403: Use the first attention calculation layer to perform attention calculation on the first aggregated feature to obtain a first attention matrix.

[0131] Optionally, a first attention calculation layer is used to perform attention calculation on the first aggregated feature to obtain a first attention matrix. Specifically, the first attention matrix can be calculated using formula (2).

[0132]

[0133] Among them, softmax is the first attention calculation, GQ is the first aggregate feature with context information, and K T is the matrix of the first linear key vector, obtained by the first normalized feature transformation. k is the dimension of the matrix of the first linear key vector, mainly used as a scaling factor, S is the first attention matrix, and n×d in S are the dimensions of S respectively.

[0134] Step 404: Use a second aggregation layer to perform aggregation calculation on the first attention matrix and the first linear value vector to obtain a second global feature.

[0135] Optionally, the first attention matrix and the first linear value vector are aggregated by a second aggregation layer. Specifically, the second global feature is obtained by performing a dot product calculation on the first attention matrix and the first linear value vector according to formula (3).

[0136] GV∈R n×d =S×V (3)

[0137] Among them, GV is the second global feature, S is the first attention matrix, V is the first linear value vector, and n×d in GV are the dimensions of GV.

[0138] Step 405: Use a third aggregation layer to perform aggregation calculation on the activation function feature of the first linear query vector and the first global feature to obtain a second aggregate feature.

[0139] Among them, the activation function is the Relu function, the first global feature is to generate a new feature vector based on the first linear query vector, the first global feature includes context information, and the aggregation calculation is the dot product calculation.

[0140] Optionally, a Relu function feature of the first linear query vector of the third aggregation layer is used and a first global feature is generated based on the first linear query vector to perform a dot product calculation to obtain a second aggregation feature.

[0141] Step 406: Use the second attention calculation layer to perform attention calculation on the second aggregated features to obtain a second attention matrix.

[0142] Optionally, a second attention calculation layer is used to perform attention calculation on the second aggregated features to obtain a second attention matrix. Specifically, the first attention matrix can be calculated by formula (4).

[0143]

[0144] Among them, softmax is the second attention calculation, Q×GQ T is the second aggregate feature with context information. k is the dimension of the matrix of the first linear key vector, mainly used as a scaling factor, U is the first attention matrix, and N×n in U are the dimensions of U respectively.

[0145] Step 407: Use the fourth aggregation layer to perform aggregation calculation on the second attention matrix and the first global feature to obtain the target global feature.

[0146] Optionally, the second attention matrix and the first global feature are aggregated by a fourth aggregation layer. Specifically, the target global feature is obtained by performing a dot product calculation on the second attention matrix and the first global feature according to formula (5).

[0147] O∈R N×d =U×GV (5)

[0148] Among them, O is the target global feature, U is the first attention matrix, GV is the first global feature, and N×d in O are the dimensions of O.

[0149] For example, combined with Figure 3, the processing flow of a linear processing layer provided in an embodiment of the present application is described. The first normalized feature is linearly processed by the linear processing layer 10221 to obtain a first linear query vector, a first linear key vector, and a first linear value vector corresponding to the first linear key vector. The first aggregation layer 10222 is used to perform aggregation calculation on the first global feature corresponding to the first linear query vector and the activation function feature of the first linear key vector to obtain a first aggregate feature with context information. The first attention calculation layer 10226 is used to perform attention calculation on the first aggregate feature to obtain a first attention matrix. The second aggregation layer 10223 is used to perform aggregation calculation on the first attention matrix and the first linear value vector to obtain a second global feature. The third aggregation layer 10224 is used to perform aggregation calculation on the activation function feature of the first linear query vector and the first global feature to obtain a second aggregate feature. The second attention calculation layer 10226 is used to perform attention calculation on the second aggregate feature to obtain a second attention matrix. The fourth aggregation layer 10225 is used to perform aggregation calculation on the second attention matrix and the first global feature to obtain a target global feature.

[0150] In the embodiment of the present application, by setting multiple linear processing layers, multiple aggregation layers and multiple attention calculation layers, the learning ability of the image segmentation network model can be enhanced by introducing an activation function and a first linear query vector to generate a new feature vector with context information, which helps the model to better capture dependencies when processing sequence data, thereby improving the performance and generalization ability of the model. The present application obtains two attention matrices through two attention calculation layers, which can reduce the complexity of the image segmentation network model calculation, and can extract disease aggregation global information from the first linear key vector and the first linear value vector, and can capture the global dependencies in the first normalized feature, providing rich context information for subsequent calculations. The second attention matrix and the first global feature aggregation calculation can effectively interact with the first normalized feature and the global information through a broadcast mechanism, and further from the image segmentation network model to capture the long-distance dependencies in the input first normalized feature, provide a rich feature representation for the output of the image segmentation network model, while reducing the difficulty of calculation, retaining the information exchange in the first normalized feature, can maintain the integrity of information transmission when processing a large amount of data, and the calculation complexity is close to linear when processing a large amount of data, which significantly improves the processing efficiency of the image segmentation network model.

[0151] On the basis of the above embodiment, the down-sampling module includes: multiple convolution layers, and the multiple convolution layers correspond to multiple pixel sizes respectively. Figure 5 A schematic diagram of the structure of a downsampling module of an image segmentation network model provided in an embodiment of the present application, such as Figure 5As shown, the downsampling module 101 includes: a plurality of convolutional layers 1011. The plurality of convolutional layers 1011 correspond to a plurality of pixel sizes respectively.

[0152] To this end, the present application provides a process for determining a first normalized feature in a semantic segmentation method for an optical remote sensing image. Figure 6 A schematic diagram of a process for determining a first normalized feature in a semantic segmentation method for an optical remote sensing image provided in an embodiment of the present application, such as Figure 6 As shown, in the above step 201, the optical remote sensing image of the target pixel size to be processed is processed by using a downsampling module to obtain a first feature map, including:

[0153] Step 601: Use multiple convolution layers to perform convolution processing on the optical remote sensing image in sequence to obtain first feature maps of multiple pixel sizes.

[0154] Wherein, multiple convolutional layers are used to obtain first feature maps of multiple pixel sizes.

[0155] For example, when there are three convolution layers, the pixel size of the input optical remote sensing image is 512×512, and the optical remote sensing image is a 3-channel image; after the first convolution layer, the pixel size of the optical remote sensing image of the first feature map obtained is 256×256, and the number of channels is 128; after the second convolution layer, the pixel size of the optical remote sensing image of the first feature map obtained is 128×128, and the number of channels is 256; after the third convolution layer, the pixel size of the optical remote sensing image of the first feature map obtained is 64×64, and the number of channels is 512. This application only takes three convolution layers as an example, and does not limit the specific number of convolution layers and the number of downsampling times. It is determined according to the specific usage scenario. The number of channels of the first feature map is set according to the actual situation and is not subject to the above embodiment.

[0156] The first normalization layer is used to normalize the first feature map to obtain a first normalized feature, including:

[0157] Step 602: Use a first normalization layer to normalize the first feature map of the minimum pixel size to obtain a first normalized feature.

[0158] Among them, the first feature map with the minimum pixel size is the first feature map obtained after the last convolution layer. According to the above example content, the first feature map with the minimum pixel size is the first feature map with a pixel size of 64×64 and a channel of 512.

[0159] Optionally, the first feature map of the minimum pixel size needs to pass through a hidden layer and a linear mapping layer before entering the normalization layer, and the specific pixel values ​​of each pixel in the first feature map of the minimum pixel size that has passed through the linear mapping layer are normalized, and the feature corresponding to the first feature map is the first normalized feature.

[0160] In an embodiment of the present application, multiple convolution layers are used to perform convolution processing on the optical remote sensing image in sequence to obtain first feature maps of multiple pixel sizes, and a first normalization layer is used to normalize the first feature map of the smallest pixel size to obtain a first normalized feature. In the present application, the amount of data calculation can be reduced by performing convolution processing on the optical remote sensing image, thereby reducing the complexity of the image segmentation network model calculation, thereby speeding up the processing speed. Normalizing the first feature map can make the dimensions of each pixel in the first special map the same, reduce the impact of the difference in different feature scales, make the model more stable, and improve the model accuracy.

[0161] Based on the above embodiment, the up-sampling module includes: multiple deconvolution layers. Figure 7 A schematic diagram of the structure of an upsampling module in an image segmentation network model provided in an embodiment of the present application, such as Figure 7 As shown, the up-sampling module 103 includes: a plurality of deconvolution layers 1031, and the plurality of deconvolution layers 1031 correspond to a plurality of pixel sizes respectively.

[0162] To this end, the present application provides a process for determining a target feature map in a semantic segmentation method for an optical remote sensing image, such as using an upsampling module in the above step 104 to upsample the second feature map to obtain a target feature map of a target pixel size, including:

[0163] A plurality of deconvolution layers are used to sequentially perform deconvolution processing on the second feature map to obtain third feature maps of a plurality of pixel sizes, and the third feature map of the maximum pixel size is determined as the target feature map.

[0164] Among them, multiple deconvolution layers are used to upsample the second feature map to obtain a third feature map of multiple pixel sizes.

[0165] For example, when there are three deconvolution layers, the pixel size of the input second feature map is 64×64, the channel is 512, the pixel size of the third feature map obtained after the first deconvolution layer is 128×128, the channel is 256; the pixel size of the third feature map obtained after the second deconvolution layer is 256×256, the channel is 128; the pixel size of the third feature map obtained after the third deconvolution layer is 512×512, the channel is 64. This application only takes three deconvolution layers as an example, and does not limit the specific number of deconvolution layers and the number of downsampling times, which are determined according to the specific usage scenario. The number of channels of the third feature map is set according to the actual situation and is not subject to the above embodiment.

[0166] Optionally, before using multiple deconvolution layers to sequentially perform deconvolution processing on the second feature map, the second feature map is respectively passed through a convolution layer normalization layer and an activation function. Then, the second feature map is deconvoluted, that is, upsampled, to sequentially obtain third feature maps of multiple pixel sizes, and the third feature map of the largest pixel size is determined as the target feature map. Among them, the third feature maps of multiple pixel sizes are the same in number and pixel size as the first feature maps of multiple pixel sizes.

[0167] In an embodiment of the present application, multiple deconvolution layers are used to sequentially deconvolve the second feature map to obtain third feature maps of multiple pixel sizes, and the third feature map of the largest pixel size is determined as the target feature map. The present application can restore image details at different pixel sizes during the deconvolution process, which helps to retain and fuse multi-scale context information in the final feature map.

[0168] Based on the above embodiment, the up-sampling module further includes: a plurality of gating modules. Figure 8 A schematic diagram of the structure of an upsampling module in another image segmentation network model provided in an embodiment of the present application is as follows: Figure 8 As shown, the up-sampling module 103 includes: a plurality of gating modules 1032, and the plurality of gating modules 1032 correspond to a plurality of pixel sizes respectively.

[0169] To this end, the present application provides a process for determining a third feature map in a semantic segmentation method for an optical remote sensing image. Fig. 9 A schematic diagram of a process for determining a third feature map in a semantic segmentation method for a remote sensing image provided in an embodiment of the present application is shown in FIG. Fig. 9 As shown, based on the above-mentioned use of multiple deconvolution layers to sequentially perform deconvolution processing on the second feature map to obtain third feature maps of multiple pixel sizes, and determining the third feature map with the largest pixel size as the target feature map, the method also includes:

[0170] Step 901: Use each gating module to perform feature fusion on the first feature map and the second feature map of the corresponding pixel size of each gating module to obtain a fused feature map of the corresponding pixel size.

[0171] Among them, each gated fusion module corresponds to a different fusion size, and the gated fusion module is used to perform feature fusion on the first feature map and the second feature map of the pixel size corresponding to each gated module. The number of gated modules is the same as the number of convolutional layers and deconvolutional layers. At this time, the pixel size corresponding to the gated fusion module, the pixel size of the first feature map, and the pixel size of the second feature map are all the same. The size of the fused feature map obtained by the fusion of the gated fusion module is also the pixel size corresponding to the gated fusion module. The first feature map is a low-level feature map obtained by downsampling, and the second feature map is a high-level feature map obtained by upsampling.

[0172] Optionally, each gating module is used to perform feature fusion on the first feature map and the second feature map of the corresponding pixel size of each gating module, that is, the first feature map and the second feature map of the same size as the gating module among the multiple first feature maps and the multiple second feature maps are feature fused to obtain a fused feature map of the corresponding pixel size.

[0173] As described above, multiple deconvolution layers are used to sequentially perform deconvolution processing on the second feature map to obtain third feature maps of multiple pixel sizes, and the third feature map with the largest pixel size is determined as the target feature map, including:

[0174] Step 902: Use each deconvolution layer to perform deconvolution processing on the fused feature map of the corresponding pixel size to obtain a third feature map of the next pixel size.

[0175] The next pixel size is a pixel size larger than the corresponding pixel size.

[0176] Optionally, each deconvolution layer is used to perform deconvolution processing on the fused feature map of the corresponding pixel size, that is, the fused feature map of the corresponding pixel size is upsampled through the deconvolution layer to obtain a third feature map of the next pixel size of the fused feature map.

[0177] In an embodiment of the present application, each gating module is used to perform feature fusion on the first feature map and the second feature map of the corresponding pixel size of each gating module to obtain a fused feature map of the corresponding pixel size, and each deconvolution layer is used to perform deconvolution processing on the fused feature map of the corresponding pixel size to obtain a third feature map of the next pixel size. The present application can obtain a third feature map with a larger pixel size, and by superimposing it with a feature map of a lower layer, the quality of upsampling can be improved and artifacts can be reduced. High-level semantic information and low-level detail information can be used simultaneously to enrich feature representation.

[0178] On the basis of the above embodiment, each gating module includes: a first mixing unit, a second mixing unit, a weight function unit and a connection function unit. Fig.10 The structural diagram of the gating module in the first image segmentation network model provided in the embodiment of the present application is as follows: Fig.10 As shown, each gating module 1032 includes: a first mixing unit 10321, a second mixing unit 10322, a weight function unit 10323 and a connection function unit 10324. Among them, the first mixing unit 10321 is used to calculate the feature map according to the weight parameter. The second mixing unit 10322 is also used to calculate the feature map according to the weight parameter. The weight function unit 10323 is used to determine the corresponding weight parameter. The connection function unit 10324 is used to splice the weighted feature map.

[0179] To this end, the present application provides a process for determining a fusion feature map in a semantic segmentation method for an optical remote sensing image. Fig.11 A schematic diagram of a flow chart for determining a fusion feature map in a semantic segmentation method for an optical remote sensing image provided in an embodiment of the present application, such as Fig.11 As shown, in the above step 1001, each gating module is used to perform feature fusion on the first feature map and the second feature map of the corresponding pixel size of each gating module to obtain a fused feature map of the corresponding pixel size, including:

[0180] Step 1101: Use a weight function unit to perform weight calculation on a first feature map and a second feature map of corresponding pixel sizes to obtain a first weight parameter.

[0181] Optionally, a weight function unit is used to calculate the weight of the first feature map and the second feature map of the corresponding pixel size. Specifically, a first weight parameter is calculated according to the input first feature map and the second feature map by the logic function in formula (6), wherein the logic function can be a Sigmoid function. The value range of the first weight parameter is between 0 and 1. The first weight parameter reflects the importance of the first feature map in the fused feature map.

[0182] W = sigmoid(X1+X2)(6)

[0183] Among them, X1 is the first feature map, X2 is the second feature map, and W is the first weight parameter.

[0184] Step 1102: Use a first mixing unit to perform weighted calculation according to a first weight parameter and a first feature map of a corresponding pixel size to obtain a first weighted feature map.

[0185] Optionally, a first mixing unit is used to perform weighted calculation according to a first weight parameter and a first feature map of a corresponding pixel size. The specific process is to calculate the first feature map of the corresponding pixel size according to the first weight parameter using formula (7) to obtain a first weighted feature map.

[0186] X1′=W·X1(7)

[0187] Among them, X1′ is the first weighted feature map, W is the first weight parameter, and X1 is the first feature map.

[0188] Step 1103: Use a second mixing unit to perform weighted calculation according to a second weight parameter corresponding to the first weight parameter and a second feature map of a corresponding pixel size to obtain a second weighted feature map.

[0189] The sum of the first weight parameter and the second weight parameter is 1, and the second weight parameter can be determined by subtracting the first weight parameter from 1.

[0190] Optionally, a second mixing unit is used to perform weighted calculation according to a second weight parameter corresponding to the first weight parameter and a second feature map of the corresponding pixel size. The specific process is to calculate the second weight parameter according to the first weight parameter, and calculate the second weighted feature map according to the second weight parameter and the second feature map through formula (8).

[0191] X′2=(1-W)·X2(8)

[0192] Among them, X′2 is the second weighted feature map, W is the first weight parameter, and X2 is the second feature map.

[0193] Step 1104: Use a connection function unit to concatenate the first weighted feature map and the second weighted feature map to obtain a fused feature map of corresponding pixel size.

[0194] Optionally, a connection function unit is used to splice the first weighted feature map and the second weighted feature map. The specific process is to obtain a fusion feature map of corresponding pixel size by splicing the first weighted feature map and the second weighted feature map according to formula (9).

[0195] Y=X1′⊕X′2(9)

[0196] Among them, Y is the fused feature map, X1′ is the first weighted feature map, and X2′ is the second weighted feature map.

[0197] In an embodiment of the present application, through the first mixing unit, the second mixing unit, the weight function unit and the connection function unit in the gating module, the importance of different scale features in the final fusion process is adaptively adjusted according to the characteristics of the input first feature map and the second feature map, so that the fusion process is more flexible and the fusion result is more accurate.

[0198] On the basis of the above embodiment, each gating module includes: a first convolution unit, a second convolution unit, a first mixing unit, a second mixing unit, a weight function unit and an adding unit. Fig.12 The structural diagram of the gating module in the second image segmentation network model provided in the embodiment of the present application is as follows: Fig.12As shown, each gating module 1032 includes: a first convolution unit 10325, a second convolution unit 10326, a first mixing unit 10321, a second mixing unit 10322, a weight function unit 10323 and an addition unit 10327. Among them, the first mixing unit 10321 is used to calculate the feature map according to the weight parameter. The second mixing unit 10322 is also used to calculate the feature map according to the weight parameter. The weight function unit 10323 is used to determine the corresponding weight parameter. The first convolution unit 10325 is used to perform a convolution operation on the input feature map. The second convolution unit 10326 is used to perform a convolution operation on the input feature map. The addition unit 10327 is used to perform addition processing on the feature map.

[0199] To this end, the present application provides a process for determining a fusion feature map in a semantic segmentation method for an optical remote sensing image. Fig.13 A schematic diagram of a flow chart for determining a fusion feature map in a second semantic segmentation method for optical remote sensing images provided in an embodiment of the present application, such as Fig.13 As shown, in the above step 1001, each gating module is used to perform feature fusion on the first feature map and the second feature map of the corresponding pixel size of each gating module to obtain a fused feature map of the corresponding pixel size, including:

[0200] Step 1301: Use a first convolution unit and a second convolution unit to perform convolution processing on a first feature map and a second feature map of corresponding pixel sizes respectively to obtain a first mapping feature and a second mapping feature.

[0201] Among them, the convolution processing is a 1×1 convolution operation.

[0202] Optionally, a first convolution unit and a second convolution unit are used to perform convolution processing on a first feature map and a second feature map of corresponding pixel sizes, respectively. The specific process is to perform convolution processing on the first feature map of corresponding pixel size by formula (10) to obtain a first mapping feature, and to perform convolution processing on the second feature map of corresponding pixel size by formula (11) to obtain a second mapping feature.

[0203] m1=Conv 1×1 (X1) (10)

[0204] m2=Conv 1×1 (X2) (11)

[0205] Among them, M1 is the first mapping feature, X1 is the first feature map, M2 is the second mapping feature, X2 is the second feature map, Conv 1×1 It is a 1×1 convolution operation.

[0206] Step 1302: Use a weight function unit to perform weight calculation on the first mapping feature and the second mapping feature to obtain a first weight parameter.

[0207] Optionally, a weight function unit is used to calculate the weight of the first mapping feature and the second mapping feature, and the specific steps are to calculate the weight according to the first mapping feature and the second mapping feature through the logic function in formula (12) to obtain a first weight parameter. The logic function can be a Sigmoid function. The value range of the first weight parameter is between 0 and 1. The first weight parameter reflects the importance of the first feature map in the fusion feature map.

[0208] W=Sigmoid(M1+M2) (12)

[0209] Wherein, W is the first weight parameter, M1 is the first mapping feature, and M2 is the second mapping feature.

[0210] Step 1303: Use the first mixing unit to perform weighted calculation according to the first weight parameter and the first mapping feature to obtain a first weighted feature map.

[0211] Optionally, a first mixing unit is used to perform weighted calculation according to a first weight parameter and the first mapping feature to determine the weight of the first mapping feature, thereby obtaining a first weighted feature map.

[0212] Step 1304: Use a second mixing unit to perform weighted calculation according to the second weight parameter corresponding to the first weight parameter and the second mapping feature to obtain a second weighted feature map.

[0213] The sum of the first weight parameter and the second weight parameter is 1, and the second weight parameter can be determined by subtracting the first weight parameter from 1.

[0214] Optionally, a second mixing unit is used to perform weighted calculation according to a second weight parameter corresponding to the first weight parameter and the second mapping feature to determine the second weight parameter, and to determine a second weighted feature map according to the second weight parameter and the second mapping feature.

[0215] Step 1305: Use an adding unit to perform addition processing on the first weighted feature map and the second weighted feature map to obtain a fused feature map of corresponding pixel size.

[0216] Optionally, an adding unit is used to perform addition processing on the first weighted feature map and the second weighted feature map. The specific process is to perform addition processing on the first weighted feature map and the second weighted feature map through formula (13) to obtain a fused feature map of corresponding pixel size.

[0217] Y=W·M1+(1-W)·M2(13)

[0218] Among them, W is the first weight parameter, M1 is the first mapping feature, M2 is the second mapping feature, W·M1 is the first weighted feature map, (1-W)·M2 is the second weighted feature map, and Y is the fusion feature map.

[0219] In the embodiment of the present application, through the first convolution unit, the second convolution unit, the first mixing unit, the second mixing unit, the weight function unit and the addition unit in the gating module, the importance of different scale features in the final fusion process is adaptively adjusted according to the characteristics of the input first feature map and the second feature map, so that the fusion process is more flexible and the fusion result is more accurate.

[0220] On the basis of the above embodiment, each gating module includes: a first convolution unit, a second convolution unit, a first mixing unit, a second mixing unit, a weight function unit and an adding unit. Fig.14 The schematic diagram of the structure of the gating module in the third image segmentation network model provided in the embodiment of the present application is as follows: Fig.14 As shown, each gating module 1032 includes: a first convolution unit 10324, a second convolution unit 10325, a first mixing unit 10321, a second mixing unit 10322, a weight function unit 10323 and a connection function unit 10324. The functions of the specific units are the same as those in the above embodiment and will not be repeated here.

[0221] To this end, the present application provides a process for determining a fusion feature map in a semantic segmentation method for an optical remote sensing image. Fig.15 A schematic diagram of a process for determining a fusion feature map in a third method for semantic segmentation of optical remote sensing images provided in an embodiment of the present application is shown in FIG. Fig.15 As shown, in the above step 1001, each gating module is used to perform feature fusion on the first feature map and the second feature map of the corresponding pixel size of each gating module to obtain a fused feature map of the corresponding pixel size, including:

[0222] Step 1501: Use a first convolution unit and a second convolution unit to perform convolution processing on a first feature map and a second feature map of corresponding pixel sizes respectively to obtain a first mapping feature and a second mapping feature.

[0223] Optionally, this step can be obtained by calculating through the above formula (10) and formula (11).

[0224] M1=Conv 1×1 (X1) (10)

[0225] M2=Conv 1×1 (X2) (11)

[0226] Among them, M1 is the first mapping feature, X1 is the first feature map, M2 is the second mapping feature, X2 is the second feature map, Comv 1×1 It is a 1×1 convolution operation.

[0227] Step 1502: Use a weight function unit to perform weight calculation on the first mapping feature and the second mapping feature to obtain a first weight parameter.

[0228] Optionally, this step can be obtained by calculating using the above formula (12).

[0229] W=Sigmoid(M1+M2) (12)

[0230] Wherein, W is the first weight parameter, M1 is the first mapping feature, and M2 is the second mapping feature.

[0231] Step 1503: Use the first mixing unit to perform weighted calculation according to the first weight parameter and the first feature map of the corresponding pixel size to obtain a first weighted feature map.

[0232] Optionally, a first mixing unit is used to perform weighted calculation according to the first weight parameter and the first feature map of the corresponding pixel size through the above formula (7) to determine the weight of the first feature map of the corresponding pixel size, thereby obtaining a first weighted feature map.

[0233] Step 1504: Use a second mixing unit to perform weighted calculation according to the second weight parameter corresponding to the first weight parameter and the second feature map to obtain a second weighted feature map.

[0234] The sum of the first weight parameter and the second weight parameter is 1, and the second weight parameter can be determined by subtracting the first weight parameter from 1.

[0235] Optionally, a second mixing unit is used to perform weighted calculation according to the second weight parameter and the second feature map of the corresponding pixel size through the above formula (8) to determine the weight of the second feature map of the corresponding pixel size, thereby obtaining a second weighted feature map.

[0236] Step 1505: Use a connection function unit to concatenate the first weighted feature map and the second weighted feature map to obtain a fused feature map of corresponding pixel size.

[0237] Optionally, a connection function unit is used to concatenate the first weighted feature map and the second weighted feature map through the above formula (9) to obtain a fused feature map of corresponding pixel size.

[0238] In the embodiment of the present application, through the first convolution unit, the second convolution unit, the first mixing unit, the second mixing unit, the weight function unit and the connection function unit in the gating module, the importance of different scale features in the final fusion process is adaptively adjusted according to the characteristics of the input first feature map and the second feature map, so that the fusion process is more flexible and the fusion result is more accurate.

[0239] Based on the above embodiment, the feature segmentation module includes: a feature mapping unit and a generation unit. Fig.16 A structural diagram of a feature segmentation module in an image segmentation network model provided in an embodiment of the present application, such as Fig.16 As shown, the feature segmentation module 200 includes: a feature mapping unit 2001 and a generation unit 2002. The feature mapping unit 2001 is used to extract higher-level semantic information and map it to a category label, and the generation unit 2002 is used to generate a corresponding semantic segmentation map.

[0240] To this end, the present application provides a process for generating a semantic segmentation map in a semantic segmentation method for optical remote sensing images. Fig.17 A schematic diagram of a flow chart of generating a semantic segmentation map in a semantic segmentation method for an optical remote sensing image provided in an embodiment of the present application, such as Fig.17 As shown, in the above step 207, a feature segmentation module is used to perform feature segmentation on the target feature map to obtain a semantic segmentation map of the optical remote sensing image, including:

[0241] Step 1701: Use a feature mapping unit to perform a convolution operation on a target feature map according to a preset number of categories to obtain a new feature map, in which the features of each pixel point have a corresponding category label and the probability of the corresponding category.

[0242] The number of preset categories is determined according to the actual situation, and the embodiments of the present application do not limit this. The size of the target feature map is H×W×C, where H×W is the height and width respectively, and C is the number of channels, which is usually equal to the number of target categories. The number of category labels of the features of each pixel in the new feature map is consistent with the number of preset categories.

[0243] Optionally, a feature mapping unit is used to perform a convolution operation on the target feature map according to a preset number of categories through formula (14) to obtain a new feature map.

[0244] F′=Conv(F,K) (14)

[0245] Among them, F is the target feature map, F′ is the new feature map, K is the convolution kernel, and Conv is the convolution operation.

[0246] Optionally, an activation function is used to calculate the probability of each pixel feature for different category labels using formula (15). The activation function may be a softmax activation function.

[0247]

[0248] Among them, P(c|F′) i represents the probability that the feature of the pixel at position i in the target feature map belongs to the analog label c, F′ i,c is the value of the pixel at position i in the target feature map corresponding to category c, and j is the number of pixels.

[0249] Step 1702: Use a generation unit to generate a semantic segmentation map based on the new feature map.

[0250] Optionally, the label category with the highest probability for each pixel is determined by formula (16) as the predicted category label of each pixel. A corresponding semantic segmentation map is generated according to the predicted category label of each pixel. In the semantic segmentation map, each pixel is assigned a category label to represent different areas, and the value of each pixel is the index of the corresponding category label. The size of the semantic segmentation map is the same as the target size of the optical remote sensing image of the target pixel size to be processed.

[0251]

[0252] in, is the predicted category label of the pixel at position i in the target feature map, P(c|F′) i It represents the probability that the feature of the pixel at position i in the target feature map belongs to the analogy label c, and argmax is the maximum probability calculation function.

[0253] In an embodiment of the present application, a semantic segmentation map is generated by a feature mapping unit and a generation unit. The feature mapping unit is used to perform a convolution operation on the target feature map according to a preset number of categories to obtain a new feature map. The features of each pixel in the new feature map have a corresponding category label and the probability of the corresponding category. The category label of the pixel is determined based on the category label with the maximum probability, thereby ensuring the accuracy of the generated semantic segmentation map and improving the precision of the generated semantic segmentation map.

[0254] On the basis of the above embodiment, the present application further provides a process for determining an optical remote sensing image of a target pixel size in a semantic segmentation method of an optical remote sensing image. Before the above step 201 uses a downsampling module to downsample the optical remote sensing image of the target pixel size to be processed to obtain a first feature map, the method further includes:

[0255] The original remote sensing image is cropped to obtain an optical remote sensing image of the target pixel size.

[0256] Among them, the cropping process can use a sliding window of 512×512 pixels to crop the entire image area to ensure that no details of the optical remote sensing image are missed.

[0257] Optionally, when the original remote sensing image is cropped, the sliding window is slid with a fixed step size s to obtain optical remote sensing images of multiple target pixel sizes. The value of s can be adjusted according to the characteristics of the specific data set, and the step size is usually smaller than the size of the sliding window to ensure that there is appropriate overlap between adjacent cropped areas.

[0258] For example, when the original remote sensing image size is H×W and the easing window size is K×K, the number of cropped images N can be calculated by formula (17).

[0259]

[0260] Among them, H and W are the sizes of the original remote sensing image, K is the size of the sliding window, and s is the step size of the sliding window.

[0261] Optionally, edge areas that cannot be divided evenly during the cropping process are filled or scaled to make the image size consistent.

[0262] For example, a bilinear interpolation method is used to fill or scale the insufficient parts through formula (17) to ensure that all image segments have the same size.

[0263] Inew(x,y)=∑ x,y I(x,y)·max(0,1-|x′-x|)·max(0,1-|y′-y|) (17)

[0264] Among them, I new (x, y) represents the pixel value of the scaled optical remote sensing image, I(x, y) is the pixel value of the original optical remote sensing image, and x, y and x′, y′ are the coordinates in the original image and the scaled image respectively.

[0265] Optionally, in the process of training the image segmentation network model, after the preset data set is trimmed, a total of 3,300 optical remote sensing images of size 512×512 are obtained, and the optical remote sensing images are divided into a training set and a test set according to a certain ratio. The training set contains 2,805 images of 512×512 pixels for model training and optimization. The test set contains 495 images of 512×512 pixels for performance evaluation and testing of the model to ensure that the model has good generalization ability on unseen data. Among them, the preset data set can be the Vaihingen data set, which includes 33 high-resolution images in near-infrared, red and green bands, with a ground sampling distance of 9 cm, and different land cover types, such as water surface, buildings, vegetation and vehicles.

[0266] In the application embodiment, the original remote sensing image is cropped to obtain an optical remote sensing image of the target pixel size, which can ensure the consistency of the pixel size of the input image segmentation network model, process the original remote sensing image in batches, reduce the amount of calculation, improve the image processing speed, and save computing resources.

[0267] Based on the same inventive concept, an embodiment of the present application also provides a semantic segmentation device for optical remote sensing images corresponding to the semantic segmentation method for optical remote sensing images. Since the principle of solving the problem by the device in the embodiment of the present application is similar to the semantic segmentation method for optical remote sensing images in the embodiment of the present application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0268] Fig.18 A schematic diagram of a semantic segmentation device for an optical remote sensing image provided in an embodiment of the present application is applied to an image segmentation network model, the image segmentation network model includes: a feature extraction module and a feature segmentation module, the feature extraction module includes: a downsampling module, a linear attention network and an upsampling module, the linear attention network includes: a first normalization layer, a linear attention module, a second normalization layer and a multilayer perceptron, the device includes:

[0269] A first sampling module 1801 is used to perform downsampling processing on the optical remote sensing image of the target pixel size to be processed by using a downsampling module to obtain a first feature map;

[0270] A first processing module 1802 is used to perform normalization processing on the first feature map using a first normalization layer to obtain a first normalized feature;

[0271] The second processing module 1803 is used to perform linear attention processing on the first normalized feature using a linear attention module to obtain a target global feature with context information;

[0272] The third processing module 1804 is used to normalize the target global feature by using a second normalization layer to obtain a second normalized feature;

[0273] A mapping module 1805 is used to perform feature mapping on the second normalized feature using a multi-layer perceptron to obtain a second feature map having the same pixel size as the first feature map;

[0274] A second sampling module 1806 is used to perform upsampling processing on the second feature map by using an upsampling module to obtain a target feature map of a target pixel size;

[0275] The segmentation module 1807 is used to perform feature segmentation on the target feature map using the feature segmentation module to obtain a semantic segmentation map of the optical remote sensing image.

[0276] In one possible implementation, the linear attention module includes: a linear processing layer, a first aggregation layer, a second aggregation layer, a third aggregation layer, a fourth aggregation layer, a first attention calculation layer, and a second attention calculation layer second processing module 1803, which is specifically used to: use the linear processing layer to perform linear processing on the first normalized feature to obtain a first linear query vector, a first linear key vector, and a first linear value vector corresponding to the first linear key vector;

[0277] Using a first aggregation layer to perform aggregation calculation on a first global feature and a first linear key vector corresponding to the first linear query vector to obtain a first aggregated feature with context information;

[0278] Using the first attention calculation layer, perform attention calculation on the first aggregated feature to obtain a first attention matrix;

[0279] A second aggregation layer is used to perform aggregation calculation on the first attention matrix and the first linear value vector to obtain a second global feature;

[0280] A third aggregation layer is used to perform aggregation calculation on the activation function feature of the first linear query vector and the first global feature to obtain a second aggregate feature;

[0281] Using the second attention calculation layer, perform attention calculation on the second aggregated features to obtain a second attention matrix;

[0282] The fourth aggregation layer is used to perform aggregation calculation on the second attention matrix and the first global feature to obtain the target global feature.

[0283] In a possible implementation, the downsampling module includes: a plurality of convolutional layers, each of which corresponds to a plurality of pixel sizes; a first sampling module 1801, specifically configured to: sequentially perform convolution processing on the optical remote sensing image using the plurality of convolutional layers to obtain a first feature map of a plurality of pixel sizes;

[0284] In a possible implementation, the first processing module 1802 is specifically configured to: perform normalization processing on the first feature map of the minimum pixel size using a first normalization layer to obtain a first normalized feature.

[0285] In one possible implementation, the upsampling module includes: multiple deconvolution layers; the multiple deconvolution layers correspond to multiple pixel sizes respectively; the second sampling module 1806 is specifically used to: use multiple deconvolution layers to sequentially deconvolve the second feature map to obtain a third feature map of multiple pixel sizes, and determine the third feature map of the largest pixel size as the target feature map.

[0286] In a possible implementation, the up-sampling module further includes: a plurality of gating modules, the plurality of gating modules respectively corresponding to a plurality of pixel sizes, and the second sampling module 1806 is further used to: use each gating module to perform feature fusion on the first feature map and the second feature map corresponding to the pixel size of each gating module to obtain a fused feature map corresponding to the pixel size;

[0287] In a possible implementation, the second sampling module 1806 is specifically configured to: use each deconvolution layer to perform deconvolution processing on the fused feature map of the corresponding pixel size to obtain a third feature map of the next pixel size.

[0288] In a possible implementation, each gating module includes: a first mixing unit, a second mixing unit, a weight function unit, and a connection function unit; a second sampling module 1806 is specifically used to: use the weight function unit to perform weight calculation on the first feature map and the second feature map of corresponding pixel sizes to obtain a first weight parameter;

[0289] Using a first mixing unit, performing weighted calculation according to a first weight parameter and a first feature map of a corresponding pixel size to obtain a first weighted feature map;

[0290] Using a second mixing unit, performing weighted calculation according to a second weight parameter corresponding to the first weight parameter and a second feature map of a corresponding pixel size, to obtain a second weighted feature map; wherein the sum of the first weight parameter and the second weight parameter is 1;

[0291] The first weighted feature map and the second weighted feature map are concatenated using a connection function unit to obtain a fused feature map of corresponding pixel size.

[0292] In a possible implementation, each gating module includes: a first convolution unit, a second convolution unit, a first mixing unit, a second mixing unit, a weight function unit, and an addition unit; the second sampling module 1806 is specifically used to: use each gating module to perform feature fusion on the first feature map and the second feature map of the corresponding pixel size of each gating module to obtain a fused feature map of the corresponding pixel size, including:

[0293] Using a first convolution unit and a second convolution unit to perform convolution processing on the first feature map and the second feature map of corresponding pixel sizes respectively, to obtain a first mapping feature and a second mapping feature;

[0294] Using a weight function unit to perform weight calculation on the first mapping feature and the second mapping feature to obtain a first weight parameter;

[0295] Using a first mixing unit, performing weighted calculation according to a first weight parameter and a first mapping feature to obtain a first weighted feature map;

[0296] Using a second mixing unit, performing weighted calculation according to a second weight parameter corresponding to the first weight parameter and a second mapping feature to obtain a second weighted feature map; wherein the sum of the first weight parameter and the second weight parameter is 1;

[0297] An adding unit is used to perform addition processing on the first weighted feature map and the second weighted feature map to obtain a fused feature map of corresponding pixel size.

[0298] In a possible implementation, each gating module includes: a first convolution unit, a second convolution unit, a first mixing unit, a second mixing unit, a weight function unit, and a connection function unit; a second sampling module 1806 is specifically used to: use the first convolution unit and the second convolution unit to perform convolution processing on the first feature map and the second feature map of the corresponding pixel size respectively, to obtain the first mapping feature and the second mapping feature;

[0299] Using a weight function unit to perform weight calculation on the first mapping feature and the second mapping feature to obtain a first weight parameter;

[0300] Using a first mixing unit, performing weighted calculation according to a first weight parameter and a first feature map of a corresponding pixel size to obtain a first weighted feature map;

[0301] Using a first mixing unit, performing weighted calculation according to a second weight parameter corresponding to the first weight parameter and a second feature map of a corresponding pixel size, to obtain a second weighted feature map; wherein the sum of the first weight parameter and the second weight parameter is 1;

[0302] The first weighted feature map and the second weighted feature map are concatenated using a connection function unit to obtain a fused feature map of corresponding pixel size.

[0303] In a possible implementation, the feature segmentation module includes: a feature mapping unit and a generation unit; a segmentation module 1807, specifically configured to: use the feature mapping unit to perform a convolution operation on the target feature map according to a preset number of categories to obtain a new feature map, wherein the features of each pixel in the new feature map have a corresponding category label and a probability of the corresponding category;

[0304] The generation unit is used to generate a semantic segmentation map based on the new feature map.

[0305] In a possible implementation, the first sampling module 1801 is further used to: perform cropping processing on the original remote sensing image to obtain an optical remote sensing image of a target pixel size.

[0306] For descriptions of the processing flow of each module in the device and the interaction flow between each module, reference may be made to the relevant descriptions in the above method embodiment, which will not be described in detail here.

[0307] The present application also provides a computer device, Fig.19 A schematic diagram of the structure of a computer device provided in an embodiment of the present application includes: a processor 1901 and a memory 1902. The memory 1902 stores machine-readable instructions executable by the processor 1901. When the computer device is running, the processor 1901 communicates with the memory 1902. When the machine-readable instructions are executed by the processor 1901, the steps of the above-mentioned semantic segmentation method of optical remote sensing images are performed.

[0308] An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned method for semantic segmentation of optical remote sensing images are executed.

[0309] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the method embodiment, and will not be repeated in this application. In the several embodiments provided in this application, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0310] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or part of the technical solution that contributes to the prior art or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, RandomAccess Memory), disk or optical disk and other media that can store program code.

[0311] The above are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be covered by the protection scope of the present application.

Claims

1. A semantic segmentation method for optical remote sensing images, characterized in that: Applied to an image segmentation network model, the image segmentation network model includes: a feature extraction module and a feature segmentation module, the feature extraction module includes: a downsampling module, a linear attention network and an upsampling module, the linear attention network includes: a first normalization layer, a linear attention module, a second normalization layer and a multilayer perceptron, the method includes: The downsampling module is used to downsample the optical remote sensing image of the target pixel size to be processed to obtain a first feature map; Using the first normalization layer to normalize the first feature map to obtain a first normalized feature; Using the linear attention module to perform linear attention processing on the first normalized feature to obtain a target global feature with context information; Normalizing the target global feature using the second normalization layer to obtain a second normalized feature; Performing feature mapping on the second normalized features using the multilayer perceptron to obtain a second feature map having the same pixel size as the first feature map; The upsampling module is used to perform upsampling processing on the second feature map to obtain a target feature map of the target pixel size; The feature segmentation module is used to perform feature segmentation on the target feature map to obtain a semantic segmentation map of the optical remote sensing image.

2. The method for semantic segmentation of optical remote sensing images according to claim 1, characterized in that: The linear attention module includes: a linear processing layer, a first aggregation layer, a second aggregation layer, a third aggregation layer, a fourth aggregation layer, a first attention calculation layer, and a second attention calculation layer; the linear attention module is used to linearly process the first normalized feature to obtain a target global feature with context information, including: Using the linear processing layer to perform linear processing on the first normalized feature to obtain a first linear query vector, a first linear key vector, and a first linear value vector corresponding to the first linear key vector; Using the first aggregation layer to perform aggregation calculation on a first global feature corresponding to the first linear query vector and the first linear key vector to obtain a first aggregated feature with context information; Using the first attention calculation layer, perform attention calculation on the first aggregated features to obtain a first attention matrix; Using the second aggregation layer to perform aggregation calculation on the first attention matrix and the first linear value vector to obtain a second global feature; Using the third aggregation layer to perform aggregation calculation on the activation function feature of the first linear query vector and the first global feature to obtain a second aggregate feature; Using the second attention calculation layer, performing attention calculation on the second aggregated features to obtain a second attention matrix; The fourth aggregation layer is used to perform aggregation calculation on the second attention matrix and the first global feature to obtain the target global feature.

3. The semantic segmentation method of optical remote sensing images according to claim 1, characterized in that: The downsampling module includes: a plurality of convolutional layers, wherein the plurality of convolutional layers correspond to a plurality of pixel sizes respectively; The downsampling module is used to process the optical remote sensing image of the target pixel size to obtain a first feature map, including: Using the multiple convolutional layers to sequentially perform convolution processing on the optical remote sensing images to obtain first feature maps of the multiple pixel sizes; The using the first normalization layer to perform normalization processing on the first feature map to obtain a first normalized feature includes: The first normalization layer is used to normalize the first feature map of the minimum pixel size to obtain the first normalized feature.

4. The method for semantic segmentation of optical remote sensing images according to claim 3, characterized in that: The upsampling module includes: a plurality of deconvolution layers; the plurality of deconvolution layers correspond to the plurality of pixel sizes respectively; the upsampling process of the second feature map by the upsampling module to obtain the target feature map of the target pixel size includes: The second feature map is sequentially deconvolved using the multiple deconvolution layers to obtain third feature maps of the multiple pixel sizes, and the third feature map of the maximum pixel size is determined as the target feature map.

5. The method for semantic segmentation of optical remote sensing images according to claim 4, characterized in that: The up-sampling module further includes: a plurality of gating modules, the plurality of gating modules respectively corresponding to the plurality of pixel sizes, and the method further includes: Using each gating module, performing feature fusion on the first feature map and the second feature map of the corresponding pixel size of each gating module to obtain a fused feature map of the corresponding pixel size; The step of sequentially performing deconvolution processing on the second feature map using the multiple deconvolution layers to obtain third feature maps of the multiple pixel sizes, and determining the third feature map of the maximum pixel size as the target feature map includes: Each deconvolution layer is used to perform deconvolution processing on the fused feature map of the corresponding pixel size to obtain a third feature map of the next pixel size.

6. The method for semantic segmentation of optical remote sensing images according to claim 5, characterized in that: Each of the gating modules comprises: a first mixing unit, a second mixing unit, a weight function unit and a connection function unit; each of the gating modules is used to perform feature fusion on the first feature map and the second feature map of the corresponding pixel size of each gating module to obtain the fused feature map of the corresponding pixel size, including: Using the weight function unit to perform weight calculation on the first feature map and the second feature map of the corresponding pixel size to obtain a first weight parameter; Using the first mixing unit, performing weighted calculation according to the first weight parameter and the first feature map of the corresponding pixel size to obtain a first weighted feature map; Using the second mixing unit, performing weighted calculation according to the second weight parameter corresponding to the first weight parameter and the second feature map of the corresponding pixel size, to obtain a second weighted feature map; wherein the sum of the first weight parameter and the second weight parameter is 1; The first weighted feature map and the second weighted feature map are concatenated using the connection function unit to obtain a fused feature map of the corresponding pixel size.

7. The method for semantic segmentation of optical remote sensing images according to claim 5, characterized in that: Each of the gating modules comprises: a first convolution unit, a second convolution unit, a first mixing unit, a second mixing unit, a weight function unit and an addition unit; The method of using each gating module to perform feature fusion on the first feature map and the second feature map of the corresponding pixel size of each gating module to obtain the fused feature map of the corresponding pixel size includes: Using the first convolution unit and the second convolution unit to perform convolution processing on the first feature map and the second feature map of the corresponding pixel size respectively, to obtain a first mapping feature and a second mapping feature; Using the weight function unit to perform weight calculation on the first mapping feature and the second mapping feature to obtain a first weight parameter; Using the first mixing unit, performing weighted calculation according to the first weight parameter and the first mapping feature to obtain a first weighted feature map; Using the second mixing unit, performing weighted calculation according to a second weight parameter corresponding to the first weight parameter and the second mapping feature to obtain a second weighted feature map; wherein the sum of the first weight parameter and the second weight parameter is 1; The adding unit is used to perform addition processing on the first weighted feature map and the second weighted feature map to obtain the fused feature map of the corresponding pixel size.

8. The method for semantic segmentation of optical remote sensing images according to claim 5, characterized in that: Each of the gating modules includes: a first convolution unit, a second convolution unit, a first mixing unit, a second mixing unit, a weight function unit and a connection function unit; The method of using each gating module to perform feature fusion on the first feature map and the second feature map of the corresponding pixel size of each gating module to obtain the fused feature map of the corresponding pixel size includes: Using the first convolution unit and the second convolution unit to perform convolution processing on the first feature map and the second feature map of the corresponding pixel size respectively, to obtain a first mapping feature and a second mapping feature; Using the weight function unit to perform weight calculation on the first mapping feature and the second mapping feature to obtain a first weight parameter; Using the first mixing unit, performing weighted calculation according to the first weight parameter and the first feature map of the corresponding pixel size to obtain a first weighted feature map; Using the first mixing unit, performing weighted calculation according to a second weight parameter corresponding to the first weight parameter and the second feature map of the corresponding pixel size, to obtain a second weighted feature map; wherein the sum of the first weight parameter and the second weight parameter is 1; The first weighted feature map and the second weighted feature map are concatenated using the connection function unit to obtain a fused feature map of the corresponding pixel size.

9. The method for semantic segmentation of optical remote sensing images according to claim 1, characterized in that: The feature segmentation module includes: a feature mapping unit and a generation unit; The step of using the feature segmentation module to perform feature segmentation on the target feature map to obtain a semantic segmentation map of the optical remote sensing image includes: The feature mapping unit is used to perform a convolution operation on the target feature map according to a preset number of categories to obtain a new feature map, wherein the features of each pixel in the new feature map have a corresponding category label and the probability of the corresponding category; The generating unit is used to generate the semantic segmentation map according to the new feature map.

10. The method for semantic segmentation of optical remote sensing images according to claim 1, characterized in that: Before the downsampling module is used to downsample the optical remote sensing image of the target pixel size to be processed to obtain the first feature map, the method further includes: The original remote sensing image is cropped to obtain an optical remote sensing image of the target pixel size.

Images