Land cover classification method and electronic device based on deep learning model

Abstract

The present disclosure provides a land cover classification method and electronic device based on a deep learning model, which can be applied to the field of image processing and pattern recognition technology. The deep learning model includes an encoder, a first decoder and a second decoder, and the land cover classification method based on the deep learning model includes: inputting a land image to be detected into the encoder, outputting a first feature map of multiple resolutions and a second feature map fused with multiple resolutions; inputting the first feature map of multiple resolutions into the first decoder, outputting a third feature map of multiple resolutions; and inputting the first feature map of multiple resolutions, the second feature map and the third feature map of multiple resolutions into the second decoder, and outputting the land cover classification result of the land image.

Description

Technical Field

[0001] The present disclosure relates to the technical field of image processing and pattern recognition, and in particular to a land cover classification method and electronic device based on a deep learning model. Background Art

[0002] Land cover classification is one of the basic data for regional planning, climate change, ecosystem assessment, environmental modeling and many other studies. It can provide quantitative and qualitative information on the state and changes of the earth's surface, and plays a very important role in resource survey, environmental monitoring, ecological protection, urban planning, agricultural production and other fields. At the same time, land cover classification can provide valuable information for land planning and management. Decision makers can rely on land cover classification data to plan land use more intelligently, achieve sustainable land management, reasonably allocate resources, reduce land pollution, increase agricultural production capacity, and promote sustainable development of cities. Therefore, in-depth research on land cover classification methods is necessary for the sustainable development of the environment.

[0003] In recent years, with the development of remote sensing and deep learning, remote sensing technology has the advantages of wide coverage, high efficiency, and less terrain restrictions. The high-resolution remote sensing images obtained can provide more detailed and accurate features of land objects. High-resolution remote sensing images play an important role in land cover classification. Although high-resolution remote sensing images contain rich information about land objects, they also introduce a lot of background information and face challenges such as similar spectral and texture features of land object categories, unbalanced category distribution, and difficulty in classifying small target categories.

[0004] The main methods of land cover classification include traditional geographic information system (GIS) methods, machine learning algorithms, semantic segmentation methods in deep learning, etc. However, using GIS methods and machine learning methods requires researchers to spend a lot of time designing features, selecting models, and adjusting parameters, and the trained models are difficult to adapt to different classification scenarios.

[0005] However, the existing land cover classification methods still have the following technical problems: the same category of objects in remote sensing images have differences in external features, such as texture, color, shape, etc., which makes it difficult for the model to identify and distinguish; different categories of objects have similarities in external features, such as shadows, lighting, angles, etc., which makes the model easy to confuse and make errors, resulting in the "salt and pepper" phenomenon and inaccurate boundaries in the predicted map. Summary of the invention

[0006] In view of the above problems, the present disclosure provides a land cover classification method and electronic device based on a deep learning model.

[0007] According to a first aspect of the present disclosure, a land cover classification method based on a deep learning model is provided, wherein the deep learning model includes an encoder, a first decoder, and a second decoder, and the method includes: inputting a land image to be detected into the encoder, outputting a first feature map of multiple resolutions and a second feature map fused with multiple resolutions; inputting the first feature map of multiple resolutions into the first decoder, outputting a third feature map of multiple resolutions; and inputting the first feature map of multiple resolutions, the second feature map, and the third feature map of multiple resolutions into the second decoder, and outputting a land cover classification result of the land image.

[0008] According to an embodiment of the present disclosure, the encoder includes a multi-scale information interaction module and a five-layer encoder structure that is the same as the UHRNet deep learning model; wherein, the land image to be detected is input into the encoder, and a first feature map of multiple resolutions and a second feature map that fuses multiple first feature maps are output, including: after the land image to be detected is convolved by the first layer of encoder structure, the first feature maps of four resolutions are output in sequence through the last four layers of encoder structure; and the first feature maps of three resolutions output by the middle three layers of encoder structure are input into the multi-scale information interaction module, and the second feature map that fuses the three resolutions is output.

[0009] According to an embodiment of the present disclosure, the first feature maps of three resolutions include a first sub-feature map of low resolution, a second sub-feature map of medium resolution, and a third sub-feature map of high resolution; the first feature maps of three resolutions output by the intermediate three-layer encoder structure are input into the multi-scale information interaction module, and the second feature map of the three resolutions is output, including: performing parameterized convolution on the first sub-feature map of low resolution and the second sub-feature map of medium resolution, respectively, to obtain a fourth sub-feature map and a fifth sub-feature map; performing large-core hole tower pooling operations on the second sub-feature map of medium resolution and the third sub-feature map of high resolution, respectively, to obtain a sixth sub-feature map and a seventh sub-feature map; splicing the feature map after fusing the fourth sub-feature map and the sixth sub-feature map with the feature map after fusing the fifth sub-feature map and the seventh sub-feature map to obtain an eighth sub-feature map; and inputting the eighth sub-feature map into the coordinate attention module to output the second feature map.

[0010] According to an embodiment of the present disclosure, performing a large-kernel hole tower pooling operation on the second sub-feature map of medium resolution to obtain a sixth sub-feature map includes: performing a convolution operation on the second sub-feature map based on three different sampling rates to obtain three first convolution feature maps; performing average pooling on the second sub-feature map to obtain a second convolution feature map; performing a dimensionality reduction operation on the second sub-feature map to obtain a third convolution feature map; and concatenating the three first convolution feature maps, the second convolution feature map and the third convolution feature map to obtain the sixth sub-feature map.

[0011] According to an embodiment of the present disclosure, the convolutional layers of each encoder structure in the five-layer encoder structure are in the form of Res2net.

[0012] According to an embodiment of the present disclosure, the first decoder includes a four-layer decoder structure that is the same as the UHRNet deep learning model; the first feature maps of the multiple resolutions are input into the first decoder, and the third feature maps of the multiple resolutions are output, including: the first feature maps of the two resolutions output by the fifth encoder structure and the fourth encoder structure are input into the first decoder structure, and the third feature map with the same resolution as the fourth encoder structure is output; the first feature map output by the third encoder structure and the third feature map output by the first decoder structure are input into the second decoder structure, and the third feature map with the same resolution as the third encoder structure is output; the first feature map output by the second encoder structure and the third feature map output by the second decoder structure are input into the third decoder structure, and the third feature map with the same resolution as the second encoder structure is output; and the third feature map output by the third decoder structure is input into the fourth decoder structure, and the third feature map with the same resolution as the first encoder structure is output.

[0013] According to an embodiment of the present disclosure, the second decoder includes a four-layer attention boundary refinement upsampling module and a large-core hole tower pooling module, and the four-layer attention boundary refinement upsampling module corresponds one-to-one to the levels of the four-layer decoder structure. The step of inputting the first feature maps of the multiple resolutions, the second feature maps and the third feature maps of the multiple resolutions into the second decoder, and outputting the land cover classification result of the land image, comprises: inputting the second feature map output by the fifth-layer encoder structure into the large-core holey tower pooling module, and outputting the fourth convolution feature map after the pooling operation; inputting the fourth convolution feature map and the third feature map output by the first-layer decoder structure into the first-layer attention boundary refinement upsampling module, and outputting the first boundary refinement feature map; inputting the first boundary refinement feature map, the first feature map output by the fourth-layer encoder structure, and the third feature map output by the second-layer decoder structure into the second-layer attention boundary refinement upsampling module, and outputting the second boundary refinement feature map; inputting the second boundary refinement feature map, the first feature map output by the third-layer encoder structure, and the third feature map output by the third-layer decoder structure into the third-layer attention boundary refinement upsampling module, and outputting the third boundary refinement feature map; inputting the third boundary refinement feature map, the second feature map, and the third feature map output by the fourth-layer decoder structure into the fourth-layer attention boundary refinement upsampling module, and outputting the land cover classification result.

[0014] According to an embodiment of the present disclosure, the first boundary refinement feature map, the first feature map output by the fourth-layer encoder structure, and the third feature map output by the second-layer decoder structure are input into a second-layer attention boundary refinement upsampling module to output a second boundary refinement feature map, including: fusing and downsampling the first feature map output by the fourth-layer encoder structure and the third feature map output by the second-layer decoder structure to obtain a first fused feature map; inputting the first boundary refinement feature map into an attention module with a spatial and channel parallel combination to output a second fused feature map; performing a transposed convolution on the second fused feature map to obtain a third fused feature map; and inputting the feature map after the first fused feature map and the third fused feature map are superimposed into a boundary refinement submodule to output the second boundary refinement feature map.

[0015] According to an embodiment of the present disclosure, it also includes: the loss function for training the deep learning model includes a boundary loss function, a sample category balance function, an optimized intersection-over-union function and a cross entropy loss function, wherein the boundary loss function is used to reduce the edge information difference of each category of sample land images.

[0016] The second aspect of the present disclosure provides a land cover classification device based on a deep learning model, including: an encoding module, used to input a land image to be detected into the encoder, and output a first feature map of multiple resolutions and a second feature map fused with multiple resolutions; a first decoding module, used to input the first feature map of the multiple resolutions into a first decoder, and output a third feature map of multiple resolutions; a second decoding module, used to input the first feature map of the multiple resolutions, the second feature map and the third feature map of the multiple resolutions into the second decoder, and output a land cover classification result of the land image.

[0017] The third aspect of the present disclosure provides an electronic device, comprising: one or more processors; a memory for storing one or more computer programs, which, when the one or more processors execute the one or more computer programs, implements the steps of the land cover classification method based on the deep learning model.

[0018] The fourth aspect of the present disclosure also provides a computer-readable storage medium having a computer program or instruction stored thereon, which, when executed by a processor, implements the steps of the land cover classification method based on the deep learning model.

[0019] The fifth aspect of the present disclosure also provides a computer program product, including a computer program or instructions, which, when executed by a processor, implement the steps of the land cover classification method based on the deep learning model.

[0020] The land cover classification method based on the deep learning model provided by the present disclosure includes at least the following technical effects:

[0021] (1) The D-UHRNet land cover classification model is constructed using UHRNet. The input image is encoded once and decoded twice to obtain the final classification result. This can improve the classification ability of different land object categories and achieve the purpose of improving the robustness of the model.

[0022] (2) The multi-scale feature information interaction module is used to improve the learning ability of ground object features, especially the extraction of similar ground object categories and small target categories, which alleviates the "salt and pepper" phenomenon to a certain extent.

[0023] (3) The attention boundary refinement upsampling module and the boundary loss are used as auxiliary loss functions to enhance the model's extraction of edge information of each category, thereby ensuring the integrity of the classification results of each category extracted and reducing the impact of the same object with different spectra. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The above contents and other purposes, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0025] Figure 1 The application scenario of the land cover classification method based on the deep learning model according to the embodiment of the present disclosure is schematically shown;

[0026] Figure 2 A flowchart of a land cover classification method based on a deep learning model according to an embodiment of the present disclosure is schematically shown;

[0027] Figure 3 The network structure diagram of D-UHRNet according to an embodiment of the present disclosure is schematically shown;

[0028] Figure 4 A schematic diagram of a network structure of a multi-scale information interaction module according to an embodiment of the present disclosure is schematically shown;

[0029] Figure 5 The network structure diagram of the large core hollow tower pooling module according to the embodiment of the present disclosure is schematically shown;

[0030] Figure 6 A schematic diagram schematically shows the structure of a convolutional layer of an encoder structure according to an embodiment of the present disclosure;

[0031] Figure 7 The network structure diagram of the attention boundary refinement upsampling module according to an embodiment of the present disclosure is schematically shown;

[0032] Figure 8 A schematic diagram schematically shows the fusion of spatial attention and channel attention according to an embodiment of the present disclosure;

[0033] Fig. 9 A land cover classification result map according to an embodiment of the present disclosure is schematically shown;

[0034] Fig.10 A block diagram of an electronic device suitable for a land cover classification method based on a deep learning model according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0035] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0036] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0037] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification, and should not be interpreted in an idealized or overly rigid manner.

[0038] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0039] In order to at least partially solve the above technical problems, the embodiments of the present disclosure perform one encoding and two decoding operations on the input land image to obtain the final land cover classification result, which can improve the classification ability of different land object categories and achieve the purpose of improving the robustness of the model. Specifically, the present disclosure provides a land cover classification method based on a deep learning model, and the deep learning model includes: an encoder, a first decoder, and a second decoder. The method includes: inputting the land image to be detected into the encoder, outputting a first feature map of multiple resolutions and a second feature map fused with multiple resolutions; inputting the first feature map of multiple resolutions into the first decoder, outputting a third feature map of multiple resolutions; and inputting the first feature map of multiple resolutions, the second feature map, and the third feature map of multiple resolutions into the second decoder, and outputting the land cover classification result of the land image.

[0040] Figure 1 The application scenario of the land cover classification method based on the deep learning model according to the embodiment of the present disclosure is schematically illustrated.

[0041] like Figure 1As shown, the application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104 and a server 105. The network 104 is used to provide a medium for a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc. A user may use at least one of the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, etc. The first terminal device 101, the second terminal device 102, and the third terminal device 103 may be various electronic devices with display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop portable computers, desktop computers, etc. The server 105 may be a server that provides various services.

[0042] It should be noted that the post-disaster building damage assessment method based on remote sensing images provided in the embodiment of the present disclosure can generally be executed by the server 105. Accordingly, the post-disaster building damage assessment device based on remote sensing images provided in the embodiment of the present disclosure can generally be set in the server 105. It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0043] The following will be based on Figure 1 The scene described by Figure 2 to Figure 8 The land cover classification method based on the deep learning model of the disclosed embodiment is described in detail.

[0044] Figure 2 A flowchart of a land cover classification method based on a deep learning model according to an embodiment of the present disclosure is schematically shown.

[0045] like Figure 2 As shown, the method 200 includes operations S210 to S230.

[0046] In operation S210, a land image to be detected is input into an encoder, and a first feature map with multiple resolutions and a second feature map with multiple resolutions are output.

[0047] In operation S220, first feature maps of a plurality of resolutions are input to a first decoder, and third feature maps of a plurality of resolutions are output.

[0048] In operation S230, the first feature maps of multiple resolutions, the second feature maps, and the third feature maps of multiple resolutions are input to a second decoder, and a land cover classification result of the land image is output.

[0049] According to an embodiment of the present disclosure, the deep learning model may be a Deep-UHRNet land cover classification model constructed based on a U-shaped High Resolution Net (U-HRNet), namely D-UHRNet.

[0050] In an embodiment of the present disclosure, D-UHRNet includes: an encoder, a first decoder, and a second decoder. The encoder of D-UHRNet includes the same five-layer encoder structure as the encoder of UHRNet, that is, the encoder includes five resolution levels, and the resolutions of the first-layer encoder structure, the second-layer encoder result, the third-layer encoder structure, the fourth-layer encoder result, and the fifth-layer encoder structure decrease in sequence. The first decoder of D-UHRNet is similar to the decoder of UHRNet and includes four resolution levels. The second decoder of D-UHRNet is similar to the first decoder and also includes four resolution levels, and the second decoder corresponds one-to-one to the levels of the first decoder.

[0051] It can be understood that the D-UHRNet body of the embodiment of the present disclosure includes 9 stages (5 stages of the encoder and 4 stages of the first decoder / second decoder) and 5 resolution streams.

[0052] According to an embodiment of the present disclosure, the land image to be detected may include a remote sensing image of the land. The land image may include at least one of the following land covers: buildings, roads, cultivated land, gardens, woodlands, grasslands, water bodies, etc.

[0053] Accordingly, the land cover classification result output by D-UHRNet for the land image to be detected also includes at least one of the above land covers. In the land cover classification result, the land cover can be indicated by a plurality of different colors.

[0054] After the land image to be detected is input into the encoder, the first feature maps of four resolutions can be output respectively through the last four layers of the five-layer encoder structure. The second feature map can be obtained by fusing the above multiple first feature maps. For example, the second feature map is obtained by fusing three first feature maps of different resolutions.

[0055] On the decoder side, the first decoder can decode the first feature maps of multiple resolutions into third feature maps of corresponding resolutions one by one. The second decoder can fuse and decode the first feature maps of multiple resolutions, the second feature maps and the third feature maps of multiple resolutions to obtain a land cover classification result that fuses multiple resolutions.

[0056] The embodiment of the present disclosure performs one encoding and two decoding operations on the input land image to obtain the final land cover classification result, which can improve the classification ability of different land object categories and achieve the purpose of improving the robustness of the model.

[0057] Figure 3 The network structure diagram of D-UHRNet according to an embodiment of the present disclosure is schematically shown.

[0058] like Figure 3 As shown in the figure, the upper side of the dotted line is the encoder of D-UHRNet, and the lower side of the dotted line is the first decoder and the second decoder of D-UHRNet. The encoder includes a multi-scale information interaction module (MIIM) and a 5-layer encoder structure, that is, 5 high resolution blocks (HRBlocks).

[0059] The following will be Figure 3 As a benchmark, the encoder side and the decoder side are described in detail respectively.

[0060] The encoder may output the first feature map at four resolutions, such as a first resolution, a second resolution, a third resolution, and a fourth resolution with successively decreasing resolutions. The specific operation of the encoder outputting the first feature map at four resolutions is as follows:

[0061] In the first stage, the first HRBlock extracts image features with the same resolution as the input land image, that is, the first resolution, through two bottleneck residual blocks, and generates two branches with different resolutions and their feature maps after downsampling. The first HRBlock outputs two branches with the first resolution and the second resolution, but does not output the first feature map.

[0062] In the second stage, for the second HRBlock, the feature maps of the first and second resolution branches are input. The above two branches first pass through the two stacked basic convolution blocks of their respective high resolution modules (HR Module) to further extract image features, and then are fused with the feature map of the other resolution branch through upsampling / downsampling to obtain two feature maps of different resolutions. At this time, the second HRBlock outputs the feature map of the first resolution as the first feature map, and downsamples the feature map of the second resolution (lower resolution) to form two new resolution branches for input into the third HRBlock.

[0063] In the third, fourth and fifth stages, the operations of the third, fourth and fifth HRBlocks are similar to those of the second HRBlock. The third, fourth and fifth HRBlocks output the first feature maps of the second resolution, the third resolution and the fourth resolution, respectively. Specifically, the structures of the second, third, fourth and fifth HRBlocks are as indicated by the dotted lines on the encoder side.

[0064] For the multi-scale information interaction module (MIIM), the multi-scale information interaction module can fuse the first feature maps of the three resolutions output by the 2nd, 3rd, and 4th HRBlocks into the second feature map, that is, fuse the first feature maps of the first resolution, the second resolution, and the third resolution into the second feature map.

[0065] For the three resolutions output by the 2nd, 3rd, and 4th HRBlocks, since the resolutions decrease successively, the first feature map of the third resolution can be used as the first sub-feature map of low resolution, the first feature map of the second resolution can be used as the second sub-feature map of medium resolution, and the first feature map of the first resolution can be used as the third sub-feature map of high resolution.

[0066] The embodiments of the present disclosure can improve the learning ability of D-UHRNet for ground feature features by adopting a multi-scale feature information interaction module, especially the extraction of similar ground feature categories and small target categories, which alleviates the "salt and pepper" phenomenon to a certain extent.

[0067] Figure 4 The schematic diagram shows a network structure diagram of a multi-scale information interaction module according to an embodiment of the present disclosure.

[0068] like Figure 4 As shown, the input of the multi-scale information interaction module (MIIM) is a first sub-feature map of low resolution (low-level), a second sub-feature map of medium resolution (mid-level) and a third sub-feature map of high resolution (high-level).

[0069] like Figure 4 As shown, the first sub-feature map of low resolution (low-level) and the second sub-feature map of mid-level (mid-level) can be respectively subjected to a parameterized convolution (Depthwise Over-parameterized Convolutional, DOConv) to obtain a fourth sub-feature map and a fifth sub-feature map;

[0070] The large kernel atrous spatial pyramid pooling operation is performed on the second sub-feature map of mid-level and the third sub-feature map of high-level, respectively, to obtain the sixth sub-feature map and the seventh sub-feature map. The large kernel atrous spatial pyramid pooling operation can be performed by the large kernel atrous spatial pyramid pooling module (LKASPP).

[0071] like Figure 4 As shown in the figure, for the fusion of low resolution and medium resolution, the matrix multiplication operation can be performed on the fourth sub-feature map output by DOConv and the sixth sub-feature map output by LKASPP; for the fusion of medium resolution and high resolution, the matrix multiplication operation can be performed on the fifth sub-feature map output by DOConv and the seventh sub-feature map output by LKASPP. After that, the outputs of the low-medium branch and the medium-high branch are fused by concatenation (Concat), and the feature map after the fusion of the two branches passes through the coordinate attention module (Coord) to obtain the eighth sub-feature map. Among them, Coord is used to compress the global spatial information of the feature map to the eighth sub-feature map.

[0072] According to an embodiment of the present disclosure, a large-kernel hole tower pooling operation is performed on a second sub-feature map of medium resolution to obtain a sixth sub-feature map, including: performing a convolution operation on the second sub-feature map based on three different sampling rates to obtain three first convolution feature maps; performing average pooling on the second sub-feature map to obtain a second convolution feature map; performing a dimensionality reduction operation on the second sub-feature map to obtain a third convolution feature map; and concatenating the three first convolution feature maps, the second convolution feature map and the third convolution feature map to obtain the sixth sub-feature map.

[0073] The embodiment of the present disclosure proposes a multi-scale information interaction module (such as Figure 4 As shown in the figure, the low-resolution shallow features, the medium-resolution mid-level features, and the high-resolution deep features are further extracted and then multiplied with each other as weights, so that the resulting feature map can contain more feature information, which can effectively reduce the occurrence of the "salt and pepper" phenomenon.

[0074] Figure 5 The network structure diagram of the large core cavity tower pooling module according to the embodiment of the present disclosure is schematically shown.

[0075] like Figure 5As shown, the large kernel atrous tower pooling module (LKASPP) convolves the input with the atrous convolution basic block LkBlock through three sampling rates rate1, rate2 and rate3 to obtain feature maps of the three sampling rates. As a specific example, the three sampling rates rate1, rate2 and rate3 can be 6, 12 and 18 respectively.

[0076] In addition, the input of LKASPP can be averaged through the pooling layer (AdaptiveAvgPool); the input of LKASPP can be reduced in dimension through the 1×1 convolution kernel (Conv1×1). Finally, the outputs of the five branches are fused into the final output through concatenation (Concat).

[0077] Large-kernel holey tower pooling operations for both medium and high resolutions can be implemented through the large-kernel holey tower pooling module (LKASPP). For example, based on three different sampling rates rate1, rate2, and rate3, the second sub-feature map is convolved to obtain three first convolution feature maps; the second sub-feature map is average pooled and dimensionally reduced by AdaptiveAvgPool and Conv1×1 respectively. The five outputs are fused to obtain the sixth sub-feature map.

[0078] According to an embodiment of the present disclosure, the convolutional layers of each encoder structure in the five-layer encoder structure are in the form of Res2net.

[0079] Figure 6 The structural diagram of the convolutional layer of the encoder structure according to the embodiment of the present disclosure is schematically shown.

[0080] like Figure 6 As shown in the figure, the input of the convolution layer is first reduced in dimension by the Conv1×1 convolution in the first branch, and then the reduced feature map is divided into four sub-channel sets: X1, X2, X3, and X4. Each sub-channel set of X1, X2, and X3 can obtain the corresponding feature subset map through convolution, and the feature map of the X4 channel is not convolved, and Y4 is X4.

[0081] For subchannel set X3, X3 is convolved (Conv) with a 3×3 convolution kernel to obtain Y3. For subchannel set X2, Y3 obtained by convolving subchannel set X3 is added to X2, and then the result of Y3+X2 is convolved (Conv) with a 3×3 convolution kernel to obtain Y2. For subchannel set X1, Y2 obtained by convolving subchannel set X2 is added to X1, and then the result of Y2+X1 is convolved (Conv) with a 3×3 convolution kernel to obtain Y1.

[0082] Concatenate Y1, Y2, Y3 and Y4 and perform Conv1×1 dimension upgrading to obtain the output of the first branch.

[0083] The input of the convolutional layer is also used as the result of the second branch, that is, the result of the residual branch, and is added to the result of the first branch to obtain the final output of the convolutional layer.

[0084] In the embodiments of the present disclosure, not only a deep learning network D-UHRNet is proposed based on the existing UHRNet semantic segmentation structure, but also a Res2net form (such as Figure 6 The convolutional layer (as shown in the figure) is used for feature extraction, which increases the receptive field range of each network layer, improves the multi-scale learning ability of features, is more suitable for extracting objects with multi-scale complex characteristics, and helps to improve the accuracy of feature extraction.

[0085] On the decoder side of D-UHRNet, the first decoder includes the same 4-layer decoder structure as UHRNet, namely 4 high resolution blocks (HRBlocks). The second decoder includes 4 layers of attention boundary refinement upsampled modules (ABRUM) and large kernel atrous tower pooling modules (LKASPP).

[0086] like Figure 4 As shown in the figure, the four HRBlocks from right to left correspond to the first layer decoder structure, the second layer decoder structure, the third layer decoder structure and the fourth layer decoder structure respectively. According to the data flow of D-UHRNet, the four HRBlocks from right to left are in the 6th stage, the 7th stage, the 8th stage and the 9th stage respectively.

[0087] In the first decoder, the input of the first three layers of the decoder structure includes two feature maps of different resolutions, and the input of the last layer of the decoder structure only includes a feature map of one resolution. The first three layers of the decoder structure retain the low-resolution features output by the previous stage, upsample them to obtain high-resolution feature maps, and then perform pooling operations and channel fusion on the feature maps of the same resolution as the previous stage to obtain the output of the decoder structure of this layer. Specifically:

[0088] In the sixth stage, the input of the first decoder structure is the first feature map of the fourth resolution output by the fifth encoder structure and the first feature map of the third resolution output by the fourth encoder structure. The first decoder structure upsamples the first feature map of the fourth resolution to the third resolution, and then performs pooling and channel fusion with the first feature map of the third resolution to obtain the third feature map of the third resolution.

[0089] In the 7th stage, the input of the second-layer decoder structure is the first feature map of the second resolution output by the third-layer encoder structure and the third feature map of the third resolution output by HRBlock in the 6th stage. The second-layer decoder structure upsamples the third feature map of the third resolution to the second resolution, and then performs pooling and channel fusion with the first feature map of the second resolution to obtain the third feature map of the second resolution.

[0090] In stage 8, the input of the third-layer decoder structure is the first feature map of the first resolution output by the second-layer encoder structure and the third feature map of the second resolution output by HRBlock in stage 7. The third-layer decoder structure upsamples the third feature map of the second resolution to the first resolution, and then performs pooling and channel fusion with the first feature map of the first resolution to obtain the third feature map of the first resolution.

[0091] In the 9th stage, the input of the fourth-layer decoder structure is only the third feature map of the first resolution output by the third-layer decoder structure, and the third feature map of the first resolution is output through the convolution operation of the convolution layer.

[0092] On the decoder side, the output of each decoder structure in the first decoder needs to be used not only as the input of the next decoder structure in the first encoder, but also as the input of the Attention Boundary Refinement Upsampling Module (ABRUM) at the same level in the second encoder. Figure 4 As shown, the second decoder includes four levels of ABRUM from right to left, and the output of each ABRUM serves as one of the inputs of the next ABRUM.

[0093] In the sixth stage, i.e. the first ABRUM, the input of the first ABRUM includes: the output of the sixth stage HRBlock and the output of the large core hole tower pooling module (LKASPP). The input of LKASPP is the output of the fifth stage (the fifth encoder structure), i.e. the first feature map of the fourth resolution. The network structure of LKASPP can be found in Figure 5 It should also be noted that the input of the first ABRUM includes two resolutions: the fourth convolutional feature map of the fourth resolution from LKASPP and the third feature map of the third resolution output by the sixth stage HRBlock.

[0094] In the 7th stage, the input of the 2nd ABRUM includes: the output of the 7th stage HRBlock (the third feature map of the second resolution), the output of the 1st ABRUM, and the output of the fourth layer encoder structure (the first feature map of the third resolution).

[0095] In the 8th stage, the input of the 3rd ABRUM includes: the output of the 8th stage HRBlock (the third feature map of the first resolution), the output of the 2nd ABRUM, and the output of the third layer encoder structure (the first feature map of the second resolution).

[0096] In the 9th stage, the input of the 4th ABRUM includes: the output of the 9th stage HRBlock (the third feature map after convolution of the first resolution), the output of the 3rd ABRUM, and the output of MIIM (the second feature map fused with multiple resolutions).

[0097] Among them, the output of the first ABRUM, the output of the second ABRUM and the output of the third ABRUM are the first boundary refinement feature map, the second boundary refinement feature map and the third boundary refinement feature map respectively.

[0098] In one embodiment, the fourth ABRUM may be connected to an output layer, so that the output of the fourth ABRUM is a land cover classification result. Alternatively, the fourth ABRUM outputs a fourth boundary refinement feature map, and the fourth boundary refinement feature map is processed into a land cover classification result through the output layer connected after the fourth ABRUM.

[0099] The embodiments of the present disclosure can enhance the extraction of edge information of each category by the D-UHRNet model by adopting the attention boundary refinement upsampling module, thereby ensuring the integrity of the classification results of each category obtained by each extraction and reducing the impact of the same object with different spectra.

[0100] Figure 7 The network structure diagram of the attention boundary refinement upsampling module according to an embodiment of the present disclosure is schematically shown.

[0101] like Figure 7 As shown, the input of the attention boundary refinement upsampling module (ABRUM) includes Mj from the encoder side, and Ni and Nj from the decoder side. Among them, Nj represents the third feature map output by the same level HRBlock in the first decoder, and Ni represents the boundary refinement feature map output by the previous level ABRUM in the second decoder.

[0102] like Figure 7 As shown in the figure, Mj from the encoder and Nj from the first decoder are fused by concatenation (Concat); then, the concatenated features are convolved (Conv1×1) using a 1×1 convolution kernel to obtain a downsampled feature map.

[0103] For Ni from the second decoder, feature extraction is performed through a dual attention module (DAM) that combines space and channels in parallel. After that, the output of DAM is transposed convolution (TransConv) to obtain the output of this branch.

[0104] Finally, the outputs of the two branches are superimposed (Add), and the superimposed feature map is input into the boundary refinement submodule (Boundary Refinement, BR). The output of BR is the output of ABRUM.

[0105] The boundary refinement submodule (BR) also includes two branches. One branch reduces the dimension through Conv3×3 convolution, ReLU activation, and Conv3×3 convolution, while the other branch does not process the input of BR. The output of BR is obtained by adding the outputs of the two branches.

[0106] In a preferred embodiment, the four ABRUMs are identical, and an output layer is further provided after the fourth ABRUM so as to output the land cover classification result through the second decoder.

[0107] Figure 8 A schematic diagram of the fusion of spatial attention and channel attention according to an embodiment of the present disclosure is schematically shown.

[0108] like Figure 8 As shown in the figure, after Ni is input into DAM, DAM divides the Ni features into three branches, one branch is used to extract spatial attention, one branch is used to extract channel attention, and the spatial attention and channel attention are fused by feature addition. In addition, the third branch of DAM is the input Ni, which is used to perform a residual connection between Ni and the features that fuse the spatial attention and channel attention, for example, by performing a residual connection by feature addition.

[0109] For the process of extracting spatial attention, the average pooling and maximum pooling features of Ni can be concatenated first, and the concatenated features can be convolved to obtain the attention information of the spatial dimension; then, the convolved spatial dimension attention information is matrix multiplied with Ni and normalized (sigmoid) to obtain spatial attention.

[0110] For the process of extracting channel attention, similarly, average pooling and maximum pooling are first performed on Ni, and then the features after average pooling and maximum pooling are sent to the multi-layer perceptron (MLP) to share information. Finally, the shared results are applied to the two channels of average pooling and maximum pooling. Then, the features of the two channels are added in turn, the added features are matrix multiplied with Ni, and the result of the matrix multiplication is normalized using the sigmoid function to obtain channel attention.

[0111] The embodiment of the present disclosure cuts the obtained public data set GID-15 into a size of 512*512, and then divides it into a training set of 18,200 images and a validation set of 1,820 images, and re-plans the categories contained in the data set into 7 categories, namely, buildings, roads, cultivated land, gardens, woodlands, grasslands, and water bodies. Therefore, after training D-UHRNet with the training set, the land cover can be divided into seven categories: buildings, roads, cultivated land, gardens, woodlands, grasslands, and water bodies.

[0112] In the training process of the D-UHRNet model, the training set data is used as the input image, and is encoded into 256-channel features by the encoder step by step, and then decoded step by step by the first decoder and the second decoder to the segmentation result map of the same size as the input image. The initial learning rate of the model is set to 0.001, and the Adam optimizer is used with a batch size of 8. The polynomial learning rate decay strategy is used: lr = base_lr*(0.5*(epoch / / 40)), base_lr is the base learning rate, and epoch refers to the process of training once using all the samples in the training set, and " / / " means dividing first and then rounding down.

[0113] According to an embodiment of the present disclosure, the loss functions used to train the D-UHRNet model include: a boundary loss function, a sample category balance function, an optimized intersection-over-union function and a cross entropy loss function, wherein the boundary loss function is used to reduce the edge information difference of each category of sample land images.

[0114] Taking into account the problem that the edge information of each land cover category is mostly incomplete and inaccurate, the present invention not only strengthens the model's learning of edge information based on the attention boundary refinement upsampling module, but also improves the loss function and proposes a boundary loss function BoundaryLoss to assist the learning of the model.

[0115] The specific definition of the boundary loss function BoundaryLoss is shown in formula (1):

[0116]

[0117] Among them, G(i,j) represents the boundary information map obtained by using the Laplace operator to extract the edge of each category. The i in G(i,j) is the x-axis coordinate, and j is the y-axis coordinate. p (i,j) and G g (i, j) represents the predicted pixel value and the real pixel value at the (i, j) coordinate respectively. According to formula (1), the difference between the pixel values ​​of the boundary information map G and the real boundary information map can be calculated to measure the defects of the edge information of the classification result.

[0118] The calculation method of the boundary information graph G can refer to formula (2):

[0119] G(x,y)&=f(x-1,y-1)+f(x,y-1)+f(x+1,y-1)+f(x-1,y)

[0120] +f(x+1,y)+f(x-1,y+1)+f(x,y+1)+f(x+1,

[0121] y+1)-8f(x,y)(2)

[0122] The loss function of the entire D-UHRNet model is composed of multiple loss functions, namely: boundary loss function BoundaryLoss, sample category balance function FocalLoss for balancing the imbalanced category distribution problem, optimized intersection over union (IOU) function LovaszSoftmax and cross entropy loss function Cross Entropy, i.e. CE. The loss function is shown in formula (3):

[0123] Loss=λ1CE+λ2FocalLoss+λ3LovaszSoftmax+λ4BoundaryLoss (3)

[0124] Among them, λ1, λ2, λ3, and λ4 are adjustment weights of the respective loss functions. In a specific embodiment, λ1, λ2, λ3, and λ4 can be set to 1, 0.3, 0.4, and 0.3, respectively.

[0125] According to an embodiment of the present disclosure, after the D-UHRNet model is trained using a training set, the trained model can be tested in a test set to obtain a land cover classification result of the test set.

[0126] Fig. 9 A land cover classification result map according to an embodiment of the present disclosure is schematically shown.

[0127] like Fig. 9As shown, Fig. 9 It includes three groups of land images to be tested (a), (c) and (e), and the land cover classification results (b), (d) and (f) corresponding to the land images (a), (c) and (e) respectively. It can be concluded from the result graph that the D-UHRNet model can clearly extract the boundaries of the categories, and each category can also be clearly distinguished, and the "salt and pepper" phenomenon is also greatly alleviated.

[0128] In addition, the embodiment of the present disclosure also calculates the average intersection-over-union (mIOU), average F1 value (mF1), accuracy (Accuracy) and recall (Recall) for the entire test sample set to evaluate the accuracy of the D-UHRNet model, and obtains the statistical results shown in Table 1.

[0129] Table 1 Land cover classification accuracy verification statistics

[0130] mIOU Recall Accuracy mF1 80.77％ 79.06％ 90.86％ 89.19

[0131] The calculation method of mIOU is shown in formula (4), and the specific calculation methods of Accuracy and mF1 values ​​are shown in formulas (5) and (6):

[0132]

[0133]

[0134]

[0135] Among them, TP c To correctly predict the number of pixels for each category, FP c is the number of pixels incorrectly predicted as each class, TN c To correctly predict the number of pixels as background, FN c is the number of pixels that are incorrectly predicted as background. N represents the number of categories in the dataset, and c represents a category in the land cover category.

[0136] The embodiments of the present disclosure adopt a boundary loss function boundaryloss as an auxiliary loss function to enhance the model's extraction of edge information of each category, thereby ensuring the integrity of the classification results of each category obtained by the extraction and reducing the impact of different spectra of the same object.

[0137] In an embodiment of the present disclosure, the land cover classification device based on the deep learning model of the embodiment may include at least one module, sub-module, unit, sub-unit, which is used to input the land image to be detected into an encoder, output a first feature map of multiple resolutions and a second feature map that fuses multiple resolutions; input the first feature map of multiple resolutions into a first decoder, output a third feature map of multiple resolutions; and input the first feature map of multiple resolutions, the second feature map and the third feature map of multiple resolutions into a second decoder, and output the land cover classification result of the land image.

[0138] It should be noted that the device part in the embodiments of the present disclosure corresponds to the method part in the embodiments of the present disclosure. The description of the device part specifically refers to the method part and will not be repeated here.

[0139] According to the embodiments of the present disclosure, any multiple modules among the modules, submodules, units, and subunits can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of the modules can be combined with at least part of the functions of other modules and implemented in one module.

[0140] According to an embodiment of the present disclosure, at least one of the modules, submodules, units, and subunits may be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or may be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or may be implemented in any one of the three implementation methods of software, hardware, and firmware, or in an appropriate combination of any of them. Alternatively, at least one of the modules, submodules, units, and subunits may be at least partially implemented as a computer program module, which may perform corresponding functions when the computer program module is executed.

[0141] Fig.10 A block diagram of an electronic device suitable for a land cover classification method based on a deep learning model according to an embodiment of the present disclosure is schematically shown.

[0142] like Fig.10As shown, the electronic device 1000 according to an embodiment of the present disclosure includes a processor 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage part 1008 into a random access memory (RAM) 1003. The processor 1001 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 1001 may also include an onboard memory for caching purposes. The processor 1001 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0143] In RAM 1003, various programs and data required for the operation of electronic device 1000 are stored. Processor 1001, ROM 1002 and RAM 1003 are connected to each other via bus 1004. Processor 1001 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 1002 and / or RAM 1003. It should be noted that the program can also be stored in one or more memories other than ROM 1002 and RAM 1003. Processor 1001 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in the one or more memories.

[0144] According to an embodiment of the present disclosure, the electronic device 1000 may further include an input / output (I / O) interface 1005, which is also connected to the bus 1004. The electronic device 1000 may further include one or more of the following components connected to the input / output I / O interface 1005: an input portion 1006 including a keyboard, a mouse, etc.; an output portion 1007 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 1008 including a hard disk, etc.; and a communication portion 1009 including a network interface card such as a LAN card, a modem, etc. The communication portion 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to the I / O interface 1005 as needed. A removable medium 1011, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1010 as needed, so that a computer program read therefrom is installed into the storage portion 1008 as needed.

[0145] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.

[0146] According to an embodiment of the present disclosure, a computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present disclosure, a computer-readable storage medium may include the ROM 1002 and / or RAM 1003 described above and / or one or more memories other than ROM 1002 and RAM 1003.

[0147] The embodiment of the present disclosure also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the above method provided by the embodiment of the present disclosure.

[0148] The above functions defined in the system / device of the embodiment of the present disclosure are performed when the computer program is executed by the processor 1001. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0149] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 1009, and / or installed from the removable medium 1011. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0150] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 1009, and / or installed from the removable medium 1011. When the computer program is executed by the processor 1001, the above functions defined in the system of the embodiment of the present disclosure are performed. According to the embodiment of the present disclosure, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.

[0151] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).

[0152] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0153] It will be appreciated by those skilled in the art that the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways, even if such combinations and / or combinations are not explicitly described in the present disclosure. In particular, the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways without departing from the spirit and teachings of the present disclosure. All of these combinations and / or combinations fall within the scope of the present disclosure.

[0154] The specific embodiments described above further illustrate the purpose, technical solutions and beneficial effects of the present disclosure. It should be understood that the above description is only a specific embodiment of the present disclosure and is not intended to limit the present disclosure. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present disclosure should be included in the protection scope of the present disclosure.

Claims

1. A land cover classification method based on a deep learning model, the deep learning model comprising: An encoder, a first decoder and a second decoder, the method comprising: Inputting a land image to be detected into the encoder, outputting a first feature map with multiple resolutions and a second feature map with multiple resolutions integrated; Inputting the first feature maps of the plurality of resolutions into a first decoder, and outputting third feature maps of the plurality of resolutions; and Inputting the first feature map of the multiple resolutions, the second feature map and the third feature map of the multiple resolutions into the second decoder, and outputting a land cover classification result of the land image; The encoder includes a multi-scale information interaction module and a five-layer encoder structure that is the same as the UHRNet deep learning model; wherein the land image to be detected is input into the encoder, and a first feature map with multiple resolutions and a second feature map with multiple resolutions is output, including: After performing a convolution operation on the land image to be detected through the first layer of encoder structure, first feature maps of four resolutions are outputted through the next four layers of encoder structure in sequence; and Inputting the first feature maps of three resolutions output by the intermediate three-layer encoder structure into the multi-scale information interaction module, and outputting a second feature map integrating the three resolutions; The first feature maps of the three resolutions include a first sub-feature map of low resolution, a second sub-feature map of medium resolution, and a third sub-feature map of high resolution; the first feature maps of the three resolutions output by the intermediate three-layer encoder structure are input into the multi-scale information interaction module, and the second feature map of the three resolutions is output, including: Performing parameterized convolution on the first sub-feature map with low resolution and the second sub-feature map with medium resolution respectively to obtain a fourth sub-feature map and a fifth sub-feature map; Performing large-core hole tower pooling operations on the second sub-feature map with medium resolution and the third sub-feature map with high resolution, respectively, to obtain a sixth sub-feature map and a seventh sub-feature map; splicing a feature map obtained by fusing the fourth sub-feature map and the sixth sub-feature map with a feature map obtained by fusing the fifth sub-feature map and the seventh sub-feature map to obtain an eighth sub-feature map; and The eighth sub-feature map is input into a coordinate attention module, and the second feature map is output.

2. The method according to claim 1, wherein: The large-core hole tower pooling operation is performed on the second sub-feature map of the medium resolution to obtain a sixth sub-feature map including: Based on three different sampling rates, a convolution operation is performed on the second sub-feature map to obtain three first convolution feature maps; Performing average pooling on the second sub-feature map to obtain a second convolutional feature map; Performing a dimensionality reduction operation on the second sub-feature map to obtain a third convolution feature map; and The three first convolutional feature maps, the second convolutional feature map and the third convolutional feature map are concatenated to obtain the sixth sub-feature map.

3. The method according to claim 1, wherein: The convolutional layers of each encoder structure in the five-layer encoder structure are in Res2net form.

4. The method according to claim 1, wherein: The first decoder includes a four-layer decoder structure identical to the UHRNet deep learning model; The step of inputting the first feature maps of the plurality of resolutions into a first decoder and outputting third feature maps of the plurality of resolutions comprises: Inputting the first feature maps of two resolutions output by the fifth layer encoder structure and the fourth layer encoder structure into the first layer decoder structure, and outputting a third feature map with the same resolution as the fourth layer encoder structure; Input the first feature map output by the third-layer encoder structure and the third feature map output by the first-layer decoder structure into the second-layer decoder structure, and output a third feature map with the same resolution as the third-layer encoder structure; Inputting the first feature map output by the second layer encoder structure and the third feature map output by the second layer decoder structure into the third layer decoder structure, and outputting the third feature map with the same resolution as the second layer encoder structure; and The third feature map output by the third layer decoder structure is input into the fourth layer decoder structure, and the third feature map with the same resolution as the first layer encoder structure is output.

5. The method according to claim 4, wherein: The second decoder includes a four-layer attention boundary refinement upsampling module and a large-core hole tower pooling module, and the four-layer attention boundary refinement upsampling module corresponds one-to-one to the layers of the four-layer decoder structure; The step of inputting the first feature map of the plurality of resolutions, the second feature map, and the third feature map of the plurality of resolutions into the second decoder and outputting the land cover classification result of the land image comprises: Inputting the second feature map output by the fifth layer encoder structure into the large core hole tower pooling module, and outputting the fourth convolution feature map after the pooling operation; Input the fourth convolutional feature map and the third feature map output by the first layer decoder structure into a first layer attention boundary refinement upsampling module, and output a first boundary refinement feature map; Input the first boundary refinement feature map, the first feature map output by the fourth layer encoder structure, and the third feature map output by the second layer decoder structure into a second layer attention boundary refinement upsampling module, and output a second boundary refinement feature map; Input the second boundary refinement feature map, the first feature map output by the third layer encoder structure, and the third feature map output by the third layer decoder structure into a third layer attention boundary refinement upsampling module, and output a third boundary refinement feature map; The third boundary refinement feature map, the second feature map, and the third feature map output by the fourth layer decoder structure are input into the fourth layer attention boundary refinement upsampling module, and the land cover classification result is output.

6. The method according to claim 5, wherein: The step of inputting the first boundary refinement feature map, the first feature map output by the fourth layer encoder structure, and the third feature map output by the second layer decoder structure into a second layer attention boundary refinement upsampling module, and outputting a second boundary refinement feature map, comprises: Fusing and downsampling the first feature map output by the fourth-layer encoder structure and the third feature map output by the second-layer decoder structure to obtain a first fused feature map; Input the first boundary refinement feature map into the attention module of spatial and channel parallel combination, and output a second fused feature map; performing a transposed convolution on the second fused feature map to obtain a third fused feature map; and The feature map obtained by superimposing the first fused feature map and the third fused feature map is input into the boundary refinement submodule, and the second boundary refinement feature map is output.

7. The method according to claim 1, further comprising: The loss functions for training the deep learning model include a boundary loss function, a sample category balance function, an optimized intersection-over-union function and a cross entropy loss function, wherein the boundary loss function is used to reduce the edge information difference of the land images of each category of samples.

8. An electronic device comprising: one or more processors; a memory for storing one or more computer programs, It is characterized in that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 7.

Images