A remote sensing object classification method based on deep learning

Abstract

The present invention discloses a remote sensing object classification method based on deep learning, which includes obtaining a remote sensing object classification data set, constructing a remote sensing image generation model, inputting the remote sensing object classification data set into the remote sensing image generation model to achieve data enhancement; constructing a neural network model for remote sensing object classification, inputting the data-enhanced remote sensing object classification data set into the neural network model for training, and generating optimal weights after training. Any remote sensing image is input into the trained model to generate a remote sensing object classification image. The present invention improves the balanced proportion of each object type by performing data enhancement on the remote sensing object image, thereby improving the overall object classification accuracy. By performing remote sensing object classification through the constructed network model, the extraction effect of irregular large-sized object types is enhanced, the problem of difficult extraction of small target details of remote sensing objects and difficult distinction of edges of complex objects is improved, and the overall accuracy of remote sensing object classification is improved.

Description

Technical Field

[0001] The present invention relates to the field of remote sensing object classification, and in particular to a remote sensing object classification method based on deep learning. Background Art

[0002] Remote sensing images are captured by satellites or aerial vehicles, recording the electromagnetic wave size of various ground objects. Remote sensing object classification is the study of analyzing remote sensing images to derive information such as the distribution of local object types. As an important area of ​​remote sensing research, remote sensing object classification plays a key role in urban planning, resource exploration, post-disaster relief, military reconnaissance, and other fields. Accurate and rapid semantic segmentation of remote sensing images is a primary challenge in remote sensing object classification.

[0003] In the early days, remote sensing object classification primarily relied on machine learning algorithms such as support vector machines (SVM) and K-Means. With the development of deep learning algorithms, various neural networks have become a major research tool in computer vision. However, due to the unique characteristics of remote sensing images, objects exhibit irregular shapes, large objects intersect with small objects, and feature edges are complex. Therefore, deep learning algorithms commonly used in image segmentation are not fully suitable for remote sensing object classification. Therefore, improving deep learning algorithms to address these unique characteristics and make them suitable for remote sensing object classification research is of paramount importance. Summary of the Invention

[0004] The purpose of the present invention is to provide a remote sensing object classification method based on deep learning, which can effectively improve the accuracy of remote sensing object classification.

[0005] The specific steps include:

[0006] Step 1: Obtain a remote sensing object classification dataset, which includes the original images of remote sensing object classification and their corresponding original annotations; divide the dataset into a training set and a validation set. The training set is used to train the neural network model, and the validation set is used to verify the accuracy of the remote sensing object classification after the neural network model is trained.

[0007] Step 2: Build a remote sensing image generation model based on the pix2pix algorithm in the conditional generative adversarial network model:

[0008] The remote sensing image generation model uses a U-Net framework, comprising an encoder, a decoder, and skip connections for linking low-level semantic information with high-level semantic information. The encoder employs a ResNet50 architecture, with a channel-wise attention mechanism incorporated into the skip connections, enabling the network to prioritize valid object features and ignore unnecessary ones. The decoder model includes upsampling and convolutional layers.

[0009] Step 3: Input the remote sensing object classification dataset from Step 1 into the remote sensing image generation model constructed in Step 2 for training. Based on the actual surface conditions and annotation map of the remote sensing object classification dataset, manually draw a label map. Input the manually drawn label map into the remote sensing image generation model to generate a remote sensing image. This remote sensing image and label map are then added to the original remote sensing object classification dataset to achieve data augmentation.

[0010] Step 4: Build a neural network model for remote sensing object classification:

[0011] The neural network model for remote sensing object classification adopts a U-shaped framework, including encoder, decoder and skip connection;

[0012] The encoder is divided into five stages. Stages 1 through 4 feed the final feature map data into the skip connections, and the fifth stage outputs the final image data to the decoder. Each stage uses both large and small convolution kernels for simultaneous image feature extraction. Using a structural reparameterization method, the large and small convolution kernels are trained synchronously, combining their feature extraction results. The optimized parameters of the small convolution kernel are then incorporated into the parameters of the large convolution kernel for dual optimization.

[0013] A channel attention mechanism is added to the skip connection structure to make the network model pay more attention to key semantic information, and it is combined with the long-term and short-term memory structure gating mechanism to enable high-level semantic information to have the ability to guide the channel weights of low-level semantic information, making the weights allocated by the channel attention mechanism more reasonable.

[0014] In the decoder, the image size is expanded through upsampling operations. In each stage, the feature data of the ground object input by the jump connection of the stage and the feature data after the upsampling operation of the previous stage are spliced, and two convolution operations are performed before upsampling operations, so that the feature image data is gradually restored to the original image size.

[0015] Step 5: Input the enhanced remote sensing feature classification dataset from step 3 into the neural network model constructed in step 4 for training. After training, the optimal weights are generated. Input any remote sensing image into the trained model to generate a remote sensing feature classification image.

[0016] By adopting the above technical solution, the present invention has the following beneficial effects: by performing data enhancement on remote sensing ground object images, the proportion of different feature types is balanced and improved, thereby improving the overall accuracy of ground object classification. The constructed network model for remote sensing ground object classification enhances the extraction of irregular large-scale ground object types, improves the difficulty in extracting small remote sensing ground object details and the difficulty in distinguishing complex ground object edges, and improves the overall accuracy of remote sensing ground object classification.

[0017] This network encoding model uses large (9×9) and small (3×3) convolution kernels for simultaneous image feature extraction to improve the network's receptive field and enhance its ability to discriminate and classify irregular, large-scale features. Using a structural reparameterization method, the large (9×9) and small (3×3) convolution kernels are trained simultaneously, combining their feature extraction results. This results in a network with both robust extraction of irregular, large-scale features and the ability to recognize complex, small objects and classify edges. Furthermore, during each training optimization, the optimized parameters of the small convolution kernels are incorporated into the parameters of the large convolution kernels for dual optimization. Due to the complex convolutional structure of this network model, the number of parameters is relatively large, which somewhat impacts training efficiency. By reducing the dimensionality of the training channels and simplifying the extraction process, training time is accelerated. The encoder allows the network to learn identity mapping information, compensating for the oversight of details and preventing network degradation. A channel attention mechanism is added to the skip connection structure to make the network model pay more attention to key semantic information, and it is combined with the long-term and short-term memory structure gating mechanism to enable high-level semantic information to have the ability to guide the channel weights of low-level semantic information, making the weights allocated by the channel attention mechanism more reasonable, so as to improve the ability to extract land features and enhance the accuracy of land feature classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 This is the neural network structure diagram of the remote sensing object classification of the present invention

[0019] Figure 2 Schematic diagram of a data set of the present invention;

[0020] Figure 3 This is the result diagram of the classification of remote sensing objects by the neural network model of the present invention. DETAILED DESCRIPTION

[0021] To make the technical solution and advantages of the present invention clearer, the following is a detailed description of the specific implementation steps of the technical solution of the present invention with reference to examples. A remote sensing feature classification method based on deep learning specifically includes the following steps:

[0022] Step 1: Take the LoveDA remote sensing object classification dataset as an example. Figure 2 The dataset shown includes original images of remote sensing object classification and their corresponding original annotations; they are randomly cut into image sizes of 256×256×3 and divided into training set and validation set in a ratio of 9:1. The training set is used to train the neural network model, and the validation set is used to verify the accuracy of the remote sensing object classification after the neural network model is trained.

[0023] Step 2: Build a remote sensing image generation model based on the pix2pix algorithm in the generative adversarial network model:

[0024] The remote sensing image generation model uses a U-Net framework, comprising an encoder, a decoder, and skip connections for linking low-level semantic information with high-level semantic information. The encoder employs a residual network structure to increase network depth and enhance feature extraction capabilities. A channel-wise attention mechanism is incorporated into the skip connections to assign weights to feature information, allowing the network model to prioritize valid object features and ignore unnecessary ones. The decoder model includes upsampling and convolutional layers.

[0025] Step 3: Input the dataset from Step 1 into the model built in Step 2 for training to generate the optimal weight parameters. Draw a label map based on the LoveDA dataset and add the generated remote sensing images to the LoveDA dataset to complete data augmentation.

[0026] Step 4: Build Figure 1 The neural network model for remote sensing object classification is shown:

[0027] The model adopts a U-shaped framework globally, including an encoder, a decoder and a jump connection;

[0028] In the encoder, there are five stages:

[0029] In the first stage, after the 256×256×3 remote sensing image data is input into the encoder, it first enters the convolution layer with 64 channels and a convolution kernel size of 7×7 for convolution. The image size after convolution is 128×128×64. The convolution result is output to the normalization layer for data normalization. Finally, the ReLU nonlinear function is used for calculation, and the calculation result is input into the 3×3 maximum pooling layer. The image size becomes 64×64×64 to reduce the image size and increase the receptive field area.

[0030] In the second stage, the 64×64×64 image data output from the first stage is convolved in a convolutional layer with 64 channels and a kernel size of 1×1. The resulting image size is 64×64×64. The convolution result is fed into a normalization layer for data normalization, and finally, a ReLU nonlinear function is applied. The result is then fed into a convolutional layer with 64 channels and a kernel size of 3×3, and another convolutional layer with 64 channels and a kernel size of 9×9. The resulting image size is 64×64×64. The convolution results are then fed into a normalization layer for data normalization, and the two normalized results are added together. Finally, a ReLU nonlinear function is applied. The result is then fed into a convolutional layer with 256 channels and a kernel size of 1×1, resulting in an image size of 64×64×256. The convolution result is then fed into a normalization layer for data normalization.

[0031] The image data is then fed into a convolutional layer with 256 channels and a 1×1 kernel size. The resulting image size is 64×64×256. The convolution result is then fed into a normalization layer for data normalization to preserve feature information, preventing loss during the convolution process and network degradation. The normalized processing structure is then added to the normalized processing result of the convolutional structure above, and the resulting image size is then calculated using the ReLU nonlinear function before output, resulting in a 64×64×256 image.

[0032] The above operation is performed twice on the output 64×64×256 image. After obtaining the 64×64×256 image for the first time in the operation, the image data is not input into the convolution layer with a channel number of 256 and a convolution kernel size of 1×1 for convolution. The remaining steps are the same as the above operation.

[0033] In the third stage, the 64×64×256 image output from the second stage is convolved in a convolutional layer with 128 channels, a kernel size of 1×1, and a stride of 2. The resulting image size is 32×32×128. The convolution result is fed into a normalization layer for data normalization, and finally, the ReLU nonlinear function is used for calculation. The calculation results are fed into a convolutional layer with 128 channels and a kernel size of 3×3, and a convolutional layer with 128 channels and a kernel size of 9×9. After convolution, the image size of both images is 32×32×128. The convolution results of both are fed into a normalization layer for data normalization, and the two normalized results are added together. Finally, the ReLU nonlinear function is used for calculation. The calculation result is input into the convolution layer with a channel number of 512 and a convolution kernel size of 1×1 for convolution. The image size after convolution is 32×32×512. The convolution result is output to the normalization layer for data normalization.

[0034] The image is then convolved through a convolutional layer with 512 channels, a kernel size of 1×1, and a stride of 2. The resulting image size is 32×32×512. The convolution result is then fed into a normalization layer for data normalization to preserve feature information, preventing loss during the convolution process and network degradation. The normalized processing structure is then added to the normalized processing result of the convolutional structure above and then processed using the ReLU nonlinear function for output, resulting in an image size of 32×32×512.

[0035] The above operation is performed three times on the output 32×32×512 image. The convolution layer stride is no longer limited to 2 during the operation. After obtaining the 32×32×512 image for the first time, the convolution layer with 512 image data channels, 1×1 convolution kernel size, and stride of 2 is not convolved. The remaining steps are consistent with the above operation.

[0036] In the fourth stage, the 32×32×512 image data output from the third stage is convolved into a convolutional layer with 256 channels, a kernel size of 1×1, and a stride of 2. The resulting image size is 16×16×256. The convolution result is fed into a normalization layer for data normalization, and finally, the ReLU nonlinear function is used for operation. The operation results are fed into a convolutional layer with 256 channels and a kernel size of 3×3, and a convolutional layer with 256 channels and a kernel size of 9×9, respectively. After convolution, the image size of both images is 16×16×256. The convolution results of both are fed into a normalization layer for data normalization, and the two normalized results are added together. Finally, the ReLU nonlinear function is used for operation. The calculation result is input into the convolution layer with a channel number of 1024 and a convolution kernel size of 1×1 for convolution. The image size after convolution is 16×16×1024. The convolution result is output to the normalization layer for data normalization.

[0037] The image is then convolved through a convolutional layer with 1024 channels, a kernel size of 1×1, and a stride of 2. The resulting image size is 16×16×1024. The convolution result is then sent to a normalization layer for data normalization to preserve feature information, preventing loss during the convolution process and network degradation. The normalized processing structure is added to the normalized processing result of the convolutional structure above, and then the ReLU nonlinear function is used for the output, resulting in an image size of 16×16×1024.

[0038] The above operation is performed three times on the output 16×16×1024 image. The stride of the convolution layer is no longer limited to 2 in the operation. After obtaining the 16×16×1024 image for the first time, the convolution layer with 1024 input channels, 1×1 convolution kernel size, and stride of 2 is not convolved. The remaining steps are the same as the above operation.

[0039] In the fifth stage, the 16×16×1024 image data output from the fourth stage is convolved into a convolutional layer with 512 channels, a kernel size of 1×1, and a stride of 2. The resulting image size is 8×8×512. The convolution result is fed into a normalization layer for data normalization, and finally, the ReLU nonlinear function is used for calculation. The calculation results are fed into a convolutional layer with 512 channels and a kernel size of 3×3, and a convolutional layer with 512 channels and a kernel size of 9×9, respectively. After convolution, the image size of both images is 8×8×512. The convolution results of both are fed into a normalization layer for data normalization, and the two normalized results are added together. Finally, the ReLU nonlinear function is used for calculation. The calculation result is input into the convolution layer with a channel number of 2048 and a convolution kernel size of 1×1 for convolution. The image size after convolution is 8×8×2048. The convolution result is output to the normalization layer for data normalization.

[0040] The image is then convolved through a convolutional layer with 2048 channels, a kernel size of 1×1, and a stride of 2. The resulting image size is 8×8×2048. The convolution result is then fed into a normalization layer for data normalization. The shortcut structure is primarily used to preserve feature information to avoid loss during the convolution process and prevent network degradation. The normalized processing structure is then added to the normalized processing results of the convolutional structure above and then calculated using the ReLU nonlinear function before outputting the resulting image size. The image size is now 8×8×2048.

[0041] The above operation is performed three times on the output 8×8×2048 image. The stride of the convolution layer is no longer limited to 2 during the operation. After obtaining the 8×8×2048 image for the first time, the convolution layer with an input channel number of 2048, a convolution kernel size of 1×1, and a stride of 2 is not convolved. The remaining steps are the same as the above operation.

[0042] Output the final image data result to the decoder;

[0043] From the first to the fourth stage, the final feature map data (64×64×64, 64×64×256, 32×32×512, 16×16×1024) will be input into the jump connection. Assume that the feature maps at different levels are set to F i , each feature map has its own characteristics due to the different sizes of the receptive fields at each level. In this structure, X t is the one-dimensional feature vector data obtained by Squeeze calculation of the feature information of the current level, S t is the hidden layer state, C t Additional data links added for multi-level memory.

[0044] Taking the t-1 layer as an example, calculate St-1 When X is input t-1 Feature information and high-level input data S t In addition, C t-1 Data. S t and C t There is a close connection between the forget gate mechanism f1 and the input gate mechanism f2. The forget gate mechanism determines how to change the feature data state based on the feature data of the higher level and the feature data of the current level:

[0045] f1=sigmoid(W1[S t , X t-1 ]+b1);

[0046] W1 and b1 are weight parameters. The sigmoid function takes values ​​between 0 and 1. When matrix elements are multiplied, elements with a value of 0 are erased to filter out important features and ignore irrelevant data.

[0047] The input gate mechanism again changes the feature data state based on the high-level feature data and the current-level feature data:

[0048]

[0049] W2, b2, and As the weight parameter, the sigmoid function filters the feature data again, and the tanh function also takes a value between 0 and 1 to sort out and summarize all the feature information data.

[0050] After data optimization, it will become C t-2 The data continues to be passed forward, and S t-1 Data is updated:

[0051] C t-1 =f1×C t +f2;

[0052] In this structure, a short-term feature data memory chain S is maintained t and long-term feature data memory chain C t The state of mutual update ensures the interaction and optimized screening of feature data at all levels. Finally, the long short-term memory structure module outputs the one-dimensional vector data to the channel attention mechanism, first performing an excitation operation to generate new feature channel weights, and then performing a weighted operation on the image feature data through a reweight operation to obtain the filtered and optimized feature information, which is input into the decoder.

[0053] In the decoder, the image size is expanded through upsampling operations. In each stage, the feature data of the ground object input by the jump connection of the stage and the feature data after the upsampling operation of the previous stage are spliced, and two convolution operations with a convolution kernel size of 3×3 are performed before upsampling operations, so that the feature image data is gradually restored to the original image size.

[0054] The decoder first receives the output data from the fifth stage of the encoder. The image size is 8×8×2048. After upsampling, the image size is 16×16×2048. After two convolutions, the image size is 16×16×1024. This image data is then concatenated with the image data from the fourth stage of the skip connection. The image size is now 16×16×2048. After upsampling, the image size is 32×32×2048. After two convolutions, the image size is 32×32×512. This image data is then concatenated with the image data from the third stage of the skip connection. The image size is now 32×32×1024. After upsampling, the image size is 64×64×1024. After two convolutions, the image size is 64×64×512. The image data at this point is concatenated with the image data output from the second stage of the skip connection. The image size is now 64×64×768, and after two convolutions, the image size is 64×64×256. The image data at this point is concatenated with the image data output from the first stage of the skip connection. The image size is now 64×64×320, and after upsampling, the image size is 256×256×320. After two convolutions, the image size is 256×256×64. Finally, convolution is performed through a convolutional layer with a number of channels equal to the number of feature types and a kernel size of 1×1.

[0055] Step 5: Input the remote sensing feature classification dataset after data enhancement in step 3 into the neural network model constructed in step 4 for training. After training, the optimal weights are generated. Load the optimal weights, perform remote sensing feature classification verification on the validation set in step 1, and visualize the feature classification results. Output is as follows: Figure 3 The final land feature classification results are shown in the figure. It can be seen from the figure that the final classification results are quite close to the original annotation situation. The boundaries of each feature type are clear and more consistent with the actual surface conditions.

[0056] Input any remote sensing image into the trained model to generate remote sensing object classification images.

Claims

1. A remote sensing feature classification method based on deep learning, characterized by: The specific steps include: Step 1: Obtain a remote sensing object classification dataset, which includes the original images of remote sensing object classification and their corresponding original annotations; divide the dataset into a training set and a validation set. The training set is used to train the neural network model, and the validation set is used to verify the accuracy of the remote sensing object classification after the neural network model is trained; Step 2: Build a remote sensing image generation model based on the pix2pix algorithm in the conditional generative adversarial network model: The remote sensing image generation model uses the U-Net framework, which includes an encoder, a decoder, and skip connections for connecting low-level semantic information with high-level semantic information. The encoder uses the ResNet50 structure, and the skip connection part adds a channel attention mechanism, which makes the network model focus on effective feature information of objects and ignore unnecessary feature information. The decoder model includes upsampling layers and convolution layers. Step 3: Input the remote sensing object classification dataset in step 1 into the remote sensing image generation model constructed in step 2 for training. According to the actual surface conditions and annotation map of the remote sensing object classification dataset, manually draw a label map; input the manually drawn label map into the remote sensing image generation model to generate a remote sensing image, and add this part of the remote sensing image and label map to the original remote sensing object classification dataset to achieve data enhancement; Step 4: Build a neural network model for remote sensing object classification: The neural network model for remote sensing object classification adopts a U-shaped framework, including encoder, decoder and skip connection; In the encoder, it is divided into five stages. The first to fourth stages will input the final ground feature map data into the jump connection, and the fifth stage will output the final image data result to the decoder. In each stage, large convolution kernels and small convolution kernels are used to extract image features simultaneously. Using the structural reparameterization method, the large convolution kernel and the small convolution kernel are trained synchronously and the feature extraction results of the two are combined. The optimized parameters of the small convolution kernel are incorporated into the parameters of the large convolution kernel for double optimization. By adding a channel attention mechanism to the skip connection structure, the network model pays more attention to key semantic information. This is combined with the gating mechanism of the long-short-term memory structure to enable high-level semantic information to guide the channel weights of low-level semantic information, making the weights allocated by the channel attention mechanism more reasonable. In the decoder, the image size is enlarged by upsampling. At each stage, the feature data input by the jump connection of the stage and the feature data after the upsampling operation of the previous stage are spliced, and two convolution operations are performed before upsampling, so that the feature image data is gradually restored to the original image size. Step 5: Input the remote sensing object classification dataset after data enhancement in step 3 into the neural network model constructed in step 4 for training, and generate the optimal weight after training; input any remote sensing image into the trained model to generate a remote sensing object classification image.

2. The remote sensing feature classification method based on deep learning according to claim 1, characterized in that: The five stages of the encoder are specifically: In the first stage, after the 256×256×3 remote sensing image data is input into the encoder, it first enters the convolution layer with 64 channels and a convolution kernel size of 7×7 for convolution. The image size after convolution is 128×128×64. The convolution result is output to the normalization layer for data normalization. Finally, the ReLU nonlinear function is used for operation. The operation result is then input into the 3×3 maximum pooling layer. The image size becomes 64×64×64 to reduce the image size and increase the receptive field area. In the second stage, the 64×64×64 image data output by the first stage is input into a convolutional layer with 64 channels and a convolution kernel size of 1×1 for convolution. After convolution, the image size is 64×64×64. The convolution result is output to a normalization layer for data normalization, and finally the ReLU nonlinear function is used for operation. The operation results are respectively input into a convolutional layer with 64 channels and a convolution kernel size of 3×3 and a convolutional layer with 64 channels and a convolution kernel size of 9×9 for convolution. After convolution, the image sizes of the two are both 64×64×64. The convolution results of the two are respectively input into a normalization layer for data normalization, the normalized results of the two are added, and finally the ReLU nonlinear function is used for operation. The operation results are input into a convolutional layer with 256 channels and a convolution kernel size of 1×1 for convolution. After convolution, the image size is 64×64×256. The convolution result is output to a normalization layer for data normalization. The image data is then input into a convolutional layer with 256 channels and a convolution kernel size of 1×1 for convolution. The image size after convolution is 64×64×256. The convolution result is output to a normalization layer for data normalization to preserve the feature information of the ground object, so as to avoid loss during the convolution process and network degradation. The normalized processing structure is added to the normalized processing result of the above convolution structure and then output after calculation using the ReLU nonlinear function. At this time, the image size is 64×64×256. The above operation is performed twice on the output 64×64×256 image. After obtaining the 64×64×256 image for the first time, the image data is not input into the convolution layer with 256 channels and 1×1 kernel size for convolution. The rest of the steps are the same as the above operation. In the third stage, the 64×64×256 image output from the second stage is input into a convolutional layer with 128 channels, a convolution kernel size of 1×1, and a stride of 2 for convolution. The image size after convolution is 32×32×128. The convolution result is output to a normalization layer for data normalization, and finally the ReLU nonlinear function is used for operation. The operation results are respectively input into a convolutional layer with 128 channels and a convolution kernel size of 3×3 and a convolution kernel size of 128 channels and a convolution kernel size of A 9×9 convolutional layer is used for convolution. After convolution, the image size of both images is 32×32×128. The convolution results of the two images are respectively input to the normalization layer for data normalization. The two normalized results are added together, and finally the ReLU nonlinear function is used for operation. The operation result is input to a convolutional layer with a channel number of 512 and a convolution kernel size of 1×1 for convolution. After convolution, the image size is 32×32×512. The convolution result is output to the normalization layer for data normalization. The image is then input into a convolutional layer with a channel number of 512, a convolution kernel size of 1×1, and a stride of 2 for convolution. The image size after convolution is 32×32×512. The convolution result is output to a normalization layer for data normalization to preserve the feature information of the ground object to avoid loss during the convolution process and network degradation. The normalized processing structure is added to the normalized processing result of the above convolution structure and then output using the ReLU nonlinear function for operation. At this time, the image size is 32×32×512. The above operation is performed three times on the output 32×32×512 image, and the convolution layer stride is no longer limited (to 2) in the operation; After obtaining the 32×32×512 image for the first time, the convolution layer with 512 image data channels, 1×1 kernel size, and stride of 2 is not convolved. The remaining steps are the same as above. In the fourth stage, the 32×32×512 image data output from the third stage is input into a convolutional layer with 256 channels, a convolution kernel size of 1×1, and a stride of 2 for convolution. The image size after convolution is 16×16×256. The convolution result is output to a normalization layer for data normalization, and finally the ReLU nonlinear function is used for operation. The operation results are respectively input into a convolutional layer with 256 channels and a convolution kernel size of 3×3 and a convolution kernel size of 256 channels and a convolution kernel size of 9. × 9 convolution layer for convolution. After convolution, the image size of both images is 16 × 16 × 256. The convolution results of the two are respectively input to the normalization layer for data normalization. The two normalized results are added and finally the ReLU nonlinear function is used for operation. The operation result is input to the convolution layer with a channel number of 1024 and a convolution kernel size of 1 × 1 for convolution. After convolution, the image size is 16 × 16 × 1024. The convolution result is output to the normalization layer for data normalization. The image is then input into a convolutional layer with a channel number of 1024, a convolution kernel size of 1×1, and a stride of 2 for convolution. The image size after convolution is 16×16×1024. The convolution result is output to a normalization layer for data normalization to preserve the feature information of the ground object, so as to avoid loss during the convolution process and avoid network degradation. The normalized processing structure is added to the normalized processing result of the above convolution structure and then output using the ReLU nonlinear function for operation. At this time, the image size is 16×16×1024. The above operation is performed three times on the output 16×16×1024 image. The convolution layer stride is no longer limited to 2. After the first 16×16×1024 image is obtained, the convolution layer with 1024 input channels, 1×1 kernel size, and stride of 2 is not convolved. The remaining steps are the same as the above operation. In the fifth stage, the 16×16×1024 image data output from the fourth stage is input into a convolutional layer with a channel number of 512, a convolution kernel size of 1×1, and a stride of 2 for convolution. The image size after convolution is 8×8×512. The convolution result is output to a normalization layer for data normalization, and finally the ReLU nonlinear function is used for operation. The operation results are respectively input into a convolutional layer with a channel number of 512 and a convolution kernel size of 3×3 and a convolution layer with a channel number of 512 and a convolution kernel size of 1×1. The convolution layer is a 9×9 convolution layer. After convolution, the size of both images is 8×8×512. The convolution results of the two are respectively input to the normalization layer for data normalization. The two normalized results are added together, and finally the ReLU nonlinear function is used for operation. The operation result is input to the convolution layer with a channel number of 2048 and a convolution kernel size of 1×1 for convolution. The image size after convolution is 8×8×2048. The convolution result is output to the normalization layer for data normalization. The image is then input into a convolutional layer with a channel number of 2048, a convolution kernel size of 1×1, and a stride of 2 for convolution. The image size after convolution is 8×8×2048. The convolution result is output to a normalization layer for data normalization. The shortcut structure is mainly used to preserve the feature information of the ground object to avoid loss during the convolution process and to avoid network degradation. The normalized processing structure is added to the normalized processing result of the above convolution structure and then output using the ReLU nonlinear function for operation. At this time, the image size is 8×8×2048. The above operation is performed three times on the output 8×8×2048 image. The stride of the convolution layer is no longer limited to 2. After the first 8×8×2048 image is obtained, the convolution layer with 2048 input channels, 1×1 kernel size, and stride of 2 is not convolved. The remaining steps are the same as the above operation. The final image data result is output to the decoder.

Images