Remote sensing image segmentation method and improved DeepLabv3+ neural network

Abstract

The application relates to the technical field of image processing, in particular to a remote sensing image segmentation method and an improved DeepLabv3+ neural network. The remote sensing image segmentation method comprises the following steps: acquiring a remote sensing image; performing first data preprocessing on the remote sensing image; performing semantic segmentation on the remote sensing image after the first data preprocessing by using the trained improved DeepLabv3+ neural network to obtain a target semantic segmentation graph of the remote sensing image; wherein the improved DeepLabv3+ neural network comprises a sequentially connected encoder subnetwork and a decoder subnetwork; the component architecture of the encoder subnetwork comprises a first convolutional layer, a first batch normalization layer, four stage modules and a first ReLU activation function which are sequentially connected and have the same component architecture as corresponding modules in a RegNetY-040 model; the component architecture of the decoder subnetwork is the same as that of the decoder of an original DeepLabv3+ neural network; and the output of the decoder subnetwork is the output of the improved DeepLabv3+ neural network. The application can improve the extraction effect of the improved DeepLabV3+ neural network on the remote sensing image.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a remote sensing image segmentation method and an improved DeepLabv3+ neural network. BACKGROUND

[0002] Segmenting regions of interest in remote sensing images based on convolutional neural networks has been widely used. Common segmentation models include U-Net, DeepLab, etc. DeepLabV3+ is a deep learning model used for image segmentation. Image segmentation refers to dividing an image into different regions and assigning a label to each region. This technology can be used to automatically extract the contours of objects in an image or separate different parts of an image.

[0003] DeepLabv3+ is a semantic segmentation network that adds a Decoder module to DeepLabv3 to refine the segmentation results, especially the segmentation results of object boundaries. The original encoder of DeepLabv3+ is ResNet or Xception.

[0004] The encoder of DeepLabV3+ uses an ASPP (Atrous Spatial Pyramid Pooling) module to extract global context information. In this way, even if some parts of the image are missing or have a lot of noise, DeepLabV3+ can still accurately segment the objects in the image.

[0005] However, the encoder of the standard DeepLabV3+ model requires a large amount of computing power due to the use of ResNet or Xception models, and the extraction effect of remote sensing images is not ideal. SUMMARY

[0006] Therefore, to solve the above technical problems, the present application provides a remote sensing image segmentation method and an improved DeepLabv3+ neural network.

[0007] To solve the above technical problems, the present application adopts a technical solution of providing a remote sensing image segmentation method, which comprises:

[0008] obtaining a remote sensing image;

[0009] performing first data preprocessing on the remote sensing image;

[0010] performing semantic segmentation on the remote sensing image after first data preprocessing using the trained improved DeepLabv3+ neural network to obtain a target semantic segmentation map of the remote sensing image;

[0011] The improved DeepLabv3+ neural network comprises a sequentially connected encoder subnetwork and a decoder subnetwork, the component architecture of the encoder subnetwork comprises a first convolutional layer, a first batch normalization layer, four stage modules and a first ReLU activation function which are sequentially connected and have the same component architecture as the corresponding modules in the RegNetY-040 model; the component architecture of the decoder subnetwork is the same as that of the decoder of the original DeepLabv3+ neural network; and the output of the decoder subnetwork is the output of the improved DeepLabv3+ neural network.

[0012] To solve the above technical problems, one technical solution adopted by the present application is to provide an improved DeepLabv3+ neural network, which comprises a sequentially connected encoder subnetwork and a decoder subnetwork, the component architecture of the encoder subnetwork comprises a first convolutional layer, a first batch normalization layer, four stage modules and a first ReLU activation function which are sequentially connected and have the same component architecture as the corresponding modules in the RegNetY-040 model; the component architecture of the decoder subnetwork is the same as that of the decoder of the original DeepLabv3+ neural network; and the output of the decoder subnetwork is the output of the improved DeepLabv3+ neural network.

[0013] Advantages: Unlike the prior art, in the present application, the component architecture of the encoder subnetwork comprises a first convolutional layer, a first batch normalization layer, four stage modules and a first ReLU activation function which are sequentially connected and have the same component architecture as the corresponding modules in the RegNetY-040 model, that is, the encoder subnetwork is designed as a lightweight model like the RegNetY model, which can efficiently run on resource-constrained devices. And the encoder subnetwork like RegNetY adopts the structure of "residual module", which can improve the accuracy of the model through residual connection. Unlike other deep neural network architectures, the encoder subnetwork in the present application, like the RegNetY model, its design parameters (such as the depth and width of each residual module) are data-driven, which allows it to be fine-tuned according to the needs of a specific task. In addition, the encoder subnetwork in the present application, like RegNetY, also adopts the "adaptive span" technology, which allows the model to better handle features of different scales. Therefore, compared with the Xception model, the encoder subnetwork of the present application can perform excellently in various computer vision tasks and achieve high accuracy with a smaller model size. Thus, the improved DeepLabV3+ neural network can improve the extraction effect of remote sensing images. BRIEF DESCRIPTION OF DRAWINGS

[0014] In order to make the technical solutions in the embodiments of the present application or the prior art clearer, the accompanying drawings needed in the embodiments or prior art description will be briefly introduced. Obviously, the accompanying drawings in the following description are only some embodiments of the present application, and other accompanying drawings can be obtained by those skilled in the art without any creative effort on the basis of these accompanying drawings.

[0015] Figure 1 is a test flow diagram of the improved DeepLabv3+ neural network in the remote sensing image segmentation method of the present application;

[0016] Figure 2 is a training flow diagram of the improved DeepLabv3+ neural network in the remote sensing image segmentation method of the present application;

[0017] Figure 3 is a structure diagram of the improved DeepLabv3+ neural network in the remote sensing image segmentation method of the present application;

[0018] Figure 4 is a connection structure diagram of four stage modules, a second convolutional layer and an ASPP module in the encoder subnetwork of the improved DeepLabv3+ neural network in the remote sensing image segmentation method of the present application;

[0019] Figure 5 is a simple structure diagram of the first block module of any one of the four stage modules in the encoder subnetwork of the improved DeepLabv3+ neural network in the remote sensing image segmentation method of the present application;

[0020] Figure 6 is a simple structure diagram of the remaining block modules of any one of the four stage modules in the encoder subnetwork of the improved DeepLabv3+ neural network in the remote sensing image segmentation method of the present application;

[0021] Figure 7 is a structure diagram of the first block module of any one of the four stage modules in the encoder subnetwork of the improved DeepLabv3+ neural network in the remote sensing image segmentation method of the present application;

[0022] Figure 8 is a structure diagram of the remaining block modules of any one of the four stage modules in the encoder subnetwork of the improved DeepLabv3+ neural network in the remote sensing image segmentation method of the present application.

[0023] Explanation of reference signs:

[0024] 10, improved DeepLabv3+ neural network; 100, encoder subnetwork; 200, decoder subnetwork; 300, second convolutional layer; 400, second batch normalization layer; 500, second ReLU activation function;

[0025] 110, first convolutional layer; 120, first batch normalization layer; 130, four stage modules; 140, first ReLU activation function;

[0026] 131, first stage module; 132, second stage module; 133, third stage module; 134, fourth stage module;

[0027] 20, block module one; 21, first main branch; 21-1, third convolutional layer; 21-2, first set of convolutional layers; 21-3, fourth convolutional layer; 22, first shortcut branch; 22-1, fifth convolutional layer; 22-2, sixth batch normalization layer; 23, third ReLU activation function; 21-4, third batch normalization layer; 21-5, fourth ReLU activation function; 21-6, fourth batch normalization layer; 21-7, fifth ReLU activation function; 21-8, fifth batch normalization layer;

[0028] 24, third main branch; 24-1, eighth convolutional layer; 24-2, sixth ReLU activation function; 24-3, ninth convolutional layer; 24-4, first Sigmoid activation function; 25, third shortcut branch; 26, first add function;

[0029] 30, block module two; 31, second main branch; 31-1, sixth convolutional layer; 31-2, second set of convolutional layers; 31-3, seventh convolutional layer; 32, second shortcut branch; 33, seventh ReLU activation function; 31-4, seventh batch normalization layer; 31-5, eighth ReLU activation function; 31-6, eighth batch normalization layer; 31-7, ninth ReLU activation function; 31-8, ninth batch normalization layer;

[0030] 34, fourth main branch; 34-1, tenth convolutional layer; 34-2, tenth ReLU activation function; 34-3, eleventh convolutional layer; 34-4, second Sigmoid activation function; 35, fourth shortcut branch; 36, second add function;

[0031] 210, ASPP module; 220, first up-sampling layer; 230, first connection function; 240, first block module. DETAILED DESCRIPTION

[0032] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0033] In the description of the present application, it should be noted that the terms "center", "longitudinal", "transverse", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer" and the like indicate the orientation or positional relationship shown in the drawings, and are only for the purpose of facilitating the description of the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation of the present application. In addition, the terms "first", "second" are only for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0034] Please refer to Figure 1 The remote sensing image segmentation method of the present application comprises steps S110 to S130.

[0035] Step S110: acquiring a remote sensing image.

[0036] Step S120: performing first data preprocessing on the remote sensing image.

[0037] Specifically, the first data preprocessing on the remote sensing image can be performed by a data preprocessing module, and the first data preprocessing includes upsampling and / or data enhancement on the remote sensing image. The data enhancement includes but is not limited to one or more of morphological changes, pixel value changes and copy-paste enhancement. The morphological changes include but are not limited to at least one or more of flipping, rotating and scaling. The pixel value changes include but are not limited to at least one or more of brightness transformation and contrast enhancement. The copy-paste enhancement includes but is not limited to randomly extracting another remote sensing image and pasting the changed area in the extracted remote sensing image into the current acquired remote sensing image in the form of an object.

[0038] Step S130: performing semantic segmentation on the remote sensing image after the first data preprocessing by using a trained improved DeepLabv3+ neural network to obtain a target semantic segmentation map of the remote sensing image.

[0039] In an example, the target semantic segmentation map can be an extracted image of a road region in the remote sensing image. In another example, the target semantic segmentation map can be an extracted image of a river region in the remote sensing image. In yet another example, the target semantic segmentation map can be an extracted image of a residential area in the remote sensing image. As mentioned above, but not limited thereto, in other examples, the target semantic segmentation map can be an extracted image of other regions in the remote sensing image.

[0040] In combination with the above, Figure 1 Referring to Figure 3 The improved DeepLabv3+ neural network 10 includes an encoder subnetwork 100 and a decoder subnetwork 200 connected in sequence. The component architecture of the encoder subnetwork 100 includes a first convolutional layer 110, a first batch normalization layer 120, four stage modules 130, and a first ReLU activation function 140 connected in sequence and having the same component architecture as the corresponding modules in the RegNetY-040 model. The component architecture of the decoder subnetwork 200 is the same as that of the decoder of the original DeepLabv3+ neural network. The output of the decoder subnetwork 200 is the output of the improved DeepLabv3+ neural network 10.

[0041] In the above manner, in the present embodiment, the component architecture of the encoder subnetwork 100 includes a first convolutional layer 110, a first batch normalization layer 120, four stage modules 130, and a first ReLU activation function 140 connected in sequence and having the same component architecture as the corresponding modules in the RegNetY-040 model, i.e., the encoder subnetwork 100 is designed as a lightweight model like the RegNetY model, which can efficiently run on resource-constrained devices. And the encoder subnetwork 100 uses the structure of the "residual module" like RegNetY, which can improve the accuracy of the model through residual connection. Unlike other deep neural network architectures, the encoder subnetwork 100 in the present application, like the RegNetY model, its design parameters (such as the depth and width of each residual module) are data-driven, which enables it to be fine-tuned according to the needs of a specific task. In addition, the encoder subnetwork 100 in the present application, like RegNetY, also uses the "adaptive span" technique, which enables the model to better handle features of different scales. Therefore, compared to the Xception model, the encoder subnetwork 100 of the present application can perform excellently in a variety of computer vision tasks and achieve high accuracy with a smaller model size. Thus, the extraction effect of the improved DeepLabV3+ neural network on remote sensing images can be improved.

[0042] After step S130, step S140 of performing morphological processing on the obtained target semantic segmentation map to convert the target semantic segmentation map in bitmap type into a target semantic segmentation map in vector type can be further included.

[0043] The target semantic segmentation map in vector type refers to an electronic map depicted by straight lines and curves, and elements in the electronic map include but are not limited to points, lines, rectangles, polygons, circles and arcs, etc. In this way, the target semantic segmentation map can be more visually displayed in the vector type.

[0044] Embodiment 2

[0045] Further, on the basis of embodiment 1, the improved embodiment 2 is combined with Figure 1 Referring to Figure 3-4 The improvement lies in that the four stage modules 130 are sequentially connected and have the same composition architecture as the corresponding modules in the RegNetY-040 model, i.e., the first stage module 131, the second stage module 132, the third stage module 133 and the fourth stage module 134. The composition architecture of the decoder subnetwork 200 includes the ASPP module 210, the first upsampling layer 220, the first connection function 230 and the first block module 240, which are sequentially connected and have the same composition architecture as the corresponding modules in the RegNetY-040 model.

[0046] The composition architecture of the first stage module 131 is composed of two sequentially connected block modules. The composition architecture of the second stage module 132 is composed of six sequentially connected block modules. The composition architecture of the third stage module 133 is composed of twelve sequentially connected block modules. The composition architecture of the fourth stage module 134 is composed of two sequentially connected block modules.

[0047] The output of the last block module of the fourth stage module 134 is the input of the ASPP module 210, and the output of the last block module of the first stage module 131 sequentially passes through the second convolutional layer 300, the second batch normalization layer 400 and the second ReLU activation function 500, and then is input into the first connection function 230 together with the output of the first upsampling layer 220. The output of the first block module 240 is the output of the improved DeepLabv3+ neural network 10.

[0048] In the above manner, in the present embodiment, the ability to extract semantic information can be improved without increasing too much computational load.

[0049] Embodiment 3

[0050] On the basis of embodiment 2, the improved embodiment 3 is combined withFigure 1 、 Figure 3-4 Referring to Figure 5-6 The first convolutional layer 110 adopts a 3x3 convolution kernel and a step length of 2. The first block module of any one of the first to fourth stage modules 134 is a block module one 20, which adopts a step length of 2 and includes a first main branch 21, a first shortcut branch 22, and a first add function 26. The input ends of the first main branch 21 and the first shortcut branch 22 are connected with the input end of the block module one 20. The output of the first main branch 21 and the output of the first shortcut branch 22 are the inputs of the first add function 26, and the output of the first add function 26 is the output of the block module one 20. The first main branch 21 includes a third convolutional layer 21-1, a first group of convolutional layers 21-2, and a fourth convolutional layer 21-3 connected in sequence. The first shortcut branch 22 includes a fifth convolutional layer 22-1. The third convolutional layer 21-1 and the fourth convolutional layer 21-3 both adopt a 1x1 convolution kernel and a step length of 1, the first group of convolutional layers 21-2 adopts a 3x3 convolution kernel and a step length of 2, and the fifth convolutional layer 22-1 adopts a 1x1 convolution kernel and a step length of 2.

[0051] Generally, the shortcut branch processes the input feature matrix differently according to the step length of the block module to which the shortcut branch belongs. Since the step length of the block module one 20 to which the first shortcut branch 22 belongs is 2, the first shortcut branch 22 generally performs down-sampling on the input feature matrix through the fifth convolutional layer 22-1 in the first shortcut branch 22, so as to reduce the resolution of the feature matrix, and make the resolution of the output feature matrix half of that of the input feature matrix.

[0052] In addition, since the step length of the block module one 20 to which the first shortcut branch 22 belongs is 2, the number of channels of the input feature matrix of the block module one 20 to which the first shortcut branch 22 belongs, the number of channels of the output feature matrix of the block module one 20 to which the first shortcut branch 22 belongs, and the bottleneck ratio of the block module one 20 to which the first shortcut branch 22 belongs satisfy the relationship (1), where the relationship (1) is:

[0053] ·······················(1)

[0054] wherein, is the number of channels of the output feature matrix of the block module one 20 to which the first shortcut branch 22 belongs, is the number of channels of the input feature matrix of the block module one 20 to which the first shortcut branch 22 belongs, The bottleneck ratio of the block module 20 to which the first shortcut branch 22 belongs.

[0055] In the above manner, in the present embodiment, since the step size of the block module 20 is 2, the spatial size of the input feature matrix is halved, thereby reducing the amount of calculation. Since the third convolutional layer 21-1 and the fourth convolutional layer 21-3 both use a 1x1 convolutional kernel and have a step size of 1, the number of channels can be changed without changing the spatial size of the input feature matrix, thereby increasing the nonlinearity of the model. Since the first group of convolutional layers 21-2 use a 3x3 convolutional kernel and have a step size of 2, local features can be extracted while reducing the spatial size of the feature matrix. Since the fifth convolutional layer 22-1 uses a 1x1 convolutional kernel and has a step size of 2, the number of channels can be changed while reducing the spatial size of the feature matrix. The above features in Embodiment 3 cooperate with each other to improve the extraction effect of the remote sensing image.

[0056] Further, the remaining block modules of any one of the first stage module 131 to the fourth stage module 134 are block modules two 30, which have a step size of 1 and include a second main branch 31, a second shortcut branch 32, and a second add function 36. The input ends of the second main branch 31 and the second shortcut branch 32 are both connected to the input end of the block module two 30. The output of the second main branch 31 and the output of the second shortcut branch 32 are inputs of the second add function 36, and the output of the second add function 36 is the output of the block module two 30 to which the second add function 36 belongs. The second main branch 31 includes a sixth convolutional layer 31-1, a second group of convolutional layers 31-2, and a seventh convolutional layer 31-3 connected in sequence. The second shortcut branch 32 does not include a convolutional layer. Among them, the sixth convolutional layer 31-1 and the seventh convolutional layer 31-3 both use a 1x1 convolutional kernel and have a step size of 1, and the second group of convolutional layers 31-2 uses a 3x3 convolutional kernel and has a step size of 1.

[0057] Generally, the shortcut branch processes the input data differently according to the step size of the block module to which the shortcut branch belongs. Since the step size of the block module two 30 to which the second shortcut branch 32 belongs is 1, the second shortcut branch 32 generally does not process the input data, but directly outputs, so that the resolution of the input data is the same as the resolution of the output data.

[0058] In addition, since the step size of the block module two 30 to which the second shortcut branch 32 belongs is 1, the number of channels of the input feature matrix of the block module two 30 to which the second shortcut branch 32 belongs and the number of channels of the output feature matrix of the block module two 30 to which the second shortcut branch 32 belongs satisfy the relationship (2), where the relationship (2) is:

[0059] (2)

[0060] wherein, is the number of channels of the feature matrix of the output of the block module two 30 to which the second shortcut branch 32 belongs, is the number of channels of the feature matrix of the input of the block module two 30 to which the second shortcut branch 32 belongs.

[0061] In the above manner, in the present embodiment, since the step size of the block module two 30 is 1, the spatial size of the feature matrix can be kept unchanged. Since the sixth convolutional layer 31-1 and the seventh convolutional layer 31-3 both use a 1x1 convolutional kernel and have a step size of 1, the number of channels can be changed without changing the spatial size of the feature matrix, thereby increasing the nonlinearity of the model. Since the second group of convolutional layers 31-2 use a 3x3 convolutional kernel and have a step size of 1, local features can be extracted while keeping the spatial size of the feature matrix unchanged.

[0062] Embodiment 4

[0063] On the basis of Embodiment 3, the improved DeepLabv3+ neural network 10 is combined with Figure 1 , Figure 3-6 Referring to Figure 7-8 , the improvement point is that the third convolutional layer 21-1, the first group of convolutional layers 21-2, the fourth convolutional layer 21-3, the sixth convolutional layer 31-1, the second group of convolutional layers 31-2, and the seventh convolutional layer 31-3 are all connected with a batch normalization layer and a ReLU activation function. In this way, the performance of the improved DeepLabv3+ neural network 10 can be improved.

[0064] Embodiment 5

[0065] On the basis of any one of Embodiments 1-4, the improved DeepLabv3+ neural network 10 is combined with Figure 1 Referring to Figure 2 , the improvement point is that steps S101-S104 are included before step S130.

[0066] Step S101: Establishing an improved DeepLabv3+ neural network.

[0067] The specific component architecture of the improved DeepLabv3+ neural network 10 can be referred to the related description in the foregoing embodiments, which will not be described here again.

[0068] Step S102: Obtaining a training set of remote sensing images and a corresponding label set; the remote sensing images in the training set have corresponding manually labeled true value semantic segmentation maps in the label set.

[0069] Specifically, the so-called artificially annotated ground truth semantic segmentation map can be a segmentation map of a target region obtained by manual annotation by professional personnel. Manual annotation includes but is not limited to a segmentation map of a target region drawn by a professional image processing software according to a drawing operation of manual input.

[0070] Step S103: performing second data preprocessing on the remote sensing images in the training set.

[0071] The second data preprocessing includes up-sampling and data enhancement on the remote sensing images. The second data preprocessing can adopt the same steps as the first data preprocessing, and thus will not be described herein.

[0072] Step S104: inputting the remote sensing images after the second data preprocessing into the improved DeepLabv3+ neural network to train the improved DeepLabv3+ neural network.

[0073] Further, the step S104 includes steps S11 to S12.

[0074] Step S11: performing semantic segmentation on the remote sensing images after the second data preprocessing by using the improved DeepLabv3+ neural network to obtain a predicted semantic segmentation map of the remote sensing images.

[0075] Step S12: calculating a loss between the predicted semantic segmentation map of the remote sensing images and a corresponding artificially annotated ground truth semantic segmentation map by using a loss function, and optimizing parameters of the neural network by using a back propagation algorithm.

[0076] Optionally, the loss function can adopt a DICE loss, and the DIOE loss satisfies a relationship formula (3), and the relationship formula (3) is as follows:

[0077] ·······················(3)

[0078] Wherein, the DICE loss is, the predicted semantic segmentation map is, the ground truth semantic segmentation map is, a total number of pixels in an intersection part of the predicted semantic segmentation map and the ground truth semantic segmentation map is, a total number of pixels in the predicted semantic segmentation map is, a total number of pixels in the ground truth semantic segmentation map is.

[0079] The reason for adopting the DICE loss is that there can be a class imbalance problem in the training data, and a target pixel ratio is small, which leads to a learning tendency of a model to a background class, and the DICE loss can alleviate this situation.

[0080] Optionally, considering that only using DICE loss may lead to unstable training process, a weighted cross-entropy loss can be introduced into the loss function, which satisfies relation (4), relation (4) is:

[0081] ·······················(4)

[0082] wherein, is the weighted cross-entropy loss, is the weight, M is the number of categories, is the one-hot vector, which only has 0 and 1, and takes 1 if the category C and the category of the pixel point of the predicted semantic segmentation map are the same, otherwise takes 0, is the probability that the pixel point of the predicted semantic segmentation map belongs to category c, and the weight satisfies relation (5), relation (5) is:

[0083] ·······················(5)

[0084] wherein, is the total number of pixels of the predicted semantic segmentation map, is the total number of pixel points belonging to category C in the ground truth semantic segmentation map.

[0085] Therefore, preferably, the loss function is the joint loss of the weighted cross-entropy loss and the DICE loss, which satisfies relation (6), relation (6) is:

[0086] ·······················(6)

[0087] wherein, is the joint loss, is the DICE loss, is the weighted cross-entropy loss.

[0088] Optionally, the step of optimizing the parameters of the neural network by the back propagation algorithm comprises: using AdamW optimizer to optimize the parameters of the neural network and using preheated cosine annealing to adjust the learning rate. In this way, the stability of the initial model training can be improved and the model can be prevented from falling into a local minimum point during the entire training process.

[0089] Embodiment 6

[0090] On the basis of any one of embodiments 1-5, it is improved that between step S110 and step S120, further comprising step S111a, and step S120 further comprises step S120a, and step S130 further comprises step S131a to step S132a. Details as follows.

[0091] Step S110: Obtain a remote sensing image.

[0092] Step S111a: The obtained remote sensing image is subjected to overlapping block processing to obtain N image subblocks; the image subblock includes an intermediate area and a peripheral area, and the intermediate area is arranged in the peripheral area, and the peripheral area between the intermediate areas of two image subblocks distributed adjacent to each other in the remote sensing image is the same.

[0093] Step S120a: The N image subblocks are respectively subjected to first data preprocessing.

[0094] Step S131a: The first data preprocessed N image subblocks are respectively subjected to semantic segmentation using the trained improved DeepLabv3+ neural network to obtain the target semantic segmentation map of the N image subblocks.

[0095] Step S132a: The intermediate areas of the N image subblocks are obtained and spliced according to the corresponding relative distribution positions of the N image subblocks in the remote sensing image to obtain the target semantic segmentation map of the remote sensing image.

[0096] In the above manner, the remote sensing image is first subjected to overlapping block processing, and then the obtained N image subblocks are respectively subjected to semantic segmentation, which can reduce the image size for single semantic segmentation. And the overlapping block processing can retain the context information of the edge pixels of the intermediate area of the image subblock, thereby improving the recognition accuracy of the edge pixels of the intermediate area of the image subblock.

[0097] Embodiment 7

[0098] The application provides an improved DeepLabv3+ neural network, which is the same as the improved DeepLabv3+ neural network 10 described above. For the parts described above, no further description is given.

[0099] Hereinafter, the improved DeepLabv3+ neural network will be described in detail with reference to the accompanying drawings. Figure 5 Referring to Figure 7 , the block module 20 will be described in more detail.

[0100] Combined with Figure 5 Referring to Figure 7The third convolutional layer 21-1 and the fifth convolutional layer 22-1 are both connected with the input of the block module one 20 through the third ReLU activation function 23. The third convolutional layer 21-1 is sequentially connected with the third batch normalization layer 21-4 and the fourth ReLU activation function 21-5 between the first group of convolutional layers 21-2. The first group of convolutional layers 21-2 is sequentially connected with the fourth batch normalization layer 21-6 and the fifth ReLU activation function 21-7. The fifth ReLU activation function 21-7 is connected with the third main branch 24 and the third shortcut branch 25 between the fourth convolutional layer 21-3, respectively. The third main branch 24 includes a mean function, the eighth convolutional layer 24-1, the sixth ReLU activation function 24-2, the ninth convolutional layer 24-3 and the first Sigmoid activation function 24-4 which are sequentially connected. The third shortcut branch 25 does not include a convolutional layer. The fourth convolutional layer 21-3 is connected with the third main branch 24 and the third shortcut branch 25 through a mul function, respectively. The fourth convolutional layer 21-3 is connected with the fifth batch normalization layer 21-8. The third shortcut branch 25 includes the sixth batch normalization layer 22-2 which is connected after the fifth convolutional layer 22-1. The output of the fifth batch normalization layer 21-8 and the output of the sixth batch normalization layer 22-2 are the input of the first add function 26. The output of the first add function 26 is the output of the block module one 20. The eighth convolutional layer 24-1 adopts a 1*1 convolutional kernel and a step of 1. The ninth convolutional layer 24-3 adopts a 1*1 convolutional kernel and a step of 1.

[0101] In this way, in the embodiment, the ability of extracting semantic information can be improved, and the calculation amount can be reduced.

[0102] Hereinafter, the block module two 30 will be described in detail. Figure 6 Referring to Figure 8 The block module two 30 will be described in detail.

[0103] Hereinafter, the block module two 30 will be described in detail. Figure 6 Referring to Figure 8The sixth convolutional layer 31-1 and the seventh convolutional layer 31-3 are connected with the input of the block module two 30 through the seventh ReLU activation function 33 respectively. The sixth convolutional layer 31-1 is sequentially connected with the second group of convolutional layers 31-2 through the seventh batch normalization layer 31-4 and the eighth ReLU activation function 31-5. The second group of convolutional layers 31-2 is sequentially connected with the eighth batch normalization layer 31-6 and the ninth ReLU activation function 31-7. The ninth ReLU activation function 31-7 is connected with the seventh convolutional layer 31-3 through the fourth main branch 34 and the fourth shortcut branch 35 respectively. The fourth main branch 34 comprises a mean function, a tenth convolutional layer 34-1, a tenth ReLU activation function 34-2, an eleventh convolutional layer 34-3 and a second Sigmoid activation function 34-4 which are sequentially connected. The fourth shortcut branch 35 does not comprise a convolutional layer. The seventh convolutional layer 31-3 is connected with the fourth main branch 34 and the fourth shortcut branch 35 through a mul function respectively. The seventh convolutional layer 31-3 is connected with the ninth batch normalization layer 31-8, and the output of the ninth batch normalization layer 31-8 and the output of the fourth shortcut branch 35 are the inputs of the second add function 36, and the output of the second add function 36 is the output of the block module two 30. The tenth convolutional layer 34-1 adopts a convolutional kernel of 1*1 and a step of 1, and the eleventh convolutional layer 34-3 adopts a convolutional kernel of 1*1 and a step of 1.

[0104] In the above manner, in the embodiment, the ability of extracting semantic information can be improved, and the calculation amount can be reduced.

[0105] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A remote sensing image segmentation method, characterized in that, The remote sensing image segmentation method includes: Acquire remote sensing images; The remote sensing image is subjected to a first data preprocessing step; The remote sensing image after the first data preprocessing is semantically segmented using a trained improved DeepLabv3+ neural network to obtain a target semantic segmentation map of the remote sensing image. The improved DeepLabv3+ neural network includes a sequentially connected encoder subnetwork and a decoder subnetwork. The encoder subnetwork consists of a first convolutional layer, a first batch of normalization layers, four stage modules, and a first ReLU activation function, all connected sequentially and identical to the corresponding modules in the RegNetY-040 model. The decoder subnetwork has the same architecture as the decoder in the original DeepLabv3+ neural network. The output of the decoder subnetwork is the output of the improved DeepLabv3+ neural network. The four stage modules are sequentially connected and have the same architecture as the corresponding modules in the RegNetY-040 model: a first stage module, a second stage module, a third stage module, and a fourth stage module; the decoder sub-network architecture includes a sequentially connected ASPP module, a first upsampling layer, a first connection function, and a first block module, which have the same architecture as the corresponding modules in the RegNetY-040 model. The first stage module consists of two sequentially connected block modules; the second stage module consists of six sequentially connected block modules; the third stage module consists of twelve sequentially connected block modules; and the fourth stage module consists of two sequentially connected block modules. Wherein, the output of the last block module of the fourth stage module is the input of the ASPP module, and the output of the last block module of the first stage module passes sequentially through the second convolutional layer, the second batch normalization layer and the second ReLU activation function, and is used as the input of the first connection function along with the output of the first upsampling layer; the output of the first block module is the output of the improved DeepLabv3+ neural network.

2. The remote sensing image segmentation method according to claim 1, characterized in that, The first convolutional layer uses a 3x3 convolutional kernel with a stride of 2; The first block module of any one of the first to fourth stage modules is block module one. Block module one has a step size of 2 and includes a first main branch, a first shortcut branch, and a first add function. The input terminals of the first main branch and the first shortcut branch are both connected to the input terminal of block module one. The output of the first main branch and the output of the first shortcut branch are the input of the first add function, and the output of the first add function is the output of the block module one to which it belongs. The first main branch includes a third convolutional layer, a first group of convolutional layers, and a fourth convolutional layer connected in sequence; the first shortcut branch includes a fifth convolutional layer; wherein the third and fourth convolutional layers both use 1×1 convolutional kernels with a stride of 1, the first group of convolutional layers uses 3×3 convolutional kernels with a stride of 2, and the fifth convolutional layer uses 1×1 convolutional kernels with a stride of 2; The remaining block modules of any one of the first to fourth stage modules are block module two. Block module two has a stride of 1 and includes a second main branch, a second shortcut branch, and a second add function. The inputs of the second main branch and the second shortcut branch are connected to the input of block module two. The outputs of the second main branch and the second shortcut branch are the inputs of the second add function, and the output of the second add function is the output of the block module two to which it belongs. The second main branch includes a sixth convolutional layer, a second group of convolutional layers, and a seventh convolutional layer connected in sequence. The second shortcut branch does not include convolutional layers. The sixth and seventh convolutional layers both use 1×1 convolutional kernels with a stride of 1, and the second group of convolutional layers uses 3×3 convolutional kernels with a stride of 1.

3. The remote sensing image segmentation method according to claim 2, characterized in that, Each of the third convolutional layer, the first group of convolutional layers, the fourth convolutional layer, the sixth convolutional layer, the second group of convolutional layers, and the seventh convolutional layer is followed by a batch normalization layer and a ReLU activation function.

4. The remote sensing image segmentation method according to claim 1, characterized in that, The step of performing semantic segmentation on the acquired remote sensing image using a trained improved DeepLabv3+ neural network to obtain a predicted semantic segmentation map includes, prior to: Establish the improved DeepLabv3+ neural network; Obtain a training set of remote sensing images and a corresponding annotation set; the remote sensing images in the training set have corresponding manually annotated ground truth semantic segmentation maps in the annotation set; Perform a second data preprocessing on the remote sensing images in the training set; The remote sensing image after the second data preprocessing is input into the improved DeepLabv3+ neural network to train the improved DeepLabv3+ neural network.

5. The remote sensing image segmentation method according to claim 4, characterized in that, The step of inputting the preprocessed remote sensing image into the improved DeepLabv3+ neural network to train the improved DeepLabv3+ neural network includes: The improved DeepLabv3+ neural network is used to perform semantic segmentation on the remote sensing image after the second data preprocessing to obtain a predicted semantic segmentation map of the remote sensing image; The loss function is used to calculate the loss between the predicted semantic segmentation map of the remote sensing image and the corresponding manually labeled ground truth semantic segmentation map, and the parameters of the neural network are optimized by the backpropagation algorithm.

6. The remote sensing image segmentation method according to claim 5, characterized in that, The loss function uses a combined loss of weighted cross-entropy loss and DICE loss.

7. The remote sensing image segmentation method according to claim 6, characterized in that, The step of optimizing the parameters of the neural network using the backpropagation algorithm includes: optimizing the parameters of the neural network using the AdamW optimizer and adjusting the learning rate using preheated cosine annealing.

8. The remote sensing image segmentation method according to claim 1, characterized in that, Between the step of acquiring the remote sensing image and the step of performing the first data preprocessing on the remote sensing image, the following steps are included: The acquired remote sensing image is subjected to overlapping block processing to obtain N image sub-blocks; each image sub-block includes a middle area and an outer area, the middle area is set within the outer area, and the outer areas of the middle areas of two adjacent image sub-blocks in the remote sensing image overlap. The step of performing the first data preprocessing on the remote sensing image includes: The first data preprocessing is performed on each of the N image sub-blocks; The step of performing semantic segmentation on the remote sensing image after the first data preprocessing using a trained improved DeepLabv3+ neural network to obtain a target semantic segmentation map includes: The N image sub-blocks after the first data preprocessing are semantically segmented sequentially using a trained improved DeepLabv3+ neural network to obtain the target semantic segmentation map of the N image sub-blocks. The middle regions of the N image sub-blocks are obtained and stitched together according to the relative distribution positions of the N image sub-blocks in the remote sensing image to obtain the target semantic segmentation map of the remote sensing image.

Images