Remote sensing image deep learning segmentation method and system based on superpixel and watershed

Abstract

The present invention discloses a remote sensing image deep learning segmentation method and system based on superpixel and watershed, which performs superpixel segmentation on remote sensing images by using superpixel segmentation algorithm, outputs superpixel segmentation result map; performs ground object segmentation on remote sensing images by using semantic segmentation convolutional neural network, outputs ground object segmentation result map; defines the geometric center of each superpixel block as the position of seed point, combines the output superpixel segmentation map and ground object segmentation result map, calculates the proportion of different ground object category labels in superpixel blocks one by one, takes the ground object category label with the largest proportion as the seed point category of the superpixel block; uses the defined seed point as input parameter, performs ground object segmentation on remote sensing images by using watershed algorithm, and outputs ground object segmentation effect map of the same size. The present invention effectively improves the visualization effect and accuracy of ground object segmentation of remote sensing images.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular discloses a remote sensing image deep learning segmentation method and system based on superpixels and watersheds. Background Art

[0002] Remote sensing images are an important data source for quickly acquiring large-scale ground information. They are widely used in many fields, such as agricultural monitoring, urban planning, and map production and updating. Remote sensing image segmentation is an important step in the remote sensing image interpretation process. It is a pixel-level intensive classification problem, and its goal is to annotate each pixel in the remote sensing image with semantic information. With the improvement of satellite observation capabilities, accurate and rapid segmentation of wide-area and high-resolution remote sensing images is of great significance in guiding production practice.

[0003] Remote sensing image object segmentation methods are divided into traditional object segmentation methods and deep learning-based object segmentation methods. Traditional object segmentation methods rely on the design and selection of artificial features. The quality and selection of features directly affect the performance of the algorithm. Traditional object segmentation methods represented by color, texture, and vegetation index run fast but generally have low segmentation performance. In recent years, with the improvement of computer computing power and the accumulation of large-scale data, object segmentation algorithms represented by convolutional neural networks such as FCN (Fully Convolutional Networks), SegNet (Sementic Segmentation), and DeepLab have developed rapidly. This type of method uses an encoder-decoder approach. First, the input image with a higher resolution is downsampled layer by layer to extract semantic features with lower resolution. Then, these semantic features are upsampled layer by layer to output the object segmentation result map of the same size as the input image.

[0004] In the remote sensing image object segmentation algorithm based on deep learning, due to the existence of the encoder-decoder structure, sporadically distributed erroneous fragments will appear in the local details of the object segmentation results. This fragmentation phenomenon is particularly obvious in high-resolution remote sensing images, affecting the accuracy and visualization effect of remote sensing image object segmentation. Summary of the invention

[0005] The present invention provides a remote sensing image deep learning segmentation method and system based on superpixel and watershed, aiming to overcome the problem of erroneous fragmented points in the object segmentation result map output by the original remote sensing image object segmentation method based on deep learning, and adopts the superpixel block fusion method to remove the erroneous fragmented point labels in the semantic segmentation result map, thereby improving the accuracy and visualization effect of semantic segmentation.

[0006] One aspect of the present invention relates to a remote sensing image deep learning segmentation method based on superpixels and watersheds, comprising the following steps:

[0007] Use superpixel segmentation algorithm to perform superpixel segmentation on remote sensing images and output superpixel segmentation result map;

[0008] Use semantic segmentation convolutional neural network to segment objects in remote sensing images and output the object segmentation result map;

[0009] The geometric center of each superpixel block is defined as the position of the seed point. Combined with the output superpixel segmentation map and the object segmentation result map, the proportion of different object category labels in the superpixel block is statistically calculated one by one, and the object category label with the largest proportion is taken as the seed point category of the superpixel block.

[0010] The defined seed points are used as input parameters, and the watershed algorithm is used to segment the remote sensing image and output the segmentation effect map of the same size.

[0011] Furthermore, the remote sensing image is segmented into superpixels using a superpixel segmentation algorithm. In the step of outputting a superpixel segmentation result map, a simple linear iterative clustering algorithm is used to segment the remote sensing image into superpixels, thereby obtaining a series of superpixel regions with well-preserved contours and neatly and compactly arranged.

[0012] Furthermore, the superpixel segmentation algorithm involves region size parameters and regularization balance parameters. In the step of performing superpixel segmentation on the remote sensing image using the superpixel segmentation algorithm and outputting the superpixel segmentation result map, the corresponding region size parameters and regularization balance parameters are set.

[0013] Furthermore, in the step of using a semantic segmentation convolutional neural network to segment objects in remote sensing images and outputting an object segmentation result map, a single remote sensing image is input into a deep learning model, and after forward propagation calculation, an object segmentation result map of the same size and with object annotation categories is output.

[0014] Furthermore, the geometric center of each superpixel block is defined as the position of the seed point. The output superpixel segmentation map and the object segmentation result map are combined, and the proportion of different object category labels in the superpixel block is calculated one by one. The object category label with the largest proportion is taken as the seed point category of the superpixel block. The object categories in the remote sensing image are defined, and the object categories include buildings, rivers, roads and vegetation.

[0015] Another aspect of the present invention relates to a remote sensing image deep learning segmentation system based on superpixels and watersheds, comprising:

[0016] The first segmentation module is used to perform super-pixel segmentation on the remote sensing image using a super-pixel segmentation algorithm and output a super-pixel segmentation result map;

[0017] The second segmentation module is used to segment the remote sensing image using a semantic segmentation convolutional neural network and output a segmentation result map;

[0018] A statistical calculation module is used to define the geometric center of each superpixel block as the position of the seed point, combine the output superpixel segmentation map and the ground object segmentation result map, and statistically calculate the proportion of different ground object category labels in the superpixel block one by one, and take the ground object category label with the largest proportion as the seed point category of the superpixel block;

[0019] The third segmentation module is used to use the defined seed point as an input parameter, use the watershed algorithm to segment the remote sensing image, and output the segmentation effect map of the same size.

[0020] Furthermore, the first segmentation module includes a first segmentation unit, which is used to perform superpixel segmentation on the remote sensing image using a simple linear iterative clustering algorithm to obtain a series of superpixel regions with well-preserved contours and neatly and compactly arranged.

[0021] Furthermore, the first segmentation module includes a setting unit, which is used to set corresponding region size parameters and regularization balance parameters.

[0022] Furthermore, the second segmentation module includes a second segmentation unit, which is used to take a single remote sensing image as input into the deep learning model, and after forward propagation calculation, output a ground object segmentation result map of the same size and with ground object annotation categories.

[0023] Furthermore, the statistical calculation module includes a definition unit, which is used to define the types of objects in the remote sensing image, and the types of objects include buildings, rivers, roads and vegetation.

[0024] The beneficial effects achieved by the present invention are:

[0025] The present invention provides a remote sensing image deep learning segmentation method and system based on superpixel and watershed, which performs superpixel segmentation on remote sensing images by using superpixel segmentation algorithm, and outputs superpixel segmentation result map; performs ground object segmentation on remote sensing images by using semantic segmentation convolutional neural network, and outputs ground object segmentation result map; defines the geometric center of each superpixel block as the position of seed point, combines the output superpixel segmentation map and ground object segmentation result map, and calculates the proportion of different ground object category labels in the superpixel block one by one, and takes the ground object category label with the largest proportion as the seed point category of the superpixel block; uses the defined seed point as input parameter, performs ground object segmentation on remote sensing images by using watershed algorithm, and outputs ground object segmentation effect map of the same size. The remote sensing image deep learning segmentation method and system based on superpixel and watershed provided by the present invention can effectively eliminate the broken points of wrong segmentation in the ground object segmentation effect map output by the deep learning model, improve the visualization effect of ground object segmentation of remote sensing images, and effectively improve the accuracy of ground object segmentation of remote sensing images. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 A schematic diagram of a flow chart of an embodiment of a remote sensing image deep learning segmentation method based on superpixels and watersheds provided by the present invention;

[0027] Figure 2 A diagrammatic illustration of an embodiment of a remote sensing image deep learning segmentation method based on superpixels and watersheds provided by the present invention;

[0028] Figure 3 A ground object segmentation result map based on the output of the deep learning model in the remote sensing image deep learning segmentation method based on superpixels and watersheds provided by the present invention;

[0029] Figure 4 The effect diagram of ground object segmentation by fusion of superpixel blocks in the remote sensing image deep learning segmentation method based on superpixel and watershed provided by the present invention;

[0030] Figure 5 This is a functional block diagram of an embodiment of a remote sensing image deep learning segmentation system based on superpixels and watersheds provided by the present invention.

[0031] Description of Figure Numbers:

[0032] 10. First segmentation module; 20. Second segmentation module; 30. Statistics calculation module; 40. Third segmentation module. DETAILED DESCRIPTION

[0033] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0034] like Figures 1 to 4As shown, the first embodiment of the present invention proposes a remote sensing image deep learning segmentation method based on superpixels and watersheds, comprising the following steps:

[0035] Step S100: Use a superpixel segmentation algorithm to perform superpixel segmentation on the remote sensing image and output a superpixel segmentation result map.

[0036] Input optical remote sensing image data, use SLIC (simple linear iterative clustering) algorithm to perform superpixel segmentation on remote sensing images, and obtain a series of superpixel regions with good contour preservation and neat and compact arrangement. Here, the superpixel algorithm involves two parameters, region size parameter (regionsize) and regularization balance parameter (regularization). In the superpixel segmentation of remote sensing images, set the corresponding region size parameter and regularization balance parameter.

[0037] Step S200: Use a semantic segmentation convolutional neural network to segment the remote sensing image and output a segmentation result map.

[0038] A single remote sensing image is used as input to the deep learning model. After forward propagation calculation, the segmentation result map of the same size and with the object classification is output. The semantic segmentation convolutional neural network can choose any method such as UNet, SegNet, DeepLabv3, etc.

[0039] Step S300, define the geometric center of each superpixel block as the position of the seed point, combine the output superpixel segmentation map and the object segmentation result map, calculate the proportion of different object category labels in the superpixel block one by one, and take the object category label with the largest proportion as the seed point category of the superpixel block.

[0040] Define the object categories in remote sensing images, including buildings, rivers, roads, and vegetation. Statistically calculate the proportion of multi-class label pixels in each superpixel block, use the geometric center of the superpixel block as the seed point position, and use the category with the largest proportion in the superpixel block as the seed point category label.

[0041] Step S400: using the defined seed point as an input parameter, using the watershed algorithm to segment the remote sensing image, and outputting a segmentation effect map of the same size.

[0042] The watershed algorithm is used to segment the objects in the remote sensing image. The defined seed points are used as input parameters, and the watershed algorithm is used to segment the objects in the remote sensing image, and the object segmentation effect map of the same size is output.

[0043] The remote sensing image deep learning segmentation method based on superpixel and watershed provided by the present invention, compared with the prior art, performs superpixel segmentation on the remote sensing image by using the superpixel segmentation algorithm, and outputs the superpixel segmentation result map; uses the semantic segmentation class convolutional neural network to perform ground object segmentation on the remote sensing image, and outputs the ground object segmentation result map; defines the geometric center of each superpixel block as the position of the seed point, combines the output superpixel segmentation map and the ground object segmentation result map, and statistically calculates the proportion of different ground object category labels in the superpixel block one by one, and takes the ground object category label with the largest proportion as the seed point category of the superpixel block; uses the defined seed point as an input parameter, and uses the watershed algorithm to perform ground object segmentation on the remote sensing image, and outputs the ground object segmentation effect map of the same size. The remote sensing image deep learning segmentation method based on superpixel and watershed provided by the present invention can effectively eliminate the broken points of wrong segmentation in the ground object segmentation effect map output by the deep learning model, improve the visualization effect of ground object segmentation of the remote sensing image, and effectively improve the accuracy of ground object segmentation of the remote sensing image.

[0044] Please see Figure 5 , Figure 5The present invention provides a functional block diagram of an embodiment of a remote sensing image deep learning segmentation system based on superpixels and watersheds. In this embodiment, the remote sensing image deep learning segmentation system based on superpixels and watersheds includes a first segmentation module 10, a second segmentation module 20, a statistical calculation module 30 and a third segmentation module 40, wherein the first segmentation module 10 is used to perform superpixel segmentation on the remote sensing image using a superpixel segmentation algorithm and output a superpixel segmentation result map; the second segmentation module 20 is used to perform ground object segmentation on the remote sensing image using a semantic segmentation convolutional neural network and output a ground object segmentation result map; the statistical calculation module 30 is used to define the geometric center of each superpixel block as the position of the seed point, combine the output superpixel segmentation map and the ground object segmentation result map, calculate the proportion of different ground object category labels in the superpixel block one by one, and take the ground object category label with the largest proportion as the seed point category of the superpixel block; the third segmentation module 40 is used to use the defined seed point as an input parameter, perform ground object segmentation on the remote sensing image using a watershed algorithm, and output a ground object segmentation effect map of the same size. Among them, the first segmentation module 10 includes a first segmentation unit, which is used to use a simple linear iterative clustering algorithm to perform superpixel segmentation on the remote sensing image to obtain a series of superpixel regions with good contours and neat and compact arrangement. The first segmentation module 10 includes a setting unit, which is used to set the corresponding region size parameters and regularization balance parameters. The second segmentation module 20 includes a second segmentation unit, which is used to input a single remote sensing image into the deep learning model, and after forward propagation calculation, output a ground object segmentation result map of the same size and with ground object annotation categories. The statistical calculation module 30 includes a definition unit, which is used to define the ground object categories in the remote sensing image, and the ground object categories include buildings, rivers, roads and vegetation.

[0045] The remote sensing image deep learning segmentation system based on superpixel and watershed provided by the present invention, compared with the prior art, adopts a first segmentation module 10, a second segmentation module 20, a statistical calculation module 30 and a third segmentation module 40, performs superpixel segmentation on the remote sensing image by using a superpixel segmentation algorithm, and outputs a superpixel segmentation result map; uses a semantic segmentation convolutional neural network to perform ground object segmentation on the remote sensing image, and outputs a ground object segmentation result map; defines the geometric center of each superpixel block as the position of the seed point, combines the output superpixel segmentation map and the ground object segmentation result map, and statistically calculates the proportion of different ground object category labels in the superpixel block one by one, and takes the ground object category label with the largest proportion as the seed point category of the superpixel block; uses the defined seed point as an input parameter, uses the watershed algorithm to perform ground object segmentation on the remote sensing image, and outputs a ground object segmentation effect map of the same size. The remote sensing image deep learning segmentation system based on the superpixel guided watershed algorithm provided by the present invention can effectively eliminate the erroneously segmented fragmented points in the object segmentation rendering output by the deep learning model, improve the visualization effect of the remote sensing image object segmentation, and effectively improve the accuracy of the remote sensing image object segmentation.

[0046] The remote sensing image deep learning segmentation method and system based on superpixel and watershed provided in this embodiment are described below with specific embodiments:

[0047] Example 1: Remote sensing image segmentation based on superpixel-guided watershed algorithm

[0048] The optical remote sensing images of urban areas taken by GF-2 were used as the data source, and a piece of the area was intercepted as the test instance data. The image data size was 512×512 and the image resolution was 0.8 meters.

[0049] Step 1: Use the SLIC algorithm to perform superpixel segmentation on the remote sensing image. The superpixel region size parameter regionsize is set to 30 pixels, and the regularization balance parameter regularization is set to 0.001, to obtain a series of superpixel regions with well-preserved contours and neat and compact arrangement.

[0050] Step 2: Input the remote sensing image into the Deeplabv3 convolutional neural network, and after forward propagation calculation, output the object segmentation result map with a size of 512×512 with object annotation categories. The object categories here include seven categories: buildings, vegetation, road networks, water bodies, bare land, industrial plants and stadiums.

[0051] Step 3: Define the location and category of the seed point. Define the geometric center of each superpixel block as the location of the seed point. Combine the superpixel segmentation map output in the first step and the object segmentation result map output in the second step, and calculate the proportion of different object category labels in the superpixel block one by one. Take the object category label with the largest proportion as the seed point category of the superpixel block.

[0052] Step 4: Use the watershed algorithm to segment the remote sensing image. Use the seed points defined in the third step as input parameters, use the watershed algorithm to segment the remote sensing image, and output the segmentation effect map of the same size.

[0053] Example 2: Remote sensing image segmentation based on superpixel-guided watershed algorithm

[0054] Optical remote sensing images of rural areas taken by Gaofen-2 were used as the data source, and a piece of the area was intercepted as the test instance data. The image data size was 512×512 and the image resolution was 0.8 meters.

[0055] Step 1: Use the SLIC algorithm to perform superpixel segmentation on the remote sensing image. The superpixel region size parameter regionsize is set to 50 pixels, and the regularization balance parameter regularization is set to 0.005, to obtain a series of superpixel regions with well-preserved contours and neat and compact arrangement.

[0056] Step 2: Input the remote sensing image into the Deeplabv3 convolutional neural network, and after forward propagation calculation, output the object segmentation result map with a size of 512×512 with object annotation categories. The object categories here include ten categories: forest land, vegetation, road network, buildings, water bodies, bare land, crops, cultivated land, greenhouses and beaches.

[0057] Step 3: Define the location and category of the seed point. Define the geometric center of each superpixel block as the location of the seed point. Combine the superpixel segmentation map output in the first step and the object segmentation result map output in the second step, and calculate the proportion of different object category labels in the superpixel block one by one. Take the object category label with the largest proportion as the seed point category of the superpixel block.

[0058] Step 4: Use the watershed algorithm to segment the remote sensing image. Use the seed points defined in the third step as input parameters, use the watershed algorithm to segment the remote sensing image, and output the segmentation effect map of the same size.

[0059] Example 3: Remote sensing image segmentation based on superpixel-guided watershed algorithm

[0060] The optical remote sensing images of the port area taken by Gaofen-2 were used as the data source, and a piece of the area was intercepted as the test instance data. The image data size was 512×512 and the image resolution was 0.8 meters.

[0061] Step 1: Use the SLIC algorithm to perform superpixel segmentation on the remote sensing image. The superpixel region size parameter regionsize is set to 30 pixels, and the regularization balance parameter regularization is set to 0.002 to obtain a series of superpixel regions with well-preserved contours and neat and compact arrangement.

[0062] Step 2: Input the remote sensing image into the Deeplabv3 convolutional neural network, and after forward propagation calculation, output the segmentation result images with a size of 512×512 with the labeled categories of the ground objects. The ground objects here include eight categories: breakwater, grocery yard, water body, industrial plant, car yard, oil tank, dock and container.

[0063] Step 3: Define the location and category of the seed point. Define the geometric center of each superpixel block as the location of the seed point. Combine the superpixel segmentation map output in the first step and the object segmentation result map output in the second step, and calculate the proportion of different object category labels in the superpixel block one by one. Take the object category label with the largest proportion as the seed point category of the superpixel block.

[0064] Step 4: Use the watershed algorithm to segment the remote sensing image. Use the seed points defined in the third step as input parameters, use the watershed algorithm to segment the remote sensing image, and output the segmentation effect map of the same size.

[0065] The present embodiment provides a remote sensing image deep learning segmentation method and system based on superpixel and watershed, such as Figure 3 and Figure 4 As shown, Figure 3 and Figure 4 This is a comparison chart of the object segmentation effects of remote sensing images based on superpixel block fusion. High-resolution optical remote sensing images of urban areas taken by GF-2 are used as test data. The object categories include seven categories: buildings, vegetation, road networks, water bodies, bare land, industrial plants and stadiums. Figure 3 This is the result map of the object segmentation output based on the deep learning object segmentation model. Figure 4 This is the ground object segmentation result map output by the remote sensing image deep learning segmentation method based on superpixels and watersheds. From the segmentation result map, it can be seen that the remote sensing image deep learning segmentation method based on superpixels and watersheds can effectively remove the fragments in the ground object segmentation result map, and has a better effect on the contour extraction of ground objects such as buildings and roads.

[0066] Although preferred embodiments of the present invention have been described, additional changes and modifications may be made to these embodiments by those skilled in the art once the basic inventive concepts are known. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention. Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A remote sensing image deep learning segmentation method based on superpixel and watershed, characterized in that: The following steps are involved: Use superpixel segmentation algorithm to perform superpixel segmentation on remote sensing images and output superpixel segmentation result map; Use semantic segmentation convolutional neural network to segment objects in remote sensing images and output the object segmentation result map; The geometric center of each superpixel block is defined as the position of the seed point. Combined with the output superpixel segmentation map and the object segmentation result map, the proportion of different object category labels in the superpixel block is statistically calculated one by one, and the object category label with the largest proportion is taken as the seed point category of the superpixel block. The defined seed points are used as input parameters, and the watershed algorithm is used to segment the remote sensing image, and the segmentation effect map of the same size is output; In the step of performing superpixel segmentation on the remote sensing image using the superpixel segmentation algorithm and outputting the superpixel segmentation result map, optical remote sensing image data is input, and the remote sensing image is segmented into superpixels using the SLIC algorithm to obtain a series of superpixel regions with good contours and neat and compact arrangement. The superpixel algorithm involves two parameters, a region size parameter and a regularization balance parameter; in the superpixel segmentation of the remote sensing image, the corresponding region size parameter and regularization balance parameter are set; In the step of using a semantic segmentation convolutional neural network to segment remote sensing images and outputting a segmentation result map, a single remote sensing image is input into a deep learning model, and after forward propagation calculation, a segmentation result map of the same size and with object annotation categories is output.

2. The remote sensing image deep learning segmentation method based on superpixel and watershed as claimed in claim 1, characterized in that: The geometric center of each superpixel block is defined as the position of the seed point, and the output superpixel segmentation map and the ground object segmentation result map are combined to statistically calculate the proportion of different ground object category labels in the superpixel block one by one, and the ground object category label with the largest proportion is taken as the seed point category of the superpixel block. In the step, the ground object categories in the remote sensing image are defined, and the ground object categories include buildings, rivers, roads and vegetation.

3. A remote sensing image deep learning segmentation system based on superpixels and watershed, characterized in that: include: A first segmentation module (10) is used to perform superpixel segmentation on the remote sensing image using a superpixel segmentation algorithm and output a superpixel segmentation result map; A second segmentation module (20) is used to perform ground object segmentation on the remote sensing image using a semantic segmentation convolutional neural network and output a ground object segmentation result map; A statistical calculation module (30) is used to define the geometric center of each superpixel block as the position of the seed point, combine the output superpixel segmentation map and the ground object segmentation result map, statistically calculate the proportion of different ground object category labels in the superpixel block one by one, and take the ground object category label with the largest proportion as the seed point category of the superpixel block; A third segmentation module (40) is used to use the defined seed point as an input parameter, perform ground object segmentation on the remote sensing image using a watershed algorithm, and output a ground object segmentation effect image of the same size; The first segmentation module (10) comprises a first segmentation unit, which is used to input optical remote sensing image data and use the SLIC algorithm to perform superpixel segmentation on the remote sensing image to obtain a series of superpixel regions with well-maintained contours and neatly and compactly arranged. The superpixel algorithm involves two parameters, a region size parameter and a regularization balance parameter. The first segmentation module (10) comprises a setting unit, wherein the setting unit is used to set corresponding region size parameters and regularization balance parameters in superpixel segmentation of remote sensing images; The second segmentation module (20) comprises a second segmentation unit, which is used to take a single remote sensing image as input to a deep learning model, and output a ground object segmentation result image of the same size and with ground object annotation categories after forward propagation calculation.

4. The remote sensing image deep learning segmentation system based on superpixel and watershed as claimed in claim 3, characterized in that: The statistical calculation module (30) comprises a definition unit, wherein the definition unit is used to define the types of ground objects in the remote sensing image, and the types of ground objects include buildings, rivers, roads and vegetation.

Images