A fast multi-scale superpixel segmentation method for synthetic aperture radar images

Abstract

A fast multi-scale superpixel segmentation method for synthetic aperture radar images includes the following steps: combining backscattering grayscale similarity, edge information similarity and homogeneity similarity to measure the final similarity of two pixels; performing a graph structure representation on the SAR image; constructing a minimum spanning tree for the SAR image; realizing fast multi-scale superpixel segmentation of the SAR image based on the minimum spanning tree, obtaining a superpixel segmentation boundary based on the segmentation result and generating a SAR image segmentation result. The present invention can instantly generate any number of superpixels with a tree structure, so that superpixel results of different levels can be segmented. The multi-scale superpixel segmentation method of the present invention can not only well maintain the image boundary, but also has a good ability to maintain local image details, and can generate segmentation results of any scale for SAR images in real time, which is helpful for obtaining information of targets of different sizes, and the execution efficiency is significantly improved.

Description

Technical Field

[0001] The present invention belongs to the field of remote sensing image interpretation, and specifically relates to a fast multi-scale super-pixel segmentation method for synthetic aperture radar images. The method constructs a minimum spanning tree of a SAR image by utilizing SAR image pixel similarity, thereby completing fast multi-scale super-pixel segmentation, thereby providing important category attribute information for SAR image content analysis and scene segmentation. Background Art

[0002] Synthetic Aperture Radar (SAR) plays a key role in remote sensing technology, providing high-resolution images of the Earth regardless of day or night or in various weather conditions. Fast segmentation of SAR images can identify different object categories, which helps in subsequent applications such as image despeckle, image classification, and target detection.

[0003] Superpixel segmentation is a popular image preprocessing method that can generate local homogeneous regions with similar attributes in an image. Using local homogeneous regions (i.e., superpixels) instead of pixels as the basic unit of image processing helps to reduce the influence of coherent speckle noise and the computational complexity of SAR image processing. So far, several superpixel generation and segmentation methods for SAR images have been proposed in the relevant literature. For example, based on simple linear iterative clustering (SLIC), superpixels are generated by defining the amplitude and position similarity of SAR images. This method has a certain robustness to coherent speckle noise. Using Mahalanobis distance instead of Euclidean distance to define pixel similarity metrics can improve the performance of SLIC algorithm on SAR images. In addition, the SAR image is first segmented into superpixels using the SLIC algorithm, and then the superpixels are merged using a density-based spatial clustering method to achieve the final segmentation. In addition to the SLIC-based algorithm, the SAR image adaptive superpixel generation method with linear feature clustering and edge constraints can generate superpixels with good boundary preservation and consider global image structure information. The edge-aware superpixel generation method can achieve superpixel segmentation of SAR images in one iteration.

[0004] Although the above methods can be used for SAR image superpixel segmentation, they cannot directly generate a series of superpixels with different details. Superpixels at a coarser level can be composed of superpixels at a finer level. This pyramid segmentation result helps to quickly extract information from SAR images. In order to achieve hierarchical or multi-scale SAR image segmentation, the number of superpixels needs to be determined according to the segmentation scale, and then the superpixel generation algorithm needs to be repeated multiple times, which is inefficient. Summary of the invention

[0005] The purpose of the present invention is to provide a fast multi-scale superpixel segmentation method for synthetic aperture radar images to address the problem of low efficiency in implementing hierarchical or multi-scale SAR image segmentation in the above-mentioned prior art. The method realizes fast multi-scale superpixel segmentation of SAR images based on a minimum spanning tree (MST), can instantly generate any number of superpixels with a tree structure, can segment superpixel results of different levels, well maintain image boundaries, and has good image local detail preservation capabilities.

[0006] In order to achieve the above object, the present invention has the following technical solutions:

[0007] A fast multi-scale superpixel segmentation method for synthetic aperture radar images comprises the following steps:

[0008] - Combine backscatter grayscale similarity, edge information similarity and homogeneity similarity to measure the final similarity of two pixels;

[0009] -Graph structure representation of SAR images;

[0010] -Construct a minimum spanning tree for SAR images;

[0011] -Accomplish fast multi-scale superpixel segmentation of SAR images based on minimum spanning tree, obtain superpixel segmentation boundaries based on segmentation results and generate SAR image segmentation results.

[0012] In a preferred embodiment of the present invention, the backscatter difference of the image is obtained based on the sample average amplitude of two 3×3 local areas of the current central pixel.

[0013] In a preferred embodiment of the present invention, the backscattering grayscale similarity between two pixels is:

[0014]

[0015] Among them, two local regions R u and R v The sample means of and L is the number of views of the SAR image.

[0016] In a preferred embodiment of the present invention, a SAR image edge detector based on a guided filter is used to extract edges from a SAR image.

[0017] In a preferred embodiment of the present invention, after obtaining the image edge, the similarity measure based on the edge information is:

[0018] σ E (u, v)=max{ESM(u), ESM(v)}

[0019] Where ESM(u) represents the edge strength map of pixel u.

[0020] In a preferred embodiment of the present invention, the homogeneity similarity is: in, and Represents the local area R u and R v The coefficient of variation of the SAR image is given by the ratio of the standard deviation to the mean.

[0021] In a preferred embodiment of the present invention, the final similarity between two pixels is the sum of the scattered grayscale similarity, the edge information similarity and the homogeneity similarity.

[0022] In a preferred embodiment of the present invention, the operation method of performing graph structure representation on the SAR image is as follows:

[0023] Assume that the SAR image G = (V, E) is a vertex set V = {v1, v2, ..., v N} and edge set E = {e1, e2, ..., e M}; each pixel of the SAR image is represented by a vertex, and every two vertices are connected based on the 8-connected neighborhood. The vertex set V is divided into K disjoint components, namely superpixels, and each component corresponds to a connected subgraph G′=(V′, E′), where V′∈V, E′∈E.

[0024] In a preferred embodiment of the present invention, the minimum spanning tree for SAR images is constructed using the Kruskal algorithm, and the specific steps are as follows:

[0025] 1) The input SAR image is regarded as a connected, weighted undirected graph, with v vertices and e edges in the graph;

[0026] 2) Obtaining edge intensity map of SAR image;

[0027] 3) Initialize all vertices as individual pixels;

[0028] 4) Initialize the minimum spanning tree to be empty. The minimum spanning tree has the same v vertices as the original graph, but no edges.

[0029] 5) Calculate the pixel similarity between all pixels of the SAR image and their neighboring pixels as the weight of the edge, and sort all e edges in ascending order according to the weight:

[0030] 6) Loop: Start from the edge with the smallest weight and traverse each edge until all the nodes in the graph are in the same connected component. If the two nodes connected by an edge are not in the same connected component, add this edge to the minimum spanning tree.

[0031] 7) Return the generated minimum spanning tree.

[0032] In a preferred embodiment of the present invention, after obtaining the multi-scale segmentation result of the SAR image, the super-pixel segmentation edge is extracted using the gradient calculation method according to the pixel segmentation label, and the mean of each super-pixel of the SAR image is calculated according to the segmentation edge to obtain the final SAR image segmentation result.

[0033] Compared with the prior art, the present invention has the following beneficial effects: according to the pixel similarity of the SAR image, a new pixel similarity is defined in combination with the backscattering grayscale similarity, edge information similarity and homogeneity similarity of the SAR image, and the pixel similarity is robust. On this basis, the minimum spanning tree of the SAR image is constructed, and finally the fast multi-scale superpixel segmentation of the SAR image is realized based on the minimum spanning tree. The method of the present invention can instantly generate any number of superpixels with a tree structure, so that superpixel results of different levels can be segmented. The multi-scale superpixel segmentation method proposed in the present invention can not only well maintain the image boundary, but also has good image local detail preservation ability, and can generate segmentation results of any scale for SAR images in real time, which is helpful for obtaining information of targets of different sizes, and the execution efficiency is significantly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a SAR image slice;

[0035] Figure 2 It is the MST diagram constructed from the SAR image;

[0036] Figure 3 This is a schematic diagram of multi-scale SAR image superpixel segmentation based on MST;

[0037] Figure 4 This is the multi-scale segmentation result of the method of the present invention for SAR images:

[0038] (a) 3000 superpixels; (b) 1500 superpixels; (c) 800 superpixels; (d) 400 superpixels;

[0039] Figure 5 This is the comparison method for SAR data segmentation results:

[0040] (a) Segmentation result of GIOESP method; (b) Segmentation result of RCBLP method; (c) Segmentation result of ALFCE method. DETAILED DESCRIPTION

[0041] The present invention is further described in detail below in conjunction with the accompanying drawings.

[0042] A fast multi-scale superpixel segmentation method for synthetic aperture radar images, the specific steps are as follows:

[0043] A. For SAR images, pixel grayscale similarity, edge intensity similarity and image inhomogeneity similarity are defined;

[0044] B. Graph structure representation of SAR images;

[0045] C. Use Kruskal algorithm to realize the minimum spanning tree of SAR images;

[0046] D. Fast multi-scale superpixel segmentation of SAR images based on minimum spanning tree;

[0047] E. Based on the segmentation results, the super-pixel segmentation boundary is obtained and the SAR image segmentation result is generated.

[0048] The specific operation method of step A is:

[0049] First, it is necessary to define a pixel similarity metric for SAR images. Due to the presence of coherent speckle noise, the traditional pixel grayscale similarity metric (such as Euclidean distance) is not applicable to SAR images. In addition, the similarity metric should also consider the image edge information and uniformity information to achieve accurate superpixel segmentation. In the segmentation method of the present invention, the sample average amplitude of two 3×3 local areas based on the current center pixel is used to obtain the backscatter difference of the image. The two local areas R u and R v The sample means of and The backscatter grayscale similarity between two pixels is:

[0050]

[0051] Where L is the number of views of the SAR image.

[0052] Edge information is crucial for defining pixel similarity metrics. When there is an edge between two pixels, they will not be able to form a minimum spanning tree. The present invention uses a SAR image edge detector based on a guided filter to extract edges from SAR images. This method can remove non-edge information including speckle noise and detail texture, while retaining edge information. After obtaining the image edge, the similarity metric based on edge information can be expressed as:

[0053] σ E (u, v)=max{ESM(u), ESM(v)}

[0054] Where ESM(u) represents the edge strength map of pixel u.

[0055] Usually, pixels in a homogeneous region have a high probability of belonging to the same minimum spanning tree. Therefore, the image homogeneity differences between them can be considered in the definition of pixel similarity. In the present invention, the homogeneity similarity is defined as:

[0056]

[0057] in, and Represents the local area R u and R v The coefficient of variation (CoV) of the SAR image can be obtained by the ratio of the standard deviation to the mean.

[0058] Finally, the present invention combines backscattering grayscale similarity, edge information similarity and homogeneity similarity to achieve the final similarity measurement of two pixels:

[0059] The specific operation method of step B is:

[0060] Assume that G = (V, E) is a vertex set V = {v1, v2, ..., v N} and edge set E = {e1, e2, ..., e M}. For SAR images, each pixel can be represented by a vertex, and each edge e k =(u,v) have weights w(e k ), that is, the difference w(u, v) between vertices u and v. In the segmentation method of the present invention, every two vertices are connected based on an 8-connected neighborhood. In order to implement superpixel segmentation for a graph G (i.e., a SAR image), the vertex set V should be divided into K disjoint components, i.e., superpixels, and each component (superpixel) corresponds to a connected subgraph G′=(V′, E′), where V′∈V, E′∈E.

[0061] The specific operation method of step C is as follows: the spanning tree is defined as a weighted, undirected and acyclic subgraph. The minimum spanning tree is actually the spanning tree with the smallest total weight of all connecting edges between vertices. For SAR image superpixel segmentation, since the pixels in the minimum spanning tree (MST) have the smallest sum of pixel similarities, the MST can represent superpixels, thereby generating local image regions with very similar properties. The Kruskal algorithm is a popular MST extraction method. Its basic idea is: assuming that the connected network is G(V, E), let the initial state of the minimum spanning tree be a non-connected graph T = (V, {}) with only n vertices and no edges, and each vertex in the graph is a connected component. Select the edge with the lowest cost in E. If the vertices attached to the edge are on different connected components in T, add this edge to T; otherwise, discard this edge and select the next edge with the lowest cost. And so on, until all vertices in T form a connected component.

[0062] MST can be used for image segmentation tasks, but few studies have applied MST to SAR image superpixel segmentation. The method of the present invention provides how to construct MST for SAR images. The specific steps are as follows:

[0063] 1) The input SAR image is regarded as a connected, weighted undirected graph, with v vertices and e edges in the graph;

[0064] 2) Obtaining edge intensity map of SAR image;

[0065] 3) Initialize all vertices as individual pixels;

[0066] 4) Initialize the minimum spanning tree to be empty. The minimum spanning tree has the same v vertices as the original graph, but no edges.

[0067] 5) Calculate the pixel similarity between all pixels of the SAR image and their neighboring pixels as the weight of the edge, and sort all e edges in ascending order according to the weight:

[0068] 6) Loop: Start from the edge with the smallest weight and traverse each edge until all the nodes in the graph are in the same connected component. If the two nodes connected by an edge are not in the same connected component, add this edge to the minimum spanning tree.

[0069] 7) Return the generated minimum spanning tree.

[0070] The specific operation method of step D is as follows: In the previous step, the Kruskal algorithm calculates the minimum spanning tree in a bottom-up manner. Through the iterative merging of trees (pixels), the entire SAR image will eventually generate a large MST, and each superpixel can be regarded as a subtree in the large MST. Figure 1 and Figure 2As shown in the figure, pixels of the same category can form subtrees with smaller edge weights, which are also smaller MSTs. However, the edge weights between subtrees formed by different regions are much larger. Therefore, it can be concluded that the generated large MST contains the hierarchical structure information of the SAR image superpixels. By setting different numbers of superpixels, hierarchical or multi-scale superpixel segmentation results can be quickly obtained.

[0071] Figure 3 A schematic diagram of multi-scale superpixel segmentation based on MST partitioning is given. Specifically, for all edges within the largest MST, they can be sorted according to the weight of the edge, and then the edge with the largest weight is deleted. Therefore, one MST can be divided into two MSTs. Then, for each small MST, the edge weight is re-sorted, and then the edge with the largest weight is further deleted. This process is performed for each MST until the number of MSTs finally generated reaches the preset number of superpixels. Therefore, it can be found that the method proposed by the present invention can actually dynamically generate any number of superpixels from the MST.

[0072] The specific operation method of step E is as follows: after obtaining the multi-scale segmentation result of the SAR image, this step further uses the gradient calculation method to extract the super-pixel segmentation edge according to the pixel segmentation label, and then calculates the mean of each super-pixel of the SAR image according to the segmentation edge, so as to obtain the final SAR image segmentation result.

[0073] Experimental effect analysis:

[0074] An airborne SAR dataset is used to verify the proposed method. The data is a part of the X-band SAR image with a resolution of 0.5 meters. The image size is 1063×1165. For evaluation and comparison, three hierarchical segmentation methods of SAR images are selected. The first is a hierarchical region merging method, namely the GIOESP method. The second is a fast region merging method based on common boundary constraints, namely the RCBLP method. The last one is an adaptive superpixel segmentation method with edge constraints, namely the ALFCE method. The first two methods belong to region merging methods, which merge initial superpixels to achieve final segmentation. The last one is a superpixel segmentation method, which can achieve hierarchical segmentation by repeatedly executing the ALFCE method.

[0075] Figure 4 Middle (a), Figure 4 Middle (b), Figure 4 (c) and Figure 4 (d) shows the multi-scale superpixel segmentation results of SAR data by the method of the present invention. For the X-band SAR image, hierarchical segmentation with 3000, 1500, 800 and 400 superpixels is given. It can be seen from the figure that more superpixels can retain more image details. Figure 4 In (a), not only the obvious image boundaries and larger building areas can be well preserved, but also the smaller isolated buildings can be well preserved. On the contrary, as the number of superpixels decreases, some small isolated buildings will be merged into their adjacent areas, resulting in some information loss. Nevertheless, the structural information of the SAR image is well preserved, which shows that the generated superpixels can well maintain the image edges and can retain the main image information. It is worth noting that the superpixel segmentation results are all obtained from the same MST, and the time required to generate all results is basically the same. Therefore, segmentation results of any scale can be obtained in real time from one MST, indicating that the present invention can achieve hierarchical segmentation of SAR images.

[0076] Figure 5 Middle (a), Figure 5 (b) and Figure 5 (c) shows the segmentation results of SAR data by GIOESP, RCBLP and ALFCE respectively. For all the results, the number of segmented regions is set to 1500. Figure 5 As can be seen in (a), GIOESP can clearly preserve most of the image edges and building areas. Figure 5 Compared to the result in (b), some small isolated building details are missing, resulting in worse segmentation effect. Figure 5 (b) shows the result of RCBLP, which has similar performance to GIOESP. Since both methods merge the initial superpixels to achieve the final segmentation, some over-merging errors are prone to occur, resulting in the loss of details of small targets. In contrast, the method of the present invention calculates the MST of the SAR image, which actually contains hierarchical structure information. The segmentation result is then obtained from the MST. Therefore, there are fewer over-merging errors and the detailed information of small targets can be well maintained. Figure 5 (c) shows the result of ALFCE, which is also a superpixel segmentation method for SAR images. It can be seen that the ALFCE method can clearly obtain the image boundary and structure information of small artificial targets. However, this method performs superpixel segmentation based on local feature clustering, which leads to some over-segmentation errors. In addition, the segmented superpixel boundary cannot fit the image boundary well. In contrast, due to the global MST of SAR images, Figure 4 The result obtained by the segmentation method of the present invention in (b) can better fit the real boundary and has fewer over-segmentation errors.

[0077] The above description is only a preferred embodiment of the present invention and is not intended to impose any limitation on the technical solution of the present invention. Those skilled in the art should understand that, without departing from the spirit and principles of the present invention, the technical solution can also be subjected to several simple modifications and substitutions, and these modifications and substitutions will also fall within the scope of protection covered by the claims.

Claims

1. A fast multi-scale superpixel segmentation method for synthetic aperture radar images, characterized in that: The following steps are involved: The final similarity of two pixels is measured by combining backscattering grayscale similarity, edge information similarity and homogeneity similarity; Graph structure representation of synthetic aperture radar SAR images; Construct the minimum spanning tree MST for SAR images; Based on the minimum spanning tree, fast multi-scale superpixel segmentation of SAR images is realized, and the superpixel segmentation boundary is obtained based on the segmentation result to generate the SAR image segmentation result; The backscatter difference of the image is obtained based on the average amplitude of samples in two 3×3 local areas based on the current center pixel; The backscatter grayscale similarity between two pixels is: Among them, two local regions R u and R v The sample means of and L is the number of views of the SAR image; The SAR image edge detector based on guided filter is used to extract the edge of SAR image. After obtaining the image edge, the edge information similarity is: σ E (u,v)=max{{ESM(u),ESM(v)} Where ESM(u) represents the edge strength map of pixel u; The homogeneity similarity is: in, and Represents the local area R u and R v The coefficient of variation of the SAR image, obtained by the ratio of the standard deviation to the mean; The final similarity of two pixels is the sum of backscattering grayscale similarity, edge information similarity, and homogeneity similarity; The operation method of graph structure representation of SAR image is as follows: Assume that SAR image G = (V, F) is an undirected weighted graph consisting of vertex set V and edge set F; each pixel of SAR image is represented by a vertex, and each two vertices are connected based on 8-connected neighborhood, and the vertex set V is divided into K non-overlapping components, i.e., superpixels, and each component corresponds to a connected subgraph G' = (V', F'), where V'∈V, F∈F'; The Kruskal algorithm is used to construct the minimum spanning tree for SAR images. The specific steps are as follows: 1) The input SAR image is regarded as a connected, weighted undirected graph with N vertices and M edges. 2) Obtaining edge intensity map of SAR image; 3) Initialize all vertices as individual pixels; 4) Initialize the minimum spanning tree to be empty. The minimum spanning tree has the same N vertices as the original graph, but no edges. 5) Calculate the pixel similarity between all pixels of the SAR image and their neighboring pixels as the weight of the edge, and sort all M edges from small to large according to the weight; 6) Loop: Start from the edge with the smallest weight and traverse each edge until all the nodes in the graph are in the same connected component. If the two nodes connected by an edge are not in the same connected component, add this edge to the minimum spanning tree. 7) Return the generated minimum spanning tree; The method realizes fast multi-scale superpixel segmentation of SAR images based on minimum spanning tree, obtains superpixel segmentation boundary based on segmentation result and generates SAR image segmentation result, including: generating a large MST for the whole SAR image, and treating each superpixel as a subtree in the large MST; sorting all edges in the largest MST according to edge weights, and deleting the edge with the largest weight; re-sorting the edge weights for each small MST, and further deleting the edge with the largest weight; until the number of MSTs finally generated reaches a preset number of superpixels; after obtaining the multi-scale segmentation result of the SAR image, further extracting superpixel segmentation edges according to pixel segmentation labels, and then calculating the mean of each superpixel of the SAR image according to the segmentation edges, so as to obtain the final SAR image segmentation result.

2. The method for fast multi-scale superpixel segmentation of synthetic aperture radar images according to claim 1, characterized in that: After obtaining the multi-scale segmentation results of the SAR image, the superpixel segmentation edge is extracted using the gradient calculation method according to the pixel segmentation label, and the mean of each superpixel of the SAR image is calculated according to the segmentation edge to obtain the final SAR image segmentation result.

Images