An integrated segmentation method of high-resolution optical and SAR images for geographic objects

Abstract

The present invention discloses a method for integrated segmentation of high-resolution optical and SAR images for geographic objects, comprising the following steps: obtaining a reliable set of geographic objects by segmenting the optical image; adaptively extracting marker points for each object, and projecting them into the SAR image based on the coarse registration result; finally, performing region growth based on the marker points in the SAR image, and finally obtaining a set of objects that match the segmented objects of the optical image. The experimental results of the present invention on multiple groups of optical and SAR images show that the optical-SAR matching object set extracted by the proposed method is closer to the actual geographic object, and the J-value can reach above 7.8, which is significantly better than the comparison method in both visual analysis and quantitative evaluation.

Description

Technical Field

[0001] The invention relates to a method for integrated image segmentation, and in particular to a method for integrated segmentation of high-resolution optical and SAR images oriented to geographic objects. Background Art

[0002] High-resolution optical and synthetic aperture radar (SAR) images are currently the two main data sources in remote sensing target monitoring applications. Optical images have rich spectral, texture and spatial detail information, which helps to achieve fine classification of surface cover. However, due to the limitations of passive imaging mechanisms, optical images are easily restricted by factors such as weather, clouds, and lighting conditions. On the other hand, synthetic aperture radar has the advantages of working all day and all weather, and its echo characteristics help to describe the height, structure and other information of the ground objects, but the lack of spectral information and the presence of speckle noise seriously affect the data reliability of SAR images.

[0003] Extracting a unified set of geographic objects from heterogeneous images is the premise for combining the advantages of optical and SAR images to carry out target recognition. The commonly used methods currently include pixel-level feature point extraction and registration, as well as heterogeneous image integrated segmentation and registration methods (YU Xianchuan, LV Zhonghua, HU Dan. Review of remote sensing image registration techniques [J]. Optics and Precision Engineering, 2013, 21 (11)). The former mainly extracts feature points based on methods such as SIFT, and then performs registration by detecting pairs of points with the same name. For example, Wang Mengmeng et al. (Optical and SAR image registration based on spatial constraints and structural features [J]. Journal of Wuhan University (Information Science Edition), 2022, 47 (01): 141-148.) proposed an optical and SAR image registration method based on spatial constraints and structural features. This method proposes strategies such as local geometric correction of images and construction of geometric structure feature descriptors, which significantly improves the registration accuracy. Nevertheless, due to the huge differences in the imaging mechanisms of optical and SAR images (Fan Jianwei, Wu Yan, Li Ming, et al. SAR and Optical Image Registration Using Nonlinear Diffusion and Phase Congruency Structural Descriptor [J]. IEEE Transactions on Geoscience and Remote Sensing, 2018, 56 (9)), it is difficult to define a reliable similarity measurement method to carry out feature point matching; on the other hand, the registration result only establishes the mapping relationship between pixels, but does not cross the semantic gap between pixels and geographic objects. The latter mainly obtains significant regional features and structural features by segmenting the image, and then aligns the optical and SAR images on this basis. For example, Xu Chuan et al. (Research on the automatic registration method of optical and SAR images based on multi-features and multi-measures [D]. Wuhan University, 2013) proposed a research on the automatic registration method of optical and SAR images based on multi-features and multi-measures. This method uses multiple segmentations and registrations, and then constructs a triangulation registration method based on global optimization. This type of method is essentially an object-oriented registration method, so the registration results have a higher reference value for target recognition applications.Nevertheless, the realization of integrated segmentation and registration requires accurate segmentation results and a certain number of closed areas. Also, due to the significant imaging differences between optics and SAR, there are incomplete targets in the segmentation results, that is, the segmentation results often do not match the real objects. Summary of the invention

[0004] Purpose of the invention: The purpose of the present invention is to provide a method for integrated segmentation of high-resolution optical and SAR images for geographic objects, which can extract a unified optical-SAR matching object set and can determine appropriate object boundaries respectively from the heterogeneous images themselves.

[0005] Technical solution: The method for integrated segmentation of high-resolution optics and SAR images of the present invention comprises the following steps:

[0006] S1, for the two sets of high-resolution optical images and SAR images collected, the SIFT algorithm is used to extract the corresponding feature points in the optical image and the SAR image, and the mapping equation from optical to SAR is obtained to achieve the rough registration of optical and SAR images;

[0007] S2, taking the optical image as a reference image and performing image segmentation, extracting the centroid of each segmented object to obtain adaptive marker points in the optical image;

[0008] S3, using the optical to SAR mapping equation obtained in step S1, mapping the centroid in the optical object to the SAR image to obtain the SAR image marker points; and obtaining a set of objects in the SAR image that match the optical object according to the region growth of the SAR image marker points.

[0009] Furthermore, in step S1, the specific steps for achieving the rough registration of the optical image and the SAR image are as follows:

[0010] S11, taking urban scenes including residential buildings, factories and streets as the object, two sets of data sets were collected, and the heterogeneous images were resampled and preprocessed using the software ENVI; taking the optical image as the benchmark, the SAR image was resampled using the bilinear interpolation method to obtain the SAR image and the optical image with the same resolution;

[0011] S12, using SIFT algorithm to extract corresponding feature points P in optical images and SAR images m , the feature point P m The calculation formula is as follows:

[0012]

[0013] Among them, D is the DOG value of the key point, is the scale space function pair P = (x, y, σ) TThe partial derivative of (x, y) is the coordinate of the key point, σ is the image scale parameter, so as to obtain the feature point set F = {f1, f2, ..., f i}, i = 1, 2, ..., n, where n is the total number of feature point pairs; f i It is composed of corresponding feature points of optical and SAR images, f i ={f gi (x i ,y i ), f si (x′ i , y′ i )},f gi (x i ,y i ) is the optical feature point, x i ,y i are the horizontal and vertical coordinates of the optical image feature points, respectively, and f si (x′ i , y′ i ) is the corresponding feature point of the SAR image, x′ i , y′ i are the horizontal and vertical coordinates of the corresponding feature points of the SAR image respectively; the Homography transformation method is used to determine the mapping equation Hom:

[0014]

[0015] Among them, (x, y, z) and (x′, y′, z′) are the normalized feature points corresponding to optical and SAR respectively. Let z = z′ = 1, and the mapping equation is obtained:

[0016]

[0017] Among them, h1 to h9 are the 9 degree of freedom parameters of the mapping equation Hom.

[0018] Furthermore, in step S2, the optical image is first segmented, and then corresponding marking points are extracted for each object. The specific steps are as follows:

[0019] S21, the software eCognition was used to complete the multi-scale image segmentation, and the visual interpretation method was used to determine the segmentation parameters (Scale, Shape, Compactness) of the two data sets, where Scale is the scale parameter, Shape is the shape factor, and Compactness is the compactness factor;

[0020] S22, let the object set extracted by image segmentation be R = {R1, R2, ..., R j}, j = 1, 2, ..., m, where m is the total number of objects segmented; for each object Rj , the coordinates of the marking points are as follows:

[0021]

[0022] Among them, x d ,y d are the horizontal and vertical coordinates of the pixel points in the object, and s is the total number of pixels;

[0023] The coordinates of the obtained marking point are marked as k j (X j , Y j ), traverse all objects and obtain the optical image object marker point set K = {k1, k2, ..., k j}.

[0024] Furthermore, in step S3, the specific steps of obtaining the SAR image matching object set are as follows:

[0025] S31, projecting the optical image object marker point set K according to the mapping equation Hom, obtaining the initial seed point set S in the SAR image = {s1, s2, ..., s j}, where s j For j (x j ,y j );

[0026] When mapped to the seed point s of the SAR image j (x j ,y j ) is outside the SAR image boundary, then calculate point s i The Euclidean distance from the center point of the SAR image is the point t with the smallest Euclidean distance ρ′. j (x′ j , y′ j ) instead of point s j (x j ,y j ) as the seed point; x j ,y j are the horizontal and vertical coordinates of the initial seed point in the SAR image, x′ j , y′ j is the horizontal and vertical coordinates of the adaptive seed point, then:

[0027]

[0028] The seed point set T in the adaptive SAR image is obtained as follows: j}, where t j t j (x′ j , y′ j);

[0029] S32, according to the seed point set T, region growth is performed for each seed point at the same time; for any seed point, if the pixel value of a certain point in the eight-neighborhood region satisfies the following formula:

[0030]

[0031] Then the point is judged to meet the growth conditions and added to the region;

[0032] Among them, (X, Y) is the coordinate of the midpoint of the eight-neighborhood of the seed point, t j (x′ j , y′ j ) is the seed point coordinate, Img is the pixel matrix of the SAR image;

[0033] Calculate the seed point t j (x′ j , y′ j The difference between the pixel value at the eight neighborhood points and the seed point pixel, all the differences are added and averaged to get the value as the threshold q j , the expression is as follows:

[0034]

[0035] Among them, u, v, and w are all independent variables. After the region growth is completed, the SAR image object set C corresponding to the optical image object set R is obtained. j}, and finally obtain a unified set of optical-SAR matching objects.

[0036] Compared with the existing technology, its remarkable effects are as follows:

[0037] 1. The use of optical image marker mapping and seed point region growth in SAR images can not only extract a unified optical-SAR matching object set, but also determine the appropriate object boundaries from the heterogeneous images themselves, thereby providing reliable analysis primitives for subsequent target recognition, etc., solving the problem that objects in optical and SAR images are difficult to match after segmentation using general segmentation methods due to differences in imaging properties, and also solving the problem that it is difficult to obtain appropriate object boundaries using general segmentation methods;

[0038] 2. The present invention performs segmentation in the optical image and extracts adaptive marker points of the segmented objects. All the marker points are then projected onto the SAR image and region growth is performed simultaneously. The obtained optical-SAR image matching object set is more consistent with the real objects, and the J-value index reaches above 7.8. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1It is a schematic diagram of the overall process of the present invention;

[0040] Figure 2(a) is a schematic diagram of the high-resolution optical images of the experimental data set 1-March 2011;

[0041] Figure 2(b) is a schematic diagram of the SAR images of the experimental data set 1-March 2011;

[0042] Figure 3(a) is a schematic diagram of the high-resolution optical images of the experimental data set 2-March 2011;

[0043] Figure 3(b) is a schematic diagram of the SAR images of the experimental data set 2-March 2011;

[0044] FIG4( a ) is a schematic diagram of the optical image segmentation results of the experimental data set 1-the present invention;

[0045] FIG4( b ) is an intention diagram of the SAR image segmentation result of the experimental data set 1-the present invention;

[0046] Figure 4(c) is a schematic diagram of the optical image segmentation results of experimental dataset 1-strategy 1;

[0047] Figure 4(d) is a schematic diagram of the SAR image segmentation results of experimental dataset 1-strategy 1;

[0048] Figure 4(e) is a schematic diagram of the optical image segmentation results of experimental dataset 1-strategy 2;

[0049] Figure 4(f) shows the SAR image segmentation result of experimental dataset 1-strategy 2;

[0050] FIG4( g ) is a schematic diagram showing the result of obtaining the specific object boundary after edge detection for the optical image segmentation result of experimental data set 1-strategy 2;

[0051] Figure 4(h) is a schematic diagram showing the result of obtaining the specific object boundary after edge detection for the SAR image segmentation result of experimental data set 1-strategy 2;

[0052] FIG5( a ) is a schematic diagram of the optical image segmentation results of the experimental data set 2-the present invention;

[0053] FIG5( b ) is an intention diagram of the SAR image segmentation result of the experimental data set 2-the present invention;

[0054] Figure 5(c) is a schematic diagram of the optical image segmentation results of experimental dataset 2-strategy 1;

[0055] Figure 5(d) is a schematic diagram of the SAR image segmentation results of experimental dataset 2-strategy 1;

[0056] Figure 5(e) is a schematic diagram of the optical image segmentation results of experimental dataset 2-strategy 2;

[0057] Figure 5(f) shows the SAR image segmentation result of experimental dataset 2-strategy 2;

[0058] FIG5( g ) is a schematic diagram showing the result of obtaining the specific object boundary after edge detection for the optical image segmentation result of experimental data set 2-strategy 2;

[0059] Figure 5(h) is a schematic diagram of the SAR image segmentation results of experimental dataset 2-strategy 2, which uses edge detection to obtain the specific object boundaries. DETAILED DESCRIPTION

[0060] The present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0061] The method of the present invention mainly includes three steps: coarse registration of heterogeneous images based on SIFT feature points; adaptive marker point extraction of optical image object sets; and extraction of SAR image matching object sets combined with mapping and region growing. The specific process is as follows Figure 1 The detailed implementation process is as follows:

[0062] Step S1: Rough registration of heterogeneous images based on SIFT feature points

[0063] Step S11: Data preparation and preprocessing

[0064] The present invention uses two sets of high-resolution optical images and SAR images of Sendai, Japan to conduct experiments, as shown in Figures 2(a), 2(b) and 3(a), 3(b). The two sets of images were collected in March 2011, so the image features have a high degree of consistency. Among them, the optical image is an IKONOS satellite image, collected in March 2011, with a spatial resolution of 1m; the SAR image is a TerraSAR-X satellite image, collected in March 2011, with a spatial resolution of 3m.

[0065] As shown in Figure 2(a), Figure 2(b) and Figure 3(a), Figure 3(b), both datasets are common urban scenes, including typical objects such as residential buildings, factories, and streets. On this basis, the currently popular remote sensing image processing commercial software ENVI is used to resample and preprocess the heterogeneous images. Taking the optical image as the benchmark, the SAR image is resampled using the bilinear interpolation method to obtain SAR images and optical images with the same resolution.

[0066] Step S12: SIFT extraction and establishment of matching equations

[0067] The present invention uses SIFT (Scale-invariant feature transform) algorithm to extract corresponding feature points in optical and SAR images, and uses this to perform rough registration of optical and SAR images. m It can be calculated by formula (1):

[0068]

[0069] Among them, D is the DOG value of the key point, is the scale space function pair P = (x, y, σ) T The partial derivative of (x, y) is the coordinate of the key point, σ is the image scale parameter, that is, the fuzzy coefficient, so as to obtain the feature point set F = {f1, f2, ..., f i}, (i = 1, 2, ..., n), where n is the total number of feature point pairs, f i It is composed of the corresponding feature points of optical and SAR images, that is, f i ={f gi (x i ,y i ), f si (x′ i , y′ i )}(i=1,2,…,n),f gi (x i ,y i ) is the optical feature point, x i ,y i are the horizontal and vertical coordinates of the optical image feature points, respectively, and f si (x′ i , y′ i ) is the corresponding feature point of the SAR image, x′ i , y′ i are the horizontal and vertical coordinates of the corresponding feature points of the SAR image, respectively. On this basis, the present invention adopts the Homography transformation method to determine the mapping equation Hom:

[0070]

[0071] Among them, (x, y, z) and (x′, y′, z′) are the normalized feature points corresponding to optical and SAR respectively, z=z′=1. Using the obtained feature point set F and the basic knowledge of the matrix, the mapping equation can be obtained:

[0072]

[0073] Among them, h1 to h9 are the 9 degree of freedom parameters of the mapping equation Hom.

[0074] Step S2: multi-scale segmentation and adaptive extraction of landmarks

[0075] In view of the fact that optical images have rich spectral, texture and spatial detail information. To this end, the optical image is regarded as a reference image and image segmentation is performed, so that the object set obtained is closer to the real geographic object. On this basis, a matching object is searched in the SAR image for each object in the optical image object set, so as to obtain a unified optical and SAR object set. Therefore, the present invention first segments the optical image, and then extracts the corresponding marker points for each object. The specific steps are as follows:

[0076] Step S21: multi-scale image segmentation

[0077] In the image segmentation stage, the present invention uses the object-oriented commercial software eCognition. Compared with the traditional multi-scale segmentation method, the most notable feature of eCognition is that during image classification and multi-scale segmentation, the homogeneity between the pixels inside the image is calculated for segmentation, and the contour of the multi-scale segmented object obtained is close to the boundary of the object. In the multi-scale segmentation process, the setting of three key parameters is mainly involved, including the scale parameter (Scale), the shape factor (Shape), and the compactness factor (Compactness), which are determined by the visual interpretation method in the present invention. The segmentation parameters (Scale, Shape, Compactness) of the two sets of data sets are (80, 0.6, 0.6) and (85, 0.4, 0.6), respectively.

[0078] Step S22: adaptive extraction of marker points

[0079] Let the object set extracted by image segmentation be R = {R1, R2, ..., R j}, (j=1, 2, ..., m) where m is the total number of objects segmented. For each object R j , according to formula (4), the coordinates of its marking points are adaptively determined:

[0080]

[0081] Among them, x d ,y d are the horizontal and vertical coordinates of the pixel points in the object, s is the total number of pixels; the coordinates of the marker point obtained are marked as k j (X j , Y j ), traverse all objects to obtain the optical image object marker point set K = {k1, k2, ..., k j}(j=1,2,…,m).

[0082] Step S3: SAR image matching and ground object extraction

[0083] Based on the mapping equation Hom and the optical image marker point set K, the present invention designs a SAR image matching ground object extraction strategy, which mainly includes SAR image marker point extraction and marker point-based region growth.

[0084] Step S31: Extracting SAR image marker points

[0085] According to the mapping equation Hom, K is projected to obtain the initial seed point set S in the SAR image = {s1, s2, ..., s j}. (j = 1, 2, ..., m), where s j That is s j (x j ,y j );

[0086] At this time, due to the significant difference between optical and SAR images, the marker points on the boundary of the optical image may be outside the boundary of the SAR image when mapped to the SAR image. Therefore, the present invention designs an adaptive compensation method: when the seed point s mapped to the SAR image j (x j ,y j ) is outside the SAR image boundary, then calculate point s i The Euclidean distance from the center point of the SAR image is the point t with the smallest Euclidean distance ρ′. j (x′ j , y′ j ) instead of point s j (x j ,y j ) as the seed point, the formula is as follows:

[0087]

[0088] The seed point set T = {t1, t2, ..., t j}(j=1, 2, ..., m), where t j That is t j (x′ j , y′ j ).

[0089] Step S32: region growth based on marker points

[0090] Based on the seed point set T, region growth is performed for each seed point at the same time. For any seed point, if the pixel value of a point in the eight-neighborhood region satisfies formula (6), then the point is determined to meet the growth condition and added to the region. In formula (6), (X, Y) is the coordinate of the point in the eight-neighborhood region of the seed point, t j (x′ j , y′j ) is the seed point coordinate, and Img is the pixel matrix of the SAR image.

[0091]

[0092] Among them, the threshold q j Adopt the following strategy to adaptively determine: Calculate the seed point t j (x′ j , y′ j ) The difference between the pixel value at the eight neighborhood points and the seed point pixel, on this basis, all the differences are added and averaged to determine q j The value of is shown in formula (7):

[0093]

[0094] Among them, u, v, and w are all independent variables. After the region growth is completed, the SAR image object set C corresponding to the optical image object set R can be obtained. j}(j=1, 2, ..., m), thereby finally obtaining a unified optical-SAR matching object set.

[0095] Experimental results and analysis

[0096] (1) Experimental setup and comparison method

[0097] Since the method of the present invention belongs to an integrated segmentation and registration method, two advanced methods of the same type were selected for comparison in the experiment, including:

[0098] Heterogeneous image fusion and segmentation method (Gao Yu, Hu Zhaoling, Fan Ru. The impact of high-resolution image fusion algorithm on the classification accuracy of coastal wetland land use [J]. Bulletin of Surveying and Mapping, 2022, (01): 116-120.), denoted as "Strategy 1";

[0099] Level set segmentation method for complex texture images based on structural features (Deng Xingtao. Research on level set segmentation method for complex texture images based on structural features [D]. Hefei University of Technology, 2021), denoted as "Strategy 2".

[0100] Among them, Strategy 1 uses the object-oriented commercial software eCognition to segment the fused image in the image segmentation stage. The segmentation parameters (Scale, Shape, Compactness) determined by the present invention based on visual interpretation are (80, 0.6, 0.6) and (85, 0.4, 0.6), respectively. In addition, since Strategy 2 segmented the optical and SAR images separately, the number of objects in the segmentation results is different. For this reason, the present invention only selects the mutually matching object pairs in optical and SAR for analysis.

[0101] (2) Experimental results and visual analysis

[0102] The method of the present invention and the comparative method were experimented based on two sets of data sets, and the results are shown in Figures 4(a)-(d) and 5(a)-(d). In addition, to facilitate visual analysis, the present invention represents typical areas in the image with wireframes of different shapes, and selects three typical objects for analysis. In Figures 4(a)-(d) and 5(a)-(d), the circular frame represents a building, the rectangular frame represents a road, and the square frame represents other man-made targets.

[0103] As shown in Figures 4(c)-(d) and 5(c)-(d), since Strategy 1 first needs to fuse heterogeneous images and then segment the fused images, the rich spectral and texture information of the optical image is affected by the SAR image to a certain extent, resulting in a decrease in the segmentation effect compared with the single optical image segmentation of the present invention. Therefore, Strategy 1 needs to comprehensively consider the optical and SAR images in the initial segmentation, which leads to poor segmentation effect. In Strategy 2, the optical and SAR images are simultaneously subjected to level set segmentation, and the segmentation results are shown in Figures 4(e)-(f) and 5(e)-(f). The edge detection method is used to extract the object boundaries in the optical and SAR images after segmentation, as shown in Figures 4(g)-(h) and 5(g)-(h). From the image analysis, it can be seen that the extraction effect of some objects in the optical image is not obvious, which causes certain difficulties in matching objects between the optical and SAR images. In addition, due to the huge imaging difference between optical and SAR images, the segmentation effects of optical images and SAR images are also quite different for the same segmentation method, which causes certain difficulties in finding corresponding objects in optical and SAR images. In the method of the present invention, as shown in FIG. 4 (a)-(b) and FIG. 5 (a)-(b), the optical image with rich spectral and texture information is selected for segmentation, and then the adaptive points of each segmented object are projected into the SAR image for region growth, thereby obtaining the object in the SAR image corresponding to the optical image. The optical-SAR object obtained by the method of the present invention can not only be well matched one by one, but also obtain a clear object boundary. In addition, although the edge of the object in the SAR image obtained by the method of the present invention is not as smooth as the edge of the object obtained by strategy 1, it is more consistent with the real boundary of the ground object in the SAR image, and the internal homogeneity of the object is higher. From the segmentation results of the SAR image in strategy 1, although the shape is consistent with the optical object, it obviously deviates from the real ground object, reducing its use value. From the segmentation results of the optical and SAR images in strategy 2, some objects in the optical image do not have large gradient value pixels, and the pixel value difference with the surrounding area is not obvious, which leads to some objects not being segmented, and there are certain differences in the segmentation effects of the optical image and the SAR image, which causes certain difficulties in obtaining a unified optical and SAR object set. The present invention uses a region growing method to add points similar to seed points into the region to form an object, thereby reducing the impact of speckle noise in the SAR image on the result, so that the formed region is closer to the real object.

[0104] In addition, for the building corresponding to the circular frame, the object obtained by the method of the present invention has a good boundary and is very similar to the corresponding object structure in the optical image, and can be well matched. However, the building object obtained by strategy 1 is significantly offset from the real building. The edge of the building object obtained by SAR image segmentation in strategy 2 is incoherent. The reason is that the SAR image is affected by speckle noise. There may be multiple points with large pixel values ​​in an object, resulting in the formation of a separate boundary during segmentation, which makes it difficult to match with the optical object.

[0105] (3) Quantitative evaluation results

[0106] In order to further quantitatively analyze the performance of different methods, the present invention selects three indicators: standard deviation, J-value, and SSIM.

[0107] The standard deviation represents the degree of dispersion of the mean of a set of data. In the present invention, the standard deviation of the pixels in each object is calculated, and then the average is taken as the measurement index of the overall effect. The formula is as follows:

[0108]

[0109] Where m is the total number of objects, N k , μ k are the number and mean of pixels in the kth object, x i is the pixel value of the i-th point in the object, σ k is the standard deviation of the pixel value of the kth object, and σ is the mean of the standard deviations of all objects.

[0110] J-value reflects the degree of homogeneity within an object, and the formula is as follows:

[0111]

[0112] Let the position z of each pixel be f (x, y) is pixel z f The pixel value of f = (1, 2, ..., F), F is the total number of pixels in the image. Determine the window size of H × H (H is the scale parameter), Z f In pixels z f The set of all pixels within the window centered at z f (x,y)∈Z f , then S T Z f The overall variance of all pixels in ω Z f The variance of all pixels in the image is calculated with 256 grayscale values ​​as the center, and then the variance is summed up.

[0113] The above two indicators are used to measure the homogeneity of a single image segmentation result. The present invention further uses SSIM to measure the similarity of matching objects in two images. At this time, since the same object may contain different pixels in heterogeneous images, the present invention uses an improved SSIM for measurement, and the formula is as follows:

[0114]

[0115] Among them, σ x , σ y are the standard deviations of the optical object and the SAR object respectively, C2 is a constant, generally taken as 0.03, σ′ xy is the covariance.

[0116] The quantitative evaluation results based on these indicators are shown in Tables 1 and 2:

[0117] Table 1 Quantitative evaluation results of dataset 1

[0118]

[0119] Table 2 Quantitative evaluation results of dataset 2

[0120]

[0121] The object obtained by the region growth of a single SAR image is quantitatively evaluated, and the standard deviation and J-value are used for analysis. From the quantitative evaluation results in Tables 1 and 2, it can be seen that the standard deviations of the SAR image region growth results of Dataset 1 and Dataset 2 obtained by the method of the present invention are only 6.4 and 8.8, which are smaller than the standard deviations of the SAR image segmentation results in the two strategies, indicating that the discrete degree of the pixel values ​​of all points in the object and the pixel mean of the object in the SAR image obtained by the method of the present invention is low. The pixel values ​​in a real object generally do not change greatly, which shows that the object in the SAR image obtained by the method of the present invention is closer to the real object and is easy to match with the optical object. From the perspective of the J-value index, the J-value values ​​of the SAR image object sets in the two sets of data obtained by the method of the present invention are 7.8 and 8.6, respectively, while the J-value values ​​of the SAR image object sets obtained by strategy 1 are 3.8 and 3.3, respectively, and the J-value values ​​of the SAR image object sets obtained by strategy 2 are 4.8 and 4.5, respectively. It shows that the homogeneity of the SAR image object objects obtained by the method of the present invention is higher and closer to the real object objects. According to the principle, the pixel points in a real geographic object generally have a high similarity. The present invention uses the method of regional growth to grow the seed points obtained after the optical marker point is projected. Since certain growth rules are established, it is easier to include points similar to the seed points in the region, so the obtained object is closer to the real object and has a higher degree of homogeneity. Based on Figures 4(g)-(h) and 5(g)-(h), it is also proved that the experimental results are consistent with the quantitative analysis results, proving that the object in the SAR image obtained by the method of the present invention is smoother inside, more homogeneous, and can be better matched with the optical object.

[0122] (4) Analysis of projection effects of different types of objects

[0123] In order to further quantitatively analyze the segmentation effects of different ground objects, the present invention selects buildings, roads and other man-made targets for statistics and evaluation. The number of different ground objects is statistically analyzed for the segmentation results of heterogeneous images, and the results are shown in Tables 3 and 4.

[0124] Table 3 Statistics of the proportion of different ground features in dataset 1

[0125]

[0126] Table 4 Statistics of the proportion of different ground features in dataset 2

[0127]

[0128] It can be seen from Table 3 and Table 4 that the method of the present invention has a good effect on the extraction of building objects in optical and SAR, and the difference between the two is within 6%, while the difference of strategy 1 is within 10%, and the difference of strategy 2 reaches 14%. In principle, since the homogeneity of building objects is high and the internal features of the objects are more, it is easier to obtain objects similar to real buildings by the region growing method of the present invention. In terms of road extraction, since a road may be divided into several regions when the optical image is multi-scale segmented, the difference of the method of the present invention reaches 7%. In strategy 1, the difference in the proportion of roads between optical images and SAR images reaches 10%, while in strategy 2, although the difference in road extraction is small, the proportion of the number of roads extracted is significantly less than the actual proportion, which has no practical significance. For the extraction of other artificial targets, the difference of the method of the present invention is 12%, while the differences of strategy 1 and strategy 2 are 18% and 9% respectively. It can be seen that in the face of the huge imaging differences between optical and SAR images, the method of the present invention can better maintain the category consistency of matching objects, which is more practical than the comparison method.

Claims

1. A method for integrated segmentation of high-resolution optical and SAR images for geographic objects, characterized in that: The steps include: S1, for the two sets of high-resolution optical images and SAR images collected, the SIFT algorithm is used to extract the corresponding feature points in the optical image and the SAR image, and the mapping equation from optical to SAR is obtained to achieve the rough registration of optical and SAR images; S2, taking the optical image as a reference image and performing image segmentation, extracting the centroid of each segmented object to obtain adaptive marker points in the optical image; S3, using the optical to SAR mapping equation obtained in step S1, mapping the centroid in the optical object to the SAR image to obtain the SAR image marker point; and obtaining a set of objects in the SAR image that match the optical object according to the region growth of the SAR image marker point; In step S1, the specific steps for achieving coarse registration of optical images and SAR images are as follows: S11, taking urban scenes including residential buildings, factories and streets as the object, two sets of data sets were collected, and the heterogeneous images were resampled and preprocessed using the software ENVI; taking the optical image as the benchmark, the SAR image was resampled using the bilinear interpolation method to obtain the SAR image and the optical image with the same resolution; S12, using SIFT algorithm to extract corresponding feature points P in optical images and SAR images m , the feature point P m The calculation formula is as follows: Among them, D is the DOG value of the key point, is the scale space function pair P = (x, y, σ) T The partial derivative of (x, y) is the coordinate of the key point, σ is the image scale parameter, so as to obtain the feature point set F = {f1, f2, ..., f i }, i = 1, 2, ..., n, where n is the total number of feature point pairs; f i It is composed of corresponding feature points of optical and SAR images, f i ={f gi (x i ,y i ), f si (x i ′ ,y i ′ )},f gi (x i ,y i ) is the optical feature point, x i ,y i are the horizontal and vertical coordinates of the optical image feature points, respectively, and f si (x i ′ ,y i ′ ) is the corresponding feature point of the SAR image, x i ′ ,y i ′ are the horizontal and vertical coordinates of the corresponding feature points of the SAR image respectively; the Homography transformation method is used to determine the mapping equation Hom: Among them, (x, y, z) and (x ′ ,y ′ ,z ′ ) are the normalized feature points corresponding to optical and SAR, respectively, let z = z ′ =1, then we get the mapping equation: Among them, h1 to h9 are the 9 degree of freedom parameters of the mapping equation Hom.

2. The method for integrated segmentation of high-resolution optical and SAR images for geographic objects according to claim 1, characterized in that: In step S2, the optical image is first segmented, and then corresponding marking points are extracted for each object. The specific steps are as follows: S21, the software eCognition was used to complete the multi-scale image segmentation, and the visual interpretation method was used to determine the segmentation parameters (Scale, Shape, Compactness) of the two data sets, where Scale is the scale parameter, Shape is the shape factor, and Compactness is the compactness factor; S22, let the object set extracted by image segmentation be R = {R1, R2, ..., R j }, j = 1, 2, ..., m, where m is the total number of objects segmented; for each object R j , the coordinates of the marking points are as follows: Among them, x d ,y d are the horizontal and vertical coordinates of the pixel points in the object, and s is the total number of pixels; The coordinates of the obtained marking point are marked as k j (X j , Y j ), traverse all objects and obtain the optical image object marker point set K = {k1, k2, ..., k j }.

3. The method for integrated segmentation of high-resolution optical and SAR images for geographic objects according to claim 2, characterized in that: In step S3, the specific steps of obtaining the SAR image matching object set are as follows: S31, project the optical image object marker point set K according to the mapping equation Hom to obtain the initial seed point set S in the SAR image = {s1, s2, ..., s j }, where s j For j (x j ,y j ); When mapped to the seed point s of the SAR image j (x j ,y j ) is outside the SAR image boundary, then calculate point s i The Euclidean distance from the midpoint of the SAR image is expressed as the Euclidean distance ρ ′ The smallest point t j (x j ′ ,y j ′ ) instead of point s j (x j ,y j ) as the seed point; x j ,y j are the horizontal and vertical coordinates of the initial seed point in the SAR image, x j ′ ,y j ′ is the horizontal and vertical coordinates of the adaptive seed point, then: The seed point set T in the adaptive SAR image is obtained as follows: j }, where t j t j (x j ′ ,y j ′ ); S32, according to the seed point set T, region growth is performed for each seed point at the same time; for any seed point, if the pixel value of a certain point in the eight-neighborhood region satisfies the following formula: Then the point is judged to meet the growth conditions and added to the region; Among them, (X, Y) is the coordinate of the midpoint of the eight-neighborhood of the seed point, t j (x j ′ ,y j ′ ) is the seed point coordinate, Img is the pixel matrix of the SAR image; Calculate the seed point t j (x j ′ ,y j ′ The difference between the pixel value at the eight neighborhood points and the seed point pixel, all the differences are added and averaged to get the value as the threshold q j , the expression is as follows: Among them, u, v, and w are all independent variables. After the region growth is completed, the SAR image object set C corresponding to the optical image object set R is obtained. j }, and finally a unified set of optical-SAR matching objects is obtained.

Images