Multi-source vector grating big data city remote sensing image fusion registration method

Abstract

According to the multi-source vector grid big data city remote sensing image fusion registration method, firstly, on a high-resolution remote sensing image, an object feature analysis method is adopted to extract ground features, and proper parameters are set for specific images and interested target ground features to optimize the extraction effect; and 2, extraction of a ground object target: not only is inherent characteristics of the object considered, but also comprehensive analysis is carried out by combining context and topology of the object and global characteristics of an image, and for different ground objects, a method of combining DSM data is adopted, and corresponding appropriate characteristic combination extraction is selected. And 3, searching homonymous elements between the extracted result and the existing vector data, matching the vector elements by adopting an approximation degree measurement model integrating position, shape and area characteristics, and performing image registration by utilizing the homonymous elements obtained by matching, thereby realizing accurate and efficient fusion registration of the multi-source vector grating big data urban remote sensing image.

Description

Technical Field

[0001] The present application relates to a big data remote sensing image registration method, and in particular to a multi-source vector-grating big data urban remote sensing image fusion registration method, belonging to the technical field of multi-source remote sensing image registration. Background Art

[0002] High-resolution remote sensing images are increasingly used in production, such as change detection and data update. The demand for spatial data integration and fusion is also increasing. However, it is difficult to achieve good results using traditional pixel-based image analysis methods on high-resolution remote sensing images. The traditional presentation of remote sensing image processing results is still raster data, while GIS processing and analysis mostly use vector data. The existing technology can obtain a variety of images from the currently developed remote sensing satellites, but lacks an efficient means to obtain the information required for production from these images. Therefore, there is an obvious efficiency gap between the result data obtained from the remote sensing images and the data used in GIS production. Fundamentally, compared with ordinary images, remote sensing images, especially high-resolution remote sensing images, contain more mottled information. The information extraction method using ordinary images cannot achieve good results. At the same time, remote sensing images also have the problem of heterogeneous objects in the same spectrum. Therefore, the most commonly used method is still the traditional manual visual interpretation, which obviously cannot meet the needs of rapid acquisition of massive data and product updates.

[0003] In addition, one of the problems currently faced by many GIS application projects is how to integrate data from different sources. In practice, different organizations and departments usually collect spatial data of the same area multiple times according to different needs. These spatial data are all describing the same objective entity and should be identical. However, due to differences in the equipment, measurement methods, collection time periods, mapping methods and other work processes, there are more or less differences between the data. Therefore, in order to obtain a fused data that combines the advantages of various data, it is necessary to integrate the above data. The integration here does not simply pile up data from different sources, but eliminates the spatial differences of these heterogeneous data, so that these data can be effectively integrated. Existing technologies cannot effectively solve these two problems.

[0004] The registration of vector and raster images is the basis of data fusion and integration, and is also the cornerstone of data change detection, automatic update and quality assessment. The registration between raster data is based on the grayscale of the image. Through the sliding window, the grayscale is used as the standard to find the same-name points, and then the image is corrected by these same-name points. The registration of vector and raster adopts the matching method, that is, first extract the features of the ground objects, and then find the same-name entities through the matching between vectors to complete the registration.

[0005] The problems that need to be solved in the prior art multi-source vector-grating remote sensing image registration and the key technical difficulties of this application include:

[0006] (1) The existing pixel-based image analysis methods are difficult to achieve good results on high-resolution remote sensing images. The traditional remote sensing image processing results are still presented in the form of raster data, while GIS processing and analysis mostly use vector data. The existing technology can obtain a variety of images from the currently developed remote sensing satellites, but lacks an efficient means to obtain the information required for production from these images. Therefore, there is a clear efficiency gap between the result data obtained from the processed remote sensing images and the data used in GIS production. Since the information contained in remote sensing images, especially high-resolution remote sensing images, is more mottled, the information extraction method using ordinary images cannot achieve good results. At the same time, remote sensing images also have the problem of different objects in the same spectrum. Therefore, the traditional manual visual interpretation is still the most commonly used method, which cannot meet the needs of rapid acquisition of massive data and product updates.

[0007] (2) One of the problems currently faced by many GIS application projects is how to integrate data from different sources. In practice, different organizations and departments usually collect spatial data of the same region multiple times according to different needs. Since these spatial data are all describing the same objective entity, they should be identical. However, due to differences in the equipment, measurement methods, collection time periods, mapping methods and other workflows, there are more or less differences between the data. Therefore, in order to obtain a fused data that combines the advantages of various data, it is necessary to integrate the above data. The integration here does not mean simply piling up data from different sources, but eliminating the spatial differences between these heterogeneous data so that these data can be effectively integrated. However, existing technologies cannot effectively solve this problem.

[0008] (3) The contradiction between remote sensing and GIS is the contradiction between the low efficiency of extracting the data required for production from remote sensing images and the high efficiency of GIS in processing data. The low efficiency of the integration and utilization of existing vector data is also a current problem. The feature analysis of existing technologies cannot make image processing and GIS more integrated. There is still a wall between the processing of remote sensing images and the analysis and processing of GIS. There is a lack of channels between the two, and the processing flow cannot be combined and integrated into one. The existing technology lacks image segmentation, object feature analysis, and vector data matching technologies for remote sensing image fusion and registration. It is impossible to effectively integrate various spatial data, and it is impossible to realize map change detection, automatic update, and quality assessment. The lack of a comprehensive approximation matching model and a control point automatic extraction method makes it impossible to realize the registration of vector data and image data. Summary of the invention

[0009] This application aims at the problem of registration of existing vector data and high-resolution remote sensing images. Based on the image segmentation method and the features of the image object to extract the image target, a vector matching method is designed, a comprehensive approximation matching model and a control point automatic extraction method are established to achieve the registration of vector data and image data. Firstly, an algorithm and process for image segmentation are proposed, and then the segmentation results are enabled to express the information of the objects of interest by setting appropriate parameters. Based on the feature analysis of the image object, the object extraction effect is optimized by combining appropriate features. For the matching of planar vector elements, a comprehensive approximation measurement model is used to match the elements. A control point automatic extraction method based on boundary network feature points is established to extract the control points required for registration on the successfully matched vector entities, and finally the accurate and efficient registration of vector data and remote sensing data is achieved.

[0010] In order to achieve the above technical effects, the technical solutions adopted in this application are as follows:

[0011] Multi-source vector-raster big data urban remote sensing image fusion registration method, through the ground object extraction, vector matching and image registration method, register vector data and raster data, realize the integration, fusion and update of multi-source data: 1) Extract image target based on object feature analysis: analyze and extract ground objects based on feature network evolution multi-scale segmentation, establish feature network evolution multi-scale segmentation algorithm and its parameter flow, take advantage of the high-resolution image target evolution characteristics in the segmentation process, make the segmentation result object more in line with the ground object extraction requirements; 2) Vector matching and image registration: based on the inherent characteristics of the ground object extraction object, the topological characteristics between objects and the global characteristics, construct a method for extracting urban buildings combined with DSM auxiliary data, and comprehensively utilize the height, spectrum, shape and contextual characteristics of urban buildings for extraction; establish a comprehensive proximity measurement model, while considering the location, shape and area information of the elements, based on the matching results, use the boundary network feature point method to extract control points to achieve image registration;

[0012] S1-Image segmentation method: adopt an object-based image analysis method, in which image segmentation is the first step of object feature analysis. The algorithm and process of image segmentation are established to generate a series of image objects. The extraction of ground objects is based on these basic image objects. By setting appropriate parameter factors, the segmentation results can express the information of the ground objects of interest;

[0013] S2-Image object feature analysis: The extraction of interesting objects is based on the unique features of this type of objects. Not only the inherent features of the object itself are considered, but also the context, topology and global features of the image are comprehensively analyzed. For different objects, the appropriate feature combination is selected for extraction;

[0014] S3-Vector matching: The elements extracted from the remote sensing image are matched with the existing vector elements to obtain the elements with the same name in the image and vector data, and the comprehensive proximity measurement model is used to match the planar vector elements;

[0015] S4-Automatic extraction of control points: Establish an automatic extraction method for control points based on boundary network feature points, extract the control points required for registration on the successfully matched vector entities, and finally realize the registration of vector data and remote sensing data.

[0016] Preferably, feature network evolution multi-scale segmentation: each time pixels are merged, the heterogeneity of the two objects before and after the merge is calculated, and the merge is performed according to the rule of minimum heterogeneity change. If the minimum growth exceeds the critical value, the merge is terminated. At the same time, a local optimal strategy is adopted in the order of merging objects. For any object A, an object B is found among the objects adjacent to it to meet the minimum heterogeneity increment rule; and for this object B, according to the same conditions, an object C is found to merge with B to cause the smallest heterogeneity change. If C=A, it indicates that the local optimal merging rule is met at this time. Otherwise, a reassignment operation is performed, and A=B, B=C is set. The above operation is repeated until a locally optimal pair of objects is found for merging. The spectral and shape features are comprehensively evaluated, and the two features are combined to evaluate the heterogeneity.

[0017] Preferably, the image evolution segmentation factor and process are:

[0018] A-Scale factor selection rules: A1- Consider the features of the actual target to be extracted, and generate multiple different segmentation extraction layers through multiple segmentation and setting different parameters. The most important basis for determining the parameter size is the size of the object; A2- Under the premise of ensuring that different types of image areas or certain target objects can be distinguished, the selection of the segmentation scale factor should try to make the resulting object size as large as possible, that is, the segmentation scale factor is large;

[0019] B-Spectral parameter setting rule: Based on the optimal scale factor rule, the spectral factor is given priority among the spectral factor and the shape factor to generate the segmentation object that best meets the spectral characteristics of the image;

[0020] C-Shape parameter setting rule: Based on the AB rule, the shape smoothness weight is given priority among the smoothness factor and the compactness factor;

[0021] Preferably, the multi-scale image segmentation algorithm process with feature network evolution as the core is as follows:

[0022] Step 1: Input the original remote sensing image;

[0023] Step 2: Set the segmentation parameters, including the segmentation scale factor s, band information weight and heterogeneity factor;

[0024] Step 3: Start region growing merging. According to the rule of minimum heterogeneity, the image object segmented last time is taken as the starting point. The first merging starts with a single pixel and calculates the heterogeneity h of adjacent objects after merging.

[0025] Step 4: Determine whether to terminate the segmentation based on the set scale factor s. If the condition is met, stop the segmentation and execute step 5. Otherwise, return to step 3 for the next segmentation.

[0026] Step 5: Terminate image segmentation and output the results.

[0027] Preferably, image target evolution features: the features shown on the image object after ground object segmentation are analyzed, and the features of the image object are divided into three categories:

[0028] (1) Intrinsic characteristics of objects: These are the most basic information of objects in images, reflecting the most intuitive characteristics of objects, including spectral characteristics, shape characteristics, texture characteristics, and various thematic characteristics of image objects;

[0029] (2) Topological features between objects: reflects the spatial positional relationship between different image objects, including the orientation relationship, adjacent relationship, relationship between objects at a certain distance, and relationship between inclusion and being included;

[0030] (3) Global features: describe the overall features of all objects or certain target objects in the image, including the number of image layers, resolution, number of objects, etc.

[0031] Preferably, vector element remote sensing image matching:

[0032] (1) The data to be matched have the same format and file type. Data of different file types should be converted first so that the converted data file formats are consistent;

[0033] (2) From the perspective of matching the same-name features in vector data, the data to be matched should be from the same region, or from overlapping bordering areas;

[0034] (3) The scales of the vector maps to be matched should be the same or similar, and the scales of the matching data should not differ too much.

[0035] (4) Based on the time phase problem, the time gap between the matching vector data should not be too large;

[0036] (5) Before matching, the overall system differences between the projection system and coordinate system of the vector data are eliminated;

[0037] (6) The vector elements involved in the matching should be consistent in semantics and geometric types;

[0038] (7) The topological relationship of all the vector data involved in the matching should be correctly established, and measures should be taken to eliminate the irregular topological relationship of pseudo nodes and hanging lines;

[0039] Vector data that meets the above conditions can be matched, and the evaluation indicators for judging whether the elements match include position similarity, topological structure similarity, orientation relationship similarity, shape similarity, and attribute similarity.

[0040] Preferably, the surface element matching: the surface element matching is performed by comprehensively analyzing the position, shape and area characteristics, and the matching relationship between the surface elements is comprehensively evaluated. It is assumed that the surface elements with matching are A and B respectively;

[0041] First, for the measurement of position features, the measured position features are calculated by formula 1:

[0042]

[0043] Among them, (x A ,y A ) and (x B ,y B ) are the geometric center points of entities A and B, d max (A,B) is the maximum distance between two points on any boundary of entities A and B;

[0044] Second, the quantification of shape features is measured by Fourier descriptor based on curvature radius. The specific formula is as follows:

[0045]

[0046] Among them, the coordinate change of a moving point P(l) on the boundary of a polygon is mapped to the complex plane as x(l)+jy(l), j is a parameter, and the coordinate of P(l) is expressed in complex form. It is a periodic function with the perimeter of the shape boundary as the period. The periodic function is expanded into the Fourier series form;

[0047] A surface feature is composed of N points as its boundary. In this point set, a point is randomly selected as the starting point, denoted as (x0, y0), and a point sequence is obtained by traversing the boundary in a counterclockwise direction, denoted as C = {(x0, y0), (x1, y1), ..., (x N-1 ,y N-1 )};

[0048] Assume (x l ,y l ) is the lth point in C, and maps it to the complex plane:

[0049]

[0050] By inverse transformation we get:

[0051]

[0052] The descriptor is normalized to ensure that the calculation of shape approximation is independent of the scaling, rotation and offset of the shape. The normalized descriptor is defined as follows:

[0053]

[0054] M is the number of terms in the Fourier series expansion. Since the low-frequency information reflecting the shape characteristics is concentrated in the first few terms of the Fourier series, the value of M should not be too large. In this application, it is 12.

[0055] Third, for the comparison of area characteristics, the following formula is used for judgment:

[0056]

[0057] Where Area A and Area B are the areas of surface elements A and B respectively;

[0058] Finally, the above three characteristics are combined in a weighted manner to obtain the following comprehensive judgment formula:

[0059]

[0060] Where i = 1, 2, 3, ω i is the weight coefficient, D i is the difference between the features of the surface elements A and B (i.e. the three features of position, shape and area mentioned above), and the difference of each feature is normalized to obtain D i ∈[0,1], so D(A,B)∈[0,1], the difference degree defines the approximation as:

[0061] Similarity=1-D(A,B) Formula 8

[0062] A comprehensive approximation measurement model is obtained to perform surface element matching.

[0063] Preferably, the control points are automatically extracted based on the boundary network feature points: the registration of vector data and images requires obtaining the control points first, and the boundary network feature point method is used to extract the control points on the elements with the same name that are successfully matched;

[0064] Suppose any pair of matching entities obtained after image matching is A and B, where A is the entity in the vectorized data and B is the entity in the original vector data, and their boundaries are E A and E B, the perimeters are l A and 1 B , the geometric center points are O A and O B , the boundary control point selection method is: for the entity boundary E A or E B , choose any point on the boundary as the starting point, denoted as p O , normalize the boundary perimeter to 1. If n control points are selected, the entire boundary is divided into n sub-segments every 1 / n perimeter distance in a clockwise (or counterclockwise) direction, and a feature point is selected in each sub-segment as a control point. In each sub-segment, the feature point is selected by the following method: the matching entity pairs have shape similarity, and the radius at each point on the boundary, that is, the distance from a point on the boundary line to the center, is used, recorded as r. By comparing the radius of all boundary points on the sub-segment, the boundary point with the largest radius is taken as the feature point. If the maximum value is not unique, the first maximum value is selected according to the sub-segment division direction. For the selection of the starting point, the control point selection method in the sub-segment is applied. By comparing the radius of all points on the entity boundary, the point with the maximum (or minimum) radius is taken as the starting point.

[0065] Preferably, the registration of vector data and images: establishes a correspondence between two images, determines corresponding geometric transformation parameters, and geometrically corrects one image using the vector data as a reference;

[0066] After completing the matching of the surface features, according to the matching results, appropriate control points are extracted on the corresponding feature objects with the same name for image geometric correction. The polynomial correction model is used for geometric correction. Assuming that (x, y) is the coordinate before geometric correction and (u, v) is the coordinate after geometric correction, the polynomial correction model is expressed as follows:

[0067]

[0068] Among them, a ij and b ij is the polynomial coefficient, N is the polynomial degree, and the above two polynomials are expanded to obtain:

[0069] x=a0+a1u+a2v+a3u 2 +a4uv+a5v 2 +…Formula 11

[0070] y=b0+b1u+b2v+b3u 2 +b4uv+b5v 2 +…Formula 12

[0071] Let the polynomial degree N = 1, which is the affine transformation. After the degree is selected, a certain number of control points are selected, and the coefficients of the polynomial are calculated using the least squares method using the coordinates of these control points to obtain the corrected coordinates. For the evaluation of error and accuracy, the root mean square error RMS is used Error To measure the accuracy of each control point, RMS Error The calculation formula is as follows:

[0072]

[0073] Among them, x and y are the coordinates of the selected control point, and and are the coordinates of the control points calculated by the polynomial, that is, the estimated coordinates of the control points.

[0074] Compared with the prior art, the innovations and advantages of this application are:

[0075] (1) This application establishes a vector and raster image registration method based on object feature analysis. First, on high-resolution remote sensing images, the types of objects are mottled and complex, and the information that a single pixel can express is very limited. The existing pixel-based image analysis method can no longer meet current needs. This application uses an object feature analysis method to extract objects. The basic unit of object feature analysis, the image object, is a set of adjacent similar pixels, which is the product of image segmentation. For specific images and objects of interest, appropriate parameters are set to optimize the extraction effect; second, the extraction of objects of interest is based on the unique features of this type of object. The extraction of object targets not only considers the inherent characteristics of the object itself, but also combines the context, topology and global features of the object for comprehensive analysis. For different objects, select the corresponding appropriate feature combination for extraction. The method of combining DSM data is used to extract urban building objects. Third, we search for elements with the same name between the extracted results and the existing vector data, use a proximity measurement model that integrates position, shape and area features to match the vector elements, and use the matched elements with the same name to perform image registration, thus achieving accurate and efficient fusion registration of multi-source vector-raster big data urban remote sensing images.

[0076] (2) The object feature analysis of the present application extracts the matching of the ground objects and vector data, and finally uses the matching results for image registration. Establishing object feature analysis is conducive to making image processing and GIS more integrated. At this stage, there is still a wall between the processing of remote sensing images and the analysis and processing of GIS. Object feature analysis is a tool to break this wall, establish a channel between the two, and integrate the processing flow into one. Designing image segmentation as the basis of object feature analysis can not only be applied to the analysis of remote sensing images, but also serve as a reference for the segmentation of images in other fields. Compared with the traditional pixel-based analysis method, the establishment of object feature analysis not only reflects the use of spectral information, but also highlights the analysis of the spectral calculation information inside the object (such as mean, variance, maximum, minimum, etc.). In addition, object feature analysis can reflect some topological relationships (such as adjacent, contained, etc.), and better reflect the spatial information of various ground objects on the image. On this basis, ground object extraction and classification operations will also have higher accuracy. The vector data matching technology of the present application can effectively integrate various spatial data to achieve map change detection, automatic update and quality assessment.

[0077] (3) This application aims at the problem of registration of existing vector data and high-resolution remote sensing images. It uses image segmentation methods and image object features to extract image targets, designs vector matching methods, establishes a comprehensive approximation matching model, and automatically extracts control points to achieve registration of vector data and image data. First, an algorithm and process for image segmentation are proposed, and then appropriate parameters are set so that the segmentation results can express the information of the objects of interest. Based on the analysis of image object features, the object extraction effect is optimized by combining appropriate features. For the matching of planar vector elements, a comprehensive approximation measurement model is used to match the elements. A method for automatically extracting control points based on boundary network feature points is established to extract the control points required for registration on the successfully matched vector entities, ultimately achieving accurate and efficient registration of vector data and remote sensing data. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] Figure 1 It is the flow chart of the multi-scale image segmentation algorithm based on feature network evolution.

[0079] Figure 2 It is a schematic diagram of automatic extraction of control points based on boundary network feature points.

[0080] Figure 3 It is a comparison chart of experimental data of this application.

[0081] Figure 4 It is a schematic diagram of the building surface vector data of the existing data in the area.

[0082] Figure 5 This is a schematic diagram of the smoothed DSM and slope data.

[0083] Figure 6 This is a schematic diagram of the experimental image segmentation results of this application.

[0084] Figure 7 This is a schematic diagram of the results of urban building extraction in the experiment of this application.

[0085] Figure 8 This is a schematic diagram of the experimental vector matching and image matching results of this application.

[0086] Fig. 9 This is a detailed list of the experimental vector matching results of this application. DETAILED DESCRIPTION

[0087] The following, in conjunction with the accompanying drawings, further describes the technical solution of the multi-source vector-raster big data urban remote sensing image fusion and registration method provided in the present application, so that technicians in this field can better understand the present application and implement it.

[0088] Due to differences in data collection and processing, there is a certain gap between remote sensing data from different sources, making it difficult to combine the advantages and characteristics of each data for comprehensive production analysis. At the same time, the efficiency and quality of remote sensing image acquisition are increasing, but the corresponding information extraction and analysis capabilities are far from keeping up with the pace. In addition, the demand for spatial data integration and fusion is also increasing significantly. Therefore, this application uses ground feature extraction, vector matching and image registration methods to register vector data and raster data to achieve multi-source data integration, fusion and update.

[0089] 1. Use object-based image analysis methods (object feature analysis) to analyze and extract objects, establish a feature network evolutionary multi-scale segmentation algorithm and its parameter flow, and make full use of the advantages of high-resolution images in the segmentation process to make the segmentation result objects more in line with the requirements of object extraction.

[0090] 2. Based on the inherent features of objects extracted from ground objects, the topological features between objects, and the global features, a method for extracting urban buildings combined with DSM auxiliary data is constructed, and the height, spectrum, shape and contextual features of urban buildings are comprehensively utilized for extraction.

[0091] 3. Vector matching and image registration, establish a comprehensive proximity measurement model, consider the position, shape and area information of the elements at the same time, use the matching results as the basis, use the boundary network feature point method to extract control points, and realize image registration.

[0092] 1. Extracting image targets based on object feature analysis

[0093] (I) Feature Network Evolution Multi-Scale Segmentation

[0094] Each time pixels are merged, the heterogeneity of the two objects before and after the merge is calculated, and the merge is performed according to the rule of minimum heterogeneity change. If the minimum increase exceeds the critical value, the merge is terminated. At the same time, a local optimal strategy is adopted in the order of merging objects. For any object A, an object B is found among the objects adjacent to it to meet the minimum heterogeneity increment rule; and for this object B, according to the same conditions, an object C is found to have the smallest heterogeneity change caused by merging with B. If C = A, it means that the local optimal merge rule is met at this time. Otherwise, a reassignment operation is performed, and A = B, B = C is set. The above operation is repeated until a local optimal pair of objects is found for merging. The spectral and shape features are comprehensively evaluated, and the two features are combined to evaluate heterogeneity.

[0095] It is known that the heterogeneity values ​​of the two adjacent objects before the initial merging of the segmentation are h1 and h2 respectively. Assume that the heterogeneity value of the new object generated after the two objects are merged is h m , then the difference in heterogeneity caused by this merger is h diff , which is the merging cost. The final merging rule is to minimize the merging cost generated by the merging and meet the requirements of the quantification of the segmentation results, that is, the mean heterogeneity of the image objects is minimized.

[0096] (II) Image evolution segmentation factors and processes

[0097] A-Scale factor selection rules: A1- Consider the features of the actual target to be extracted, and generate multiple different segmentation extraction layers through multiple segmentation and setting different parameters. The most important basis for determining the parameter size is the size of the object (i.e., the number of pixels or area); A2- Under the premise of ensuring that different types of image areas or certain target objects can be distinguished, the selection of the segmentation scale factor should make the resulting object size as large as possible, that is, the segmentation scale factor is large (maximum area rule);

[0098] B-Spectral parameter setting rule: Based on the optimal scale factor rule, the spectral factor is given priority among the spectral factor and the shape factor to generate the segmentation object that best meets the spectral characteristics of the image;

[0099] C-Shape parameter setting rule: Based on the AB rule, the shape smoothness weight is given priority among the smoothness factor and the compactness factor;

[0100] The process of the multi-scale image segmentation algorithm with feature network evolution as the core is as follows:

[0101] Step 1: Input the original remote sensing image;

[0102] Step 2: Set the segmentation parameters, including the segmentation scale factor s, band information weight and heterogeneity factor;

[0103] Step 3: Start region growing merging. According to the rule of minimum heterogeneity, the image object segmented last time is taken as the starting point. The first merging starts with a single pixel and calculates the heterogeneity h of adjacent objects after merging.

[0104] Step 4: Determine whether to terminate the segmentation based on the set scale factor s. If the condition is met, stop the segmentation and execute step 5. Otherwise, return to step 3 for the next segmentation.

[0105] Step 5: Terminate image segmentation and output the results. Figure 1 shown.

[0106] 3.3 Image target evolution characteristics

[0107] The features of the image objects after ground object segmentation are analyzed and divided into three categories:

[0108] (1) Intrinsic characteristics of objects: These are the most basic information of objects in images, reflecting the most intuitive characteristics of objects, including spectral characteristics, shape characteristics, texture characteristics, and various thematic characteristics of image objects;

[0109] (2) Topological features between objects: reflects the spatial positional relationship between different image objects, including the orientation relationship, adjacent relationship, relationship between objects at a certain distance, and relationship between inclusion and being included;

[0110] (3) Global features: describe the overall features of all objects or certain target objects in the image, including the number of image layers, resolution, number of objects, etc.

[0111] 2. Vector Matching and Image Registration

[0112] The object feature analysis method is used to extract urban buildings as vector objects to be matched, and a surface vector data matching method is adopted. At the same time, a boundary network feature point method is used to extract control points for image registration.

[0113] 1. Vector element remote sensing image matching

[0114] (1) The data to be matched have the same format (i.e., vector data) and the same file type. Data of different file types should be converted first so that the converted data file formats are consistent;

[0115] (2) From the perspective of matching the same-name features in vector data, the data to be matched should be from the same region, or from overlapping bordering areas;

[0116] (3) The scales of the vector maps to be matched should be the same or similar, and the scales of the matching data should not differ too much.

[0117] (4) Based on the time phase problem, the time gap between the matching vector data should not be too large;

[0118] (5) Before matching, the overall system differences between the projection system and coordinate system of the vector data are eliminated;

[0119] (6) The vector elements involved in the matching should be consistent in semantics and geometric types;

[0120] (7) The topological relationship of all the vector data involved in the matching should be correctly established, and measures should be taken to eliminate the irregular topological relationship of pseudo nodes and hanging lines;

[0121] Vector data that meets the above conditions can be matched, and the evaluation indicators for judging whether the elements match include position similarity, topological structure similarity, orientation relationship similarity, shape similarity, and attribute similarity.

[0122] (II) Area feature matching

[0123] The location, shape and area characteristics are comprehensively analyzed to match the surface elements, and the matching relationship between the surface elements is comprehensively evaluated. The following assumes that the matching surface elements are A and B respectively;

[0124] First, for the measurement of position features, the measured position features are calculated by formula 1:

[0125]

[0126] Among them, (x A ,y A ) and (x B ,y B ) are the geometric center points of entities A and B, d max (A,B) is the maximum distance between two points on any boundary of entities A and B;

[0127] Second, the quantification of shape features is measured by Fourier descriptor based on curvature radius. The specific formula is as follows:

[0128]

[0129] Among them, the coordinate change of a moving point P(l) on the boundary of a polygon is mapped to the complex plane as x(l)+jy(l), j is a parameter, and the coordinate of P(l) is expressed in complex form. It is a periodic function with the perimeter of the shape boundary as the period. The periodic function is expanded into the Fourier series form;

[0130] A surface feature is composed of N points as its boundary. In this point set, a point is randomly selected as the starting point, denoted as (x0, y0). A circle is traversed counterclockwise on the boundary to obtain a point sequence, denoted as C = {(x0, y0), (x1, y1), ..., (x N-1 ,y N-1 )};

[0131] Assume (x l ,y l ) is the lth point in C, and maps it to the complex plane:

[0132]

[0133] By inverse transformation we get:

[0134]

[0135] The descriptor is normalized to ensure that the calculation of shape approximation is independent of the scaling, rotation and offset of the shape. The normalized descriptor is defined as follows:

[0136]

[0137] M is the number of terms in the Fourier series expansion. Since the low-frequency information reflecting the shape characteristics is concentrated in the first few terms of the Fourier series, the value of M should not be too large. In this application, it is 12.

[0138] Third, for the comparison of area characteristics, the following formula is used for judgment:

[0139]

[0140] Where Area A and Area B are the areas of surface elements A and B respectively;

[0141] Finally, the above three characteristics are combined in a weighted manner to obtain the following comprehensive judgment formula:

[0142]

[0143] Where i = 1, 2, 3, ω i is the weight coefficient, D i is the difference between the features of the surface elements A and B (i.e. the three features of position, shape and area mentioned above), and the difference of each feature is normalized to obtain D i ∈[0,1], so D(A,B)∈[0,1], the difference degree defines the approximation as:

[0144] Similarity=1-D(A,B) Formula 8

[0145] A comprehensive approximation measurement model is obtained to perform surface element matching.

[0146] (III) Automatic extraction of control points based on boundary network feature points

[0147] The registration of vector data and images requires obtaining control points first, and the boundary network feature point method is used to extract control points on the successfully matched features with the same name;

[0148] Suppose any pair of matching entities obtained after image matching is A and B, where A is the entity in the vectorized data and B is the entity in the original vector data, and their boundaries are E A and E B , the perimeters are l A and 1 B , the geometric center points are O A and O B , the boundary control point selection method is: for the entity boundary E A or E B , choose any point on the boundary as the starting point, denoted as p O , normalize the boundary perimeter to 1. If n control points are selected, the entire boundary is divided into n sub-segments every 1 / n perimeter distance in a clockwise (or counterclockwise) direction, and a feature point is selected in each sub-segment as a control point. In each sub-segment, the feature point is selected by the following method: the matching entity pairs have shape similarity, and the radius at each point on the boundary, that is, the distance from a point on the boundary line to the center, is used, recorded as r. By comparing the radius of all boundary points on the sub-segment, the boundary point with the maximum (or minimum) radius is taken as the feature point. If the maximum (or minimum) value is not unique, the first maximum value is selected according to the sub-segment division direction. For the selection of the starting point, the control point selection method in the sub-segment is applied. By comparing the radius of all points on the entity boundary, the point with the maximum (or minimum) radius is taken as the starting point. The schematic diagram is shown as follows: Figure 2 shown.

[0149] 4. Registration of vector data and images

[0150] Establish the correspondence between two images, determine the corresponding geometric transformation parameters, and use the vector data as a reference to geometrically correct an image;

[0151] After completing the matching of the surface features, according to the matching results, appropriate control points are extracted on the corresponding feature objects with the same name for image geometric correction. The polynomial correction model is used for geometric correction. Assuming that (x, y) is the coordinate before geometric correction and (u, v) is the coordinate after geometric correction, the polynomial correction model is expressed as follows:

[0152]

[0153] Among them, a ij and b ij is the polynomial coefficient, N is the polynomial degree, and the above two polynomials are expanded to obtain:

[0154] x=a0+a1u+a2v+a3u 2 +a4uv+a5v 2 +…Formula 11

[0155] y=b0+b1u+b2v+b3u 2 +b4uv+b5v 2 +…Formula 12

[0156] Let the polynomial degree N = 1, which is the affine transformation. After the degree is selected, a certain number of control points are selected, and the coefficients of the polynomial are calculated using the least squares method using the coordinates of these control points to obtain the corrected coordinates. For the evaluation of error and accuracy, the root mean square error RMS is used Error To measure the accuracy of each control point, RMS Error The calculation formula is as follows:

[0157]

[0158] Among them, x and y are the coordinates of the selected control point, and and are the coordinates of the control points calculated by the polynomial, that is, the estimated coordinates of the control points.

[0159] 3. Experiment and Analysis

[0160] According to the method of the present application, appropriate high-resolution remote sensing image data and existing vector data of the corresponding area are selected, and urban building objects are taken as objects. First, the object feature analysis method of the present application is used to extract urban buildings in the remote sensing image to obtain a series of surface vector elements. The extraction result is then used as a reference element, and the existing urban building vector data is used as the element to be matched. The comprehensive approximation measurement model of the present application is used to match the surface vector elements of urban buildings. Finally, based on the matching results, the boundary network feature point method is used to extract control points on the successfully matched urban buildings for image geometry correction, thereby realizing the registration of vector and raster images.

[0161] (I) Experimental data

[0162] The data used is a high-resolution panchromatic remote sensing image with a resolution of 0.5 m, such as Figure 3 As shown in the left picture, the size is 2005pxl×2005pxl, and the existing vector data is the building surface vector data of the area, such as Figure 4 As shown in Figure 2, the DSM data of the same region is used as an important basis for extracting urban buildings. The resolution is 2.5m. Figure 3 As shown in the picture on the right.

[0163] (II) Urban building extraction results and analysis

[0164] First, the experimental image data is observed and analyzed, and it is found that the terrain of the area described by the image is relatively flat, without much undulation. For the house buildings to be extracted in this application, in terms of spectral characteristics, the spectral characteristics between different house buildings are significantly different, but the internal spectral characteristics of the top of the same house building are generally consistent, and this feature is used as a reference in the extraction process; in terms of shape characteristics, the contour boundaries of different house buildings are different in shape and have no obvious commonality, but in terms of size, compared with the trees, vehicles and other objects in the image, the house buildings are all shown to have a certain area, so this feature is also used as an important factor in the extraction process; at the same time, the most intuitive method to extract the building houses in the flat area is to use its height characteristics and combine DSM data for extraction. Directly using DSM data for extraction, a better extraction effect is obtained overall, but the boundary shape of some urban buildings cannot be clearly extracted, so further optimization is needed. Analyzing the boundaries of urban buildings, it is found that due to the large drop, the slope value of the boundary of urban buildings is large, and the slope data can be obtained by calculation from the DSM data, so this feature is used to first extract its boundaries, and then further extraction is performed.

[0165] Based on the above analysis, the slope data is first obtained. Since the resolution of DSM data is lower than that of image data, in order to make the slope data boundary smooth, the DSM is first smoothed by Gaussian filtering, and then the surface calculation is performed to obtain the slope data as shown in the figure. Figure 5 shown.

[0166] In order to extract boundary data, image segmentation is first performed on the slope data. The experiment found that after one segmentation, objects with larger slope values ​​can be segmented, while objects with larger slope values ​​are not segmented well, that is, objects with larger slope values ​​need to be segmented again into objects with medium slope values ​​and objects with high slope values. The results after two segmentations are as follows: Figure 6 As shown, the left picture is the result of the first segmentation, and the right picture is the result of continued segmentation based on the first segmentation.

[0167] Based on the image objects obtained by the above segmentation, the average slope of all objects is calculated, and the critical value is set to 50. The objects with a slope average greater than 50 are the steep boundaries to be extracted.

[0168] Before extracting urban buildings, in order to eliminate interference from other factors, we first extract the ground whose height is obviously lower than that of urban buildings. Using DSM height data, we calculate the maximum and minimum values ​​of the height, and then divide this interval into ten equal parts. We take a ten-point point close to the minimum value as the critical value, and objects below this critical value are all ground.

[0169] Use DSM height data to extract urban buildings. Since the ground objects have been extracted, the difference between the average height of the remaining objects and the ground objects is calculated to extract urban buildings. The critical value is set to 10. The objects whose average height difference with the average height of the ground objects is greater than 10 are urban buildings.

[0170] Now that we have obtained the boundary and the urban building object, we need to combine the two to obtain the urban building object with a good outline. The boundary object adjacent to the urban building object is selected as the object to be studied. The adjacent, i.e., shared boundary length is greater than 0 pixels. Since the objects on the roof of the same urban building have spectral similarity, in order to comprehensively consider the three bands of RGB, the spectral characteristic value is expressed as R+B+C. The spectral characteristics of the urban building object are used as the standard value. When the difference between the spectral characteristic value of the object and the standard value is less than the critical value (here set to 30), the object is re-extracted as an urban building object.

[0171] Using the ratio (i.e. ) Set the critical value to 0.36 to remove vegetation; use the relative boundary length and set the critical value to 0.5 to complete the urban buildings; use the area critical value method and set the critical value to 1000 to remove some small objects. The final extraction results are as follows Figure 7 As shown in the figure, the contour boundary is more consistent with the boundary characteristics of urban buildings, and the situations of missed extraction and wrong extraction have been effectively improved.

[0172] (III) Planar vector matching and image registration

[0173] The above urban building extraction results are used as the vector elements to be matched, and the existing urban building vector numbers are used as reference elements, such as Figure 8 As shown in the left figure. Now we use the comprehensive approximation measurement model for vector matching, where the shape approximation is measured by the Fourier shape descriptor based on the complex function of the curvature radius. The weights of the three features of position, shape and area are ω1=0.1, ω2=0.5, ω3=0.4 respectively. The critical value of the approximation for judging whether the match is finally made is 0.8. The matching model calculation and results are shown in Fig. 9 shown.

[0174] The matching results are shown as follows Figure 8As shown in the right figure, overall, the matching results are consistent with reality. The objects that have not been matched successfully are all incomplete urban buildings and some small urban building objects on the extracted image boundaries. There are no corresponding urban buildings in the existing vector data.

[0175] The image is then registered, and the boundary network feature point method is used to extract control points on the successfully matched urban building objects, and the above control points are used to perform image geometric correction.

[0176] The experiment uses remote sensing images and existing vectors as data, and combines the method of this application to conduct an experiment on the registration of image and vector data. Specifically, urban buildings are extracted by combining DSM data and vector results are exported, which are matched with existing vector data. Control points are extracted between successfully matched entities through the boundary network feature point method. Finally, these control points are used to realize the registration of remote sensing images and vector data, verifying the feasibility of the method of this application.

Claims

1. A multi-source vector-grating big data urban remote sensing image fusion registration method, characterized in that: Through the method of object extraction, vector matching and image registration, the vector data and raster data are registered to realize the integration, fusion and update of multi-source data: 1) Extract image targets based on object feature analysis: Analyze and extract objects based on feature network evolution multi-scale segmentation, establish feature network evolution multi-scale segmentation algorithm and its parameter flow, and take advantage of the evolutionary features of high-resolution image targets in the segmentation process, so that the segmentation result objects are more in line with the requirements of object extraction; 2) Vector matching and image registration: Based on the inherent features of objects extracted from ground objects, the topological features between objects, and the global features, a method for extracting urban buildings combined with DSM auxiliary data is constructed, and the height, spectrum, shape, and context features of urban buildings are comprehensively used for extraction; A comprehensive proximity measurement model is established, taking into account the position, shape and area information of the elements. Based on the matching results, the boundary network feature point method is used to extract control points to achieve image registration. S1-Image segmentation method: adopt an object-based image analysis method, in which image segmentation is the first step of object feature analysis. The algorithm and process of image segmentation are established to generate a series of image objects. The extraction of ground objects is based on these basic image objects. By setting appropriate parameter factors, the segmentation results can express the information of the ground objects of interest; S2-Image object feature analysis: The extraction of interesting objects is based on the unique features of this type of objects. Not only the inherent features of the object itself are considered, but also the context, topology and global features of the image are comprehensively analyzed. For different objects, the appropriate feature combination is selected for extraction; S3-Vector matching: The elements extracted from the remote sensing image are matched with the existing vector elements to obtain the elements with the same name in the image and vector data, and the comprehensive proximity measurement model is used to match the planar vector elements; S4-Automatic extraction of control points: Establish an automatic extraction method for control points based on boundary network feature points, extract the control points required for registration on the successfully matched vector entities, and finally realize the registration of vector data and remote sensing data.

2. According to the multi-source vector-grating big data urban remote sensing image fusion registration method of claim 1, it is characterized by: Feature network evolutionary multi-scale segmentation: Each time a pixel is merged, the heterogeneity of the two objects before and after the merge is calculated, and the merge is performed according to the rule of minimum heterogeneity change. If the minimum growth exceeds the critical value, the merge is terminated. At the same time, a local optimal strategy is adopted in the order of merging objects. For any object A, an object B is found among the objects adjacent to it to meet the minimum heterogeneity increment rule; and for this object B, according to the same conditions, an object C is found to merge with B to cause the smallest heterogeneity change. If C = A, it indicates that the local optimal merging rule is met at this time. Otherwise, a reassignment operation is performed, and A = B, B = C is set. The above operation is repeated until a locally optimal pair of objects is found for merging. The spectral and shape features are comprehensively evaluated, and the two features are combined to evaluate the heterogeneity.

3. According to the multi-source vector-raster big data urban remote sensing image fusion registration method of claim 1, it is characterized by: Image evolution segmentation factors and processes: A-Scale factor selection rules: A1- Consider the features of the actual target to be extracted, and generate multiple different segmentation extraction layers through multiple segmentation and setting different parameters. The most important basis for determining the parameter size is the size of the object; A2- Under the premise of ensuring that different types of image areas or certain target objects can be distinguished, the selection of the segmentation scale factor should try to make the resulting object size as large as possible, that is, the segmentation scale factor is large; B-spectral parameter setting rule: Based on the optimal scale factor rule, the spectral factor is given priority among the spectral factor and the shape factor to generate the segmentation object that best meets the spectral characteristics of the image; C-Shape parameter setting rule: Based on the AB rule, the shape smoothness weight is given priority among the smoothness factor and the compactness factor.

4. According to the multi-source vector-raster big data urban remote sensing image fusion registration method of claim 3, it is characterized in that: The process of the multi-scale image segmentation algorithm with feature network evolution as the core is as follows: Step 1: Input the original remote sensing image; Step 2: Set the segmentation parameters, including the segmentation scale factor s, band information weight and heterogeneity factor; Step 3: Start region growing merging. According to the rule of minimum heterogeneity, the image object segmented last time is taken as the starting point. The first merging starts with a single pixel and calculates the heterogeneity h of adjacent objects after merging. Step 4: Determine whether to terminate the segmentation based on the set scale factor s. If the condition is met, stop the segmentation and execute step 5. Otherwise, return to step 3 for the next segmentation. Step 5: Terminate image segmentation and output the results.

5. The multi-source vector-raster big data urban remote sensing image fusion registration method according to claim 1 is characterized in that: Image target evolution features: Analyze the features of the image objects after ground object segmentation and divide the features of the image objects into three categories: (1) Intrinsic characteristics of objects: These are the most basic information of objects in images, reflecting the most intuitive characteristics of objects, including spectral characteristics, shape characteristics, texture characteristics, and various thematic characteristics of image objects; (2) Topological features between objects: reflects the spatial positional relationship between different image objects, including the orientation relationship, adjacent relationship, relationship between objects at a certain distance, and relationship between inclusion and being included; (3) Global features: describe the overall features of all objects or certain target objects in the image, including the number of image layers, resolution, number of objects, etc.

6. The multi-source vector-raster big data urban remote sensing image fusion registration method according to claim 1 is characterized in that: Vector feature remote sensing image matching: (1) The data to be matched have the same format and file type. Data of different file types should be converted first so that the converted data file formats are consistent; (2) From the perspective of matching the same-name features in vector data, the data to be matched should be from the same region, or from overlapping bordering areas; (3) The scales of the vector maps to be matched should be the same or similar, and the scales of the matching data should not differ too much; (4) Based on the time phase problem, the time gap between the matching vector data should not be too large; (5) Before matching, the overall system differences between the projection system and coordinate system of the vector data are eliminated; (6) The vector elements involved in the matching should be consistent in semantics and geometric types; (7) The topological relationship of all the vector data involved in the matching should be correctly established, and measures should be taken to eliminate the irregular topological relationship of pseudo nodes and hanging lines; Vector data that meets the above conditions can be matched, and the evaluation indicators for judging whether the elements match include position similarity, topological structure similarity, orientation relationship similarity, shape similarity, and attribute similarity.

7. The multi-source vector-raster big data urban remote sensing image fusion registration method according to claim 1 is characterized in that: Area element matching: Comprehensively analyze the position, shape and area characteristics to match the area elements, and comprehensively evaluate the matching relationship between the area elements. The following assumes that the matching area elements are A and B respectively; First, for the measurement of position features, the measured position features are calculated by formula 1: Among them, (x A ,y A ) and (x B ,y B ) are the geometric center points of entities A and B, d max (A,B) is the maximum distance between two points on any boundary of entities A and B; Second, the quantification of shape features is measured by Fourier descriptor based on curvature radius. The specific formula is as follows: Among them, the coordinate change of a moving point P(l) on the boundary of a polygon is mapped to the complex plane as x(l)+jy(l), j is a parameter, and the coordinate of P(l) is expressed in complex form. It is a periodic function with the perimeter of the shape boundary as the period. The periodic function is expanded into the Fourier series form; A surface feature is composed of N points as its boundary. In this point set, a point is randomly selected as the starting point, denoted as (x0, y0), and a point sequence is obtained by traversing the boundary in a counterclockwise direction, denoted as C = {(x0, y0), (x1, y1), ..., (x N-1 ,y N-1 )}; Assume (x l ,y l ) is the lth point in C, and maps it to the complex plane: By inverse transformation we get: The descriptor is normalized to ensure that the calculation of shape approximation is independent of the scaling, rotation and offset of the shape. The normalized descriptor is defined as follows: M is the number of terms in the Fourier series expansion. Since the low-frequency information reflecting the shape characteristics is concentrated in the first few terms of the Fourier series, the value of M should not be too large. In this application, it is 12. Third, for the comparison of area characteristics, the following formula is used for judgment: Where Area A and Area B are the areas of surface elements A and B respectively; Finally, the above three characteristics are combined in a weighted manner to obtain the following comprehensive judgment formula: Where i = 1, 2, 3, ωi is the weight coefficient, D i is the difference between the features of the surface elements A and B (i.e. the three features of position, shape and area mentioned above), and the difference of each feature is normalized to obtain D i ∈[0,1], so D(A,B)∈[0,1], the difference degree defines the approximation as: Similarity=1-D(A,B) Formula 8 A comprehensive approximation measurement model is obtained to perform surface element matching.

8. The multi-source vector-raster big data urban remote sensing image fusion registration method according to claim 1 is characterized in that: Automatic extraction of control points based on boundary network feature points: The registration of vector data and images requires obtaining control points first, and the boundary network feature point method is used to extract control points on the successfully matched features with the same name; Suppose any pair of matching entities obtained after image matching is A and B, where A is the entity in the vectorized data and B is the entity in the original vector data, and their boundaries are E A and E B , the perimeters are l A and 1 B , the geometric center points are O A and O B , the boundary control point selection method is: for the entity boundary E A or E B , choose any point on the boundary as the starting point, denoted as p O , normalize the boundary perimeter to 1. If n control points are selected, the entire boundary is divided into n sub-segments in a clockwise (or counterclockwise) direction at every 1 / n perimeter distance. A feature point is selected in each sub-segment as a control point, wherein the feature point is selected in each sub-segment by the following method: the matching entity pairs have shape similarity, and the radius at each point on the boundary, that is, the distance from a point on the boundary line to the center, is used, denoted as r. By comparing the radius of all boundary points on the sub-segment, the boundary point with the largest radius is taken as the feature point. If the maximum value is not unique, the first maximum value is selected according to the sub-segment division direction. For the selection of the starting point, the control point selection method in the sub-segment is applied. By comparing the radius of all points on the entity boundary, the point with the maximum radius is taken as the starting point.

9. The multi-source vector-raster big data urban remote sensing image fusion registration method according to claim 1, characterized in that: Registration of vector data and images: establish the correspondence between two images, determine the corresponding geometric transformation parameters, and use the vector data as a reference to geometrically correct an image; After completing the matching of the surface features, according to the matching results, appropriate control points are extracted on the corresponding feature objects with the same name for image geometric correction. The polynomial correction model is used for geometric correction. Assuming that (x, y) is the coordinate before geometric correction and (u, v) is the coordinate after geometric correction, the polynomial correction model is expressed as follows: Among them, a ij and b ij is the polynomial coefficient, N is the polynomial degree, and the above two polynomials are expanded to obtain: x=a0+a1u++a2v+a3u 2 +a4uv+a5v 2 +… Formula 11 y=b0+b1u+b2v+b3u 2 +b4uv+b5v 2 +… formula 12 Let the polynomial degree N = 1, which is the affine transformation. After the degree is selected, a certain number of control points are selected, and the coefficients of the polynomial are calculated using the least squares method using the coordinates of these control points to obtain the corrected coordinates. For the evaluation of error and accuracy, the root mean square error RMS is used Error To measure the accuracy of each control point, RMS Error The calculation formula is as follows: Among them, x and y are the coordinates of the selected control point, and and are the coordinates of the control points calculated by the polynomial, that is, the estimated coordinates of the control points.

Images