Intelligent segmentation method based on multi-source geographic data fusion

Abstract

The invention discloses an intelligent segmentation method based on multi-source geographic data fusion, and the method employs a combination method of a neural network technology, digital image operation and space vector operation to form a fusion updating method for current spatial geographic data, simplifies and fuses multiple boundaries to reduce invalid information, and improves the efficiency. And face graph segmentation is carried out by using the boundary lines extracted from the picture, so that an original geographic face graph is effectively updated by taking the remote sensing image as a reference. According to the method, the boundaries are converted into the images, re-extraction is carried out through a morphological operation method, equivalently, the boundaries which are very close in position are fused, and tiny deviation of the boundaries and the actual boundaries in the images is avoided.

Description

Technical Field

[0001] The invention belongs to the technical field of geographic data processing and relates to an intelligent segmentation method based on multi-source geographic data fusion. Background Art

[0002] With the continuous development of artificial intelligence technology, many remote sensing image segmentation and classification methods based on neural networks have emerged in the field of remote sensing. However, most of them require the production of a large number of samples with regional annotations according to specific application scenarios, which is time-consuming and laborious. In the field of geographic information, three national land surveys have been carried out. If the spatial distribution of land use in a certain area needs to be updated according to remote sensing images, the traditional method is to regenerate various land use maps according to remote sensing images in the entire range, and then overlay and analyze them with the current land status layer. The workload and calculation amount are huge, and there will be a large number of original boundaries that will move slightly. Summary of the invention

[0003] The problem solved by the present invention is to provide an intelligent segmentation method based on multi-source geographic data fusion, simplify the preparation of the neural network model, and avoid the situation where a large number of original boundaries have slight offsets.

[0004] The present invention is achieved through the following technical solutions:

[0005] An intelligent segmentation method based on multi-source geographic data fusion includes the following operations:

[0006] 1) The geographical scope of the current vector map is used as the scope of the latest remote sensing image to be extracted; the remote sensing image is divided into blocks according to a regular grid and expanded by a certain number of pixels to form image slices, and the image is preprocessed including denoising and color equalization;

[0007] 2) Submit the image to the image convolutional neural network for predictive segmentation, infer the categories of clustered pixels in the image, identify the boundary pixels between classes, segment or classify the different pixel groups generated by categories, and generate a predictive segmentation / classification grid map, where the same pixel values ​​are the same, and different categories will form a clear boundary;

[0008] 3) Vectorizing the predicted segmentation / classification grid map, and converting the areas where the same type of pixels are clustered into several polygonal boundaries with common edges between adjacent categories;

[0009] Convert all extracted polygon boundaries into line graphics, and make a buffer zone of 1-2 pixels width for the polygon boundary lines to form a strip surface graphics;

[0010] 4) extracting the center line of the strip surface figure as the common boundary of the multi-layer boundary covering the transition boundary of adjacent categories in the strip surface figure;

[0011] 5) Perform a morphological opening operation on the image of the strip-shaped surface graphic to obtain the result A;

[0012] Perform multiple morphological erosions on the image of the strip-shaped surface until all pixels in the image are black, and then take the previous result image as result B;

[0013] Subtract result A from result B pixel by pixel to obtain a rasterized primary skeleton line map, and refine it using the Zhang-Suen algorithm to extract lines from the raster image; then convert the skeleton lines in the raster image into a line coordinate sequence to generate a line layer extracted by slice range;

[0014] 6) Import the current vector surface layer from the source data, traverse the surface graphics, and use the boundary lines of the line layer to split the surface graphics that intersect with it;

[0015] For each face, the area of ​​the small polygons generated after line segmentation is calculated. If the area is less than the threshold or the size of the segmented section is less than the threshold, the small polygons are merged back into the original face; otherwise, the segmentation result is retained and the update of the survey map is completed.

[0016] The size of the outward expansion pixels is based on the edge after the block division, and the image slices can be connected to the outward expansion parts to form a whole;

[0017] If the outward expansion pixel is too small, the information at the edge of the block will be lost, resulting in gaps at the edges of the blocks after merging; if it is too large, the repeated area at the edges will be expanded, which will increase the amount of invalid calculations.

[0018] After converting the polygonal boundaries into line graphics, the boundaries that need to be merged are added to generate the buffer zone.

[0019] The method for extracting the center line of the strip surface graphic is:

[0020] Calculate directly on the vector polygon data, first merge the adjacent original polygon faces, then perform triangulation and extract the center line of the strip surface figure based on its internal triangles;

[0021] Alternatively, the strip vector surface graphic is converted into a binary raster image, wherein the polygonal surface range is 1 and the outside of the surface is 0.

[0022] The morphological opening operation is to perform a dilation-erosion opening operation and remove small hanging adhesions and tiny closed areas.

[0023] The traversal is performed in the order stored in the layer set;

[0024] The threshold is determined by the minimum area required by the application scenario, and is generally an integer multiple of the pixel area.

[0025] Compared with the prior art, the present invention has the following beneficial technical effects:

[0026] The intelligent segmentation method based on multi-source geographic data fusion proposed in the present invention can perform image recognition based on convolutional neural networks, and can fully utilize the existing prediction models trained by huge data sets as a basis to construct a current map boundary update plan, thereby omitting or simplifying the preparation process of the previous model of "sample collection", "label preparation" and "sample training" that must be carried out in most neural network prediction models; and in the work of updating the current vector boundary from remote sensing images, the boundary of the current layer is completely retained, and the latest simplified boundary is extracted according to the remote sensing image segmentation. The current layer data is supplemented and updated in the original image by adding new boundaries, so as to avoid a large number of small boundary movements caused by comprehensive re-update.

[0027] The intelligent segmentation method based on multi-source geographic data fusion proposed in the present invention uses segmentation lines from multiple sources to update the land map, including segmentation lines output by a remote sensing image segmentation algorithm, terrain lines calculated and produced by DEM raster data, and existing vector lines; the result of merging these lines is used to segment the land map to achieve the goal of updating the land map and avoiding slight deviations of the original boundaries.

[0028] The present invention utilizes a combination of neural network technology, digital image operation, and space vector operation to form a fusion update method for current spatial geographic data, simplifies and fuses multiple boundaries to reduce invalid information, and performs surface graphics segmentation based on boundary lines extracted from images, thereby achieving effective updating of the original geographic surface graphics based on remote sensing images. In the case of multiple boundaries, in order to remove overlapping boundaries, the general processing method is to retain all or partially delete them. If the deletion method is inappropriate, a slight offset will be caused between the boundary and the actual boundary in the image; the present invention converts the boundary into an image again, and re-extracts it using a morphological operation method, which is equivalent to fusing boundaries that are very close in position to avoid offsets. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 Schematic diagram of the process of the present invention;

[0030] Figure 2 It is the multi-layer boundary at the transition point between adjacent categories formed locally after segmentation and classification of remote sensing images;

[0031] Figure 3 The rasterized buffer strip boundary (white) and the extracted centerline (red);

[0032] Figure 4 Superimposing the current surface graphics to be segmented on the extracted boundary lines;

[0033] Figure 5 This is a screenshot of the original image and the original national border in the actual scene. The red border is the national border;

[0034] Figure 6 The boundary (green) extracted from the image is used to crop the updated land boundary of the land graphics. DETAILED DESCRIPTION

[0035] See also Figure 1 The present invention proposes an intelligent segmentation method based on multi-source geographic data fusion, which includes the following operations:

[0036] 1) The geographical scope of the current vector map is used as the scope of the latest remote sensing image to be extracted; the remote sensing image is divided into blocks according to a regular grid and expanded by a certain number of pixels to form image slices, and the image is preprocessed including denoising and color equalization;

[0037] 2) Submit the image to the image convolutional neural network for predictive segmentation, infer the categories of clustered pixels in the image, identify the boundary pixels between classes, segment or classify the different pixel groups generated by categories, and generate a predictive segmentation / classification grid map, where the same pixel values ​​are the same, and different categories will form a clear boundary;

[0038] 3) Vectorizing the predicted segmentation / classification grid map, converting the areas where similar pixels are clustered into several polygonal boundaries with common edges between adjacent categories;

[0039] Convert all extracted polygon boundaries into line graphics, and make a buffer zone of 1-2 pixels width for the polygon boundary lines to form a strip surface graphics;

[0040] 4) extracting the center line of the strip surface figure as the common boundary of the multi-layer boundary covering the transition boundary of adjacent categories in the strip surface figure;

[0041] 5) Perform a morphological opening operation on the image of the strip-shaped surface graphic to obtain the result A;

[0042] Perform multiple morphological erosions on the image of the strip-shaped surface until all pixels in the image are black, and then take the previous result image as result B;

[0043] Subtract result A from result B pixel by pixel to obtain a rasterized primary skeleton line map, and refine it using the Zhang-Suen algorithm to extract lines from the raster image; then convert the skeleton lines in the raster image into a line coordinate sequence to generate a line layer extracted by slice range;

[0044] 6) Import the current vector surface layer from the source data, traverse the surface graphics, and use the boundary lines of the line layer to split the surface graphics that intersect with it;

[0045] For each face, the area of ​​the small polygons generated after line segmentation is calculated. If the area is less than the threshold or the size of the segmented section is less than the threshold, the small polygons are merged back into the original face; otherwise, the segmentation result is retained and the update of the survey map is completed.

[0046] The retained segmentation result is the final result map of this period. It can replace the geometry of the map of the previous period in the corresponding position record as the surface graphic geometry (the entity is a series of regular coordinate sets) on the final corresponding position record of this period. It is generally completed by importing it into the spatial database.

[0047] Furthermore, the size of the outward expansion pixels is based on the edge after the block division, and the image slices can be connected with the outward expansion parts to form a whole;

[0048] If the outward expansion pixel is too small, the information at the edge of the block will be lost, resulting in gaps at the edges of the blocks after merging; if it is too large, the repeated area at the edges will be expanded, which will increase the amount of invalid calculations.

[0049] A specific implementation example is given below for updating the map patches of land survey.

[0050] Example 1

[0051] An intelligent segmentation method based on multi-source geographic data fusion includes the following operations:

[0052] Step 1: Take the geographical scope of the current vector map as the scope of the latest remote sensing image to be extracted; divide the remote sensing image into image slices and perform preprocessing;

[0053] Since remote sensing images generally have large pixel sizes, they need to be divided into blocks to form slices for easy processing. Therefore, a regular grid size is used as the standard, and a certain number of pixels (such as 50 pixels) are expanded to divide the image cropping range; the expanded pixels are for subsequent separate processing of the blocks and slices, and the expanded parts can also be combined to form a final whole.

[0054] The reason for expanding by a certain number of pixels here is to ensure that the results of subsequent regular grid block processing can be connected. If it is too small, information at the edge of the block will be lost, resulting in gaps at the edges of the blocks after merging. If it is too large, the repeated area at the edges will be expanded, increasing the amount of invalid calculations.

[0055] The image cropping results also need to be pre-processed by denoising, color equalization and other operations;

[0056] Specifically, there are many methods available for image denoising. In this embodiment, the median filtering method is used for image denoising.

[0057] Color equalization uses histogram equalization, which are commonly used techniques in digital image processing.

[0058] Step 2: Submit the image to the image convolutional neural network for prediction and segmentation;

[0059] Using network communication, the image data formed by block slicing is transmitted to the image convolutional neural network training model, and the uploaded image is predicted and segmented (inferring which category the clustered pixels belong to, or only identifying the boundary pixels between classes), and the predicted result image is returned.

[0060] The convolutional neural network training model can directly use the training results of existing organizations or institutions on large data sets, or use a pre-trained model plus a small amount of data sets collected according to specific application scenarios, conduct a small amount of training to adjust the neural network parameters, and use these models for final predictions.

[0061] A pre-trained model is a model that has been trained with a data set, such as VGG16 / 19, Resnet and other models (excellent networks that have been debugged and do not need to modify their network structure), and uses a large data set as a training set, such as Imagenet, COCO and other trained model parameters; this saves time and computing resources, and can quickly achieve better results.

[0062] The prediction result image returned by the convolutional neural network training model contains different pixel groups generated by category segmentation or classification. The pixel values ​​of the same category are the same, and there will be obvious boundaries between different categories.

[0063] Step 3: Using the raster vectorization method, the area where similar pixels are clustered in the prediction result image is converted into a number of polygons with common edges between adjacent categories;

[0064] When a raster image is converted into a vector polygon, jagged polygon boundaries originating from small pixel squares will be generated; if these boundaries are directly simplified and smoothed, several polygons that originally shared the boundaries will overlap and have gaps at the common edges of the polygons after processing.

[0065] In addition, due to the complexity and uncertainty of the geographical plots represented by remote sensing images, after neural network recognition, a large number of multi-layer boundaries will appear at the boundaries of different categories (such as Figure 2 As shown, the color image is a remote sensing image, the raster image is below it, and the image on the right is the raster image converted into a vector polygon). There is only 1 to 2 pixels between these layers, which will result in a large amount of invalid boundary information in the resulting image.

[0066] Therefore, all extracted polygonal boundaries are converted into line graphics, and after adding other boundaries that need to be fused (such as valleys and ridge lines), a buffer zone with a width of 1 to 2 pixels is made for these boundary lines to form a strip surface graphic; it covers the multi-layer boundaries at the transition boundaries of most different adjacent categories.

[0067] The boundaries to be merged depend on the specific application scenario.

[0068] Step 4: Extract the center line of the strip surface figure as the common boundary of the aforementioned multi-layer boundaries.

[0069] There are two types of solutions for this step: Solution 1, directly calculate on the vector polygon data, first merge adjacent original polygon faces to avoid overlapping areas between faces, and then perform triangulation based on the internal triangles to extract the center line of the strip surface figure [such as CN114972624A];

[0070] Solution 2: First, convert the strip vector surface graphics into a binary rasterized image, where the polygonal surface range is 1 and the outside of the surface is 0. Since the shape of the strip surface graphics is relatively consistent after rasterization, it is convenient for the subsequent centerline extraction process.

[0071] This embodiment adopts solution 2. Figure 3 Shown is the rasterized buffer strip boundary (white) and the extracted centerline (red);

[0072] Step 5: Rasterize the binary image to form a strip-shaped distribution result image, and first perform expansion and corrosion operations to obtain result A, in which small hanging adhesions and tiny closed areas are removed.

[0073] Then the original binary raster image is eroded multiple times until all pixels in the image are black after a certain erosion, and the previous intermediate result image is taken as result B;

[0074] Subtract result A from result B pixel by pixel to obtain a rasterized primary skeleton line graph (result A and result B are intermediate results, both of which are a raster image);

[0075] The raster image of the primary skeleton line is further refined using the Zhang-Suen algorithm (the purpose of refinement is to facilitate the extraction of lines from the raster image, and the primary skeleton line is too thick and uneven to be converted into a line graphic), and then the skeleton lines in the final raster image are converted and extracted into a line coordinate sequence to finally generate a line layer.

[0076] Step 6: Import the current vector surface layer from the source data, traverse the surface graphics therein (traverse in the order stored in the layer set); use the boundary line extracted according to the slice range in step 5 to split the surface graphics that have an intersecting relationship with it; Figure 4What is shown is the boundary line superimposed on the current surface graphics to be segmented;

[0077] For each face, the area of ​​the small polygons generated after line segmentation is calculated. If the area is smaller than a specific area value or the size of the segmented section is smaller than a threshold, the small polygons are merged back into the original face shape.

[0078] The threshold is determined by the minimum area required by the application scenario. The size of a single pixel corresponds to the geographic size of the actual location and is generally expressed in pixels, which is an integer multiple of the pixel area size.

[0079] Specifically, Figure 5 This is a screenshot of the original image and the original national boundary in the actual scene (before processing). The red boundary is the national boundary.

[0080] Figure 6 As shown, at the same position, the boundary (green) extracted from the image is used to crop the land graphics and finally used to update the land boundary; it can be seen that the method of the present invention can avoid slight deviation of the original boundary.

[0081] The above embodiments are preferred examples for implementing the present invention, and the present invention is not limited to the above embodiments. Any non-essential additions and substitutions made by those skilled in the art based on the technical features of the technical solution of the present invention shall fall within the protection scope of the present invention.

Claims

1. An intelligent segmentation method based on multi-source geographic data fusion, characterized in that: The following operations are included: 1) The geographical scope of the current vector map is used as the scope of the latest remote sensing image to be extracted; the remote sensing image is divided into blocks according to a regular grid and expanded by a certain number of pixels to form image slices, and the image is preprocessed including denoising and color equalization; 2) Submit the image to the image convolutional neural network for predictive segmentation, infer the categories of clustered pixels in the image, identify the boundary pixels between classes, segment or classify the different pixel groups generated by categories, and generate a predictive segmentation / classification grid map, where the same pixel values ​​are the same, and different categories will form a clear boundary; 3) Vectorizing the predicted segmentation / classification grid map, converting the areas where similar pixels are clustered into several polygonal boundaries with common edges between adjacent categories; Convert all extracted polygon boundaries into line graphics, and make a buffer zone of 1-2 pixels width for the polygon boundary lines to form a strip surface graphics; 4) extracting the center line of the strip surface figure as the common boundary of the multi-layer boundary covering the transition boundary of adjacent categories in the strip surface figure; 5) Perform a morphological opening operation on the image of the strip-shaped surface graphic to obtain the result A; Perform multiple morphological erosions on the image of the strip-shaped surface until all pixels in the image are black, and then take the previous result image as result B; Subtract result A from result B pixel by pixel to obtain a rasterized primary skeleton line map, and refine it using the Zhang-Suen algorithm to extract lines from the raster image; then convert the skeleton lines in the raster image into a line coordinate sequence to generate a line layer extracted by slice range; 6) Import the current vector surface layer from the source data, traverse the surface graphics, and use the boundary lines of the line layer to split the surface graphics that intersect with it; For each face, the area of ​​the small polygons generated after line segmentation is calculated. If the area is less than the threshold or the size of the segmented section is less than the threshold, the small polygons are merged back into the original face; otherwise, the segmentation result is retained and the update of the survey map is completed.

2. The intelligent segmentation method based on multi-source geographic data fusion according to claim 1, characterized in that: The size of the outward expansion pixels is based on the edge after the block division, and the image slices can be connected to the outward expansion parts to form a whole; If the outward expansion pixel is too small, the information at the edge of the block will be lost, resulting in gaps at the edges of the blocks after merging; if it is too large, the repeated area at the edges will be expanded, which will increase the amount of invalid calculations.

3. The intelligent segmentation method based on multi-source geographic data fusion according to claim 1, characterized in that: The denoising is performed by using a median filtering method to perform image denoising; The color equalization is performed by histogram equalization.

4. The intelligent segmentation method based on multi-source geographic data fusion according to claim 1, characterized in that: After converting the polygonal boundaries into line graphics, the boundaries that need to be merged are added to generate the buffer zone.

5. The intelligent segmentation method based on multi-source geographic data fusion according to claim 1, characterized in that: The method for extracting the center line of the strip surface graphic is: Calculate directly on the vector polygon data, first merge the adjacent original polygon faces, then perform triangulation and extract the center line of the strip surface figure based on its internal triangles; Alternatively, the strip vector surface graphic is converted into a binary raster image, wherein the polygonal surface range is 1 and the outside of the surface is 0.

6. The intelligent segmentation method based on multi-source geographic data fusion according to claim 1, characterized in that: The morphological opening operation is to perform a dilation-erosion opening operation and remove small hanging adhesions and tiny closed areas.

7. The intelligent segmentation method based on multi-source geographic data fusion according to claim 1, characterized in that: The traversal is performed in the order stored in the layer set; The threshold is determined by the minimum area required by the application scenario, and is generally an integer multiple of the pixel area.

Images