Large-scale framing map automatic fusion method

Abstract

The invention discloses an automatic fusion method for a large-scale framing map, and the method comprises the following steps: S1, training an image segmentation model, so as to achieve the extraction and segmentation of a map region; s2, registering a graphic area calibrated by the image segmentation model, and restoring the graphic area into a map image of a standard sheet; and S3, determining the position sequence of different framing pictures in the full-width map, splicing a large number of framing pictures, and finally splicing to form the full-width map. According to the method, the automatic fusion of the framing maps in a large-scale scene is realized, the splicing process of the framing maps is remarkably simplified through automatic operation, a large amount of manual operation is reduced, manpower is saved, and the splicing efficiency is greatly improved.

Description

Technical Field

[0001] The invention belongs to the technical field of surveying and mapping, and in particular is an automatic fusion method for large-scale segmented maps. Background Art

[0002] In the last century and the early 21st century, maps were all made on paper. For some scenes with large map ranges and high resolutions, maps could only exist in the form of paper tiles due to map size limitations. These historical map data have important research significance and use value in key areas such as urban planning iteration, urban history research, cultural heritage protection, and natural resource rights confirmation.

[0003] However, it is very difficult to use these maps at present. First, the paper map needs to be scanned to form an electronic image, and then the image needs to be processed and registered manually to be processed into a usable electronic image. Due to the huge number of map frames and the cumbersome manual image processing process, the processing of map images in the use of map frames is usually time-consuming and laborious, and it is difficult to meet the usage standards. Summary of the invention

[0004] In view of the above problems, the present invention is proposed to provide a large-scale segmented map automatic fusion method that overcomes the above problems or at least partially solves the above problems.

[0005] To achieve the above object, the present invention adopts the following technical solutions:

[0006] A large-scale segmented map automatic fusion method, the method comprising the following steps:

[0007] S1. Train the image segmentation model to extract and segment the map area;

[0008] S2, registering the graphic area calibrated by the image segmentation model and restoring it to a map image of a standard map sheet;

[0009] S3. Determine the position order of different sub-frames in the full-frame map, and splice a large number of sub-frames to form a full-frame map.

[0010] Optionally, step S1 includes:

[0011] Select image segmentation model;

[0012] Prepare map image datasets in the surveying and mapping field;

[0013] Train an image segmentation model.

[0014] Optionally, preparation of map image datasets in the surveying and mapping field includes:

[0015] Collect and select tile maps to manually label images;

[0016] Mark the map area by selecting the vertices at the edge of the map area, and then obtain the label image through binarization;

[0017] The number of full-scale maps is expanded through the dataset augmentation method, and the dataset is divided into training set and test set according to the proportion. The image segmentation model is trained based on the dataset.

[0018] Optionally, the map area extraction uses the original image as input, and the trained image segmentation model network outputs a binary mask of the image. The map area is extracted and restored based on the binary mask, specifically including:

[0019] Use cv2.findcounters() function to extract the boundary vertices of the mask;

[0020] Use cv2.approxPolyDP() function to approximate the boundary polygon and reduce the number of vertices. The parameter epsilon is set to 0.01 times the perimeter of the boundary. The return value is the approximated closed set of vertices, denoted by N = {n1, n2, ..., n i ,…n |N|};

[0021] Use cv2.minAreaRect() method to calculate the minimum enclosing rectangle of the vertex set, and classify the vertex set into four categories according to the distance between the vertex and the four sides of the outer rectangle, and obtain N1, N2, N3, and N4, which represent the corresponding point sets of the four sides of the approximate quadrilateral respectively;

[0022] Use the RANSAC algorithm to fit four straight line equations l1, l2, l3, l4 to the four point sets N1, N2, N3, N4 respectively, and calculate the intersection points of the straight lines to obtain the vertices d1, d2, d3, d4 of the quadrilateral area;

[0023] For vertex n i , whose projection point on the corresponding straight line is n i ′, so that the vertex vector [n1,n2,...,n |N| ]The projection vector on the quadrilateral is [n1′, n2′, ..., n |N| ′], use cv2.getPerspectiveTranfrom() function to calculate the projection matrix and use cv2.warpPerspective() function to perform projection transformation on the original image to map the map area to the quadrilateral area;

[0024] Let h and w be the height and width of the standard rectangular map. For the vertex matrix [d1, d2, d3, d4] of the quadrilateral, calculate its projection matrix mapped to the standard map matrix [[0, 0], [w, 0], [w, h], [0, h]], and perform projection transformation to map the map area to the standard rectangular map, so that the original image is restored to a map image with a standard map style.

[0025] Optionally, the graphic area calibrated by the image segmentation model is registered through a map image restoration algorithm, and the mapping relationship between the edge vertices of the segmented map image and the standard map sheet is described, and computer vision technology is used to perform graphic transformation to correct the impact of deformation on the map area.

[0026] Optionally, step S3 includes:

[0027] According to the similarity law of pixels on the adjacent edges of two adjacent sub-frames, an algorithm is designed to describe the similarity relationship between the boundaries of different sub-frames.

[0028] The similarity is measured, and based on the similarity, the candidate adjacent edges are determined. Rules are designed to determine the final adjacency relationship of the sub-images, and the splicing and fusion of all the sub-images are completed.

[0029] Optionally, the splicing of the sub-frames is based on the automatic fusion of edge features, including:

[0030] For the adjusted standard map, the image pixels with a width of one pixel on the four sides are intercepted as edge vectors. For the horizontal and vertical edges of the frame i, we get Respectively represent the edge vectors of the top, bottom, left and right sides;

[0031] For the sub-images i and j, the upper boundary edge vector is calculated by the dynamic time warping algorithm and the lower boundary edge vector The minimum regularization distance is denoted as and the left boundary edge vector and the right boundary edge vector The minimum regular distance between The distance metric is Manhattan distance;

[0032] For the upper boundary of tile i, select The corresponding edge of the corresponding tile image j is taken as the candidate adjacent edge, and The score of the candidate adjacent edge pair is used to measure the similarity between the two edges. Similarly, the candidate adjacent edges and similarity of each edge can be found;

[0033] For an m×n combination of sub-frames, there should be m(n-1)n(m-1) pairs of adjacent edges. According to the scores of each pair of adjacent edges from high to low, m(n-1)+n(m-1) pairs of adjacent edges that do not conflict are selected as the final adjacent edges and the splicing and fusion of the sub-frames are performed to complete the automatic fusion of the full-frame image.

[0034] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0035] 1. The present invention realizes the automatic fusion of tiled maps in large-scale scenes. Through automated operation, the tiled map splicing process is significantly simplified, a large number of manual operations are reduced, manpower is saved and the splicing efficiency is greatly improved. In the map area extraction and segmentation steps, the mIoU of the map area segmentation reached 98.63%, effectively developing a map image segmentation model suitable for the surveying and mapping field.

[0036] 2. The map stitching accuracy of the present invention based on anchor point statistics reaches 96%, which significantly improves the stitching effect and map usability. This technology has broad application prospects in key areas such as urban planning iteration, urban history research, cultural heritage protection, and natural resource rights confirmation, providing an effective tool to better support urban development. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 A schematic diagram of a large-scale segmented map automatic fusion method provided in an embodiment of the present application;

[0038] Figure 2 This is a schematic diagram of a label image according to an embodiment of the present application;

[0039] Figure 3 This is a rendering of the original image of the embodiment of the present application restored to a map image of a standard map format;

[0040] Figure 4 It is a schematic diagram of the splicing process (partial). After determining the splicing order, execute the splicing diagram according to the corresponding position;

[0041] Figure 5 This is a schematic diagram of the splicing effect;

[0042] Figure 6 Schematic diagram of anchor point pair marking in the map stitching accuracy process based on anchor point statistics. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0044] See also Figure 1 This embodiment provides a large-scale segmented map automatic fusion method, the method comprising the following steps:

[0045] S1. Train the image segmentation model to extract and segment the map area.

[0046] Step S1 includes:

[0047] Select an image segmentation model.

[0048] Prepare map image datasets in the surveying and mapping field.

[0049] Train the image segmentation model to ensure the model's ability to accurately segment map images.

[0050] Preparation of map image datasets in the surveying and mapping field includes:

[0051] 500 typical partitioned maps were collected and selected for manual labeling of label images. The label image is a binary mask of the map area (the map area value is 255 and the background area value is 0).

[0052] The map area is marked by selecting the vertices at the edge of the map area, and then the label image is obtained through binarization. The specific effect is shown in the attached figure. Figure 2 shown.

[0053] The number of full-scale maps was expanded to 1,000 by using the dataset augmentation method, and divided into training set and test set in a ratio of 9:1. The image segmentation model was trained based on this dataset. The training parameter settings are as follows:

[0054]

[0055]

[0056] The map area extraction takes the original image as input, and the trained image segmentation model network outputs the binary mask of the image. The map area is extracted and restored based on the binary mask, which specifically includes:

[0057] Use cv2.findcounters() function to extract the boundary vertices of the mask.

[0058] Use cv2.approxPolyDP() function to approximate the boundary polygon and reduce the number of vertices. The parameter epsilon is set to 0.01 times the perimeter of the boundary. The return value is the approximated closed set of vertices, denoted by N = {n1, n2, ..., n i ,…n |N|}.

[0059] Use cv2.minAreaRect() method to calculate the minimum enclosing rectangle of the vertex set, and classify the vertex set into four categories according to the distance between the vertex and the four sides of the outer rectangle, and get N1, N2, N3, N4, which represent the corresponding point sets of the four sides of the approximate quadrilateral respectively.

[0060] Use the RANSAC algorithm to fit four straight line equations l1, l2, l3, l4 to the four point sets N1, N2, N3, N4 respectively, and calculate the intersection of the straight lines to obtain the vertices d1, d2, d3, d4 of the quadrilateral area.

[0061] For vertex n i , whose projection point on the corresponding straight line is n i ′, so that the vertex vector [n1, n2, ..., n |N| ]The projection vector on the quadrilateral is [n1′, n2′, ..., n |N| ′], use cv2.getPerspectiveTranfrom() function to calculate the projection matrix and use cv2.warpPerspective() function to perform projection transformation on the original image to map the map area to the quadrilateral area.

[0062] Let h and w be the height and width of the standard rectangular map sheet. For the vertex matrix [d1, d2, d3, d4] of the quadrilateral, calculate its projection matrix mapped to the standard map sheet matrix [[0,0], [w,0], [w,h], [0,h]], and perform projection transformation to map the map area to the standard rectangular map sheet, so that the original image is restored to a map image of the standard map sheet style. The specific effect is shown in the attached figure. Figure 3 shown.

[0063] S2. Register the graphic area calibrated by the image segmentation model and restore it to a map image of a standard sheet size.

[0064] The graphic area calibrated by the image segmentation model is registered through the map image restoration algorithm, and the mapping relationship between the edge vertices of the segmented map image and the standard map sheet is described. Computer vision technology is used to perform graphic transformation to correct the impact of deformation on the map area.

[0065] S3. Determine the position order of different sub-frames in the full-frame map, and stitch a large number of sub-frames together to form a full-frame map. The specific effect is shown in the attached figure. Figure 4 , Attachment Figure 5 and attached Figure 6 shown.

[0066] Step S3 includes:

[0067] According to the similarity law of pixels on the adjacent edges of two adjacent sub-images, an algorithm is designed to describe the similarity relationship between the boundaries of different sub-images.

[0068] The similarity is measured, and based on the similarity, the candidate adjacent edges are determined. Rules are designed to determine the final adjacency relationship of the sub-images, and the splicing and fusion of all the sub-images are completed.

[0069] The splicing of the sub-frames is based on the automatic fusion of edge features. Ideally, the pixels of the adjacent edges of adjacent sub-frames should be exactly the same. In the use of sub-frames, due to possible map preparation errors, scanning image problems, deformation and contamination interference, there will be some differences in the pixels of adjacent edges, but they are still very similar on a large scale. And this similarity is difficult to be caused by other non-adjacent edges. Therefore, this method statistically calculates the similarity between the edges of the sub-frames, and constructs a candidate set of adjacent edges based on the similarity to determine the adjacency relationship between different sub-frames, including:

[0070] For the adjusted standard map, the image pixels with a width of one pixel on the four sides are intercepted as edge vectors. For the horizontal and vertical edges of the frame i, we get Represent the edge vectors of the top, bottom, left and right sides respectively.

[0071] For the sub-images i and j, the upper boundary edge vector is calculated by the dynamic time warping algorithm and the lower boundary edge vector The minimum regularization distance is denoted as and the left boundary edge vector and the right boundary edge vector The minimum regular distance between The distance metric is Manhattan distance.

[0072] For the upper boundary of tile i, select The corresponding edge of the corresponding tile image j is taken as the candidate adjacent edge, and The score of the candidate adjacent edge pair is used to measure the similarity between the two edges. Similarly, the candidate adjacent edges and similarity of each edge can be found.

[0073] For an m×n combination of segmented images, there should be m(n-1)+n(m-1) pairs of adjacent edges. According to the scores of each adjacent edge pair from high to low, m(n-1)+n(m-1) pairs of adjacent edges that do not conflict are selected as the final adjacent edges and the segmented images are spliced ​​and fused to complete the automatic fusion of the full image.

[0074] This embodiment realizes the automated fusion of segmented maps in large-scale scenarios. Through automated operations, the segmented map splicing process is significantly simplified, a large amount of manual operations are reduced, manpower is saved and the splicing efficiency is greatly improved. In the map area extraction and segmentation steps, the mIoU of the map area segmentation reached 98.63%, and a map image segmentation model suitable for the surveying and mapping field was effectively developed. In addition, the map splicing accuracy based on anchor point statistics reached 96%, which significantly improved the splicing effect and map usability. This technology has broad application prospects in key areas such as urban planning iteration, urban history research, cultural heritage protection, and natural resource rights confirmation, providing an effective tool to better support urban development.

[0075] The above contents are further detailed descriptions of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the protection scope of the present invention.

Claims

1. A large-scale segmented map automatic fusion method, characterized in that: The method comprises the following steps: S1. Train the image segmentation model to extract and segment the map area; S2, registering the graphic area calibrated by the image segmentation model and restoring it to a map image of a standard map sheet; S3. Determine the position order of different sub-frames in the full-frame map, and splice a large number of sub-frames to form a full-frame map.

2. The large-scale segmented map automatic fusion method according to claim 1, characterized in that: Step S1 includes: Select image segmentation model; Prepare map image datasets in the surveying and mapping field; Train an image segmentation model.

3. The large-scale segmented map automatic fusion method according to claim 1, characterized in that: Preparation of map image datasets in the surveying and mapping field includes: Collect and select tile maps to manually label images; Mark the map area by selecting the vertices at the edge of the map area, and then obtain the label image through binarization; The number of full-scale maps is expanded through the dataset augmentation method, and the dataset is divided into training set and test set according to the proportion. The image segmentation model is trained based on the dataset.

4. The large-scale segmented map automatic fusion method according to claim 1, characterized in that: The map area extraction takes the original image as input, and the trained image segmentation model network outputs the binary mask of the image. The map area is extracted and restored based on the binary mask, which specifically includes: Use cv2.findcounters() function to extract the boundary vertices of the mask; Use cv2.approxPolyDP() function to approximate the boundary polygon and reduce the number of vertices. The parameter epsilon is set to 0.01 times the perimeter of the boundary. The return value is the closed set of approximated vertices, denoted by N = {n1, n2, ..., n i , ..., n |N| }; Use cv2.minAreaRect() method to calculate the minimum enclosing rectangle of the vertex set, and classify the vertex set into four categories according to the distance between the vertex and the four sides of the outer rectangle, and obtain N1, N2, N3, and N4, which represent the corresponding point sets of the four sides of the approximate quadrilateral respectively; Use the RANSAC algorithm to fit four straight line equations l1, l2, l3, l4 to the four point sets N1, N2, N3, N4 respectively, and calculate the intersection points of the straight lines to obtain the vertices d1, d2, d3, d4 of the quadrilateral area; For vertex n i , whose projection point on the corresponding straight line is n i ′, so that the vertex vector [n1, n2, ..., n |N| ]The projection vector on the quadrilateral is [n1′, n2′, ..., n |N| ′], use cv2.getPerspectiveTranfrom() function to calculate the projection matrix and use cv2.warpPerspective() function to perform projection transformation on the original image to map the map area to the quadrilateral area; Let h and w be the height and width of the standard rectangular map. For the vertex matrix [d1, d2, d3, d4] of the quadrilateral, calculate its projection matrix mapped to the standard map matrix [[0, 0], [w, 0], [w, h], [0, h]], and perform projection transformation to map the map area to the standard rectangular map, so that the original image is restored to a map image with a standard map style.

5. The large-scale segmented map automatic fusion method according to claim 1, characterized in that: The graphic area calibrated by the image segmentation model is registered through the map image restoration algorithm, and the mapping relationship between the edge vertices of the segmented map image and the standard map sheet is described. Computer vision technology is used to perform graphic transformation to correct the impact of deformation on the map area.

6. The large-scale segmented map automatic fusion method according to claim 1, characterized in that: Step S3 includes: According to the similarity law of pixels on the adjacent edges of two adjacent sub-frames, an algorithm is designed to describe the similarity relationship between the boundaries of different sub-frames. The similarity is measured, and based on the similarity, the candidate adjacent edges are determined. Rules are designed to determine the final adjacency relationship of the sub-images, and the splicing and fusion of all the sub-images are completed.

7. The large-scale segmented map automatic fusion method according to claim 1, characterized in that: The splicing of the sub-frames is based on the automatic fusion of edge features, including: For the adjusted standard map, the image pixels with a width of one pixel on the four sides are intercepted as edge vectors. For the horizontal and vertical edges of the frame i, we get Respectively represent the edge vectors of the top, bottom, left and right sides; For the sub-images i and j, the upper boundary edge vector is calculated by the dynamic time warping algorithm and the lower boundary edge vector The minimum regularization distance is denoted as and the left boundary edge vector and the right boundary edge vector The minimum regular distance between The distance metric is Manhattan distance; For the upper boundary of tile i, select The corresponding edge of the corresponding tile image j is taken as the candidate adjacent edge, and The score of the candidate adjacent edge pair is used to measure the similarity between the two edges. Similarly, the candidate adjacent edges and similarity of each edge can be found; For an m×n combination of segmented images, there should be m(n-1)+n(m-1) pairs of adjacent edges. According to the scores of each adjacent edge pair from high to low, m(n-1)+n(m-1) pairs of adjacent edges that do not conflict are selected as the final adjacent edges and the segmented images are spliced ​​and fused to complete the automatic fusion of the full image.

Images