High-resolution remote sensing image road extraction method considering network topological relation

Abstract

The invention discloses a high-resolution remote sensing image road extraction method considering a network topology relation, and belongs to the patent field of the geographic information system and intelligent traffic field, and the basic steps are as follows: obtaining an intersection category in a to-be-detected image and a maximum intersection detection target frame with a direction; extracting information such as a road surface, a road center line and an edge line of the to-be-detected image; automatically constructing the topological relation of the road network by using the geometrical relation between the detected road intersection target frame and the extracted road center line; and finally storing the topological relation of the road network. Wherein the intersection recognition model and the road information semantic segmentation model are obtained by training according to a rotating frame road intersection target detection data set and a road segmentation data set. The method has the advantages of being simple, effective, easy to implement and the like, the topological information of the road network is automatically constructed on the basis of the intersection information and the road center line information, and the efficiency of road extraction and topological relation establishment is improved.

Description

Technical Field

[0001] The present invention belongs to the patent field of geographic information system and intelligent transportation, and specifically relates to a method for extracting roads based on remote sensing images and constructing road topological relationships. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] With the continuous expansion of urban scale and the continuous increase in traffic volume, the importance of managing and planning urban road networks has become increasingly prominent. Traditional urban road management methods mainly rely on manual inspections and field surveys, which are time-consuming, labor-intensive and inefficient. The development of modern remote sensing technology and deep learning technology has provided new solutions for road management. Remote sensing technology can obtain high-resolution remote sensing images through satellites, aircraft, etc., providing important data support for the management of urban road networks.

[0004] Most existing studies focus on the extraction of road surfaces and road lines. According to the different extraction methods, they can be divided into traditional extraction methods and deep learning semantic segmentation extraction methods. Traditional extraction methods mainly extract roads from remote sensing images based on the shape and texture feature information of the road. Deep learning extraction algorithms can extract deep abstract features from remote sensing images, overcome the shortcomings of traditional extraction methods such as slow extraction speed and poor effect, and can effectively improve the accuracy and efficiency of road information recognition. However, the type of intersections and the connectivity relationship between roads (i.e., topological relationship) are also important aspects of road information, which are of great significance for autonomous driving and high-precision map construction. In particular, the topological information of roads needs to take into account multiple factors such as road connectivity and intersections. In order to accurately capture this information, it is necessary to integrate technologies from multiple fields such as image processing, machine learning, and computational geometry to solve the problem of extracting road topological information with a comprehensive strategy. Therefore, extracting road topological information based on remote sensing images is still a challenging problem.

[0005] The present invention provides a method for extracting roads and their topological information based on remote sensing images and deep learning technology, which can realize the automation of road segmentation and intersection information detection, and then accurately extract the road centerline in the remote sensing image, and vectorize it, and finally complete the construction of the road network topological structure. The advantage of this method is that it can realize the automated processing of large-scale remote sensing image data, improve the efficiency of road extraction and the establishment of its topological relationship, and provide data support for traffic management and urban planning.

[0006] Beneficial Effects

[0007] Compared with the prior art, the present invention has the following beneficial effects:

[0008] The present invention provides a method for constructing road topology information based on high-resolution remote sensing images. The main process is to extract road surfaces, road centerlines, road edge lines and intersection information with angle information from high-resolution remote sensing images through a trained deep learning model, and vectorize the road centerlines, and then construct the topological structure of the entire road network according to the geographical location relationship between the intersections and the vectorized road centerlines; in large-scale road topology information extraction tasks, the present invention can efficiently generate the road network topological structure of the target area. Summary of the invention

[0009] In order to solve the problem that the existing technology cannot completely construct road topology information when directly extracting roads from remote sensing images, the present invention proposes an innovative method that can not only effectively extract road information from remote sensing images, but also further construct the topological structure of the road.

[0010] To achieve the above object, the present invention provides the following technical solutions:

[0011] The present invention provides a method for extracting roads from high-resolution remote sensing images taking into account network topology relationships, which is characterized by comprising the following steps:

[0012] 1. Extracting road networks with topological relationships

[0013] Step 1: Identify road intersections, including: Using the rotating box road intersection target detection dataset Itr trn Train the rotating box target detection deep learning model to obtain the road intersection detection model M Itr ; Get a remote sensing image X with a spatial resolution of 0.3 meters for the area to be processed, downsample X to obtain remote sensing images X0, X1, and X2 with spatial resolutions of 1 meter, 2 meters, and 4 meters, respectively, and record the geographic coordinate ranges of these remote sensing images. Sliding window crop the remote sensing images of each spatial resolution and create an index to obtain a road intersection detection dataset B composed of cropped remote sensing images X0, X1, and X2; Use the road intersection detection model M Itr Detect the road intersections in the road intersection detection dataset B and obtain a set B containing preliminary road intersection detection results. i , for B i The bounding box in is used for non-maximum suppression and coordinate transformation to filter out the "valid road intersections" and obtain the final intersection detection result set B Geo ;

[0014] Step 2: Extract the road centerline and post-process it, including: obtaining a remote sensing image X with a spatial resolution of 0.3 meters for the area to be processed, cropping the remote sensing image with a sliding window and creating an index, and obtaining a road segmentation dataset C composed of the cropped remote sensing image X; using the road segmentation dataset Rodtrn Train the road segmentation deep learning model to obtain the road segmentation model M Rod ; Use road segmentation model M Rod Extract the road segmentation dataset C and obtain the road centerline set C rod =(y0,y1,...,y k ,...,y n )(where y k represents the road centerline extraction result of the kth sliding window cropped image); according to the index of C rod The elements in y are stitched together. Since the sliding window will cause overlapping areas between the images, in the stitching process, first find the image that is consistent with y. k Adjacent extraction results and their overlapping areas, use the bilinear interpolation algorithm to process the pixels of each extraction result in the overlapping area to obtain the complete road centerline segmentation result C' rod ; Finally, combined with the geographic coordinate range and spatial resolution corresponding to C, C' rod Perform binarization, smoothing, skeleton extraction, and vectorization operations to initially obtain a vectorized road centerline set

[0015] Step 3: Automatically construct the road intersection node arc segment model (topological structure). First, calculate, group and sort the key points; then, traverse the B in step 1. Geo The bounding box of the vectorized road is calculated by calculating the coordinates of the center point of the bounding box and the center line of the bounding box and the vectorized road. The number of intersecting edges Num type And the coordinates of each intersection, further calculate the coordinates of the intersection node; judge Num type Whether it corresponds to the bounding box category, if so, use the bounding box to crop and remove the Then, according to the intersection type and the calculated intersection node coordinates, the corresponding road intersection centerline model is constructed. Finally and Merge them. If they do not correspond, skip the current road intersection. Repeat the above steps until all "valid road intersections" have completed the construction of node arc models, and finally output and save them in the form of vector models.

[0016] 2. Make a model training data set

[0017] Step 4: Create a road intersection detection dataset, including: using 1-8m spatial resolution remote sensing images (all remote sensing images in the present invention are true color satellite remote sensing images composed of green band, blue band and red band) to create a rotating bounding box road intersection target detection dataset Itr trn, including at least three types: "cross", "Y" and "T";

[0018] Step 5: Create a road semantic segmentation dataset, including: Use 0.2-0.6m spatial resolution remote sensing images to create a road segmentation dataset Rod trn . BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0020] Figure 1 A flowchart for the specific implementation of a method for extracting roads from high-resolution remote sensing images taking into account network topological relationships;

[0021] Figure 2 It is a schematic diagram of the center point, intersection point and boundary midpoint of the intersection boundary box;

[0022] Figure 3 Modeling process for intersections;

[0023] Figure 4 Modeling process for T-junction;

[0024] Figure 5 Modeling process for Y-shaped intersection;

[0025] Figure 6 Annotate examples for intersection target boxes;

[0026] Figure 7 Annotation example of samples for segmentation of road surface, road edge line, and road center line. DETAILED DESCRIPTION

[0027] In order to facilitate ordinary technicians in the field to understand and implement the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the implementation examples described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0028] Feature definition:

[0029] For the intersection bounding box set B and the road centerline set C, after completing the modeling of all intersections, the road network G = {V, E} containing topological information is obtained, where V and E represent the nodes and arcs of the road network respectively, and the detailed meanings are as follows:

[0030] ①Intersection bounding box set B:

[0031] B={b1,b2,b3,...,b m}

[0032] Among them, b k Represents the kth intersection bounding box in the intersection bounding box set B. Each intersection bounding box contains the following information:

[0033]

[0034] Where P k Represents the set of geographic coordinates of the corner points of the bounding box of the kth intersection, Represents the coordinates of the i-th corner point of the k-th intersection bounding box (i=1,2,3,4), type k Indicates the type of the kth intersection;

[0035] ②Road centerline set C:

[0036] In the centerline set C, the default initial value of the driving direction of each road section is set to two-way driving. Therefore, the road centerline set C can also be represented as a directed graph C(N,E), where

[0037] N={n|n is the centerline intersection point in the centerline set}

[0038] N is the node set of the center line set C, E is the set of center line segments, and segment e (e∈E) connects two nodes, where the starting node is n. s ∈N and target node n t ∈N; In addition, in the road segment e, the starting node n s and target node n e There are several shape points in between; the attributes of segment e include the starting node n s and target node n t id, longitude and latitude coordinates of two nodes and shape points, length of road section, driving direction of lane;

[0039] ③The total number of edges where the bounding box and the center line geometry intersect edge :

[0040] Assume that Num type Indicates the number of intersections between the centerline set and the bounding box of a specific intersection. The Num type =4, Num of Y-shaped and T-shaped intersections type =3, when Num type When <3, it means that the current intersection does not need to be topologically constructed. We assume that the number of intersection edges between the intersection centerline set and the intersection boundary box is Num edge ;

[0041] like Figure 1 As shown, the present invention discloses a method for extracting roads from high-resolution remote sensing images taking into account network topological relationships, and the specific steps include:

[0042] 1. Extracting road networks with topological relationships

[0043] Step 1: Identify road intersections using the rotating box road intersection target detection dataset Itr trn Train the rotating box target detection deep learning model to obtain the road intersection detection model M Itr , which can be used to detect intersections with different angles and inclinations; obtain a remote sensing image X with a spatial resolution of 0.3 meters in the area to be processed, downsample X, and obtain remote sensing images X0, X1, and X2 with spatial resolutions of 1 meter, 2 meters, and 4 meters, respectively, and record the geographic coordinate ranges of these remote sensing images, set the window size to 512×512 pixels, and the sliding step size in the row and column directions is 256 pixels. The remote sensing images of each spatial resolution are cropped by sliding windows and indexes are created to obtain the road intersection detection dataset B composed of cropped remote sensing images X0, X1, and X2; use model M Itr Detect the road intersections in each image in the road intersection detection dataset B and obtain a preliminary road intersection detection result set B i ; Use the geographic coordinate range and spatial resolution of the remote sensing images X0, X1, and X2 to output each detected target frame, target frame confidence, and target frame type into a vector format, and then use the non-maximum suppression algorithm to remove the target frames with low confidence or located at the edge of the sliding window in the detection results of the remote sensing images with the same spatial resolution to obtain the final detection results The final output result of traversing remote sensing image X2 If the target box in A target frame in or If there is an overlap or inclusion relationship between the target boxes in , the target box is marked as a "valid road intersection" and then traversed If the target box in A target frame and If a target frame in the image overlaps or contains a certain target frame, it is also marked as a "valid road intersection". Finally, all target frames marked as "valid road intersections" are regarded as real road intersections in the area. These target frames and categories are output to obtain the final intersection detection result set B. Geo ;

[0044] In step 1, the non-maximum suppression algorithm is used for the detection results of remote sensing images with the same spatial resolution. The target frames between all windows adjacent to the current sliding window can be suppressed, and the target frames with lower confidence can also be suppressed. The specific implementation process is as follows:

[0045] Assuming the original image is I, the sliding window size is h×h, and the moving step between windows is s, the sliding window method can be expressed as:

[0046] a. Window division: Divide the image I into n windows of equal size, that is:

[0047] I1,I2,...,I n

[0048] Where n is the total number of windows, satisfying:

[0049]

[0050] W and H represent the width and height of the original image I, respectively, in pixels;

[0051] b. Window movement: For each window I i , move its position to get the adjacent window, that is:

[0052] I1,I2,...,I n →I1',I2',...,I n '

[0053] Among them, window I i 'The upper left corner coordinates are:

[0054]

[0055] in, Indicates the number of windows that the image can be divided into horizontally.

[0056] It represents the number of windows that the image can be divided into vertically. mod represents the modulo operation. For example, (i-1)modM represents the remainder of (i-1) divided by M.

[0057] c. Target detection: For each window I i ', and send it to the rotating box target detection model to obtain the score of the target in the window and the location information of the target, that is:

[0058] s i =[s i1 ,s i2 ,...,s in ]

[0059] d i =[(x i1 ,y i1 ,w i1 ,h i1 ,θ i1 ),(x i2 ,yi2 ,w i2 ,h i2 ,θ i2 ),...,(x in ,y in ,w in ,h in ,θ in )]

[0060] Among them, s i Indicates window I i 'The confidence of all target boxes in (x ij ,y ij ) indicates window I i 'The coordinates of the center point of the j-th target box, w ij and h ij Represent the width and height of the target box, θ ij Expressed as the angle of the target box;

[0061] d. Non-maximum suppression of the target box: For all sliding windows I1,I2,...,I n The detected target frame is deduplicated and filtered using the non-maximum suppression method to obtain the final target frame. The non-maximum suppression process for the rotated target frame is different from that for the regular target frame.

[0062] Assume there are two rotating target frames R1 and R2, their position information can be expressed as the center point coordinates x i ,y i , major axis length h i , short axis length w i and the rotation angle θ i To express;

[0063] Define the intersection region R of R1 and R2 int and the union region R uni :

[0064] R int =RBB_Intersect(R1,R2)=R1∩R2

[0065] R uni =RBB_Union(R1,R2)=R1∪R2

[0066] Among them, RBB_Intersect and RBB_Union represent the intersection and union of the rotated target boxes respectively;

[0067] Then calculate the intersection-over-union ratio of the rotated target box, that is, the value of RBB_IoU is:

[0068]

[0069] Area represents the area of ​​the polygon;

[0070] After that, non-maximum suppression is performed on the rotating target box of the sliding window:

[0071] (a) For all sliding windows I1,I2,...,I n , sort all the rotation target frames of category k from high to low according to the corresponding confidence s, and record the sorted rotation target frame list as

[0072] Where m is the number of rotated target boxes of category k, and D1 is the highest confidence score;

[0073] (b) Traverse the target box list D k , for the rotating target box Calculate the IoU value between it and the remaining rotated target frames, and delete the target frames whose IoU value is greater than the threshold T from the target frame list; specifically, suppose the rotated target frame With rotating target box The IoU value is The threshold is T, then delete the target box that meets the following conditions

[0074] RBB_IoU(D i ,D j )>T,j>i

[0075] Finally, the remaining rotated target frames are the set of rotated target frames retained after non-maximum suppression;

[0076] Perform the above steps for each category separately, and finally merge the set of retained rotated target boxes of each category to obtain the non-maximum suppression results of all categories and mark them as "valid road intersections";

[0077] Step 2: Extract road centerline and post-processing, including: obtaining a remote sensing image X with a spatial resolution of 0.3 meters for the area to be processed, setting the window size to 512×512 pixels, and the sliding step size in the row and column directions to 256 pixels, cutting the remote sensing image into sliding windows and creating an index, and obtaining a road segmentation dataset C composed of the cut remote sensing image X; using the road segmentation dataset Rod trn Train the road segmentation deep learning model to obtain the road segmentation model M Rod ; Use road segmentation model M Rod Extract the road segmentation dataset C and obtain the road centerline set C rod =(y0,y1,...,y k ,...,y n )(where y krepresents the road centerline extraction result of the kth sliding window cropped image); according to the index of C rod The elements in y are stitched together. Since the sliding window will cause overlapping areas between the images, in the stitching process, first find the image that is consistent with y. k Adjacent extraction results and their overlapping areas, use the bilinear interpolation algorithm to process the pixels of each extraction result in the overlapping area to obtain the complete road centerline segmentation result C' rod ; Finally, combined with the geographic coordinate range and spatial resolution corresponding to C, C' rod Perform binarization, smoothing, skeleton extraction, and vectorization operations to initially obtain a vectorized road centerline set

[0078] In step 2, the specific implementation process of the bilinear interpolation algorithm is:

[0079] Since a sliding window with a step size of 256 pixels is used to crop remote sensing images, it is assumed that a 256×512 pixel overlapping area S will be generated between the two adjacent images S1 and S2. 12 , an overlapping area S of 512×256 pixels will be generated between the two adjacent images S1 and S3. 13 , for the overlapping area S 12 For each pixel (x, y) in the image, its pixel value can be calculated by following the steps below:

[0080] a. Calculate the offset dx,dy of S1 relative to S2:

[0081]

[0082] in, represents the pixel position in S1, represents the pixel position in S2;

[0083] b. Calculate the overlap area S of S1 and S2 12 Weight:

[0084]

[0085] c. Take the pixel values ​​from S1 and S2 and perform weighted average according to the weights to get the final pixel value:

[0086]

[0087] Similarly, for the overlapping area S between S1 and S3 13 , we use the same calculation method to first calculate the offset and weight value of S1 relative to S3, and finally get the pixel value of the overlapping area:

[0088]

[0089] Since a sliding window with a step size of 256 pixels is used to crop the remote sensing image in both the row and column directions, the four segmentation results S1, S2, S3, and S4 are all binary matrices of size H×H. They have the same overlapping area. The road centerline results of the overlapping area are fused to obtain a new segmentation result S:

[0090] a. Determine the location and size of the overlapping area, represented by R, where R is a 256×256 matrix;

[0091] b. For each pixel (x, y) in the overlapping area, calculate its position and weight in the four segmentation results. Assume that the pixel position in the i-th segmentation result is (x i ,y i ), then the offset relative to the segmentation result can be expressed as:

[0092]

[0093] Calculate the weight of the pixel in the four segmentation results:

[0094] w i =(1-dx i )×(1-dy i ),i=1

[0095] w i =dx i ×(1-dy i ),i=2

[0096] w i =(1-dx i )×dy i ,i=3

[0097] w i =dx i ×dy i ,i=4

[0098] c. For each pixel (x, y) in the overlapping area, take the pixel values ​​at that position from the four segmentation results and perform weighted averaging according to their weights to obtain the final bilinear mixed pixel value:

[0099] S(x,y)=w1×S1(x,y)+w2×S2(x,y)+w3×S3(x,y)+w4×S4(x,y)

[0100] Among them, S irepresents the pixel value in the i-th segmentation result; for pixels outside the overlapping area, directly take the pixel value from the corresponding segmentation result into S;

[0101] In step 2, raster road vectorization includes binarization and smoothing of the spliced ​​road centerlines, using a skeleton extraction algorithm to extract the road skeleton of the entire area, and finally using a topology-based tracking method to vectorize the road to obtain a road centerline vector map that does not contain any topological information;

[0102] The specific process of extracting the road skeleton of the entire area using the skeleton extraction algorithm based on branch points is as follows:

[0103] a. Input binary image: I(x,y), where x,y∈Z represents the row and column numbers of the binary image, I(x,y)∈{0,1};

[0104] b. Initialize an empty set S to save skeleton pixels:

[0105] c. Calculate branch points: For all x, y∈Z, if I(x, y) = 1 and N(x, y) ≥ 3, then (x, y) is a branch point, where N(x, y) is the number of foreground pixels in the 8-neighborhood of pixel I(x, y), that is, if the value of a pixel in the 8 directions of the current pixel, i.e., above, below, left, right, upper left, upper right, lower left, and lower right, is 1, then the pixel is regarded as a foreground pixel in the 8-neighborhood;

[0106] d. Skeleton extraction: For each branch point (x, y), search along the connected path of the foreground pixels. When reaching another branch point or the end point, stop searching. Add the pixels (x', y') traversed during the search to S:

[0107] S = S ∪ {(x', y') | (x', y') is the foreground pixel searched from (x, y)}

[0108] e. Output: Returns a set S containing skeleton pixels;

[0109] The specific process of using the topology-based tracking method to vectorize roads is as follows:

[0110] a. Take the extracted skeleton S(x,y) as the input image, S(x,y)∈{0,1}, where 1 is foreground pixel and 0 is background pixel;

[0111] b. Calculate the connected components. For each foreground pixel (x, y), mark all adjacent foreground pixels as the same connected component, and obtain multiple connected components C1, C2, C3, ..., C n ;

[0112] c. Initialize the vector shape and create an empty vector shape G;

[0113] d. Vectorization, returning the vector graphics G;

[0114] Step 3: Automatically construct the road intersection node arc segment model (topological structure). First, calculate, group and sort the key points; then, traverse the B in step 1. Geo The bounding box of the vectorized road is calculated by calculating the coordinates of the center point of the bounding box and the center line of the bounding box and the vectorized road. The number of intersecting edges Num type And the coordinates of each intersection, further calculate the coordinates of the intersection node; judge Num type Whether it corresponds to the bounding box category, if so, use the bounding box to crop and remove the Then, according to the intersection type and the calculated intersection node coordinates, the corresponding road intersection centerline model is constructed. Finally and Merge them. If they do not correspond, skip the current road intersection. Repeat the above steps until all "valid road intersections" have completed the construction of node arc models, and finally output and save them in the form of vector models.

[0115] In step 3, the specific implementation process of key point calculation, grouping and sorting is as follows:

[0116] a. Orientation determination of each edge of the intersection bounding box:

[0117] (a) Assume that the current bounding box is the kth intersection b k , and its four corner points are simplified as

[0118] {p1,p2,p3,p4},p i =(x i ,y i ), the four edges of the intersection boundary box can be defined as the lines between two consecutive corner points, such as: edge1 = {p1, p2}, edge2 = {p2, p3}, edge3 = {p3, p4} and edge4 = {p4, p1}, and the midpoint coordinates of each edge are {ct1, ct2, ct3, ct4};

[0119] (b) Use the diagonal point coordinates of the intersection bounding box to calculate the center point coordinates of the current intersection bounding box. o ,y o );

[0120] (c) Find the edge i The normal vector

[0121] like Figure 2 The diagram is a diagram of the intersection, center point and midpoint of the target box boundary. The center point of the intersection boundary box is used as the starting point for normal vector calculation. The vector formed by the center point of the boundary box and the midpoint of each edge is used as the normal vector of the current edge. The edge of the intersection boundary box is i The normal vector is

[0122] They respectively represent the direction from the center point o to the edge i The center point c i A vector of , where:

[0123]

[0124] (d) Determine edge i The orientation d relative to the bounding box i :

[0125] if and Then the judgment edge is on the right side of the rectangular box, that is, facing east;

[0126] if and Then the judgment edge is on the left side of the rectangular box, that is, facing west;

[0127] if and Then the judgment edge is located on the upper side of the rectangular frame, that is, facing north;

[0128] if and Then the judgment edge is located at the lower side of the rectangular frame, that is, facing south;

[0129]

[0130] b. Get the intersection point set between the intersection bounding box and the road centerline:

[0131] Assume that the intersection point set between the intersection bounding box set B and the road centerline set C is BC, then

[0132] BC={bc1,bc2,...,bc m}

[0133] Where m is the number of intersection bounding boxes, assuming that the kth intersection bounding box b in the intersection bounding box set B k The set of intersections with the road centerline set C is bc k ,in

[0134]

[0135] in Represents the edge of the intersection bounding box i The intersection point set with the road centerline set C, h is b k The number of intersections between the road centerline set C. Assuming that the number of intersections between the edges edge1, edge2, edge3, and edge4 of the intersection bounding box and the centerline set C are n1, n2, n3, and n4 respectively, the intersection set contained in the i-th edge of the intersection bounding box is:

[0136]

[0137] Indicates bc k The j-th intersection of

[0138] c. BC according to the edge of the bounding box k Sort the elements in:

[0139] (a) Set the intersection points on each edge According to the direction of the edge d i Sort by north, east, south, and west;

[0140] (b) All the intersection points belonging to the same edge Sort by the following rules:

[0141] If edge i The corresponding direction is north or south, for edge i The intersection of The elements in are sorted from west to east, that is, they are sorted from small to large according to the horizontal coordinates of the elements;

[0142] If edge i The corresponding direction is east or west, for edge i The intersection of The elements in are sorted from south to north, that is, they are sorted from small to large according to the vertical coordinates of the elements;

[0143] (c) bc k Every intersection point As d i The index of the set is represented as In order to clearly determine the specific location of the current intersection on the intersection boundary box during the subsequent intersection topology construction process;

[0144] In step 3, the number and position relationship of the intersection points between the vector road centerline and the target frame are determined as follows: a. Figure 3 As shown, the center line of the crossroad should ensure that there are 4, 6 or 8 intersections with the target frame (i.e., Numtype =4 or Num type =6 or Num type =8), and the intersection points are distributed on four sides, then the node coordinates of the crossroads can be calculated;

[0145] (a)Num type =4 o'clock crossroads:

[0146] like Figure 3 In (a1), two auxiliary lines are drawn to connect the points on the opposite sides of the intersection boundary box. The intersection point c1 of the auxiliary lines can be used as the node n1 of the current intersection. The specific calculation process is expressed as follows: c1 is the auxiliary line and The intersection point between them, assuming c1=(x c1 ,y c1 ), by c1, The three points are collinear, so we can get the collinear equation By c1,

[0147] The three points are collinear, so we can get the collinear equation Finally, calculate x c1 and c1 , then the node coordinates can be expressed as n1=c1=(x c1 ,y c1 );

[0148] (b)Num type =6 o'clock crossroads:

[0149] With Num type =4, the node coordinate calculation method is the same. Figure 3 In (b1), auxiliary lines are drawn by connecting the intersection points on the opposite sides of the intersection boundary box. The two intersection points c1 and c2 generated between the three auxiliary lines are calculated and c1 and c2 are used as the two nodes n of the current intersection. w1 、n e1 , the node coordinates are represented by n w1 =c1=(x c1 ,y c1 ), n e1 =c2=(x c2 ,y c2 ); Figure 5 In (b2), similar to (b1), the node coordinates are expressed as n s1 =c1=(x c1 ,y c1 ),

[0150] n n1 =c2=(x c2 ,y c2 );

[0151] (c)Num type = Crossroads at 8 o'clock:

[0152] like Figure 3 In (d1), connect the intersection points on the opposite sides of the intersection boundary box to make 4 auxiliary lines, and calculate the coordinates of the intersection points c1, c2, c3, c4 of the auxiliary lines, and the corresponding node is n n1 、n s1 、n n2 、n s2 ; and Num type =4, the node coordinates are calculated in the same way, and the node coordinates are expressed as n s1 =c1=(x c1 ,y c1 ),

[0153] n e1 =c2=(x c2 ,y c2 ), n s2 =c3=(x c3 ,y c3 ), n e2 =c4=(x c4 ,y c4 );

[0154] b. Figure 4 and Figure 5 As shown in the figure, there are high similarities between the T-shaped intersection and the Y-shaped intersection. There are three intersections between the bounding box and the center line of both the T-shaped intersection and the Y-shaped intersection, and in most cases the intersections are distributed on the three sides of the intersection bounding box. The difference is that the angle formed by the intersection of the road centerline and the bounding box at the T-shaped intersection is close to a right angle, while at least one of the angles formed by the intersection of the road centerline and the bounding box at the Y-shaped intersection is not close to a right angle.

[0155] like Figure 4 , the number of intersection points on the T-shaped intersection boundary box is 3, 4, 5, and 6, that is, Num type =3, 4, 5, 6; according to the sorted intersection set bc k middle The corresponding direction d i ,exist Figure 6 Medium, d i ='South', that is, there is a topological relationship between the east, west and north roads of the intersection boundary box; determine the number of intersections Num type , calculate the node coordinates;

[0156] (a)Num type =3 T-junction:

[0157] like Figure 4 In (a1), connect the points on the opposite sides of the intersection boundary box to make an auxiliary line, and calculate the coordinates of the midpoint c1 of the auxiliary line as the node n1 of the current intersection, using direction d i The value of The index condition of the auxiliary line is calculated by the coordinates c1 of the midpoint of the auxiliary line. Let:

[0158]

[0159] in, Indicates that the intersection is at bc k The i-th direction in is 'West', Indicates that the intersection is at bc k The i-th direction in is 'East', and the coordinates of c1 and node n1 are calculated:

[0160]

[0161] (b)Num type =4 T-junction:

[0162] like Figure 4 In (b1), and Num type =3, draw an auxiliary line connecting the intersection points on the opposite sides of the intersection boundary box, and calculate the coordinates of the midpoint c1 of the auxiliary line. Calculate the distance D between the two intersection points on the north intersection boundary box in the figure, and then draw a line of length 1 along both ends of the auxiliary line based on the coordinates of the midpoint c1 of the auxiliary line. The extension line has two ends. The coordinates are the node coordinates of the intersection. The detailed calculation process is as follows:

[0163]

[0164] where i = 1, 2

[0165] in, Indicates in bc k The i-th intersection point with direction 'North';

[0166] calculate and The distance D between

[0167]

[0168] calculate The slope of the line between them is k1,

[0169]

[0170] Calculate the coordinates of c1,

[0171]

[0172] make but The coordinates of

[0173]

[0174] When the slope k1 = 0,

[0175] (c)Num type = T-junction at 5 o'clock:

[0176] like Figure 4 In (c1), connect the intersection points on the opposite sides of the intersection boundary box to make two auxiliary lines in the order from south to north, and calculate the coordinates of the midpoints of the auxiliary lines c1 and c2, and compare them with Num type =3 is similar to the T-shaped intersection, the two nodes The position is consistent with c1 and c2, and the detailed coordinates are:

[0177]

[0178] (d)Num type =6 o'clock T-shaped intersection:

[0179] like Figure 4 In (d1), connect the intersection points on the opposite sides of the intersection boundary box to make two auxiliary lines in the order from south to north, and calculate the coordinates of the midpoints of the auxiliary lines c1 and c2, and compare them with Num type =4, the distance D between the two intersection points on the northbound intersection boundary box in the figure is calculated, and then a distance of length 1 is made along both ends of the auxiliary line based on the midpoint coordinates c1 and c2 of the two auxiliary lines. The extension line, the four ends of the extension line The coordinates of are the node coordinates of the intersection;

[0180] in The calculation method of the coordinates is the same as Num type =4 when computing nodes The method of calculating the coordinates of is exactly the same; When the coordinates of The slope of the line between them is k2, and the remaining calculation steps are Num type =4, the method of calculating node coordinates is similar;

[0181] c. Figure 5The number of intersection points on the boundary box of the Y-shaped intersection is 3, 4, 5, and 6, which is similar to the T-shaped intersection. However, when constructing auxiliary lines, it is necessary to consider the impact of this non-vertical intersection on the calculation of node positions, expressed as Num type =3 as an example to introduce the node calculation: For the nodes of the Y-shaped intersection, it is only required to take the intersection of the extension line of the slope of the road centerline at the intersection of the centerline and the intersection boundary box and the auxiliary line as the node of the current intersection, such as Figure 5 The intersection of (a1), (a2), and (a3) The auxiliary line intersects with the center line of the road at n1. The specific calculation method is as follows:

[0182]

[0183] in, Indicates that the intersection is at bc k The i-th direction in is 'West', Indicates that the intersection is at bc k The i-th direction in is 'East';

[0184] For point Sample another point 3 meters outside the intersection boundary box along the center line. Assume that the point is

[0185] Then n1 represents the passing point and The straight line and the and The intersection of the straight lines establishes the following equation:

[0186]

[0187] Then solve and

[0188] like Figure 5 In the cases (b1) and (d1), we first find and The nodes on the line are and The coordinates of the connecting line are solved according to the distance from the two nodes in the T shape to the center point, so as to obtain the node coordinates in the entire intersection;

[0189] 2. Make a model training data set

[0190] Step 4: Use 1-8m spatial resolution remote sensing images (all remote sensing images in the present invention are true color satellite remote sensing images composed of green band, blue band and red band) to create a rotating bounding box road intersection target detection dataset Itr trn, including at least three types of intersections: "cross", "Y" and "T", and the number of samples in each category should be as consistent as possible; among them, the example of mapping the three types of intersections in remote sensing images is shown in Figure 6 ,When annotating the target box, it is necessary to keep the boundary of the target box parallel to the edge line of the road as much as possible;

[0191] Step 5: Use 0.2-0.6m spatial resolution remote sensing images to create a road segmentation dataset Rod trn , including three types of segmentation samples: road surface, road edge line, and road center line; see the example of making a road segmentation dataset Figure 7 When marking the road surface, road edge line, and road center line of the road segmentation dataset, it should be noted that the road edge line is the edge of the road surface sample, and the road center line should be located between the road edge line and the road surface. The width of the road edge line and the road center line in the sample image is 1 pixel.

Claims

1. A method for extracting roads from high-resolution remote sensing images taking into account network topology relationships, characterized in that: The following steps are involved:

1. Extracting road networks with topological relationships Step 1: Identify road intersections, including: Using the rotating box road intersection target detection dataset Itr trn Train the rotating box target detection deep learning model to obtain the road intersection detection model M Itr ; Get a remote sensing image X with a spatial resolution of 0.3 meters for the area to be processed, downsample X to obtain remote sensing images X0, X1, and X2 with spatial resolutions of 1 meter, 2 meters, and 4 meters, respectively, and record the geographic coordinate ranges of these remote sensing images. Sliding window crop the remote sensing images of each spatial resolution and create an index to obtain a road intersection detection dataset B composed of cropped remote sensing images X0, X1, and X2; Use the road intersection detection model M Itr Detect the road intersections in the road intersection detection dataset B and obtain a set B containing preliminary road intersection detection results. i , for B i The bounding box in is used for non-maximum suppression and coordinate transformation to filter out "valid road intersections" and obtain the final intersection detection result set B Geo ; Step 2: Extract the road centerline and post-process it, including: obtaining a remote sensing image X with a spatial resolution of 0.3 meters for the area to be processed, cropping the remote sensing image with a sliding window and creating an index, and obtaining a road segmentation dataset C composed of the cropped remote sensing image X; using the road segmentation dataset Rod trn Train the road segmentation deep learning model to obtain the road segmentation model M Rod ; Use road segmentation model M Rod Extract the road segmentation dataset C and obtain the road centerline set C rod =(y0,y1,...,y k ,...,y n )(where y k represents the road centerline extraction result of the kth sliding window cropped image); according to the index of C rod The elements in y are stitched together. Since the sliding window will cause overlapping areas between the images, in the stitching process, first find the image that is consistent with y. k Adjacent extraction results and their overlapping areas, the pixels of each extraction result in the overlapping area are processed using the bilinear interpolation algorithm to obtain the complete road centerline segmentation result C r ' od ; Finally, combined with the geographic coordinate range and spatial resolution corresponding to C, r ' od Perform binarization, smoothing, skeleton extraction, and vectorization operations to initially obtain a vectorized road centerline set Step 3: Automatically construct the road intersection node arc segment model (topological structure). First, calculate, group and sort the key points; then, traverse the B in step 1. Geo The bounding box of the vectorized road is calculated by calculating the coordinates of the center point of the bounding box and the center line of the bounding box and the vectorized road. The number of intersecting edges Num type And the coordinates of each intersection, further calculate the coordinates of the intersection node; judge Num type Whether it corresponds to the bounding box category, if so, use the bounding box to crop and remove the Then, according to the intersection type and the calculated intersection node coordinates, the corresponding road intersection centerline model is constructed. Finally and Merge them. If they do not correspond, skip the current road intersection. Repeat the above steps until all "valid road intersections" have completed the construction of node arc models, and finally output and save them in the form of vector models.

2. Make a model training data set Step 4: Create a road intersection detection dataset, including: using 1-8m spatial resolution remote sensing images (all remote sensing images in the present invention are true color satellite remote sensing images composed of green band, blue band and red band) to create a rotating bounding box road intersection target detection dataset Itr trn , including at least three types: "cross", "Y" and "T"; Step 5: Create a road semantic segmentation dataset, including: Use 0.2-0.6m spatial resolution remote sensing images to create a road segmentation dataset Rod trn .

Images