A regularized road extraction method based on convolution and graph neural network
By combining convolutional neuron networks and graph neuron networks, open source data is used for road pavement extraction, which solves the problems of incomplete and irregular extraction in complex scenarios, and realizes regular road pavement and centerline correction, achieving close to human drawing Level effect.
Patent Information
- Application Number
- CN202111474496.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-03
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2041-12-03
AI Technical Summary
The prior art is difficult to achieve regular and complete road surface extraction in complex scenarios, especially due to occlusion problems caused by the influence of light, which leads to incomplete and irregular extraction results, which are far lower than the human drawing level.
Using a framework based on convolutional neuron network and graph neuron network, combined with open source road centerline data, transforms the regular width prediction problem of road extraction into centerline nodes, and extracts advanced features through convolutional neuron network and graph neurons. The network combines to achieve end-to-end regular road surface extraction and centerline offset correction.
The regular road surface extraction is achieved close to the level of human drawing, which improves the irregularity and incompleteness of the segmentation method, and can correct the offset position of the road center line.
Smart Images

Figure CN114399509B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a regularized road extraction method based on convolution and graph neural network, which can realize the automatic extraction of regularized road pavement, effectively improve the integrity and regularity of road extraction results, and can partially replace the production work of road pavement. It can be applied to geographic information updating, urban planning, autonomous driving and other fields. Background Art
[0002] As a basic geographic data, regular road surface maps have important and extensive applications in urban planning, geographic information update, car navigation and autonomous driving. However, despite the rapid development of earth observation technology and deep learning technology in recent years, the construction and update of road surface maps is still a task that relies on manual operation.
[0003] In recent years, many deep learning road pavement extraction methods based on high-resolution remote sensing images have been proposed. Most of these methods are semantic segmentation methods. Although they have achieved remarkable results, in complex scenes, due to the influence of lighting, trees, buildings and shadows in remote sensing images will block the road. They usually produce poorly connected, incomplete and irregular extraction results, which are far below the level of human drawing. At present, there is no method that can extract regular, complete and human-level road pavement. Summary of the invention
[0004] In order to solve the bottleneck of the current road extraction problem and realize fully automatic extraction of road pavement close to manual drawing, on the one hand, some easily accessible road-related data can be collected and used as a supplement, and on the other hand, an effective regularization method needs to be applied. The development of open source road centerline data such as OpenStreetMap and graph neural network has made it possible to solve the above problems.
[0005] Therefore, in order to solve the problems of incomplete and irregular extraction results of the current mainstream segmentation methods in road extraction, it is necessary to combine open source data and regularization methods to study a new road extraction framework structure to achieve fully automatic extraction of road pavement close to the level of human drawing.
[0006] In view of the shortcomings of the prior art, the present invention provides a framework for regular road surface extraction from remote sensing images based on convolutional neural networks and graph neural networks. By making full use of open source road centerline data and introducing graph neural networks into regularized road width regression, the road surface extraction problem is transformed into a regularized width prediction problem of centerline nodes. For the first time, end-to-end extraction of regularized road surfaces reaching the human drawing level is achieved, and the offset centerline position can be corrected at the same time.
[0007] The technical solution used to achieve the purpose of the present invention is a regularized road extraction method based on convolution and graph neural network, comprising the following steps:
[0008] Step 1: Generate an equidistant vector road centerline map structure based on a given image and a corresponding road centerline annotation file, and calculate the width information of the road nodes;
[0009] Step 2: Using the road width samples and centerline vector information (i.e., centerline graph structure) constructed in step 1, the entire image area is divided into sub-graphs, and the regularized width prediction network composed of convolutional neural networks and graph neural networks is trained in batches to predict the left and right width information of each road node on the centerline. The convolutional neural network extracts high-level abstract features and road boundary features of each node, and concatenates them with the node position features to form the feature vector of each node as the input of the graph neural network; the graph neural network propagates and aggregates node feature information in the centerline graph structure, optimizes the node features as a whole, and finally predicts the left and right regularized width information of each node;
[0010] Step 3, using the left and right width information of the road nodes and the centerline graph structure obtained in step 2, a regular road surface is generated, and the offset centerline is corrected to the center position.
[0011] Furthermore, the specific implementation of step 1 includes the following sub-steps:
[0012] Step 1.1, centerline vectorization: vectorize the road centerline annotations corresponding to the remote sensing image into a vector centerline graph structure consisting of nodes and lines, and resample the nodes to be equidistant;
[0013] Step 1.2, calculate the node direction vector: calculate the node direction vector according to the node degree. If the node degree is 1, that is, there is only one neighbor node, then the direction is calculated based on the neighbor node and the node; if the node degree is 2, that is, there are two neighbor nodes, then the direction is calculated based on the two neighbor nodes; if the node degree is greater than 2, that is, there are three or more neighbor nodes, then the direction is not calculated;
[0014] Step 1.3, calculate the left and right widths of the node: According to the node direction calculated in step 1.2, for nodes with a degree less than or equal to 2, the node vertical direction can be obtained. The vertical direction intersects with the corresponding road surface label to obtain the left and right widths of the node; for nodes with a degree greater than 2, no processing is performed.
[0015] Furthermore, the specific implementation of step 2 includes the following sub-steps:
[0016] Step 2.1, construct the centerline subgraph: Due to the limitation of GPU, all nodes in the image cannot be input into the network at one time. It is necessary to construct a subgraph based on the vector centerline graph generated in step 1.1. A random starting point is selected from the nodes of the centerline graph. Starting from this starting point, a centerline subgraph is constructed by random depth-first or breadth-first search for a quantitative number of nodes (such as 64 or 128) as a batch of the input network. The next random starting point selection skips the processed nodes until the entire centerline graph is covered, that is, the entire image is processed;
[0017] Step 2.2, crop the node block: according to the centerline sub-graph constructed in step 2.1 and the node direction calculated in step 1.2, crop the image with a size of 256×256 pixels and the centerline with the node position as the center, and rotate it to the vertical direction so that the road edge is located on the left and right sides of the node;
[0018] Step 2.3, predict node width: input the node image and center line block produced in step 2.2 into the proposed regularized width prediction network to predict the width values on the left and right sides of the node.
[0019] Furthermore, the regularized width prediction network includes a convolutional neural network feature extraction part and a graph neural network width regularization regression part. Among them, the main part of the convolutional neural network feature extraction includes eleven consecutively stacked convolutional layers and three fully connected layers connected thereto. The convolution kernel size of the first convolutional layer is 5×5, with a step size of 2; the convolution kernel size of the second, fourth, sixth, eighth, and tenth convolutional layers is 3×3, with a step size of 1; the convolution kernel size of the third, fifth, seventh, ninth, and eleventh convolutional layers is 3×3, with a step size of 2; at the same time, a convolution layer with a convolution kernel of 3×3 and a step size of 1 is connected to the fourth layer, and a rough road edge extraction result is generated through a Softmax activation function, forming a branch part of the convolutional neural network. In the main part of the convolutional neural network, a 64-dimensional node high-order feature vector is generated through three fully connected layers; the branch part of the convolutional neural network generates a 62-dimensional node edge feature vector through three fully connected layers; the position of the node in the input block is used as a 2-dimensional position feature vector, which is concatenated with the 64-dimensional node high-order feature vector and the 62-dimensional node edge feature vector to generate a 128-dimensional feature vector for each node as the input feature of each node in the graph neural network.
[0020] Furthermore, the graph neural network includes three GAT layers. The first layer has four attention heads, which convert the 128-dimensional feature vector into a 64-dimensional vector; the second layer has four attention heads, which convert the 64×4-dimensional vector into an 8-dimensional vector; the last layer has only one attention head, which converts the 8×4-dimensional vector into a 4-dimensional vector, representing the position coordinates of the edge nodes on both sides of the node, and the width of the two sides of the node can be obtained through the coordinates. There are two types of centerline graph structures for the input graph neural network. Both graph structures are generated by the centerline vector graph structure obtained in step 1.1. They share the same set of nodes, but the connection relationship of the edges is different. The first graph structure assumes that information propagation only occurs on the same logical road. The same logical road means that the angle between adjacent nodes on the road is less than 30°. The second graph structure assumes that the information propagation between nodes is limited to between two adjacent endpoints or intersections, that is, between road sections.
[0021] Furthermore, the loss function of the regularized width prediction network includes two parts: a rough road edge prediction mean square error loss function and a final width regression mean square error loss function. The weights of these two parts are set to 100 and 1 according to experience.
[0022] Furthermore, the specific implementation of step 3 includes the following sub-steps:
[0023] Step 3.1, generate regularized road surface: According to the road centerline vector and the left and right widths of the nodes predicted in step 2, the left and right boundary node positions of each node can be obtained. For the intersection node, no prediction is made in step 2. The width information is obtained by calculating the average width of the neighboring nodes around it and determining the edge node position in each direction corresponding to the neighboring node. Connect the edge nodes in sequence to obtain the regularized road surface map;
[0024] Step 3.2, correct the offset center line: obtain the center position of the road node by arithmetic averaging the left and right widths of the node predicted in step 2, and then obtain the corrected center line.
[0025] Compared with the prior art, the present invention has the following advantages and beneficial effects:
[0026] This paper proposes a new framework for regularized road surface extraction (and centerline correction) from remote sensing images and existing road centerline data, transforming the road surface extraction task into width inference in a graph structure pre-built from road centerlines. This is the first regularized road surface extraction framework based on graph neural networks.
[0027] The proposed road extraction framework consists of a convolutional neural network and a graph neural network. The front-end convolutional neural network extracts both high-level abstract features for subsequent graph neural network regression and boundary features for identifying roads from the background. The back-end graph neural network propagates and aggregates high-level features in the road centerline vector map to achieve global optimization prediction of road width and regularized road pavement extraction, improving the limitations of irregular and incomplete segmentation results.
[0028] The method proposed by the present invention can perform post-processing on the prediction of the width of both sides of each road vertex and realize the center line offset correction at the same time. BRIEF DESCRIPTION OF THE DRAWINGS
[0029] Figure 1 It is the overall framework diagram of the present invention.
[0030] Figure 2 This is a diagram of the regularized width prediction network structure in the present invention.
[0031] Figure 3 It is a schematic diagram of the graph structure of the input graph neural network in the present invention.
[0032] Figure 4 It is a schematic diagram of drawing a regularized road surface and correcting a deviated center line in the present invention. DETAILED DESCRIPTION
[0033] The specific implementation of the present invention is described below through examples and drawings.
[0034] The embodiment of the present invention provides a regularized road extraction method based on a convolutional graph neural network, comprising the following steps:
[0035] Step 1: Create a road width sample library based on the existing images and the corresponding road surface and centerline annotation files, generate an equidistant vector road centerline map structure, and calculate the width information of the road nodes;
[0036] Step 2: Using the road width samples and centerline vector information constructed in step 1, the entire image area is divided into sub-graphs and a regularized width prediction network composed of a convolutional neural network and a graph neural network is trained in batches to obtain the left and right width information of each node.
[0037] Step 3, using the left and right width information of the road nodes and the centerline graph structure obtained in step 2, a regular road surface is generated, and the offset centerline is corrected to the center position.
[0038] Furthermore, the specific implementation of step 1 includes the following sub-steps:
[0039] Step 1.1, centerline vectorization. The road centerline annotations corresponding to the remote sensing image are vectorized into a vector centerline graph structure consisting of nodes and lines, and the nodes are resampled to be equidistant to facilitate the subsequent information propagation in the graph neural network.
[0040] Step 1.2, calculate the node direction vector. Calculate the node direction vector based on the node degree. If the node degree is 1, that is, there is only one neighbor node, then the direction is calculated based on the neighbor node and the node; if the node degree is 2, that is, there are two neighbor nodes, then the direction is calculated based on the two neighbor nodes; if the node degree is greater than 2, that is, there are three or more neighbor nodes, then the direction is not calculated.
[0041] Step 1.3, calculate the left and right width of the node. For nodes with a degree less than or equal to 2 in the road centerline graph, the node vertical direction can be obtained according to the node direction calculated in step 1.2. The left and right width of the node can be obtained by intersecting the vertical direction with the corresponding road surface label; for nodes with a degree greater than 2 in the road centerline graph, no processing is performed.
[0042] Furthermore, the specific implementation of step 2 includes the following sub-steps:
[0043] Step 2.1, construct the centerline sample subgraph. Due to the limitation of GPU, all nodes in the image cannot be input into the network at one time, so it is necessary to construct the sample subgraph according to the vector centerline graph generated in step 1.1. By selecting a random starting point from the nodes of the centerline graph, starting from this starting point, a random depth-first or breadth-first search is performed to quantitatively search the number of nodes (such as 64, 128) to form a centerline sample subgraph as a batch of the input network. The next time a random starting point is selected, the processed nodes are skipped, and the next batch of subgraphs are obtained through random depth-first or breadth-first search, and so on, until all nodes of the entire centerline graph are covered, that is, the entire image is processed.
[0044] Step 2.2, crop the node block, including the image, centerline map and road edge label corresponding to the node. Based on the centerline subgraph constructed in step 2.1 and the node direction calculated in step 1.2, crop the image, centerline and road label block with a size of 256×256 pixels with the node position as the center, and rotate it to the vertical direction so that the road edge is located on the left and right sides of the node to facilitate the network to identify the direction; process the road label block, resample it to one-fourth of the original size, that is, 64×64 pixels, and use the canny operator to extract the edge to generate the road edge label block.
[0045] Step 2.3, training the regularized width prediction network. Input the node image and centerline block produced in step 2.2 into the regularized width prediction network, and supervise the training of the regularized width prediction network with the left and right widths generated in step 1.3 and the road edge label block generated in step 2.2 to obtain the left and right width values of the input node.
[0046] The above-mentioned regularized width prediction network consists of a convolutional neural network feature extraction part and a graph neural network regularized width regression part. Among them, the convolutional neural network feature extraction includes a main part and a branch part. The main part includes eleven layers of continuously stacked convolutional layers and three fully connected layers connected afterwards. Each convolutional layer uses batch normalization to speed up the training speed. The convolution kernel size of the first convolutional layer is 5×5, and the step size is 2; the convolution kernel size of the second, fourth, sixth, eighth, and tenth convolutional layers is 3×3, and the step size is 1; the convolution kernel size of the third, fifth, seventh, ninth, and eleventh convolutional layers is 3×3, and the step size is 2; at the same time, a convolution layer with a convolution kernel of 3×3 and a step size of 1 is connected to the fourth layer, and a rough road edge extraction result is generated through a Softmax activation function, which constitutes the branch part of the convolutional neural network. The main part of the convolutional neural network passes through three fully connected layers to generate a 64-dimensional node high-order feature vector; the branch part of the convolutional neural network passes through three fully connected layers to generate a 62-dimensional node edge feature vector; the position of the node in the input block is used as a 2-dimensional position feature vector, which is concatenated with the 64-dimensional node high-order feature vector and the 62-dimensional node edge feature vector to generate a 128-dimensional feature vector for each node as the input feature of each node in the graph neural network.
[0047] The centerline graph structure of the input graph neural network in the present invention includes two types, namely, the logical graph structure and the road segment graph structure. Both graph structures are generated by the centerline vector graph structure obtained in step 1.1. They share the same node set, but the edge connection relationship is different. The first logical graph structure assumes that the turning angle between nodes of a logically identical road is less than 30°, and assumes that information propagation is only carried out on the same logical road, that is, only logically connected road nodes have a connection relationship in the graph structure; the second road segment graph structure assumes that the information propagation between nodes is limited to two adjacent endpoints or intersections, that is, only road nodes in the same road segment have a connection relationship in the graph structure.
[0048] The regularized width regression part of the graph neural network includes three layers of GAT [1]Layer, propagates and aggregates feature information between nodes in the above two graph structures. The first layer has four attention heads, each of which converts the input 128-dimensional feature vector into a 64-dimensional vector, and finally concatenates the results of each attention head; the second layer has four attention heads, each of which converts the 64×4-dimensional vector into an 8-dimensional vector, and finally concatenates the results of each attention head; the last layer has only one attention head, which converts the 8×4-dimensional vector into a 4-dimensional vector, representing the position coordinates of the edge nodes on both sides of the node, through which the width of both sides of the node can be obtained. The loss function of the network includes two parts: a rough road edge prediction mean square error loss function and a final width regression mean square error loss function. The weights of these two parts are set to 100 and 1 according to experience.
[0049] [1] P. G.Cucurull, A.Casanova et al., “Graph attentionnetworks,” arXiv preprint arXiv:1710.10903, 2017.
[0050] Furthermore, the specific implementation of step 3 includes the following sub-steps:
[0051] Step 3.1, generate a regularized road surface. For nodes with a degree less than or equal to 2 in the centerline graph, the road left and right boundary node positions corresponding to each node can be obtained based on the road centerline vector and the left and right widths of the node predicted in step 2. For intersection nodes, no prediction is made in step 2. The width information is obtained by calculating the average width of the neighboring nodes around it and determining the edge node position in each direction corresponding to the neighboring nodes. Connect the obtained edge nodes in sequence to obtain a regularized road surface graph.
[0052] Step 3.2, correct the offset center line. The center position of the road node is obtained by arithmetic averaging the left and right widths of the node predicted in step 2, and then the corrected center line is obtained.
[0053] The invention provides a regularized road extraction method based on convolutional graph neural network. Figure 1 It can be seen that it is divided into four parts in total, namely data preprocessing, convolutional neural network extraction of node features, graph neural network regression width, and finally generating regularized road surface and corrected offset road centerline.
[0054] First, the data is preprocessed in three levels. In the first level, in order to preserve the integrity of road connections, the preprocessing part does not crop the image into fixed small blocks, but retains the overall connection relationship of the image area by generating an equidistant vector centerline map structure from the centerline annotation; in the second level, due to the limitation of GPU, the node information in the centerline map of the entire area cannot be input into the network at one time, so it is necessary to process n nodes in the centerline map in batches to form a centerline subgraph. The subgraph is obtained by randomly selecting n-1 adjacent nodes of the starting point by depth-first or breadth-first search. When constructing the next subgraph, the processed nodes are skipped and searched until the entire centerline map area is covered; in the third level, each node in the subgraph is processed, and the image block, centerline block and road label block centered on each node in the subgraph are cropped and rotated to the vertical direction. The size of the image block is 256×256 pixels. The road label block is extracted using the canny operator to extract the road edge and resampled to one-fourth of the original size, that is, 64×64 pixels.
[0055] Then, the image blocks and centerline blocks of n nodes in a batch (i.e., a subgraph) are sent to the regularized width extraction network. The network structure is as follows: Figure 2 As shown. The network consists of a convolutional neural network feature extraction part and a graph neural network regularized width regression part. Among them, the convolutional neural network feature extraction includes a main part and a branch part. The main part includes eleven layers of continuously stacked convolutional layers and three layers of fully connected layers. Each convolutional layer uses batch normalization to speed up the training speed. The convolution kernel size of the first convolutional layer is 5×5, and the step size is 2; the convolution kernel size of the second, fourth, sixth, eighth, and tenth convolutional layers is 3×3, and the step size is 1; the convolution kernel size of the third, fifth, seventh, ninth, and eleventh convolutional layers is 3×3, and the step size is 2; at the same time, a convolution layer with a convolution kernel of 3×3 and a step size of 1 is connected to the fourth layer, and a rough road edge extraction result is generated through a Softmax activation function, which constitutes the branch part of the convolutional neural network. The main part of the convolutional neural network generates a 64-dimensional node high-order feature vector through three layers of fully connected layers; the branch part of the convolutional neural network generates a 62-dimensional node edge feature vector through three layers of fully connected layers; the position of the node in the input block is used as a 2-dimensional position feature vector, which is concatenated with the 64-dimensional node high-order feature vector and the 62-dimensional node edge feature vector to generate a 128-dimensional feature vector for each node as the input feature of each node in the graph neural network. The regularized width regression part of the graph neural network includes three layers of GAT layers, with four, four and one attention heads respectively, which gradually convert the 128-dimensional features into 4×64, 4×8 and 4-dimensional features as the final output of the network, representing the position coordinates of the edge nodes on both sides of the node, through which the width of the node on both sides can be obtained.
[0056] In the graph neural network part, there are two types of graph structures for information propagation, the logical graph structure and the segment structure, such as Figure 3 As shown in the figure, connected road nodes are represented by the same number. Both graph structures are generated from the original centerline vector graph structure. They share the same set of nodes, but the edge connection relationships are different. The first logical graph structure assumes that the turning angle between nodes of a logically identical road is less than 30°, and assumes that information propagation is only carried out on the same logical road, that is, only logically connected road nodes have a connection relationship in the graph structure; the second road segment graph structure assumes that information propagation between nodes is limited to two adjacent endpoints or intersections, that is, only road nodes in the same road segment have a connection relationship in the graph structure.
[0057] Finally, a regularized road surface map is generated based on the network prediction results, and the offset road centerline is corrected, such as Figure 4 As shown in the figure. The convolutional neural network can obtain the rough width result of each node through training. The graph neural network optimizes the node width and smoothes the road surface by propagating and aggregating the width information between nodes. Finally, a regular road surface map can be generated by connecting the edge points on both sides calculated by the predicted width of the road. The center line of the road after the offset is corrected can be obtained by arithmetic averaging the widths on both sides.
[0058] The specific embodiments described herein are merely examples of the spirit of the present invention. Those skilled in the art may make various modifications or additions to the specific embodiments described or replace them in similar ways, but they will not deviate from the spirit of the present invention or exceed the scope defined by the appended claims.
Claims
1. A regularized road extraction method based on convolution and graph neural network, It is characterized in that The steps include: Step 1: Generate an equidistant vector road centerline map structure based on a given image and a corresponding road centerline annotation file, and calculate the width information of the road nodes; Step 2: Using the road width samples and centerline graph structure constructed in step 1, the entire image area is divided into sub-graphs, and the regularized width prediction network composed of the convolutional neural network and the graph neural network is trained in batches to predict the left and right width information of each road node on the centerline. The convolutional neural network extracts the high-level abstract features and road boundary features of each node, and concatenates them with the node position features to form the feature vector of each node as the input of the graph neural network. The graph neural network propagates and aggregates the node feature information in the centerline graph structure, optimizes the node features as a whole, and finally predicts the left and right regularized width information of each node. The regularized width prediction network includes a convolutional neural network feature extraction part and a graph neural network width regularization regression part; wherein, the convolutional neural network feature extraction main part includes eleven layers of continuously stacked convolutional layers and three layers of fully connected layers; the convolution kernel size of the first convolutional layer is 5×5, and the step size is 2; the convolution kernel size of the second, fourth, sixth, eighth, and tenth convolutional layers is 3×3, and the step size is 1; the convolution kernel size of the third, fifth, seventh, ninth, and eleventh convolutional layers is 3×3, and the step size is 2; at the same time, a convolution layer with a convolution kernel of 3×3 and a step size of 1 is connected to the fourth layer, through A rough road edge extraction result is generated through a Softmax activation function to form the branch part of the convolutional neural network; in the main part of the convolutional neural network, a 64-dimensional node high-order feature vector is generated through three layers of fully connected layers; the branch part of the convolutional neural network passes through three layers of fully connected layers to generate a 62-dimensional node edge feature vector; the position of the node in the input block is used as a 2-dimensional position feature vector, which is concatenated with the 64-dimensional node high-order feature vector and the 62-dimensional node edge feature vector to generate a 128-dimensional feature vector for each node as the input feature of each node in the graph neural network; The graph neural network includes three GAT layers, the first layer has four attention heads, which converts the 128-dimensional feature vector into a 64-dimensional vector; the second layer has four attention heads, which converts the 64×4-dimensional vector into an 8-dimensional vector; the last layer has only one attention head, which converts the 8×4-dimensional vector into a 4-dimensional vector, representing the position coordinates of the edge nodes on both sides of the node, and the width of both sides of the node can be obtained through the coordinates; the centerline graph structure of the input graph neural network includes two types, both of which are generated by the centerline vector graph structure obtained in step 1, they share the same node set, but the connection relationship of the edges is different; the first graph structure assumes that information propagation is only carried out on the same logical road, and the same logical road means that the angle between adjacent nodes of the road is less than 30°; the second graph structure assumes that the information propagation between nodes is limited to between two adjacent endpoints or intersections, that is, between road sections; Step 3, using the left and right width information of the road nodes and the centerline graph structure obtained in step 2, a regular road surface is generated, and the offset centerline is corrected to the center position.
2. A regularized road extraction method based on convolution and graph neural network as claimed in claim 1, Features: The specific implementation of step 1 includes the following sub-steps: Step 1.1, centerline vectorization: vectorize the road centerline annotations corresponding to the remote sensing image into a vector centerline graph structure consisting of nodes and lines, and resample the nodes to be equidistant; Step 1.2, calculate the node direction vector: calculate the node direction vector according to the node degree; if the node degree is 1, that is, there is only one neighbor node, then the direction is calculated based on the neighbor node and the node; if the node degree is 2, that is, there are two neighbor nodes, then the direction is calculated based on the two neighbor nodes; if the node degree is greater than 2, that is, there are three or more neighbor nodes, then the direction is not calculated; Step 1.3, calculate the left and right widths of the node: According to the node direction calculated in step 1.2, for nodes with a degree less than or equal to 2, the node vertical direction can be obtained. The vertical direction intersects with the corresponding road surface label to obtain the left and right widths of the node; for nodes with a degree greater than 2, no processing is performed.
3. A regularized road extraction method based on convolution and graph neural network as claimed in claim 1, Features: The specific implementation of step 2 includes the following sub-steps: Step 2.1, construct the centerline subgraph: Due to the limitation of GPU, all nodes in the image cannot be input into the network at one time. It is necessary to construct a subgraph based on the vector centerline graph generated in step 1.
1. A random starting point is selected from the nodes of the centerline graph. Starting from this starting point, a random depth-first or breadth-first search is performed to quantify the number of nodes to form a centerline subgraph as a batch of input to the network. The next random starting point selection skips the processed nodes until the entire centerline graph is covered, that is, the entire image is processed. Step 2.2, crop the node block: according to the centerline subgraph constructed in step 2.1 and the node direction calculated in step 1.2, crop the image and centerline of size n×n pixels with the node position as the center, and rotate it to the vertical direction so that the road edge is located on the left and right sides of the node; Step 2.3, predict node width: input the node image and center line block produced in step 2.2 into the proposed regularized width prediction network to predict the width values on the left and right sides of the node.
4. A regularized road extraction method based on convolution and graph neural network as claimed in claim 1, Features: The loss function of the regularized width prediction network includes two parts: a rough road edge prediction mean square error loss function and a final width regression mean square error loss function. The weights of these two parts are set to 100 and 1 according to experience.
5. A regularized road extraction method based on convolution and graph neural network as claimed in claim 1, Features: The specific implementation of step 3 includes the following sub-steps: Step 3.1, generate regularized road surface: according to the road centerline vector and the left and right widths of the nodes predicted in step 2, the left and right boundary node positions of each node can be obtained; for the intersection node, no prediction is made in step 2, and the width information is obtained by calculating the average width of the neighboring nodes around it and determining the edge node position in each direction corresponding to the neighboring nodes; the edge nodes are connected in sequence to obtain the regularized road surface map; Step 3.2, correct the offset center line: obtain the center position of the road node by arithmetic averaging the left and right widths of the node predicted in step 2, and then obtain the corrected center line.
