Global-local decoding remote sensing image road network extraction method

Abstract

The invention provides a global-local decoding remote sensing image road network extraction method, and relates to the technical field of remote sensing image road network extraction, and the method comprises the steps: firstly achieving the query of road nodes in a remote sensing image through a Transform model, respectively mapping the query of the road nodes into two-dimensional coordinates and 36-dimensional direction descriptors of the road nodes, and executing the construction of a primary network; and on the basis, performing iterative retrieval local query on the road end points in the primary road network to complete the road network to obtain a final road network. Through the method and the device, the problems of low accuracy and integrity of road network extraction and long road network retrieval time during road network extraction in the prior art are solved.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing image road network extraction, and in particular to a remote sensing image road network extraction method of global-local decoding. Background Art

[0002] Remote sensing image road network extraction refers to extracting the road centerline in remote sensing images by manual delineation or machine learning model recognition. As an important basic geographic information, the road network reflects the structure and spatial layout of the road and has important applications in map updating, autonomous driving, and urban planning.

[0003] In the field of road network extraction from remote sensing images, the method of manually outlining roads is time-consuming and costly. Automatic extraction of road networks from remote sensing images based on deep learning models is a hot topic in current research. For example, there are two main types. One is to use the global information of the image to output the road network in parallel. This method encodes the road network into a tensor representation, supervises training and predicts whether each pixel is the center point of the road and the connection direction of other pixels, and then forms the road network result through the connection algorithm. However, there is a problem of connection confusion in the actual data production process. This method quantizes the road direction in 6 directions, which makes it easy to connect to other lanes incorrectly when multiple lanes are adjacent. The other type is to iteratively output the road network using local information of the image. This method first uses the semantic segmentation network to extract the intersection point in the image as the initial point. Then, the initial point is used as the local image center point to iteratively search the road segment to form a road segment starting from the initial point, until all the initial points are searched to form the final road network result. However, in the actual data production process, compared with the parallel output road network method, there is a problem of slow retrieval speed. Each time this method iterates and retrieves the next vertex, it needs to use the influence of the previous vertex area to perform model prediction, which results in a retrieval time that is significantly higher than the parallel output road network method. Summary of the invention

[0004] The present invention provides a global-local decoding remote sensing image road network extraction method, which is used to solve the problems in the prior art of low accuracy and completeness of road network extraction and long road network retrieval time.

[0005] The present invention provides a method for extracting road networks from remote sensing images by global-local decoding, comprising: Obtain a remote sensing image slice of the road network to be extracted, and call the Transformer encoder to encode the remote sensing image slice to obtain an initialized road query; Construct noise samples according to the ground truth coordinates of the road network to be extracted; Input the initialized road query and the noise sample into the Transformer decoder for decoding processing to obtain a candidate road query that completes information exchange, and call the Hungarian algorithm to perform bipartite matching on the candidate road query to obtain a target road query; Calling the global prediction head to map the target road query into 2D coordinates and 36-dimensional direction descriptors of road nodes respectively, and inputting the combined features constructed by using the 2D coordinates and the 36-dimensional direction descriptors into the topological connection network to obtain the primary road network of the road network ground to be extracted; In the process of iteratively searching the road endpoints in the primary road network, the road network is completed for the road endpoints to obtain the final road network of the ground to be extracted.

[0006] In some embodiments, calling a Transformer encoder to encode the remote sensing image slice to obtain an initialized road query includes: Extract multi-scale feature maps of each remote sensing image slice through the Transformer encoder; Perform multi-scale hole attention calculation on the multi-scale feature map according to the ground sampling points to obtain a corresponding sampling point query; Select multiple sampling point queries with the largest confidence as the initial road query.

[0007] In some embodiments, the step of constructing a noise sample according to the ground truth coordinates of the road network to be extracted includes: For each ground truth coordinate in the road network to be extracted, two random noise values ​​are added; When the random noise value is less than or equal to the preset noise threshold, the ground truth coordinates with the added random noise value are taken as positive samples; When the random noise value is greater than the noise threshold and less than twice the noise threshold, the ground truth coordinates with the added random noise value are taken as negative samples; The positive and negative samples of each ground truth coordinate are used as noise samples.

[0008] In some embodiments, the combined features constructed by the 2D coordinates and the 36-dimensional direction descriptors are input into a topological connection network to obtain a primary road network of the road network ground to be extracted, including: Mapping the 2D coordinates into coordinate feature representations through a coordinate multilayer perceptron, and mapping the 36-dimensional direction descriptors into direction feature representations through a direction multilayer perceptron; The coordinate feature representation and the direction feature representation are concatenated to obtain a combined feature; In the topological connection network, for each target road node, adjacent road nodes of the target road node are determined, and combined features of the target road node and the adjacent road nodes are merged to obtain a merged feature; Calling an activation function to activate the merged features to obtain a corresponding feature tensor; Calling the fully connected layer to map the feature tensor to obtain the corresponding tensor value; When the tensor value is greater than the tensor threshold, the target road node is connected to the adjacent road nodes to obtain a primary road network of the road network ground to be extracted.

[0009] In some embodiments, in the process of iteratively searching the road endpoints in the primary road network, completing the road network of the road endpoints to obtain the final road network of the road network ground to be extracted includes: Constructing an initial retrieval point set according to road endpoints in the primary road network, wherein the road endpoints are road nodes whose number of adjacent road nodes in the primary road network is less than 2; Selecting a center point from the initial search point set according to a preset iteration strategy; For each selected center point, a local remote sensing image is extracted from the remote sensing image slice, and a local road network grid result is extracted from the primary road network; A local road query of the center point is determined according to the local remote sensing image and the local road network grid result, and a road network is completed in the primary road network according to the local road query to obtain a final road network of the ground to be extracted.

[0010] In some embodiments, the preset iteration strategy includes: In the first round of iteration, a road endpoint is randomly selected from the initial search point set as the starting center point; If the number of road nodes in the local road query of the starting center point is 0, randomly select a road endpoint from the candidate point set as the center point of the next iteration, wherein the candidate point set is a point set consisting of the remaining road endpoints in the initial search point set except the starting center point; If the number of road nodes in the local road query is 1, the road node in the local road query is used as the center point of the next iteration; If the number of road nodes in the local road query is 2, randomly select a road node in the local road query as the center point of the next iteration, and add the remaining road nodes in the local road query to the candidate point set; When the candidate point set is an empty set, the iteration ends.

[0011] In some embodiments, the method further comprises: The Transformer encoder, the Transformer decoder, and the topological connection network are integrated into a road network extraction model, and the training process of the road network extraction model includes: Acquire a road network remote sensing image sample, wherein the road network remote sensing image sample includes a road network connection true value; Inputting the road network remote sensing image sample into the road network extraction model for forward propagation; In the parallel stage of forward propagation, a parallel stage loss function is constructed according to the road network connection true value and the coordinate prediction value, direction prediction value, and node connection prediction value of the road node output by the road network extraction model; In the iterative phase of forward propagation, an iterative phase loss function is constructed according to the true value of the road network connection and the predicted probability value of the road endpoint output by the road network extraction model; The sum of the parallel stage loss function and the iterative stage loss function is used as the final loss function, and the final loss function is used to perform back propagation in the road network extraction model to update the parameters of the road network extraction model.

[0012] In some embodiments, the process of constructing the road network connection true value includes: In the true value road map of the road network remote sensing image sample, rasterization is performed according to line segments of preset pixel lengths; When a road node in the road network remote sensing image sample is within the range of the line segment, determining the road node as a valid node; For each valid node, a corresponding projection point is determined in the true value road map, where the projection point is a center line point in the true value road map with the smallest Euclidean distance to the valid node; According to the connection relationship of the projection points, the connectivity relationship of the road nodes in the road network remote sensing image sample is determined, and based on the connectivity relationship, the corresponding road nodes in the road network remote sensing image sample are connected to obtain the road network connection true value.

[0013] In some embodiments, the process of constructing the parallel stage loss function includes: Determine the coordinate true value, direction true value, and node connection true value of the road node from the road network connection true value; Constructing a position loss according to the true value of the coordinate and the predicted value of the coordinate; Constructing a direction loss according to the true direction value and the predicted direction value; Constructing a connection loss according to the true value of the node connection and the predicted value of the node connection; The position loss, the direction loss and the connection loss are weighted respectively according to the preset first position weight, direction weight and connection weight, and the weighted results are summed to obtain a parallel stage loss function, wherein the values ​​of the direction weight and the connection weight are positively correlated with the training rounds of the road network extraction model; The process of constructing the loss function in the iteration phase includes: Determining the true probability value of the road endpoint from the road network connection true value; Constructing a probability loss of a road endpoint according to the true probability value and the predicted probability value; The position loss and the probability loss are weighted respectively by a preset second position weight and a probability weight, and the weighted results are summed to obtain an iterative phase loss function.

[0014] The present invention also provides a global-local decoding remote sensing image road network extraction device, comprising: The encoding module is used to obtain the remote sensing image slices of the road network to be extracted, and call the Transformer encoder to encode the remote sensing image slices to obtain the initialization road query; A construction module, for constructing noise samples according to the ground truth coordinates of the road network to be extracted; A decoding module is used to input the initialization road query and the noise sample into the Transformer decoder for decoding processing to obtain a candidate road query that completes information exchange, and call the Hungarian algorithm to perform two-part matching on the candidate road query to obtain a target road query; A global query module is used to call a global prediction head to map the target road query into 2D coordinates and 36-dimensional direction descriptors of road nodes, and input the combined features constructed by using the 2D coordinates and the 36-dimensional direction descriptors into a topological connection network to obtain a primary road network of the road network to be extracted; The local query module is used to complete the road network of the road endpoints in the primary road network during the iterative search of the road endpoints to obtain the final road network of the ground to be extracted.

[0015] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the remote sensing image road network extraction method using global-local decoding as described in any one of the above is implemented.

[0016] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the remote sensing image road network extraction method of global-local decoding as described in any one of the above is implemented.

[0017] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the remote sensing image road network extraction method of global-local decoding as described in any one of the above is implemented.

[0018] The method for extracting road networks from remote sensing images with global-local decoding provided by the present invention first implements the query of road nodes in remote sensing images through a Transformer model, and maps the query of road nodes to 2D coordinates and 36-dimensional direction descriptors of road nodes, respectively, so as to refine the directions of road nodes, alleviate the problem of unclear directions of road nodes in multi-lane scenarios, and avoid confusion in road node connections. In addition, the present invention also constructs noise samples for the prediction of road nodes, reduces the semantic ambiguity of complex intersections, and improves the prediction accuracy of road points in complex road networks. Next, a road network construction process from global to local is constructed, and most of the road networks can be extracted at a fast speed through global queries. In local queries, only iterative searches of road endpoints are performed to complete the road network, which ensures the integrity of the road network extraction. Compared with the prior art, the number of iterative vertices can be greatly reduced, shortening the road network retrieval time. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced one by one below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 It is a flow chart of the global-local decoding remote sensing image road network extraction method provided by the present invention.

[0021] Figure 2 It is a schematic diagram of the global-local decoding remote sensing image road network extraction method provided by the present invention.

[0022] Figure 3 This is a road node representation diagram provided by the present invention.

[0023] Figure 4 It is a schematic diagram of the effect of remote sensing image road network extraction provided by the present invention.

[0024] Figure 5 It is a comparison chart of the effects of road network extraction from various remote sensing images provided by the present invention.

[0025] Figure 6 It is a structural schematic diagram of a remote sensing image road network extraction device with global-local decoding provided by the present invention.

[0026] Figure 7 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] The following is a flow chart of the method for extracting road network from remote sensing images by global-local decoding according to the present invention, as shown in the accompanying drawings. Figure 1 As shown, the method includes the following steps 101 to 105.

[0029] Step 101: obtain a remote sensing image slice of the road network to be extracted, and call the Transformer encoder to encode the remote sensing image slice to obtain an initialized road query.

[0030] First, obtain the remote sensing image of the road network to be extracted, and then crop the remote sensing image to obtain remote sensing image slices. The cropping process can be to evenly crop the remote sensing image of size h×w×c into remote sensing image slices of size s×s×c through a non-overlapping sliding window of size s×s. Among them, h is the image height, w is the image width, c is the number of image channels, and s is the sliding window size with the same width and height. If the height of the last row of remote sensing image slices in the sliding window is less than s, the part of the last column with a width less than s is reversely cropped to obtain remote sensing image slices of size s×s×c.

[0031] Then, for each remote sensing image slice, the Transformer encoder is called to encode the remote sensing image slice to obtain an initialized road query. Here, the multi-scale feature map of the remote sensing image slice is first extracted by the Transformer encoder, and then a sampling point query is performed on the multi-scale feature map to obtain an initialized road query. In an embodiment of the present invention, the road query can be a pixel coordinate value representing a road portion in a remote sensing image, which is used to determine a specific road node in a remote sensing image slice.

[0032] Specifically, if Figure 2As shown in the figure, the multi-scale feature map of each remote sensing image slice is first extracted through the Transformer encoder. Here, the backbone network in the Transformer encoder is used to extract feature maps with feature scales of H / 4×W / 4×256, H / 8×W / 8×256, H / 16×W / 16×256, and H / 32×W / 32×256. The corresponding position code is also added in the process of extracting the feature map.

[0033] Next is the Transformer encoding process. The Transformer encoder consists of a multi-scale deformable attention module and a multi-layer perceptron, which aims to fuse long-range information into multi-scale features. First, multiple ground sampling points are determined on the multi-scale feature map, and then multi-scale dilated attention calculation is performed on the multi-scale feature map according to the ground sampling points to obtain the corresponding sampling point query. The multi-scale dilated attention (MSDA) calculation is implemented through the multi-scale deformable attention module, and the formula is as follows: (1) In the above formula (1), , q and Respectively represent the multi-scale feature map, road query and the reference point coordinates corresponding to the road query, , and They represent the mth attention head, lth feature level and kth sampling point in the model respectively. and are all learnable weight matrices, Indicates road query q, Attention head, Level characteristics, The attention weights of the sampling points. To adapt to The feature layer is normalized to the reference point to zoom in or out. represents the learnable sampling offset parameter. Represented in the feature map Road Query The sampling location.

[0034] like Figure 2As shown in the figure, during the Transformer encoding process, through the above formula (1), the Transformer encoder will output multiple sampling point queries, all of which are likely to represent road areas. Then, the top N sampling point queries are selected, specifically, multiple sampling point queries with the highest confidence are selected as the initial road queries, that is, N is 500, and the confidence of the sampling point queries is calculated and then sorted from large to small according to the confidence. Finally, the 500 sampling point queries with the highest confidence are selected from the head of the sequence. These sampling point queries are most likely to represent road areas, so they can be used as the initial road queries.

[0035] Step 102: construct noise samples according to the ground truth coordinates of the road network to be extracted.

[0036] In order to enhance the adaptability to noise, the embodiment of the present invention also constructs a denoising strategy, which participates in the prediction process of road query by adding noise of ground truth (GT) coordinates in the initial road query. By predicting the road area under the influence of noise, the possible noise influence can be effectively eliminated in the process of road area prediction.

[0037] In some embodiments, noise samples are constructed according to the ground truth coordinates of the road network to be extracted. Specifically, first, for each ground truth coordinate in the road network to be extracted, two random noise values ​​are added, which are respectively expressed as ( )as well as( ). Then, the embodiment of the present invention performs the distinction between positive and negative samples on the ground truth coordinates with added random noise.

[0038] When the random noise value is less than or equal to the preset noise threshold, the ground truth coordinates with the added random noise value are taken as positive samples. The noise threshold value is , if satisfied as well as , it means that the ground truth coordinates with added random noise values ​​can be used as positive samples.

[0039] When the random noise value is greater than the noise threshold and less than twice the noise threshold, the ground truth coordinates of the random noise value are added as negative samples. Here, if as well as , it means that the ground truth coordinates with added random noise values ​​can be used as negative samples. Among them, The meaning of is the preset noise size hyperparameter, which is 5 here.

[0040] In this way, a positive sample and a negative sample can be constructed for each ground truth coordinate, and then the positive sample and negative sample of each ground truth coordinate are used as noise samples.

[0041] The embodiments of the present invention, when predicting road queries by extracting multi-scale feature maps, participate in the prediction by constructing corresponding noise samples, and can also make predictions when noise may affect road predictions. This ensures the adaptability of the road network extraction process to noise, reduces semantic ambiguity in complex intersections, and improves the prediction accuracy of road points in complex road networks.

[0042] Step 103: Input the initialized road query and the noise sample into the Transformer decoder for decoding processing to obtain a candidate road query that completes information exchange, and call the Hungarian algorithm to perform two-part matching on the candidate road query to obtain a target road query.

[0043] like Figure 2 As shown in the figure, after constructing the noise sample, the next step is the decoding process of the Transformer decoder, that is, Transformer decoding. Here, the initialized road query and the noise sample are input into the Transformer decoder for decoding processing to obtain the candidate road query that completes the information interaction. The structure of the Transformer decoder is the same as that of the encoder, and is also composed of a 6-layer multi-head attention module, a multi-scale deformable attention module, and a multi-layer perceptron. The candidate road query can not only represent the possible road area, but also represent the information interaction between roads, so as to facilitate the accurate determination of road nodes in the road area, and the query extraction process is completed.

[0044] Next, in order to determine accurate road node information based on the road query, the embodiment of the present invention performs two-part matching on the candidate road query by calling the Hungarian algorithm to obtain the target road query. Here, the true value of each road information can be determined from the remote sensing image of the road network ground to be extracted, and then the candidate road query is matched with the true value of the road information by the Hungarian algorithm, and the best match is selected, and then the target road query is filtered out from the candidate road query from the best match. The target road query can accurately reflect the road nodes in the road area, which is convenient for road network construction.

[0045] Step 104: Call the global prediction head to map the target road query into the 2D coordinates and 36-dimensional direction descriptors of the road nodes respectively, and input the combined features constructed using the 2D coordinates and 36-dimensional direction descriptors into the topological connection network to obtain the primary road network of the ground to be extracted.

[0046] After determining the target road query of the road network to be extracted in step 103, the first stage of global query of road network extraction is performed next. In the process of global query, the coordinates and directions of the road nodes are first determined according to the target road query.

[0047] like Figure 2As shown in the figure, in the process of global query, for the target road query, the global prediction head is called here to map the target road query into the 2D coordinates and 36D direction descriptors of the road nodes. The global prediction head is composed of two different multi-layer perceptrons (MLPs), specifically the coordinate multi-layer perceptron and the direction multi-layer perceptron, which are used for the coordinate prediction and direction prediction of the road nodes respectively.

[0048] See also Figure 3 , Figure 3 The following example shows a schematic diagram of the 2D coordinates and 36-dimensional direction descriptor of a road node: Figure 3 As shown on the left side of the figure, the center of the circle represents the coordinates of the center point of the road. Horizontally to the right is 0 degrees, and counterclockwise every 10 degrees is an increment, with a total of 36 increments. When the direction reaches 360 degrees, it coincides with the 0 degree direction, and the direction value is 0. If this road node has adjacent road nodes in directions 3, 9, 21, and 27, its direction is represented as follows Figure 3 As shown on the right, the road node coordinates are the coordinates of the center point of the road, which belongs to a 2-dimensional vector (X, Y). In the vector of the 36-dimensional direction descriptor, the 3rd, 9th, 21st, and 27th positions are 1, and the rest are 0.

[0049] Next, for each road node, the combined features constructed by the 2D coordinates and 36-dimensional direction descriptors are input into the topological connection network to obtain the primary road network of the road network to be extracted. Here, the coordinates and directions of the road nodes are used as the features of the road nodes and input into the topological connection network. In the topological connection network, the tensor values ​​of the adjacent road nodes are calculated, and the corresponding adjacent road nodes are selected for connection according to the preset tensor threshold, so as to obtain the primary road network of the road network to be extracted.

[0050] The following is a detailed introduction to the primary road network construction process. Figure 2 As shown in the figure, the features of the road nodes are first processed. Here, the 2D coordinates are mapped to coordinate feature representation (Point Embedding) through the coordinate multilayer perceptron, and the 36-dimensional direction descriptor is mapped to the direction feature representation Direct Embedding through the direction multilayer perceptron. Then, the coordinate features and direction features are fused, and the coordinate feature representation and the direction feature representation are spliced ​​to obtain the combined features, which are finally sent to the global topological connection network to execute the construction of the primary road network.

[0051] The input received by the topological connection network is the feature combination of the road node and the adjacent road nodes. The topological connection network can be regarded as a fully connected neural network model. Therefore, in the topological connection network, for each target road node, the adjacent road nodes of the target road node are first determined, and the combined features of the target road node and the adjacent road nodes are merged to obtain the merged features. Each road node has adjacent road nodes, so here for each road node, the adjacent road nodes are first determined, and then the combined features of the road nodes are merged. Combined features with adjacent road nodes Merge and get the merged features, recorded as ,in is the maximum number of adjacent nodes.

[0052] Next, the activation function is called to activate the merged features and obtain the corresponding feature tensor. The feature vector is then activated by the Relu activation function of the fully connected layer to obtain The feature vector is then activated by the sigmoid function of the fully connected layer to obtain The feature tensor of .

[0053] Finally, the fully connected layer is called to map the feature tensor to obtain the corresponding tensor value. The feature tensor is then input into a fully connected layer and mapped through the sigmoid function of the fully connected layer to obtain the corresponding tensor value, recorded as .

[0054] When constructing a primary road network, the embodiment of the present invention presets a tensor threshold of 0.5. When the tensor value is greater than the tensor threshold, the target road node is connected to the adjacent road node to obtain the primary road network of the road network surface to be extracted. Here, by performing tensor calculation on the merged features of each target road node, when the calculated tensor number is greater than 0.5, the target road node can be connected to the adjacent road node, so that each target road node can select the corresponding adjacent road node for connection, and finally form a primary road network.

[0055] In the embodiment of the present invention, in the process of building a road network from global to local, a preliminary road network structure is first built based on the predicted road nodes through the global query stage, which can quickly extract most of the road network with high efficiency, reducing the retrieval time and calculation amount of subsequent road network completion.

[0056] Step 105: In the process of iteratively searching the road endpoints in the primary road network, the road network is completed for the road endpoints to obtain the final road network of the ground to be extracted.

[0057] Through step 104, the first stage global query process of road network extraction is completed, and then the second stage of road network extraction, the local query process, is entered. This part is mainly for completing the dead-end roads existing in the road network, that is, for completing the road endpoints. The road endpoints are road nodes with less than 2 adjacent road nodes in the primary road network. In this process, first determine the road endpoints to be completed, and perform local road queries based on the local remote sensing images and local road network raster results corresponding to the road endpoints, and then perform road network completion. In this process, it is necessary to perform an iterative retrieval process of road endpoints, and perform a road network completion once in each retrieval process to obtain the final road network of the ground to be extracted. The effect schematic diagram of the final road network can be shown as follows Figure 4 As shown, Figure 4 The yellow road node segments are the final road network extracted.

[0058] Specifically, firstly, an initial retrieval point set is constructed according to the road endpoints in the primary road network. A road endpoint is a road node whose number of adjacent road nodes in the primary road network is less than 2. Here, all road endpoints are added to the initial retrieval point set, which is recorded as ,in , Indicates the number of adjacent nodes for subsequent iterative retrieval.

[0059] Then, according to the preset iteration strategy, the center point is selected from the initial search point set. For each selected center point, the local remote sensing image is extracted from the remote sensing image slice, and the local road network grid result is extracted from the primary road network. A local remote sensing image with a width and height size of 128×128 is intercepted as the center, and a local road network grid result with a width and height size of 128×128 is also extracted from the corresponding primary road network.

[0060] Then, the local road query of the center point is determined based on the local remote sensing image and the local road network grid results. Figure 2 As shown in the figure, in the local query process, the local remote sensing image and the local road network raster results are first input into the backbone network for feature extraction. The local road network raster results are subjected to feature extraction through the backbone network to obtain the corresponding mask features, while the local remote sensing image is subjected to feature extraction through the backbone network to obtain the corresponding RGB features. The backbone network can be a Transformer model, and then the mask features and RGB features are subjected to feature fusion. Finally, the fused features are input into the query extraction module for query extraction, and the corresponding local road query is output.

[0061] Next, the road network is completed in the primary road network based on the local road query to obtain the final road network of the road network to be extracted. Here, the local road query is still feature mapped by the prediction head, and the local road query is mapped to the coordinates of the corresponding road endpoints, which are then input into the topological connection network for tensor calculation, and then the adjacent road nodes whose tensor values ​​are less than the tensor threshold of 0.5 are determined, and the road endpoints and adjacent road nodes are connected to achieve road network completion.

[0062] Of course, in some embodiments, in order to improve the accuracy of local road query, such as Figure 2 As shown, the embodiment of the present invention can also be used to slice the remote sensing image with the center point Cut multiple pieces for the center ( Figure 2 In the example, two local remote sensing images with a size of 128×128 are given, and multiple 128×128 local road network grid results are also extracted from the corresponding primary road network. The first local remote sensing image and local road network grid results are extracted with features through the backbone network, and then the corresponding local road query is output through the query extraction module. This local road query is extracted with features from the backbone network together with the second local remote sensing image and local road network grid results, and then the corresponding local road query is output through the query extraction module. The local road query outputted last in this process is used as the final local road query of the road endpoint.

[0063] In the embodiment of the present invention, in the local query process of the second stage of road network extraction, the central point is selected from the initial search point set by iteration, and the local road query of each road endpoint is calculated to complete the primary road network. In the local query, only the iterative search of the road endpoints is performed to complete the road network, which ensures the integrity of the road network extraction. Compared with the prior art, the number of iterative vertices can be greatly reduced, and the road network search time can be shortened.

[0064] In some embodiments, in order to ensure the integrity of road network extraction, the embodiments of the present invention select the center point from the initial retrieval point set according to a preset iteration strategy to achieve road network completion of road endpoints. The specific iteration process of the preset iteration strategy is introduced below.

[0065] The main purpose of the iteration strategy is to determine the center point selected in each iteration process. Here, in the first iteration, a road endpoint is randomly selected from the initial retrieval point set as the starting center point. Select any road endpoint As the starting center point, the next The neighbor road nodes are used for judgment.

[0066] If the number of road nodes in the local road query of the starting center point is 0, a road endpoint is randomly selected from the candidate point set as the center point of the next iteration, where the candidate point set is the point set consisting of the remaining road endpoints in the initial retrieval point set except the starting center point.

[0067] Here, if the number of road nodes in the local road query of the starting center point is determined to be 0, it means that in this local road query, the neighboring road nodes of the starting center point cannot be found, and the road network cannot be constructed for the starting center point, so it is directly skipped and this round of iteration ends. Then, another road endpoint other than the starting center point is selected from the initial search point set as the center point of the next round of iteration to perform local road query.

[0068] If the number of road nodes in the local road query is 1, the road node in the local road query is used as the center point of the next iteration.

[0069] Here, if it is determined that the number of road nodes in the local road query of the starting center point is 1, it means that in this local road query, a neighboring road node of the starting center point can be found, then this road node in the local road query is used as the center point of the next round of iteration, and then according to the local road query, the road network construction of the starting center point and this road node is executed to complete the road network.

[0070] If the number of road nodes in the local road query is 2, a road node in the local road query is randomly selected as the center point of the next iteration, and the remaining road nodes in the local road query are added to the candidate point set.

[0071] Here, if it is determined that the number of road nodes in the local road query of the starting center point is 2, it means that in this local road query, two neighboring road nodes of the starting center point can be found, and then one of the two neighboring road nodes is randomly selected as the center point of the next iteration, and then according to the local road query, the road network construction of the starting center point and the road node randomly selected in the local road query is executed to complete the road network. The road nodes that are not selected in the two neighboring road nodes are added to the candidate point set.

[0072] Among them, in each round of iteration, after the road network is completed for the selected center point, the next road endpoint is selected from the candidate point set to perform iteration. When the candidate point set is an empty set, it means that all road endpoints have completed the road network completion and the iteration ends.

[0073] The embodiment of the present invention performs iterative search on road endpoints to calculate local road queries for each road endpoint and determine the neighboring road nodes of the road endpoint, thereby completing the road network. That is, the integrity of the road network extraction is guaranteed, and compared with the prior art, the number of iterative vertices can be greatly reduced, and the road network search time can be shortened.

[0074] In some embodiments, considering that the road network structure of some road networks to be extracted is complex and there are many road nodes, there are still defects in the accuracy of road network extraction, and it is impossible to meet some high requirements for the extraction of road network structure. Therefore, an embodiment of the present invention also proposes a method of using a road network extraction model to perform road network extraction. By integrating the Transformer encoder and the Transformer decoder and the topological connection network into a road network extraction model, that is, the Transformer encoder used in the above step 101, the Transformer decoder used in step 103, and the topological connection network used in step 104 are integrated into a road network extraction model. The training process of the road network extraction model is introduced below.

[0075] First, obtain road network remote sensing image samples, where the road network remote sensing image samples include road network connection truth values. Here, the road network remote sensing image samples are some remote sensing images of the road network ground to be extracted, and the road network connection truth values ​​are the truth data extracted from the truth road map with the road network structure annotated, which are used as training labels to construct the loss function. The truth data specifically includes: the true value of the coordinates of the road nodes, the true value of the direction, the true value of the node connection, and the true probability value of the road endpoint.

[0076] At the beginning of training, the road network remote sensing image samples are input into the road network extraction model for forward propagation, which is completed in two stages.

[0077] In the parallel stage of forward propagation, a parallel stage loss function is constructed based on the true value of the road network connection and the coordinate prediction value, direction prediction value, and node connection prediction value of the road nodes output by the road network extraction model.

[0078] Specifically, when the road network extraction model performs road network extraction, it will calculate the road query to predict the coordinate prediction value and direction prediction value of each road node. When constructing the primary road network, the connection relationship of the road nodes is determined by the coordinate prediction value and direction prediction value of the road nodes, thereby obtaining the node connection prediction value of the corresponding road node. Therefore, in the forward propagation process, the parallel stage loss function can be constructed based on the true value data including the coordinate prediction value, direction prediction value, node connection prediction value and road network connection true value to train the road network extraction model for the accuracy of road network extraction.

[0079] In the iterative stage of forward propagation, the iterative stage loss function is constructed based on the true value of the road network connection and the predicted probability value of the road endpoint output by the road network extraction model.

[0080] In the process of road network completion, the road network extraction model first determines each road endpoint in the primary road network, wherein the location information of the road endpoint also includes the predicted probability of the road endpoint. In the embodiment of the present invention, the predicted probability value of the road endpoint and the true value data included in the road network connection true value are selected to construct the iterative phase loss function to train the road network extraction model for the road network completion accuracy of the road endpoint.

[0081] Finally, the sum of the parallel stage loss function and the iterative stage loss function is taken as the final loss function, and the final loss function is used to backpropagate in the road network extraction model to update the parameters of the road network extraction model.

[0082] By presetting the maximum number of iterations, when the final loss function begins to converge or reaches the maximum number of iterations, the iteration is stopped and the training process ends. The trained road network extraction model can perform road network extraction for various remote sensing images of the road network to be extracted to obtain the corresponding final road network.

[0083] The embodiment of the present invention designs an end-to-end road network extraction method, which directly performs road network extraction by designing a road network extraction model. When training the road network extraction model, by constructing a parallel phase loss function and an iterative phase loss function, not only the accuracy of the road network extraction model for road network extraction is ensured, but also the accuracy of the road network completion of the road endpoints by the road network extraction model is ensured. It can meet some requirements for the extraction of road network structures with high requirements.

[0084] Further, based on the above embodiment, the construction process of the road network connection true value is described below. Figure 5 As shown, Figure 5 (a) in the figure is a true road map of a road network remote sensing image sample with annotated road network structure. In the true road map of the road network remote sensing image sample, rasterization is first performed according to line segments of a preset pixel length, which can be 5 pixels.

[0085] When the road node in the road network remote sensing image sample is within the range of the line segment, the road node is determined as a valid node. Through these methods, the road nodes that can be displayed in the true value road map can be determined as valid nodes.

[0086] For each valid node, the corresponding projection point is determined in the true road map, so that the mapping relationship between the road nodes in the road network remote sensing image sample and the projection points in the true road map can be established. The projection point is the center line point with the smallest Euclidean distance to the valid node in the true road map. According to this standard, the projection point can be mapped one by one to the road node.

[0087] In the true value road map, the projection points are the road nodes of the standard road network structure, so there is a connection relationship between each projection point. Since the projection points have been mapped one-to-one with the road nodes in the road network remote sensing image sample, the connectivity relationship of the road nodes in the road network remote sensing image sample can be determined based on the connection relationship of the projection points.

[0088] Finally, based on the connectivity relationship, the corresponding road nodes are connected in the road network remote sensing image sample to obtain the road network connection truth value. By connecting the road nodes, the road network structure of the road network remote sensing image sample can be determined, and then it is easy to determine the true value data such as the coordinate true value, direction true value, node connection true value, and true probability value of the road endpoint one by one as the road network connection true value.

[0089] In the embodiment of the present invention, by establishing a mapping relationship between the road network remote sensing image sample and the true value road map, the real road network structure of the road network remote sensing image sample is determined, and then the road network connection true value. This can be used as a training label for the road network extraction model to construct a final loss function to train the road network extraction model, ensuring the training effectiveness of the model and the accuracy of the road network extraction.

[0090] In some embodiments, the road network extraction model depends on the final loss function constructed, and the final loss function is composed of the parallel stage loss function And the loss function in the iteration stage The two parts are as follows: the construction process of the parallel stage loss function and the iterative stage loss function.

[0091] For the parallel stage loss function The embodiment of the present invention first determines the true value of the coordinates of the road node from the true value of the road network connection. , Direction True Value , node connection true value .

[0092] Next, according to the true value of the coordinates And the coordinate prediction value , construct the position loss The position loss uses the L1 loss function, which is expressed as the following formula: (2) And, according to the true value of the direction And the direction prediction value , construct the direction loss Here, considering that most road nodes in the road network have two directions, only at the intersection will there be three or more directions. In order to deal with the problem of category imbalance, the direction loss of the road node here is It is constructed through the FocalLoss loss function and is expressed as: (3) In addition, the true value is connected according to the node And the node connection prediction value , construct the connection loss In the road network connection part, the connection between road nodes is a binary classification problem, that is, whether the connection with the neighboring road node is correct or failed. The loss function of the binary classification problem is It is constructed through the BCEWithLogitsLoss loss function, expressed as: (4) Loss function in parallel build phase Therefore, the embodiment of the present invention presets the first position weight for the loss function of the three parts respectively. , Direction Weight , connection weight , used to balance the loss values ​​of the three parts.

[0093] According to the preset first position weight , Direction Weight , connection weight For position loss , Direction loss and connection loss Weighted, and the weighted results are summed to obtain the parallel stage loss function , expressed as: (5) in, , , The position loss is , Direction loss and connection loss However, in the actual training process of the road network extraction model, the position prediction of the road nodes is unstable in the early stage, so the road network extraction model is trained with the position loss of the road nodes in the early stage. The main one will be given a greater weight, while the direction weight and connection weights The value of is positively correlated with the training epoch of the road network extraction model, that is, as the number of iterations increases, in the middle and late stages of training, the direction weight and connection weights The corresponding assignment will also become larger.

[0094] Specifically, as the number of iterations increases, the convergence direction of the road network extraction model is obvious, and the direction loss and connection loss The weight of The specific settings are , and the connection loss The specific settings are , epoch is the current training round of the road network connection model.

[0095] For the iterative stage loss function , the embodiment of the present invention still determines the true probability value of the road endpoint from the road network connection true value, where the true probability value is the true prediction probability of the road endpoint. In the true value road graph, the connection relationship between the road endpoint and the neighboring nodes is determined one by one, with a probability of 1, so is 1.

[0096] Next, according to the true probability value and predicted probability value , construct the probability loss of road endpoints , constructed through the BCEWithLogitsLoss loss function, expressed as: (6) In addition, the loss function in the iteration stage It also includes the coordinate loss of the road nodes, that is, , here, by the preset second position weight , probability weight For position loss , Probability loss Weighted, and summed the weighted results to get the loss function of the iteration stage , expressed as: (7) In the embodiment of the present invention, during the process of constructing the loss function of the road network extraction module, a parallel stage loss function is constructed by predicting the coordinates, directions and connections of the road nodes, so that the model can accurately predict the positions, directions and connection relationships of the road nodes in the primary road network, and an iterative stage loss function is constructed by the predicted probability values ​​of the road endpoints, so that the model can complete the road network for the road endpoints in the primary road network, thereby ensuring the accuracy of the road network extraction.

[0097] In order to verify the effectiveness of the road network extraction model of the present invention, the embodiment of the present invention also sets up a comparative experiment, using the commonly used Sat2Graph, RNGDet and other models for road network extraction in the prior art to conduct comparative experiments, and compare them with the real value road map. Figure 5 As shown in the figure, (b) is the road network extraction effect of the Sat2Graph model, (c) is the road network extraction effect of the RNGDet model, (d) is the road network extraction effect of the road network extraction model proposed by the present invention, and (a) is the ground truth road map. Figure 5 It can be proved that the road network extraction model proposed in the present invention is superior to the models used for road network extraction such as Sat2Graph and RNGDet in terms of road network extraction completeness and road node accuracy.

[0098] The global-local decoding remote sensing image road network extraction device provided by the present invention is described below. The global-local decoding remote sensing image road network extraction device described below and the global-local decoding remote sensing image road network extraction method described above can be referenced to each other.

[0099] like Figure 6As shown, the global-local decoding remote sensing image road network extraction device specifically includes: an encoding module 601, which is used to obtain a remote sensing image slice of the road network ground to be extracted, and call the Transformer encoder to encode the remote sensing image slice to obtain an initial road query; a construction module 602, which is used to construct a noise sample according to the ground true coordinates of the road network ground to be extracted; a decoding module 603, which is used to input the initial road query and the noise sample into the Transformer decoder for decoding processing to obtain a candidate road query that completes information interaction, and call the Hungarian algorithm to perform two-part matching on the candidate road query to obtain a target road query; a global query module 604, which is used to call the global prediction head to map the target road query into a 2D coordinate and a 36-dimensional direction descriptor of a road node respectively, and input the combined features constructed by using the 2D coordinate and the 36-dimensional direction descriptor into a topological connection network to obtain a primary road network of the road network ground to be extracted; a local query module 605, which is used to complete the road network of the road endpoints in the iterative retrieval process of the road endpoints in the primary road network to obtain the final road network of the road network ground to be extracted.

[0100] It should be noted that the beneficial effects of the global-local decoding remote sensing image road network extraction device here and the global-local decoding remote sensing image road network extraction method mentioned above can correspond to each other, so the beneficial effects of the global-local decoding remote sensing image road network extraction device will not be repeated here.

[0101] Figure 7 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 7As shown, the electronic device may include: a processor (processor) 710 , a communication interface (Communications Interface) 720 , a memory (memory) 730 and a communication bus 740 , wherein the processor 710 , the communication interface 720 , and the memory 730 communicate with each other through the communication bus 740 . The processor 710 can call the logic instructions in the memory 730 to execute the remote sensing image road network extraction method of global-local decoding, which includes: obtaining a remote sensing image slice of the road network ground to be extracted, and calling the Transformer encoder to encode the remote sensing image slice to obtain an initial road query; constructing a noise sample according to the ground true coordinates of the road network ground to be extracted; inputting the initial road query and the noise sample into the Transformer decoder for decoding processing to obtain a candidate road query that completes information exchange, and calling the Hungarian algorithm to perform bipartite matching on the candidate road query to obtain a target road query; calling the global prediction head to map the target road query into 2D coordinates and 36-dimensional direction descriptors of road nodes respectively, and inputting the combined features constructed using the 2D coordinates and the 36-dimensional direction descriptors into the topological connection network to obtain a primary road network of the road network ground to be extracted; in the process of iteratively retrieving the road endpoints in the primary road network, completing the road network of the road endpoints to obtain the final road network of the road network ground to be extracted.

[0102] In addition, the logic instructions in the above-mentioned memory 730 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0103] On the other hand, the present invention also provides a computer program product, the computer program product includes a computer program, the computer program can be stored on a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can execute the global-local decoding remote sensing image road network extraction method provided by the above methods, the method comprising: obtaining a remote sensing image slice of the road network to be extracted, and calling a Transformer encoder to encode the remote sensing image slice to obtain an initialized road query; constructing a noise sample according to the ground true coordinates of the road network to be extracted; inputting the initialized road query and the noise sample into The Transformer decoder performs decoding processing to obtain candidate road queries that complete information exchange, and calls the Hungarian algorithm to perform bipartite matching on the candidate road queries to obtain target road queries; the global prediction head is called to map the target road queries into 2D coordinates and 36-dimensional direction descriptors of road nodes respectively, and the combined features constructed using the 2D coordinates and the 36-dimensional direction descriptors are input into the topological connection network to obtain the primary road network of the ground to be extracted; in the process of iterative retrieval of road endpoints in the primary road network, the road endpoints are completed to obtain the final road network of the ground to be extracted.

[0104] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a remote sensing image road network extraction method for executing the global-local decoding provided by the above-mentioned methods, the method comprising: obtaining a remote sensing image slice of the ground of the road network to be extracted, and calling a Transformer encoder to encode the remote sensing image slice to obtain an initialized road query; constructing a noise sample according to the ground true coordinates of the ground of the road network to be extracted; inputting the initialized road query and the noise sample into a Transformer decoder for decoding processing to obtain a candidate road query that completes information exchange, and calling a Hungarian algorithm to perform bipartite matching on the candidate road query to obtain a target road query; calling a global prediction head to map the target road query into 2D coordinates and 36-dimensional direction descriptors of road nodes respectively, and inputting the combined features constructed using the 2D coordinates and the 36-dimensional direction descriptors into a topological connection network to obtain a primary road network of the ground of the road network to be extracted; in the process of iteratively retrieving the road endpoints in the primary road network, completing the road network of the road endpoints to obtain the final road network of the ground of the road network to be extracted.

[0105] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0106] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0107] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for extracting road networks from remote sensing images by global-local decoding, characterized in that: include: Obtain a remote sensing image slice of the road network to be extracted, and call the Transformer encoder to encode the remote sensing image slice to obtain an initialized road query; Construct noise samples according to the ground truth coordinates of the road network to be extracted; Input the initialized road query and the noise sample into the Transformer decoder for decoding processing to obtain a candidate road query that completes information exchange, and call the Hungarian algorithm to perform bipartite matching on the candidate road query to obtain a target road query; Calling the global prediction head to map the target road query into 2D coordinates and 36-dimensional direction descriptors of road nodes respectively, and inputting the combined features constructed by using the 2D coordinates and the 36-dimensional direction descriptors into the topological connection network to obtain the primary road network of the road network ground to be extracted; In the process of iteratively searching the road endpoints in the primary road network, the road network is completed for the road endpoints to obtain the final road network of the ground to be extracted.

2. The method for extracting road networks from remote sensing images by global-local decoding according to claim 1, characterized in that: The calling of the Transformer encoder to encode the remote sensing image slice to obtain an initialized road query includes: Extract multi-scale feature maps of each remote sensing image slice through the Transformer encoder; Perform multi-scale hole attention calculation on the multi-scale feature map according to the ground sampling points to obtain a corresponding sampling point query; Select multiple sampling point queries with the largest confidence as the initial road query.

3. The method for extracting road network from remote sensing images by global-local decoding according to claim 1, characterized in that: The step of constructing noise samples according to the ground truth coordinates of the road network to be extracted includes: For each ground truth coordinate in the road network to be extracted, two random noise values ​​are added; When the random noise value is less than or equal to the preset noise threshold, the ground truth coordinates with the added random noise value are taken as positive samples; When the random noise value is greater than the noise threshold and less than twice the noise threshold, the ground truth coordinates with the added random noise value are taken as negative samples; The positive and negative samples of each ground truth coordinate are used as noise samples.

4. The method for extracting road network from remote sensing images by global-local decoding according to claim 1, characterized in that: The combined features constructed by using the 2D coordinates and the 36-dimensional direction descriptors are input into a topological connection network to obtain a primary road network of the road network ground to be extracted, including: Mapping the 2D coordinates into coordinate feature representations through a coordinate multilayer perceptron, and mapping the 36-dimensional direction descriptors into direction feature representations through a direction multilayer perceptron; The coordinate feature representation and the direction feature representation are concatenated to obtain a combined feature; In the topological connection network, for each target road node, adjacent road nodes of the target road node are determined, and combined features of the target road node and the adjacent road nodes are merged to obtain a merged feature; Calling an activation function to activate the merged features to obtain a corresponding feature tensor; Calling the fully connected layer to map the feature tensor to obtain the corresponding tensor value; When the tensor value is greater than the tensor threshold, the target road node is connected to the adjacent road nodes to obtain a primary road network of the road network ground to be extracted.

5. The method for extracting road network from remote sensing images by global-local decoding according to claim 1, characterized in that: In the process of iteratively searching the road endpoints in the primary road network, the road network is completed for the road endpoints to obtain the final road network of the road network to be extracted, including: Constructing an initial retrieval point set according to road endpoints in the primary road network, wherein the road endpoints are road nodes whose number of adjacent road nodes in the primary road network is less than 2; Selecting a center point from the initial search point set according to a preset iteration strategy; For each selected center point, a local remote sensing image is extracted from the remote sensing image slice, and a local road network grid result is extracted from the primary road network; A local road query of the center point is determined according to the local remote sensing image and the local road network grid result, and a road network is completed in the primary road network according to the local road query to obtain a final road network of the ground to be extracted.

6. The method for extracting road network from remote sensing images by global-local decoding according to claim 5, characterized in that: The preset iteration strategy includes: In the first round of iteration, a road endpoint is randomly selected from the initial search point set as the starting center point; If the number of road nodes in the local road query of the starting center point is 0, randomly select a road endpoint from the candidate point set as the center point of the next iteration, wherein the candidate point set is a point set consisting of the remaining road endpoints in the initial search point set except the starting center point; If the number of road nodes in the local road query is 1, the road node in the local road query is used as the center point of the next iteration; If the number of road nodes in the local road query is 2, randomly select a road node in the local road query as the center point of the next iteration, and add the remaining road nodes in the local road query to the candidate point set; When the candidate point set is an empty set, the iteration ends.

7. The method for extracting road network from remote sensing images by global-local decoding according to claim 1, characterized in that: The method further comprises: The Transformer encoder, the Transformer decoder, and the topological connection network are integrated into a road network extraction model, and the training process of the road network extraction model includes: Acquire a road network remote sensing image sample, wherein the road network remote sensing image sample includes a road network connection true value; Inputting the road network remote sensing image sample into the road network extraction model for forward propagation; In the parallel stage of forward propagation, a parallel stage loss function is constructed according to the road network connection true value and the coordinate prediction value, direction prediction value, and node connection prediction value of the road node output by the road network extraction model; In the iterative phase of forward propagation, an iterative phase loss function is constructed according to the true value of the road network connection and the predicted probability value of the road endpoint output by the road network extraction model; The sum of the parallel stage loss function and the iterative stage loss function is used as the final loss function, and the final loss function is used to perform back propagation in the road network extraction model to update the parameters of the road network extraction model.

8. The method for extracting road network from remote sensing images by global-local decoding according to claim 7, characterized in that: The process of constructing the road network connection true value includes: In the true value road map of the road network remote sensing image sample, rasterization is performed according to line segments of preset pixel lengths; When a road node in the road network remote sensing image sample is within the range of the line segment, determining the road node as a valid node; For each valid node, a corresponding projection point is determined in the true value road map, where the projection point is a center line point in the true value road map with the smallest Euclidean distance to the valid node; According to the connection relationship of the projection points, the connectivity relationship of the road nodes in the road network remote sensing image sample is determined, and based on the connectivity relationship, the corresponding road nodes in the road network remote sensing image sample are connected to obtain the road network connection true value.

9. The method for extracting road network from remote sensing images by global-local decoding according to claim 7, characterized in that: The construction process of the parallel stage loss function includes: Determine the coordinate true value, direction true value, and node connection true value of the road node from the road network connection true value; Constructing a position loss according to the true value of the coordinate and the predicted value of the coordinate; Constructing a direction loss according to the true direction value and the predicted direction value; Constructing a connection loss according to the true value of the node connection and the predicted value of the node connection; The position loss, the direction loss and the connection loss are weighted respectively according to the preset first position weight, direction weight and connection weight, and the weighted results are summed to obtain a parallel stage loss function, wherein the values ​​of the direction weight and the connection weight are positively correlated with the training rounds of the road network extraction model; The process of constructing the loss function in the iteration phase includes: Determining the true probability value of the road endpoint from the road network connection true value; Constructing a probability loss of a road endpoint according to the true probability value and the predicted probability value; The position loss and the probability loss are weighted respectively by a preset second position weight and a probability weight, and the weighted results are summed to obtain an iterative phase loss function.

10. A global-local decoding remote sensing image road network extraction device, characterized in that: include: The encoding module is used to obtain the remote sensing image slices of the road network to be extracted, and call the Transformer encoder to encode the remote sensing image slices to obtain the initialization road query; A construction module, for constructing noise samples according to the ground truth coordinates of the road network ground to be extracted; A decoding module is used to input the initialization road query and the noise sample into the Transformer decoder for decoding processing to obtain a candidate road query that completes information exchange, and call the Hungarian algorithm to perform two-part matching on the candidate road query to obtain a target road query; A global query module is used to call a global prediction head to map the target road query into 2D coordinates and 36-dimensional direction descriptors of road nodes, and input the combined features constructed by using the 2D coordinates and the 36-dimensional direction descriptors into a topological connection network to obtain a primary road network of the road network to be extracted; The local query module is used to complete the road network of the road endpoints in the primary road network during the iterative search of the road endpoints to obtain the final road network of the ground to be extracted.

Images