Remote sensing image path planning methods, systems, equipment and media

Abstract

This invention relates to a remote sensing image path planning method, system, device, and medium. The remote sensing image path planning method includes: employing a combination of D-LinkNet and HED network structures to obtain a fully convolutional neural network road extraction model, with side output layers and a fusion layer; constructing a new loss function to replace the original loss function of the D-LinkNet network structure, which is constructed by adding cross-entropy and Dice coefficient losses; training the road extraction model based on a preset training set; using the trained road extraction model to extract roads in a selected study area, obtaining a road membership map in raster data form; and performing path planning on the road membership raster map based on the A* algorithm to generate a travel path. This invention improves the accuracy of image road extraction and can generate reasonable path planning schemes even in the absence of road network data.

Description

Technical Field

[0001] This invention relates to the field of path planning technology. In particular, it relates to a method, system, device, and medium for path planning of remote sensing images. Background Technology

[0002] Traditional route planning methods mainly rely on road vector data to construct road networks and then perform route planning based on the vector network. For areas where road vector data is missing but passable roads exist, route planning mainly relies on road network vectors or digital elevation model (DEM) data.

[0003] Research on road network-less path planning using high-resolution optical remote sensing data for complex scenarios is an interdisciplinary and comprehensive research field involving multiple aspects such as geographic information science, remote sensing science, and road traffic science. Currently, research in this field mainly focuses on how to extract key information from high-resolution optical remote sensing data. Traditional methods for accessibility analysis of complex scenarios based on remote sensing data generally involve first extracting the road network after an event occurs from remote sensing imagery, then comparing it with the road network before the event to determine road accessibility; or first using the road network before the event to determine road blockage along the road direction, and then determining road accessibility. In terms of efficiency, these methods generally involve a large number of spatial operations between vectors and raster layers, making the steps complex and inefficient. Regarding accuracy, firstly, although the accuracy of road extraction methods from imagery is constantly improving, problems such as road network discontinuity still exist, making it impossible to guarantee effective road network updates. Secondly, road extraction methods using imagery cannot completely guarantee the representation of regional accessibility; for example, wasteland and farmland may not be identified as accessible areas under special conditions. Meanwhile, due to factors such as registration errors between images and roads and road simplification in map reduction, roads and images cannot be perfectly matched, which limits the accuracy of the accessibility analysis results.

[0004] Deep learning algorithms have played a crucial role in extracting road-related information, such as road accessibility assessment and road blockage information identification. Deep learning technologies, including convolutional neural networks and attention mechanisms, have demonstrated excellent performance in tasks like image recognition, classification, and segmentation, providing new solutions for road information extraction from remote sensing images. By training deep learning models, it is possible to accurately identify road blockage information and assess road accessibility in remote sensing images, significantly improving processing efficiency and accuracy. With the continuous development of deep learning technology, several new methods and techniques have been introduced into research on remote sensing road blockage information extraction. For example, deep learning models based on attention mechanisms can improve their ability to identify road features by learning the weight assignments of different features; transfer learning techniques can utilize pre-trained models to improve the learning efficiency for new tasks; and generative adversarial networks (GANs) can generate more realistic remote sensing images, providing more training data for road information extraction.

[0005] However, research on road-related information extraction based on deep learning still faces several challenges. First, road features in remote sensing images are complex and varied, with significant differences in road type, material, and width across different regions, posing difficulties for extracting road obstruction information. Second, remote sensing images may contain a large amount of non-road interference information, such as buildings and trees, which also affects the extraction results. To address these challenges, researchers have employed various methods to improve the accuracy of road information extraction. How to construct a high-resolution optical remote sensing path planning model without a road network suitable for complex scenarios, thereby reducing interference from non-road areas and improving the accuracy of road information extraction, is an urgent problem to be solved. Summary of the Invention

[0006] This invention provides a remote sensing image path planning method, system, device, and medium to solve the problems in the prior art, which involve a lot of spatial operations between vectors and grids, are complex and inefficient, have untimely road network updates, cannot fully represent road accessibility, and cannot perfectly match roads with images.

[0007] To achieve the above objectives, in a first aspect, the present invention relates to a remote sensing image path planning method, comprising:

[0008] A road extraction model of a fully convolutional neural network is obtained by combining the D-LinkNet network structure and the HED network structure. In the decoding area of ​​the D-LinkNet network structure, a side output layer is set after each feature layer of different scales. Each side output layer outputs an edge map. A fusion layer is also set to connect the output ends of the side output layers to synthesize the edge maps output by each side output layer.

[0009] A new loss function is constructed to replace the original loss function of the D-LinkNet network structure, which is constructed by adding the cross-entropy and Dice coefficient loss. The new loss function is the sum of the loss of the road extraction model output, which is added according to their respective weights, and the loss of each side output layer.

[0010] The road extraction model is trained based on a preset training set;

[0011] The trained road extraction model is used to extract roads in the selected study area, resulting in a road membership map in raster data form.

[0012] The A* algorithm is used to perform path planning on the road membership raster map to generate a travel path.

[0013] Preferably, the side output layer consists of a 1*1 convolutional layer and a deconvolutional layer to ensure that the edge map of the side output matches the size of the original image;

[0014] The fusion layer uses the convolutional layer to learn the optimal combination weights between the outputs of different side output layers.

[0015] Preferably, a new loss function is constructed to replace the original D-LinkNet loss function, which is constructed by adding the cross-entropy and Dice coefficient losses. Specifically:

[0016] Set the inter-class balance coefficient β to calculate the positive sample weights Pos_W of the cross-entropy:

[0017] in, Count_Neg is the total number of negative samples in the road target image, and Count_Pos is the total number of positive samples in the road target image.

[0018] Set the distance weight Dis_W, which uses the distance from the cell to the nearest road centerline as the calculation basis. Effect_Dis is the preset maximum effective distance, which is the distance that contributes to the calculation of road features based on the actual study area. Dis(i) is the pixel distance from the i-th pixel to the nearest road centerline.

[0019] Mathematical morphological thinning is performed on the road target image to obtain the road centerline;

[0020] Find the pixel distance from the nearest road centerline;

[0021] Based on the image resolution, the pixel distance is converted into a spatial Euclidean distance in meters, and then substituted into the formula for distance weight calculation.

[0022] The distance weight Dis_W is obtained, and the pixel's Pos_W and Dis_W are multiplied together to obtain the final weight calculation result, which is then input into the cross-entropy function with the weights as the new loss function of the network.

[0023] Based on the new loss function, the D-LinkNet network structure is improved.

[0024] Preferably, training the road extraction model based on a preset training set includes:

[0025] The training and validation sample sets are created by converting the road network vectors into graticule to obtain road samples, which are then used as the input target layer of the neural network.

[0026] Both the target layer and the indicator layer use a combination of open-source datasets and collected historical data, and the data structures of the training sample set and the validation sample set both use open-source datasets.

[0027] The road extraction model is trained based on the training samples in the training sample set;

[0028] The road extraction model is validated based on the validation samples to obtain a trained road extraction model.

[0029] Preferably, the step of performing path planning on the road membership raster map based on the A* algorithm to generate a travel path specifically involves:

[0030] The A* algorithm uses an evaluation function f(n) to select the next node to expand. At each point of expansion, the node with the smallest f(n) value is selected and inserted into the list of possible paths to generate a path. The evaluation function f(n) consists of two parts: f(n) = g(n) + h(n), where g(n) is the actual cost from the starting point to node n, and h(n) is the estimated cost from node n to the destination.

[0031] The A* algorithm defines h(n) as the sum of the straight-line distance from node n to the destination and the difficulty of traversing node n:

[0032] h(n) = Dist(n) + Diff(n)

[0033] Diff(n) = 1 - DoM(n)

[0034] In the formula, Dist(n) is the straight-line distance from node n to the destination, Diff(n) is the difficulty of passing through node n, which is located in the interval [0,1], and DoM(n) is the road membership degree of node n.

[0035] Preferably, this includes: adjusting the numerical range of the road membership degree of node n, whereby the adjusted h(n) is defined as:

[0036] h(n) = Dist(n) + Diff(n) * Dist(s,e), where Dist(n) is the straight-line distance from node n to the destination, Diff(n) is the difficulty of traversing node n, and Dist(s,e) is the straight-line distance from the starting point to the destination.

[0037] Preferably, the method of generating a travel path by performing path planning on the road membership raster map based on the A* algorithm further includes: sequentially scaling up the road membership map by multiples from large to small until the difference between the path planning results of two adjacent scales is less than a set threshold, at which point the calculation is stopped and the result of the last path planning is retained as the travel path.

[0038] Preferably, the step of stopping the operation when the difference between the path planning results of two adjacent scales is less than a set threshold is specifically as follows: the path planning result at an n-fold scale up is resampled to obtain a first path planning result, so that the first path planning result and the path planning result at an n-fold scale up have the same resolution; the path planning result at an n-fold scale up is subjected to a morphological expansion operation of 2 times to obtain a second path planning result; the operation stops when the ratio of the line width of the second path planning result to the line width of the first path planning result at a 2n-fold scale up after resampling is greater than a set threshold T, where T∈(1,0), and the result of the last path planning is retained, where n is a positive integer.

[0039] To achieve the above objectives, in a second aspect, the present invention relates to a remote sensing image path planning system, comprising:

[0040] The model building module is used to combine the D-LinkNet network structure and the HED network structure to obtain a fully convolutional neural network road extraction model. In the decoding area of ​​the D-LinkNet network structure, a side output layer is set after each feature layer of different scales. Each side output layer outputs an edge map. A fusion layer is also set to connect the output ends of the side output layers to synthesize the edge maps output by each side output layer.

[0041] The loss function update module is used to construct a new loss function to replace the original loss function of the D-LinkNet network structure, which is constructed by adding the cross-entropy and Dice coefficient loss. The new loss function calculates the loss of the final fused road extraction model output and the loss of each side output layer, and adds them together according to the weights as the total loss.

[0042] The model training module is used to train the road extraction model based on a preset training set;

[0043] The road membership map acquisition module is used to extract roads in the selected study area using the trained road extraction model, and obtain a road membership map in the form of raster data.

[0044] The route planning module is used to perform route planning on the road membership raster map based on the A* algorithm and generate travel routes.

[0045] To achieve the above objectives, in a third aspect, the present invention also relates to an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the above-described remote sensing image path planning method.

[0046] To achieve the above objectives, in a fourth aspect, the present invention also relates to a computer-readable storage medium storing instructions that, when executed, perform the aforementioned remote sensing image path planning method.

[0047] The present invention relates to a remote sensing image path planning method, system, device, and medium, which has the following advantages compared to the prior art:

[0048] This invention enables the direct extraction of path planning results from remote sensing imagery with complex scene conditions. The method, based on the HEDLink-RoadNet neural network and the adaptive A* algorithm, allows for direct path planning from remote sensing imagery without relying on road vector data. A high-resolution optical remote sensing road network-free path planning model suitable for complex scenes is constructed, thus achieving path planning without the need for pre-constructed road networks. The neural network effectively extracts and fuses road condition feature information in complex scenes. A deep neural network-based path planning method is used to acquire path planning information from remote sensing imagery, achieving end-to-end information extraction and replacing the cumbersome and time-consuming road extraction and vectorization steps in traditional remote sensing path planning methods. This not only simplifies the processing, lowers the technical threshold and computational complexity, but also significantly improves the efficiency and accuracy of path planning information acquisition. Based on this, the accuracy and efficiency of path planning are improved, ensuring the stable operation of the path planning system in complex scenes.

[0049] This method has applications in many scenarios, such as disaster emergency response, geological survey route analysis, or route planning in remote areas without road infrastructure. It not only improves the flexibility and adaptability of route planning systems but also provides new solutions for scenarios that traditional route planning methods struggle to cover.

[0050] By combining the D-LinkNet network structure with the HED (Holistically-Nested Edge Detection) network structure, a network structure more suitable for road extraction from remote sensing images was constructed, referencing both network structures. At the same time, a new loss function was proposed and replaced the original D-LinkNet loss function, which was constructed by adding the cross entropy and Dice coefficient losses, thereby improving the accuracy of road extraction from images.

[0051] This method can generate reasonable route planning schemes even in the absence of road network data. Furthermore, for areas with available road network data, it eliminates the reliance on road network vector data, thus achieving route planning without vector calculations and road network updates, generating accurate route planning results that meet actual needs. Attached Figure Description

[0052] Figure 1 The following is a flowchart of a remote sensing image path planning method according to Embodiment 1 of the present invention. Figure 1 ;

[0053] Figure 2 The following is a flowchart of a remote sensing image path planning method according to Embodiment 1 of the present invention. Figure 2 ;

[0054] Figure 3 This is a network structure diagram of a road extraction model combining the D-LinkNet network structure and the HED network structure in a remote sensing image path planning method according to Embodiment 1 of the present invention.

[0055] Figure 4 This is a schematic diagram of the structure of a remote sensing image path planning system according to Embodiment 2 of the present invention;

[0056] Figure 5 This is a schematic diagram of the structure of an electronic device according to Embodiment 3 of the present invention. Detailed Implementation

[0057] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.

[0058] Example 1

[0059] A remote sensing image path planning method, please refer to Figures 1-5For path planning without a road network in complex scenarios using high-resolution optical remote sensing data, this method can be implemented using intelligent electronic devices with a central processing unit, such as unmanned vehicles, robot dogs, personal computers, smart terminals, servers, etc. Figure 1 As shown, the procedure includes the following steps: S101 to S106.

[0060] S101 combines the D-LinkNet network structure with the HED network structure to obtain a fully convolutional neural network road extraction model. In the decoding area of ​​the D-LinkNet network structure, a side output layer is set after each feature layer of different scales. Each side output layer outputs an edge map. A fusion layer is also set to connect the output ends of the side output layers to synthesize the edge maps output by each side output layer.

[0061] The specific combination of the D-LinkNet network structure and the HED network structure is as follows: the decoding region structure of the original D-LinkNet network structure is deleted, and a side output layer from the HED network structure is introduced after each feature layer at different scales. Each side output layer outputs an edge map, as shown in the specific structure below. Figure 3 The diagram shows side output layers of different sizes within the decoding region. Additionally, a fusion layer, used in the HED network structure to connect all side output layers, is introduced to synthesize the edge maps output by each side output layer. The specific structure is shown below. Figure 3 The structure after deconvolution in the middle decoding region is shown.

[0062] The side output layer consists of 1x1 convolutional and deconvolutional layers to ensure that the edge map of the side output matches the size of the original image. The fusion layer uses convolutional layers to learn the optimal combination weights between different outputs. Specifically, it uses a convolutional kernel to connect four feature layers (each with a size of 1024x1024 after deconvolution) to the final activation output layer. This convolutional kernel can continuously adjust its weights during network learning, thus the trained result is the optimal combination weights required for fusing the side output layer results.

[0063] The D-LinkNet network structure consists of three parts: a residual network encoding region, a central region, and a decoding region, following an encoder-decoder architecture. The encoding region accurately encodes road information into feature information; the decoding region maps these encoded road features to the spatial domain to achieve accurate road segmentation. The D-LinkNet network model introduces dilated convolutional layers in the central region without adding additional learning parameters, greatly simplifying the training process. In the construction of the encoding region, D-LinkNet uses one initial convolutional module and four residual modules based on a ResNet34 pre-trained network structure. The residual modules improve the generalization representation ability of region blocks through skip connections. In the core part of the central region, D-LinkNet implements a hybrid connection system combining serial and parallel connections. D-LinkNet receives 1024×1024 images as input and uses a ResNet34 pre-trained on the ImageNet dataset as its encoder. ResNet34 was originally designed for classifying medium-resolution images of size 256×256. Using pooling layers could reduce the resolution of the central feature map and lose spatial information. Therefore, D-LinkNet uses dilated convolutional layers to enhance the network's ability to perceive road feature information. Compared to using pooling layers, dilated convolutional layers can maintain high-resolution feature maps while avoiding the loss of spatial information.

[0064] The design philosophy of the HED network structure is also applicable to road extraction. Firstly, HED was initially designed for edge detection; therefore, the network was defined from the outset as a pure binary classification problem, requiring only accurate edge location rather than strong semantic information, making it more suitable for road recognition tasks. Secondly, the design philosophy of the HED network aims for each layer to output useful information, which is then combined through an ensemble approach. Shallow layers are used for recognizing object outlines, while deeper layers, containing a larger receptive field, are better suited for recognizing semantic information such as object categories. Finally, the HED network structure incorporates a unique loss function to address the class imbalance problem.

[0065] This invention incorporates the design features of D-LinkNet and HED network structures, and proposes a more targeted network structure for image road extraction problems—HEDLink-RoadNet.

[0066] HEDLink-RoadNet primarily integrates the advantages of D-LinkNet and HED network structures in terms of both network architecture design and loss function design.

[0067] In terms of network architecture design, a key feature of the HED model is the addition of side-output layers after convolutional layers at different levels. Each side-output layer outputs an edge map, and a fusion layer merges the edge maps output by each side-output layer to obtain the final edge detection result. This strategy effectively combines feature representations from different levels, greatly improving the accuracy and robustness of edge detection.

[0068] The D-LinkNet network structure is modified based on the design of the side output layer by adding a side output layer after each feature layer at different scales: In the decoder part of D-LinkNet, each feature map typically corresponds to a different scale of the original image. After each such feature layer, a side output layer can be added. This layer typically consists of convolutional layers (possibly 1x1 convolutions to adjust the number of channels) and possibly upsampling layers to ensure that the edge map of the side output matches the size of the original image or has a similar resolution. The goal of each side output layer is to capture and emphasize the edge information at that feature level. This can be achieved by training these layers to output a probability map or edge map, where each pixel value represents the probability that the location is an edge.

[0069] To generate the final edge detection result, a fusion layer needs to be designed to synthesize the edge maps from the various side output layers. This fusion layer can be a simple weighted averaging layer or a more complex network structure, such as using convolutional layers to learn the optimal combination weights between the outputs of different layers. To improve the model's flexibility, the fusion layer uses convolutional layers where the weights are learned during training rather than manually set. This allows the model to automatically adjust the contribution of outputs from different layers to adapt to specific tasks and datasets.

[0070] S102 constructs a new loss function to replace the original loss function of the D-LinkNet network structure, which was constructed by adding the cross-entropy and Dice coefficient losses. The new loss function calculates the loss of the final fused road extraction model output and the loss of each side output layer, and adds them together according to their respective weights to obtain the total loss.

[0071] To fully utilize the information from all side output layers and promote their learning during training, a multi-scale loss function was designed. This function not only calculates the loss of the final fused output but also calculates the loss of each side output layer separately, and then sums these losses with certain weights to obtain the total loss. Considering that the importance of outputs at different levels may vary, their losses can be weighted. Typically, the weights of outputs from higher layers (closer to the original image size) can be set higher because they contain richer detail information. The specific network structure of HEDLink-RoadNet is as follows: Figure 2 As shown.

[0072] In this embodiment, the specific steps for setting the new loss function are as follows:

[0073] S1021 sets the inter-class balance coefficient β to calculate the positive sample weights Pos_W for cross-entropy:

[0074] in, Count_Neg is the total number of negative samples in the road target image, and Count_Pos is the total number of positive samples in the road target image.

[0075] S1022 sets the distance weight Dis_W, which uses the distance from the cell to the nearest road centerline as the calculation basis. Effect_Dis is the preset maximum effective distance, which is the distance that contributes to the calculation of road features based on the actual study area. Dis(i) is the pixel distance from the i-th pixel to the nearest road centerline.

[0076] S1023 performs mathematical morphological thinning on the road target image to obtain the road centerline;

[0077] S1024 calculates the pixel distance from the nearest road centerline;

[0078] S1025 converts the pixel distance into spatial Euclidean distance in meters based on the image resolution and substitutes it into the formula to calculate the distance weight.

[0079] S1026 obtains the distance weight Dis_W, multiplies the pixel's Pos_W and Dis_W to obtain the final weight calculation result, and inputs it into the weighted cross-entropy function as a new loss function for the network.

[0080] S1027 improves the D-LinkNet network structure based on a new loss function.

[0081] S103 trains the road extraction model based on a pre-set training set.

[0082] In some embodiments, S103 includes:

[0083] S1031 creates training and validation sample sets by converting road network vectors into graticule to obtain road samples, which are then used as the input target layer of the neural network.

[0084] Both the target layer and the indicator layer of S1032 use a combination of open-source datasets and collected historical data. The data structures of the training sample set and the validation sample set both use open-source datasets.

[0085] Before conducting network training, it is essential to create training and validation datasets. The target layer of the datasets can utilize information provided by existing road networks or open-source road network data such as OpenStreetMap (OSM). The data structure of the datasets references open-source datasets, while also making reasonable modifications and adjustments based on the model's input requirements. Ensuring that the samples cover diverse terrains and landforms guarantees the model's adaptability to various real-world scenarios.

[0086] S1033 trains the road extraction model based on training samples in the training sample set;

[0087] S1034 validates the road extraction model based on the validation samples to obtain a trained road extraction model.

[0088] S104 uses a trained road extraction model to extract roads in the selected study area, obtaining a road membership map in raster data form.

[0089] S105 uses the A* algorithm to perform path planning on the road membership raster map and generate travel routes.

[0090] A traffic difficulty map is generated based on the road membership result map. Following the principle of minimum traffic difficulty, path planning is performed to generate a traffic path between the start and end points. An adaptive A* path search algorithm is proposed to search for the optimal path on the traffic difficulty map.

[0091] The A* pathfinding algorithm is a widely used heuristic search algorithm, particularly suitable for solving optimal path search and discovery problems in graphs. It is widely applied in many fields, including robot navigation, game development, and map navigation. The A* algorithm calculates the cost of every possible node reachable from the current position and then selects the node with the lowest cost to add to the search space. This newly added node is then used to generate more possible paths. The A* algorithm combines the advantages of Dijkstra's algorithm and the greedy best-first search algorithm by introducing a heuristic function to guide the search process, thereby improving search efficiency while ensuring the optimal path is found.

[0092] In this embodiment, S105 specifically refers to:

[0093] The A* algorithm uses an evaluation function f(n) to select the next node to expand. At each point of expansion, the node with the smallest f(n) value is selected and inserted into the list of possible paths. The evaluation function f(n) consists of two parts: f(n) = g(n) + h(n), where g(n) is the actual cost from the starting point to node n, and h(n) is the estimated cost from node n to the destination.

[0094] The A* algorithm defines h(n) as the sum of the straight-line distance from node n to the destination and the difficulty of traversing node n:

[0095] h(n) = Dist(n) + Diff(n)

[0096] Diff(n) = 1 - DoM(n)

[0097] In the formula, Dist(n) is the straight-line distance from node n to the destination, Diff(n) is the difficulty of passing through node n, which is located in the interval [0,1], and DoM(n) is the road membership degree of node n.

[0098] The basic steps of the A* algorithm can be summarized as follows:

[0099] 1. Initialization: Create two lists: an open list (OPEN list) to store nodes to be examined, and a closed list (CLOSE list) to store nodes that have already been examined. Add the starting point to the OPEN list and set its g and f values.

[0100] 2. Select a node: Select the node with the smallest f value from the OPEN table as the current node, and move it from the OPEN table to the CLOSE table.

[0101] 3. Expand Nodes: For each neighboring node of the current node, calculate its g, h, and f values. If the neighboring node is not in the OPEN list, add it to the OPEN list; if the neighboring node is already in the OPEN list, compare the f values ​​of the neighboring node and the current node, and retain the smaller f value.

[0102] 4. Repeat steps: Repeat steps 2 and 3 until the endpoint is found or the OPEN table is empty.

[0103] 5. Backtracking path: Starting from the end point, backtrack to the starting point according to the pointer of the parent node to form the shortest path.

[0104] The selection of the heuristic function h(n) has a significant impact on the algorithm's performance. It should reflect some approximation of the actual cost from the current node to the destination, ensuring the algorithm's optimality. Guided by the heuristic function h(n), the A* algorithm can effectively reduce the search space and avoid unnecessary searches, thereby improving search efficiency. The selection of the heuristic function h(n) has a significant impact on the algorithm's performance and needs to be designed according to the characteristics of the specific problem to adapt to different application scenarios. In practical applications, it may be necessary to consider the complexity of the map and the limitations of computing resources to balance search speed and search accuracy. During the search process, the A* algorithm may encounter multiple nodes with the same f value. In this case, certain strategies (such as prioritizing the expansion of nodes with smaller g values) are needed to avoid getting trapped in local optima.

[0105] Based on this, an adaptive A* algorithm is proposed, which can adapt to different distance units and image resolutions, avoiding the influence of distance units and image resolution on the results. It is used to directly apply the road extraction results from the neural network to path planning. Since the road extraction results from the neural network are a raster-form road membership map, they need to be converted into travel difficulty. The numerical range of the road membership of node n is adjusted. By adjusting the numerical range of the road membership of node n, the influence of the unit of measurement can be avoided, thus automatically adapting to different distance units. Specifically, the straight-line distance from the starting node to the ending node is used as a magnification factor to adjust the numerical range of the travel difficulty.

[0106] By employing an upscaling method on the road membership raster data, these problems can be effectively alleviated, and the efficiency of the algorithm can be improved, while the path planning time can be reduced. When roads in the image are clear and obvious, different upscaling factors have little impact on path planning. However, for unclear, hidden roads in mountainous areas, excessive upscaling may cause fragmentation in the road extraction results, and different upscaling factors have a significant impact on path planning. To improve the efficiency of the A* algorithm, the algorithm further includes: sequentially upscaling the road membership map from large to small factors until the difference between the path planning results of two adjacent scales is less than a set threshold, at which point the calculation stops, and the result of the last path planning is retained as the travel path.

[0107] In this embodiment, the road membership degree of node n is adjusted in terms of numerical range, and the adjusted h(n) is defined as:

[0108] h(n) = Dist(n) + Diff(n) * Dist(s,e), where Dist(n) is the straight-line distance from node n to the destination, Diff(n) is the difficulty of traversing node n, and Dist(s,e) is the straight-line distance from the starting point to the destination.

[0109] By employing an upscaling method on the road membership raster data, these problems can be effectively alleviated, and the efficiency of the algorithm can be improved, while the path planning time can be reduced. When the roads in the image are clear and obvious, different upscaling factors have little impact on path planning. However, for unclear, hidden roads in mountainous areas, excessive upscaling may cause fragmentation in the road extraction results, and different upscaling factors have a significant impact on path planning.

[0110] In this embodiment, in order to improve the efficiency of the A* algorithm, S105 further includes: scaling up the road membership map by multiples from large to small until the difference between the path planning results of two adjacent scales is less than a set threshold, then stopping the calculation and retaining the result of the last path planning as the travel path.

[0111] The specific criteria for determining the difference between the two instances are as follows:

[0112] Specifically, the path planning result at an n-fold upscaling scale is resampled to obtain the first path planning result, which has the same resolution as the path planning result at an n-fold upscaling scale. The path planning result at an n-fold upscaling scale is then subjected to a morphological expansion operation of 2 times to obtain the second path planning result. The operation stops when the ratio of the line width of the second path planning result to the line width of the first path planning result at a 2n-fold upscaling scale after resampling is greater than a set threshold T, where T∈(1,0). The result of the last path planning is then retained, where n is a positive integer.

[0113] The ratio t of the line width of the second path planning result to the line width of the first path planning result after resampling and scaling by 2n times:

[0114]

[0115] Among them, I 2n For 2n times upscaling results, I n The scaling factor is n, ∩ represents the intersection of the results, and ∪ represents the union of the results. The value of n is 4, 2, or 1. When the original image size is 1024*1024 pixels, the corresponding upscaled image sizes are 128*128 pixels, 256*256 pixels, and 512*512 pixels, respectively. The calculation stops when t is greater than the set threshold T, and the result of the last path planning is retained. Testing and comparison show that setting T to 0.85 can simultaneously guarantee efficiency and accuracy.

[0116] Example 2

[0117] A remote sensing image path planning system is provided for path planning without a road network for high-resolution optical remote sensing data in complex scenes. In this embodiment, the system can be implemented using intelligent electronic devices with a central processing unit, such as unmanned vehicles, robot dogs, personal computers, smart terminals, servers, etc. Please refer to [link to relevant documentation]. Figure 4 It includes a model building module 61, a loss function update module 62, a model training module 63, a road membership graph acquisition module 64, and a path planning module 65.

[0118] The model building module 61 is used to combine the D-LinkNet network structure and the HED network structure to obtain a fully convolutional neural network road extraction model. In the decoding area of ​​the D-LinkNet network structure, a side output layer is set after each feature layer of different scales. Each side output layer outputs an edge map. A fusion layer is also set to connect the output ends of the side output layers to synthesize the edge maps output by each side output layer.

[0119] The side output layer consists of a 1*1 convolutional layer and a deconvolutional layer to ensure that the edge map output by the side matches the size of the original image.

[0120] The fusion layer uses convolutional layers to learn the optimal combination weights between the outputs of different side output layers.

[0121] The loss function update module 62 is used to construct a new loss function to replace the original loss function of the D-LinkNet network structure, which is constructed by adding the cross-entropy and Dice coefficient loss. The new loss function calculates the loss of the final fused road extraction model output and the loss of each side output layer, and adds them together according to the weights as the total loss.

[0122] The model training module 63 is used to train the road extraction model based on a preset training set;

[0123] The road membership map acquisition module is used to extract roads in the selected study area using a trained road extraction model, and obtain a road membership map in the form of raster data.

[0124] The route planning module 64 is used to perform route planning on the road membership raster map based on the A* algorithm to generate travel routes.

[0125] In this embodiment, the loss function update module 62 is specifically used for:

[0126] A new loss function is constructed to replace the original D-LinkNet loss function, which sums the cross-entropy and Dice coefficient losses. Specifically:

[0127] Set the inter-class balance coefficient β to calculate the positive sample weights Pos_W for cross-entropy:

[0128] in, Count_Neg is the total number of negative samples in the road target image, and Count_Pos is the total number of positive samples in the road target image.

[0129] Set the distance weight Dis_W, which uses the distance from the cell to the nearest road centerline as the calculation basis. Effect_Dis is the preset maximum effective distance, which is the distance that contributes to the calculation of road features based on the actual study area. Dis(i) is the pixel distance from the i-th pixel to the nearest road centerline.

[0130] Mathematical morphological thinning is performed on the road target image to obtain the road centerline;

[0131] Find the pixel distance from the nearest road centerline;

[0132] Based on the image resolution, the pixel distance is converted into spatial Euclidean distance in meters, and then substituted into the formula to calculate the distance weight.

[0133] The distance weight Dis_W is obtained, and the pixel's Pos_W and Dis_W are multiplied together to obtain the final weight calculation result, which is then input into the cross-entropy function with attached weights as a new loss function for the network.

[0134] Based on the new loss function, the D-LinkNet network structure is improved.

[0135] In this embodiment, the path planning module 64 specifically uses the A* algorithm to select the next node to be expanded using the evaluation function f(n). At each point of expansion, the node with the smallest f(n) value is selected and inserted into the linked list of possible paths to generate a travel path. The evaluation function f(n) consists of two parts: f(n) = g(n) + h(n), where g(n) is the actual cost from the starting point to node n, usually the distance or consumption, and h(n) is the estimated cost (heuristic function) from node n to the destination.

[0136] The A* algorithm defines h(n) as the sum of the straight-line distance from node n to the destination and the difficulty of traversing node n:

[0137] h(n) = Dist(n) + Diff(n)

[0138] Diff(n) = 1 - DoM(n)

[0139] In the formula, Dist(n) is the straight-line distance from node n to the destination, Diff(n) is the difficulty of passing through node n, which is located in the interval [0,1], and DoM(n) is the road membership degree of node n.

[0140] The adjusted h(n) is defined as:

[0141] h(n) = Dist(n) + Diff(n) * Dist(s,e), where Dist(n) is the straight-line distance from node n to the destination, Diff(n) is the difficulty of traversing node n, and Dist(s,e) is the straight-line distance from the starting point to the destination.

[0142] To improve the efficiency of the A* algorithm, the path planning module 64 also includes: scaling up the road membership graph by multiples from large to small until the difference between the path planning results of two adjacent scales is less than a set threshold, at which point the calculation stops and the result of the last path planning is retained as the travel path.

[0143] The calculation stops when the difference between the path planning results of two adjacent scales is less than a set threshold. Specifically, the path planning results at an n-fold scale are resampled to obtain the first path planning result, which has the same resolution as the path planning results at an n-fold scale. The path planning results at an n-fold scale are then subjected to a morphological expansion operation of 2 times to obtain the second path planning result. The calculation stops when the ratio of the line width of the second path planning result to the line width of the first path planning result at a 2n-fold scale after resampling is greater than a set threshold T, where T∈(1,0). The result of the last path planning is retained, where n is a positive integer.

[0144] The remote sensing image path planning system of this embodiment is implemented in the same way as the remote sensing image path planning method described in Embodiment 1, and will not be repeated here.

[0145] Example 3

[0146] like Figure 5As shown, this embodiment relates to an electronic device including at least one processor and a memory communicatively connected to the at least one processor. The memory stores a computer program executable by the at least one processor. The computer program is executed by the at least one processor to enable the at least one processor to execute a remote sensing image path planning method according to Embodiment 1, and to achieve the corresponding beneficial effects of the remote sensing image path planning method, which will not be elaborated further here. The electronic device provided in this embodiment can be a personal computer, such as a desktop computer, all-in-one computer, laptop computer, tablet computer, etc., or it can be a mobile phone, wearable device, PDA, etc. In this embodiment, the electronic device is a flight control computer. The electronic device is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0147] The components of the electronic device 3 may include, but are not limited to: at least one processor 4, at least one memory 5, and a bus 6 connecting different system components (including memory 5 and processor 4).

[0148] Bus 6 includes a data bus, an address bus, and a control bus.

[0149] The memory 5 may include volatile memory, such as random access memory (RAM) 51 and / or cache memory 52, and may further include read-only memory (ROM) 53.

[0150] The memory 5 may also include a program / utility 55 having a set (at least one) of program modules 54, including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0151] The processor 4 executes various functional applications and data processing by running computer programs stored in the memory 5, such as the remote sensing image path planning method described above.

[0152] Electronic device 3 can also communicate with one or more external devices 7 (e.g., keyboard, pointing device, etc.). This communication can be performed through input / output (I / O) interface 8. Furthermore, electronic device 3 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 9. Figure 5 As shown, network adapter 9 communicates with other modules of electronic device 3 via bus 6. It should be understood that, although... Figure 5Not shown, it can be combined with electronic device 3 to use other hardware and / or software modules, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (disk array) systems, tape drives, and data backup storage systems.

[0153] It should be noted that although several units / modules or sub-units / modules of the electronic device have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of the present invention, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided and embodied by multiple units / modules.

[0154] Example 4

[0155] This invention relates to a computer-readable storage medium storing instructions that, when executed, perform a remote sensing image path planning method according to Embodiment 1. The execution process and effects are the same as those of the remote sensing image path planning method described in Embodiment 1, and will not be repeated here.

[0156] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0157] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A remote sensing image path planning method, characterized in that, include: A road extraction model of a fully convolutional neural network is obtained by combining the D-LinkNet network structure and the HED network structure. In the decoding area of ​​the D-LinkNet network structure, a side output layer is set after each feature layer of different scales. Each side output layer outputs an edge map. A fusion layer is also set to connect the output ends of the side output layers to synthesize the edge maps output by each side output layer. A new loss function is constructed to replace the original loss function of the D-LinkNet network structure, which is constructed by adding the cross-entropy and Dice coefficient loss. The new loss function is the sum of the loss of the road extraction model output, which is added according to their respective weights, and the loss of each side output layer. The road extraction model is trained based on a preset training set; The trained road extraction model is used to extract roads in the selected study area, resulting in a road membership map in raster data form. Path planning is performed on the road membership raster map based on the A* algorithm to generate travel routes; Specifically, a new loss function is constructed to replace the original D-LinkNet loss function, which is constructed by adding the cross-entropy and Dice coefficient losses. Set inter-class balance coefficient To calculate the positive sample weights of the cross-entropy. : in, ; , This represents the total number of negative sample pixels in the road target image. This represents the total number of positive pixels in the road target image; Set distance weights based on the distance from the cell to the nearest road centerline. ,in , This is a preset maximum effective distance, which is a distance selected based on the actual study area that contributes to the calculation of road characteristics. It is the first The pixel distance from the nearest road centerline; Mathematical morphological thinning is performed on the road target image to obtain the road centerline; Find the pixel distance from the nearest road centerline; Based on the image resolution, the pixel distance is converted into a spatial Euclidean distance in meters, and then substituted into the formula for distance weight calculation. Obtain distance weights , will the pixel and The multiplication, as the final weight calculation result, is input into the cross-entropy function with accompanying weights and used as the new loss function of the network; Based on the new loss function, the D-LinkNet network structure is improved.

2. The remote sensing image path planning method according to claim 1, characterized in that, The side output layer consists of a 1*1 convolutional layer and a deconvolutional layer to ensure that the edge map of the side output matches the size of the original image. The fusion layer uses the convolutional layer to learn the optimal combination weights between the outputs of different side output layers.

3. The remote sensing image path planning method according to claim 1, characterized in that, The process of training the road extraction model based on a preset training set includes: Training and validation sample sets are created by converting road network vectors into lattices to obtain road samples, which are then used as the target layer of the neural network. Both the target layer and the indicator layer use a combination of open-source datasets and collected historical data, and the data structures of the training sample set and the validation sample set both use open-source datasets. The road extraction model is trained based on the training samples in the training sample set; The road extraction model is validated based on the validation samples to obtain a trained road extraction model.

4. The remote sensing image path planning method according to claim 1, characterized in that, The method of generating travel routes by performing path planning on the road membership raster map based on the A* algorithm is as follows: The A* algorithm uses an evaluation function. To select the next node to expand, at each point of expansion, the node with the smallest... The node with the value is inserted into the linked list of possible paths to generate a travel path, where the evaluation function... It consists of two parts: ,in The actual cost from the starting point to node n. This represents the estimated cost from node n to the destination. A* Algorithm Definition The sum of the straight-line distance from node n to the destination and the difficulty of traversing node n: ； ； In the formula, Let n be the straight-line distance from node n to the endpoint. Let be the difficulty of traversing node n, which lies in the interval [0, 1]. Let n be the road membership degree of node n.

5. The remote sensing image path planning method according to claim 4, characterized in that, include: The numerical range of the road membership degree of node n is adjusted, and the adjusted value is... Defined as: In the formula, Let n be the straight-line distance from node n to the endpoint. Let n be the difficulty of traversing node n. This is the straight-line distance from the starting point to the ending point.

6. The remote sensing image path planning method according to claim 4, characterized in that, The method of generating a travel path by performing path planning on the road membership raster map based on the A* algorithm further includes: sequentially scaling up the road membership map from large to small scales until the difference between the path planning results of two adjacent scales is less than a set threshold, at which point the calculation stops and the result of the last path planning is retained as the travel path.

7. The remote sensing image path planning method according to claim 6, characterized in that, The operation stops when the difference between the path planning results of two adjacent scales is less than a set threshold. Specifically, the path planning results at a scale of 2n are resampled to obtain a first path planning result, so that the first path planning result has the same resolution as the path planning result at a scale of n. The path planning result at a scale of n is then subjected to a morphological expansion operation of 2 times to obtain a second path planning result. The operation stops when the ratio of the line width of the second path planning result to the line width of the first path planning result at a scale of 2n after resampling is greater than a set threshold T, where T∈(1,0). The result of the last path planning is retained, where n is a positive integer.

8. A remote sensing image path planning system, characterized in that, include: The model building module is used to combine the D-LinkNet network structure and the HED network structure to obtain a fully convolutional neural network road extraction model. In the decoding area of ​​the D-LinkNet network structure, a side output layer is set after each feature layer of different scales. Each side output layer outputs an edge map. A fusion layer is also set to connect the output ends of the side output layers to synthesize the edge maps output by each side output layer. The loss function update module is used to construct a new loss function to replace the original loss function of the D-LinkNet network structure, which is constructed by adding the cross-entropy and Dice coefficient loss. The new loss function calculates the loss of the final fused road extraction model output and the loss of each side output layer, and adds them together according to the weights as the total loss. The model training module is used to train the road extraction model based on a preset training set; The road membership map acquisition module is used to extract roads in the selected study area using the trained road extraction model, and obtain a road membership map in the form of raster data. The route planning module is used to perform route planning on the road membership raster map based on the A* algorithm and generate travel routes. Specifically, the loss function update module is used for: Set inter-class balance coefficient To calculate the positive sample weights of the cross-entropy. : in, ; , This represents the total number of negative sample pixels in the road target image. This represents the total number of positive pixels in the road target image; Set distance weights based on the distance from the cell to the nearest road centerline. ,in , This is a preset maximum effective distance, which is a distance selected based on the actual study area that contributes to the calculation of road characteristics. It is the first The pixel distance from the nearest road centerline; Mathematical morphological thinning is performed on the road target image to obtain the road centerline; Find the pixel distance from the nearest road centerline; Based on the image resolution, the pixel distance is converted into a spatial Euclidean distance in meters, and then substituted into the formula for distance weight calculation. Obtain distance weights , will the pixel and The multiplication, as the final weight calculation result, is input into the cross-entropy function with accompanying weights and used as the new loss function of the network; Based on the new loss function, the D-LinkNet network structure is improved.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements a remote sensing image path planning method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: The storage medium stores instructions that, when executed, perform a remote sensing image path planning method as described in any one of claims 1-7.

Images