Road network data generation method and device, equipment, storage medium and program product

Abstract

The invention provides a road network data generation method and device, electronic equipment, a computer readable storage medium and a computer program product, and relates to the technical fields of unmanned aerial vehicle aerial photography, map generation, automatic driving navigation and the like. The method comprises the following steps: determining each target area lacking road network data; based on the representative points of each target area, generating a complete graph in which a connection relationship is established between any two representative points; determining a flight planning route based on the complete graph and a preset path optimization algorithm; obtaining an orthographic aerial image shot by the unmanned aerial vehicle flying according to the flight planning route; and determining road network data of each target area based on the orthographic aerial image. According to the method, the flight height advantage of the unmanned aerial vehicle is fully utilized, a reasonable flight planning path is arranged for the unmanned aerial vehicle, so that the orthographic aerial image used for extracting the road network data is conveniently obtained, missing road network data can be well complemented, and better automatic driving service can be conveniently provided.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, specifically to technical fields such as drone aerial photography, map generation, and autonomous driving navigation, and in particular to a road network data generation method, device, electronic device, computer-readable storage medium, and computer program product. Background Art

[0002] Road network data is mainly collected by collection vehicles equipped with professional equipment. However, due to the influence of road speed and traffic capacity, the lack of new road perception methods (official information, user reports) and the actual physical distance limit, the collection cost of new roads is high, the response speed is slow, or even impossible to collect. Summary of the invention

[0003] The embodiments of the present disclosure provide a method, device, electronic device, computer-readable storage medium, and computer program product for generating road network data.

[0004] In a first aspect, an embodiment of the present disclosure proposes a method for generating road network data, including: determining each target area for which road network data is missing; based on representative points of each target area, generating a complete graph in which a connection relationship is established between any two representative points; based on the complete graph and a preset path optimization algorithm, determining a flight planning route; obtaining orthophoto aerial images taken by a drone flying along the flight planning route; and determining the road network data of each target area based on the orthophoto aerial images.

[0005] In the second aspect, an embodiment of the present disclosure proposes a road network data generating device, comprising: a target area determination unit, configured to determine each target area for which road network data is missing; a complete graph generation unit, configured to generate a complete graph in which a connection relationship is established between any two representative points based on representative points of each target area; a flight planning route determination unit, configured to determine the flight planning route based on the complete graph and a preset path optimization algorithm; an aerial image acquisition unit, configured to acquire orthophoto aerial images taken by a drone flying along the flight planning route; and a road network data generation unit, configured to determine the road network data of each target area based on the orthophoto aerial images.

[0006] In a third aspect, an embodiment of the present disclosure provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can implement the road network data generation method described in any implementation method in the first aspect when executing.

[0007] In a fourth aspect, an embodiment of the present disclosure provides a non-transitory computer-readable storage medium storing computer instructions, which are used to enable a computer to implement the road network data generation method described in any implementation method of the first aspect when executed.

[0008] In a fifth aspect, an embodiment of the present disclosure provides a computer program product comprising a computer program, which, when executed by a processor, can implement the steps of the road network data generation method described in any implementation manner in the first aspect.

[0009] The road network data generation scheme provided by the present disclosure, for target areas where road network data is missing, first abstracts the area into representative points, then constructs a complete graph by connecting the representative points, and then determines a flight planning route suitable for UAV flight based on the complete graph and a preset path optimization algorithm, so that the UAV can obtain the required orthophoto aerial images of the target area by shooting according to the flight planning route, and finally determine the road network data of each target area from the orthophoto aerial images. That is, this scheme makes full use of the flight altitude advantage of the UAV, and conveniently obtains the orthophoto aerial images used to extract road network data by arranging a reasonable flight planning path for it, which can better complete the missing road network data, so as to provide better autonomous driving services.

[0010] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Other features, objects and advantages of the present disclosure will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0012] Figure 1 is an exemplary system architecture in which the present disclosure may be applied;

[0013] Figure 2 A flowchart of a method for generating road network data provided by an embodiment of the present disclosure;

[0014] Figure 3 A flowchart of a method for determining a target area based on a user trajectory provided by an embodiment of the present disclosure;

[0015] Figure 4 A flowchart of a method for generating a flight planning route and obtaining orthophoto aerial images provided in an embodiment of the present disclosure;

[0016] Figure 5A flowchart of a method for obtaining target area road network data through an image segmentation network provided by an embodiment of the present disclosure;

[0017] Figure 6 A flowchart of another method for obtaining target area road network data through an image segmentation network provided by an embodiment of the present disclosure;

[0018] Figure 7 A structural block diagram of a road network data generating device provided in an embodiment of the present disclosure;

[0019] Figure 8 A schematic diagram of the structure of an electronic device suitable for executing a road network data generating method provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0020] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description. It should be noted that the embodiments in the present disclosure and the features in the embodiments may be combined with each other without conflict.

[0021] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0022] Figure 1 An exemplary system architecture 100 is shown to which embodiments of the road network data generation method, apparatus, electronic device, and computer-readable storage medium of the present disclosure can be applied.

[0023] like Figure 1 As shown, the system architecture 100 may include terminal devices 101, 102, a server 103, a network 104, and a drone 105. The network 104 is used to provide a medium for communication links between different terminal devices 101, 102 and the server 103, and between the server 103 and the drone 105. The network 104 may include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

[0024] The terminal devices 101 and 102 carried by the user can record the relevant information of the user under the premise of authorization by the user, and send it to the server 103 through the network 104. The server 103 can further generate relevant instructions for controlling the drone 105 based on the analysis and processing of the relevant information received from the user, so as to control the drone 105 to fly and receive relevant images taken by the drone 105 during the flight. Various applications for realizing information communication between the terminal devices 101 and 102, the server 103 and the drone 105 can be installed, such as trajectory recording and collection applications, road network data generation applications, drone flight control applications, etc.

[0025] The terminal devices 101, 102, the server 103 and the drone 105 are usually hardware, and can also be software in a specific scenario (such as a simulation scenario). When the terminal devices 101 and 102 are hardware, they can be various electronic devices with display screens, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, servers or server clusters, etc.; when the terminal devices 101 and 102 are software, they can be installed in the electronic devices listed above, which can be implemented as multiple software or software modules, or as a single software or software module, which is not specifically limited here; when the server 103 is hardware, it can be implemented as a distributed server cluster composed of multiple servers, or as a single server; when the server is software, it can be implemented as multiple software or software modules, or as a single software or software module, which is not specifically limited here; when the drone 105 is hardware, it can be implemented as different models of drone devices with flight and shooting capabilities; when the drone 105 is software, it can be implemented as software or software modules integrated with flight and shooting functions, or as a single software or software module, which is not specifically limited here.

[0026] The server 103 can provide various services through various built-in applications. Taking the road network data generation application that can provide road network data generation service as an example, the server 105 can achieve the following effects when running the road network data generation application: first, according to the relevant user trajectories recorded in the terminal devices 101 and 102 and the historical map information recorded locally received through the network 104, the target areas where the road network data is missing are determined; then, based on the representative points of each target area, a complete graph with a connection relationship established between any two representative points is generated; then, based on the complete graph and a preset path optimization algorithm, the flight planning route is determined; next, the flight planning route is sent to the drone 103 through the network 104, and then the orthophoto aerial images taken by the drone 103 along the flight planning route are received; finally, the road network data of each target area is determined based on the orthophoto aerial images.

[0027] It should be noted that the relevant user trajectories recorded in the terminal devices 101 and 102 can be obtained from the terminal devices 101 and 102 through the network 104, or can be pre-stored in the server 103 in various ways. Therefore, when the server 103 detects that the data has been stored locally (for example, starting to process the tasks to be processed before), it can choose to directly obtain the data locally. In this case, the exemplary system architecture 100 may not include the terminal devices 101 and 102.

[0028] Since the generation of road network data requires more computing resources and stronger computing power, the road network data generation method provided in the subsequent embodiments of the present disclosure is generally performed by the server 103 with stronger computing power and more computing resources. Accordingly, the road network data generation device is generally also set in the server 103. However, it should also be pointed out that when the terminal devices 101 and 102 also have computing power and computing resources that meet the requirements, the terminal devices 101 and 102 can also complete the above-mentioned various operations that are handed over to the server 103 through the road network data generation application installed thereon, and then output the same result as the server 103. Especially in the case where there are multiple terminal devices with different computing powers at the same time, but the road network data generation application determines that the terminal device has stronger computing power and more remaining computing resources, it can let the terminal device perform the above-mentioned operations, thereby appropriately reducing the computing pressure of the server 103. Accordingly, the road network data generation device can also be set in the terminal devices 101 and 102. In this case, the exemplary system architecture 100 may also not include the server 103 and the network 104.

[0029] It should be understood that Figure 1 The number of terminal devices, networks, servers, and drones in the embodiment is only for illustration. Any number of terminal devices, networks, servers, and drones may be provided as required.

[0030] Please refer to Figure 2 , Figure 2 A flowchart of a method for generating road network data provided by an embodiment of the present disclosure, wherein process 200 includes the following steps:

[0031] Step 201: Determine each target area where road network data is missing;

[0032] This step is intended to be performed by the execution subject of the road network data generation method (e.g. Figure 1 The server 103 shown in the figure determines an area where road network data that meets the requirements is missing. For the convenience of the following description, the area where road network data that meets the requirements is missing is referred to as a target area.

[0033] Among them, road network data refers to data that describes the road system and its connection relationships, including information such as the name, type, location, length, intersection, number of lanes, and types of different lanes. These data are usually used in applications such as traffic planning, navigation systems, and geographic information systems to help analyze and optimize traffic flow and routes, and can also be used to support autonomous driving services for autonomous vehicles.

[0034] If any road network data has never been collected in a certain area or only a small amount of road network data has been collected, it can be determined that the area lacks road network data that meets the requirements. If complete road network data has been collected in a certain area before but the freshness is low (for example, it has not been updated for more than 3 months or more), then the area can also lack road network data that meets the requirements.

[0035] Specifically, the target area can be determined by screening and determining from a map library or road network data, or exclusively determined based on the condition that the non-target area has road network data that meets the requirements, or reversely deduced at the practical level based on whether there are some sections of the user trajectory that cannot be matched with the road network data.

[0036] An implementation method including but not limited to may be:

[0037] Areas where the collection vehicle cannot collect road network data by moving are identified as target areas. That is, considering that the road network data of each area is currently collected by the collection vehicle moving on the corresponding road, some areas where the collection vehicle cannot move or cannot collect complete road network data during the moving process should belong to the target area.

[0038] Step 202: Based on the representative points of each target area, a complete graph is generated in which a connection relationship is established between any two representative points;

[0039] On the basis of step 201, this step aims to generate a complete graph in which a connection relationship is established between any two representative points based on the representative points of each target area by the above-mentioned execution subject. That is, the representative point is a point that can represent the corresponding target area. Specifically, any point in the area that can represent the area can be used as the representative point, such as the center point or mass point of the area; and the edge connecting any two representative points not only refers to the connection relationship, but also can reflect the path and path length between the objects represented by the two points in a suitable form in the required scenario, for example, by matching the edge length of the path length, or by giving different (distance) weights to the edge when it cannot be reflected by different edge lengths.

[0040] A feasible implementation method is to simplify the target area and represent it with its center point, and then use spectral clustering (spectral clustering is a clustering algorithm based on graph theory, which is used to deal with clustering problems of complex structures, especially suitable for discovering clusters of non-convex shapes. It uses the eigenvalues ​​and eigenvectors of the graph to reduce the dimension of data and cluster the data) to perform clustering and obtain various density areas, so as to construct a complete graph based on these density areas.

[0041] Among them, the complete graph is a basic concept in graph theory. It is an undirected graph in which each pair of different vertices is connected by an edge. In other words, a complete graph is a graph with an edge between each pair of vertices, and all vertices are connected by edges. As the basic graph type in graph theory, the complete graph plays an important role in many application fields due to its complete edge connection characteristics. It is the basis of many theoretical and algorithmic studies, and it can help understand the structural properties of graphs and the complexity of algorithms.

[0042] This step aims to construct a complete graph by connecting the representative points of each target area, so as to generate interconnected paths between all areas through the complete graph, so as to facilitate subsequent path optimization. For example, find the shortest loop so that the drone can visit all cities and pass through only once. This scheme is used in this step to find the optimal path between different areas, and the same principle can also be used to find the optimal path between sub-areas within a region.

[0043] An implementation method including but not limited to may be:

[0044] First, determine the representative point of each target area, which may include: a center point or a mass point; then, connect any two representative points to obtain a complete graph in which any two representative points are connected by an edge, and the edge has a weight that matches the distance between the two connected representative points, so that the distance between the two areas corresponding to the two connected representative points can be reflected by the size of the weight.

[0045] Step 203: Determine the flight planning route based on the complete graph and a preset path optimization algorithm;

[0046] Based on step 202, this step aims to determine the flight planning route by the above-mentioned execution subject based on the complete graph and the preset path optimization algorithm. Since the path between any two representative points has been presented in the complete graph by constructing edges, a suitable path optimization algorithm is used to find a flight planning path suitable for the UAV flight to pass through as many representative points as possible at one time in the shortest route based on the complete graph.

[0047] Among them, the path optimization algorithm aims to find the best path from the starting point to the end point in the graph, usually optimized according to the shortest distance or the lowest cost. For example, Dijkstra algorithm (principle: starting from the starting point, gradually expanding to the unvisited vertices in the graph, selecting the vertices of the current shortest path for processing until the end point is found. Applicable to graphs with non-negative edge weights), Bellman-Ford algorithm (principle: through multiple relaxation operations, the shortest path estimate of each vertex in the graph is updated, which can handle negative weight edges, but cannot handle negative weight cycles), Floyd-Warshall algorithm (principle: using dynamic programming methods, by updating the shortest paths between all vertex pairs in the graph to calculate the shortest paths of all vertex pairs. Applicable to graphs with negative edge weights but no negative weight cycles), simulated annealing algorithm (simulated annealing algorithm is a heuristic algorithm used to find the optimal solution, especially suitable for dealing with combinatorial optimization problems), etc.

[0048] Step 204: obtaining orthophoto aerial images taken by the drone flying along the planned flight route;

[0049] Based on step 203, this step aims to obtain the orthophoto aerial images taken by the drone flying along the flight planning route by the above-mentioned execution subject. Specifically, to achieve the purpose of this step, the above-mentioned execution subject can first send the flight planning path to the drone in the task execution state, so that the drone flies according to the received flight planning path, and uses the carried camera to complete the image shooting of the passing area during the flight.

[0050] Among them, orthophoto aerial images are geometrically corrected aerial or drone-photographed images that accurately reflect the actual surface conditions geographically. This type of image eliminates distortion caused by shooting angles and terrain undulations, ensuring that each point in the image is aligned with the actual geographic coordinate system. The result is an image with true scale and accurate geographic location that can be used for mapping, geographic information system analysis, road network information collection, and spatial planning.

[0051] Step 205: Determine the road network data of each target area based on the orthophoto aerial image.

[0052] On the basis of step 204, this step aims to enable the above-mentioned execution subject to obtain the road network data of each target area through appropriate analysis and processing based on the relevant image content contained in the orthophoto aerial image, thereby completing the originally missing road network data so that the target area also has road network data that meets the requirements.

[0053] Specifically, considering that the orthophoto aerial images contain relevant information that can determine the road network data, in order to better extract and more efficiently obtain the road network data of each target area, especially when the orthophoto aerial images contain images of all target areas, it is inevitable that some images that do not require road network data will be included when passing through different target areas, and the images of each target area will also contain some images that are irrelevant to the determination of the road network data. Therefore, it is also possible to first split the images of each target area by segmentation, and then extract some key image content related to the road network data of the road from the images of each target area, and then combine the key image content to perform preliminary confirmation and correction of the road network data, and finally obtain the road network data of each target area. Furthermore, in this process, for a target area containing multiple lanes, it is also possible to split it into images corresponding to each lane before extracting the road network data, and then determine the corresponding road network data for each lane image.

[0054] The road network data generation method provided by the disclosed embodiment, for the target area where the road network data is missing, first abstracts the area into representative representative points, then constructs a complete graph by connecting the representative points, and then determines the flight planning route suitable for the UAV flight based on the complete graph and the preset path optimization algorithm, so that the UAV can obtain the required orthophoto aerial images of the target area by shooting according to the flight planning route, and finally determine the road network data of each target area from the orthophoto aerial images. That is, this solution makes full use of the flight altitude advantage of the UAV, and conveniently obtains the orthophoto aerial images used to extract the road network data by arranging a reasonable flight planning path for it, which can better complete the missing road network data, so as to provide better autonomous driving services.

[0055] In step 201 of the previous embodiment, a method for determining the target area is provided based on whether the collection vehicle can collect road network data by traveling. In order to deepen the understanding of how to determine the target area, this embodiment further Figure 3 Another implementation is provided, that is, how to determine the target area based on the user trajectory, wherein process 300 includes the following steps:

[0056] Step 301: Determine a user trajectory set with missing road network data;

[0057] This step aims to first collect all user trajectories with missing road network data by the above-mentioned execution subject, and form a user trajectory set formed by multiple user trajectories.

[0058] It should be noted that the user trajectory with missing road network data described in this step refers to the situation where, in the process of the user traveling along the trajectory, the matching road network data cannot be found on some sections of the trajectory, and thus the corresponding services and support cannot be provided to the users traveling on that section. It is understandable that the reason for this situation is that the road network data of the corresponding sections is missing in the map database, so the target area can be reversely determined through the problems reflected in user practice. Specifically, the set of trajectories that cannot be supplemented due to the lack of existing data (field pictures, satellite pictures, etc.) and cannot be verified in the current period can be obtained from the data engine (DataEngine, English abbreviation: DE) team periodically (such as weekly or monthly).

[0059] Step 302: determining the trajectory speed of each user trajectory in the user trajectory set respectively, and splitting the walking and cycling trajectory set from the user trajectory set according to the trajectory speed;

[0060] Based on step 301, this step aims to distinguish walking tracks, cycling tracks and vehicle tracks according to the speed of the user passing through each user track, and further separate the walking and cycling tracks that are low-speed and only applicable to walking or cycling.

[0061] That is, the vehicle trajectory means that the corresponding road section can be traveled by motor vehicles, so the collection vehicle for collecting road network data should also be able to pass through, and the road network data of the road section can be collected by arranging the collection vehicle to move. Therefore, it is more necessary to focus on the road sections or areas corresponding to the walking and cycling trajectories where the road network data cannot be obtained by the collection vehicle.

[0062] Step 303: performing clustering processing on each walking and cycling trajectory in the walking and cycling trajectory set to obtain a fused trajectory;

[0063] On the basis of step 302, this step aims to perform clustering processing on each walking and cycling trajectory in the walking and cycling trajectory set by the above-mentioned execution subject, so as to fit similar trajectories in the same area by clustering, thereby obtaining a fused trajectory.

[0064] Specifically, the DBSCAN (Density-Based Spatial Clustering of Applications with Noise) algorithm can be used, and 100,000 times the distance between trajectory points is used as the radius distance of the e-neighborhood to cluster the trajectories in the trajectory set, so that similar trajectories are aggregated into one, and PV (position and speed) information is added, which includes the number of walking trajectories and the number of cycling trajectories, so as to facilitate screening according to a pre-set threshold.

[0065] Step 304: The area corresponding to the partial trajectory in the fused trajectory that cannot be matched to any existing road network data is determined as the target area.

[0066] On the basis of step 303, this step aims to determine, by the execution subject, the area corresponding to the partial trajectory in the fused trajectory that cannot be matched to any existing road network data as the target area.

[0067] An implementation method including but not limited to may be:

[0068] First, the part of the fusion trajectory that cannot be matched to any existing road network data is determined as the road network data missing trajectory; then, the road network data missing trajectory is convex hulled to obtain a convex hull; finally, the target area is determined based on the area covered by the convex hull. Specifically, the centroid of the convex hull can be first determined, and then the target area is obtained by taking the centroid as the area center and the preset length as the area radius. Of course, other representative points of the convex hull can also be used as the area center and other lengths as the radius to obtain a similar target area, and the target area may not be a circle, but may be other more appropriate shapes that better match the actual situation.

[0069] Among them, the main purpose of convex hull technology when processing trajectory data is to simplify and optimize data representation: by generating the convex hull (the smallest convex polygon) of the trajectory, reducing the number of complex trajectory data points and retaining key boundary information. The role is to reduce the size of the data set, make data processing and storage more efficient, and accurately extract the spatial boundaries of the trajectory to facilitate the analysis of trajectory coverage. It has the following advantages: simplified data reduces computational complexity and improves analysis speed; simplified data is easier to visualize, helping to intuitively understand the overall shape and coverage area of ​​the trajectory; in spatial analysis, path planning and regional monitoring, reducing redundant points improves processing accuracy and efficiency. That is, when processing and analyzing trajectory data, convex hull technology helps to better understand and apply trajectory information by removing redundant details.

[0070] This embodiment provides a method for determining a target area with missing road network data based on a user trajectory through steps 301 to 304, so that the target area can be reversely inferred based on the feedback of the user trajectory, which can complement the method of forwardly determining the target area to make the determined target area more accurate and comprehensive.

[0071] Based on any of the above embodiments, this embodiment further comprises: Figure 4 A method for generating a flight planning route and obtaining orthophoto aerial images is provided to deepen the understanding of how to obtain a flight planning route based on a complete graph path optimization algorithm and then obtain an orthophoto aerial image. The process 400 includes the following steps:

[0072] Step 401: Based on the weights of the edges connecting the representative points in the complete graph and the simulated annealing algorithm, a complete flight planning route passing through all the representative points is calculated;

[0073] This step aims to calculate the complete flight planning route passing through all representative points based on the weights of the edges connecting the representative points in the complete graph and the simulated annealing algorithm by the above-mentioned execution subject.

[0074] Among them, the weight of the edge connecting each representative point is used to reflect the actual distance of the path. When the simulated annealing algorithm is applied to the path optimization problem of the complete graph, simulated annealing can be used to find a path that passes through all destinations and is as short as possible, thereby avoiding exceeding the maximum flight distance of the drone.

[0075] The general process of finding the shortest path based on a complete graph using the simulated annealing algorithm can be seen in the following points:

[0076] 1) Initialization: Randomly generate an initial path (visit all vertices in a loop).

[0077] 2) Define the objective function:

[0078] The objective function is usually the total length or total cost of the path. The length of the current path is calculated as an evaluation criterion for the current solution.

[0079] 3) Generate neighborhood solutions: Make small random adjustments to the current path, such as swapping the positions of two vertices in the path, to generate a new path.

[0080] 4) Calculate the objective function value of the new solution: calculate the length or cost of the new path.

[0081] 5) Acceptance criterion: If the new path is better (shorter in length), then accept the new path as the current path. If the new path is worse, then accept it with a certain probability to avoid falling into the local optimal solution. The acceptance probability usually depends on the current temperature and the degree of degradation of the solution.

[0082] 6) Gradually lower the temperature: The temperature controls the probability of accepting a worse solution. As the algorithm proceeds, the temperature is gradually lowered to reduce the possibility of accepting an inferior solution and eventually converge to a better solution.

[0083] 7) Termination condition: When the temperature drops to a very low level or after a certain number of iterations, the algorithm terminates and returns the optimal path currently found.

[0084] That is, the simulated annealing algorithm helps to explore possible solutions and find better solutions in the path optimization of the complete graph through the combination of randomness and step-by-step optimization.

[0085] Step 402: In response to the planned flight distance of the complete flight planning route exceeding the maximum flight distance of the UAV, the complete flight planning route is split into at least two sub-flight planning routes that only pass through some representative points according to the maximum flight distance;

[0086] This step is based on the situation that the planned flight distance of the complete flight planning route calculated in step 401 exceeds the maximum flight distance of the drone. Since it exceeds the maximum flight distance of the drone, it means that the drone cannot pass through all target areas in a single flight. Therefore, at least one drone needs to fly twice to obtain orthophoto aerial images of all target areas. This step aims to split the complete flight planning route into at least two sub-flight planning routes that only pass through some representative points according to the maximum flight distance, and the sum of the number of times each sub-flight planning route passes through any representative point is not less than 1, that is, after being split into multiple sub-flight planning routes, for the sake of safety, the areas passed by different flight planning routes can partially overlap. Furthermore, it can also be ensured that each target area is passed through twice through multiple flights, that is, the orthophoto aerial images of each target area taken during two flights are obtained, so as to avoid as much as possible the possible omissions or missing of key information due to single flight shooting.

[0087] Step 403: Obtain orthophoto aerial images taken by the drone flying along different sub-flight planning routes in sequence.

[0088] Based on step 402, this step aims to enable the above-mentioned execution subject to obtain the orthophoto aerial images taken by the drone flying along different sub-flight planning routes in sequence, that is, when the route has been split into at least two sub-flight planning routes, the same drone can be controlled to fly and shoot successively along different sub-flight planning routes.

[0089] This embodiment provides a solution for calculating a complete flight planning route based on a complete graph and a simulated annealing algorithm through steps 401 to 403, and further splits the complete flight planning path when the flight distance corresponding to the complete flight planning route exceeds the maximum flight distance of the drone, so that the drone can finally obtain orthophoto aerial images of all target areas through multiple consecutive flight shootings. That is, the advantage of the simulated annealing algorithm in calculating the shortest path is fully utilized.

[0090] On the basis of any of the above embodiments, in order to deepen the understanding of how to determine the road network data of each target area according to the orthophoto aerial image, this embodiment further Figure 5 A method for obtaining target area road network data through an image segmentation network is provided, wherein the process 500 includes the following steps:

[0091] Step 501: using a preset image segmentation network to segment the orthophoto aerial image to obtain mask images of different roads in each target area;

[0092] This step aims to enable the above-mentioned execution subject to first use the image segmentation network to segment the mask images of different roads contained in each target area from the orthophoto aerial image.

[0093] Among them, the image segmentation network is a deep learning model used to divide an image into several regions with specific labels. Common network structures include U-Net, SegNet, DeepLab, etc. These networks can recognize and segment different objects or regions from images through training, and are usually used for semantic segmentation tasks; the mask image is a binary image or multi-channel image generated by image segmentation, in which the value of each pixel indicates which specific region or category the pixel belongs to. The mask image is usually used to represent the segmentation results of certain specific regions in the image, that is, the mask image can distinguish the target area from the background, which is convenient for subsequent analysis and processing; the segmentation network processes the orthophoto aerial image and outputs the mask image, which represents that each mask image corresponds to a different road area in the image and shows the spatial distribution of each area. The mask image may be a binary image (indicating whether it belongs to a road) or a multi-channel image (indicating different road types).

[0094] That is, this step is to apply image segmentation technology to process orthophoto aerial images to obtain mask images that can clearly identify road areas, thereby achieving accurate identification and extraction of target areas such as roads through precise image segmentation, facilitating subsequent extraction of corresponding roads or determination of road network data.

[0095] A specific implementation method including but not limited to may be:

[0096] A semantic-based image segmentation network is used to segment the orthophoto aerial image to obtain mask images of each lane contained in the target area; wherein the types of lanes include motor vehicle lanes, non-motor vehicle lanes and sidewalks, and the number of lanes of each type includes multiple.

[0097] Taking U-Net as an image segmentation network as an example, the specific process can be:

[0098] First, the encoder in the image segmentation network is used to segment the multi-scale fusion features of each lane contained in the target area from the orthophoto aerial image; then, the decoder in the image segmentation network is used to map the multi-scale fusion features into road masks carrying semantic information; then, the segmenter in the image segmentation network is used to segment the image corresponding to the road mask of the lane type of non-motorized lanes and sidewalks in the orthophoto aerial image into mask images (the reason why the road mask of the motor vehicle lane is not segmented is also because the road network data of the motor vehicle lane can be obtained more conveniently by the collection vehicle, and only the non-motorized lanes and sidewalks that the collection vehicle cannot pass through are more suitable for arranging drones to collect and extract and identify road network data).

[0099] Furthermore, in order to improve the performance of U-Net, orthophoto aerial image data containing walking and cycling roads can be collected in advance, and the categories of pixel points can be marked one by one to construct a multi-label data set of walking roads and cycling roads, and then data enhancement and preprocessing operations are performed on the multi-label data set to improve data diversity and robustness; the processed multi-label data set is then used to perform additional training on the basic U-Net model to further improve the performance of the model on the training set through back propagation; finally, the test set is used to evaluate the model's accuracy, recall rate F1 index and other performance in the walking and cycling road extraction tasks, thereby obtaining a U-Net model that is more suitable for the scenarios targeted by the present disclosure and has more accurate segmentation.

[0100] Step 502: preprocessing the mask images of different roads, and splicing the mask images of the roads after the preprocessing according to the road position information recorded in the standard map to obtain a spliced ​​road mask image;

[0101] Based on step 501, this step aims to perform preprocessing operations on the mask images of different roads by the above-mentioned execution subject, so as to further highlight the key content in the mask image, shield or eliminate the part of the content irrelevant to the extraction of road network information, and splice the mask images of each road after the preprocessing operation according to the road position information recorded in the standard map to obtain the spliced ​​road mask image. In order to facilitate splicing and combining map information, the orthophoto image can also carry the name and coordinate information of the corresponding area.

[0102] Step 503: extracting the road network structure from the spliced ​​road mask image, and determining the road network data of the target area based on the road network structure.

[0103] Based on step 502, this step aims to extract the road network structure (including road centerline, road width and other information) from the spliced ​​road mask image by the above-mentioned execution subject, and further determine the road network data of the target area based on the road network structure. In this process, external instructions can be combined to modify and adjust or supplement relevant information.

[0104] This embodiment provides a method of first using an image segmentation network to segment the orthophoto aerial image through steps 501 to 503 to obtain mask images of different roads in each area, and then preprocessing the mask images to highlight key content, and then splicing them in combination with the positional relationship between the roads recorded in the map to obtain a spliced ​​road mask image, and finally extracting the road structure and determining the road network data therefrom to improve the accuracy of the road network data finally obtained.

[0105] In the above Figure 5 On the basis of the embodiment shown, in order to further deepen the understanding of the specific implementation process, this embodiment also Figure 6 A more specific implementation is provided, wherein the process 600 includes the following steps:

[0106] Step 601: using a preset image segmentation network to segment the orthophoto aerial image to obtain mask images of different roads in each target area;

[0107] To facilitate segmentation, the orthophoto images can also be cropped into small pictures in advance to facilitate subsequent processing.

[0108] Step 602: performing dilation and / or erosion operations on mask images of different roads to obtain noise-removed images;

[0109] Step 603: Fill in the noise points to eliminate the road gaps in the image, and obtain a filled image;

[0110] That is, step 602 and step 603 are intended to be performed by the execution subject to successively use dilation operation and / or erosion operation to eliminate noise points and fill in existing road gaps, so as to use the filled image as the image obtained after the preprocessing operation.

[0111] Step 604: stitching the completed images according to the road location information recorded in the annotated map to obtain a stitched road mask image;

[0112] Step 605: extracting the road network structure from the spliced ​​road mask image;

[0113] Step 606: obtaining a modification instruction for the road topological relationship of different roads in the road network structure, and obtaining supplementary content of attribute information of different roads in the road network structure;

[0114] The purpose of this step is to enable the above-mentioned execution entity to obtain modification instructions for the road topological relationship of different roads in the road network structure and to obtain supplementary content of attribute information for different roads in the road network structure, so as to correct and supplement the road network structure extracted in step 605 according to the modification instructions and supplementary content of attribute information, so as to finally obtain more accurate road network data.

[0115] Step 607: Correct the road network structure according to the modification instruction and the supplementary content of the attribute information to obtain the road network data of the target area.

[0116] Steps 605 to 607 are for the above-mentioned execution subject to further extract the road network structure in the mask image, including information such as the road centerline and road width, and finally correct the road network structure by modifying the road topological relationship and supplementing the road attribute information to finally obtain the road network data.

[0117] Further references Figure 7 As an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of a road network data generating device, and the device embodiment is Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0118] like Figure 7 As shown, the road network data generating device 500 of this embodiment may include: a target area determining unit 701, a complete graph generating unit 702, a flight planning route determining unit 703, an aerial image acquiring unit 704, and a road network data generating unit 705. Among them, the target area determining unit 701 is configured to determine each target area with missing road network data; the complete graph generating unit 702 is configured to generate a complete graph with a connection relationship established between any two representative points based on the representative points of each target area; the flight planning route determining unit 703 is configured to determine the flight planning route based on the complete graph and a preset path optimization algorithm; the aerial image acquiring unit 704 is configured to acquire the orthophoto aerial image taken by the drone flying along the flight planning route; the road network data generating unit 705 is configured to determine the road network data of each target area based on the orthophoto aerial image.

[0119] In this embodiment, the specific processing and technical effects of the target area determination unit 701, the complete graph generation unit 702, the flight planning route determination unit 703, the aerial image acquisition unit 704, and the road network data generation unit 705 in the road network data generation device 700 can be referred to in detail. Figure 2The relevant descriptions of steps 201 - 205 in the corresponding embodiment are not repeated here.

[0120] In some optional implementations of this embodiment, the target area determining unit 701 may include:

[0121] A user trajectory set determination subunit is configured to determine a user trajectory set for which road network data is missing;

[0122] The step-and-ride trajectory set determination subunit is configured to respectively determine the trajectory speed of each user trajectory in the user trajectory set, and separate the step-and-ride trajectory set from the user trajectory set according to the trajectory speed;

[0123] The trajectory fusion unit is configured to perform clustering processing on each walking and cycling trajectory in the walking and cycling trajectory set to obtain a fused trajectory;

[0124] The target area determination subunit is configured to determine the area corresponding to the partial trajectory in the fused trajectory that cannot be matched to any existing road network data as the target area.

[0125] In some optional implementations of this embodiment, the target area determination subunit may include:

[0126] The confirmed trajectory determination module is configured to determine a part of the fused trajectory that cannot be matched to any existing road network data as a road network data missing trajectory;

[0127] A convex hull processing module is configured to perform convex hull processing on the missing trajectory of the road network data to obtain a convex hull;

[0128] The target area determination module is configured to determine the target area based on the area covered by the convex hull.

[0129] In some optional implementations of this embodiment, the target area determination module may be further configured to:

[0130] Determine the centroid of the convex hull;

[0131] The target area is obtained by taking the centroid as the area center and the preset length as the area radius.

[0132] In some optional implementations of this embodiment, the target area determining unit 701 may be further configured to:

[0133] The area where the collection vehicle cannot collect road network data by moving is determined as the target area.

[0134] In some optional implementations of this embodiment, the complete graph generating unit 702 may be further configured to:

[0135] Determine representative points of each target area; wherein the representative points include: center points or mass points;

[0136] Connect any two representative points to obtain a complete graph in which any two representative points are connected by an edge; wherein the edge has a weight matching the distance between the two connected representative points.

[0137] In some optional implementations of this embodiment, the flight planning route determination unit 703 may be further configured to:

[0138] Based on the weights of the edges connecting the representative points in the complete graph and the simulated annealing algorithm, the complete flight planning route passing through all the representative points is calculated.

[0139] In some optional implementations of this embodiment, the road network data generating device 700 may further include:

[0140] The route splitting unit is configured to split the complete flight planning route into at least two sub-flight planning routes that only pass through some representative points according to the maximum flight distance in response to the planned flight distance of the complete flight planning route exceeding the maximum flight distance of the drone; wherein the sum of the number of times each sub-flight planning route passes through any representative point is not less than 1;

[0141] Correspondingly, the aerial image acquisition unit 704 may be further configured as follows:

[0142] Obtain orthophoto aerial images taken by the UAV flying along different sub-flight planning routes in sequence.

[0143] In some optional implementations of this embodiment, the road network data generating unit 705 may include:

[0144] The mask image segmentation subunit is configured to segment the orthophoto aerial image to obtain mask images of different roads in each target area using a preset image segmentation network;

[0145] The preprocessing and splicing subunit is configured to perform preprocessing operations on the mask images of different roads, and splice the mask images of the roads after the preprocessing operations according to the road position information recorded in the standard map to obtain the spliced ​​road mask images;

[0146] The road network structure extraction and road network data determination subunit is configured to extract the road network structure from the spliced ​​road mask image and determine the road network data of the target area based on the road network structure.

[0147] In some optional implementations of this embodiment, the mask image segmentation subunit includes:

[0148] The lane-level mask image segmentation module is configured to use a semantic-based image segmentation network to segment the orthophoto aerial image to obtain the mask image of each lane contained in the target area; wherein the types of lanes include motor vehicle lanes, non-motor vehicle lanes and sidewalks, and the number of lanes of each type includes multiple.

[0149] In some optional implementations of this embodiment, the lane-level mask image segmentation module is further configured to:

[0150] Using the encoder in the image segmentation network, the multi-scale fusion features of each lane contained in the target area are segmented from the orthophoto aerial image;

[0151] Using the decoder in the image segmentation network, the multi-scale fusion features are mapped into a road mask carrying semantic information;

[0152] The image corresponding to the road mask of the lane of the non-motorized vehicle lane and the sidewalk in the orthophoto aerial image is segmented into a mask image by using the segmenter in the image segmentation network.

[0153] In some optional implementations of this embodiment, the preprocessing and splicing subunit is further configured to:

[0154] Performing dilation and / or erosion operations on mask images of different roads to obtain noise-removed images;

[0155] Fill in the noise points to eliminate the road gaps in the image and obtain the completed image;

[0156] The completed images are spliced ​​according to the road location information recorded in the annotated map to obtain a spliced ​​road mask image.

[0157] In some optional implementations of this embodiment, the road network structure extraction and road network data determination subunit includes a road network data determination module configured to determine the road network data of the target area based on the road network structure, and the road network data determination module is further configured to:

[0158] Obtaining instructions for modifying the road topological relationship of different roads in the road network structure;

[0159] Obtain additional attribute information for different roads in the road network structure;

[0160] According to the modification instructions and the supplementary content of the attribute information, the road network structure is corrected to obtain the road network data of the target area.

[0161] This embodiment exists as a device embodiment corresponding to the above method embodiment. The road network data generation device provided by this embodiment, for the target area where the road network data is missing, first abstracts the area into representative representative points, and then constructs a complete graph by connecting the representative points, and then determines the flight planning route suitable for the UAV flight based on the complete graph and the preset path optimization algorithm, so that the UAV can obtain the required orthophoto aerial images of the target area by shooting according to the flight planning route, and finally determine the road network data of each target area from the orthophoto aerial images. That is, this solution makes full use of the flight altitude advantage of the UAV, and conveniently obtains the orthophoto aerial images used to extract the road network data by arranging a reasonable flight planning path for it, which can better complete the missing road network data, so as to provide better automatic driving services.

[0162] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the road network data generation method described in any of the above embodiments can be implemented when the at least one processor executes.

[0163] According to an embodiment of the present disclosure, the present disclosure further provides a readable storage medium, which stores computer instructions, and the computer instructions are used to enable a computer to implement the road network data generation method described in any of the above embodiments when executed.

[0164] According to an embodiment of the present disclosure, the present disclosure also provides a computer program product, which, when executed by a processor, can implement the steps of the road network data generation method described in any of the above embodiments.

[0165] Figure 8 A schematic block diagram of an example electronic device 800 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0166] like Figure 8As shown, the device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0167] A number of components in the device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0168] The computing unit 801 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above, such as the road network data generation method. For example, in some embodiments, the road network data generation method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the road network data generation method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform the road network data generation method in any other appropriate manner (e.g., by means of firmware).

[0169] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0170] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0171] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0172] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0173] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0174] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and virtual private servers (VPS) services.

[0175] According to the technical solution of the disclosed embodiment, for the target area where the road network data is missing, the area is first abstracted into representative points, and then a complete graph is constructed by connecting the representative points, and then a flight planning route suitable for the UAV flight is determined based on the complete graph and the preset path optimization algorithm, so that the UAV can obtain the required orthophoto aerial images of the target area by shooting according to the flight planning route, and finally the road network data of each target area can be determined from the orthophoto aerial images. That is, this solution makes full use of the flight altitude advantage of the UAV, and conveniently obtains the orthophoto aerial images used to extract the road network data by arranging a reasonable flight planning path for it, which can better complete the missing road network data, so as to provide better autonomous driving services.

[0176] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0177] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for generating road network data, comprising: Identify target areas where road network data is missing; Based on the representative points of each target area, a complete graph is generated in which a connection relationship is established between any two representative points; Determine the flight planning route based on the complete graph and a preset path optimization algorithm; Obtaining orthophoto aerial images taken by the drone flying along the planned flight route; The road network data of each target area is determined based on the orthophoto aerial image.

2. The method according to claim 1, wherein: The step of determining target areas where road network data is missing includes: Determine the user trajectory set with missing road network data; Determine the trajectory speed of each user trajectory in the user trajectory set respectively, and separate the walking and cycling trajectory set from the user trajectory set according to the trajectory speed; Performing clustering processing on each walking and cycling trajectory in the walking and cycling trajectory set to obtain a fused trajectory; An area corresponding to a portion of the fused trajectory that cannot be matched to any existing road network data is determined as the target area.

3. The method according to claim 2, wherein: The step of determining the area corresponding to the partial track in the fused track that cannot be matched to any existing road network data as the target area includes: Determine a portion of the fused trajectory that cannot be matched to any existing road network data as a road network data missing trajectory; Performing convex hull processing on the missing trajectory of the road network data to obtain a convex hull; The target area is determined based on the area covered by the convex hull.

4. The method according to claim 3, wherein: The determining the target area based on the area covered by the convex hull comprises: determining the centroid of the convex hull; The target area is obtained by taking the centroid as the area center and the preset length as the area radius.

5. The method according to claim 1, wherein: The step of determining target areas where road network data is missing includes: An area where the collection vehicle cannot collect road network data by traveling is determined as the target area.

6. The method according to claim 1, wherein: The step of generating a complete graph in which a connection relationship is established between any two representative points based on the representative points of each target area includes: Determine a representative point of each target area; wherein the representative point includes: a center point or a mass point; Connect any two of the representative points to obtain a complete graph in which any two of the representative points are connected by an edge; wherein the edge has a weight matching the distance between the two connected representative points.

7. The method according to claim 1, wherein: Determining the flight planning route based on the complete graph and a preset path optimization algorithm includes: Based on the weights of the edges connecting the representative points in the complete graph and a simulated annealing algorithm, a complete flight planning route passing through all the representative points is calculated.

8. The method according to claim 7, further comprising: In response to the planned flight distance of the complete flight planning route exceeding the maximum flight distance of the UAV, the complete flight planning route is split into at least two sub-flight planning routes that only pass through some of the representative points according to the maximum flight distance; wherein the sum of the number of times each of the sub-flight planning routes passes through any of the representative points is not less than 1; Correspondingly, the obtaining of the orthophoto aerial image taken by the drone flying along the planned flight route includes: Obtain orthophoto aerial images taken by the drone flying along different sub-flight planning routes in sequence.

9. The method according to any one of claims 1 to 8, wherein: The determining of the road network data of each target area based on the orthophoto aerial image comprises: Using a preset image segmentation network to segment the orthophoto aerial images to obtain mask images of different roads in each of the target areas; Preprocessing the mask images of the different roads, and splicing the mask images of the roads after the preprocessing according to the road position information recorded in the standard map to obtain a spliced ​​road mask image; A road network structure is extracted from the spliced ​​road mask image, and road network data of the target area is determined based on the road network structure.

10. The method according to claim 9, wherein: The method of using a preset image segmentation network to segment the orthophoto aerial image to obtain mask images of different roads in each target area includes: A semantic-based image segmentation network is used to segment the orthophoto aerial image to obtain a mask image of each lane contained in the target area; wherein the types of lanes include motor vehicle lanes, non-motor vehicle lanes and sidewalks, and the number of lanes of each type includes multiple.

11. The method according to claim 10, wherein: The method of segmenting the orthophoto aerial image using a semantic-based image segmentation network to obtain a mask image of each lane contained in the target area includes: Using the encoder in the image segmentation network, segment the orthophoto aerial image to obtain multi-scale fusion features of each lane contained in the target area; Mapping the multi-scale fusion features into a road mask carrying semantic information using a decoder in the image segmentation network; The image corresponding to the road mask of the lane of the non-motorized vehicle lane and the sidewalk in the orthophoto aerial image is segmented into the mask image by using the segmenter in the image segmentation network.

12. The method according to claim 9, wherein: The preprocessing operation is performed on the mask images of the different roads, and the mask images of the roads after the preprocessing operation are spliced ​​according to the road position information recorded in the standard map to obtain the spliced ​​road mask images, including: Performing dilation and / or erosion operations on mask images of different roads to obtain noise-removed images; Filling the road gap in the noise-eliminated image to obtain a filled image; Each of the completed images is spliced ​​according to the road position information recorded in the annotated map to obtain the spliced ​​road mask image.

13. The method according to claim 9, wherein: The determining the road network data of the target area based on the road network structure includes: Obtaining instructions for modifying road topological relationships between different roads in the road network structure; Obtaining supplementary content of attribute information of different roads in the road network structure; According to the modification instruction and the supplementary content of the attribute information, the road network structure is corrected to obtain the road network data of the target area.

14. A road network data generating device, comprising: A target area determination unit is configured to determine each target area where road network data is missing; A complete graph generating unit is configured to generate a complete graph in which a connection relationship is established between any two representative points based on the representative points of each target area; A flight planning route determination unit is configured to determine the flight planning route based on the complete graph and a preset path optimization algorithm; An aerial image acquisition unit is configured to acquire orthophoto aerial images taken by the drone flying along the flight planning route; The road network data generating unit is configured to determine the road network data of each of the target areas based on the orthophoto aerial image.

15. The device according to claim 14, wherein: The target area determination unit comprises: A user trajectory set determination subunit is configured to determine a user trajectory set for which road network data is missing; a step cycling trajectory set determination subunit, configured to respectively determine a trajectory speed of each user trajectory in the user trajectory set, and separate a step cycling trajectory set from the user trajectory set according to the trajectory speed; A trajectory fusion unit is configured to perform clustering processing on each walking and cycling trajectory in the walking and cycling trajectory set to obtain a fused trajectory; The target area determination subunit is configured to determine the area corresponding to the partial trajectory in the fused trajectory that cannot be matched to any existing road network data as the target area.

16. The device according to claim 15, wherein: The target area determination subunit comprises: A confirmed trajectory determination module is configured to determine a part of the fused trajectory that cannot be matched to any existing road network data as a road network data missing trajectory; A convex hull processing module is configured to perform convex hull processing on the missing trajectory of the road network data to obtain a convex hull; The target area determination module is configured to determine the target area based on the area covered by the convex hull.

17. The device according to claim 16, wherein: The target area determination module is further configured to: determining the centroid of the convex hull; The target area is obtained by taking the centroid as the area center and the preset length as the area radius.

18. The device according to claim 14, wherein: The target area determination unit is further configured to: An area where the collection vehicle cannot collect road network data by traveling is determined as the target area.

19. The device according to claim 14, wherein: The complete graph generation unit is further configured to: Determine a representative point of each target area; wherein the representative point includes: a center point or a mass point; Connect any two of the representative points to obtain a complete graph in which any two of the representative points are connected by an edge; wherein the edge has a weight matching the distance between the two connected representative points.

20. The device according to claim 14, wherein: The flight planning route determination unit is further configured to: Based on the weights of the edges connecting the representative points in the complete graph and a simulated annealing algorithm, a complete flight planning route passing through all the representative points is calculated.

21. The apparatus according to claim 20, further comprising: a route splitting unit, configured to, in response to the planned flight distance of the complete flight planning route exceeding the maximum flight distance of the drone, split the complete flight planning route into at least two sub-flight planning routes that only pass through some of the representative points according to the maximum flight distance; wherein the sum of the number of times each of the sub-flight planning routes passes through any of the representative points is not less than 1; Correspondingly, the aerial image acquisition unit is further configured to: Obtain orthophoto aerial images taken by the drone flying along different sub-flight planning routes in sequence.

22. The device according to any one of claims 14 to 21, wherein: The road network data generating unit comprises: The mask image segmentation subunit is configured to segment the orthophoto aerial image using a preset image segmentation network to obtain mask images of different roads in each of the target areas; The preprocessing and splicing subunit is configured to perform a preprocessing operation on the mask images of the different roads, and splice the mask images of the roads after the preprocessing operation according to the road position information recorded in the standard map to obtain a spliced ​​road mask image; The road network structure extraction and road network data determination subunit is configured to extract the road network structure from the spliced ​​road mask image and determine the road network data of the target area based on the road network structure.

23. The device according to claim 22, wherein: The mask image segmentation subunit comprises: The lane-level mask image segmentation module is configured to use a semantic-based image segmentation network to segment the orthophoto aerial image to obtain mask images of each lane contained in the target area; wherein the types of lanes include motor vehicle lanes, non-motor vehicle lanes and sidewalks, and the number of lanes of each type includes multiple.

24. The device according to claim 23, wherein: The lane-level mask image segmentation module is further configured to: Using the encoder in the image segmentation network, segment the orthophoto aerial image to obtain multi-scale fusion features of each lane contained in the target area; Mapping the multi-scale fusion features into a road mask carrying semantic information using a decoder in the image segmentation network; The image corresponding to the road mask of the lane of the non-motorized vehicle lane and the sidewalk in the orthophoto aerial image is segmented into the mask image by using the segmenter in the image segmentation network.

25. The device according to claim 22, wherein: The preprocessing and splicing subunit is further configured to: Performing dilation and / or erosion operations on mask images of different roads to obtain noise-removed images; Filling the road gap in the noise-eliminated image to obtain a filled image; Each of the completed images is spliced ​​according to the road position information recorded in the annotated map to obtain the spliced ​​road mask image.

26. The device according to claim 22, wherein: The road network structure extraction and road network data determination subunit comprises a road network data determination module configured to determine the road network data of the target area based on the road network structure, and the road network data determination module is further configured to: Obtaining instructions for modifying road topological relationships between different roads in the road network structure; Obtaining supplementary content of attribute information of different roads in the road network structure; According to the modification instruction and the supplementary content of the attribute information, the road network structure is corrected to obtain the road network data of the target area.

27. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the road network data generation method described in any one of claims 1-13.

28. A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute the road network data generation method according to any one of claims 1 to 13.

29. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the steps of the road network data generating method according to any one of claims 1 to 13 are implemented.

Images