Road network generation method, intelligent device and computer-readable storage medium

Abstract

The present application relates to the field of autonomous driving technology, and specifically to a road network generation method, intelligent device and computer-readable storage medium, aiming to solve the technical problem of how to realize the automatic and low-cost generation of road networks in low-speed scenarios. To this end, the present application obtains a point cloud map of the area to be tested. According to the point cloud map and image data, a semantic feature map of the area to be tested is obtained. The road network generation result of the area to be tested is obtained according to the semantic feature map. Through the above configuration, the present application uses image data to perform semantic understanding of the point cloud map, obtain a semantic feature map, and predict the road network generation result based on the semantic feature map. The prediction process emphasizes the understanding of the semantic information of the area to be tested, and does not rely on the historical vehicle driving trajectory in the user return data. It has the advantages of high coverage, less manual intervention, high degree of automation, and low cost.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and specifically to a road network generation method, an intelligent device, and a computer-readable storage medium. Background Art

[0002] With the development of autonomous driving technology, the use of functions has gradually expanded from high-speed to urban areas and low-speed parking, charging and swapping. However, HD / SD maps have not yet fully covered low-speed scenarios. Traditional HD / SD maps have a long acquisition cycle and high manual participation, which limits their expansion and update speed. The algorithm for obtaining maps based on semantics integrates geographic information data from a wide range of users, and has the advantages of low data collection costs and high data freshness.

[0003] For low-speed scenarios, road network topology construction often relies on historical vehicle driving trajectories in user data. However, since there are often a large number of unavailable historical vehicle driving trajectories, it is necessary to manually design rules for screening, which not only limits the degree of automation of road network topology construction, but also increases costs.

[0004] Accordingly, the art needs a new road network generation solution to solve the above problems. Summary of the invention

[0005] In order to overcome the above-mentioned defects, the present application is proposed to solve or at least partially solve the technical problem of how to achieve automatic and low-cost generation of road networks in low-speed scenarios.

[0006] In a first aspect, a road network generation method is provided, the method comprising:

[0007] Obtaining a point cloud map of the area to be measured; the point cloud map is generated based on point cloud data obtained by performing multiple rounds of data collection on the area to be measured; the data obtained by the multiple rounds of data collection also includes image data;

[0008] Acquire a semantic feature map of the area to be measured according to the point cloud map and the image data corresponding to the point cloud data;

[0009] According to the semantic feature map, a road network generation result of the area to be tested is obtained.

[0010] In a technical solution of the above-mentioned road network generation method, the step of obtaining the semantic feature map of the area to be measured according to the point cloud map and the image data corresponding to the point cloud data includes:

[0011] Performing semantic segmentation on the image data to obtain a semantic segmentation result of the image data;

[0012] Fusing the point cloud map with the semantic segmentation result to obtain a global semantic point cloud fusion result;

[0013] The semantic feature map is obtained according to the global semantic point cloud fusion result.

[0014] In a technical solution of the above-mentioned road network generation method, fusing the point cloud map with the semantic segmentation result to obtain a global semantic point cloud fusion result includes:

[0015] The single-frame semantic segmentation result and the single-frame point cloud map with the same timestamp are fused to obtain a single-frame semantic point cloud;

[0016] All single-frame semantic point clouds are stitched together to obtain the global semantic point cloud fusion result.

[0017] In a technical solution of the above-mentioned road network generation method, obtaining the semantic feature map according to the global semantic point cloud fusion result includes:

[0018] The global semantic point cloud fusion result is divided into grids, and the semantic information in each grid is fused to obtain the global semantic point cloud fusion result after grid semantic fusion;

[0019] According to the global semantic point cloud fusion result after the grid semantic fusion, BEV perspective rendering is performed to obtain the semantic feature map.

[0020] In a technical solution of the above-mentioned road network generation method, the fusing of semantic information in each grid includes:

[0021] The semantic information in the grid is fused according to the semantic observation distance and / or the semantic segmentation confidence in the grid.

[0022] In a technical solution of the above road network generation method, obtaining the road network generation result of the area to be tested according to the semantic feature map includes:

[0023] According to a first preset size, the semantic feature map is cropped to obtain a plurality of semantic feature thumbnails;

[0024] For each semantic feature thumbnail, performing image depth feature extraction on the semantic feature thumbnail to obtain image depth features of the semantic feature thumbnail;

[0025] According to the image depth feature, a road network prediction result of the semantic feature thumbnail is obtained; the road network prediction result is a correlation relationship between a road centerline and an intersection point within the range of the semantic feature thumbnail;

[0026] According to the road network prediction results of all semantic feature graphs, the road network generation result of the area to be tested is obtained.

[0027] In a technical solution of the above-mentioned road network generation method, obtaining the road network prediction result of the semantic feature thumbnail according to the image depth feature includes:

[0028] According to the image depth feature, obtaining the road centerline and intersection points contained in the semantic feature thumbnail;

[0029] According to the obtained road center lines and intersection points, association relationship prediction is performed to obtain the road network prediction result of the semantic feature mini-graph.

[0030] In a technical solution of the above road network generation method, obtaining the road network generation result of the area to be tested based on the road network prediction results of all semantic feature graphs includes:

[0031] According to the road network prediction results of all semantic feature graphs, the road center lines and intersection points in the road network prediction results are used as nodes, and the association relationships are used as edges to construct the road network topology relationship of the area to be tested;

[0032] According to the road network topology, a road network generation result of the area to be tested is obtained.

[0033] In a technical solution of the above road network generation method, there is an overlapping area of ​​a second preset size between adjacent semantic feature graphs;

[0034] The step of obtaining a road network generation result of the area to be tested according to the road network topology relationship includes:

[0035] Obtaining the distances of the same type of road network elements in the overlapping area; the road network elements include road centerlines and intersections;

[0036] For the road network elements of the same type whose distance is less than a preset distance threshold, the road network elements are merged;

[0037] Based on the road network topological relationship after the road network elements are fused, the road network generation result of the area to be tested is obtained.

[0038] In a technical solution of the above-mentioned road network generation method, the data obtained by the multiple data collections also include positioning sensor data;

[0039] The step of obtaining a point cloud map of the area to be measured includes:

[0040] A point cloud map of the area to be measured is obtained according to the point cloud data and the positioning sensor data.

[0041] In a second aspect, an intelligent device is provided, comprising at least one processor; and a memory communicatively connected to the at least one processor; wherein a computer program is stored in the memory, and when the computer program is executed by the at least one processor, the method described in any one of the technical solutions of the above-mentioned road network generation method is implemented.

[0042] In a third aspect, a computer-readable storage medium is provided, wherein a plurality of program codes are stored therein, wherein the program codes are suitable for being loaded and run by a processor to execute the method described in any one of the technical solutions of the above-mentioned road network generation method.

[0043] The above one or more technical solutions of the present application have at least one or more of the following beneficial effects:

[0044] In implementing the technical solution of the road network generation method provided by the present application, the present application obtains a point cloud map of the area to be tested. Based on the point cloud map and the image data, a semantic feature map of the area to be tested is obtained. Based on the semantic feature map, the road network generation result of the area to be tested is obtained. Through the above configuration method, the present application uses image data to perform semantic understanding of the point cloud map, obtain a semantic feature map, and predict the road network generation result based on the semantic feature map. The prediction process emphasizes the understanding of the semantic information of the area to be tested, does not rely on the historical vehicle driving trajectories in the user return data, and has the advantages of high coverage, less manual intervention, high degree of automation, and low cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The disclosure of the present application will become easier to understand with reference to the accompanying drawings. It is easy for those skilled in the art to understand that these drawings are only for illustrative purposes and are not intended to limit the scope of protection of the present application. Among them:

[0046] Figure 1 It is a schematic flow chart of the main steps of a road network generation method according to an embodiment of the present application;

[0047] Figure 2 It is a schematic diagram of the main module composition structure of a road network generation method according to an implementation method of an embodiment of the present application;

[0048] Figure 3 It is a flowchart of the main steps of obtaining a semantic feature map according to an implementation method of an embodiment of the present application;

[0049] Figure 4 It is a flowchart of the attention steps of obtaining a road network generation result according to a semantic feature map according to an implementation method of an embodiment of the present application;

[0050] Figure 5is a road network prediction result of a semantic feature thumbnail according to an example of an embodiment of the present application;

[0051] Figure 6 is a schematic flow chart of the main steps of step S102 according to an implementation of an embodiment of the present application;

[0052] Figure 7 is a schematic diagram of the main steps of step S103 according to an implementation of an embodiment of the present application;

[0053] Figure 8 It is a schematic diagram of the main structure of a smart device according to an embodiment of the present application.

[0054] Reference numerals:

[0055] 11: memory; 12: processor. DETAILED DESCRIPTION

[0056] Some embodiments of the present application are described below with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are only used to explain the technical principles of the present application and are not intended to limit the protection scope of the present application.

[0057] In the description of the present application, "module" and "processor" may include hardware, software or a combination of the two. A module may include hardware circuits, various suitable sensors, communication ports, memory, and may also include software parts, such as program code, or a combination of software and hardware. The processor may be a central processing unit, a microprocessor, an image processor, a digital signal processor or any other suitable processor. The processor has data and / or signal processing functions. The processor may be implemented in software, hardware or a combination of the two. Computer-readable storage media include any suitable medium that can store program code, such as a disk, a hard disk, an optical disk, a flash memory, a read-only memory, a random access memory, etc. The term "A and / or B" means all possible combinations of A and B, such as only A, only B or A and B. The term "at least one A or B" or "at least one of A and B" has a similar meaning to "A and / or B" and may include only A, only B or A and B. The singular terms "one" and "the" may also include plural forms.

[0058] The relevant user personal information that may be involved in the various embodiments of this application is strictly in accordance with the requirements of laws and regulations, following the principles of legality, legitimacy and necessity, based on the reasonable purposes of business scenarios, to process the personal information that users actively provide during the use of products / services or generated due to the use of products / services, as well as the personal information obtained with the user's authorization.

[0059] The user personal information processed by this application may vary depending on the specific product / service scenario. It is subject to the specific scenario of the user using the product / service, and may involve the user's account information, device information, driving information, vehicle information or other relevant information. This application will treat the user's personal information and its processing with a high degree of diligence.

[0060] This application attaches great importance to the security of user personal information and has taken security protection measures that meet industry standards and are reasonable and feasible to protect the user's information and prevent personal information from being accessed, publicly disclosed, used, modified, damaged or lost without authorization.

[0061] Refer to the appendix Figure 1 , Figure 1 is a schematic diagram of the main steps of a road network generation method according to an embodiment of this application. As Figure 1 shown, the road network generation method in the embodiment of this application mainly includes the following steps S101 to step S103.

[0062] Step S101: Obtain a point cloud map of the area to be measured; the point cloud map is generated based on the point cloud data obtained from multiple trips of data collection in the area to be measured; the data obtained from multiple trips of data collection also includes image data.

[0063] In this embodiment, multiple trips of data collection can be performed on the area to be measured to obtain multiple trips of point cloud data and image data of the area to be measured. A point cloud map of the area to be measured can be obtained based on the multiple trips of point cloud data collected.

[0064] In one implementation, the data obtained from multiple trips of data collection can also include motion sensor data. Based on the multiple trips of point cloud data and motion sensor data, multiple trips of point cloud maps of the area to be measured can be obtained, and the pose estimation of the point cloud map with global consistency can be obtained according to the motion sensor data, so as to obtain a point cloud map with global consistency. Among them, the motion sensor can be an IMU (Inertial Measurement Unit), GNSS (Global Navigation Satellite System), wheel speed meter, etc. A point cloud map can be obtained based on the commonly used point cloud map acquisition methods in the art, and this application does not limit this.

[0065] In one implementation, data collection can be performed on the area to be measured based on a vehicle equipped with a lidar, a camera, and a motion sensor to obtain the results of multiple trips of data collection. Multiple trips of data collection results can be obtained by performing data collection on the area to be measured multiple times based on the same vehicle; multiple trips of data collection results can also be obtained by performing data collection on the area to be measured based on multiple different vehicles.

[0066] Step S102: Obtain a semantic feature map of the area to be tested based on the point cloud map and the image data corresponding to the point cloud data.

[0067] In this embodiment, semantic information of the point cloud map can be understood based on the image data corresponding to the point cloud data to obtain a semantic feature map of the area to be measured. The image data corresponding to the point cloud data is image data with the same timestamp as the point cloud data, that is, image data collected at the same time as the point cloud data.

[0068] Step S103: Obtain the road network generation result of the area to be tested according to the semantic feature map.

[0069] In this embodiment, the road network generation result of the area to be tested can be obtained according to the semantic feature map.

[0070] In one implementation, the semantic feature map can be input into a deep learning model to predict the association between road network elements and road network elements to obtain a road network generation result for the area to be tested. The road network elements can include road centerlines and intersections.

[0071] In one implementation, the deep learning model may be a Transformer model. The Transformer model uses a semantic feature map as a model input to predict the road network generation result. It only needs to annotate the model training data during the model training phase to achieve automatic prediction of the road network generation result. No manual intervention is required during the prediction phase, which has the advantages of being more efficient and more automated.

[0072] Based on the method described in steps S101 to S103 above, the embodiment of the present application obtains a point cloud map of the area to be tested. According to the point cloud map and the image data, a semantic feature map of the area to be tested is obtained. According to the semantic feature map, the road network generation result of the area to be tested is obtained. Through the above configuration method, the embodiment of the present application uses image data to perform semantic understanding of the point cloud map, obtain a semantic feature map, and predict the road network generation result based on the semantic feature map. The prediction process emphasizes the understanding of the semantic information of the area to be tested, does not rely on the historical vehicle driving trajectory in the user return data, and has the advantages of high coverage, less manual intervention, high degree of automation, and low cost.

[0073] Step S102 and step S103 are further described below.

[0074] In one implementation of the present application, see the attached Figure 6 , Figure 6 FIG. 1 is a flow chart of the main steps of step S102 according to an implementation of an embodiment of the present application. Figure 6As shown, step S102 may further include the following steps S1021 to S1023:

[0075] Step S1021: Perform semantic segmentation on the image data to obtain a semantic segmentation result of the image data.

[0076] In this embodiment, semantic segmentation can be performed on each frame of image data to extract semantic information of the environment of the test area, especially the road, including but not limited to semantic information of drivable areas, road signs, parking spaces, gates, buildings, vegetation, sidewalks, etc., so as to obtain semantic segmentation results.

[0077] In one implementation, the collected multi-pass image data may be input into an image semantic segmentation model to obtain different semantic information, and the semantic information may be encoded using three-channel RGB to obtain a semantic segmentation result.

[0078] Step S1022: Fuse the point cloud map with the semantic segmentation result to obtain a global semantic point cloud fusion result.

[0079] In this implementation, step S1022 may further include the following steps S10221 and S10222:

[0080] Step S10221: Fuse the single-frame semantic segmentation result and the single-frame point cloud map with the same timestamp to obtain a single-frame semantic point cloud.

[0081] Step S10222: stitching all single-frame semantic point clouds to obtain a global semantic point cloud fusion result.

[0082] In this embodiment, since the image data and the point cloud data are collected at the same timestamp, the single-frame semantic segmentation result and the single-frame point cloud map with the same timestamp can be fused, that is, the point cloud map and the image data can be converted to the same coordinate system according to the external parameters between the camera and the laser radar and the camera projection model, for example, the point cloud map can be projected onto the semantic segmentation result to obtain a single-frame semantic point cloud. And according to the position and posture of the point cloud map, the single-frame semantic point cloud is spliced ​​to obtain the global semantic point cloud fusion result.

[0083] Step S1023: Obtain a semantic feature map according to the global semantic point cloud fusion result.

[0084] In this implementation, step S1023 may further include the following steps S10231 and S10232:

[0085] Step S10231: divide the global semantic point cloud fusion result into grids, and fuse the semantic information in each grid to obtain the global semantic point cloud fusion result after grid semantic fusion.

[0086] In this embodiment, the global semantic point cloud fusion result can be grid-divided. Since the point clouds falling into the same grid may carry different semantic information, the semantic information in each grid can be fused to reduce the influence of noise and obtain the global semantic point cloud fusion result after grid semantic fusion.

[0087] In one implementation, information weights may be constructed based on one or more parameters such as semantic observation distance within a grid, semantic segmentation confidence, etc., and semantic information within the same grid may be fused based on the constructed information weights to obtain a global semantic point cloud fusion result after grid semantic fusion. For example, for multiple semantic information within the same grid, a weight result of each semantic information may be calculated based on the information weight of the semantic information, and semantic information with a lower weight result may be removed, leaving only the semantic information with the highest weight result as the semantic information within the grid.

[0088] Step S10232: Perform BEV perspective rendering according to the global semantic point cloud fusion result after raster semantic fusion to obtain a semantic feature map.

[0089] In this embodiment, since the embodiment of the present application only needs to pay attention to the topology of road elements, using the entire global semantic point cloud fusion result will result in data redundancy in the subsequent processing process. Therefore, in order to reduce the amount of calculation, the BEV (Bird's Eye View) perspective rendering can be performed based on the global semantic point cloud fusion result after raster semantic fusion to obtain a semantic feature map from the BEV perspective, retaining the features within the preset height range of the road surface and above the road surface. Among them, those skilled in the art can set the value of the preset height according to the needs of the actual application.

[0090] In one implementation of the present application, see the attached Figure 7 , Figure 7 FIG. 1 is a schematic diagram of the main steps of step S103 according to an implementation of an embodiment of the present application. Figure 7 As shown, step S103 may further include the following steps S1031 to S1034:

[0091] Step S1031: cropping the semantic feature map according to a first preset size to obtain a plurality of semantic feature thumbnails.

[0092] In this embodiment, since the area to be tested is generally a low-speed scene area, the low-speed scene covers a large area and there are cases where the coverage areas are greatly different, the obtained semantic feature map can be cut according to the first preset size to obtain multiple semantic feature small images, and the road network prediction is first performed based on the feature small images. Among them, those skilled in the art can set the first preset size according to the needs of actual applications.

[0093] In one embodiment, when the semantic feature map is cropped, there may be an overlapping area of ​​a second preset size between adjacent semantic feature sub-maps, so as to facilitate the subsequent splicing of the road network topological relationship of the semantic feature sub-maps to obtain the road network generation result of the area to be tested. Among them, those skilled in the art can set the second preset size according to the needs of actual applications.

[0094] In a specific example, the first preset size may be 150 meters×150 meters, and the second preset size may be 10 meters.

[0095] Step S1032: for each semantic feature thumbnail, extract image depth features of the semantic feature thumbnail to obtain image depth features of the semantic feature thumbnail.

[0096] In this implementation, image depth feature extraction may be performed on each semantic feature thumbnail to obtain image depth features of the semantic feature thumbnail.

[0097] In one implementation, a deep convolutional network (such as a ResNet (Residual Network) or other network) may be applied to extract image depth features from the semantic feature sub-graph to obtain image depth features of the semantic feature sub-graph.

[0098] Step S1033: Obtain a road network prediction result of the semantic feature thumbnail according to the image depth feature; the road network prediction result is the correlation relationship between the road centerline and the intersection points within the semantic feature thumbnail.

[0099] In this implementation, step S1033 may further include the following steps S10331 and S10332:

[0100] Step S10331: According to the image depth feature, obtain the road centerline and intersection points contained in the semantic feature thumbnail.

[0101] In this implementation, the road centerline and intersection points in the semantic feature thumbnail can be obtained based on the image depth features.

[0102] In one implementation, the road centerline and intersection points in the semantic feature sub-image can be extracted based on the image depth features based on the deep neural network of the Transformer model. Specifically, the Transformer model can perform feature encoding on the image depth features, detect road network elements based on the encoding results and the target query mechanism of the Transformer model, and obtain the road centerline and intersection points. Since there are no manually designed anchor frames in the road network element detection process, end-to-end training of the Transformer model can be achieved. Among them, the road centerline can be represented by a 2D point set with a direction, and the intersection can be represented by a single 2D point.

[0103] Step S10332: perform association relationship prediction based on the acquired road center lines and intersection points to obtain the road network prediction result of the semantic feature thumbnail.

[0104] In this embodiment, the association relationship between the acquired road center lines and intersection points can be predicted, so as to obtain the road network prediction result of the semantic feature graph.

[0105] In one implementation, all the tuples of road centerlines and intersections contained in the feature graph can be traversed to determine whether there is an association between different tuples; and when there is an association, which endpoint of the road centerline the intersection is associated with, etc. Specifically, since the Transformer model in the aforementioned steps has already acquired the image depth features (i.e., captured sufficient environmental feature information) and the positional relationship of the road network elements (i.e., correlation), the image depth features can be compressed and encoded based on the image depth features and the positional relationship of the road network elements, and the encoding results can be input into a binary classification network to predict the association relationship between the road centerline and the intersection points, so as to efficiently obtain the road network prediction results of the semantic feature graph.

[0106] In one implementation, the binary classification network may be a binary classification network based on a multilayer perceptron (MLP).

[0107] Please refer to the attached Figure 5 , Figure 5 This is a road network prediction result of a semantic feature graph according to an example of an embodiment of the present application. Figure 5 The figure shows the road centerline prediction results, intersection point prediction results, association relationship prediction results, road edges and road surface signs in the semantic feature sub-image.

[0108] Step S1034: Obtain the road network generation result of the area to be tested based on the road network prediction results of all semantic feature graphs.

[0109] In this implementation, step S1034 may further include the following steps S10341 and S10342:

[0110] Step S10341: Based on the road network prediction results of all semantic feature graphs, the road center lines and intersection points in the road network prediction results are used as nodes, and the association relationships are used as edges to construct the road network topology relationship of the area to be tested.

[0111] In this embodiment, the idea of ​​graph can be used to construct the road network topology relationship between the road network elements in the area to be tested by taking the road center lines and intersection points of the semantic feature graph as nodes and the association relationships as edges.

[0112] Step S10342: Obtain the road network generation result of the area to be tested according to the road network topology relationship.

[0113] In this implementation, step S10342 may further include the following steps S103421 to S103423:

[0114] Step S103421: Obtain the distances between the same type of road network elements in the overlapping area between the semantic feature mini-graphs.

[0115] Step S103422: For the road network elements of the same type whose distance is less than a preset distance threshold, the road network elements are merged.

[0116] Step S103423: Based on the road network topology relationship after the fusion of road network elements, obtain the road network generation result of the area to be tested.

[0117] In this embodiment, when the semantic feature thumbnails are cut, there are overlapping areas between adjacent semantic feature thumbnails. Therefore, the distance between the same type of road network elements in the overlapping area can be obtained, and the road network elements whose distance is less than the preset distance threshold can be fused, and then the road network generation result of the area to be tested is obtained based on the fused road network topological relationship. Among them, those skilled in the art can set the distance threshold according to the needs of the actual application.

[0118] In one implementation, the distance between road network elements may be measured using Euclidean distance.

[0119] In a specific example, the distance threshold may be 3 meters.

[0120] Combine the following Figures 2 to 4 , the road network generation method of the embodiment of the present application is further explained. Figure 2 It is a schematic diagram of the main module composition structure of a road network generation method according to an implementation method of an embodiment of the present application; Figure 3It is a flowchart of the main steps of obtaining a semantic feature map according to an implementation method of an embodiment of the present application; Figure 4 It is a flowchart of the attention steps for obtaining a road network generation result based on a semantic feature map according to an implementation method of an embodiment of the present application.

[0121] like Figure 2 As shown, the road network generation method is mainly implemented by a semantic feature mapping module 21 and a road network generation model 22. Multiple data collection results (data collection result 1, data collection result 2, ..., data collection result N) of the test area can be collected, and the data collection results include Lidar (laser radar) / camera data and IMU / wheel speed meter data. The multiple data collection results are input into the semantic feature mapping module 21 to obtain a semantic feature map; the semantic feature map is input into the road network generation module 22 to obtain a road network generation result.

[0122] like Figure 3 As shown, the function of the semantic feature mapping module can be implemented based on the following steps S201 to S205.

[0123] Step S201: Obtain a point cloud map based on multiple Lidar / IMU / GNSS / wheel speed meter data and determine the position and posture of a single frame point cloud map.

[0124] Step S202: performing semantic segmentation on the image data according to the image data collected by the camera in multiple passes to obtain a semantic segmentation result.

[0125] Step S203: Combine the external parameters of multiple camera passes to fuse the single-frame point cloud map and the single-frame semantic segmentation result to obtain a global semantic point cloud fusion result.

[0126] Step S204: performing raster semantic fusion on the global semantic point cloud fusion result to obtain a global semantic point cloud fusion result after raster semantic fusion.

[0127] Step S205: performing BEV perspective rendering on the global semantic point cloud fusion result after raster semantic fusion to obtain a semantic feature map.

[0128] like Figure 4 As shown, the functions of the road network generation module can be implemented based on the following steps S301 to S307.

[0129] Step S301: Cut based on the semantic feature map to obtain multiple semantic feature thumbnails (feature thumbnails) Figure 1 Small features Figure 2 ,…,feature thumbnail N).

[0130] Step S302: Obtain image depth features of the semantic feature thumbnail through a deep convolutional neural network.

[0131] Step S303: extracting the road centerline based on the image depth feature.

[0132] Step S304: extracting intersection points based on image depth features.

[0133] Step S305: predicting the association relationship between the road centerline and the intersection points.

[0134] Step S306: Obtain the road network prediction result of the semantic feature thumbnail.

[0135] Step S307: Based on the road network prediction results of the semantic feature graph, the road network generation results of the area to be tested are obtained by splicing.

[0136] It should be pointed out that although the various steps in the above embodiments are described in a specific order, those skilled in the art can understand that in order to achieve the effect of the present application, different steps do not have to be executed in such an order, and they can be executed simultaneously (in parallel) or in other orders. These adjusted schemes are equivalent to the technical schemes described in this application, and therefore will also fall within the scope of protection of this application.

[0137] It is understood by those skilled in the art that all or part of the processes in the method for implementing the above-mentioned embodiment of the present application can also be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable storage medium may include: any entity or device, medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory, random access memory, electrical carrier signal, telecommunication signal and software distribution medium, etc. that can carry the computer program code.

[0138] Another aspect of the present application also provides a computer-readable storage medium.

[0139] In an embodiment of a computer-readable storage medium according to the present application, the computer-readable storage medium may be configured to store a program for executing the road network generation method of the above method embodiment, and the program may be loaded and run by a processor to implement the above road network generation method. For ease of explanation, only the parts related to the embodiment of the present application are shown. For specific technical details not disclosed, please refer to the method part of the embodiment of the present application. The computer-readable storage medium may be a storage device formed by various electronic devices. Optionally, the computer-readable storage medium in the embodiment of the present application is a non-temporary computer-readable storage medium.

[0140] Another aspect of the present application also provides a smart device.

[0141] In an embodiment of an intelligent device according to the present application, the intelligent device may include at least one processor; and a memory connected to the at least one processor in communication; wherein a computer program is stored in the memory, and when the computer program is executed by the at least one processor, the method described in any of the above embodiments is implemented. The intelligent device described in the present application may include a driving device, a smart car, a robot, and the like. Figure 8 , Figure 8 FIG. 4 exemplarily shows that the memory 11 and the processor 12 are communicatively connected via a bus.

[0142] In some embodiments of the present application, the smart device may further include at least one sensor for sensing information. The sensor is communicatively connected to any type of processor mentioned in the present application. Optionally, the smart device may further include an autonomous driving system for guiding the smart device to drive itself or assist in driving. The processor communicates with the sensor and / or the autonomous driving system to complete the method described in any of the above embodiments.

[0143] So far, the technical solution of the present application has been described in conjunction with an embodiment shown in the accompanying drawings, but it is easy for those skilled in the art to understand that the protection scope of the present application is obviously not limited to these specific embodiments. Without departing from the principles of the present application, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present application.

Claims

1. A road network generation method, characterized in that: The method comprises: Obtaining a point cloud map of the area to be measured; the point cloud map is generated based on point cloud data obtained by performing multiple rounds of data collection on the area to be measured; the data obtained by the multiple rounds of data collection also includes image data; Performing semantic segmentation on the image data corresponding to the point cloud data to obtain a semantic segmentation result of the image data; The single-frame semantic segmentation result and the single-frame point cloud map with the same timestamp are fused to obtain a single-frame semantic point cloud; All single-frame semantic point clouds are stitched together to obtain the global semantic point cloud fusion result; According to the global semantic point cloud fusion result, a semantic feature map of the area to be measured is obtained; According to the semantic feature map, a road network generation result of the area to be tested is obtained.

2. The road network generation method according to claim 1, characterized in that: The obtaining of the semantic feature map according to the global semantic point cloud fusion result includes: The global semantic point cloud fusion result is divided into grids, and the semantic information in each grid is fused to obtain the global semantic point cloud fusion result after grid semantic fusion; According to the global semantic point cloud fusion result after the grid semantic fusion, BEV perspective rendering is performed to obtain the semantic feature map.

3. The road network generation method according to claim 2, characterized in that: The fusing of semantic information in each grid includes: The semantic information in the grid is fused according to the semantic observation distance and / or the semantic segmentation confidence in the grid.

4. The road network generation method according to claim 1, characterized in that: The step of obtaining a road network generation result of the area to be tested according to the semantic feature map includes: According to a first preset size, the semantic feature map is cropped to obtain a plurality of semantic feature thumbnails; For each semantic feature thumbnail, extract image depth features of the semantic feature thumbnail to obtain image depth features of the semantic feature thumbnail; According to the image depth feature, a road network prediction result of the semantic feature thumbnail is obtained; the road network prediction result is a correlation relationship between a road centerline and an intersection point within the range of the semantic feature thumbnail; According to the road network prediction results of all semantic feature graphs, the road network generation result of the area to be tested is obtained.

5. The road network generation method according to claim 4, characterized in that: The step of obtaining the road network prediction result of the semantic feature thumbnail according to the image depth feature includes: According to the image depth feature, obtaining the road centerline and intersection points contained in the semantic feature thumbnail; According to the obtained road center lines and intersection points, association relationship prediction is performed to obtain the road network prediction result of the semantic feature thumbnail.

6. The road network generation method according to claim 4, characterized in that: The step of obtaining the road network generation result of the area to be tested based on the road network prediction results of all semantic feature graphs includes: According to the road network prediction results of all semantic feature graphs, the road center lines and intersection points in the road network prediction results are used as nodes, and the association relationships are used as edges to construct the road network topology relationship of the area to be tested; According to the road network topology, a road network generation result of the area to be tested is obtained.

7. The road network generation method according to claim 6, characterized in that: There is an overlapping area of ​​a second preset size between adjacent semantic feature thumbnails; The step of obtaining a road network generation result of the area to be tested according to the road network topology relationship includes: Obtaining the distances of the same type of road network elements in the overlapping area; the road network elements include road centerlines and intersections; For the road network elements of the same type whose distance is less than a preset distance threshold, the road network elements are merged; Based on the road network topological relationship after the fusion of road network elements, the road network generation result of the area to be tested is obtained.

8. The road network generation method according to any one of claims 1 to 7, characterized in that: The data obtained by the multiple data collection also includes positioning sensor data; The step of obtaining a point cloud map of the area to be measured includes: A point cloud map of the area to be measured is obtained according to the point cloud data and the positioning sensor data.

9. A smart device, characterized in that: include: at least one processor; and, a memory communicatively coupled to the at least one processor; Wherein, a computer program is stored in the memory, and when the computer program is executed by the at least one processor, the road network generation method according to any one of claims 1 to 8 is implemented.

10. A computer-readable storage medium storing a plurality of program codes, characterized in that: The program code is suitable for being loaded and run by a processor to execute the road network generation method according to any one of claims 1 to 8.

Images