Road boundary generation method and device, computer program product and electronic equipment

Abstract

The invention relates to a road boundary generation method and device, a computer program product and electronic equipment, and relates to the technical field of boundary detection, and the method comprises the steps: determining a target scene, carrying out the data collection of the target scene, obtaining the data of the target scene, and generating a three-dimensional point cloud map of the target scene based on the data of the target scene; and converting the three-dimensional point cloud map into a two-dimensional map, generating a two-dimensional map boundary based on the two-dimensional map, and projecting the two-dimensional map boundary into the three-dimensional point cloud map to obtain the road boundary of the target scene. According to the invention, the efficiency and flexibility of boundary generation are improved.

Description

Background Art

[0002] The rapid development of automation and intelligent technology in recent years has promoted the further improvement of key technologies for autonomous driving. Mines have become one of the scenarios for the implementation of autonomous driving due to their closed roads and relatively simple environment. Key technologies for autonomous driving can be divided into three major technologies: environmental perception, planning and decision-making, and tracking and control. Accurately identifying and constructing boundaries within the mine area is the key to achieving decision-making, planning and control.

[0003] In the related art, the methods for generating boundaries include: image-based road boundary generation methods, semantic segmentation-based road boundary generation methods, and ground segmentation-based road boundary generation methods. Among them, in the image-based road boundary generation method, the image quality is low because the lighting conditions for night operations in the mine design are difficult to meet and there is a lot of dust at the mine operation site; in the semantic segmentation-based method, a large amount of data sets are required, the data annotation is complex, and the hardware equipment computing power requirements are high; in the ground segmentation-based method, the mine scenes are all unstructured roads, and there are indeed irregular changes in the roads.

[0004] Therefore, a new road boundary generation method needs to be provided.

[0005] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0006] The purpose of the present disclosure is to provide a road boundary generation method, a road boundary generation device, a computer program product and an electronic device, thereby overcoming the problem of low road boundary generation efficiency caused by the limitations and defects of related technologies at least to a certain extent.

[0007] According to one aspect of the present disclosure, a road boundary generation method is provided, comprising:

[0008] Determine a target scene, collect data on the target scene to obtain target scene data, and generate a three-dimensional point cloud map of the target scene based on the target scene data;

[0009] The three-dimensional point cloud map is converted into a two-dimensional map, a two-dimensional map boundary is generated based on the two-dimensional map, and the two-dimensional map boundary is projected into the three-dimensional point cloud map to obtain a road boundary of the target scene.

[0010] In an exemplary embodiment of the present disclosure, collecting data on the target scene to obtain target scene data includes:

[0011] Determine a collection starting point in the target scene, and obtain lidar data, inertial navigation data, and real-time dynamic positioning data in a path starting from the collection starting point and ending at the collection starting point;

[0012] The laser radar data, the inertial navigation data and the real-time dynamic positioning data are time synchronized and external parameter calibrated to obtain the target scene data.

[0013] In an exemplary embodiment of the present disclosure, generating a three-dimensional point cloud map of the target scene based on the target scene data includes:

[0014] Acquire an IMU pre-integration factor, an odometer factor, and a GPS factor, and jointly optimize the target scene data according to the IMU pre-integration factor, the odometer factor, and the GPS factor to obtain a state quantity;

[0015] Based on the state quantity, the three-dimensional point cloud map is generated.

[0016] In an exemplary embodiment of the present disclosure, converting the three-dimensional point cloud map into a two-dimensional map includes:

[0017] Acquire the number of point clouds included in the three-dimensional point cloud map, and determine the pixel resolution according to the number of point clouds;

[0018] According to the point cloud included in the three-dimensional point cloud map and the pixel resolution, the size of the two-dimensional map and the first coordinates of the point cloud in the three-dimensional point cloud map are obtained, and the second coordinates of the point cloud in the two-dimensional map are obtained;

[0019] The two-dimensional map is generated according to the size of the two-dimensional map and the second coordinates.

[0020] In an exemplary embodiment of the present disclosure, after converting the three-dimensional point cloud map into a two-dimensional map, the method further includes:

[0021] Generate a projection relationship of the point cloud according to a first coordinate of the point cloud in the three-dimensional point cloud map and a second coordinate of the point cloud in the two-dimensional map;

[0022] A projection matrix is ​​generated according to the projection relationship.

[0023] In an exemplary embodiment of the present disclosure, projecting the two-dimensional map boundary into the three-dimensional point cloud map to obtain the road boundary of the target scene includes:

[0024] The point cloud in the two-dimensional map boundary is projected into the three-dimensional point cloud map according to the inverse matrix of the projection matrix to obtain the road boundary of the target scene.

[0025] In an exemplary embodiment of the present disclosure, after obtaining the road boundary of the target scene, the method further includes:

[0026] Acquire laser radar data collected during the data collection process, store the laser radar data in a first grid, and store the three-dimensional point cloud map of the target scene in a second grid;

[0027] Comparing the first grid and the second grid, determining first data newly added in the first grid and a first point cloud corresponding to the first data in the second grid;

[0028] The first point cloud is deleted from the three-dimensional point cloud map, and a second point cloud corresponding to the first data is added to the three-dimensional point cloud map, and the road boundary is updated according to the second point cloud.

[0029] According to one aspect of the present disclosure, there is provided a road boundary generating device, comprising:

[0030] A point cloud map generation module is used to determine a target scene, collect data on the target scene, obtain target scene data, and generate a three-dimensional point cloud map of the target scene based on the target scene data;

[0031] The road boundary generation module is used to convert the three-dimensional point cloud map into a two-dimensional map, generate a two-dimensional map boundary based on the two-dimensional map, and project the two-dimensional map boundary into the three-dimensional point cloud map to obtain the road boundary of the target scene.

[0032] According to one aspect of the present disclosure, a computer program product is provided, including a computer program, wherein when the computer program is executed by a processor, the road boundary generating method described in any one of the above exemplary embodiments is implemented.

[0033] According to one aspect of the present disclosure, there is provided an electronic device, including:

[0034] A processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the road boundary generation method described in any of the above exemplary embodiments by executing the executable instructions.

[0035] The disclosed embodiment provides a road boundary generation method, which determines a target scene, collects data for the target scene, obtains target scene data, generates a three-dimensional point cloud map of the target scene based on the target scene data; converts the three-dimensional point cloud map into a two-dimensional map, generates a two-dimensional map boundary based on the two-dimensional map, and projects the two-dimensional map boundary into the three-dimensional point cloud map to obtain the road boundary of the target scene. On the one hand, after determining the target scene, data is collected for the target scene to obtain target scene data, a three-dimensional point cloud map is generated according to the target scene data, the three-dimensional point cloud data is converted into a two-dimensional map, the image boundary is determined in the two-dimensional map, and the determined boundary is projected into the three-dimensional point cloud map, without taking a photo of the target scene, solving the problem that the image-based road boundary generation method in the related art affects boundary generation due to image quality, and improving the efficiency of boundary generation; on the other hand, after obtaining the three-dimensional point cloud map, only the three-dimensional point cloud map needs to be converted, and a large amount of data sets do not need to be collected, and the data does not need to be annotated, which simplifies the process of road boundary generation; on the other hand, compared with the method based on ground segmentation, the road boundary generation method of the present application does not require the existence of structured roads in the target scene, which improves the flexibility of road boundary generation.

[0036] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0038] Figure 1 A flowchart of a road boundary generation method according to an exemplary embodiment of the present disclosure is schematically shown.

[0039] Figure 2 A flow chart of a method for collecting data on the target scene to obtain target scene data according to an exemplary embodiment of the present disclosure is schematically shown.

[0040] Figure 3 A flowchart of a method for generating a three-dimensional point cloud map of a target scene based on the target scene data according to an example embodiment of the present disclosure is schematically shown.

[0041] Figure 4 A schematic diagram of a three-dimensional point cloud map according to an example embodiment of the present disclosure is schematically shown.

[0042] Figure 5 A flowchart of a method for converting the three-dimensional point cloud map into a two-dimensional map according to an example embodiment of the present disclosure is schematically shown.

[0043] Figure 6 A schematic diagram schematically illustrates a two-dimensional map according to an example embodiment of the present disclosure.

[0044] Figure 7 A flowchart of a method for generating a road boundary after converting the three-dimensional point cloud map into a two-dimensional map according to an exemplary embodiment of the present disclosure is schematically shown.

[0045] Figure 8 A schematic diagram schematically illustrates a two-dimensional map boundary according to an example embodiment of the present disclosure.

[0046] Fig. 9 A schematic diagram schematically illustrates a road boundary of a target scene according to an example embodiment of the present disclosure.

[0047] Fig.10 A flowchart of a method for generating a road boundary after obtaining the road boundary of the target scene according to an exemplary embodiment of the present disclosure is schematically shown.

[0048] Fig.11 A block diagram schematically shows a road boundary generating device according to an exemplary embodiment of the present disclosure.

[0049] Fig.12 An electronic device for implementing the road boundary generating method according to an exemplary embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0050] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as being limited to the examples set forth herein; on the contrary, these embodiments are provided so that the present disclosure will be more comprehensive and complete, and the concepts of the example embodiments are fully conveyed to those skilled in the art. The described features, structures, or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced while omitting one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known technical solutions are not shown or described in detail to avoid obscuring various aspects of the present disclosure.

[0051] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0052] The rapid development of automation and intelligent technology in recent years has promoted the further improvement of key technologies for autonomous driving. Mines have become one of the scenarios for the implementation of autonomous driving due to their closed roads and relatively simple environment. Key technologies for autonomous driving can be divided into three technologies: environmental perception, planning and decision-making, and tracking and control. Accurately identifying and constructing boundaries within the mine area is the key to achieving decision-making, planning and control within the mine.

[0053] In the related art, methods for generating road boundaries include: an image-based road boundary generation method, a semantic segmentation-based road boundary generation method, and a ground segmentation-based road boundary generation method.

[0054] Among them, the image-based road boundary generation method is suitable for scenes with good lighting conditions and high image data quality. However, the lighting conditions for night operations in mines are difficult to meet and there is a lot of dust at the mine operation site, which affects the image data quality.

[0055] Based on the semantic segmentation method, the point cloud data is segmented through a trained semantic segmentation network to obtain the road boundary. However, this method requires a large amount of data set, complex data labeling and high computing power requirements for hardware equipment.

[0056] Based on the ground segmentation method, the boundary of the ground point cloud obtained by ground segmentation is fitted to obtain the road boundary curve. This method has a good effect on structured roads, but the mining scenes are all unstructured roads and there are irregular changes in the roads.

[0057] Based on one or more of the above problems, this exemplary embodiment first provides a road boundary generation method, which can be run on a terminal device; of course, those skilled in the art can also run the method of the present invention on other platforms as required, and this exemplary embodiment does not specifically limit this. Figure 1 As shown, the road boundary generation method may include step S110 and step S120:

[0058] Step S110: determining a target scene, collecting data on the target scene to obtain target scene data, and generating a three-dimensional point cloud map of the target scene based on the target scene data;

[0059] Step S120. Convert the three-dimensional point cloud map into a two-dimensional map, generate a two-dimensional map boundary based on the two-dimensional map, project the two-dimensional map boundary into the three-dimensional point cloud map, and obtain the road boundary of the target scene.

[0060] The above road boundary generation method determines the target scene, collects data for the target scene, obtains the target scene data, generates a three-dimensional point cloud map of the target scene based on the target scene data; converts the three-dimensional point cloud map into a two-dimensional map, generates a two-dimensional map boundary based on the two-dimensional map, and projects the two-dimensional map boundary into the three-dimensional point cloud map to obtain the road boundary of the target scene. On the one hand, after determining the target scene, data is collected for the target scene to obtain the target scene data, a three-dimensional point cloud map is generated according to the target scene data, the three-dimensional point cloud data is converted into a two-dimensional map, the image boundary is determined in the two-dimensional map, and the determined boundary is projected into the three-dimensional point cloud map, and there is no need to take a photo of the target scene, which solves the problem that the image-based road boundary generation method in the related art affects the boundary generation due to the image quality, and improves the efficiency of boundary generation; on the other hand, after obtaining the three-dimensional point cloud map, it is only necessary to convert the three-dimensional point cloud map, and there is no need to collect a large number of data sets, and there is no need to annotate the data, which simplifies the process of road boundary generation; on the other hand, compared with the method based on ground segmentation, the road boundary generation method of the present application does not require the existence of structured roads in the target scene, which improves the flexibility of road boundary generation.

[0061] Hereinafter, each step involved in the road boundary generation method according to the exemplary embodiment of the present disclosure is explained and illustrated in detail.

[0062] First, the application scenario and purpose of the exemplary embodiment of the present disclosure are explained and illustrated. Specifically, the exemplary embodiment of the present disclosure can be applied to collect data in a scene with a closed road and a simple environment, and then generate a road boundary of the scene.

[0063] In the present disclosure, based on the determined target scene, after the target scene is determined, data collection is performed on the target scene to obtain target scene data. After obtaining the target scene data, a three-dimensional point cloud map is generated according to the target scene data, the three-dimensional point cloud map is converted into a two-dimensional map, an image boundary is generated according to the point cloud included in the two-dimensional map, and after the image boundary is generated, the image boundary is projected into the three-dimensional point cloud map to obtain the road boundary of the target scene.

[0064] Next, step S110 and step S120 are explained and illustrated in detail.

[0065] In step S110, a target scene is determined, data is collected for the target scene to obtain target scene data, and a three-dimensional point cloud map of the target scene is generated based on the target scene data.

[0066] In this example embodiment, the target scene may be a scene with a closed road and a relatively simple environment, specifically, a mine, etc., and the target scene is not specifically limited in this example embodiment. After determining the target scene, data can be collected for the target scene, and the data can be collected by a car, which can be an unmanned car or an ordinary car, and this is not specifically limited in this example embodiment. After the target scene data is collected, a three-dimensional point cloud map can be generated based on the target scene data.

[0067] In this example embodiment, reference Figure 2 As shown, the data collection of the target scene to obtain the target scene data includes:

[0068] Step S210. Determine a collection starting point in the target scene, and obtain laser radar data, inertial navigation data, and real-time dynamic positioning data in a path starting from the collection starting point and ending at the collection starting point;

[0069] Step S220. Time synchronization and external parameter calibration are performed on the laser radar data, the inertial navigation data and the real-time dynamic positioning data to obtain the target scene data.

[0070] In the following, step S210 and step S220 will be further explained and illustrated. Specifically, first determine the collection starting point when the car collects data in the target scene. After determining the collection starting point, the car starts from the collection starting point, collects the data of the target scene at one time, and returns to the collection starting point. After the car returns to the collection starting point, the data collected by the car is obtained. The collected data may include lidar data, inertial navigation data, and RTK (Real-Time Kinematic, real-time dynamic positioning technology) data. After obtaining the lidar data, inertial navigation data, and RTK data, the lidar data, inertial navigation data, and RTK data can be time synchronized, and the data that has completed time synchronization can be calibrated with external parameters to obtain the target scene data. In addition, the speed of the car should be lower than 15km / h during the data collection process.

[0071] After obtaining the target scene data, a three-dimensional point cloud map can be generated based on the target scene data. Figure 3 As shown, generating a three-dimensional point cloud map of the target scene based on the target scene data includes:

[0072] Step S310. Obtain an IMU pre-integration factor, an odometer factor, and a GPS factor, and jointly optimize the target scene data according to the IMU pre-integration factor, the odometer factor, and the GPS factor to obtain a state quantity;

[0073] Step S320. Generate the three-dimensional point cloud map based on the state quantity.

[0074] Below, step S310 and step S320 will be further explained and illustrated. Specifically, in the mining scene, the scene is relatively empty and has few laser feature points. It is basically a degraded scene. If only lidar data is used to build a map, the lack of constraints in certain state directions will lead to errors in state quantity solution, which will in turn lead to map construction errors. Therefore, multi-data fusion mapping is performed in this application. First, the IMU (Inertial Measurement Unit) pre-integration factor, odometer factor and GPS factor are obtained; secondly, the acquired target scene data is input into the neural network, and the IMU pre-integration factor, odometer factor and GPS factor are introduced into the neural network. The target scene data is jointly optimized to obtain the state quantity, wherein the state quantity includes position, posture, etc. In this example embodiment, the state quantity is not specifically limited. After obtaining the state quantity, a three-dimensional point cloud map can be constructed based on the state quantity. Figure 4 A three-dimensional point cloud map is shown.

[0075] In step S120, the three-dimensional point cloud map is converted into a two-dimensional map, a two-dimensional map boundary is generated based on the two-dimensional map, and the two-dimensional map boundary is projected into the three-dimensional point cloud map to obtain the road boundary of the target scene.

[0076] After obtaining the three-dimensional point cloud map, the three-dimensional point cloud map can be converted into a two-dimensional map. After obtaining the two-dimensional map, a two-dimensional map boundary can be interactively generated in the two-dimensional map. After obtaining the two-dimensional map boundary, the two-dimensional map boundary can be back-projected into the three-dimensional point cloud map to generate a road boundary in the three-dimensional point cloud map.

[0077] In this example embodiment, reference Figure 5 As shown, converting the three-dimensional point cloud map into a two-dimensional map includes:

[0078] Step S510. Obtain the number of point clouds included in the three-dimensional point cloud map, and determine the pixel resolution according to the number of point clouds;

[0079] Step S520. According to the point cloud included in the three-dimensional point cloud map and the pixel resolution, obtain the size of the two-dimensional map and the first coordinates of the point cloud in the three-dimensional point cloud map, and obtain the second coordinates of the point cloud in the two-dimensional map;

[0080] Step S530: Generate the two-dimensional map according to the size of the two-dimensional map and the second coordinates.

[0081] Below, step S510-step S530 will be further explained and illustrated. Specifically, the point cloud and the number of point clouds included in the three-dimensional point cloud map are obtained, and the pixel resolution r is determined according to the number of point clouds. After the pixel resolution r is obtained, the size of the two-dimensional map is determined according to the pixel resolution r and the number of point clouds. If the maximum and minimum x-axis coordinates of the point cloud in the three-dimensional point cloud map are maxX and minX, respectively, and the maximum and minimum y-axis coordinates are maxY and minY, respectively, then the width of the two-dimensional map is width = (maxX-minX) / r, and the height is height = (maxY-minY) / r. After obtaining the size of the two-dimensional map, the second coordinate of each point cloud in the three-dimensional point cloud map can be obtained according to the first coordinate and pixel resolution of each point cloud in the three-dimensional point cloud map. When the three-dimensional coordinates of the point cloud are x, y, and z, the pixel coordinates (m, n) in the two-dimensional map are: m = (maxX-x) / r, m = (maxY-y) / r. After obtaining the size of the two-dimensional map and the second coordinate of each point cloud in the three-dimensional point cloud map in the two-dimensional map, the two-dimensional map can be generated according to the size of the two-dimensional map and the second coordinate. Figure 4 The two-dimensional map generated by the three-dimensional point cloud map shown in Figure 6 shown.

[0082] In the process of converting a 3D point cloud map to a 2D map, the projection matrix can also be determined based on the projection relationship in the conversion process. Figure 7 As shown, after converting the three-dimensional point cloud map into a two-dimensional map, the method further includes:

[0083] Step S710. Generate a projection relationship of the point cloud according to the first coordinates of the point cloud in the three-dimensional point cloud map and the second coordinates of the point cloud in the two-dimensional map;

[0084] Step S720: Generate a projection matrix according to the projection relationship.

[0085] In the following, step S710 and step S720 are further explained and illustrated. Specifically, according to the first coordinate of each point cloud in the three-dimensional point cloud map and the second coordinate of each point cloud in the two-dimensional map, the projection relationship of the point cloud is generated, and a projection matrix M with a dimension of 2*3 is generated according to the projection relationship.

[0086] After the two-dimensional map is generated, the two-dimensional map boundary can be interactively generated in the two-dimensional map, wherein the interactive generation is to draw the two-dimensional map boundary in the two-dimensional map according to the second coordinate of the point cloud through image editing software, according to Figure 6 The two-dimensional map shown depicts the two-dimensional map boundary reference Figure 8 shown.

[0087] In this example embodiment, after obtaining the two-dimensional map boundary, the two-dimensional map boundary can be projected again into the three-dimensional point cloud map according to the projection matrix to generate the road boundary of the target scene. The projecting of the two-dimensional map boundary into the three-dimensional point cloud map to obtain the road boundary of the target scene includes:

[0088] The point cloud in the two-dimensional map boundary is projected into the three-dimensional point cloud map according to the inverse matrix of the projection matrix to obtain the road boundary of the target scene.

[0089] Specifically, the inverse matrix N of the projection matrix M is obtained, and the point cloud coordinates corresponding to the boundary of the two-dimensional map are back-projected to the three-dimensional point cloud map according to the inverse matrix N. If the second coordinate p(px,py) of the point cloud in the two-dimensional map is back-projected into a three-dimensional point P(x,y,z), P=N*p. Figure 8 The 2D map boundary shown is back-projected to Figure 4 The three-dimensional point cloud map shown in the figure shows the road boundary of the target scene. Fig. 9 shown.

[0090] Furthermore, since the operating area of ​​the mine scene changes rapidly, the boundary of the changing area of ​​the mine scene can be updated based on the global boundary using the local point cloud road boundary after the vehicle operation. Fig.10 As shown, after obtaining the road boundary of the target scene, the method further includes:

[0091] Step S1010. Acquire laser radar data collected during the data collection process, store the laser radar data in the first grid, and store the three-dimensional point cloud map of the target scene in the second grid;

[0092] Step S1020. Compare the first grid and the second grid to determine the newly added first data in the first grid and the first point cloud corresponding to the first data in the second grid;

[0093] Step S1030. Delete the first point cloud in the three-dimensional point cloud map, add a second point cloud corresponding to the first data to the three-dimensional point cloud map, and update the road boundary according to the second point cloud.

[0094] In the following, step S1010 to step S1030 will be further explained and illustrated. Specifically, when acquiring the laser radar data collected during the data acquisition process, the laser radar data can be stored in the first three-dimensional grid, and the three-dimensional point cloud map of the target scene can be stored in the second three-dimensional grid. The first three-dimensional grid and the second three-dimensional grid are matched for occupancy, that is, the number of point clouds included in each grid in the first three-dimensional grid and the second three-dimensional grid is compared to determine the first data newly added in the laser radar data, wherein the three-dimensional point cloud map does not include the point cloud corresponding to the first data. After the first data is acquired, the second point cloud corresponding to the first data can be added to the three-dimensional point cloud map, and at the same time, the first point cloud in the three-dimensional point cloud map can also be deleted. The first point cloud can be determined based on the point cloud included in the three-dimensional grid. For example, in the three-dimensional grid, when the number of point clouds included in grid 1 of the first three-dimensional grid and grid 1 of the second three-dimensional grid is different, the point cloud included in grid 1 of the second three-dimensional grid is determined as the first point cloud. After deleting the first point cloud, the map boundary can be updated based on the newly added second point cloud. When updating the boundary, the ground of the three-dimensional point cloud map can be segmented, and the boundary of the ground can be fitted to obtain the boundary of the changed area, and the boundary of the changed area can be updated to the road boundary of the target scene.

[0095] The road boundary generation method provided by the example embodiment of the present disclosure has at least the following advantages: on the one hand, after determining the target scene, data collection is performed on the target scene to obtain target scene data, a three-dimensional point cloud map is generated according to the target scene data, the three-dimensional point cloud data is converted into a two-dimensional map, the image boundary is determined in the two-dimensional map, and the determined boundary is projected into the three-dimensional point cloud map, without taking pictures of the target scene, thereby solving the problem of image-based road boundary generation methods in related technologies that boundary generation is affected by image quality, and improving the efficiency of boundary generation; on the other hand, after obtaining the three-dimensional point cloud map, only the three-dimensional point cloud map needs to be converted, without the need to collect a large amount of data sets, and without the need to annotate the data, thereby simplifying the process of road boundary generation; on the other hand, compared with the method based on ground segmentation, the road boundary generation method of the present application does not require the existence of structured roads in the target scene, thereby improving the flexibility of road boundary generation.

[0096] The exemplary embodiment of the present disclosure also provides a road boundary generating device, referring to Fig.11 As shown, it may include: a point cloud map generation module 1110 and a road boundary generation module 1120. Among them:

[0097] The point cloud map generation module 1110 is used to determine a target scene, collect data on the target scene, obtain target scene data, and generate a three-dimensional point cloud map of the target scene based on the target scene data;

[0098] The road boundary generation module 1120 is used to convert the three-dimensional point cloud map into a two-dimensional map, generate a two-dimensional map boundary based on the two-dimensional map, and project the two-dimensional map boundary into the three-dimensional point cloud map to obtain the road boundary of the target scene.

[0099] The specific details of each module in the above-mentioned road boundary generation device have been described in detail in the corresponding road boundary generation method, so they will not be repeated here.

[0100] In an exemplary embodiment of the present disclosure, the point cloud map generation module includes:

[0101] A data acquisition module is used to determine a collection starting point in the target scene, and obtain laser radar data, inertial navigation data, and RTK data in a path starting from the collection starting point and ending at the collection starting point;

[0102] The target scene data acquisition module is used to perform time synchronization and external parameter calibration on the laser radar data, the inertial navigation data and the RTK data to obtain the target scene data.

[0103] In an exemplary embodiment of the present disclosure, the point cloud map generation module includes:

[0104] A data optimization module, used to obtain an IMU pre-integration factor, an odometer factor, and a GPS factor, and jointly optimize the target scene data according to the IMU pre-integration factor, the odometer factor, and the GPS factor to obtain a state quantity;

[0105] A three-dimensional map generation module is used to generate the three-dimensional point cloud map based on the state quantity.

[0106] In an exemplary embodiment of the present disclosure, the road boundary generation module includes:

[0107] A resolution determination module, used to obtain the number of point clouds included in the three-dimensional point cloud map, and determine the pixel resolution according to the number of point clouds;

[0108] A point cloud mapping module, used to obtain the size of the two-dimensional map and the first coordinates of the point cloud in the three-dimensional point cloud map and the second coordinates of the point cloud in the two-dimensional map according to the point cloud included in the three-dimensional point cloud map and the pixel resolution;

[0109] A map conversion module is used to generate the two-dimensional map according to the size of the two-dimensional map and the second coordinates.

[0110] In an exemplary embodiment of the present disclosure, the road boundary generation module further includes:

[0111] A projection relationship determination module, used to generate a projection relationship of the point cloud according to a first coordinate of the point cloud in the three-dimensional point cloud map and a second coordinate of the point cloud in the two-dimensional map;

[0112] The projection matrix determination module is used to generate a projection matrix according to the projection relationship.

[0113] In an exemplary embodiment of the present disclosure, the road boundary generation module includes:

[0114] The boundary generation module is used to project the point cloud in the boundary of the two-dimensional map into the three-dimensional point cloud map according to the inverse matrix of the projection matrix to obtain the road boundary of the target scene.

[0115] In an exemplary embodiment of the present disclosure, the road boundary generation module further includes:

[0116] A data storage module, used to obtain the laser radar data collected during the data collection process, store the laser radar data in the first grid, and store the three-dimensional point cloud map of the target scene in the second grid;

[0117] A data comparison module, used to compare the first grid and the second grid to determine the first data newly added in the first grid and the first point cloud corresponding to the first data in the second grid;

[0118] A road boundary updating module is used to delete the first point cloud in the three-dimensional point cloud map, add a second point cloud corresponding to the first data to the three-dimensional point cloud map, and update the road boundary according to the second point cloud.

[0119] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0120] In addition, although the steps of the method in the present disclosure are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.

[0121] In an exemplary embodiment of the present disclosure, an electronic device capable of implementing the above method is also provided.

[0122] Those skilled in the art will appreciate that various aspects of the present disclosure may be implemented as systems, methods or program products. Therefore, various aspects of the present disclosure may be specifically implemented in the following forms, namely: complete hardware implementation, complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to herein as "circuits", "modules" or "systems".

[0123] Refer to the following Fig.12 1200 according to this embodiment of the present disclosure is described. Fig.12 The electronic device 1200 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0124] like Fig.12 As shown, the electronic device 1200 is in the form of a general computing device. The components of the electronic device 1200 may include, but are not limited to: the at least one processing unit 1210, the at least one storage unit 1220, a bus 1230 connecting different system components (including the storage unit 1220 and the processing unit 1210), and a display unit 1240.

[0125] The storage unit stores program codes, which can be executed by the processing unit 1210, so that the processing unit 1210 performs the steps according to various exemplary embodiments of the present disclosure described in the above “Exemplary Method” section of this specification. For example, the processing unit 1210 can perform the following steps: Figure 1 The step S110 shown in the figure is as follows: determining a target scene, collecting data for the target scene to obtain target scene data, and generating a three-dimensional point cloud map of the target scene based on the target scene data; S120: converting the three-dimensional point cloud map into a two-dimensional map, generating a two-dimensional map boundary based on the two-dimensional map, and projecting the two-dimensional map boundary into the three-dimensional point cloud map to obtain a road boundary of the target scene.

[0126] The storage unit 1220 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 12201 and / or a cache storage unit 12202 , and may further include a read-only storage unit (ROM) 12203 .

[0127] The storage unit 1220 may also include a program / utility 12204 having a set (at least one) of program modules 12205, such program modules 12205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0128] Bus 1230 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0129] The electronic device 1200 may also communicate with one or more external devices 1300 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 1200, and / or communicate with any device that enables the electronic device 1200 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 1250. In addition, the electronic device 1200 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 1260. As shown, the network adapter 1260 communicates with other modules of the electronic device 1200 via a bus 1230. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 1200, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0130] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0131] In an exemplary embodiment of the present disclosure, a computer-readable storage medium is also provided, on which a program product capable of implementing the above method of the present specification is stored. In some possible implementations, various aspects of the present disclosure may also be implemented in the form of a program product, which includes a program code, and when the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps according to various exemplary implementations of the present disclosure described in the above "Exemplary Method" section of the present specification.

[0132] According to the program product for implementing the above method in the embodiment of the present disclosure, it can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present disclosure is not limited thereto. In this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, an apparatus or a device.

[0133] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0134] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, in which readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0135] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0136] Program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0137] In addition, the above-mentioned figures are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present disclosure, and are not intended to be limiting. It is easy to understand that the processes shown in the above-mentioned figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0138] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The specification and examples are to be considered as exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

Claims

1. A road boundary generation method, characterized in that: include: Determine a target scene, collect data on the target scene to obtain target scene data, and generate a three-dimensional point cloud map of the target scene based on the target scene data; The three-dimensional point cloud map is converted into a two-dimensional map, a two-dimensional map boundary is generated based on the two-dimensional map, and the two-dimensional map boundary is projected into the three-dimensional point cloud map to obtain a road boundary of the target scene.

2. The method according to claim 1, characterized in that The step of collecting data on the target scene to obtain target scene data includes: Determine a collection starting point in the target scene, and obtain lidar data, inertial navigation data, and real-time dynamic positioning data in a path starting from the collection starting point and ending at the collection starting point; The laser radar data, the inertial navigation data and the real-time dynamic positioning data are time synchronized and external parameter calibrated to obtain the target scene data.

3. The method according to claim 1, characterized in that The generating a three-dimensional point cloud map of the target scene based on the target scene data comprises: Obtaining an IMU pre-integration factor, an odometer factor, and a GPS factor, and jointly optimizing the target scene data according to the IMU pre-integration factor, the odometer factor, and the GPS factor to obtain a state quantity; Based on the state quantity, the three-dimensional point cloud map is generated.

4. The method according to claim 1, characterized in that The converting the three-dimensional point cloud map into a two-dimensional map comprises: Acquire the number of point clouds included in the three-dimensional point cloud map, and determine the pixel resolution according to the number of point clouds; According to the point cloud included in the three-dimensional point cloud map and the pixel resolution, the size of the two-dimensional map and the first coordinates of the point cloud in the three-dimensional point cloud map are obtained, and the second coordinates of the point cloud in the two-dimensional map are obtained; The two-dimensional map is generated according to the size of the two-dimensional map and the second coordinates.

5. The method according to claim 4, characterized in that After converting the three-dimensional point cloud map into a two-dimensional map, the method further includes: Generate a projection relationship of the point cloud according to a first coordinate of the point cloud in the three-dimensional point cloud map and a second coordinate of the point cloud in the two-dimensional map; A projection matrix is ​​generated according to the projection relationship.

6. The method according to claim 5, characterized in that The projecting the two-dimensional map boundary into the three-dimensional point cloud map to obtain the road boundary of the target scene includes: The point cloud in the two-dimensional map boundary is projected into the three-dimensional point cloud map according to the inverse matrix of the projection matrix to obtain the road boundary of the target scene.

7. The method according to claim 1, characterized in that After obtaining the road boundary of the target scene, the method further includes: Acquire laser radar data collected during the data collection process, store the laser radar data in a first grid, and store the three-dimensional point cloud map of the target scene in a second grid; Comparing the first grid and the second grid, determining first data newly added in the first grid and a first point cloud corresponding to the first data in the second grid; The first point cloud is deleted from the three-dimensional point cloud map, and a second point cloud corresponding to the first data is added to the three-dimensional point cloud map, and the road boundary is updated according to the second point cloud.

8. A road boundary generating device, characterized in that: include: A point cloud map generation module is used to determine a target scene, collect data on the target scene, obtain target scene data, and generate a three-dimensional point cloud map of the target scene based on the target scene data; The road boundary generation module is used to convert the three-dimensional point cloud map into a two-dimensional map, generate a two-dimensional map boundary based on the two-dimensional map, and project the two-dimensional map boundary into the three-dimensional point cloud map to obtain the road boundary of the target scene.

9. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. An electronic device, characterized in that: include: Processor; and A memory for storing executable instructions of the processor; wherein the processor is configured to perform the method of any one of claims 1-7 by executing the executable instructions.

Images