Map generation method, device, medium, program product and unmanned vehicle

Abstract

The present invention discloses a map generation method, device, medium, program product and unmanned vehicle, which are applied to the field of Internet of Things technology. The method includes obtaining time-synchronized navigation map information and multi-source sensor time series data; generating a driving area map according to point cloud data and driving area image features in the multi-source sensor time series data; adjusting the road topology information of the driving area map according to the road topology data of the navigation map information, and adding the navigation prompt information of the navigation map information as semantic information to the adjusted driving area map to generate a vehicle driving map. The present invention can solve the problem of poor safety of NOA without map, and effectively ensure the safe completion of the autonomous driving task.

Description

Technical Field

[0001] The present invention relates to the technical field of Internet of Things, and in particular to a map generation method, electronic equipment, non-volatile storage medium, computer program product and unmanned vehicle. Background Art

[0002] With the rapid development of artificial intelligence technology and Internet of Things technology, intelligent driving technology has also developed accordingly. The autonomous driving map used to plan the movement map of the autonomous driving vehicle affects the intelligent driving vehicle to complete the driving task efficiently and safely.

[0003] In related technologies, smart driving vehicles achieve smart driving based on mapless NOA (Navigate on Autopilot). Although mapless NOA does not rely on high-precision maps, it is necessary to build local scene maps online for obstacle avoidance and path planning. However, since the local scene maps of related technologies are based on semantic segmentation and instance segmentation, this will lead to problems such as discontinuous lane lines, missed detection, and false detection, and it is impossible to ensure the safe completion of autonomous driving tasks.

[0004] In view of this, improving the safety of map-free NOA and safely completing autonomous driving tasks are issues that technical personnel in this field need to solve.

[0005] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present application, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0006] The present invention provides a map generation method, an electronic device, a non-volatile storage medium, a computer program product and an unmanned vehicle, which effectively improve the safety of map-free NOA and ensure the safe completion of autonomous driving tasks.

[0007] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0008] In one aspect, the present invention provides a map generation method, comprising:

[0009] Get time-synchronized navigation map information and multi-source sensor timing data.

[0010] A driving area map is generated according to the point cloud data and driving area image features in the multi-source sensor time series data.

[0011] The road topology information of the driving area map is adjusted accordingly according to the road topology data of the navigation map information, and the navigation prompt information of the navigation map information is added as semantic information to the adjusted driving area map to generate a vehicle driving map.

[0012] In a first exemplary implementation, the acquiring time-synchronized navigation map information includes:

[0013] Get the global navigation map of the current frame.

[0014] Based on the fixed position of the target navigation prompt information in the global navigation map, the target navigation prompt information is obtained from the corresponding position of the current frame global navigation map.

[0015] The target vehicle is located in the current frame global navigation map, and based on the position information of the target vehicle in the current frame global navigation map, the road congestion status information in the target area ahead of the target vehicle is obtained.

[0016] In a second exemplary embodiment, the acquiring the target navigation prompt information from a corresponding position of the global navigation map of the current frame based on the fixed position of the target navigation prompt information in the global navigation map includes:

[0017] Based on the fixed position of the target navigation prompt information in the global navigation map, the current frame global navigation map is segmented by a preset segmentation frame to obtain a target area block.

[0018] A pre-built prompt information recognition network model is called to process the target area block to obtain target navigation prompt information.

[0019] In a third exemplary implementation, the acquiring time-synchronized navigation map information includes:

[0020] Get the local navigation map of the current frame.

[0021] A target vehicle is located in the local navigation map of the current frame, and lane information is determined based on position information of the target vehicle in the local navigation map of the current frame.

[0022] Road topology data is generated according to the lane area, double yellow line and lane line in the lane information.

[0023] In a fourth exemplary embodiment, determining lane information based on position information of the target vehicle in the local navigation map of the current frame includes:

[0024] Based on the position information of the target vehicle in the local navigation map of the current frame, a target topology area for extracting the road topology data is determined.

[0025] The local navigation map of the current frame is layered according to color, and interference information of colors that do not belong to lane information is removed.

[0026] The target topological area with interference information removed is subjected to color segmentation processing to obtain the lane area, double yellow line and lane line as lane information.

[0027] In a fifth exemplary implementation, obtaining time-synchronized navigation map information and multi-source sensor time series data includes:

[0028] Acquire 3D point cloud data, vehicle motion and posture data, and driving scene image data as multi-source sensor time series data;

[0029] Get the local navigation map and global navigation map with timestamp.

[0030] Based on the timestamp, time alignment processing is performed on the local navigation map, the global navigation map and the multi-source sensor time series data.

[0031] In a sixth exemplary implementation, generating a driving area map according to the point cloud data and driving area image features in the multi-source sensor time series data includes:

[0032] The point cloud data at the current moment after point cloud matching and the road surface splicing point cloud data at the previous moment before the current moment are spliced, and the spliced ​​data is used as dense point cloud data.

[0033] By performing semantic segmentation processing on the acquired driving scene image data, the driving area image information is obtained.

[0034] Based on the driving area image information and according to the correspondence between image pixels and three-dimensional points of the point cloud, non-road area data in the dense point cloud data is eliminated to obtain road surface dense point cloud data.

[0035] A driving area map is generated according to the road surface dense point cloud data, the driving scene image data and the driving area image information.

[0036] In a seventh exemplary implementation, before stitching the point cloud data at the current moment after point cloud matching and the road surface stitching point cloud data at the previous moment before the current moment, the method includes:

[0037] Obtain laser point cloud data, vehicle motion data, and vehicle motion posture data with the current time stamp.

[0038] According to the time stamp of each three-dimensional point of the laser point cloud data, the corresponding vehicle motion data and vehicle motion posture data are interpolated.

[0039] According to the vehicle movement information and vehicle posture information, the interpolated point cloud data is compensated accordingly to obtain the point cloud distortion calibration data as the point cloud data at the current moment.

[0040] In an eighth exemplary implementation, generating a driving area map according to the road surface dense point cloud data, the driving scene image data and the driving area image information includes:

[0041] According to the driving scene image data and the driving area image information, a projection image feature on a driving area plane perpendicular to the height direction is obtained.

[0042] The dense point cloud data of the road surface is processed using a point cloud target detection method to obtain point cloud bird's-eye view features.

[0043] The projection image features and the point cloud bird's-eye view features are fused, and the fused bird's-eye view features are encoded and decoded to obtain a driving area map at the current moment's bird's-eye view.

[0044] In a ninth exemplary implementation, obtaining, according to the driving scene image data and the driving area image information, a projection image feature on a driving area plane perpendicular to the height direction includes:

[0045] Mask processing is performed on the driving area image information to obtain a driving area mask image.

[0046] Image features are extracted from the driving scene image data, and mask processing is performed on the image features according to the driving area mask image.

[0047] The sensor calibration parameters are used to transform the mask image features to a bird's-eye view to obtain the projection image features.

[0048] In a tenth exemplary implementation, adjusting the road topology information of the driving area map accordingly according to the road topology data of the navigation map information includes:

[0049] A lane adjustment ratio is determined according to the lane width in the driving area map and the lane width in the road topology data, and the driving area map is adjusted accordingly according to the lane adjustment ratio.

[0050] According to the angle between the lane line of the current area where the target vehicle is located and the horizontal direction, the lane angle of the road topology data is adjusted accordingly.

[0051] The target reference area is determined in the adjusted lane topology data with the position of the target vehicle in the adjusted lane topology data as the center and the width and height of the adjusted driving area map as the reference.

[0052] The lane lines of the driving area map are corrected based on the lane lines in the target reference area.

[0053] The present invention also provides an electronic device, comprising a processor, wherein the processor is configured to implement the steps of the map generating method as described in any of the preceding items when executing a computer program stored in a memory.

[0054] Finally, the present invention also provides a non-volatile storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the map generation method as described in any of the preceding items are implemented.

[0055] The present invention also provides a computer program product, comprising a computer program / instruction, which implements the steps of the map generation method as described in any of the preceding items when executed by a processor.

[0056] Finally, the present invention also provides an unmanned vehicle, including an automatic driving perception system, wherein the automatic driving perception system is used to implement the steps of the map generation method as described in any of the preceding items when executing a computer program stored in a memory.

[0057] The technical solution provided by the present invention has the advantages of generating a driving area map based on point cloud data collected by multi-source sensors and image features of the driving area, and then using the road topology data of the local navigation information of the navigation map to correct the errors of the driving area map obtained based on the multi-source sensors, effectively improving the accuracy of the vehicle driving map finally generated. The global navigation prompt information of the navigation map is added as semantic information to the vehicle driving map, further improving the accuracy and practicality of the vehicle driving map. The method is simple and reliable, and can effectively improve the traffic capacity and safety of NOA vehicles without maps.

[0058] In addition, the present invention also provides corresponding electronic devices, non-volatile storage media, computer program products and unmanned vehicles for implementing the map generation method, which further makes the method more practical. The electronic devices, non-volatile storage media, computer program products and unmanned vehicles have corresponding advantages.

[0059] The technical features mentioned above, the technical features to be mentioned below, and the technical features shown separately in the drawings can be arbitrarily combined with each other, as long as the combined technical features are not contradictory. All feasible feature combinations are technical contents clearly recorded in this article. Any of the multiple sub-features contained in the same sentence can be applied independently, and does not have to be applied together with other sub-features. It should be understood that the above general description and the detailed description below are only exemplary and cannot limit the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the technical solutions of the present invention or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0061] Figure 1 A schematic diagram of a flow chart of a map generation method provided by the present invention;

[0062] Figure 2 A schematic diagram of a driving area map in an exemplary embodiment provided by the present invention;

[0063] Figure 3 A schematic diagram of a lane line topology diagram in an exemplary embodiment provided by the present invention;

[0064] Figure 4 A schematic diagram of an exemplary preprocessing process of dense point cloud data of a road surface provided by the present invention;

[0065] Figure 5 A schematic diagram of a lane line recognition process in an exemplary embodiment provided by the present invention;

[0066] Figure 6 A structural diagram of a specific implementation of the map generating device provided by the present invention;

[0067] Figure 7 A structural diagram of a specific implementation of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0068] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. Among them, the terms "first", "second", "third", "fourth", etc. in the specification and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations of the two are intended to cover non-exclusive inclusions. The term "exemplary" means "used as an example, embodiment or illustrative". Any embodiment described here as "exemplary" is not necessarily interpreted as being superior or better than other embodiments.

[0069] In the field of intelligent driving technology, high-speed NOA (Navigate on Autopilot) is usually used to achieve intelligent driving. High-speed NOA has a single working condition, low complexity, and requires reliance on high-precision map data. However, due to the high cost, low coverage, and slow update of high-precision maps, it is unable to meet and support the application of NOA in intelligent driving vehicles. In order to solve the high-speed NOA problem, related technologies use mapless NOA that does not rely on prior information such as high-precision maps to complete driving tasks such as automatic lane switching, autonomous overtaking and lane changing, detouring obstacles, adaptively adjusting cruise speed, responding to traffic lights, and automatically passing intersections on navigable urban roads.

[0070] Although mapless NOA does not rely on high-precision maps, in essence, when avoiding obstacles and planning paths, related technologies still need to use methods such as HDMapNet (a high-precision map algorithm based on deep learning) or SegFormer (a semantic segmentation algorithm) to build local scene maps online. HDMapNet uses a pure visual solution to build local maps, while SegFormer is based on a lidar method. Although the two use different data, they are both based on semantic segmentation and target instance segmentation. In actual tests, methods based on semantic segmentation and target instance segmentation will have problems such as discontinuous lane lines, missed detections, and false detections, and cannot meet actual safety requirements. In addition, there is a related technology based on ground marking radar visual fusion mapping based on image color. This method uses two-dimensional images captured in real time by the on-board camera of the autonomous driving vehicle and point cloud data sensed in real time by the on-board lidar to build a point cloud map. The three-dimensional point cloud data of the road surface extracted from the point cloud map is colored by the road color in the two-dimensional image to obtain the three-dimensional colored point cloud corresponding to the road surface, and the corresponding colored ground markings are obtained through a series of processing, so as to realize the automatic mapping of the ground markings. While ensuring the accuracy of the map, the map elements are extracted according to the rich semantic information of the ground markings provided in the two-dimensional image. However, since the color information of the two-dimensional image is easily affected by factors such as lighting and weather, there are certain problems in the final mapping effect, and it cannot meet the actual safety needs. In addition, a mapping method of the related technology is: identifying preset visual features from multiple target images taken by multiple cameras of the vehicle at the current moment, identifying target features matching the visual features from the electronic map of the autonomous driving navigation, and then determining the vehicle's posture at the current moment according to the position of the target feature in the electronic map of the autonomous driving navigation and the position of the visual feature in the target image, and finally determining the key frame through positioning in combination with the overhead perception map, so as to form a local map. This method uses the navigation map for self-vehicle positioning, and does not use the navigation map information for local mapping. The map finally constructed also cannot meet the safety requirements. This shows that the relevant technology cannot meet users' safety needs for intelligent driving, and cannot ensure that autonomous driving vehicles can safely complete autonomous driving tasks.

[0071] In view of this, the present invention makes full use of the local navigation information and global navigation information of the navigation map, and corrects the errors in the bev local map obtained by the vehicle's own sensors. Adding semantic information to the corrected local map according to the global navigation prompt information to generate a local long-distance light semantic map can effectively improve the traffic capacity and safety of NOA vehicles without a map. After introducing the technical solution of the present invention, various non-limiting embodiments of the present invention are described in detail below. In order to better illustrate the present invention, numerous specific details are given in the specific embodiments below. Those skilled in the art should understand that the present invention can also be implemented without these specific details. In other examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present invention.

[0072] First see Figure 1 , Figure 1 A flowchart of a map generation method provided in this embodiment may include the following contents:

[0073] S101: Acquire time-synchronized navigation map information and multi-source sensor time series data.

[0074] In order to improve the accuracy of map generation, before obtaining the multi-source sensor time series data, the calibration of each sensor of the vehicle can be completed through the internal mechanism of the vehicle. The sensors include but are not limited to inertial measurement units, odometers, lidars, and forward cameras, and the coordinate system transformation relationship between the sensors is obtained. Exemplarily, the calibration parameters of the vehicle before leaving the factory can be directly used. The vehicle of this embodiment can be any vehicle with an automatic driving function, and is not limited to automatic driving vehicles or unmanned vehicles. The multi-source sensor time series data is the data collected by each sensor, and the navigation map information includes local road topology information and global navigation information. It is a navigation map generated by the automatic driving perception system of the unmanned automatic driving vehicle. In order to reduce the difficulty of lane line matching, the local navigation information can be set to a God's perspective. In order to further improve the accuracy of map generation, the time alignment of the navigation map information and the multi-source sensor time series data can be ensured through a data time synchronization mechanism.

[0075] S102: Generate a driving area map according to the point cloud data and driving area image features in the multi-source sensor time series data.

[0076] Among them, the point cloud data includes laser point cloud data collected by the lidar, vehicle motion posture data collected by the inertial measurement unit, and vehicle motion distance information collected by the odometer. The driving area image feature is the vehicle driving image data collected from the on-board camera during the vehicle driving process, which is used to represent the vehicle's drivable area information. According to the lidar point cloud data and the driving area image features, the local point cloud map at the current moment can be obtained, and then deep learning is used to obtain the point cloud data of the road surface area at the current moment based on the point cloud data, and the lane line information of the vehicle's drivable area information is combined to obtain the local map at the current moment. For the convenience of description, it is defined as a driving area map.

[0077] S103: adjusting the road topology information of the driving area map according to the road topology data of the navigation map information, and adding the navigation prompt information of the navigation map information as semantic information to the adjusted driving area map to generate a vehicle driving map.

[0078] Since the previous step is based on deep learning methods to identify lane lines, deep learning cannot completely and accurately identify all lane lines. For example, the same lane line may be intermittent. In addition, it is easy to identify errors or fail to identify at intersections and main and auxiliary road junctions. This leads to the generation of the driving area map in the previous step. The driving area map may have problems such as intermittent lane lines and missed recognition of far-end lane lines. Figure 2 As shown, in order to obtain a high-precision map, this step corrects the driving area map of S102 based on the local navigation road topology map information, further corrects the wrong lane line type error, and repairs the lane line discontinuity problem. In order to make the map safer and improve the vehicle's traffic capacity, this step adds semantic information of the global navigation map information. The semantic information includes but is not limited to road speed limits, traffic light location / status information, main and auxiliary road switching prompt information, turn prompt information, and congestion conditions. The vehicle driving map finally generated can be applied to roadside perception, smart transportation, V2X (vehicle to everything, vehicle-to-external information exchange), intelligent security, and collaborative perception to assist vehicles in completing driving tasks more safely and efficiently.

[0079] In the technical solution provided in this embodiment, a driving area map is generated based on the point cloud data and driving area image features collected by multi-source sensors, and then the road topology data of the local navigation information of the navigation map is used to correct the errors of the driving area map obtained based on the multi-source sensors, effectively improving the accuracy of the vehicle driving map finally generated. The global navigation prompt information of the navigation map is added as semantic information to the vehicle driving map, further improving the accuracy and practicality of the vehicle driving map. This method is simple and reliable, and can effectively improve the traffic capacity and safety of NOA vehicles without maps.

[0080] In the above embodiment, there is no limitation on how to time-align the navigation map information and the multi-source sensor time series data. This embodiment also provides a time alignment method, which may include the following contents:

[0081] Acquire three-dimensional point cloud data, vehicle motion and posture data, and driving scene image data as multi-source sensor time series data; acquire local navigation maps and global navigation maps with timestamps; and perform time alignment processing on local navigation maps, global navigation maps, and multi-source sensor time series data based on timestamps.

[0082] In this embodiment, considering that the navigation device usually does not have a time synchronization interface, when acquiring each frame of local navigation information and global navigation information, its timestamp information can be acquired at the same time. For example, the timestamp information of the navigation data can be acquired through a serial port, a network port, etc., and finally the timestamp data is synchronized with the sensor data.

[0083] As can be seen from the above, this embodiment ensures the time synchronization of navigation data and sensor data through timestamps, which can not only reduce the complexity of map generation, but also help improve the accuracy of map generation.

[0084] In the above embodiment, there is no limitation on how to obtain navigation map information and what information of the navigation map is obtained. This embodiment also provides an exemplary implementation of navigation map information, which may include the following contents:

[0085] Data acquisition of global navigation information in navigation map information: obtain the current frame global navigation map; based on the fixed position of the target navigation prompt information in the global navigation map, obtain the target navigation prompt information from the corresponding position of the current frame global navigation map; locate the target vehicle in the current frame global navigation map, and based on the position information of the target vehicle in the current frame global navigation map, obtain the road congestion status information in the target area ahead of the target vehicle.

[0086] The data obtained in the global navigation information of this embodiment includes some prompt information at fixed positions, including but not limited to road signs and road names, and some data are not fixed in the position of the global navigation information, such as lane speed limits and road congestion status. For some prompt information in a fixed position in the global navigation information, in order to avoid description, it is defined as target navigation prompt information. The target navigation prompt information includes but is not limited to current road information, including straight, turning, etc., safety prompt information, such as traffic sign information such as construction ahead and school ahead, and can also be road name information. Exemplarily, based on the fixed position of the target navigation prompt information in the global navigation map, the current frame global navigation map can be segmented by a pre-set segmentation frame to obtain the target area block; the pre-built prompt information recognition network model is called to process the target area block to obtain the target navigation prompt information. Traffic sign information is usually displayed at a fixed position on the global navigation map. A segmentation frame can be set in advance. After the image of the area is segmented and extracted, the prompt information recognition network model can be used to identify and extract the target through the pre-built prompt information recognition network model. For example, yolo (detection network name) and MobilenetSSD (detection network name) can be pre-built and trained using corresponding methods. Traffic sign information can be obtained through the output of the prompt information recognition network model. Road name information is also displayed at a fixed position on the global navigation map. Accordingly, a segmentation frame can be set in advance. After the image of the area is segmented and extracted, the prompt information recognition network model can also have text recognition capabilities. Accordingly, the model can be trained through an image sample set that identifies text so that it has text recognition capabilities, thereby obtaining the current road information through text recognition capabilities. In addition, for some other prompt information, such as turn prompts ahead, traffic light prompts, etc. The display method is relatively fixed at first. The local image of the location of the information is obtained through color segmentation, and the text recognition network is further used to identify relevant information, such as the distance from the traffic light or intersection to the vehicle, the status of the traffic light, etc. For the situation that the prompt information to be obtained in the global navigation information is not in the fixed position of the global navigation information, such as lane speed limit and road congestion status, the position of the current vehicle in the global navigation map can be identified first. For the convenience of description, this embodiment defines the current vehicle as the target vehicle. Usually in the global navigation information, the vehicle itself will be represented by a special vehicle. Here, the position of the vehicle itself is identified through the target detection network. This position information is used to obtain the speed limit, congestion and other status information in front of the vehicle. Usually on the navigation map, according to the current road congestion status and the location of occurrence, the global planning path, such as the color information of part of the position of the thick green line in the actual global navigation map, is modified. According to the current vehicle position, the road congestion status information near the front of the vehicle can be obtained. Usually, the road speed limit information is a fixed style. The speed limit sign position can be effectively obtained through the detection network, and then combined with the position of the vehicle in the global navigation map, the current road speed limit information can be further identified.

[0087] For data acquisition of the local navigation map in the navigation map information: obtain the local navigation map of the current frame; locate the target vehicle in the local navigation map of the current frame, and determine the lane information based on the position information of the target vehicle in the local navigation map of the current frame; generate road topology data based on the lane area, double yellow line and lane line in the lane information.

[0088] In this embodiment, since the navigation map has no real distance reference, the subsequent lane line matching can be facilitated by identifying the current position of the vehicle. The lane information includes but is not limited to the lane area, double yellow line and lane line. An exemplary method for obtaining lane information is: based on the position information of the target vehicle in the local navigation map of the current frame, determine the target topology area for extracting road topology data; layer the local navigation map of the current frame according to color, and remove interference information of the color that does not belong to the lane information; perform color segmentation on the target topology area with the interference information removed to obtain the lane area, double yellow line and lane line as lane information. In order to facilitate the subsequent acquisition of lane information, reduce the amount of data processing, and improve the efficiency of lane information acquisition, the local navigation map can be color layered, such as medium green track line, light blue-gray green belt, etc., and unnecessary interference areas can be eliminated according to color information. Since the colors of the road surface and lane lines in the navigation map are relatively fixed in the local navigation information, color segmentation is used to obtain lane areas, double yellow lines, lane lines and other information from the new navigation map obtained by eliminating interference information, and road topology data is constructed based on these lane information. The road topology data can be as follows Figure 3 The lane line topology diagram is shown.

[0089] From the above, it can be seen that this embodiment obtains traffic prompt information and traffic sign information of global navigation information, and obtains lane line topology data of local navigation information. Through these data, the vehicle driving map is enriched and corrected. It is simple and reliable, and effectively improves the NOA traffic capacity and safety.

[0090] The above embodiment does not limit how to generate a driving area map. Based on the above embodiment, the present invention also provides an exemplary implementation method for generating a driving area map according to point cloud data and driving area image features in multi-source sensor time series data, which may include the following contents:

[0091] like Figure 4As shown, the point cloud data at the current moment after point cloud matching and the road surface splicing point cloud data at the previous moment are spliced, and the spliced ​​data is used as dense point cloud data; the driving area image information is obtained by semantic segmentation of the acquired driving scene image data; based on the driving area image information, according to the correspondence between the image pixels and the three-dimensional points of the point cloud, the non-road area data in the dense point cloud data is eliminated to obtain the road surface dense point cloud data; according to the road surface dense point cloud data, the driving scene image data and the driving area image information, a driving area map is generated.

[0092] In this embodiment, considering that the single-frame point cloud data is relatively sparse, especially the mechanical laser radar is affected by its own line number, it is very easy to miss small targets, resulting in low lane line recognition accuracy. Therefore, this embodiment splices the point cloud data in time sequence to effectively improve the density of the point cloud data. Exemplarily, a method for densifying point cloud data is: considering the spatiotemporal continuity of the point cloud data, the point cloud data is matched using a point cloud matching algorithm, ICP (Iterative Closest Point, iterative closest point algorithm), NDT (Normal Distribution Transform, normal distribution transform). Due to the movement of the vehicle, the position and posture of the vehicle at the current moment and the previous moment have changed, resulting in a certain position and posture change between the two frames of point cloud data. The inertial measurement unit can obtain the posture information of the vehicle at different times, and the odometer can obtain the moving distance between the two moments of the vehicle. Through these two data, the posture change data can be estimated. In order to obtain more accurate matching results and reduce the amount of calculation, the initial data of the posture change between the point cloud data at the current moment and the previously spliced ​​point cloud can be obtained by the odometer and the inertial measurement unit. After the point cloud matching calculation is completed, the point cloud at the current moment is superimposed on the historical spliced ​​point cloud data to obtain dense point cloud data.

[0093] In order to reduce the amount of calculation and improve the accuracy of lane line calculation, and avoid lane line being missed and misunderstood, this embodiment can further pre-process the dense point cloud data, that is, remove the point cloud data of non-road area. Among them, an exemplary implementation method of obtaining the road area is: obtain the image collected by the camera facing the vehicle, such as a far-field camera, a near-field camera, a wide-angle camera, etc., that is, the driving scene image data, and use the semantic segmentation algorithm to extract the road area from the driving scene image data. Then, the calibration parameters of the camera and the laser radar are used to obtain the correspondence between the image pixels and the point cloud points. Among them, the exemplary acquisition method of the correspondence between the two is: according to the three-dimensional coordinate system of the camera imaging system and the laser radar system, through a finite number of known spatial homonymous points, the spatial transformation relationship of the two sets of coordinate systems can be obtained. This process is called calibration. When two sets of spatial correspondences are obtained, the correspondence between any spatial homonymous points in the two sets of coordinate systems can be obtained. Based on this principle, combined with the imaging principle, the correspondence between the image pixel points and the spatial points can be obtained first, and then based on the spatial point correspondence, it can be corresponded to the point cloud data. According to the correspondence between image pixels and point clouds, it is possible to identify which point clouds belong to roads, so as to make point-by-point judgments on the point cloud data and eliminate non-road point cloud data. Then the point cloud data in non-road areas is removed, and finally, according to the pre-set focus area, such as Figure 2 The dense point cloud data of the road area in front of the vehicle is obtained by taking a 100m×20m area as an example, that is, obtaining the dense point cloud data of the road surface.

[0094] In order to further improve the processing flexibility and accuracy of point cloud densification, this embodiment also provides a point cloud densification method: considering the spatiotemporal continuity of point cloud data, a point cloud matching algorithm, such as ICP (Iterative Closest Point) and NDT (Normal Distribution Transform), can be used to match point clouds. For the same scene, two frames of point cloud data are obtained from different distances or angles. Since the two frames of point cloud data have overlapping areas, that is, the spatial structure of the point clouds is similar, the ICP or NDT algorithm can be used to obtain the transformation relationship between the two frames of point cloud data based on the point cloud data in the overlapping area, that is, the rotation transformation matrix R and the translation vector T are obtained. In order to obtain more accurate matching results and reduce the amount of matching calculations, this step can obtain vehicle motion data through the odometer and vehicle motion posture data through the inertial strategy unit, and use it as the initial posture change parameter between the point cloud data at the current moment and the previously spliced ​​point cloud as the initial value of the iterative calculation, which can effectively reduce the number of iterations. After the point cloud matching calculation is completed, the dense point cloud at the previous moment, that is, time t-1, is transformed by obtaining the rotation transformation matrix R and the translation vector T. As the vehicle continues to move forward, the dense point cloud data after the transformation at time t-1 enters the area behind the front of the vehicle. Therefore, it is necessary to remove the point cloud data that is not at the location of the vehicle at the current moment in the transformed spliced ​​point cloud data. For example, the laser radar coordinate system can be used as the vehicle coordinate system, and the point cloud data that is not at the location of the vehicle at the current moment in the transformed spliced ​​point cloud data is removed by eliminating the part with an x ​​coordinate less than 0 in the dense point cloud after the transformation at time t-1, and then splicing it with the point cloud data at the current moment t to obtain the dense point cloud data at the current moment.

[0095] In order to further improve the recognition accuracy of low raised obstacles or potholes, considering that the mechanical LiDAR (Light Laser Detection and Ranging, laser radar) will produce distortion along the direction of vehicle movement during vehicle driving due to its data receiving principle limitation, and the distortion increases with the increase of vehicle speed, which is very likely to cause a judgment error in the longitudinal direction consistent with the direction of the vehicle head. That is, if the distortion is not eliminated, it will cause false detection and missed detection of lane information recognition. Based on the above embodiment, this embodiment also corrects the distortion of the laser radar point cloud data through the point cloud data collected by the odometer and the inertial measurement unit, such as Figure 4 As shown, the distortion correction process may include the following:

[0096] The laser point cloud data, vehicle motion data and vehicle motion posture data with the timestamp at the current moment are obtained; the corresponding vehicle motion data and vehicle motion posture data are interpolated according to the timestamp of each three-dimensional point of the laser point cloud data; the interpolated point cloud data is compensated accordingly according to the vehicle movement information and vehicle posture information to obtain the point cloud distortion calibration data as the point cloud data at the current moment.

[0097] Among them, the odometer is used to obtain the vehicle's movement distance information within a unit time period, and the inertial measurement unit is used to obtain the vehicle's movement posture data within the corresponding unit time period. The mechanical laser radar usually has a data frame rate of 10hz. Correspondingly, the unit time period can be set to 0.1 seconds, that is, the odometer is used to obtain the vehicle's movement distance information within 0.1 seconds, and the inertial measurement unit is used to obtain the vehicle's movement posture data within 0.1 seconds. Of course, it can also be flexibly selected according to the actual application scenario, which does not affect the implementation of the present invention. Each point in the laser point cloud data of this embodiment is accompanied by timestamp information. For each point in the point cloud, the acquired vehicle motion data and vehicle posture data are interpolated according to its timestamp information, and their coordinates are compensated. Taking the compensation of vehicle motion data as an example, the actual time of the current data frame acquisition is , for a point cloud data point , whose uncompensated coordinates are , whose timestamp is If The vehicle moves s meters along the x-axis during the time, so the coordinates after compensation are In actual application scenarios, when the vehicle's moving direction or posture changes, corresponding compensation can be made to improve the recognition accuracy of low raised obstacles or potholes.

[0098] In order to effectively reduce the amount of calculation and eliminate the interference of moving targets, this embodiment also provides an implementation method for generating a local driving area map based on the image drivable area and the road point cloud, which may include the following contents:

[0099] According to the driving scene image data and the driving area image information, the projection image features on the plane of the driving area perpendicular to the height direction are obtained; the dense point cloud data of the road surface are processed using the point cloud target detection method to obtain the point cloud bird's-eye view features; the projection image features and the point cloud bird's-eye view features are fused, and the fused bird's-eye view features are encoded and decoded to obtain the driving area map under the bird's-eye view at the current moment.

[0100] In this embodiment, the driving scene image data is the image data collected during the driving process of the vehicle. It is understandable that there will be dynamic targets such as vehicles and pedestrians in the driving scene, that is, the driving area image data collected by the vehicle contains dynamic targets such as vehicles and pedestrians. These targets will have a certain impact on the acquisition of autonomous driving map data, so they need to be eliminated as much as possible. Any semantic segmentation network, such as unet (semantic segmentation network name) and enet (semantic segmentation network name), can be used to identify the drivable area in the image to obtain the driving area image information representing the drivable area of ​​the road surface. This embodiment realizes lane line recognition from a bird's-eye view, such as Figure 5 As shown, it is necessary to obtain image bev features and point cloud bev features. Image bev features refer to the projection of image features on a plane perpendicular to the height direction, that is, the projection image features of this embodiment, and point cloud bev features refer to the projection of point clouds on a plane perpendicular to the height direction, that is, the point cloud bird's-eye view features of the basic embodiment.

[0101] Among them, the acquisition process of the projection image features can be: the image features of the driving scene image data can be extracted through the backbone network of any deep learning network model, and the backbone network includes but is not limited to ResNet (backbone network name), VGG (backbone network name), and EfficientNet (backbone network name). In order to improve the data processing efficiency, the mask information of the driving area image information can be constructed, and the image data can be directly masked by the mask information to effectively improve the data processing efficiency. The corresponding implementation process may include: masking the driving area image information to obtain the driving area mask image; extracting image features from the driving scene image data, and masking the image features according to the driving area mask image; using the sensor calibration parameters to convert the mask image features to a bird's-eye view to obtain the projection image features. In other words, this embodiment masks the image features through the driving area mask image, and then uses the calibration parameters to convert the image features to the bev view.

[0102] Among them, the acquisition process of the point cloud bird's-eye view feature may include: for dense point cloud data on the road surface, the feature extraction and Neck (module name) module of any point cloud target detection algorithm can be used to obtain the point cloud Bev feature. Among them, point cloud target detection algorithms such as Pointpillars (algorithm name), Second (algorithm name), Centerpoin (algorithm name), the feature extraction module is used to extract the basic features of the image and generate feature maps of different scales. The Neck module helps the network to better detect targets of different scales by fusing features from different levels, and can also improve the feature representation ability, especially for the detection of small targets and complex scenes. In addition, the Neck module usually performs downsampling and upsampling operations on the feature map to better capture information of different scales. After the image bev features and point cloud bev features are obtained, the image bev features and point cloud bev features are stacked, and then sent to the encoder to obtain the fused bev features, and further through the decoder, the lane line recognition result of the bev perspective at the current moment is obtained as a driving area map.

[0103] From the above, it can be seen that this embodiment helps to improve the lane line recognition accuracy by densifying and correcting the point cloud data; then eliminating non-road point cloud data from the point cloud data, which not only reduces the amount of calculation required for the map generation process, but also helps to improve the lane line calculation accuracy.

[0104] It is understandable that the local driving area map generated in S102 and the lane topology data of the navigation map information have great differences in perspective, scale, etc., and it is impossible to accurately use the road topology data of the navigation map information for correction without knowing the accurate perspective and scale parameters. Based on this, the present invention also provides a high-precision map correction method, which may include the following contents:

[0105] According to the lane width in the driving area map and the lane width in the road topology data, the lane adjustment ratio is determined, and the driving area map is adjusted accordingly according to the lane adjustment ratio; according to the angle between the lane line in the current area where the target vehicle is located and the horizontal direction, the lane angle of the road topology data is adjusted accordingly; with the position of the target vehicle in the adjusted lane topology data as the center and the width and height of the adjusted driving area map as the reference, the target reference area is determined in the adjusted lane topology data; according to the lane lines in the target reference area, the lane lines of the driving area map are corrected.

[0106] Among them, the driving area map is a square area centered on the vehicle, which corresponds to a certain area in the real world at a certain ratio, such as 51.2 meters × 51.2 meters. Therefore, the relationship between its lane lines is the spatial distance relationship in the real world. In order to facilitate display on the navigator, the lane topology map may be compressed into a rectangular area, which is only a scalar without the concept of actual distance. Therefore, there is a large difference in scale between the lane topology map obtained from the local navigation information and the generated driving area map, and they cannot be matched directly. This embodiment first adjusts the lane width of the generated driving area map to the lane topology map: calculate the lane width of the driving area map separately and lane widths from road topology data , get the scale transformation coefficient , using the scaling factor Scale the driving area map. Then adjust the lane angle in the road topology data: calculate the angle between the lane line near the current vehicle and the horizontal direction, so that the lane line of the current vehicle in the lane topology map corresponding to the road topology data is consistent with the lane line near the current vehicle in the driving area map. Finally, establish a corresponding relationship between the driving area map and the corresponding area in the lane topology map: take the current vehicle position in the lane topology map corresponding to the road topology data as the center, and take the width and height of the driving area map after the scale adjustment as the reference to obtain the corresponding area in the lane topology map corresponding to the road topology data, and correct the lane lines in the driving area map according to the lane lines in the corresponding area of ​​the lane topology map.

[0107] As can be seen from the above, this embodiment adjusts the parameters of the lane topology map and the driving area map, and determines the regional correspondence between the two, so as to accurately use the road topology data of the navigation map information to correct it, thereby effectively improving the traffic capacity and safety of NOA vehicles without a map.

[0108] It should be noted that there is no strict order of execution between the steps in the present invention. As long as they comply with the logical order, these steps can be executed simultaneously or in a certain preset order. Figure 1 , Figure 4 and Figure 5 This is just a schematic and does not mean that this is the only execution order.

[0109] The present invention also provides a corresponding device for the map generation method, which further makes the method more practical. Among them, the device can be described from the perspective of functional modules and hardware. The map generation device provided by the present invention is introduced below. The device is used to implement the map generation method provided by the present invention. In this embodiment, the map generation device may include or be divided into one or more program modules. The one or more program modules are stored in a storage medium and executed by one or more processors to complete the map generation method disclosed in Example 1. The program module referred to in this embodiment refers to a series of computer program instruction segments that can complete specific functions, which is more suitable for describing the execution process of the map generation device in the storage medium than the program itself. The following description will specifically introduce the functions of each program module of this embodiment. The map generation device described below and the map generation method described above can be referenced to each other.

[0110] From the perspective of functional modules, see Figure 6 , Figure 6 The structure diagram of a map generation device provided in this embodiment in a specific implementation manner, the device may include:

[0111] The data acquisition module 601 is used to acquire time-synchronized navigation map information and multi-source sensor time series data.

[0112] The initial map generation module 602 is used to generate a driving area map according to the point cloud data and driving area image features in the multi-source sensor time series data.

[0113] The map generation module 603 is used to adjust the road topology information of the driving area map according to the road topology data of the navigation map information, and add the navigation prompt information of the navigation map information as semantic information to the adjusted driving area map to generate a vehicle driving map.

[0114] Exemplarily, in some implementations of the present embodiment, the above-mentioned data acquisition module 601 can also be used to: obtain the global navigation map of the current frame; based on the fixed position of the target navigation prompt information in the global navigation map, obtain the target navigation prompt information from the corresponding position of the global navigation map of the current frame; locate the target vehicle in the global navigation map of the current frame, and based on the position information of the target vehicle in the global navigation map of the current frame, obtain the road congestion status information in the target area ahead of the target vehicle.

[0115] As an exemplary implementation method of the above embodiment, the above data acquisition module 601 can be further used for: based on the fixed position of the target navigation prompt information in the global navigation map, segmenting the current frame global navigation map through a pre-set segmentation frame to obtain the target area block; calling the pre-built prompt information recognition network model to process the target area block to obtain the target navigation prompt information.

[0116] Exemplarily, in some other implementations of this embodiment, the above-mentioned data acquisition module 601 can also be used to: obtain the local navigation map of the current frame; locate the target vehicle in the local navigation map of the current frame, and determine the lane information based on the position information of the target vehicle in the local navigation map of the current frame; generate road topology data according to the lane area, double yellow line and lane line in the lane information.

[0117] As an exemplary implementation method of the above embodiment, the above data acquisition module 601 can be further used to: determine the target topology area for extracting road topology data based on the position information of the target vehicle in the local navigation map of the current frame; layer the local navigation map of the current frame according to color, and remove interference information of colors that do not belong to lane information; perform color segmentation on the target topology area with interference information removed to obtain lane areas, double yellow lines and lane lines as lane information.

[0118] Exemplarily, in some other implementations of this embodiment, the above-mentioned data acquisition module 601 can also be used to: acquire three-dimensional point cloud data, vehicle motion and posture data, and driving scene image data as multi-source sensor time series data; acquire local navigation maps and global navigation maps with timestamps; and perform time alignment processing on local navigation maps, global navigation maps, and multi-source sensor time series data based on timestamps.

[0119] Exemplarily, in some other implementations of the present embodiment, the above-mentioned initial map generation module 602 can also be used to: stitch the point cloud data at the current moment after point cloud matching and the road surface stitching point cloud data at the previous moment, and use the stitching data as dense point cloud data; obtain the driving area image information by performing semantic segmentation processing on the acquired driving scene image data; based on the driving area image information, according to the correspondence between the image pixels and the three-dimensional points of the point cloud, eliminate the non-road area data in the dense point cloud data to obtain the road surface dense point cloud data; generate a driving area map according to the road surface dense point cloud data, the driving scene image data and the driving area image information.

[0120] As an exemplary implementation method of the above embodiment, the above initial map generation module 602 can be further used to: obtain laser point cloud data, vehicle motion data and vehicle motion posture data carrying a timestamp at the current moment; interpolate the corresponding vehicle motion data and vehicle motion posture data according to the timestamp of each three-dimensional point of the laser point cloud data; compensate the interpolated point cloud data accordingly according to the vehicle movement information and vehicle posture information to obtain point cloud distortion calibration data as the point cloud data at the current moment.

[0121] As another exemplary implementation method of the above embodiment, the above initial map generation module 602 can be further used to: obtain projection image features on the plane of the driving area perpendicular to the height direction according to the driving scene image data and the driving area image information; use the point cloud target detection method to process the dense point cloud data of the road surface to obtain the point cloud bird's-eye view features; fuse the projection image features and the point cloud bird's-eye view features, and encode and decode the fused bird's-eye view features to obtain the driving area map under the bird's-eye view at the current moment.

[0122] As another exemplary implementation method of the above embodiment, the above initial map generation module 602 can be further used to: perform mask processing on the driving area image information to obtain a driving area mask image; extract image features from the driving scene image data, and perform mask processing on the image features according to the driving area mask image; use sensor calibration parameters to convert the mask image features to a bird's-eye view to obtain projection image features.

[0123] Exemplarily, in some other implementations of the present embodiment, the map generation module 603 may also be used to: determine the lane adjustment ratio based on the lane width in the driving area map and the lane width in the road topology data, and adjust the driving area map accordingly according to the lane adjustment ratio; adjust the lane angle of the road topology data accordingly according to the angle between the lane line in the current area where the target vehicle is located and the horizontal direction; determine the target reference area in the adjusted lane topology data with the position of the target vehicle in the adjusted lane topology data as the center and the width and height of the adjusted driving area map as the reference; and make inventory corrections to the lane lines of the driving area map according to the lane lines in the target reference area.

[0124] The map generating device mentioned above is described from the perspective of functional modules. Furthermore, the present invention also provides an electronic device, which is described from the perspective of hardware. Figure 7 The following is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention in one implementation manner. Figure 7 As shown, the electronic device includes a memory 70 for storing a computer program; and a processor 71 for implementing the steps of the map generating method mentioned in any of the above embodiments when executing the computer program.

[0125] Among them, the processor 71 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and the processor 71 may also be a controller, a microcontroller, a microprocessor or other data processing chip. The processor 71 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 71 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 71 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 71 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0126] The memory 70 may include one or more computer non-volatile storage media, which may be non-transitory. The memory 70 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices and flash memory storage devices. In some embodiments, the memory 70 may be an internal storage unit of an electronic device, such as a hard disk of a server. In other embodiments, the memory 70 may also be an external storage device of an electronic device, such as a plug-in hard disk equipped on a server, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Further, the memory 70 may also include both an internal storage unit of an electronic device and an external storage device. The memory 70 may not only be used to store application software and various types of data installed in the electronic device, such as: the code of the program in the process of executing the map generation method, etc., but also be used to temporarily store data that has been output or is to be output. In this embodiment, the memory 70 is at least used to store the following computer program 701, wherein, after the computer program is loaded and executed by the processor 71, the relevant steps of the map generation method disclosed in any of the aforementioned embodiments can be implemented. In addition, the resources stored in the memory 70 may also include an operating system 702 and data 703, etc., and the storage method may be temporary storage or permanent storage. The operating system 702 may include Windows, Unix, Linux, etc. The data 703 may include but is not limited to data corresponding to the map generation result, etc.

[0127] In some embodiments, the electronic device may further include a display screen 72, an input / output interface 73, a communication interface 74 or a network interface, a power supply 75 and a communication bus 76. Among them, the display screen 72 and the input / output interface 73, such as a keyboard, belong to the user interface, and exemplary user interfaces may also include standard wired interfaces, wireless interfaces, and the like. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode) touch device, and the like. The display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device and to display a visual user interface. The communication interface 74 may exemplarily include a wired interface and / or a wireless interface, such as a WI-FI interface, a Bluetooth interface, and the like, which are generally used to establish a communication connection between the electronic device and other electronic devices. The communication bus 76 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, and the like. The bus may be divided into an address bus, a data bus, a control bus, and the like. For ease of representation, Figure 7 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0128] Those skilled in the art will understand that Figure 7 The structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the figure, for example, it may also include a sensor 77 to implement various functions.

[0129] It is understandable that if the map generation method in the above embodiment is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a non-volatile storage medium. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the relevant technology or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and executes all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes but is not limited to: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), electrically erasable programmable ROM, register, hard disk, multimedia card, card-type memory (such as SD or DX memory, etc.), magnetic memory, removable disk, CD-ROM, magnetic disk or optical disk, etc. Various media that can store program codes. Based on this, the present invention also provides a non-volatile storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the map generation method described in any of the above embodiments are executed.

[0130] It is understandable that if the map generation method in the above embodiment is implemented in the form of a software functional unit and sold or used as an independent product, the computer software product does not need to be stored in a physical storage medium. For example, it can be directly transmitted to a computer or other device with information processing capabilities through a wired network or a wireless network to perform all or part of the steps of the methods of various embodiments of the present invention. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the relevant technology or all or part of the technical solution can be embodied in the form of a software product. Based on this, the present invention also provides a computer program product that stores a computer program. When the computer program is executed by a processor, the steps of the map generation method recorded in any of the above embodiments are performed.

[0131] The present invention also provides an unmanned vehicle, which completely reuses the sensors of the autonomous driving vehicle without the need to install other additional sensors, that is, the vehicle-side GPS (Global Positioning System), inertial measurement unit, odometer, lidar, and forward camera are used, and the forward camera includes a telephoto camera and a short-focus camera. On this basis, data is collected by each sensor and transmitted to an autonomous driving perception system. The autonomous driving perception system obtains multi-source sensor time series data, and combines it with navigation map information. When executing a computer program stored in a memory, the steps of the map generation method recorded in any of the above embodiments are implemented, and finally a map with added semantic information and corrected road topology is generated, effectively improving the traffic capacity and safety of NOA vehicles without a map.

[0132] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the hardware disclosed in the embodiments, including electronic devices, non-volatile storage media, computer program products and unmanned vehicles, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0133] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0134] The map generation method, electronic device, non-volatile storage medium, computer program product and unmanned vehicle provided by the present invention are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. It should be pointed out that based on the embodiments of the present invention, for ordinary technicians in this technical field, all other embodiments obtained without creative work are within the scope of protection of the present invention. Without departing from the principles of the present invention, the present invention can also be improved and modified in a number of ways, and these improvements and modifications also fall within the scope of protection of the present invention.

Claims

1. A map generation method, characterized in that: include: Acquire time-synchronized navigation map information and multi-source sensor time series data; wherein the navigation map information includes local road topology information and global navigation information; Generate a driving area map according to the point cloud data and driving area image features in the multi-source sensor time series data; According to the road topology data of the navigation map information, the road topology information of the driving area map is adjusted accordingly, and the navigation prompt information of the navigation map information is added as semantic information to the adjusted driving area map to generate a vehicle driving map; wherein the navigation prompt information is the semantic information of the global navigation map; Wherein, the road topology information of the driving area map is adjusted accordingly according to the road topology data of the navigation map information, including: Determining a lane adjustment ratio according to the lane width in the driving area map and the lane width in the road topology data, and adjusting the driving area map accordingly according to the lane adjustment ratio; According to the angle between the lane line of the current area where the target vehicle is located and the horizontal direction, the lane angle of the road topology data is adjusted accordingly; Determine a target reference area in the adjusted lane topology data, taking the position of the target vehicle in the adjusted lane topology data as the center and the width and height of the adjusted driving area map as the reference; The lane lines of the driving area map are corrected according to the lane lines in the target reference area.

2. The map generation method according to claim 1, characterized in that: The step of obtaining time-synchronized navigation map information includes: Get the global navigation map of the current frame; Based on the fixed position of the target navigation prompt information in the global navigation map, acquiring the target navigation prompt information from the corresponding position of the global navigation map of the current frame; The target vehicle is located in the current frame global navigation map, and based on the position information of the target vehicle in the current frame global navigation map, the road congestion status information in the target area ahead of the target vehicle is obtained.

3. The map generation method according to claim 2, characterized in that: Based on the fixed position of the target navigation prompt information in the global navigation map, acquiring the target navigation prompt information from the corresponding position of the current frame global navigation map includes: Based on the fixed position of the target navigation prompt information in the global navigation map, the current frame global navigation map is segmented by a preset segmentation frame to obtain a target area block; A pre-built prompt information recognition network model is called to process the target area block to obtain target navigation prompt information.

4. The map generation method according to claim 1, characterized in that: Get time-synchronized navigation map information, including: Get the local navigation map of the current frame; Positioning a target vehicle in the local navigation map of the current frame, and determining lane information based on position information of the target vehicle in the local navigation map of the current frame; Road topology data is generated according to the lane area, double yellow line and lane line in the lane information.

5. The map generation method according to claim 4, characterized in that: Determining lane information based on the position information of the target vehicle in the local navigation map of the current frame includes: Determine a target topology area for extracting the road topology data based on the position information of the target vehicle in the local navigation map of the current frame; The local navigation map of the current frame is layered according to color, and interference information of colors that do not belong to lane information is removed; The target topological area with interference information removed is subjected to color segmentation processing to obtain the lane area, double yellow line and lane line as lane information.

6. The map generation method according to claim 1, characterized in that: The step of acquiring time-synchronized navigation map information and multi-source sensor time series data includes: Acquire 3D point cloud data, vehicle motion and posture data, and driving scene image data as multi-source sensor time series data; Get local navigation map and global navigation map with timestamp; Based on the timestamp, time alignment processing is performed on the local navigation map, the global navigation map and the multi-source sensor time series data.

7. The map generation method according to any one of claims 1 to 6, characterized in that: Generate a driving area map according to the point cloud data and driving area image features in the multi-source sensor time series data, including: Splicing the point cloud data at the current moment after point cloud matching and the road surface splicing point cloud data at the previous moment before the current moment, and using the spliced ​​data as dense point cloud data; By performing semantic segmentation processing on the acquired driving scene image data, the driving area image information is obtained; Based on the driving area image information, according to the correspondence between the image pixels and the three-dimensional points of the point cloud, the non-road area data in the dense point cloud data is eliminated to obtain the road dense point cloud data; A driving area map is generated according to the road surface dense point cloud data, the driving scene image data and the driving area image information.

8. The map generation method according to claim 7, characterized in that: Before splicing the point cloud data at the current moment after point cloud matching and the road surface splicing point cloud data at the previous moment before the current moment, the method includes: Obtain laser point cloud data, vehicle motion data, and vehicle motion posture data with timestamp at the current moment; According to the timestamp of each three-dimensional point of the laser point cloud data, interpolation processing is performed on the corresponding vehicle motion data and vehicle motion posture data; According to the vehicle movement information and vehicle posture information, the interpolated point cloud data is compensated accordingly to obtain the point cloud distortion calibration data as the point cloud data at the current moment.

9. The map generation method according to claim 7, characterized in that: Generating a driving area map according to the road surface dense point cloud data, the driving scene image data and the driving area image information includes: Obtaining projection image features on a plane of the driving area perpendicular to the height direction according to the driving scene image data and the driving area image information; Using a point cloud target detection method to process the dense point cloud data of the road surface, and obtain a point cloud bird's-eye view feature; The projection image features and the point cloud bird's-eye view features are fused, and the fused bird's-eye view features are encoded and decoded to obtain a driving area map at the current moment's bird's-eye view.

10. The map generation method according to claim 7, characterized in that: According to the driving scene image data and the driving area image information, a projection image feature on a driving area plane perpendicular to the height direction is obtained, including: Performing mask processing on the driving area image information to obtain a driving area mask image; Extracting image features from the driving scene image data, and performing mask processing on the image features according to the driving area mask image; The sensor calibration parameters are used to transform the mask image features to a bird's-eye view to obtain the projection image features.

11. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the processor is used to implement the steps of the map generating method according to any one of claims 1 to 10 when executing a computer program stored in the memory.

12. A non-volatile storage medium, characterized in that: The non-volatile storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the map generation method according to any one of claims 1 to 10 are implemented.

13. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the map generation method according to any one of claims 1 to 10 are implemented.

14. An unmanned vehicle, characterized in that: It includes an autonomous driving perception system, which is used to implement the steps of the map generation method as described in any one of claims 1 to 10 when executing a computer program stored in a memory.

Images