High-precision map road label determination method and device, and electronic equipment

Abstract

This application discloses a method, apparatus, and electronic device for determining road signs in high-precision maps. The method includes: acquiring current laser point cloud data, road image data, and vehicle positioning data; performing semantic segmentation on the road image data to obtain image semantic segmentation results, the image semantic segmentation results including image semantic segmentation results of drivable road areas and image semantic segmentation results of road signs; determining the three-dimensional point cloud data of the road signs corresponding to the image semantic segmentation results of the road signs based on the laser point cloud data and the image semantic segmentation results; and determining the road sign data of the high-precision map based on the three-dimensional point cloud data of the road signs and the vehicle positioning data. This application's method for determining road signs in high-precision maps utilizes laser point cloud data, camera image data, and vehicle positioning data to determine road sign data in high-precision maps, improving the calculation accuracy of road sign data in high-precision maps.

Description

Technical Field

[0001] This application relates to the field of high-precision map technology, and in particular to a method, apparatus and electronic device for determining road signs in high-precision maps. Background Technology

[0002] With the development of autonomous driving technology, lane line detection and the recognition of planar and three-dimensional markings on the road have gradually become important components of environmental perception in the autonomous driving process. High-precision maps are the foundation and an important component of autonomous driving systems, providing important auxiliary functions such as scene perception, autonomous positioning, real-time decision-making, and route planning.

[0003] Determining road signs is a crucial step in creating high-precision maps. However, existing methods for determining road signs still suffer from low computational accuracy, resulting in high-precision maps that cannot provide accurate and reliable support for autonomous driving. Summary of the Invention

[0004] This application provides a method, apparatus, and electronic device for determining road signs in high-precision maps, so as to improve the calculation accuracy of road signs in high-precision maps.

[0005] The embodiments of this application adopt the following technical solutions:

[0006] In a first aspect, embodiments of this application provide a method for determining road signs in a high-precision map, wherein the method includes:

[0007] Acquire current laser point cloud data, road image data, and vehicle positioning data;

[0008] The road image data is semantically segmented to obtain image semantic segmentation results, which include image semantic segmentation results of the drivable area of ​​the road surface and image semantic segmentation results of road signs.

[0009] Based on the laser point cloud data and the image semantic segmentation result, determine the three-dimensional point cloud data of the road sign corresponding to the image semantic segmentation result of the road sign;

[0010] Based on the 3D point cloud data of the road signs and the vehicle positioning data, the road sign data of the high-precision map is determined.

[0011] Optionally, determining the 3D point cloud data of the road sign corresponding to the image semantic segmentation result of the road sign based on the laser point cloud data and the image semantic segmentation result includes:

[0012] Based on the transformation relationship between the lidar and camera images, the lidar point cloud data is projected onto the corresponding road image data to obtain the image projection points corresponding to the lidar point cloud data.

[0013] Based on the image projection points corresponding to the laser point cloud data and the image semantic segmentation results, a plane fitting is performed on the image projection points corresponding to the laser point cloud data to obtain the fitted plane.

[0014] Based on the fitted plane and the image semantic segmentation result of the road sign, the three-dimensional point cloud data of the road sign corresponding to the image semantic segmentation result of the road sign is determined.

[0015] Optionally, the step of performing plane fitting on the image projection points corresponding to the laser point cloud data based on the image projection points corresponding to the laser point cloud data and the image semantic segmentation result to obtain the fitted plane includes:

[0016] Based on the image projection points corresponding to the laser point cloud data and the image semantic segmentation results of the drivable area of ​​the road surface, the image projection points of the drivable area of ​​the road surface corresponding to the laser point cloud data are determined.

[0017] Plane fitting is performed on the image projection points of the drivable area of ​​the road surface corresponding to the laser point cloud data to obtain the plane of the drivable area of ​​the road surface.

[0018] Optionally, the image semantic segmentation result of the road signage includes the image semantic segmentation result of the stereo signage. The step of performing plane fitting on the image projection points corresponding to the laser point cloud data based on the image projection points corresponding to the laser point cloud data and the image semantic segmentation result to obtain the fitted plane includes:

[0019] Based on the image projection points corresponding to the laser point cloud data and the image semantic segmentation results of the stereo marker, the image projection points of the stereo marker corresponding to the laser point cloud data are determined.

[0020] Plane fitting is performed on the image projection points of the stereoscopic marker corresponding to the laser point cloud data to obtain the stereoscopic marker plane.

[0021] Optionally, the fitted plane includes a road surface drivable area plane and a 3D sign plane, and the image semantic segmentation result of the road sign includes the image semantic segmentation result of the planar sign and the image semantic segmentation result of the 3D sign. Determining the 3D point cloud data of the road sign corresponding to the image semantic segmentation result of the road sign based on the fitted plane and the image semantic segmentation result of the road sign includes:

[0022] Based on the drivable area plane of the road surface, the image semantic segmentation result of the plane marker is transformed into the vehicle coordinate system to obtain the three-dimensional point cloud data of the plane marker;

[0023] Based on the 3D sign plane, the image semantic segmentation result of the 3D sign is transformed into the vehicle coordinate system to obtain the 3D point cloud data of the 3D sign.

[0024] Optionally, determining the road sign data of the high-precision map based on the three-dimensional point cloud data of the road signs and the vehicle positioning data includes:

[0025] The three-dimensional point cloud data of the road signs are fitted to obtain the three-dimensional fitting results of the road signs;

[0026] Based on the vehicle positioning data, the three-dimensional fitting result of the road markings is converted to the world coordinate system to obtain the road marking data of the high-precision map.

[0027] Optionally, fitting the three-dimensional point cloud data of the road sign to obtain the three-dimensional fitting result of the road sign includes:

[0028] Determine the road sign type corresponding to the 3D point cloud data of the road sign;

[0029] The three-dimensional point cloud data of the road sign is fitted using a preset fitting strategy corresponding to the road sign type to obtain the corresponding three-dimensional fitting result of the road sign.

[0030] Secondly, embodiments of this application also provide a road sign determination device for high-precision maps, wherein the device includes:

[0031] The acquisition unit is used to acquire current laser point cloud data, road image data, and vehicle positioning data.

[0032] A semantic segmentation unit is used to perform semantic segmentation on the road image data to obtain image semantic segmentation results, which include image semantic segmentation results of the drivable area of ​​the road surface and image semantic segmentation results of road signs.

[0033] The first determining unit is used to determine the three-dimensional point cloud data of the road sign corresponding to the image semantic segmentation result of the road sign based on the laser point cloud data and the image semantic segmentation result;

[0034] The second determining unit is used to determine the road sign data of the high-precision map based on the three-dimensional point cloud data of the road signs and the vehicle positioning data.

[0035] Thirdly, embodiments of this application also provide an electronic device, including:

[0036] Processor; and

[0037] A memory configured to store computer-executable instructions, which, when executed, cause the processor to perform any of the methods described above.

[0038] Fourthly, embodiments of this application also provide a computer-readable storage medium that stores one or more programs, which, when executed by an electronic device including multiple applications, cause the electronic device to perform any of the methods described above.

[0039] The at least one technical solution adopted in this application embodiment can achieve the following beneficial effects: The road sign determination method for high-precision maps in this application embodiment first acquires current laser point cloud data, road image data, and vehicle positioning data; then, it performs semantic segmentation on the road image data to obtain image semantic segmentation results, which include image semantic segmentation results of the drivable area of ​​the road surface and image semantic segmentation results of road signs; then, based on the laser point cloud data and the image semantic segmentation results, it determines the three-dimensional point cloud data of the road sign corresponding to the image semantic segmentation results of the road sign; finally, based on the three-dimensional point cloud data of the road sign and the vehicle positioning data, it determines the road sign data of the high-precision map. The road sign determination method for high-precision maps in this application embodiment fuses laser point cloud data, camera image data, and vehicle positioning data, improving the calculation accuracy of road sign data in high-precision maps. Attached Figure Description

[0040] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0041] Figure 1 This is a flowchart illustrating a method for determining road signs in a high-precision map according to an embodiment of this application.

[0042] Figure 2 This is a schematic diagram of the structure of a road sign determination device for a high-precision map according to an embodiment of this application;

[0043] Figure 3 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0045] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0046] This application provides a method for determining road signs in high-precision maps, such as... Figure 1 The diagram shows a flowchart of a method for determining road signs in a high-precision map according to an embodiment of this application. The method includes at least the following steps S110 to S140:

[0047] Step S110: Obtain the current laser point cloud data, road image data, and vehicle positioning data.

[0048] The method for determining road signs in high-precision maps according to this application can be executed by a vehicle. The vehicle is equipped with a lidar, camera, and positioning module for data collection. When determining road sign data in the high-precision map, it is necessary to first obtain the lidar point cloud data around the vehicle collected by the lidar, the road image data collected by the camera, and the vehicle positioning data output by positioning modules such as RTK (Real-time kinematic).

[0049] Since different modules collect data at different frequencies, in order to ensure the accuracy of subsequent data processing, time synchronization processing can also be performed on data from different sources. For example, interpolation methods can be used to time-align laser point cloud data, road image data, and vehicle positioning data.

[0050] Step S120: Perform semantic segmentation on the road image data to obtain image semantic segmentation results, which include image semantic segmentation results of the drivable area of ​​the road surface and image semantic segmentation results of road signs.

[0051] For road image data, a pre-trained semantic segmentation model can be used to perform semantic segmentation on the road image, thereby segmenting out the drivable area of ​​the road surface and road signs contained in the road image. Since the road image captured by the vehicle camera may not only contain the road area, but may also involve non-road areas, the drivable area of ​​the road surface segmented here refers to the area in the road image where vehicles can drive. Road signs can be mainly divided into planar signs and three-dimensional signs. Planar signs may include lane lines, arrows and stop lines, etc., while three-dimensional signs may include road signs, traffic restriction signs and traffic lights, etc.

[0052] The aforementioned semantic segmentation model can be trained based on existing convolutional neural networks such as FCN and YOLO. The specific training method can be determined by those skilled in the art based on existing technology, and no specific limitations are made here.

[0053] Step S130: Based on the laser point cloud data and the image semantic segmentation result, determine the three-dimensional point cloud data of the road sign corresponding to the image semantic segmentation result of the road sign.

[0054] Since the image semantic segmentation results obtained in the aforementioned steps can only provide two-dimensional road information, the construction of high-precision maps requires three-dimensional road sign information. Laser point cloud data can provide three-dimensional point cloud information. Therefore, by combining laser point cloud data and the corresponding image semantic segmentation results, the three-dimensional point cloud data of the road sign corresponding to the image semantic segmentation results of the road sign can be determined.

[0055] Step S140: Determine the road sign data of the high-precision map based on the three-dimensional point cloud data of the road sign and the vehicle positioning data.

[0056] After obtaining the 3D point cloud data of road signs, it is necessary to further determine the absolute position information of the 3D point cloud data in 3D space. Therefore, the final 3D road sign data can be determined by combining the vehicle RTK positioning data corresponding to the 3D point cloud data of road signs, which serves as the basic data in the high-precision map.

[0057] The road sign determination method for high-precision maps in this application integrates laser point cloud data, camera image data, and vehicle positioning data, thereby improving the calculation accuracy of road sign data in high-precision maps.

[0058] In some embodiments of this application, determining the 3D point cloud data of the road sign corresponding to the image semantic segmentation result of the road sign based on the laser point cloud data and the image semantic segmentation result includes: projecting the laser point cloud data onto the corresponding road image data according to the transformation relationship between the lidar and the camera image to obtain image projection points corresponding to the laser point cloud data; performing plane fitting on the image projection points corresponding to the laser point cloud data based on the image projection points corresponding to the laser point cloud data and the image semantic segmentation result to obtain a fitted plane; and determining the 3D point cloud data of the road sign corresponding to the image semantic segmentation result of the road sign based on the fitted plane and the image semantic segmentation result of the road sign.

[0059] In this embodiment of the application, when determining the three-dimensional point cloud data of the road sign corresponding to the image semantic segmentation result of the road sign, the LiDAR and camera images can be jointly calibrated first to obtain the transformation relationship between the LiDAR and camera images. Based on this transformation relationship, the LiDAR point cloud data can be projected onto the road image to obtain the image projection points corresponding to the LiDAR point cloud data in the road image.

[0060] Since the perception of drivable areas and three-dimensional signs by lidar may be incomplete due to angle or occlusion, the embodiments of this application can perform plane fitting on the image projection points corresponding to the lidar point cloud data in the road image based on the image semantic segmentation results, thereby obtaining a more complete fitting plane for the three-dimensional point cloud. Finally, based on the fitted plane and the image semantic segmentation results of the road signs, the three-dimensional point cloud data of the road signs corresponding to the image semantic segmentation results of the road signs can be determined.

[0061] In some embodiments of this application, the step of performing plane fitting on the image projection points corresponding to the laser point cloud data based on the image projection points corresponding to the laser point cloud data and the image semantic segmentation result to obtain the fitted plane includes: determining the image projection points of the road drivable area corresponding to the laser point cloud data based on the image projection points corresponding to the laser point cloud data and the image semantic segmentation result of the road drivable area; and performing plane fitting on the image projection points of the road drivable area corresponding to the laser point cloud data to obtain the road drivable area plane.

[0062] Since LiDAR can distinguish between three-dimensional markings such as road signs and the entire drivable area of ​​the road surface, but cannot distinguish between planar markings such as lane lines or arrows specifically included in the drivable area of ​​the road surface, this embodiment of the application can first perform planar fitting on the image projection points corresponding to the laser point cloud data in the entire drivable area of ​​the road surface when performing planar fitting on the image projection points corresponding to the laser point cloud data. That is, after projecting the laser point cloud data into the road image, the projection points falling into the drivable area of ​​the road surface can be determined according to the segmented drivable area of ​​the road surface in the road image. By performing planar fitting on these projection points, the plane of the drivable area of ​​the road surface can be obtained.

[0063] In some embodiments of this application, the image semantic segmentation result of the road sign includes the image semantic segmentation result of the stereo sign. The step of performing plane fitting on the image projection points corresponding to the laser point cloud data based on the image projection points corresponding to the laser point cloud data and the image semantic segmentation result to obtain the fitted plane includes: determining the image projection points of the stereo sign corresponding to the laser point cloud data based on the image projection points corresponding to the laser point cloud data and the image semantic segmentation result of the stereo sign; and performing plane fitting on the image projection points of the stereo sign corresponding to the laser point cloud data to obtain the stereo sign plane.

[0064] As mentioned earlier, the image semantic segmentation results of road signs can include the image semantic segmentation results of three-dimensional signs such as road signs. Based on the image semantic segmentation results of three-dimensional signs, after projecting the laser point cloud data onto the road image, the projection points falling into the image semantic segmentation area of ​​the three-dimensional sign can be determined. By performing plane fitting on these projection points, the plane of the three-dimensional sign can be obtained.

[0065] Through the above two embodiments, the drivable road area plane can be obtained by fitting the image semantic segmentation results of the drivable road area and the corresponding 3D point cloud projection points, and the stereo sign plane can be obtained by fitting the image semantic segmentation results of the stereo sign and the corresponding 3D point cloud projection points. On the one hand, 3D road information is obtained based on laser point cloud data, and on the other hand, the problem of incomplete laser point cloud data is made up for by plane fitting.

[0066] In some embodiments of this application, the fitted plane includes a road surface drivable area plane and a three-dimensional sign plane. The image semantic segmentation result of the road sign includes the image semantic segmentation result of the planar sign and the image semantic segmentation result of the three-dimensional sign. Determining the three-dimensional point cloud data of the road sign corresponding to the image semantic segmentation result of the road sign based on the fitted plane and the image semantic segmentation result of the road sign includes: converting the image semantic segmentation result of the planar sign to a vehicle coordinate system based on the road surface drivable area plane to obtain the three-dimensional point cloud data of the planar sign; and converting the image semantic segmentation result of the three-dimensional sign to a vehicle coordinate system based on the three-dimensional sign plane to obtain the three-dimensional point cloud data of the three-dimensional sign.

[0067] After obtaining the drivable road surface plane, the planar markings such as lane lines, stop lines, and arrows contained in the image semantic segmentation results of road signs can be projected onto the vehicle coordinate system based on the drivable road surface plane and pre-calibrated camera intrinsic and extrinsic parameters, thereby obtaining the 3D point cloud data of the planar markings. Similarly, after obtaining the 3D marking plane, the 3D markings such as road signs and traffic restriction signs contained in the image semantic segmentation results of road signs can be projected onto the vehicle coordinate system based on the 3D marking plane and pre-calibrated camera intrinsic and extrinsic parameters, thereby obtaining the 3D point cloud data of the 3D markings.

[0068] In autonomous driving scenarios, the vehicle coordinate system is a special moving coordinate system mainly used to describe the motion of the vehicle. Its origin coincides with the center of the rear axle of the vehicle. When the vehicle is stationary on a level road, the X-axis is generally parallel to the ground and points in front of the vehicle, the Z-axis generally passes through the center of the rear axle of the vehicle and points upward, and the Y-axis generally points to the left side of the vehicle.

[0069] In some embodiments of this application, determining the road sign data of the high-precision map based on the three-dimensional point cloud data of the road sign and the vehicle positioning data includes: fitting the three-dimensional point cloud data of the road sign to obtain a three-dimensional fitting result of the road sign; and converting the three-dimensional fitting result of the road sign to the world coordinate system based on the vehicle positioning data to obtain the road sign data of the high-precision map.

[0070] The three-dimensional point cloud data of road signs determined in the aforementioned embodiments includes both planar and stereoscopic three-dimensional point cloud data, which are all a series of three-dimensional points. When applied to high-precision maps, the three-dimensional point cloud data of different types of road signs can be fitted separately to obtain the three-dimensional fitting result of each road sign. Since the three-dimensional fitting result of the road sign is in the vehicle coordinate system, it can be further combined with the vehicle's RTK measurement value to convert the three-dimensional fitting result of the road sign into GPS (Global Positioning System) coordinates in the world coordinate system.

[0071] In some embodiments of this application, fitting the three-dimensional point cloud data of the road sign to obtain a three-dimensional fitting result of the road sign includes: determining the road sign type corresponding to the three-dimensional point cloud data of the road sign; and fitting the three-dimensional point cloud data of the road sign using a preset fitting strategy corresponding to the road sign type to obtain a three-dimensional fitting result of the corresponding road sign.

[0072] Different types of road signs require different fitting methods. For example, the 3D point cloud data of lane lines can be fitted as curves, the 3D point cloud data of stop lines can be fitted as straight lines, and the 3D point cloud data of arrows can be fitted as curves and straight lines based on the corner points, that is, the outline of the arrow can be fitted. For the 3D point cloud data of 3D signs, the 3D outline points of the 3D signs can be fitted as curves.

[0073] In addition, the 3D fitting results of all road signs can be vectorized. For example, only the endpoints of the fitting equation and a small number of intermediate sampling points can be saved, thereby reducing the storage space occupied.

[0074] In summary, the road sign determination method for high-precision maps in this application has achieved at least the following technical effects:

[0075] 1) Based on laser point cloud data, the production accuracy of planar and three-dimensional markers in high-precision maps has been improved;

[0076] 2) The RTK measurement values ​​of the vehicle are spatiotemporally aligned with the laser point cloud data and road image data, which improves the calculation accuracy of the stereoscopic markers in the high-precision map;

[0077] 3) By fitting the plane of the drivable area of ​​the road surface, the calculation accuracy of the plan markings was improved;

[0078] 4) By extracting the contour points of the 3D signs and arrow signs and converting them to the vehicle coordinate system for vectorization, the calculation accuracy of the 3D signs and arrow signs is improved.

[0079] This application embodiment also provides a road sign determination device 200 for high-precision maps, such as... Figure 2 As shown, a schematic diagram of a road sign determination device for a high-precision map according to an embodiment of this application is provided. The device 200 includes: an acquisition unit 210, a semantic segmentation unit 220, a first determination unit 230, and a second determination unit 240, wherein:

[0080] Acquisition unit 210 is used to acquire current laser point cloud data, road image data and vehicle positioning data;

[0081] Semantic segmentation unit 220 is used to perform semantic segmentation on the road image data to obtain image semantic segmentation results, the image semantic segmentation results including image semantic segmentation results of the drivable area of ​​the road surface and image semantic segmentation results of road signs;

[0082] The first determining unit 230 is used to determine the three-dimensional point cloud data of the road sign corresponding to the image semantic segmentation result of the road sign based on the laser point cloud data and the image semantic segmentation result;

[0083] The second determining unit 240 is used to determine the road sign data of the high-precision map based on the three-dimensional point cloud data of the road sign and the vehicle positioning data.

[0084] In some embodiments of this application, the first determining unit 230 is specifically used for: projecting the laser point cloud data onto the corresponding road image data according to the transformation relationship between the lidar and the camera image, to obtain the image projection points corresponding to the laser point cloud data; performing plane fitting on the image projection points corresponding to the laser point cloud data according to the image semantic segmentation result, to obtain the fitted plane; and determining the three-dimensional point cloud data of the road sign corresponding to the image semantic segmentation result of the road sign according to the fitted plane and the image semantic segmentation result of the road sign.

[0085] In some embodiments of this application, the first determining unit 230 is specifically used to: determine the image projection points of the road drivable area corresponding to the laser point cloud data based on the image projection points corresponding to the laser point cloud data and the image semantic segmentation results of the road drivable area; and perform plane fitting on the image projection points of the road drivable area corresponding to the laser point cloud data to obtain the road drivable area plane.

[0086] In some embodiments of this application, the image semantic segmentation result of the road sign includes the image semantic segmentation result of the stereo sign. The first determining unit 230 is specifically used to: determine the image projection point of the stereo sign corresponding to the laser point cloud data based on the image projection point corresponding to the laser point cloud data and the image semantic segmentation result of the stereo sign; and perform plane fitting on the image projection point of the stereo sign corresponding to the laser point cloud data to obtain the stereo sign plane.

[0087] In some embodiments of this application, the fitted plane includes a road drivable area plane and a three-dimensional sign plane, and the image semantic segmentation result of the road sign includes the image semantic segmentation result of the planar sign and the image semantic segmentation result of the three-dimensional sign. The first determining unit 230 is specifically used to: convert the image semantic segmentation result of the planar sign to the vehicle coordinate system according to the road drivable area plane to obtain the three-dimensional point cloud data of the planar sign; and convert the image semantic segmentation result of the three-dimensional sign to the vehicle coordinate system according to the three-dimensional sign plane to obtain the three-dimensional point cloud data of the three-dimensional sign.

[0088] In some embodiments of this application, the second determining unit 240 is specifically used to: fit the three-dimensional point cloud data of the road sign to obtain the three-dimensional fitting result of the road sign; and convert the three-dimensional fitting result of the road sign to the world coordinate system according to the vehicle positioning data to obtain the road sign data of the high-precision map.

[0089] In some embodiments of this application, the second determining unit 240 is specifically used to: determine the road sign type corresponding to the three-dimensional point cloud data of the road sign; and fit the three-dimensional point cloud data of the road sign using a preset fitting strategy corresponding to the road sign type to obtain the three-dimensional fitting result of the corresponding road sign.

[0090] It is understood that the road sign determination device for high-precision maps described above can implement each step of the road sign determination method for high-precision maps provided in the foregoing embodiments. The relevant explanations of the road sign determination method for high-precision maps are applicable to the road sign determination device for high-precision maps, and will not be repeated here.

[0091] Figure 3 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Please refer to it. Figure 3At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and memory. The memory may include main memory, such as high-speed random-access memory (RAM), or non-volatile memory, such as at least one disk drive. Of course, the electronic device may also include other hardware required for other business operations.

[0092] The processor, network interface, and memory can be interconnected via an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 3 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0093] Memory is used to store programs. Specifically, programs may include program code, which includes computer operation instructions. Memory may include main memory and non-volatile memory, and provides instructions and data to the processor.

[0094] The processor reads the corresponding computer program from non-volatile memory into main memory and then executes it, forming a road sign determination device for a high-precision map at the logical level. The processor executes the program stored in memory and specifically performs the following operations:

[0095] Acquire current laser point cloud data, road image data, and vehicle positioning data;

[0096] The road image data is semantically segmented to obtain image semantic segmentation results, which include image semantic segmentation results of the drivable area of ​​the road surface and image semantic segmentation results of road signs.

[0097] Based on the laser point cloud data and the image semantic segmentation result, determine the three-dimensional point cloud data of the road sign corresponding to the image semantic segmentation result of the road sign;

[0098] Based on the 3D point cloud data of the road signs and the vehicle positioning data, the road sign data of the high-precision map is determined.

[0099] The above is as stated in this application. Figure 1The method executed by the high-precision map road sign determination device disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0100] The electronic device can also perform Figure 1 The method for determining road signs in medium- and high-precision maps, and the implementation of the road sign determination device in high-precision maps. Figure 1 The functions of the embodiments shown are not described in detail here.

[0101] This application also proposes a computer-readable storage medium that stores one or more programs, the programs including instructions that, when executed by an electronic device including multiple applications, enable the electronic device to perform... Figure 1 The method executed by the road sign determination device for high-precision maps in the illustrated embodiment is specifically used to perform the following:

[0102] Acquire current laser point cloud data, road image data, and vehicle positioning data;

[0103] The road image data is semantically segmented to obtain image semantic segmentation results, which include image semantic segmentation results of the drivable area of ​​the road surface and image semantic segmentation results of road signs.

[0104] Based on the laser point cloud data and the image semantic segmentation result, determine the three-dimensional point cloud data of the road sign corresponding to the image semantic segmentation result of the road sign;

[0105] Based on the 3D point cloud data of the road signs and the vehicle positioning data, the road sign data of the high-precision map is determined.

[0106] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0107] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0108] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0109] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0110] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0111] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0112] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0113] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0114] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0115] The above description is merely an embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of this application should be included within the scope of the claims of this application.

Claims

1. A method for determining road signs in a high-precision map, wherein, The method includes: Acquire current laser point cloud data, road image data, and vehicle positioning data; The road image data is semantically segmented to obtain image semantic segmentation results, which include image semantic segmentation results of the drivable area of ​​the road surface and image semantic segmentation results of road signs. Based on the laser point cloud data and the image semantic segmentation result, determine the three-dimensional point cloud data of the road sign corresponding to the image semantic segmentation result of the road sign; Based on the three-dimensional point cloud data of the road signs and the vehicle positioning data, the road sign data of the high-precision map is determined; The process of determining the road sign data for the high-precision map based on the 3D point cloud data of the road signs and the vehicle positioning data includes: The three-dimensional point cloud data of the road signs are fitted to obtain the three-dimensional fitting results of the road signs; Based on the vehicle positioning data, the three-dimensional fitting result of the road markings is converted to the world coordinate system to obtain the road marking data of the high-precision map.

2. The method as described in claim 1, wherein, The step of determining the 3D point cloud data of the road sign corresponding to the image semantic segmentation result of the road sign based on the laser point cloud data and the image semantic segmentation result includes: Based on the transformation relationship between the lidar and camera images, the lidar point cloud data is projected onto the corresponding road image data to obtain the image projection points corresponding to the lidar point cloud data. Based on the image projection points corresponding to the laser point cloud data and the image semantic segmentation results, a plane fitting is performed on the image projection points corresponding to the laser point cloud data to obtain the fitted plane. Based on the fitted plane and the image semantic segmentation result of the road sign, the three-dimensional point cloud data of the road sign corresponding to the image semantic segmentation result of the road sign is determined.

3. The method as described in claim 2, wherein, The step of performing plane fitting on the image projection points corresponding to the laser point cloud data based on the image projection points corresponding to the laser point cloud data and the image semantic segmentation result to obtain the fitted plane includes: Based on the image projection points corresponding to the laser point cloud data and the image semantic segmentation results of the drivable area of ​​the road surface, the image projection points of the drivable area of ​​the road surface corresponding to the laser point cloud data are determined. Plane fitting is performed on the image projection points of the drivable area of ​​the road surface corresponding to the laser point cloud data to obtain the plane of the drivable area of ​​the road surface.

4. The method as described in claim 2, wherein, The image semantic segmentation result of the road signage includes the image semantic segmentation result of the 3D signage. The step of performing plane fitting on the image projection points corresponding to the laser point cloud data based on the image projection points and the image semantic segmentation result to obtain the fitted plane includes: Based on the image projection points corresponding to the laser point cloud data and the image semantic segmentation results of the stereo marker, the image projection points of the stereo marker corresponding to the laser point cloud data are determined. Plane fitting is performed on the image projection points of the stereoscopic marker corresponding to the laser point cloud data to obtain the stereoscopic marker plane.

5. The method as described in claim 2, wherein, The fitted plane includes a drivable road surface plane and a 3D sign plane. The image semantic segmentation result of the road sign includes the image semantic segmentation result of the planar sign and the image semantic segmentation result of the 3D sign. Determining the 3D point cloud data of the road sign corresponding to the image semantic segmentation result of the road sign based on the fitted plane and the image semantic segmentation result of the road sign includes: Based on the drivable area plane of the road surface, the image semantic segmentation result of the plane marker is transformed into the vehicle coordinate system to obtain the three-dimensional point cloud data of the plane marker; Based on the 3D sign plane, the image semantic segmentation result of the 3D sign is transformed into the vehicle coordinate system to obtain the 3D point cloud data of the 3D sign.

6. The method of claim 1, wherein, The process of fitting the 3D point cloud data of the road sign to obtain the 3D fitting result of the road sign includes: Determine the road sign type corresponding to the 3D point cloud data of the road sign; The three-dimensional point cloud data of the road sign is fitted using a preset fitting strategy corresponding to the road sign type to obtain the corresponding three-dimensional fitting result of the road sign.

7. A road sign determination device for high-precision maps, wherein, The device includes: The acquisition unit is used to acquire current laser point cloud data, road image data, and vehicle positioning data. A semantic segmentation unit is used to perform semantic segmentation on the road image data to obtain image semantic segmentation results, which include image semantic segmentation results of the drivable area of ​​the road surface and image semantic segmentation results of road signs. The first determining unit is used to determine the three-dimensional point cloud data of the road sign corresponding to the image semantic segmentation result of the road sign based on the laser point cloud data and the image semantic segmentation result; The second determining unit is used to determine the road sign data of the high-precision map based on the three-dimensional point cloud data of the road sign and the vehicle positioning data; The second determining unit is specifically used for: The three-dimensional point cloud data of the road signs are fitted to obtain the three-dimensional fitting results of the road signs; Based on the vehicle positioning data, the three-dimensional fitting result of the road markings is converted to the world coordinate system to obtain the road marking data of the high-precision map.

8. An electronic device, comprising: processor; as well as A memory configured to store computer-executable instructions, which, when executed, cause the processor to perform the method of any one of claims 1 to 6.

9. A computer-readable storage medium storing one or more programs, which, when executed by an electronic device including a plurality of applications, cause the electronic device to perform the method of any one of claims 1 to 6.

Images