High-precision map single-element updating method and device based on non-incremental road scene

Abstract

The application relates to a high-precision map single-element updating method and device based on a non-incremental road scene, which comprises the following steps: acquiring prior difference 2D pixel coordinates of a single target element of a high-precision map to be updated, one or more trajectory correlation images, pose information of a trajectory and point cloud data; estimating a depth containing difference position information images based on a depth estimation model; estimating difference position information in a world coordinate system according to the depth containing the difference information images and the prior difference 2D pixel coordinates; projecting and segmenting point clouds corresponding to the difference position information based on the difference position information and the pose information of the trajectory; and inversely calculating pixel coordinates of each pixel to be updated into the world coordinate system and vectorizing points in the world coordinate system corresponding to the pixel coordinates. The application positions a position to be updated by introducing 2D difference information and three-dimensional reconstruction technology, and improves the precision and efficiency of single-target-element production of a high-precision map.

Description

Technical Field

[0001] This invention belongs to the field of high-precision map production technology, specifically relating to a method and apparatus for updating single elements of high-precision maps based on non-new road scenarios. Background Technology

[0002] Against the backdrop of the rapid development of autonomous driving technology, high-precision maps for autonomous driving have emerged, playing an indispensable role in the safety of ADAS (Advanced Driver Assistance Systems). They are crucial in vehicle positioning, decision-making, and real-world simulation verification. However, the prerequisite for high-precision maps to realize their maximum value is sufficient timeliness, which places high demands on the frequency of map updates. In fact, for any map provider to make its map products more competitive, besides meeting customer needs, another core factor for commercialization is maintaining quality while reducing costs and increasing efficiency. This requires maximizing the automation of map production and updates. Compared to the production of full-feature high-precision map base maps, local single-element map updates are characterized by fragmentation, relatively simple elements, and small data volume. To address these characteristics and meet the high timeliness requirements of map updates, an automated map update solution, different from that of full-feature map production, needs to be developed. Summary of the Invention

[0003] To improve the timeliness and efficiency of single-feature updates in high-precision maps, a first aspect of this invention provides a method for updating single-features in high-precision maps based on non-new road scenes. The method includes: acquiring prior difference 2D pixel coordinates of a single target feature in the high-precision map to be updated, one or more trajectory-associated images, trajectory pose information, and point cloud data; estimating the depth of an image containing difference location information from the one or more trajectory-associated images based on a depth estimation model; estimating difference location information in the world coordinate system based on the depth of the image containing difference information and the prior difference 2D pixel coordinates; projecting and segmenting the point cloud corresponding to the difference location information based on the difference location information and the trajectory pose information to obtain pixel coordinates of one or more pixels of the target feature; and inversely calculating the pixel coordinates of each pixel to be updated into the world coordinate system and vectorizing the corresponding points in the world coordinate system.

[0004] In some embodiments of the present invention, estimating the depth of the image containing differential location information from one or more trajectory-related images based on the depth estimation model includes: estimating the depth of the image containing differential location information from one or more trajectory-related images based on a monocular depth estimation model.

[0005] In some embodiments of the present invention, estimating the difference location information in the world coordinate system based on the depth of the image containing difference information and the prior difference 2D pixel coordinates includes: transforming the difference information from the pixel coordinate system to the WGS84 world coordinate system based on the depth of the image containing difference information and the prior difference 2D pixel coordinates using the PNP perspective principle.

[0006] In some embodiments of the present invention, the step of projecting and segmenting the point cloud corresponding to the difference location information based on the pose information of the difference location information and the trajectory to obtain the pixel coordinates of one or more pixels of the target element includes: projecting the point cloud corresponding to the difference location information based on the pose information of the difference location information and the trajectory to obtain a projected image; segmenting the projected image using a deep learning-based image segmentation model to obtain the pixel coordinates of one or more pixels of the target element.

[0007] Furthermore, the step of projecting the point cloud corresponding to the difference position information based on the difference position information and the pose information of the trajectory to obtain a projected image includes: taking the coordinates of the difference position as the projection center, rotating the point cloud data using the pos information of the trajectory point closest to the projection center, and rotating the point cloud data to the road direction as due north; and projecting the rotated point cloud towards the road direction to obtain a projected image.

[0008] In the above embodiments, the step of back-calculating the pixel coordinates of each pixel to be updated to the world coordinate system and vectorizing the corresponding points in the world coordinate system includes: extracting the contour from the pixel coordinates of one or more pixels of the target feature; back-calculating the pixel coordinates of each pixel to be updated back to the WGS84 coordinate system; and then vectorizing the contour and its corresponding pixel points according to the preset feature creation standard.

[0009] A second aspect of the present invention provides a high-precision map single-element update device based on a non-newly added road scene, comprising: an acquisition module, configured to acquire prior difference 2D pixel coordinates of a single target element in the high-precision map to be updated, one or more trajectory-associated images, trajectory pose information, and point cloud data; a first estimation module, configured to estimate the depth of an image containing difference location information from the one or more trajectory-associated images based on a depth estimation model; a second estimation module, configured to estimate the difference location information in the world coordinate system based on the depth of the image containing difference information and the prior difference 2D pixel coordinates; a segmentation module, configured to project and segment the point cloud corresponding to the difference location information based on the difference location information and the trajectory pose information to obtain pixel coordinates of one or more pixels of the target element; and a back-calculation module, configured to back-calculate the pixel coordinates of each pixel to be updated into the world coordinate system and vectorize the corresponding points in the world coordinate system.

[0010] A third aspect of the present invention provides an electronic device comprising: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the high-precision map single-element update method based on non-new road scenarios provided in the first aspect of the present invention.

[0011] In a fourth aspect, the present invention provides a computer-readable medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the high-precision map single-element update method based on non-new road scenarios provided in the first aspect of the present invention.

[0012] The beneficial effects of this invention are:

[0013] High-precision map base map production involves large amounts of data. Automated traffic element extraction schemes are complex and time-consuming, failing to meet the requirements for high-timeliness updates. Given the fragmented nature of map updates, the relatively simple nature of elements, and the small data volume, this paper customizes and optimizes the automated full-element base map production scheme. Instead of indiscriminately processing the entire collected data, it introduces 2D difference information and 3D reconstruction technology to locate the areas to be updated, thus meeting the requirements for efficient single-element updates. Compared to purely manual map updates, this significantly improves efficiency; compared to the full-element production scheme, this scheme precisely locates the elements to be updated, greatly reducing redundant data. Attached Figure Description

[0014] Figure 1 This is a schematic diagram of the basic process of a high-precision map single-element update method based on non-new road scenarios in some embodiments of the present invention;

[0015] Figure 2 This is a schematic diagram illustrating the specific process of a high-precision map single-element update method based on non-new road scenarios in some embodiments of the present invention.

[0016] Figure 3 This is a schematic diagram of the structure of a high-precision map single-element update device based on non-new road scenarios in some embodiments of the present invention;

[0017] Figure 4 This is a schematic diagram of the structure of an electronic device in some embodiments of the present invention. Detailed Implementation

[0018] The principles and features of the present invention are described below with reference to the accompanying drawings. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention.

[0019] refer to Figure 1 and Figure 2In a first aspect of the present invention, a method for updating a single element of a high-precision map based on a non-newly added road scene is provided, comprising: S100. acquiring prior difference 2D pixel coordinates of a single target element in the high-precision map to be updated, one or more trajectory-associated images, pose information of the trajectory, and point cloud data; S200. estimating the depth of the image containing difference location information from the one or more trajectory-associated images based on a depth estimation model; S300. estimating the difference location information in the world coordinate system based on the depth of the image containing difference information and the prior difference 2D pixel coordinates; S400. projecting and segmenting the point cloud corresponding to the difference location information based on the difference location information and the pose information of the trajectory to obtain the pixel coordinates of one or more pixels of the target element; S500. calculating the pixel coordinates of each pixel to be updated back into the world coordinate system, and vectorizing the corresponding points in the world coordinate system.

[0020] In step S100 of some embodiments of the present invention, the prior difference 2D pixel coordinates of a single target feature in the high-precision map to be updated, one or more trajectory-related images, the pose information of the trajectory, and point cloud data are obtained; the POS data mainly includes GPS data and IMU data, namely the exterior orientation elements in oblique photogrammetry: (latitude, longitude, elevation, heading angle (Phi), pitch angle (Omega), and roll angle (Kappa)).

[0021] In step S200 of some embodiments of the present invention, estimating the depth of the image containing differential location information from one or more trajectory-related images based on the depth estimation model includes: estimating the depth of the image containing differential location information from one or more trajectory-related images based on a monocular depth estimation model.

[0022] In step S300 of some embodiments of the present invention, estimating the difference location information in the world coordinate system based on the depth of the image containing difference information and the prior difference 2D pixel coordinates includes: transforming the difference information from the pixel coordinate system to the WGS84 world coordinate system based on the depth of the image containing difference information and the prior difference 2D pixel coordinates using the PNP perspective principle.

[0023] Specifically, based on the prior difference 2D pixel coordinates in step S100 and the depth estimated in step S200 for the corresponding image, the difference position information is transformed from the pixel coordinate system to the WGS84 world coordinate system in combination with the PNP perspective transformation principle.

[0024] In step S400 of some embodiments of the present invention, the step of projecting and segmenting the point cloud corresponding to the difference position information based on the pose information of the difference position information and the trajectory to obtain the pixel coordinates of one or more pixels of the target element includes:

[0025] S401. Based on the difference position information and the pose information of the trajectory, project the point cloud corresponding to the difference position information to obtain a projected image;

[0026] S402. The projected image is segmented using a deep learning-based image segmentation model to obtain the pixel coordinates of one or more pixels of the target element. Specifically, an image segmentation model is trained using deep learning technology, and the pixel coordinates of the corresponding element are segmented using the model.

[0027] Furthermore, the step of projecting the point cloud corresponding to the difference position information and the pose information of the trajectory based on the difference position information to obtain a projected image includes: using the coordinates of the difference position as the projection center, rotating the point cloud data using the pos information of the nearest trajectory point to the projection center, and rotating the point cloud data to the road direction with due north; projecting the rotated point cloud towards the road direction to obtain a projected image. Specifically, based on the difference position information in the WGS84 world coordinate system obtained in step S300, the point cloud of the corresponding region is cropped out; using the difference position coordinates of the cropped point cloud as the projection center, rotating the point cloud data using the pos information of the nearest trajectory point, and rotating the point cloud data to the road direction with due north (so that erroneous results can be eliminated according to orientation during subsequent vectorization).

[0028] In step S500 of the above embodiment, the step of recalculating the pixel coordinates of each pixel to be updated into the world coordinate system and vectorizing the corresponding points in the world coordinate system includes: S501. Extracting the contour from the pixel coordinates of one or more pixels of the target feature; S502. Recalculating the pixel coordinates of each pixel to be updated back into the WGS84 coordinate system, and then vectorizing the contour and its corresponding pixel points according to the preset feature creation standard.

[0029] Specifically, the outline is extracted from the segmented element pixel coordinates, and then calculated back to the WGS84 coordinate system based on the projection information. The shape points are then vectorized according to the element creation standards. Manual verification is then performed: the vectorized results of the automatically updated elements are validated to ensure that all results fully comply with the standards. Finally, the vectorized results are written to the database to complete the output.

[0030] It is understood that the segmentation model and target recognition model in the above steps include YOLO series, U-NET or other semantic recognition-based neural networks used to recognize or segment images.

[0031] refer to Figure 2 In one embodiment of the present invention, the following steps are included:

[0032] 1. Input data acquisition:

[0033] Input data includes prior difference 2D pixel coordinates, images associated with the trajectory, trajectory pos information, and point cloud data;

[0034] 2. Depth estimation:

[0035] Using deep learning technology, a monocular depth estimation model is trained, and the model is used to estimate the depth of images containing difference information.

[0036] 3. Three-dimensional reconstruction

[0037] Based on the prior difference 2D pixel coordinates from step 1 and the depth estimated in step 2 from the corresponding image, and combined with the PNP perspective transformation principle, the difference location information is transformed from the pixel coordinate system to the WGS84 world coordinate system.

[0038] 4. Point cloud cropping

[0039] Based on the difference position information in the WGS84 world coordinate system obtained in step 3, the point cloud of the corresponding region is cropped out.

[0040] 5. Point cloud projection

[0041] Based on the point cloud cropped in step 4, the position coordinates of the differences are used as the projection center. The point cloud data is rotated using the information of the nearest trajectory point pos, and the rotation value of the point cloud data is set to the road direction as due north (so that erroneous results can be eliminated according to the orientation during the subsequent vectorization process).

[0042] 6. Segmentation of projection diagram

[0043] Using deep learning technology, an image segmentation model is trained, and the model is used to segment the pixel coordinates of corresponding elements.

[0044] 7. Vectorization

[0045] Based on the pixel coordinates of the elements segmented in step 6, the contours are extracted. The contours are then calculated back to the WGS84 coordinate system based on the projection information. Finally, the shape points are vectorized according to the element creation standards.

[0046] 8. Manual verification

[0047] This step is a quality control process, verifying the vectorized results of the features extracted by the automated update to ensure that all results fully comply with the standards.

[0048] 9. Results are entered into the database.

[0049] Finally, the vectorization results from step 8 are written to the database to complete the output.

[0050] Example 2

[0051] refer to Figure 3 In a second aspect, the present invention provides a high-precision map single-element update device 1 based on a non-new road scene, comprising: an acquisition module 11, configured to acquire prior difference 2D pixel coordinates of a single target element in the high-precision map to be updated, one or more trajectory-associated images, pose information of the trajectory, and point cloud data; a first estimation module 12, configured to estimate the depth of the image containing difference location information from the one or more trajectory-associated images based on a depth estimation model; a second estimation module 13, configured to estimate the difference location information in the world coordinate system based on the depth of the image containing difference information and the prior difference 2D pixel coordinates; a segmentation module 14, configured to project and segment the point cloud corresponding to the difference location information based on the difference location information and the pose information of the trajectory to obtain the pixel coordinates of one or more pixels of the target element; and a reverse calculation module 15, configured to reverse calculate the pixel coordinates of each pixel to be updated into the world coordinate system and vectorize the corresponding points in the world coordinate system.

[0052] In some embodiments of the present invention, the segmentation module 14 includes: a projection unit, used to project the point cloud corresponding to the difference position information based on the difference position information and the pose information of the trajectory to obtain a projected image; and a segmentation unit, used to segment the projected image using a deep learning-based image segmentation model to obtain the pixel coordinates of one or more pixels of the target element.

[0053] Example 3

[0054] refer to Figure 4 In a third aspect, the present invention provides an electronic device comprising: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the high-precision map single-element update method based on non-new road scenarios of the first aspect of the present invention.

[0055] Electronic device 500 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 502 or a program loaded from storage device 508 into random access memory (RAM) 503. RAM 503 also stores various programs and data required for the operation of electronic device 500. The processing unit 501, ROM 502, and RAM 503 are interconnected via bus 504. Input / output (I / O) interface 505 is also connected to bus 504.

[0056] Typically, the following devices can be connected to I / O interface 505: input devices 506 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 507 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 508 including, for example, hard disks; and communication devices 509. Communication device 509 allows electronic device 500 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 4 An electronic device 500 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively. Figure 4 Each box shown can represent a device or multiple devices as needed.

[0057] Specifically, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 509, or installed from a storage device 508, or installed from a ROM 502. When the computer program is executed by a processing device 501, it performs the functions defined in the methods of embodiments of this disclosure. It should be noted that the computer-readable medium described in embodiments of this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In embodiments of this disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In embodiments of this disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0058] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable medium carries one or more computer programs, which, when executed by the electronic device, cause the electronic device to:

[0059] Computer program code for performing the operations of embodiments of this disclosure can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages—such as Java, Smalltalk, C++, and Python—and conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be entirely computed by the user.

[0060] Execute on the machine, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server.

[0061] Execution is performed on the above. In cases involving remote computers, the remote computer can be connected to the user's computer through any type of network—including a local area network (LAN) or a wide area network (WAN)—or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0062] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, the flowcharts or...

[0063] Each box in the block diagram can represent a module, segment, or portion of code, which contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the boxes may also be...

[0064] The actions may occur in a different order than those shown in the accompanying drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved.

[0065] It is determined by the specific circumstances. It should be noted that each block in the block diagram and / or flowchart, as well as combinations of blocks in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0066] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for updating single-feature maps of high-precision maps based on non-newly added road scenarios, characterized in that, include: Obtain the prior difference 2D pixel coordinates of a single target feature in the high-precision map to be updated, one or more trajectory-related images, trajectory pose information, and point cloud data; Based on the depth estimation model, the depth of the image containing the difference location information is estimated from one or more trajectory-related images; Based on the depth of the image containing the difference location information and the prior difference 2D pixel coordinates, estimate the difference location information in the world coordinate system; Based on the difference location information and the pose information of the trajectory, the point cloud corresponding to the difference location information is projected and segmented to obtain the pixel coordinates of one or more pixels of the target element. The pixel coordinates of each pixel of the target element are calculated back to the world coordinate system, and the corresponding points in the world coordinate system are vectorized.

2. The method for updating single-feature high-precision maps based on non-newly added road scenarios according to claim 1, characterized in that, The method of estimating the depth of an image containing differential location information from one or more trajectory-related images based on a depth estimation model includes: Based on a monocular depth estimation model, the depth of an image containing differential location information is estimated from one or more trajectory-related images.

3. The method for updating single-feature high-precision maps based on non-newly added road scenarios according to claim 1, characterized in that, The step of estimating the difference location information in the world coordinate system based on the depth of the image containing the difference location information and the prior difference 2D pixel coordinates includes: Based on the depth of the image containing the difference location information and the prior difference 2D pixel coordinates, the difference information is transformed from the pixel coordinate system to the WGS84 world coordinate system using the PNP perspective principle.

4. The method for updating single-feature high-precision maps based on non-newly added road scenarios according to claim 1, characterized in that, The step of projecting and segmenting the point cloud corresponding to the difference location information based on the pose information of the trajectory to obtain the pixel coordinates of one or more pixels of the target element includes: Based on the difference location information and the pose information of the trajectory, the point cloud corresponding to the difference location information is projected to obtain a projected image; The projected image is segmented using a deep learning-based image segmentation model to obtain the pixel coordinates of one or more pixels of the target element.

5. The method for updating single-feature high-precision maps based on non-newly added road scenarios according to claim 4, characterized in that, The step of projecting the point cloud corresponding to the difference position information based on the pose information of the trajectory to obtain the projected image includes: The coordinates of the difference positions are used as the projection center. The point cloud data is rotated using the pos information of the trajectory point closest to the projection center, and the point cloud data is rotated to the north direction of the road. The rotated point cloud is projected onto the road direction to obtain a projected image.

6. The method for updating single-feature high-precision maps based on non-newly added road scenarios according to any one of claims 1 to 5, characterized in that, The step of calculating the pixel coordinates of each pixel of the target element back to the world coordinate system and vectorizing the corresponding points in the world coordinate system includes: Extract the contour from the pixel coordinates of one or more pixels of the target feature; The pixel coordinates of each pixel of the target feature are calculated back to the WGS84 coordinate system, and then the outline and its corresponding pixels are vectorized according to the preset feature production standards.

7. A high-precision map single-element update device based on non-new road scenarios, characterized in that, include: The acquisition module is used to acquire the prior difference 2D pixel coordinates of a single target feature in the high-precision map to be updated, one or more trajectory-related images, the pose information of the trajectory, and point cloud data. The first estimation module is used to estimate the depth of the image containing the difference location information from one or more trajectory-related images based on the depth estimation model. The second estimation module is used to estimate the difference location information in the world coordinate system based on the depth of the image containing the difference location information and the prior difference 2D pixel coordinates. The segmentation module is used to project and segment the point cloud corresponding to the difference location information based on the pose information of the difference location information and the trajectory, so as to obtain the pixel coordinates of one or more pixels of the target element; The inverse calculation module is used to inversely calculate the pixel coordinates of each pixel of the target feature into the world coordinate system, and vectorize the corresponding points in the world coordinate system.

8. The high-precision map single-element update device based on non-newly added road scenarios according to claim 7, characterized in that, The segmentation module includes: The projection unit is used to project the point cloud corresponding to the difference position information based on the pose information of the difference position information and the trajectory to obtain a projected image; The segmentation unit is used to segment the projected image using a deep learning-based image segmentation model and obtain the pixel coordinates of one or more pixels of the target element.

9. An electronic device, comprising: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the high-precision map single-feature update method based on non-new road scenes as described in any one of claims 1 to 6.

10. A computer-readable medium having a computer program stored thereon, wherein, When the computer program is executed by the processor, it implements the high-precision map single-feature update method based on non-new road scenarios as described in any one of claims 1 to 6.

Images