Mapping method and apparatus, electronic device, and storage medium

Abstract

Embodiments of the present disclosure disclose a mapping method and device, electronic equipment and storage medium. The method comprises: determining a laser radar odometry pose based on a plurality of sensor data; determining a center coordinate of a target road mark according to the laser radar odometry pose; determining a odometry constraint condition according to the laser radar odometry pose; optimizing the laser radar odometry pose associated with the target road mark according to at least the laser radar odometry pose associated with the target road mark, the center coordinate of the target road mark and the odometry constraint condition, to obtain an optimized laser radar odometry pose associated with the target road mark; and obtaining a high-precision map based on the optimized laser radar odometry pose associated with the target road mark. The present disclosure improves the precision and consistency of laser mapping, especially for long-distance GPS data missing or indoor large-scale closed-loop application scenarios.

Description

Technical Field

[0001] This disclosure relates to the field of autonomous driving technology, and in particular to a mapping method, apparatus, electronic device, and storage medium. Background Technology

[0002] High-precision mapping technology is one of the core technologies in the field of autonomous driving. Currently, the mainstream mapping algorithms mainly adopt the method of fusing LiDAR data with high-precision GPS (Global Positioning System) data.

[0003] Among them, the lidar maintains an incremental lidar odometry based on SLAM (Simultaneous Localization and Mapping) technology, but this incremental lidar odometry will have accumulated errors, which will lead to a decrease in mapping accuracy. Summary of the Invention

[0004] To address or at least partially address the aforementioned technical problems, this disclosure provides a mapping method, apparatus, electronic device, and storage medium. By using semantic information-assisted mapping, the accuracy and consistency of mapping are improved, especially for application scenarios with long-distance missing GPS data, where mapping accuracy can be significantly enhanced.

[0005] In a first aspect, embodiments of this disclosure provide a mapping method, the method comprising:

[0006] The pose of the lidar odometer is determined based on data from multiple sensors.

[0007] The center coordinates of the target road sign can be determined based on the pose of the lidar odometer and the original point cloud of the vehicle-mounted lidar, or based on the pose of the lidar odometer, the original point cloud of the vehicle-mounted lidar, and the image captured by the vehicle-mounted camera.

[0008] Odometer constraints are constructed based on the lidar odometer pose.

[0009] The LiDAR odometry pose associated with the target landmark is optimized based on at least the LiDAR odometry pose associated with the target landmark, the center coordinates of the target landmark, and the odometry constraints, to obtain the optimized LiDAR odometry pose associated with the target landmark.

[0010] A high-precision map is obtained based on the optimized LiDAR odometry pose associated with the target landmarks.

[0011] Secondly, embodiments of this disclosure also provide a mapping apparatus, the apparatus comprising:

[0012] The first determining module is used to determine the pose of the lidar odometer based on data from multiple sensors.

[0013] The second determining module is used to determine the center coordinates of the target road sign based on the pose of the lidar odometer and the original point cloud of the vehicle-mounted lidar, or to determine the center coordinates of the target road sign based on the pose of the lidar odometer, the original point cloud of the vehicle-mounted lidar, and the image acquired by the vehicle-mounted camera device.

[0014] The construction module is used to construct odometer constraints based on the pose of the lidar odometer.

[0015] The first optimization module is used to optimize the lidar odometry pose associated with the target road sign based at least on the lidar odometry pose associated with the target road sign, the center coordinates of the target road sign, and the odometry constraint conditions, so as to obtain the optimized lidar odometry pose associated with the target road sign.

[0016] The mapping module is used to obtain a high-precision map based on the optimized LiDAR odometry pose associated with the target landmarks.

[0017] Thirdly, embodiments of this disclosure also provide an electronic device, the electronic device comprising: one or more processors; a storage device for storing one or more programs; and when the one or more programs are executed by the one or more processors, causing the one or more processors to implement the mapping method as described above.

[0018] Fourthly, embodiments of this disclosure also provide a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the mapping method described above.

[0019] The mapping method provided in this disclosure is a semantic information-assisted laser mapping scheme. This scheme adds the observation of target landmarks and the constraint association between target landmarks and lidar odometers in the map optimization. In this way, when eliminating accumulated errors in application scenarios with missing GPS data over long distances or in closed-loop detection scenarios, the consistency between target landmarks and lidar odometers can be increased, thereby improving the mapping accuracy. Attached Figure Description

[0020] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0021] Figure 1This is a flowchart of a mapping method according to an embodiment of this disclosure;

[0022] Figure 2 This is a schematic diagram of an architecture for optimizing the pose of a lidar odometer in an embodiment of this disclosure;

[0023] Figure 3 This is a schematic diagram of the structure of a mapping device according to an embodiment of the present disclosure;

[0024] Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of this disclosure. Detailed Implementation

[0025] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0026] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0027] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0028] Currently, commonly used autonomous driving mapping algorithms primarily employ a fusion of LiDAR and high-precision GPS data. The point cloud generated by LiDAR scanning is used to maintain an incremental LiDAR odometry system via SLAM (Simultaneous Localization and Mapping). This odometry system outputs the vehicle's real-time odometry pose, but this pose contains accumulated errors. Therefore, when the current frame's LiDAR odometry pose corresponds to observed high-precision GPS data or a loop closure detection is successful, the current frame's odometry pose is optimized to eliminate the accumulated errors. These accumulated errors are then propagated forward in an evenly distributed manner to previous frames' LiDAR odometry poses, achieving overall correction of the LiDAR odometry pose. This method assumes that the accumulated error of the incremental LiDAR odometry pose grows linearly across the six degrees of freedom of the pose. If the accumulated error exhibits linear growth (e.g., mapping along a straight line), applying the error-sharing method can effectively eliminate the accumulated error.

[0029] However, in practical autonomous driving applications, in long-distance scenarios without GPS data (such as tunnels) or large-scale indoor closed-loop scenarios, the cumulative error of LiDAR odometry pose is difficult to satisfy the assumption of linear growth. For example, the cumulative error of angle at sharp turns is larger than that in straight-line driving. This leads to some pose overcorrection and some pose undercorrection when distributing the error during forward propagation, making it difficult to accurately correct the LiDAR odometry pose before the current frame, which in turn leads to a decrease in the accuracy of the constructed map.

[0030] To address the aforementioned issues, this disclosure provides a semantic information-assisted laser mapping scheme. This scheme adds the observation of target landmarks and the constraint association between target landmarks and lidar odometers during map optimization. In application scenarios with missing GPS data over long distances or in closed-loop detection scenarios to eliminate accumulated errors, it can increase the consistency between target landmarks and lidar odometers, thereby improving mapping accuracy.

[0031] Figure 1 This is a flowchart illustrating a mapping method according to an embodiment of this disclosure. The method can be executed by a mapping device, which can be implemented in software and / or hardware, and can be configured in an electronic device. Figure 1 As shown, the method may specifically include the following steps:

[0032] Step 110: Determine the pose of the lidar odometer based on data from multiple sensors.

[0033] For example, the pose of the lidar odometry system is determined using a specific SLAM algorithm based on point clouds from lidar scanning, images acquired by an onboard camera, data detected by an inertial measurement unit, and data from wheel speedometers. The specific SLAM algorithm could be, for example, LOAM, ORB-SLAM, or Cartographer.

[0034] Step 120: Determine the center coordinates of the target road sign based on the pose of the lidar odometer and the original point cloud of the vehicle-mounted lidar, or determine the center coordinates of the target road sign based on the pose of the lidar odometer, the original point cloud of the vehicle-mounted lidar, and the image acquired by the vehicle-mounted camera.

[0035] Among them, target road signs refer to specific road signs scanned by vehicle-mounted LiDAR or captured by vehicle-mounted cameras, such as streetlights, utility poles, or lane lines.

[0036] Optionally, if the target landmark is a columnar object, such as a street lamp or utility pole, its center coordinates can be determined as follows:

[0037] For a single frame of raw point cloud, the ground point cloud is removed based on the height information to obtain the target point cloud after removing the ground point cloud; the target point cloud is converted to the global coordinate system based on the LiDAR odometry pose to generate a global point cloud map; clustering operations are performed on the point clouds in the global point cloud map to obtain multiple point cloud clusters; the center coordinates of the target landmark are determined based on the multiple point cloud clusters.

[0038] Determining the center coordinates of the target landmark based on the plurality of point cloud clusters includes:

[0039] From the plurality of point cloud clusters, a target point cloud cluster conforming to a preset shape is determined; the global coordinates of the center of the target point cloud cluster are determined; and the global coordinates of the center of the target point cloud cluster are determined as the center coordinates of the target landmark.

[0040] If the target road sign is located on the ground, such as an arrow or lane line, its center coordinates can be determined as follows:

[0041] Semantic segmentation is performed on a single-frame image to obtain the pixels of the target road signs included in the single-frame image. Pixels of different target road signs are distinguished by different colors, such as white for arrow pixels and yellow for lane line pixels. The point cloud closest to the timestamp of the single-frame image is converted to the coordinate system of the single-frame image to obtain a semantic single-frame point cloud. The semantic single-frame point cloud is marked with the color category information (e.g., white, yellow) of the target road sign pixels. The semantic single-frame point cloud is converted to a global coordinate system according to the LiDAR odometry pose to generate a semantic point cloud map. A color-based clustering algorithm is used to cluster the point cloud data in the semantic point cloud map to obtain multiple semantic point cloud clusters. The global coordinates of the center of each semantic point cloud cluster are determined as the center coordinates of the target road sign.

[0042] Step 130: Construct odometer constraints based on the LiDAR odometer pose.

[0043] For example, constructing odometry constraints based on the lidar odometry pose includes:

[0044] The odometer constraint conditions are constructed based on the lidar odometer pose, the lidar odometer pose increment, and the third preset information matrix associated with the lidar odometer.

[0045] Step 140: Optimize the lidar odometry pose associated with the target landmark based at least on the lidar odometry pose associated with the target landmark, the center coordinates of the target landmark, and the odometry constraints, to obtain the optimized lidar odometry pose associated with the target landmark.

[0046] The lidar odometry pose associated with the target road sign refers to the pose at which the target road sign can be observed. Specifically, in this pose, the vehicle's onboard lidar can scan the target road sign, or the onboard camera can capture an image of the target road sign. (See reference...) Figure 2 The diagram illustrates an architecture for optimizing the pose of a LiDAR odometer. Labels 210a and 210b represent the LiDAR odometer pose output over time, and labels 220a and 220b represent target landmarks. Specifically, when the target landmark is the one represented by label 220a, the associated LiDAR odometer pose is the one represented by label 210a; when the target landmark is the one represented by label 220b, the associated LiDAR odometer pose is the one represented by label 210b.

[0047] There is a relative positional relationship between the target road sign and the vehicle. This relative positional relationship can be determined based on point clouds and / or images. This relative positional relationship can be used as a constraint condition based on the target road sign to correct the vehicle pose (i.e., the LiDAR odometry pose).

[0048] Optionally, the method further includes: optimizing all the lidar odometry poses based on the optimized lidar odometry pose associated with the target landmark, to obtain all the optimized lidar odometry poses.

[0049] Alternatively, based on the LiDAR odometry pose associated with the target landmark, the center coordinates of the target landmark, and the odometry constraints, not only can the LiDAR odometry pose associated with the target landmark be optimized, but also the poses of all LiDAR odometry outputs can be optimized.

[0050] For example, under odometry constraints, the odometry poses of all LiDAR systems can be optimized based on the optimized target landmark-associated ODAR poses to obtain optimized ODAR poses for all systems. (Reference) Figure 2 As shown, under the conditions of the LiDAR odometry pose 210a associated with the target landmark 220a, the center coordinates of the target landmark 220a, and the odometry constraint 211, the optimized LiDAR odometry pose 210a associated with the target landmark is obtained. Then, under the condition of the odometry constraint 211, the LiDAR odometry pose 210b is optimized based on the optimized LiDAR odometry pose 210a associated with the target landmark (i.e., propagated backward based on the optimized LiDAR odometry pose 210a associated with the target landmark), and so on, to obtain all the optimized LiDAR odometry poses.

[0051] Optionally, in some embodiments, when a vehicle passes through an obstructed road section, such as an underground tunnel, the vehicle is usually unable to obtain high-precision GPS data. In order to ensure the mapping accuracy of the road section, the pose of the lidar odometer in the road section can be corrected by referring to the position of the target road sign in the road section.

[0052] Furthermore, in road sections where vehicles can normally acquire high-precision GPS data, the position of the LiDAR odometer can be corrected by referring to the position of the target road signs. By increasing the constraint of the vehicle's position on the target road signs, the accuracy of mapping can be improved.

[0053] For example, optimizing the lidar odometry pose associated with the target landmark based at least on the lidar odometry pose associated with the target landmark, the center coordinates of the target landmark, and the odometry constraints to obtain the optimized lidar odometry pose associated with the target landmark includes the following steps:

[0054] 131. Construct a first constraint condition based on the lidar odometry pose associated with the target landmark.

[0055] Specifically, the following first loss function is constructed based on the lidar odometry pose associated with the target landmark:

[0056]

[0057] Among them, F landmark1 Let l represent the first loss function, k represent the number of target landmarks, and l represent the number of target landmarks. p N represents the center coordinates of the p-th target landmark in the global coordinate system. p T represents the frame number of the lidar odometry pose. g T represents the pose of the lidar odometry in the g-th frame. h Ω represents the pose of the lidar odometry in the h-th frame. l1 Represents the first preset information matrix, △T gh This represents the increment between the lidar odometry pose in the g-th frame and the lidar odometry pose in the h-th frame;

[0058] The first loss function is determined as the first constraint condition.

[0059] 132. And / or, construct a second constraint condition based on the lidar odometry pose, the center coordinates of the target landmark, and the coordinates of the center coordinates in the local coordinate system of the point cloud corresponding to the lidar odometry pose.

[0060] Specifically, based on the lidar odometry pose, the center coordinates of the target landmark, and the coordinates of the center coordinates in the local coordinate system of the point cloud frame corresponding to the lidar odometry pose, the following second loss function is constructed:

[0061]

[0062] Among them, F landmark2 Let k represent the second loss function, and l represent the number of target landmarks. p N represents the center coordinates of the p-th target landmark in the global coordinate system. p T represents the frame number of the lidar odometry pose. g Ω represents the pose of the lidar odometry in the g-th frame. l2Represents the second preset information matrix, l p local Indicates the center coordinates l p The coordinates of the point cloud frame corresponding to the pose of the lidar odometry in the g-th frame;

[0063] The second loss function is defined as the second constraint condition.

[0064] 133. Optimize the lidar odometry pose associated with the target landmark based at least on the first constraint and / or the second constraint and the odometry constraint to obtain the optimized lidar odometry pose associated with the target landmark.

[0065] The process of constructing odometer constraints based on the lidar odometer pose includes:

[0066] The odometer constraint conditions are constructed based on the lidar odometer pose, the lidar odometer pose increment, and the third preset information matrix associated with the lidar odometer.

[0067] Specifically, based on the lidar odometry pose, the lidar odometry pose increment, and the third preset information matrix associated with the lidar odometry, a third loss function is constructed as follows, and this third loss function is determined as the odometry constraint condition:

[0068]

[0069] Among them, F odom Let T represent the third loss function, n represent the number of frames for the lidar odometry pose, and T represent the third loss function. Lj Indicates the pose of the lidar odometry in the j-th frame, T Lj+1 Indicates the LiDAR odometry pose and ΔT in the (j+1)th frame. Lj Ω represents the lidar odometry pose increment in the j-th frame. Lj This represents the third preset information matrix.

[0070] Optionally, the lidar odometry pose associated with the target landmark is optimized based at least on the first constraint and / or the second constraint and the odometry constraint to obtain the optimized lidar odometry pose associated with the target landmark, including:

[0071] A total loss function is generated based on the first loss function and / or the second loss function, and the third loss function; the total loss function is solved, and the lidar odometry pose corresponding to the minimum value of the total loss function is determined as the optimized lidar odometry pose associated with the target landmark.

[0072] Furthermore, in some embodiments, the method further includes:

[0073] The global positioning system (GPS) constraints are constructed based on the coordinate values ​​of the GPS pose, the coordinate values ​​of the lidar odometry pose, and the fourth preset information matrix associated with the GPS.

[0074] The pose of the lidar odometry associated with the target landmark is optimized based on the first constraint, the second constraint, the odometry constraint, and the global positioning system constraint to obtain the optimized pose of the lidar odometry associated with the target landmark.

[0075] Specifically, based on the coordinates of the GPS pose, the coordinates of the LiDAR odometry pose, and the fourth preset information matrix associated with the GPS, a fourth loss function is constructed, and this fourth loss function is determined as the GPS constraint condition:

[0076]

[0077] Among them, F gps Let m represent the fourth loss function, m represent the number of frames corresponding to the LiDAR pose in the GPS pose, and t represent the fourth loss function. Li t represents the coordinates of the LiDAR pose corresponding to the GPS pose in the i-th frame. Gi Ω represents the coordinates of the GPS pose in the i-th frame. gj This represents the fourth preset information matrix.

[0078] A total loss function is generated based on the first loss function, the second loss function, the third loss function, and the fourth loss function; the total loss function is solved, and the lidar odometry pose corresponding to the minimum value of the total loss function is determined as the optimized lidar odometry pose associated with the target landmark.

[0079] Specifically, the total loss function F = F landmark1 +F landmark2 +F odom +F gps The problem of solving the total loss function F is a nonlinear least squares problem. The goal is to minimize F by optimizing the poses of all LiDAR odometry units and the center coordinates of target landmarks. In practical applications, the LM (Levenberg-Marquardt) algorithm of the GTSM optimizer or the Gauss-Newton method can be used to solve the problem.

[0080] Step 150: Obtain a high-precision map based on the optimized LiDAR odometry pose associated with the target landmark.

[0081] General overview, for reference, etc. Figure 2 The diagram illustrates an architecture for optimizing the pose of a LiDAR odometry system. Labels 210a and 210b represent the odometry pose output by the odometry over time; labels 220a and 220b represent target landmarks; label 230 represents GPS data; label 211 represents the constraint imposed by the odometry on the odometry pose; label 221 represents the constraint imposed by the target landmarks on the odometry pose; and label 231 represents the constraint imposed by the GPS data on the odometry pose. In summary, by optimizing the odometry pose through multiple constraints, a highly accurate optimized odometry pose is obtained. A high-precision map is then built based on the optimized odometry pose.

[0082] The mapping method provided in this embodiment can be applied to mapping of local road segments or to mapping of the entire road network.

[0083] The mapping method provided in this embodiment proposes a semantically information-assisted laser mapping method by combining LiDAR odometry. This method mainly includes semantic extraction of target landmarks, data association between target landmarks and LiDAR odometry poses, and construction of a loss function for target landmarks in pose graph optimization. Using this method can improve the accuracy and consistency of laser mapping, especially for applications with long-distance missing GPS data or large-scale indoor closed-loop systems, significantly improving the quality and accuracy of laser mapping.

[0084] Figure 3 This is a schematic diagram of the structure of a mapping device according to an embodiment of this disclosure. Figure 3 As shown: The device includes: a first determining module 310, a second determining module 320, a construction module 330, a first optimization module 340, and a mapping module 350.

[0085] The system comprises the following modules: a first determining module 310, used to determine the pose of a lidar odometer based on data from multiple sensors; a second determining module 320, used to determine the center coordinates of a target road sign based on the lidar odometer pose and the original point cloud of the vehicle-mounted lidar, or based on the lidar odometer pose, the original point cloud of the vehicle-mounted lidar, and an image acquired by a vehicle-mounted camera; a construction module 330, used to construct odometer constraints based on the lidar odometer pose; a first optimization module 340, used to optimize the lidar odometer pose associated with the target road sign based at least on the lidar odometer pose associated with the target road sign, the center coordinates of the target road sign, and the odometer constraints, to obtain an optimized lidar odometer pose associated with the target road sign; and a mapping module 350, used to obtain a high-precision map based on the optimized lidar odometer pose associated with the target road sign.

[0086] Optionally, a second optimization module is also included, which is used to optimize all the lidar odometry poses based on the optimized lidar odometry poses associated with the target landmarks, so as to obtain all the optimized lidar odometry poses.

[0087] Optionally, the first optimization module 340 includes: a construction unit, configured to construct a first constraint condition based on the lidar odometry pose associated with the target landmark; and / or, to construct a second constraint condition based on the lidar odometry pose, the center coordinates of the target landmark, and the coordinates of the center coordinates in the local coordinate system of the point cloud corresponding to the lidar odometry pose; and an optimization unit, configured to optimize the lidar odometry pose associated with the target landmark based at least on the first constraint condition and / or the second constraint condition and the odometry constraint condition, to obtain an optimized lidar odometry pose associated with the target landmark.

[0088] Optionally, the optimization unit includes: a construction subunit, configured to construct the odometer constraint conditions based on the lidar odometer pose, the lidar odometer pose increment, and a third preset information matrix associated with the lidar odometer; construct global positioning system constraint conditions based on the coordinate values ​​of the global positioning system pose, the coordinate values ​​of the lidar odometer pose, and the fourth preset information matrix associated with the global positioning system; and an optimization subunit, configured to optimize the lidar odometer pose associated with the target landmark based on the first constraint conditions, the second constraint conditions, the odometer constraint conditions, and the global positioning system constraint conditions, to obtain the optimized lidar odometer pose associated with the target landmark.

[0089] Optionally, the construction unit is specifically used to: construct the following first loss function based on the lidar odometry pose associated with the target landmark:

[0090]

[0091] Among them, F landmark1 Let l represent the first loss function, k represent the number of target landmarks, and l represent the number of target landmarks. p N represents the center coordinates of the p-th target landmark in the global coordinate system. p T represents the frame number of the lidar odometry pose. g T represents the pose of the lidar odometry in the g-th frame. h Ω represents the pose of the lidar odometry in the h-th frame. l1 Represents the first preset information matrix, △T gh This represents the increment between the lidar odometry pose in frame g and frame h.

[0092] Based on the lidar odometry pose, the center coordinates of the target landmark, and the coordinates of the center coordinates in the local coordinate system of the point cloud frame corresponding to the lidar odometry pose, the following second loss function is constructed:

[0093]

[0094] Among them, F landmark2 Let k represent the second loss function, and l represent the number of target landmarks. p N represents the center coordinates of the p-th target landmark in the global coordinate system. p T represents the frame number of the lidar odometry pose. g Ω represents the pose of the lidar odometry in the g-th frame. l2 Represents the second preset information matrix, l p local Indicates the center coordinates l p The coordinates of the point cloud frame corresponding to the pose of the lidar odometry in the g-th frame under local coordinates.

[0095] Optionally, the second determining module 320 includes: a filtering unit, used to remove ground point clouds from the original point cloud of a single frame based on height information to obtain a target point cloud after removing the ground point clouds; a first conversion unit, used to convert the target point cloud to a global coordinate system based on the LiDAR odometry pose to generate a global point cloud map; a first clustering unit, used to perform clustering operations on the point clouds in the global point cloud map to obtain multiple point cloud clusters; and a first determining unit, used to determine the center coordinates of the target landmark based on the multiple point cloud clusters.

[0096] Furthermore, the first determining unit is specifically used to: determine a target point cloud cluster that conforms to a preset shape from the plurality of point cloud clusters; determine the global coordinates of the center of the target point cloud cluster; and determine the global coordinates of the center of the target point cloud cluster as the center coordinates of the target landmark.

[0097] Furthermore, the second determining module 320 includes: a segmentation unit, used to perform semantic segmentation based on a single-frame image to obtain the pixels of the target road signs included in the single-frame image, wherein the pixels of different target road signs are distinguished by different colors; a second conversion unit, used to convert the point cloud closest to the timestamp of the single-frame image to the coordinate system of the single-frame image to obtain a semantic single-frame point cloud, wherein the semantic single-frame point cloud is marked with the color category information of the pixels of the target road signs; a third conversion unit, used to convert the semantic single-frame point cloud to a global coordinate system according to the LiDAR odometry pose to generate a semantic point cloud map; a second clustering unit, used to perform clustering operations on the point cloud data in the semantic point cloud map based on a color-based clustering algorithm to obtain multiple semantic point cloud clusters; and a second determining unit, used to determine the global coordinates of the center of each semantic point cloud cluster as the center coordinates of the target road sign.

[0098] The mapping apparatus provided in this embodiment can execute the steps in the mapping method provided in this embodiment, and has the execution steps and beneficial effects, which will not be described in detail here.

[0099] Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of this disclosure. See below for details. Figure 4 It shows a schematic diagram of a structure suitable for implementing the electronic device 500 in the embodiments of this disclosure. Figure 4 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0100] like Figure 4 As shown, the electronic device 500 may include a processing device (e.g., a central processing unit, a graphics processor, etc.) 501, which can perform various appropriate actions and processes to implement the methods of the embodiments described herein, based on a program stored in a read-only memory (ROM) 502 or a program loaded from a storage device 508 into a random access memory (RAM) 503. The RAM 503 also stores various programs and data required for the operation of the electronic device 500. The processing device 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0101] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts, thereby implementing the mapping method as described above. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 509, or installed from a storage device 508, or installed from a ROM 502. When the computer program is executed by the processing device 501, it performs the functions defined in the methods of embodiments of this disclosure.

[0102] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0103] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the aforementioned mapping method.

[0104] Optionally, when one or more of the above-described procedures are executed by the electronic device, the electronic device may also perform other steps described in the above embodiments.

[0105] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0106] Option 1: A mapping method, the method comprising:

[0107] The pose of the lidar odometer is determined based on data from multiple sensors.

[0108] The center coordinates of the target road sign can be determined based on the pose of the lidar odometer and the original point cloud of the vehicle-mounted lidar, or based on the pose of the lidar odometer, the original point cloud of the vehicle-mounted lidar, and the image captured by the vehicle-mounted camera.

[0109] Odometer constraints are constructed based on the lidar odometer pose.

[0110] The LiDAR odometry pose associated with the target landmark is optimized based on at least the LiDAR odometry pose associated with the target landmark, the center coordinates of the target landmark, and the odometry pose, to obtain the optimized LiDAR odometry pose associated with the target landmark.

[0111] A high-precision map is obtained based on the optimized LiDAR odometry pose associated with the target landmarks.

[0112] Option 2, the method described in Option 1, further includes:

[0113] Based on the optimized LiDAR odometry pose associated with the target landmark, all LiDAR odometry poses are optimized to obtain all optimized LiDAR odometry poses.

[0114] Solution 3: According to the method described in Solution 1, the step of optimizing the lidar odometry pose associated with the target landmark based at least on the lidar odometry pose associated with the target landmark, the center coordinates of the target landmark, and the odometry constraints to obtain the optimized lidar odometry pose associated with the target landmark includes:

[0115] The first constraint is constructed based on the lidar odometry pose associated with the target landmark;

[0116] And / or, construct a second constraint condition based on the lidar odometry pose, the center coordinates of the target landmark, and the coordinates of the center coordinates in the local coordinate system of the point cloud corresponding to the lidar odometry pose;

[0117] The pose of the lidar odometry associated with the target landmark is optimized based on at least the first constraint and / or the second constraint and the odometry constraint to obtain the optimized pose of the lidar odometry associated with the target landmark. Scheme 4: According to the method in Scheme 3, the step of constructing the first constraint based on the lidar odometry pose associated with the target landmark includes:

[0118] Based on the lidar odometry pose associated with the target landmark, the following first loss function is constructed:

[0119]

[0120] Among them, F landmark1 Let l represent the first loss function, k represent the number of target landmarks, and l represent the number of target landmarks. p N represents the center coordinates of the p-th target landmark in the global coordinate system. p T represents the frame number of the lidar odometry pose. g T represents the pose of the lidar odometry in the g-th frame. h Ω represents the pose of the lidar odometry in the h-th frame. l1 Represents the first preset information matrix, △T gh This represents the increment between the lidar odometry pose in frame g and frame h.

[0121] Option 5: According to the method described in Option 3, the step of constructing the second constraint condition based on the lidar odometry pose, the center coordinates of the target landmark, and the coordinates of the center coordinates in the local coordinate system of the point cloud frame corresponding to the lidar odometry pose includes:

[0122] Based on the lidar odometry pose, the center coordinates of the target landmark, and the coordinates of the center coordinates in the local coordinate system of the point cloud frame corresponding to the lidar odometry pose, the following second loss function is constructed:

[0123]

[0124] Among them, F landmark2 Let k represent the second loss function, and l represent the number of target landmarks. p N represents the center coordinates of the p-th target landmark in the global coordinate system. p T represents the frame number of the lidar odometry pose. g Ω represents the pose of the lidar odometry in the g-th frame. l2 Represents the second preset information matrix, l p local Indicates the center coordinates l p The coordinates of the point cloud frame corresponding to the pose of the lidar odometry in the g-th frame;

[0125] Solution 6: According to the method described in Solution 3, the step of optimizing the lidar odometry pose associated with the target landmark based at least on the first constraint and / or the second constraint and the odometry constraint to obtain the optimized lidar odometry pose associated with the target landmark includes:

[0126] The odometer constraint conditions are constructed based on the lidar odometry pose, the lidar odometry pose increment, and the third preset information matrix associated with the lidar odometry.

[0127] The global positioning system (GPS) constraints are constructed based on the coordinate values ​​of the GPS pose, the coordinate values ​​of the lidar odometry pose, and the fourth preset information matrix associated with the GPS.

[0128] The pose of the lidar odometry associated with the target landmark is optimized based on the first constraint, the second constraint, the odometry constraint, and the global positioning system constraint to obtain the optimized pose of the lidar odometry associated with the target landmark.

[0129] Option 7: The method described in any one of Options 1-6, wherein determining the center coordinates of the target road sign based on the pose of the lidar odometer and the original point cloud of the vehicle-mounted lidar includes:

[0130] For a single frame of raw point cloud, the ground point cloud is removed based on the height information to obtain the target point cloud after removing the ground point cloud.

[0131] Based on the pose of the lidar odometry, the target point cloud is converted to the global coordinate system to generate a global point cloud map.

[0132] Clustering operations are performed on the point clouds in the global point cloud map to obtain multiple point cloud clusters;

[0133] The center coordinates of the target landmark are determined based on the multiple point cloud clusters.

[0134] Option 8: According to the method described in Option 7, determining the center coordinates of the target landmark based on the plurality of point cloud clusters includes:

[0135] From the plurality of point cloud clusters, determine the target point cloud cluster that conforms to a preset shape;

[0136] Determine the global coordinates of the center of the target point cloud cluster;

[0137] The global coordinates of the center of the target point cloud cluster are determined as the center coordinates of the target landmark.

[0138] Option 9: The method described in any one of Options 1-6, wherein determining the center coordinates of the target road sign based on the pose of the lidar odometer, the original point cloud of the vehicle-mounted lidar, and the image acquired by the vehicle-mounted camera includes:

[0139] Semantic segmentation is performed based on a single frame image to obtain the pixels of the target road signs included in the single frame image, wherein pixels of different target road signs are distinguished by different colors;

[0140] The point cloud closest to the timestamp of the single frame image is converted into the coordinate system of the single frame image to obtain the semantic single frame point cloud, in which the color category information of the pixels of the target road sign is marked.

[0141] Based on the LiDAR odometry pose, the semantic single-frame point cloud is converted to the global coordinate system to generate a semantic point cloud map.

[0142] A color-based clustering algorithm is used to perform clustering operations on the point cloud data in the semantic point cloud map to obtain multiple semantic point cloud clusters.

[0143] The global coordinates of the center of each semantic point cloud cluster are determined as the center coordinates of the target landmark.

[0144] Option 10: A mapping device, comprising:

[0145] The first determining module is used to determine the pose of the lidar odometer based on data from multiple sensors.

[0146] The second determining module is used to determine the center coordinates of the target road sign based on the pose of the lidar odometer and the original point cloud of the vehicle-mounted lidar, or to determine the center coordinates of the target road sign based on the pose of the lidar odometer, the original point cloud of the vehicle-mounted lidar, and the image acquired by the vehicle-mounted camera device.

[0147] The construction module is used to construct odometer constraints based on the pose of the lidar odometer.

[0148] The first optimization module is used to optimize the lidar odometry pose associated with the target road sign based at least on the lidar odometry pose associated with the target road sign, the center coordinates of the target road sign, and the odometry constraint conditions, so as to obtain the optimized lidar odometry pose associated with the target road sign.

[0149] The mapping module is used to obtain a high-precision map based on the optimized LiDAR odometry pose associated with the target landmarks.

[0150] Option 11: An electronic device, the electronic device comprising:

[0151] One or more processors;

[0152] Storage device for storing one or more programs;

[0153] When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of schemes 1-9.

[0154] Option 12: A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as described in any one of Options 1-9.

[0155] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

Claims

1. A mapping method, characterized in that, The method includes: The pose of the lidar odometer is determined based on data from multiple sensors. The center coordinates of the target road sign can be determined based on the pose of the lidar odometer and the original point cloud of the vehicle-mounted lidar, or based on the pose of the lidar odometer, the original point cloud of the vehicle-mounted lidar, and the image captured by the vehicle-mounted camera. Odometer constraints are constructed based on the lidar odometer pose. The lidar odometry pose associated with the target landmark is optimized based at least on the pose of the lidar odometry associated with the target landmark, the center coordinates of the target landmark, and the odometry constraints, to obtain the optimized lidar odometry pose associated with the target landmark. The lidar odometry pose associated with the target landmark refers to the pose in which the target landmark can be observed. A high-precision map is obtained based on the optimized LiDAR odometry pose associated with the target landmarks; The step of optimizing the lidar odometry pose associated with the target landmark based at least on the lidar odometry pose associated with the target landmark, the center coordinates of the target landmark, and the odometry constraints to obtain the optimized lidar odometry pose associated with the target landmark includes: The first constraint is constructed based on the lidar odometry pose associated with the target landmark; The second constraint condition is constructed based on the pose of the lidar odometry, the center coordinates of the target landmark, and the coordinates of the center coordinates in the local coordinate system of the point cloud corresponding to the pose of the lidar odometry. The pose of the lidar odometry associated with the target landmark is optimized based on at least the first constraint, the second constraint, and the odometry constraint to obtain the optimized pose of the lidar odometry associated with the target landmark.

2. The method according to claim 1, characterized in that, Also includes: Based on the optimized LiDAR odometry pose associated with the target landmark, all LiDAR odometry poses are optimized to obtain all optimized LiDAR odometry poses.

3. The method according to claim 1, characterized in that, The step of constructing the first constraint condition based on the lidar odometry pose associated with the target landmark includes: Based on the lidar odometry pose associated with the target landmark, the following first loss function is constructed: Among them, F landmark1 Let N represent the first loss function, k represent the number of target landmarks, and N represent the number of target landmarks. p T represents the frame number of the lidar odometry pose. g T represents the pose of the lidar odometry in the g-th frame. h Ω represents the pose of the lidar odometry in the h-th frame. l1 Represents the first preset information matrix, △T gh This represents the increment between the lidar odometry pose in frame g and frame h.

4. The method according to claim 1, characterized in that, The method is based on the pose of the lidar odometer and the center coordinates of the target landmark (l) p ) and the coordinates of the center coordinates in the local coordinate system of the point cloud frame corresponding to the lidar odometry pose (l p local Construct the second constraint, including: Based on the pose of the lidar odometer and the center coordinates of the target landmark (l p ) and the coordinates of the center coordinates in the local coordinate system of the point cloud frame corresponding to the lidar odometry pose (l p local Construct the second loss function as follows: Among them, F landmark2 Let k represent the second loss function, and l represent the number of target landmarks. p N represents the center coordinates of the p-th target landmark in the global coordinate system. p T represents the frame number of the lidar odometry pose. g Ω represents the pose of the lidar odometry in the g-th frame. l2 Represents the second preset information matrix, l p local Indicates the center coordinates l p The coordinates of the point cloud frame corresponding to the pose of the lidar odometry in the g-th frame under local coordinates.

5. The method according to claim 1, characterized in that, The step of optimizing the lidar odometry pose associated with the target landmark based at least on the first constraint, the second constraint, and the odometry constraint to obtain the optimized lidar odometry pose associated with the target landmark includes: The odometer constraint conditions are constructed based on the lidar odometry pose, the lidar odometry pose increment, and the third preset information matrix associated with the lidar odometry. The constraints of the Global Positioning System (GPS) are constructed based on the coordinate values ​​of the GPS pose, the coordinate values ​​of the LiDAR odometry pose, and the fourth preset information matrix associated with the GPS. The pose of the lidar odometry associated with the target landmark is optimized based on the first constraint, the second constraint, the odometry constraint, and the global positioning system constraint to obtain the optimized pose of the lidar odometry associated with the target landmark.

6. The method according to any one of claims 1-5, characterized in that, Determining the center coordinates of the target road sign based on the pose of the lidar odometer and the original point cloud of the vehicle-mounted lidar includes: For a single frame of raw point cloud, the ground point cloud is removed based on the height information to obtain the target point cloud after removing the ground point cloud. Based on the pose of the lidar odometry, the target point cloud is converted to the global coordinate system to generate a global point cloud map. Clustering operations are performed on the point clouds in the global point cloud map to obtain multiple point cloud clusters; The center coordinates of the target landmark are determined based on the multiple point cloud clusters.

7. A mapping device, characterized in that, include: The first determining module is used to determine the pose of the lidar odometer based on data from multiple sensors. The second determining module is used to determine the center coordinates of the target road sign based on the pose of the lidar odometer and the original point cloud of the vehicle-mounted lidar, or to determine the center coordinates of the target road sign based on the pose of the lidar odometer, the original point cloud of the vehicle-mounted lidar, and the image acquired by the vehicle-mounted camera device. The construction module is used to construct odometer constraints based on the pose of the lidar odometer. The first optimization module is used to optimize the lidar odometry pose associated with the target landmark based at least on the lidar odometry pose associated with the target landmark, the center coordinates of the target landmark, and the odometry constraints, to obtain the optimized lidar odometry pose associated with the target landmark, wherein the lidar odometry pose associated with the target landmark refers to the pose in which the target landmark can be observed. The mapping module is used to obtain a high-precision map based on the optimized LiDAR odometry pose associated with the target landmarks; The step of optimizing the lidar odometry pose associated with the target landmark based at least on the lidar odometry pose associated with the target landmark, the center coordinates of the target landmark, and the odometry constraints to obtain the optimized lidar odometry pose associated with the target landmark includes: The first constraint is constructed based on the lidar odometry pose associated with the target landmark; The second constraint condition is constructed based on the pose of the lidar odometry, the center coordinates of the target landmark, and the coordinates of the center coordinates in the local coordinate system of the point cloud corresponding to the pose of the lidar odometry. The pose of the lidar odometry associated with the target landmark is optimized based on at least the first constraint, the second constraint, and the odometry constraint to obtain the optimized pose of the lidar odometry associated with the target landmark.

8. An electronic device, characterized in that, The electronic device includes: One or more processors; Storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-6.

Images