A method and apparatus for processing high-precision map data

Abstract

This application discloses a high-precision map data processing method and apparatus. The method determines the overlapping area of ​​a first point cloud map segment and a second point cloud map segment, then associates the first point cloud belonging to the first point cloud map segment within the overlapping area with the matching second point cloud belonging to the second point cloud map segment. Based on the associated first and second point clouds, a precise first coordinate transformation matrix between the first and second point cloud map segments can be determined. Based on the precise first coordinate transformation matrix, a second coordinate transformation matrix for transforming the first point cloud map segment to the world coordinate system, and a third coordinate transformation matrix for transforming the second point cloud map segment to the world coordinate system, the second and third coordinate transformation matrices are optimized. Based on this, the first and second point cloud map segments are aggregated, eliminating ghosting that occurs during the aggregation process and obtaining a high-precision global map.

Description

Technical Field

[0001] This application relates to the field of map data processing technology, and in particular to a high-precision map data processing method and apparatus. Background Technology

[0002] Maps play a vital role in people's lives and travel. Before a map is created, various tools are needed to collect sub-maps, such as Inertial Measurement Unit (IMU), Global Positioning System (GPS), Real Time Kinematic (RTK), or Light Detection and Ranging (LIDAR) (e.g., LiDAR). The collected sub-maps are then fused together to form a global map.

[0003] In the process of LiDAR data acquisition and mapping, multiple sub-maps are often pre-generated due to factors such as personnel work schedules, computing resources, and operating environment. These sub-maps are then globally aggregated and stitched together to form a global map. Currently, stitching multiple sub-maps together to form a global map often results in significant errors, such as ghosting. Therefore, improving the accuracy of the global map warrants further research. Summary of the Invention

[0004] To address the aforementioned technical problems, this disclosure is proposed. Embodiments of this disclosure provide a high-precision map data processing method and apparatus.

[0005] According to one aspect of this disclosure, a high-precision map data processing method is provided, comprising:

[0006] Determine the overlapping area of ​​the first point cloud map segment and the second point cloud map segment, and the first point cloud map segment and the second point cloud map segment belong to the first point cloud map;

[0007] Determine the first and second point clouds of the overlapping area. The first point cloud belongs to the first point cloud map segment, and the second point cloud belongs to the second point cloud map segment.

[0008] Connect the first point cloud and the second point cloud;

[0009] Determine the first coordinate transformation matrix between the first point cloud map segment and the second point cloud map segment based on the first point cloud and the second point cloud;

[0010] Based on the first coordinate transformation matrix, the initial second coordinate transformation matrix, and the initial third coordinate transformation matrix, the target second coordinate transformation matrix and the target third coordinate transformation matrix are determined. The second coordinate transformation matrix is ​​used to transform the first point cloud map fragment to the world coordinate system, and the third coordinate transformation matrix is ​​used to transform the second point cloud map fragment to the world coordinate system.

[0011] The first and second point cloud map fragments are aggregated based on the target's second and third coordinate transformation matrices to obtain the global map.

[0012] According to another aspect of the present disclosure, a high-precision map data processing apparatus is provided, comprising:

[0013] The overlapping area determination module is used to determine the overlapping area of ​​the first point cloud map segment and the second point cloud map segment, wherein the first point cloud map segment and the second point cloud map segment belong to the first point cloud map.

[0014] The point cloud association module is used to determine the first and second point clouds in the overlapping area. The first point cloud belongs to the first point cloud map segment, and the second point cloud belongs to the second point cloud map segment.

[0015] The point cloud association module is also used to associate the first point cloud with the second point cloud;

[0016] The coordinate transformation matrix determination module is used to determine the first coordinate transformation matrix between the first point cloud map segment and the second point cloud map segment based on the first point cloud and the second point cloud.

[0017] The coordinate transformation matrix determination module is also used to determine the target second coordinate transformation matrix and the target third coordinate transformation matrix based on the first coordinate transformation matrix, the initial second coordinate transformation matrix, and the initial third coordinate transformation matrix. The second coordinate transformation matrix is ​​used to transform the first point cloud map fragment to the world coordinate system, and the third coordinate transformation matrix is ​​used to transform the second point cloud map fragment to the world coordinate system.

[0018] The map aggregation module is used to aggregate the first point cloud map fragment and the second point cloud map fragment based on the target's second coordinate transformation matrix and the target's third coordinate transformation matrix to obtain a global map.

[0019] According to another aspect of the present disclosure, a computer-readable storage medium is provided, the storage medium storing a computer program for performing the high-precision map data processing method described in any of the above embodiments of the present disclosure.

[0020] According to another aspect of the present disclosure, an electronic device is provided, the electronic device comprising:

[0021] processor;

[0022] Memory used to store the processor's executable instructions;

[0023] The processor is configured to read the executable instructions from the memory and execute the instructions to implement the high-precision map data processing method described in any of the above embodiments of this disclosure.

[0024] This disclosure provides a high-precision map data processing method and apparatus. First, it associates the point clouds in the overlapping area between a first point cloud map segment and a second point cloud map segment to determine the coordinate transformation matrix between the two point cloud map segments. Then, based on the initial coordinate transformation matrices corresponding to each of the two point cloud map segments and the coordinate transformation matrix between the point cloud map segments, it obtains the target coordinate transformation matrix corresponding to each of the two point cloud map segments. Based on the target coordinate transformation matrices corresponding to the two point cloud map segments, the point cloud map segments are aggregated to obtain a global map. In this scheme, since the overlapping area represents the same real physical world region, by associating the point clouds of the first point cloud map segment located in the overlapping area with the point clouds of the matching second point cloud map segment located in the overlapping area, the coordinate transformation matrix between the point cloud map segments can be accurately obtained based on this association, thereby obtaining the accurate target coordinate transformation matrix corresponding to each point cloud map segment. This eliminates ghosting during the point cloud map segment aggregation process and improves the accuracy of the global map. Attached Figure Description

[0025] The above and other objects, features, and advantages of this application will become more apparent from the more detailed description of the embodiments of this application in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the embodiments of this application to explain this application and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0026] Figure 1 This is the scenario diagram to which this disclosure applies.

[0027] Figure 2 This is a flowchart illustrating a high-precision map data processing method provided in an exemplary embodiment of this disclosure.

[0028] Figure 3 This is a flowchart illustrating a method for constructing a first point cloud map provided in another exemplary embodiment of this disclosure.

[0029] Figure 4 This is a schematic flowchart of a method for associating point clouds provided in an exemplary embodiment of this disclosure.

[0030] Figure 5This is a schematic flowchart of an exemplary embodiment of the present disclosure of a method for determining overlapping regions.

[0031] Figure 6 This is a schematic flowchart of a method for determining a first coordinate transformation matrix provided in an exemplary embodiment of this disclosure.

[0032] Figure 7 This is a flowchart illustrating a high-precision map data processing method provided in another exemplary embodiment of this disclosure.

[0033] Figure 8 This is a structural diagram of a high-precision map data processing apparatus provided in an exemplary embodiment of the present disclosure.

[0034] Figure 9 This is a structural diagram of a high-precision map data processing apparatus provided in another exemplary embodiment of this disclosure.

[0035] Figure 10 This is a structural diagram of an electronic device provided in an exemplary embodiment of this application. Detailed Implementation

[0036] Hereinafter, exemplary embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present disclosure, and not all embodiments of the present disclosure, and it should be understood that the present disclosure is not limited to the exemplary embodiments described herein.

[0037] It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of this disclosure.

[0038] Those skilled in the art will understand that the terms "first," "second," etc., in the embodiments of this disclosure are only used to distinguish different steps, devices, or modules, and do not represent any specific technical meaning, nor do they indicate a necessary logical order between them.

[0039] It should also be understood that in the embodiments disclosed herein, "a plurality of" may refer to two or more, and "at least one" may refer to one, two or more.

[0040] It should also be understood that any component, data or structure mentioned in the embodiments of this disclosure can generally be understood as one or more unless expressly defined or given to the contrary in the context.

[0041] Furthermore, the term "and / or" in this disclosure is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this disclosure generally indicates that the preceding and following related objects have an "or" relationship.

[0042] It should also be understood that the description of the various embodiments in this disclosure emphasizes the differences between the various embodiments, and the similarities or similarities can be referred to each other. For the sake of brevity, they will not be described in detail.

[0043] At the same time, it should be understood that, for ease of description, the dimensions of the various parts shown in the accompanying drawings are not drawn according to actual scale.

[0044] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit this disclosure or its application or use.

[0045] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0046] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0047] The embodiments disclosed herein can be applied to electronic devices such as terminal devices, computer systems, and servers, and can operate together with a wide range of other general-purpose or special-purpose computing system environments or configurations. Examples of well-known terminal devices, computing systems, environments, and / or configurations suitable for use with electronic devices such as terminal devices, computer systems, and servers include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computer systems, and distributed cloud computing environments including any of the above systems, etc.

[0048] Electronic devices such as terminal devices, computer systems, and servers can be described in the general context of computer system executable instructions (such as program modules) executed by a computer system. Typically, program modules can include routines, programs, object programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types. Computer systems / servers can be implemented in distributed cloud computing environments, where tasks are executed by remote processing devices linked through communication networks. In distributed cloud computing environments, program modules can reside on local or remote computing system storage media, including storage devices.

[0049] Application Overview

[0050] Currently, the main approach to obtaining high-precision maps involves aggregating the high-precision positioning poses generated by RTK+IMU with the point cloud sub-maps produced by LiDAR, and then optimizing the positioning poses to obtain a global point cloud map. However, the high cost of using RTK+IMU hinders the widespread adoption of this approach. Furthermore, in certain road sections with poor RTK signal strength, such as tunnels and overpasses, the positioning performance is often suboptimal, and the positioning poses calculated solely through post-calculation methods are often insufficiently accurate, failing to address the ghosting issue that occurs during sub-map aggregation.

[0051] Therefore, this application provides a high-precision map data processing method and apparatus. First, it associates the point clouds in the overlapping area between a first point cloud map segment and a second point cloud map segment to determine the coordinate transformation matrix between the two point cloud map segments. Then, based on the initial coordinate transformation matrices corresponding to each of the two point cloud map segments and the coordinate transformation matrix between the point cloud map segments, it obtains the target coordinate transformation matrix corresponding to each of the two point cloud map segments. Based on the target coordinate transformation matrices corresponding to each of the two point cloud map segments, the point cloud map segments are aggregated to obtain a global map. In this scheme, since the overlapping area represents the same real physical world region, by associating the point clouds of the first point cloud map segment located in the overlapping area with the point clouds of the matching second point cloud map segment located in the overlapping area, the coordinate transformation matrix between the point cloud map segments can be accurately obtained based on this association, thereby obtaining the accurate target coordinate transformation matrix corresponding to each point cloud map segment. This eliminates ghosting during the point cloud map segment aggregation process and improves the accuracy of the global map.

[0052] Exemplary System

[0053] The embodiments disclosed herein can be applied to application scenarios that require the construction of high-precision maps, such as assisted driving or autonomous driving.

[0054] For example, in assisted driving or autonomous driving applications, the vehicle's onboard image sensor can capture images of the road in which the vehicle is located to obtain images containing the road. Onboard laser sensors and onboard positioning terminals are installed on the vehicle. The onboard laser sensor determines the point cloud dataset used to build the map, and the onboard positioning terminal obtains global positioning system information. Then, a high-precision map is built using the scheme of the embodiments of this disclosure.

[0055] Figure 1 This is a scenario diagram to which this disclosure applies. For example... Figure 1 As shown, the embodiments of this disclosure are applied in assisted driving or autonomous driving application scenarios, wherein the high-precision map data processing device 100 can be connected to the vehicle-mounted laser sensor 200, the vehicle-mounted image sensor 300 and the vehicle-mounted positioning terminal 400.

[0056] The high-precision map data processing device 100 can be electrically connected to the vehicle-mounted laser sensor 200, the vehicle-mounted image sensor 300, and the vehicle-mounted positioning terminal 400, respectively. For example, if the high-precision map data processing device 100 is an on-board computer, it can be electrically connected to the vehicle-mounted laser sensor 200, the vehicle-mounted image sensor 300, and the vehicle-mounted positioning terminal 400, respectively. Alternatively, the connection between the high-precision map data processing device 100 and the vehicle-mounted laser sensor 200, the vehicle-mounted image sensor 300, and the vehicle-mounted positioning terminal 400 can also be a network connection.

[0057] The vehicle-mounted laser sensor 200 can generate a point cloud dataset and transmit it to the high-precision map data processing device 100. Additionally, the vehicle-mounted image sensor 300 can capture images including roads and transmit these images to the high-precision map data processing device 100. The vehicle-mounted positioning terminal 400 can acquire Global Positioning System (GPS) information and transmit this information to the high-precision map data processing device 100.

[0058] After acquiring a point cloud dataset, the high-precision map data processing device 100 can determine the overlapping area between a first point cloud map segment and a second point cloud map segment in multiple point cloud map segments, and associate the point clouds of the first and second point cloud map segments located in the overlapping area. Then, based on the associated point clouds, it determines a first coordinate transformation matrix between the first and second point cloud map segments, and optimizes the coordinate transformation matrices corresponding to the first and second point cloud map segments respectively according to the first coordinate transformation matrix, thereby obtaining a high-precision global map after aggregation.

[0059] For example, the device 100 for building semantic maps may be a server in a vehicle (e.g., an in-vehicle smart terminal) to enable assisted driving or autonomous driving functions.

[0060] Exemplary methods

[0061] Figure 2 This is a schematic flowchart of a high-precision map data processing method provided in an exemplary embodiment of this disclosure. This embodiment can be applied to electronic devices, such as... Figure 2 As shown, it includes the following steps:

[0062] Step 201: Determine the overlapping area of ​​the first point cloud map segment and the second point cloud map segment. The first point cloud map segment and the second point cloud map segment belong to the first point cloud map.

[0063] The vehicle-mounted laser sensor can generate the first point cloud map. The first point cloud map is segmented to generate multiple point cloud map segments, including the first point cloud map segment and the second point cloud map segment. The point cloud map segments can also be called submaps.

[0064] Among the multiple point cloud map fragments, there exists a first point cloud map fragment and a second point cloud map fragment. Parts of the point cloud in the first point cloud map fragment and parts of the point cloud in the second point cloud map fragment represent the same physical region of the real world. Therefore, the parts of the first point cloud map fragment and the parts of the second point cloud map fragment form an overlapping area.

[0065] Step 202: Determine the first point cloud and the second point cloud in the overlapping area. The first point cloud belongs to the first point cloud map segment, and the second point cloud belongs to the second point cloud map segment.

[0066] The first point cloud is a portion of the point cloud in the first point cloud map segment mentioned above, and the second point cloud is a portion of the point cloud in the second point cloud map segment mentioned above.

[0067] Step 203: Associate the first point cloud and the second point cloud.

[0068] A point cloud is a collection of points, each representing a real-world object. Each point has attribute information to identify it, such as its physical location in the real world and the semantic information of the object it represents. Therefore, it is possible to associate the first point cloud with the second point cloud based on the attribute information of the point cloud.

[0069] Step 204: Determine the first coordinate transformation matrix between the first point cloud map segment and the second point cloud map segment based on the associated first point cloud and second point cloud.

[0070] The first coordinate transformation matrix is ​​used to convert the position coordinates of points in the first point cloud into the position coordinates of matching points in the second point cloud.

[0071] The coordinate transformation matrix can also be understood as the transformation matrix between the first pose of the vehicle-mounted laser sensor corresponding to the first point cloud map segment and the second pose of the vehicle-mounted laser sensor corresponding to the second point cloud map segment. It is a homogeneous transformation matrix composed of a rotation matrix and a translation vector.

[0072] Step 205: Based on the first coordinate transformation matrix, the initial second coordinate transformation matrix, and the initial third coordinate transformation matrix, determine the target second coordinate transformation matrix and the target third coordinate transformation matrix. The second coordinate transformation matrix is ​​used to transform the first point cloud map fragment to the world coordinate system, and the third coordinate transformation matrix is ​​used to transform the second point cloud map fragment to the world coordinate system.

[0073] Each point cloud map segment has its own independent relative coordinate system, which can be transformed to the world coordinate system using a coordinate transformation matrix.

[0074] Step 206: Aggregate the first point cloud map fragment and the second point cloud map fragment according to the target second coordinate transformation matrix and the target third coordinate transformation matrix to obtain the global map.

[0075] After optimizing the initial second and third coordinate transformation matrices, the target second and third coordinate transformation matrices are obtained, respectively. Then, the first and second point cloud map fragments are aggregated based on the target second and third coordinate transformation matrices to obtain a high-precision global map.

[0076] This disclosure provides a high-precision map data processing method and apparatus. First, it associates the point clouds in the overlapping area between a first point cloud map fragment and a second point cloud map fragment to determine the coordinate transformation matrix between the two point cloud map fragments. Then, based on the initial coordinate transformation matrices corresponding to each of the two point cloud map fragments and the coordinate transformation matrix between the point cloud map fragments, it obtains the target coordinate transformation matrix corresponding to each of the two point cloud map fragments. Based on the target coordinate transformation matrices corresponding to the two point cloud map fragments, the point cloud map fragments are aggregated to obtain a global map. In this scheme, since the overlapping area represents the same real physical world region, by associating the point clouds of the first point cloud map fragment located in the overlapping area with the point clouds of the matching second point cloud map fragment located in the overlapping area, the coordinate transformation matrix between the point cloud map fragments can be accurately obtained based on this association, thus obtaining the accurate target coordinate transformation matrix corresponding to each point cloud map fragment. This eliminates ghosting during the point cloud map fragment aggregation process and improves the accuracy of the global map.

[0077] In one example, the methods for obtaining the first point cloud map fragment and the second point cloud map fragment include:

[0078] First, a point cloud dataset formed by the vehicle's surrounding environment is acquired by an onboard laser sensor, and an image corresponding to the point cloud dataset is acquired by an onboard image sensor. Then, based on the correspondence between the pixel semantics in the point cloud dataset and the image, multiple point cloud map segments of the first point cloud map are determined, including the first point cloud map segment and the second point cloud map segment. Global Positioning System (GPS) information collected by the onboard positioning terminal is acquired. Based on the GPS information, the position coordinates of the point clouds in the multiple point cloud map segments are transformed to the world coordinate system.

[0079] In the above example, this disclosure determines a first point cloud map with semantic labels by the correspondence between the point cloud dataset and the semantics of pixels in the image, thereby generating multiple point cloud map fragments with semantic labels, and then transforms the position coordinates of the point clouds in the multiple point cloud map fragments to the world coordinate system, so as to facilitate the subsequent association between the point clouds in the first point cloud map fragment and the point clouds in the second point cloud map fragment.

[0080] In one example, the methods for obtaining a point cloud dataset include:

[0081] The original point cloud dataset is acquired by using an onboard laser sensor, and then the motion distortion of the points in the original point cloud dataset is corrected by an inertial measurement unit (IMU) to obtain a point cloud dataset with higher accuracy.

[0082] The IMU can be installed on the vehicle. Through the pre-integration of the IMU, the pose of the vehicle-mounted laser sensor at each point in the point cloud dataset at the corresponding time can be estimated. Based on this, the points can be transformed so that the pose of the vehicle-mounted laser sensor corresponding to each point is the same, thereby realizing the motion distortion correction of the point cloud dataset.

[0083] In this exemplary embodiment, motion distortion correction is performed on the points in the point cloud dataset, and the point cloud dataset composed of the points after motion distortion correction is used as the point cloud dataset for constructing the first point cloud map, thereby reducing the impact of motion distortion and improving the accuracy of the first point cloud map.

[0084] In one example, multiple point cloud map segments of the first point cloud map are determined based on the correspondence between the point cloud dataset and the pixel semantics in the image, including:

[0085] The correspondence between points in the point cloud dataset and pixels in the image is determined. Then, based on the semantics of pixels in the image and the correspondence, the semantics of points in the point cloud dataset are determined. Next, a first point cloud map is constructed based on the semantics of points in the point cloud dataset. This first point cloud map is a semantic map. Finally, the first point cloud map is segmented to obtain multiple point cloud map fragments.

[0086] The methods for determining the correspondence between points in a point cloud dataset and pixels in an image include:

[0087] The first coefficient is determined based on the time it takes for the vehicle-mounted image sensor to capture an image and the time it takes for the vehicle-mounted laser sensor to emit a laser beam, using the following formula:

[0088] k = (tc - tl0) / (tl1 - tl0) Formula (1);

[0089] Where k is the first coefficient, tc is the time when the vehicle image sensor captures the image, tl0 is the time when the vehicle laser sensor emits the first laser, tl1 is the time when the vehicle laser sensor emits the second laser, the first laser is the laser corresponding to the last point cloud before tc, and the second laser is the laser corresponding to the first point cloud after tc.

[0090] Determine the pose of the vehicle-mounted laser sensor;

[0091] Based on the pose of the vehicle-mounted laser sensor and the first coefficient, the second coefficient is determined using the following formula:

[0092]

[0093] Where TC is the second coefficient, T1 is the pose of the vehicle-mounted laser sensor at tl0, and T2 is the pose of the vehicle-mounted laser sensor at tl1. The linear difference between the quaternion interpolation of rotation and translation representing pose. Multiplication of the quaternion representing the pose with the constant value of the translation vector;

[0094] Based on the second coefficient, the correspondence between points in the point cloud dataset and pixels in the image is determined using the following formula:

[0095] Px = K × T cl ×TC×P L Formula (3);

[0096] Where Px is the coordinate of a pixel in the image in the image coordinate system, K is the intrinsic parameter calibration matrix of the vehicle image sensor, and T cl P is the extrinsic calibration matrix for transforming the point cloud dataset from map coordinates to image coordinates. L The coordinates of the point cloud of the second laser in the map coordinate system.

[0097] Based on this embodiment, the correspondence between points in the point cloud dataset and pixels in the image can be determined, so that the semantics of each point in the point cloud dataset can be determined according to the correspondence and the semantics of the pixels in the image.

[0098] In the above exemplary embodiment, the pose of the vehicle-mounted laser sensor can be determined through the following steps:

[0099] First, based on the aforementioned point cloud dataset, the white noise and zero bias of the vehicle-mounted inertial measurement unit (IMU) are adjusted.

[0100] Then, the pose of the vehicle-mounted lidar is determined based on the global positioning system (GPS) and an inertial measurement unit adjusted for white noise and zero bias.

[0101] An IMU typically contains a three-axis gyroscope and three-directional accelerometers, enabling it to measure the angular velocity and acceleration of an object in three-dimensional space. However, as the measurement time increases, the IMU's white noise and bias gradually increase, affecting its measurement accuracy.

[0102] To address this issue, in the exemplary embodiments of this disclosure, the white noise and zero-bias of the IMU are adjusted based on the aforementioned point cloud dataset. This point cloud dataset has undergone motion distortion correction; by adjusting it, the white noise and zero-bias of the IMU can be reduced.

[0103] After adjusting the white noise and zero bias of the IMU, the pose of the vehicle-mounted LiDAR sensor can be determined using GPS and the IMU. The GPS receiver receives ephemeris parameters and time information from satellites and calculates its position based on this information. Additionally, the vehicle-mounted IMU measures the vehicle's angular velocity and acceleration in three-dimensional space. In this combination, the pose of the vehicle-mounted LiDAR can be determined.

[0104] In the first point cloud map construction method disclosed herein, it is necessary to determine the correspondence between points in the point cloud dataset and pixels in the image based on the pose of the vehicle-mounted laser sensor. Furthermore, when constructing the first point cloud map based on the semantics of points in the point cloud dataset, it is often also necessary to determine the vehicle's pose based on the pose of the vehicle-mounted laser sensor to achieve vehicle localization, and based on the vehicle's localization and the point cloud dataset, determine the positions of the road and the various objects contained around the road.

[0105] Existing mapping technologies typically rely on IMUs and real-time kinematic (RTK) systems to determine the pose of LiDAR. However, RTK is expensive, leading to higher costs for existing mapping technologies and hindering their widespread adoption.

[0106] However, in the scheme provided in the exemplary embodiments of this disclosure, the pose of the lidar can be determined by IMU and GPS. The cost of GPS is generally lower than that of RTK, thus reducing the cost required to build semantic maps and promoting the application of semantic map building technology.

[0107] Furthermore, in the solution provided by the exemplary embodiments of this disclosure, when determining the pose of the vehicle-mounted laser sensor, the white noise and zero bias of the IMU are adjusted using the point cloud dataset after motion distortion correction, thereby improving the accuracy of IMU measurement, which in turn improves the accuracy of determining the pose of the vehicle-mounted laser sensor, and further improves the accuracy of the first point cloud map, so as to facilitate the subsequent construction of a high-precision global map.

[0108] Figure 3 This is a flowchart illustrating a method for constructing a first point cloud map provided by another exemplary embodiment of this disclosure. This flowchart is an example of the overall process of constructing a first point cloud map using the scheme of the exemplary embodiment of this disclosure.

[0109] See Figure 3 In the method for constructing a first point cloud map provided in the exemplary embodiments of this disclosure, an on-board image sensor, an on-board laser sensor, an IMU, a GPS, and an electronic device performing the method are applied.

[0110] In the process of building the first point cloud map, the vehicle-mounted image sensor can capture images containing roads and transmit the images to an electronic device, which performs semantic segmentation on the images to obtain the semantics of the pixels in the images.

[0111] The vehicle-mounted laser sensor acquires a point cloud dataset and transmits it to an electronic device, which then performs motion distortion correction on the dataset. Additionally, the electronic device can adjust the white noise and zero-bias of the IMU using the motion-distortion-corrected point cloud dataset, and determine the pose of the vehicle-mounted laser sensor using GPS and the adjusted IMU.

[0112] Because vehicle-mounted laser sensors and vehicle-mounted image sensors are typically installed in different locations, it's possible for the laser sensor to be unobstructed while the image sensor is obstructed. This can result in areas with high pixel depth in the image, making semantic interpretations based on pixels in these areas inaccurate. Furthermore, when the vehicle-mounted image sensor captures distant objects, its resolution is lower. Therefore, a single pixel in the image may be associated with multiple points in the point cloud dataset, meaning the same pixel corresponds to multiple points in the dataset. In such cases, the depth of the pixel's region is also high, leading to inaccurate semantic interpretations based on pixels in that region. These high-depth pixels represent dynamic obstacles. Depth filtering can remove these dynamic obstacles, thereby improving the accuracy of the initial point cloud map.

[0113] In addition, the electronic device needs to determine not only the point cloud dataset but also the image containing the road, that is, to establish the association between the point cloud dataset and the image, and to determine the correspondence between the points in the point cloud dataset and the pixels in the image. Then, based on the semantics of the pixels in the image and the correspondence between the points in the point cloud dataset and the pixels in the image, the semantics of the points in the point cloud dataset are determined, and a first point cloud map is constructed based on the semantics of the points in the point cloud dataset. The first point cloud map is then segmented to obtain multiple point cloud map fragments.

[0114] Figure 4 This is a schematic flowchart of a method for associating point clouds provided in an exemplary embodiment of this disclosure. See also: Figure 4 In another exemplary embodiment of this disclosure, the above-described Figure 2 or Figure 3 Based on the illustrated embodiment, the first point cloud and the second point cloud are associated through the following steps:

[0115] Step S2031: Determine the semantic categories of all points in the first point cloud and the semantic categories of all points in the second point cloud.

[0116] For example, the first point cloud includes {P1, P2, P3, P4}, and the second point cloud includes {P5, P6, P7, P8}. The semantic categories of the points in the first point cloud are tree, tree, lane, and person, respectively, and the semantic categories of the points in the second point cloud are tree, tree, lane, and dog, respectively.

[0117] Step S2032: Identify two points that belong to the first point cloud and the second point cloud respectively and have the same semantic category.

[0118] Based on the example in step S2031, for example, if the points in the first point cloud belonging to trees include {P1, P2} and the points in the second point cloud include {P5, P6}, then there are four ways to combine two points with tree semantic labels, including {P1, P5}, {P1, P6}, {P2, P5}, and {P2, P6}, that is, one of the two points belongs to the first point cloud and the other point belongs to the second point cloud; if the points in the first point cloud belonging to lanes include {P3} and the points in the second point cloud include {P7}, then the two points with lane semantic labels are {P3, P7}.

[0119] Step S2033: If the distance between two points with the same semantic category meets the first preset condition, then associate the two points with the same semantic category.

[0120] The first preset condition is set according to actual needs. For example, the first preset condition is "less than the first threshold, and the first threshold is 1 meter".

[0121] It should be understood that points in the first point cloud and points in the second point cloud that are semantically related and meet the first preset conditions can also be called two matching points, that is, these two points represent the same object in the physical world.

[0122] Based on the example in step S2032, for example, among two points with tree labels, if the distance between {P1, P5} and {P2, P6} meets the first preset condition, then P1 and P5, and P2 and P6 are associated respectively; if the distance between {P3, P7} meets the first preset condition, then P3 and P7 are associated.

[0123] In the above embodiments, by semantically associating points in the first point cloud and points in the second point cloud that meet the first preset conditions, matching points can be accurately obtained, so that the first coordinate transformation matrix between the first point cloud map segment and the second point cloud map segment can be accurately calculated.

[0124] Figure 5 This is a schematic flowchart of an exemplary embodiment of the present disclosure providing a method for determining overlapping regions. See also: Figure 5 In another exemplary embodiment of this disclosure, based on the embodiments shown above, the overlapping area of ​​the first point cloud map segment and the second point cloud map segment can be determined by the following steps:

[0125] Step S2011: Determine the physical locations of all points in the first point cloud map segment and the physical locations of all points in the second point cloud map segment.

[0126] It should be understood that the physical location of a point in a point cloud map refers to the physical location of the object represented by that point in the real world. This physical location can be represented by three-dimensional physical coordinates, which typically include three dimensions: x, y, and z.

[0127] Step S2012: Construct a k-dimensional tree based on all points of the first point cloud map fragment.

[0128] Among them, the k-dimensional tree is a data structure used in computer science to organize and represent a set of points in k-dimensional space. It is a binary search tree with constraints.

[0129] Step S2013: If a point is found through k-tree search where the distance between the physical location of the point in the second point cloud map segment and the physical location of the point in the first point cloud map segment meets the second preset condition, then the overlapping area is determined based on the point in the second point cloud map segment that meets the second preset condition and the point in the first point cloud map.

[0130] The second preset condition can be set as needed. For example, the second preset condition is "less than the second threshold, where the second threshold is 10 meters".

[0131] In one example, the first point cloud map fragment is composed of the above. Figure 3 The IMU+GPS method shown is used to determine the position, and after converting the point cloud to the world coordinate system based on the GPS pose information, the above-mentioned steps are performed. Figure 5 The method shown can guarantee the accuracy of the overlapping area.

[0132] In the example above, a k-dimensional tree is constructed based on the points in the first point cloud map segment, and the points in the second point cloud map segment that meet the second preset conditions are searched for with the points in the first point cloud map to determine the overlapping area. Using this method, the calculation is not only simple and fast, but also the accuracy of the overlapping area is high.

[0133] Figure 6 This is a schematic flowchart of a first coordinate transformation matrix determination method provided in an exemplary embodiment of this disclosure, such as... Figure 6 As shown, in another exemplary embodiment of this disclosure, the first coordinate transformation matrix can be determined by the following steps based on the embodiments shown above:

[0134] Step S2041: Determine the associated feature point set pairs in the first point cloud and the second point cloud. The feature point set pair includes the first feature point set and the second feature point set. The first feature point set belongs to the first point cloud, and the second feature point set belongs to the second point cloud.

[0135] For example, the first feature point set P S1 ={P1, P2}, the second feature point set P S2 = {P5, P6}, where P S1 The semantic labels for each point in the text are, in order: tree, tree, P S2 The semantic labels for each point are: tree, tree. Furthermore, the physical distances between P1 and P5, and between P2 and P6, both meet the first preset condition. That is, P1 and P5 are related, and P2 and P6 are related.

[0136] Step S2042: Determine the third feature point set based on the first feature point set and the first coordinate transformation matrix.

[0137] Specifically, the position coordinates of the first feature point set are transformed by the first coordinate transformation matrix to obtain a new feature point set, which is the third feature point set.

[0138] For example, the third feature point set P S2 ={P5', P6'}, the third feature point set P S2 ' is the first feature point set P S1 The result is obtained by multiplying the position coordinate matrix and the first coordinate transformation matrix T1, as shown in the following formula:

[0139] P′S2 =T1×P S1 Formula (4);

[0140] Where T1 represents a 6-degree-of-freedom matrix, P S1 Let T1 be a matrix representing the position coordinates of P1 and P2. For example, the coordinates of P1 are (x1, y1, z1), and the coordinates of P2 are (x2, y2, z2). Since T1 represents a 6-DOF matrix, including rotation and translation, and its size is 4*4, then P... S1 The matrix represented in formula (4) is The size is 4*2, so we can see that P S1 If there are n points, then the size of the matrix it represents is 4*n.

[0141] Step S2043: If the first loss value meets the third preset condition, determine the target first coordinate transformation matrix. The first loss value represents the Mahalanobis distance of the covariance between the position of the third feature point set and the position of the second feature point set.

[0142] For example, the third preset condition is that the first loss value is minimized.

[0143] Specifically, as shown in the following formula:

[0144] Loss1=min‖T1×P S1 -P S2 || Ω Formula (5);

[0145] Where Ω represents the calculation of covariance, ||| Ω This represents the calculation of Mahalanobis distance, min represents the function for calculating the minimum value, and Loss1 represents the loss value. When Loss1 is minimized, T1 corresponds to the first coordinate transformation matrix of the target. P S2 The matrix represented in formula (5) is shown in the above P. S1 This will not be elaborated upon here.

[0146] Similarly, when the first and second feature point sets each include multiple semantic labels, the method for determining the target's coordinate transformation matrix is ​​as follows:

[0147] Assume the first feature point set P S1 ={P1, P2, P3, P4}, the second feature point set P S2 = {P5, P6, P7, P8}, where P S1 The semantic labels for each point in the text are, in order: tree, tree, lane, person, P S2The semantic labels for each point are, in order: tree, tree, lane, person. Furthermore, the physical distances between P1 and P5, P2 and P6, P3 and P7, and P4 and P8 all meet the first preset condition. That is, P1 and P5 are associated, P2 and P6 are associated, P3 and P7 are associated, and P4 and P8 are associated.

[0148] The first feature point set P S1 Based on semantic categories, it is divided into several subsets, including: P S11 P S12 P S13 P S11 ={P1, P2}, P S12 ={P3}, P S13 ={P4}, corresponding to the second feature point set P S2 Includes: P S21 P S22 P S23 P S21 ={P5, P6}, P S22 ={P7}, P S23 ={P8}, then according to formula (4), we can obtain:

[0149] Loss2=min(‖T1×P S11 -P S21 || Ω +‖T1×P S12 -P S22 || Ω +‖T1×P S13 -P S23 || Ω ) formula (6);

[0150] Therefore, when the first feature point set P S1 Second feature point set P S2 When the point cloud is divided into n subsets according to semantic categories (i.e., there are n semantic categories in the point cloud), the formula for determining the first coordinate transformation matrix of the target is as follows:

[0151]

[0152] Where j = 1, 2, ..., n. T1, the first coordinate transformation matrix of the target, corresponds to the minimum loss value. P S1j and P S2j The position coordinate matrix represented in formula (7) is shown in the above P. S1 This will not be elaborated upon here.

[0153] For example, the above loss is calculated using the iterative closest point (ICP) algorithm.

[0154] In the example above, the third feature point set is determined based on the associated first feature point set and the first coordinate transformation matrix. Then, the first loss value is determined based on the Mahalanobis distance of the covariance between the position of the third feature point set and the position of the second feature point set. When the first loss value meets the third preset condition, a high-precision target first coordinate transformation matrix can be determined so as to optimize the second and third coordinate transformation matrices in the future.

[0155] In one example, the target second coordinate transformation matrix and the target third coordinate transformation matrix are matrices in which the second loss value meets the fourth preset condition under the constraint of the first coordinate transformation matrix. The second loss value represents the Mahalanobis distance of the covariances among the first, second and third coordinate transformation matrices.

[0156] For example, the fourth preset condition is that the second loss value is the minimum value.

[0157] The calculation methods for the second and third coordinate transformation matrices of the target are explained below using the pose graph optimization algorithm, as shown in the following formulas:

[0158]

[0159] Where θ1 represents the second coordinate transformation matrix, θ2 represents the third coordinate transformation matrix, and T 12 Ω represents the first coordinate transformation matrix. 12 The covariance between the first, second, and third coordinate transformation matrices is represented by |||. Ω This indicates the calculation of Mahalanobis distance. This represents the subtraction between the Lie algebra and the pose matrix.

[0160] For example, θ1 is a 6-DOF matrix, θ1 = [yaw1 pitch1 row1 x1 y1 z1], where yaw1, pitch1, and row1 are values ​​related to Euler angles, and x1, y1, and z1 are values ​​related to position coordinates. θ2 is a 6-DOF matrix, θ1 = [yaw2 pitch2 row2 x2 y2 z2], where yaw2, pitch2, and row2 are values ​​related to Euler angles, and x2, y2, and z2 are values ​​related to position coordinates.

[0161] First, construct a pose graph, where θ1 and θ2 are used as nodes, T 12θ1 and θ2 are used as edges to constrain them. Then, θ1 and θ2 are continuously adjusted until the value of Loss4 is minimized (i.e., the derivative of the loss function is 0), thus obtaining the optimal θ1 and θ2.

[0162] Similarly, for cases where multiple point cloud map segments have overlapping areas, the formula for calculating the coordinate transformation matrix corresponding to each point cloud map segment is as follows:

[0163]

[0164] Where G represents the aforementioned multiple point cloud map segments, θ i θ represents the coordinate transformation matrix used to transform the i-th point cloud map fragment to the world coordinate system. j This represents the coordinate transformation matrix used to transform the j-th point cloud map fragment to the world coordinate system, where the i-th point cloud map fragment and the j-th point cloud map fragment have overlapping areas, and T ij Let represent the coordinate transformation matrix between the coordinates of the i-th point cloud map segment and the coordinates of the j-th point cloud map segment. Using formula (9), the coordinate transformation matrix corresponding to each point cloud map segment that is globally optimal can be obtained.

[0165] In the example above, the coordinate transformation matrix between two point cloud map segments is used as a constraint. The coordinate transformation matrix corresponding to each point cloud map segment for transformation to the world coordinate system is adjusted until the second loss value meets the fourth preset condition. This results in the globally optimal coordinate transformation matrix for each point cloud map segment for transformation to the world coordinate system. This allows the point cloud map segments to be aggregated based on the globally optimal coordinate transformation matrix corresponding to each point cloud map segment, eliminating the ghosting generated during the aggregation process and thus obtaining a high-precision global aggregated map.

[0166] Figure 7 This is a flowchart illustrating a high-precision map data processing method provided in another exemplary embodiment of this disclosure. This flowchart is an example of the overall process of constructing a high-precision map using the scheme of the exemplary embodiment of this disclosure.

[0167] See Figure 7 First, the overlapping area of ​​the first point cloud map segment and the second point cloud map segment is determined. Then, the first point cloud of the first point cloud map segment and the second point cloud of the second point cloud map segment in the overlapping area are semantically associated. Based on the semantically associated first point cloud and second point cloud, the first coordinate transformation matrix is ​​determined using the ICP algorithm.

[0168] Next, based on the first coordinate transformation matrix, the initial second coordinate transformation matrix, and the initial third coordinate transformation matrix, the graph optimization algorithm is used to determine the globally optimal target second coordinate transformation matrix and target third coordinate transformation matrix. Then, based on the globally optimal target second coordinate transformation matrix and target third coordinate transformation matrix, the first point cloud map fragment and the second point cloud map fragment are aggregated to obtain a high-precision global map.

[0169] In one example, odometry pose is estimated using simultaneous localization and mapping (SLAM), and then 3D reconstruction of the point cloud is performed based on the odometry pose to obtain either a first point cloud map fragment or a second point cloud map fragment.

[0170] In one example, an initial second coordinate transformation matrix or an initial third coordinate transformation matrix is ​​determined based on the odometry pose and GPS information.

[0171] Exemplary device

[0172] Figure 8 This is a structural diagram of a high-precision map data processing apparatus provided in an exemplary embodiment of this disclosure. This high-precision map data processing apparatus can be installed in electronic devices such as terminal devices and servers, or on objects such as vehicles, to execute the high-precision map data processing method of any of the embodiments described above. Figure 8 As shown, the high-precision map data processing device in this embodiment includes an overlapping area determination module 301, a point cloud association module 302, a coordinate transformation matrix determination module 303, and a map aggregation module 304.

[0173] The overlapping area determination module 301 is used to determine the overlapping area of ​​the first point cloud map segment and the second point cloud map segment, wherein the first point cloud map segment and the second point cloud map segment belong to the first point cloud map.

[0174] The point cloud association module 302 is used to determine the first point cloud and the second point cloud in the overlapping area. The first point cloud belongs to the first point cloud map segment, and the second point cloud belongs to the second point cloud map segment.

[0175] The point cloud association module 302 is also used to associate the first point cloud and the second point cloud.

[0176] The coordinate transformation matrix determination module 303 is used to determine the first coordinate transformation matrix between the first point cloud map segment and the second point cloud map segment based on the first point cloud and the second point cloud.

[0177] The coordinate transformation matrix determination module 303 is also used to determine the target second coordinate transformation matrix and the target third coordinate transformation matrix based on the first coordinate transformation matrix, the initial second coordinate transformation matrix and the initial third coordinate transformation matrix, wherein the second coordinate transformation matrix is ​​used to transform the first point cloud map fragment to the world coordinate system, and the third coordinate transformation matrix is ​​used to transform the second point cloud map fragment to the world coordinate system.

[0178] The map aggregation module 304 is used to aggregate the first point cloud map fragment and the second point cloud map fragment according to the target second coordinate transformation matrix and the target third coordinate transformation matrix to obtain a global map.

[0179] This disclosure provides a high-precision map data processing method and apparatus. First, it associates the point clouds in the overlapping area between a first point cloud map segment and a second point cloud map segment to determine the coordinate transformation matrix between the two point cloud map segments. Then, based on the initial coordinate transformation matrices corresponding to each of the two point cloud map segments and the coordinate transformation matrix between the point cloud map segments, it obtains the target coordinate transformation matrix corresponding to each of the two point cloud map segments. Based on the target coordinate transformation matrices corresponding to the two point cloud map segments, the point cloud map segments are aggregated to obtain a global map. In this scheme, since the overlapping area represents the same real physical world region, by associating the point clouds of the first point cloud map segment located in the overlapping area with the point clouds of the matching second point cloud map segment located in the overlapping area, the coordinate transformation matrix between the point cloud map segments can be accurately obtained based on this association, thereby obtaining the accurate target coordinate transformation matrix corresponding to each point cloud map segment. This eliminates ghosting during the point cloud map segment aggregation process and improves the accuracy of the global map.

[0180] In one feasible example, the target second coordinate transformation matrix and the target third coordinate transformation matrix are matrices in which the second loss value meets the fourth preset condition under the constraint of the first coordinate transformation matrix. The second loss value represents the Mahalanobis distance of the covariances among the first coordinate transformation matrix, the second coordinate transformation matrix and the third coordinate transformation matrix.

[0181] Furthermore, Figure 9 This is a structural diagram of a high-precision map data processing apparatus provided in another exemplary embodiment of this disclosure, see below. Figure 9 The structural diagram shown illustrates that, in one feasible example, the point cloud association module 302 includes:

[0182] The semantic category determination unit 3021 is used to determine the semantic category of all points in the first point cloud and the semantic category of all points in the second point cloud.

[0183] The semantic category determination unit 3021 is also used to determine two points that belong to the first point cloud and the second point cloud respectively and have the same semantic category.

[0184] The point cloud association unit 3022 is used to associate two points with the same semantic category if the distance between two points with the same semantic category meets a first preset condition.

[0185] In one feasible example, the overlapping region determination module 301 includes:

[0186] The physical location determination unit 3011 is used to determine the physical locations of all points in the first point cloud map segment and the physical locations of all points in the second point cloud map segment.

[0187] k-tree building unit 3012 is used to build a k-tree based on all points of the first point cloud map fragment.

[0188] The overlapping region determination unit 3013 is used to determine the overlapping region based on the points in the second point cloud map segment and the points in the first point cloud map segment if the distance between the physical location of the points in the second point cloud map segment and the physical location of the points in the first point cloud map segment meets the second preset condition through k-tree search.

[0189] In one feasible example, the coordinate transformation matrix determination module 303 includes:

[0190] The feature point set pair determination unit 3031 is used to determine the associated feature point set pairs in the first point cloud and the second point cloud. The feature point set pair includes a first feature point set and a second feature point set, where the first feature point set belongs to the first point cloud and the second feature point set belongs to the second point cloud.

[0191] The feature point set determination unit 3031 is also used to determine the third feature point set based on the first feature point set and the first coordinate transformation matrix.

[0192] The coordinate transformation matrix determination unit 3032 is used to determine the target first coordinate transformation matrix when the first loss value meets the third preset condition. The first loss value represents the Mahalanobis distance of the covariance between the position of the third feature point set and the position of the second feature point set.

[0193] In the above Figure 8 or Figure 9 Based on the illustrated embodiment, in another exemplary embodiment of this disclosure, the high-precision map data processing apparatus further includes:

[0194] The point cloud dataset acquisition module is used to acquire point cloud datasets formed by the vehicle's surrounding environment, collected by the vehicle-mounted laser sensor.

[0195] The image acquisition module is used to acquire images corresponding to the point cloud dataset collected by the vehicle-mounted image sensor;

[0196] The point cloud map segment determination module is used to determine multiple point cloud map segments of the first point cloud map based on the correspondence between the point cloud dataset and the pixel semantics in the image. The multiple point cloud map segments include the first point cloud map segment and the second point cloud map segment.

[0197] The Global Positioning System (GPS) information acquisition module acquires GPS information collected by the vehicle-mounted positioning terminal.

[0198] The location coordinate transformation module is used to transform the location coordinates of multiple point cloud map fragments to the world coordinate system based on GPS information.

[0199] Exemplary electronic devices

[0200] Below, for reference Figure 10 This describes an electronic device according to embodiments of the present application. The electronic device may be either or both of a first device 100 and a second device 200, or a standalone device independent of them, which may communicate with the first and second devices to receive acquired input signals from them.

[0201] Figure 10 A block diagram of an electronic device according to an embodiment of this application is illustrated.

[0202] like Figure 10 As shown, the electronic device 10 includes one or more processors 11 and memory 12.

[0203] The processor 11 may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 10 to perform desired functions.

[0204] The memory 12 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 11 may execute the program instructions to implement the sound source localization methods of the various embodiments of this application described above and / or other desired functions. Various contents such as input signals, signal components, and noise components may also be stored in the computer-readable storage medium.

[0205] In one example, the electronic device 10 may also include an input device 13 and an output device 14, which are interconnected via a bus system and / or other forms of connection mechanism (not shown).

[0206] For example, when the electronic device is a first device 100 or a second device 200, the input device 13 can be the aforementioned microphone or microphone array for capturing the input signal from the sound source. When the electronic device is a standalone device, the input device 13 can be a communication network connector for receiving the acquired input signals from the first device 100 and the second device 200.

[0207] In addition, the input device 13 may also include, for example, a keyboard, a mouse, etc.

[0208] The output device 14 can output various information to the outside, including determined distance information, direction information, etc. The output device 14 may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0209] Of course, for the sake of simplicity, Figure 10 Only some of the components of the electronic device 10 relevant to this application are shown in this illustration; components such as buses, input / output interfaces, etc., are omitted. In addition, the electronic device 10 may include any other suitable components depending on the specific application.

[0210] Exemplary computer program products and computer-readable storage media

[0211] In addition to the methods and devices described above, embodiments of this application may also be computer program products, which include computer program instructions that, when executed by a processor, cause the processor to perform the steps in the sound source localization methods according to various embodiments of this application as described in the "Exemplary Methods" section of this specification.

[0212] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this application. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0213] Furthermore, embodiments of this application may also be computer-readable storage media storing computer program instructions thereon, which, when executed by a processor, cause the processor to perform the steps in the sound source localization methods according to various embodiments of this application described in the "Exemplary Methods" section of this specification.

[0214] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may, for example, include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0215] The basic principles of this application have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this application are merely examples and not limitations, and should not be considered as essential features of each embodiment of this application. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the application to the necessity of employing the aforementioned specific details for implementation.

[0216] The block diagrams of devices, apparatuses, devices, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0217] It should also be noted that in the apparatus, equipment, and methods of this application, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions of this application.

[0218] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this application. Therefore, this application is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0219] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this application to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A method for processing high-precision map data, comprising: Determine the overlapping area of ​​the first point cloud map segment and the second point cloud map segment, wherein the first point cloud map segment and the second point cloud map segment belong to the first point cloud map; Determine the first point cloud and the second point cloud of the overlapping region, wherein the first point cloud belongs to the first point cloud map segment and the second point cloud belongs to the second point cloud map segment; Associate the first point cloud and the second point cloud, which have the same semantic category; Based on the associated first point cloud and second point cloud, determine the first coordinate transformation matrix between the first point cloud map segment and the second point cloud map segment; Based on the first coordinate transformation matrix, the second coordinate transformation matrix, and the third coordinate transformation matrix, a target second coordinate transformation matrix and a target third coordinate transformation matrix are determined. The second coordinate transformation matrix is ​​used to transform the first point cloud map fragment to the world coordinate system, and the third coordinate transformation matrix is ​​used to transform the second point cloud map fragment to the world coordinate system. The target second coordinate transformation matrix and the target third coordinate transformation matrix are matrices where, under the constraint of the first coordinate transformation matrix, the second loss value meets a fourth preset condition. The second loss value represents the Mahalanobis distance of the covariances among the first coordinate transformation matrix, the second coordinate transformation matrix, and the third coordinate transformation matrix. The first point cloud map fragment and the second point cloud map fragment are aggregated based on the target second coordinate transformation matrix and the target third coordinate transformation matrix to obtain a global map.

2. The method according to claim 1, wherein, The association of the first point cloud and the second point cloud, which have the same semantic category, includes: Determine the semantic categories of all points in the first point cloud and the semantic categories of all points in the second point cloud; Identify two points that belong to the first point cloud and the second point cloud respectively and have the same semantic category; If the distance between two points with the same semantic category meets the first preset condition, then the two points with the same semantic category are associated.

3. The method according to claim 1 or 2, further comprising: Acquire a point cloud dataset formed by the vehicle's surrounding environment, collected by an onboard laser sensor; Acquire images corresponding to the point cloud dataset collected by the vehicle-mounted image sensor; Based on the correspondence between the point cloud dataset and the pixel semantics in the image, multiple point cloud map segments of the first point cloud map are determined, including the first point cloud map segment and the second point cloud map segment. Acquire GPS information collected by the vehicle-mounted positioning terminal; The location coordinates of the multiple point cloud map fragments are converted to the world coordinate system based on the GPS information.

4. The method according to claim 1 or 2, wherein, Determining the overlapping area of ​​the first point cloud map segment and the second point cloud map segment includes: Determine the physical locations of all points in the first point cloud map segment and the physical locations of all points in the second point cloud map segment; Construct a k-dimensional tree based on all points of the first point cloud map fragment; If a point is found through the k-tree search where the distance between the physical location of the point in the second point cloud map segment and the physical location of the point in the first point cloud map segment meets the second preset condition, then the overlapping area is determined based on the point in the second point cloud map segment that meets the second preset condition and the point in the first point cloud map segment.

5. The method according to claim 2, wherein, Determining the first coordinate transformation matrix between the first point cloud map segment and the second point cloud map segment based on the first point cloud and the second point cloud includes: Determine the associated feature point set pairs in the first point cloud and the second point cloud. The feature point set pair includes a first feature point set and a second feature point set, where the first feature point set belongs to the first point cloud and the second feature point set belongs to the second point cloud. The third feature point set is determined based on the first feature point set and the first coordinate transformation matrix; If the first loss value meets the third preset condition, the target first coordinate transformation matrix is ​​determined, where the first loss value represents the Mahalanobis distance of the covariance between the position of the third feature point set and the position of the second feature point set.

6. A high-precision map data processing device, comprising: The overlapping area determination module is used to determine the overlapping area of ​​the first point cloud map segment and the second point cloud map segment, wherein the first point cloud map segment and the second point cloud map segment belong to the first point cloud map. The point cloud association module is used to determine the first point cloud and the second point cloud of the overlapping area, wherein the first point cloud belongs to the first point cloud map segment and the second point cloud belongs to the second point cloud map segment. The point cloud association module is also used to associate the first point cloud and the second point cloud that have the same semantic category; The coordinate transformation matrix determination module is used to determine the first coordinate transformation matrix between the first point cloud map segment and the second point cloud map segment based on the first point cloud and the second point cloud; The coordinate transformation matrix determination module is further configured to determine a target second coordinate transformation matrix and a target third coordinate transformation matrix based on the first coordinate transformation matrix, the second coordinate transformation matrix, and the third coordinate transformation matrix. The second coordinate transformation matrix is ​​used to transform the first point cloud map fragment to the world coordinate system, and the third coordinate transformation matrix is ​​used to transform the second point cloud map fragment to the world coordinate system. The target second coordinate transformation matrix and the target third coordinate transformation matrix are matrices whose second loss value meets a fourth preset condition under the constraint of the first coordinate transformation matrix. The second loss value represents the Mahalanobis distance of the covariances among the first, second, and third coordinate transformation matrices. The map aggregation module is used to aggregate the first point cloud map fragment and the second point cloud map fragment according to the target second coordinate transformation matrix and the target third coordinate transformation matrix to obtain a global map.

7. A computer-readable storage medium storing a computer program for performing the high-precision map data processing method according to any one of claims 1-5.

8. An electronic device, the electronic device comprising: processor; Memory used to store the processor's executable instructions; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the high-precision map data processing method according to any one of claims 1-5.

Images