Point cloud alignment method, apparatus, device, and storage medium

Abstract

This application discloses a point cloud alignment method, apparatus, device, and storage medium, belonging to the field of high-precision mapping. The method includes: acquiring a first point cloud and a second point cloud; calibrating at least one pair of corresponding points in the first and second point clouds based on calibration operations, wherein the corresponding point pair includes a first point in the first point cloud and a second point in the second point cloud; optimizing a first acquisition pose and a second acquisition pose based on the positional differences between the first and second points in the at least one pair of corresponding points; and aligning the first and second point clouds based on the optimized first acquisition pose and the optimized second optimized pose. During the point cloud alignment process, it is not necessary to consider whether the point cloud is a rigid body. Furthermore, it is not necessary to consider whether objects in the environment where the point cloud was acquired are moving. This improves the accuracy of the point cloud alignment results.

Description

Technical Field

[0001] This application relates to the field of high-precision mapping, and in particular to a point cloud alignment method, apparatus, device, and storage medium. Background Technology

[0002] In the process of building high-precision maps, it is necessary to collect point clouds corresponding to roads using acquisition devices. For example, acquisition devices can be deployed on vehicles, and the point clouds corresponding to the roads can be collected as the vehicles travel on the roads. Since each acquisition can only collect point clouds corresponding to a part of the road, it is necessary to merge the point clouds collected multiple times. In the process of merging point clouds, the acquisition pose of the acquisition device at the time of point cloud acquisition is required. Due to errors in the acquisition pose of the acquisition device and time errors at different acquisition times, it is possible that points collected from different batches of point clouds for the same point in the world coordinate system may be projected to different positions in the world coordinate system, resulting in point cloud misalignment.

[0003] In related technologies, the acquired point cloud is divided into regions according to the direction of the acquired point cloud (e.g., each region is 30m apart), and the Iterative Closest Point (ICP) algorithm is used to optimize the pose constraints (including rotation and translation) of the point cloud within the region, thereby obtaining the optimized acquisition pose of the acquisition device, and then the point cloud is aligned based on the optimized acquisition pose.

[0004] The ICP algorithm relies on the assumption that the point cloud within a region is a rigid body. However, during point cloud acquisition, complex phenomena such as multiple refractions and reflections occur in the real environment, meaning the assumption of a rigid point cloud is not always valid. When non-rigid point clouds exist, the point cloud alignment results using the ICP algorithm are affected by these non-rigid point clouds, leading to poor alignment performance. Therefore, the accuracy of point cloud alignment achieved through the above methods is low. Summary of the Invention

[0005] This application provides a point cloud alignment method, apparatus, device, and storage medium, which can improve the accuracy of point cloud alignment. The technical solution is as follows:

[0006] According to one aspect of this application, a point cloud alignment method is provided, the method comprising:

[0007] Obtain a first point cloud and a second point cloud, where the points in the first point cloud and the second point cloud intersect;

[0008] Based on the calibration operation, at least one pair of corresponding points are calibrated in the first point cloud and the second point cloud, wherein the pair of corresponding points includes a first point in the first point cloud and a second point in the second point cloud;

[0009] Based on the positional difference between the first point and the second point in the at least one pair of identical point pairs, the first acquisition pose and the second acquisition pose are optimized. The first acquisition pose includes the pose of the first acquisition device when acquiring the first point cloud, and the second acquisition pose includes the pose of the second acquisition device when acquiring the second point cloud.

[0010] Based on the optimized first acquisition pose and the optimized second pose, the first point cloud and the second point cloud are aligned.

[0011] According to another aspect of this application, a point cloud alignment apparatus is provided, the apparatus comprising:

[0012] The acquisition module is used to acquire a first point cloud and a second point cloud, wherein the points in the first point cloud and the second point cloud intersect;

[0013] The calibration module is used to calibrate at least one pair of corresponding points in the first point cloud and the second point cloud based on calibration operations, wherein the pair of corresponding points includes a first point in the first point cloud and a second point in the second point cloud;

[0014] The optimization module is used to optimize the first acquisition pose and the second acquisition pose based on the positional difference between the first point and the second point in the at least one pair of corresponding point pairs. The first acquisition pose includes the pose of the first acquisition device when acquiring the first point cloud, and the second acquisition pose includes the pose of the second acquisition device when acquiring the second point cloud.

[0015] The alignment module is used to align the first point cloud and the second point cloud based on the optimized first acquisition pose and the optimized second optimized pose.

[0016] In an optional design, the optimization module includes:

[0017] The projection submodule is used to project the first point in the at least one pair of corresponding point pairs onto the world coordinate system to obtain the first position of the first point in the world coordinate system; the projection submodule is also used to project the second point in the at least one pair of corresponding point pairs onto the world coordinate system to obtain the second position of the second point in the world coordinate system.

[0018] The optimization module is used to optimize the first acquisition pose and the second acquisition pose with the constraint that the positional difference between the first position and the second position of the at least one pair of corresponding points is minimized.

[0019] In an optional design, the optimization module is used for:

[0020] A first pose map of the first acquisition device is constructed based on the first acquisition pose of the first point cloud at each moment; a second pose map of the second acquisition device is constructed based on the second acquisition pose of the second point cloud at each moment.

[0021] The first residual function is determined based on the difference between the first position and the second position of the at least one pair of corresponding points;

[0022] The first pose graph and the second pose graph are optimized using the optimization objective of the first residual function as a constraint.

[0023] The first residual function is a residual function whose optimization objective is to minimize the sum of the differences between the first position and the second position of the at least one pair of corresponding points.

[0024] In an optional design, the optimization module is used for:

[0025] Using the optimization objective of the first residual function as a constraint, the first acquisition trajectory in the first pose graph is optimized, and the second acquisition trajectory in the second pose graph is optimized.

[0026] The first acquisition trajectory includes the acquisition pose of the first acquisition device corresponding to the first point, and the trajectory formed by the acquisition pose of the first acquisition device when acquiring points other than the first point in the first point cloud according to the time sequence; the second acquisition trajectory includes the acquisition pose of the second acquisition device corresponding to the second point, and the trajectory formed by the acquisition pose of the second acquisition device when acquiring points other than the second point in the second point cloud according to the time sequence.

[0027] In an optional design, the projection submodule is used for:

[0028] Obtain the acquisition pose of the first acquisition device under the first point cloud timestamp, where the first point cloud timestamp is the timestamp of the acquisition of the first point;

[0029] The first position is determined based on the position of the first point in the first point cloud and the acquisition pose of the first acquisition device under the timestamp of the first point cloud;

[0030] Obtain the acquisition pose of the second acquisition device under the second point cloud timestamp, where the second point cloud timestamp is the timestamp of the acquisition of the second point;

[0031] The second position is determined based on the position of the second point in the second point cloud and the acquisition pose of the second acquisition device under the timestamp of the second point cloud.

[0032] In an optional design, the projection submodule is used for:

[0033] Based on the first RTK calculated pose adjacent to the first point cloud timestamp, the acquisition pose of the first acquisition device under the first point cloud timestamp is determined by interpolation. The first RTK calculated pose is the acquisition pose of the first acquisition device when acquiring the first point cloud, which is calculated based on RTK technology.

[0034] Based on the second RTK calculated pose adjacent to the second point cloud timestamp, the acquisition pose of the second acquisition device under the second point cloud timestamp is determined by interpolation. The second RTK calculated pose is the acquisition pose of the second acquisition device when acquiring the second point cloud, which is calculated based on the RTK technology.

[0035] In an optional design, the projection submodule is used for:

[0036] Obtain the confidence level of the acquisition pose of the first acquisition device under the first point cloud timestamp;

[0037] The first position is determined based on the position of the first point in the first point cloud and the first weighted pose of the first acquisition device under the timestamp of the first point cloud. The first weighted pose is determined by the product of the acquisition pose of the first acquisition device under the timestamp of the first point cloud and the first weight. The first weight is positively correlated with the confidence level of the acquisition pose of the first acquisition device under the timestamp of the first point cloud.

[0038] Obtain the confidence level of the acquisition pose of the second acquisition device under the second point cloud timestamp;

[0039] The second position is determined based on the position of the second point in the second point cloud and the second weighted pose of the second acquisition device under the second point cloud timestamp. The second weighted pose is determined by the product of the acquisition pose of the second acquisition device under the second point cloud timestamp and the second weight. The second weight is positively correlated with the confidence level of the acquisition pose of the second acquisition device under the second point cloud timestamp.

[0040] In an optional design, the optimization module is used for:

[0041] The first pose graph and the second pose graph are optimized based on minimizing the sum of the first residual function and the second residual function.

[0042] The second residual function is an optimization objective that minimizes the sum of the differences between the first gravity direction and the second gravity direction of the first acquisition device, and minimizes the sum of the differences between the first gravity direction and the second gravity direction of the second acquisition device. The first gravity direction of the first acquisition device is the measured gravity direction when the first acquisition device acquires the first point cloud, and the second gravity direction of the first acquisition device is the gravity direction calculated based on the acquisition pose when the first acquisition device acquires the first point cloud. The first gravity direction of the second acquisition device is the measured gravity direction when the second acquisition device acquires the second point cloud, and the second gravity direction of the second acquisition device is the gravity direction calculated based on the acquisition pose when the second acquisition device acquires the second point cloud.

[0043] In an optional design, the device further includes:

[0044] The determination module is used to determine the second residual function based on the sum of the cross product results of the first gravity direction and the second gravity direction at each moment of the first point cloud collected by the first acquisition device, and the sum of the cross product results of the first gravity direction and the second gravity direction at each moment of the second point cloud collected by the second acquisition device.

[0045] In an optional design, the first gravity direction of the first acquisition device includes the gravity direction of the first acquisition device at each moment of acquiring the first point cloud, measured based on RTK technology; the second gravity direction of the first acquisition device includes the gravity direction of the first acquisition device at each moment of acquiring the first point cloud, calculated based on a first rotation matrix and the true gravity direction; the first rotation matrix is ​​obtained by solving based on the RTK technology; the first rotation matrix is ​​used to reflect the rotational change of the gravity direction of the first acquisition device relative to the true gravity direction at each moment of acquiring the first point cloud.

[0046] The first gravity direction of the second acquisition device includes the gravity direction of the second acquisition device at each moment of acquiring the second point cloud, measured based on the RTK technology. The second gravity direction of the second acquisition device includes the gravity direction of the second acquisition device at each moment of acquiring the second point cloud, calculated based on the second rotation matrix and the true gravity direction. The second rotation matrix is ​​obtained by solving based on the RTK technology and is used to reflect the rotational change of the gravity direction of the second acquisition device relative to the true gravity direction at each moment of acquiring the second point cloud.

[0047] In an optional design, the alignment module is used for:

[0048] The first point cloud is projected onto the world coordinate system based on the optimized first acquisition pose, and the second point cloud is projected onto the world coordinate system based on the optimized second acquisition pose, to obtain an aligned first point cloud and second point cloud.

[0049] In an optional design, the device further includes:

[0050] The display module is used to render and display the source point cloud in the user interface, wherein the source point cloud includes the first point cloud and the second point cloud;

[0051] The calibration module is used for:

[0052] In response to a calibration operation on points in the displayed source point cloud, the point pairs calibrated by the calibration operation are obtained as the at least one pair of points with the same name.

[0053] According to another aspect of this application, a computer device is provided, the computer device including a processor and a memory, the memory storing at least one instruction, at least one program, code set or instruction set, the at least one instruction, the at least one program, the code set or instruction set being loaded and executed by the processor to implement the point cloud alignment method as described above.

[0054] According to another aspect of this application, a computer-readable storage medium is provided, wherein at least one instruction, at least one program, code set, or instruction set is stored therein, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the point cloud alignment method as described above.

[0055] According to another aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the point cloud alignment method provided in various alternative implementations of the above aspects.

[0056] The beneficial effects of the technical solution provided in this application include at least the following:

[0057] By using at least one pair of calibrated corresponding points, and based on the positional differences of the points in the corresponding point pairs, the first acquisition pose of the first acquisition device and the second acquisition pose of the second acquisition device can be optimized. The optimized first and second acquisition poses enable point cloud alignment of the first and second point clouds. During point cloud alignment, by calibrating corresponding point pairs for non-rigid point clouds, the non-rigid point clouds can be divided into regions based on the calibrated corresponding point pairs, thereby aligning the non-rigid point clouds by region. Since local point clouds can be considered as rigid bodies, it is not necessary to consider whether the point clouds are rigid bodies; the point cloud alignment result is not affected by non-rigid point clouds. Furthermore, even when objects in the acquisition environment move, the calibrated corresponding point pairs can be used to align the point clouds corresponding to objects that have not moved, thus also eliminating the need to consider whether objects in the environment are moving. This improves the accuracy of the point cloud alignment result. Attached Figure Description

[0058] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0059] Figure 1 This is a schematic diagram of a point cloud provided in an exemplary embodiment of this application;

[0060] Figure 2 This is a schematic diagram of a point cloud provided in an exemplary embodiment of this application;

[0061] Figure 3 This is a schematic diagram of an in-vehicle system provided in an exemplary embodiment of this application;

[0062] Figure 4 This is a schematic diagram of a point cloud acquisition process provided in an exemplary embodiment of this application;

[0063] Figure 5 This is a schematic diagram of a point cloud acquisition process provided in an exemplary embodiment of this application;

[0064] Figure 6 This is a schematic diagram illustrating the process of aligning point clouds provided in an exemplary embodiment of this application;

[0065] Figure 7 This is a flowchart illustrating a point cloud alignment method provided in an exemplary embodiment of this application;

[0066] Figure 8This is a flowchart illustrating a point cloud alignment method provided in an exemplary embodiment of this application;

[0067] Figure 9 This is a schematic diagram of a user interface provided in an exemplary embodiment of this application;

[0068] Figure 10 This is a schematic diagram of the acquisition pose provided in an exemplary embodiment of this application;

[0069] Figure 11 This is a schematic diagram of the acquisition pose provided in an exemplary embodiment of this application;

[0070] Figure 12 This is a schematic diagram illustrating the interpolation-based determination of the acquisition pose provided in an exemplary embodiment of this application;

[0071] Figure 13 This is a schematic diagram of a data acquisition pose based on gravity direction constraint optimization provided in an exemplary embodiment of this application;

[0072] Figure 14 This is a schematic diagram of an optimized acquisition pose provided by an exemplary embodiment of this application;

[0073] Figure 15 This is a schematic diagram of a user interface provided in an exemplary embodiment of this application;

[0074] Figure 16 This is a schematic diagram of a high-precision map provided in an exemplary embodiment of this application;

[0075] Figure 17 This is a schematic diagram of the structure of a point cloud alignment device provided in an exemplary embodiment of this application;

[0076] Figure 18 This is a schematic diagram of the structure of an optimized module provided in an exemplary embodiment of this application;

[0077] Figure 19 This is a schematic diagram of the structure of a point cloud alignment device provided in an exemplary embodiment of this application;

[0078] Figure 20 This is a schematic diagram of the structure of a point cloud alignment device provided in an exemplary embodiment of this application;

[0079] Figure 21 This is a schematic diagram of the structure of a computer device provided in an exemplary embodiment of this application.

[0080] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. Detailed Implementation

[0081] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0082] First, let's introduce the terms used in the embodiments of this application:

[0083] Point cloud: refers to a set of vectors in a three-dimensional coordinate system. These vectors are usually represented in the form of three-dimensional coordinates (x, y, z), and are generally used to represent the shape of the outer surface of an object. For example, P k ={x k y k , z k} represents the k-th point in space, and the point cloud = {P1, P2, P3, ..., P...} v} represents a set of point cloud data, where k and v are positive integers. For example, the point cloud above was acquired by an acquisition device with point cloud acquisition capabilities, such as a 3D scanning device, like a LiDAR.

[0084] The acquisition device scans the environment at a certain frequency to obtain multiple frames of point clouds. The computer then accumulates the acquired point clouds frame by frame to form the final point cloud. For each frame of point cloud, the acquisition pose of the acquisition device at the time of acquisition is the basis for stitching this frame of point cloud with other point clouds.

[0085] Pose: An object has six degrees of freedom in space, namely, the translational degrees of freedom along the three Cartesian coordinate axes (x, y, z) and the rotational degrees of freedom about these three coordinate axes, namely the yaw angle, pitch angle, and roll angle. In the embodiments of this application, the acquisition pose of the acquisition device refers to the pose of the acquisition device in the world coordinate system.

[0086] Real-time Kinematics (RTK) technology, also known as carrier phase differential technology, provides real-time 3D positioning results of a device in a specified coordinate system with centimeter-level accuracy. In RTK, a base station collects satellite data and transmits its observations and station coordinates to a mobile station (the device being positioned) via a data link. The mobile station then performs real-time carrier phase differential processing (in less than one second) on the collected satellite data and the received data to obtain centimeter-level positioning results. Differential processing aims to separate the errors of the Global Positioning System (GPS). By installing a mobile base station at a reference point with a known location, the deviation of the positioning signal can be determined. This deviation is then sent to the mobile station, which obtains more accurate location information.

[0087] In the process of building high-precision maps, it is necessary to collect point clouds corresponding to roads using data acquisition devices. For example, when collecting point clouds corresponding to roads, the acquisition devices are deployed on vehicles, and the vehicles collect point clouds as they travel on the roads. Since the acquisition range of these devices is limited, each acquisition can only collect point clouds corresponding to a portion of the road area. Therefore, multiple acquisitions are required, and the point clouds collected from these multiple acquisitions need to be merged. Due to errors in the pose of the measurement devices and time errors at different acquisition times, points collected from different batches of point clouds may appear at different positions when projected onto the world coordinate system, resulting in point cloud misalignment.

[0088] In related technologies, the ICP algorithm is used to optimize the pose constraints (including rotation and translation) of point clouds, and point cloud alignment is performed based on the optimized pose constraints. The ICP algorithm assumes that the point cloud within the region is a rigid body. However, during point cloud acquisition, complex phenomena such as multiple refractions and reflections occur in the real environment, meaning the assumption of a rigid point cloud is not always valid. This leads to low accuracy in point cloud alignment achieved through this method. Furthermore, during multiple point cloud acquisitions, some objects in the environment may move. The ICP algorithm is affected by this movement, resulting in poor alignment performance as it cannot filter out the impact of object movement on point cloud alignment.

[0089] For example, Figure 1 This is a schematic diagram of a point cloud provided in an exemplary embodiment of this application. For example... Figure 1 As shown, in the point cloud 101 obtained from multiple acquisitions, the point cloud obtained from multiple acquisitions is distorted due to multiple refractions and reflections of objects in the road.

[0090] For example, Figure 2 This is a schematic diagram of a point cloud provided in an exemplary embodiment of this application. For example... Figure 2 As shown, in the point cloud 201 obtained from multiple acquisitions, the point cloud obtained from multiple acquisitions is misaligned because the boom of the tower crane moves (rotates) during the multiple acquisitions.

[0091] The point cloud alignment method provided in this application can optimize the first acquisition pose of the first acquisition device when acquiring the first point cloud and the second acquisition pose of the second acquisition device when acquiring the second point cloud based on at least one pair of corresponding points calibrated in the first and second point clouds. The optimized acquisition poses can then be used to align the first and second point clouds. This process does not require consideration of whether the point clouds are rigid bodies or whether objects in the environment are moving.

[0092] Figure 3 This is a schematic diagram of an in-vehicle system provided in an exemplary embodiment of this application. The in-vehicle system includes: a Global Navigation Satellite System (GNSS) 110, a lidar radar 120, and a host computer 130 deployed in the vehicle. The host computer 130 is a computer device.

[0093] During the point cloud acquisition and pose determination process, GNSS 110 is used to determine the vehicle's pose at various moments during point cloud acquisition using RTK technology. LiDAR 120 is used to scan the surrounding environment within a preset radius to acquire the corresponding point cloud data.

[0094] The GNSS 110 and LiDAR 120 are respectively connected to the host computer 130, and the acquired data is transmitted to the host computer 130 through these communication connections. The host computer 130 constructs a point cloud corresponding to the environment based on the data acquired by the GNSS 110 and LiDAR 120. For example, during the point cloud construction process, the host computer 130 executes the point cloud alignment method provided in this application embodiment to align the constructed point cloud, obtaining an accurate aligned point cloud. A high-precision map is then constructed based on the aligned point cloud, thereby improving the accuracy of the high-precision map.

[0095] Taking the acquisition of point clouds corresponding to roads as an example, the process of acquiring point clouds via the aforementioned vehicles will be explained. Multiple acquisitions are required during the acquisition of point clouds corresponding to roads. Therefore, the vehicle needs to travel multiple times on the road. For example, multiple travels include traveling in different lanes traveling in the same direction on the same road, and traveling in the opposite lane on the same road.

[0096] For example, Figure 4 This is a schematic diagram illustrating the point cloud acquisition process provided in an exemplary embodiment of this application. For example... Figure 4 As shown, the road consists of four lanes traveling in the same direction. During the point cloud acquisition process, vehicle 401 needs to travel once on the left side of the road to acquire the first point cloud. Vehicle 401 also needs to travel once on the right side of the road to acquire the second point cloud. The first and second point clouds are then projected onto the world coordinate system to obtain the point cloud corresponding to the road.

[0097] For example, Figure 5 This is a schematic diagram illustrating the point cloud acquisition process provided in an exemplary embodiment of this application. For example... Figure 5As shown, the road includes lanes in two directions, each with two lanes traveling in the same direction. During the acquisition of the point cloud corresponding to the road, vehicle 501 needs to travel once in the first direction to acquire the first point cloud. Vehicle 501 also needs to travel once in the second direction to acquire the second point cloud. The first and second directions are opposite directions on the road. The first and second point clouds are then projected onto the world coordinate system to obtain the point cloud corresponding to the road. During the acquisition of the first and second point clouds while traveling in opposite directions, the lidar in vehicle 501 acquires the point clouds of the opposite sides of the road environment. For example, the point clouds of the opposite sides of a traffic sign. Additionally, the lidar in vehicle 501 supports rotation, thus it can also acquire the point cloud of the same side of the road environment.

[0098] It should be noted that during multiple data collection sessions, the vehicles used in different sessions may be the same or different, the lidar used in different sessions may be the same or different, and the vehicle speeds may be the same or different during different sessions. This application does not impose limitations on these aspects.

[0099] Figure 6 This is a schematic diagram illustrating a process for aligning point clouds according to an exemplary embodiment of this application. For example... Figure 6 As shown, in step S1, the computer device acquires a source point cloud. Optionally, the source point cloud includes a first point cloud and a second point cloud, and the points in the first and second point clouds intersect. After acquiring the source point cloud, the computer device can display the source point cloud in the user interface.

[0100] In step S2, the computer device obtains the result of a manual assessment of whether the displayed source point cloud is aligned. If the source point cloud is aligned, the process ends. If it is not aligned, step S3 is executed.

[0101] In step S3, the computer device acquires at least one pair of points with the same name manually marked in the displayed source point cloud. This pair of points includes a first point in the first point cloud and a second point in the second point cloud. For example, in the displayed source point cloud, a pair of points with the same name is manually marked; this pair consists of the points in the first and second point clouds corresponding to the same points in the world coordinate system as determined by the manual.

[0102] In step S4, the computer device acquires the first acquisition pose of the first acquisition device acquiring the first point cloud, and the second acquisition pose of the second acquisition device acquiring the second point cloud. The first acquisition pose is the pose of the first acquisition device in the world coordinate system when acquiring the first point cloud, and the second acquisition pose is the pose of the second acquisition device in the world coordinate system when acquiring the second point cloud. Optionally, the first and second acquisition poses are obtained based on RTK technology. The acquisition poses form the acquisition trajectory of the acquisition device when acquiring the source point cloud.

[0103] In step S5, the computer device constructs a first residual function based on the positional difference between the first and second points in the same point pair in the world coordinate system. It also constructs a second residual function based on the difference between the measured and calculated gravity direction when the first acquisition device acquires the first point cloud, and the difference between the measured and calculated gravity direction when the second acquisition device acquires the second point cloud. The calculated gravity direction is determined based on the actual gravitational acceleration and the rotation matrix when the acquisition device acquires the point cloud. The rotation matrix reflects the rotational change of the measured gravity direction relative to the actual gravity direction when the acquisition device acquires the point cloud. Then, based on minimizing the sum of the first and second residual functions as the optimization objective, the first acquisition pose of the first acquisition device and the second acquisition pose of the second acquisition device are optimized to obtain optimized first and second acquisition poses. Finally, the first point cloud is reprojected to the world coordinate system based on the optimized first acquisition pose, and the second point cloud is reprojected to the world coordinate system based on the optimized second acquisition pose, thus obtaining the source point cloud after point cloud alignment processing.

[0104] Based on at least one pair of calibrated corresponding points, and according to the positional differences of the points in the corresponding point pairs, the first acquisition pose of the first acquisition device and the second acquisition pose of the second acquisition device can be optimized. The optimized first and second acquisition poses enable point cloud alignment between the first and second point clouds. During point cloud alignment, it is not necessary to consider whether the point cloud is a rigid body, nor is it necessary to consider whether objects in the environment are moving. This improves the accuracy of the point cloud alignment results.

[0105] Figure 7 This is a flowchart illustrating a point cloud alignment method provided in an exemplary embodiment of this application. This method can be used on a computer device or a client on a computer device. Figure 7 As shown, the method includes:

[0106] Step 702: Obtain the first point cloud and the second point cloud.

[0107] The first point cloud is the point cloud acquired by the first acquisition device. The second point cloud is the point cloud acquired by the second acquisition device. The points in the first and second point clouds intersect. The acquisition object corresponding to the first point cloud intersects with the acquisition object corresponding to the second point cloud. That is, during the acquisition of the first and second point clouds, both acquisition devices acquired the same acquisition object. The first and second point clouds can form a source point cloud. By projecting the first point cloud onto the world coordinate system using the acquisition pose of the first acquisition device when acquiring the first point cloud (before optimization), and by projecting the second point cloud onto the world coordinate system using the acquisition pose of the second acquisition device when acquiring the second point cloud (before optimization), the source point cloud can be obtained.

[0108] The first and second data acquisition devices mentioned above are lidar. Optionally, the first data acquisition device is deployed on a vehicle, and it acquires a first point cloud while the vehicle is traveling on the road. The second data acquisition device is also deployed on a vehicle, and it acquires a second point cloud while the vehicle is traveling on the road. The first and second data acquisition devices may be the same or different, and the vehicle on which the first and second data acquisition devices are located may be the same or different. During the point cloud acquisition process, the driving parameters of the vehicle on which the first and second data acquisition devices are located may be the same or different.

[0109] Optionally, the computer device can also acquire more point clouds. For example, it can acquire a first point cloud, a second point cloud, and a third point cloud, and then align the first point cloud, the second point cloud, and the third point cloud in a manner that aligns the first point cloud and the second point cloud. This application embodiment does not limit the number of point clouds to be aligned.

[0110] Step 704: Based on the calibration operation, calibrate at least one pair of points with the same name in the first point cloud and the second point cloud.

[0111] This calibration operation refers to a manual operation. When a manual determination indicates that the first point cloud and the second point cloud are misaligned, a manual calibration operation will be used to calibrate the corresponding point pair. This corresponding point pair includes the first point in the first point cloud and the second point in the second point cloud. Points in the corresponding point pair are misaligned points in both the first and second point clouds. Points in the corresponding point pair are semantically identical points in both the first and second point clouds. Optionally, the corresponding point pair can also include more points calibrated in the first and second point clouds by the calibration operation. A corresponding point pair is a manually determined point that is the same point in the world coordinate system but not the same point in the source point cloud; the non-identical points belong to the first and second point clouds respectively. For example, the first point cloud and the second point cloud include points collected from the same sign, but the points in the lower left part of the license plate in the source point cloud are misaligned with the points in the second point cloud. In this case, a person can select the points in the first point cloud and the points in the second point cloud corresponding to the lower left corner of the license plate as a pair of points with the same name.

[0112] Optionally, after acquiring the source point cloud, the computer device can display the source point cloud in a user interface. The user interface provides point cloud alignment functionality. By observing the source point cloud displayed in the user interface, a person can determine whether the source point cloud is aligned. In the case of misalignment, through calibration operations on the source point cloud displayed in the user interface, the computer device can calibrate at least one pair of corresponding points in the first and second point clouds.

[0113] Optionally, the above calibration operation can also be implemented using a machine learning model. This model identifies misaligned points in the input point cloud, thereby triggering the calibration operation. The computer device then labels the identified misaligned points as pairs of corresponding points. This machine learning model is trained based on aligned sample point clouds.

[0114] Step 706: Optimize the first acquisition pose and the second acquisition pose based on the positional difference between the first point and the second point in at least one pair of identical point pairs.

[0115] The first acquisition pose includes the pose of the first acquisition device when acquiring the first point cloud, and the second acquisition pose includes the pose of the second acquisition device when acquiring the second point cloud. The first acquisition pose can include the pose of the first acquisition device at various times or at some times when acquiring the first point cloud, and the second acquisition pose can include the pose of the second acquisition device at various times or at some times when acquiring the second point cloud. The first acquisition pose and the second acquisition pose are poses in the world coordinate system.

[0116] When merging the first and second point clouds into a source point cloud, both the first and second acquisition poses are used. By minimizing the positional difference between the first and second points in the world coordinate system as the optimization objective, the first and second acquisition poses can be optimized. This ensures that the first and second points are aligned in the world coordinate system. The acquisition trajectory is formed by the continuous movement of the acquisition device and is composed of acquisition poses, with correlations between the various acquisition poses within the trajectory. This method not only optimizes the acquisition poses of the first and second acquisition devices when acquiring the first point and the second point, but also optimizes the acquisition poses of the first and second acquisition devices when acquiring other points.

[0117] Optionally, based on the above optimization objectives, the computer device can further optimize the first acquisition pose and the second acquisition pose by minimizing the difference between the first and second gravity directions of the first acquisition device and the second acquisition device. Here, the first gravity direction of the first acquisition device is the measured gravity direction when the first acquisition device acquires the first point cloud, and the second gravity direction of the first acquisition device is the gravity direction calculated based on the acquisition pose of the first acquisition device when acquiring the first point cloud. Similarly, the first gravity direction of the second acquisition device is the measured gravity direction when the second acquisition device acquires the second point cloud, and the second gravity direction of the second acquisition device is the gravity direction calculated based on the acquisition pose of the second acquisition device when acquiring the second point cloud.

[0118] Step 708: Align the first point cloud and the second point cloud according to the optimized first acquisition pose and the optimized second pose.

[0119] The computer device projects the first point cloud onto the world coordinate system based on the optimized first acquisition pose, and projects the second point cloud onto the world coordinate system based on the optimized second acquisition pose, thereby enabling point cloud alignment of the first and second point clouds.

[0120] It should be noted that the aforementioned computer equipment is capable of providing Figure 3 The host computer in the vehicle-mounted system shown. This computer device can also be a server or terminal. The server or terminal acquires relevant data collected by the vehicle-mounted system, thereby enabling it to perform the point cloud alignment method through the steps described above.

[0121] In summary, the method provided in this embodiment optimizes the first acquisition pose of the first acquisition device and the second acquisition pose of the second acquisition device based on the positional differences of the points in the calibrated at least one pair of corresponding points. The optimized first and second acquisition poses enable point cloud alignment of the first and second point clouds. During point cloud alignment, calibrating corresponding point pairs on non-rigid point clouds allows for region division of the non-rigid point clouds based on these pairs, thus enabling point cloud alignment by region. Since local point clouds can be considered rigid, there is no need to consider whether the point clouds are rigid, and the point cloud alignment result is not affected by non-rigid point clouds. Furthermore, even when objects in the acquisition environment move, the calibrated corresponding point pairs enable point cloud alignment for objects that have not moved, thus eliminating the need to consider object movement. This improves the accuracy of the point cloud alignment result.

[0122] Figure 8 This is a flowchart illustrating a point cloud alignment method provided in an exemplary embodiment of this application. This method can be used on a computer device or a client on a computer device. Figure 8 As shown, the method includes:

[0123] Step 802: Render and display the source point cloud in the user interface. The source point cloud includes the first point cloud and the second point cloud.

[0124] Computer equipment acquires a first point cloud and a second point cloud, and projects them onto a world coordinate system to obtain a source point cloud. The points in the first and second point clouds intersect. The objects acquired in the first and second point clouds also intersect. That is, during the acquisition of the first and second point clouds, both acquisition devices acquired the same objects. This user interface provides point cloud alignment functionality.

[0125] Optionally, the source point cloud may also include more point clouds. For example, the source point cloud may include a first point cloud, a second point cloud, and a third point cloud. The computer device, according to the method provided in the embodiments of this application, can perform point cloud alignment on the first point cloud, the second point cloud, and the third point cloud.

[0126] For example, Figure 9 This is a schematic diagram of a user interface provided in an exemplary embodiment of this application. Figure 9 As shown, the computer device displays a user interface 901, which displays a source point cloud 902. The user interface 901 also displays controls that trigger point cloud alignment of the source point cloud 902.

[0127] Step 804: In response to the calibration operation for points in the displayed source point cloud, obtain the point pairs calibrated by the calibration operation as at least one pair of points with the same name.

[0128] Based on the aforementioned calibration operation, the computer equipment can calibrate at least one pair of corresponding points in the first point cloud and the second point cloud. This calibration operation refers to a manual operation. The corresponding point pair includes a first point in the first point cloud and a second point in the second point cloud. Points in the corresponding point pair are misaligned points in the first and second point clouds. Points in the corresponding point pair are semantically identical points in both the first and second point clouds. Optionally, the corresponding point pair can also include more points calibrated by the calibration operation in the first and second point clouds. A corresponding point pair is a manually determined point that is the same point in the world coordinate system but not the same point in the source point cloud; the non-identical points belong to the first point cloud and the second point cloud respectively.

[0129] By observing the source point cloud displayed in the user interface, a human can determine whether the source point cloud is aligned. In the case of misalignment, through calibration operations on the source point cloud displayed in the user interface, the computer device can calibrate at least one pair of corresponding points in the first and second point clouds.

[0130] For example, continue to refer to Figure 9 The user interface 901 displays a corresponding point pair marking control 903. Upon receiving a trigger operation on the corresponding point pair marking control 903, the computer device can obtain the point pairs marked by the calibration operation as at least one corresponding point pair, based on the user's calibration operation on the points in the displayed source point cloud 902. The user interface 901 also displays a corresponding point list 604, which is used to display the calibrated corresponding point pairs and information about the points in the corresponding point pairs.

[0131] Step 806: Optimize the first acquisition pose and the second acquisition pose based on the positional difference between the first point and the second point in at least one pair of identical point pairs.

[0132] The first acquisition pose includes the pose of the first acquisition device when acquiring the first point cloud, and the second acquisition pose includes the pose of the second acquisition device when acquiring the second point cloud. Optionally, the positional difference between the first point and the second point refers to the positional difference between the first point and the second point in the world coordinate system.

[0133] The computer device projects the first point from at least one pair of corresponding point pairs onto the world coordinate system, thereby obtaining the first position of the first point in the world coordinate system. It also projects the second point from at least one pair of corresponding point pairs onto the world coordinate system, thereby obtaining the second position of the second point in the world coordinate system. Then, by minimizing the positional difference between the first and second positions of at least one pair of corresponding point pairs, the first and second acquisition poses can be optimized.

[0134] The process of optimizing the acquired pose using the first residual function:

[0135] Optionally, the optimization of the first and second acquisition poses is achieved by optimizing the first pose graph and the second pose graph. The computer device constructs the first pose graph of the first acquisition device based on the first acquisition pose at each moment of acquiring the first point cloud. Similarly, it constructs the second pose graph of the second acquisition device based on the second acquisition pose at each moment of acquiring the second point cloud. Optionally, the first and second acquisition poses are obtained through RTK technology. The computer device determines a first residual function based on the difference between the first and second positions of at least one pair of corresponding point pairs. Then, the first pose graph and the second pose graph are optimized using the optimization objective of the first residual function as a constraint. The first residual function is a residual function whose optimization objective is to minimize the sum of the differences between the first and second positions of at least one pair of corresponding point pairs.

[0136] For example, the expression for the first residual function is as follows:

[0137]

[0138] Where p1 represents the position of the first point in the point cloud coordinate system of the first point cloud, and the point cloud coordinate system of the first point cloud is a coordinate system centered on the first acquisition device. It is determined based on the acquisition pose of the first acquisition device when acquiring the first point. p1 represents the position of the first point in the world coordinate system. p2 represents the position of the second point in the point cloud coordinate system of the second point cloud, which is a coordinate system centered on the second acquisition device. It is determined based on the acquisition pose of the second acquisition device when acquiring the second point. The location of the second point in the world coordinate system is shown below. Optionally, when projecting the first and second points into the world coordinate system, since the first acquisition device is mounted on the vehicle and the measured acquisition pose is the vehicle's pose, to improve accuracy, the vehicle's acquisition pose needs to be converted from the vehicle's coordinate system to the acquisition device's coordinate system, thus obtaining the acquisition pose of the acquisition device projected onto the first point. The vehicle coordinate system is a coordinate system centered on the vehicle. The acquisition device's coordinate system is a coordinate system centered on the acquisition device, i.e., the point cloud coordinate system. The processing of the second point is similar.

[0139] When multiple pairs of corresponding points exist, the first residual function is determined based on the sum of the positional differences between the points in each pair. In this case, the expression for the first residual function is:

[0140] Furthermore, when a set of points of the same name includes more than two points, the first residual function is determined based on the sum of the differences between each pair of points in the world coordinate system.

[0141] In the first point cloud, the first point may be acquired multiple times. For example, the first point may be acquired at acquisition time 1, acquisition time 2, and acquisition time 3. In the second point cloud, the second point may be acquired multiple times, for example, at acquisition time a and acquisition time b. Furthermore, the pose at each acquisition may differ, thus the position in the world coordinate system may also differ. In this case, the computer device will cross-subtract the positions of the points acquired multiple times in the world coordinate system to obtain the first residual function. For example, the position of the first point in the first point cloud is x, and the position of the second point in the second point cloud is y. The acquisition pose of the first acquisition device at acquisition time 1 is T1, the acquisition pose at acquisition time 2 is T2, and the acquisition pose at acquisition time 3 is T3. The acquisition pose of the second acquisition device at acquisition time a is Ta, and the acquisition pose at acquisition time b is Tb. Then the difference between the positions of the first point and the second point in the world coordinate system is (x*T1-y*Ta)+(x*T2-y*Ta)+(x*T3-y*Ta)+(x*T1-y*Tb)+(x*T2-y*Tb)+(x*T3-y*Tb).

[0142] In optimizing the first and second pose maps, the computer device optimizes the first acquisition trajectory in the first pose map and the second acquisition trajectory in the second pose map, using the optimization objective of the first residual function as a constraint. The first acquisition trajectory includes the acquisition pose of the first acquisition device corresponding to the first point, and the trajectory formed sequentially by the acquisition poses of the first acquisition device when acquiring points other than the first point in the first point cloud. That is, the acquisition trajectory of the first acquisition device when acquiring the first point cloud. The second acquisition trajectory includes the acquisition pose of the second acquisition device corresponding to the second point, and the trajectory formed sequentially by the acquisition poses of the second acquisition device when acquiring points other than the second point in the second point cloud. That is, the acquisition trajectory of the second acquisition device when acquiring the second point cloud. The acquisition trajectory is formed by the continuous movement of the acquisition device, and is composed of acquisition poses, with correlations between the acquisition poses in the trajectory. This method not only optimizes the acquisition poses of the first acquisition device when acquiring the first point and the second device when acquiring the second point, but also optimizes the acquisition poses of the first and second acquisition devices when acquiring other points.

[0143] The process of projecting points from a point cloud onto the world coordinate system:

[0144] During the projection of the first point onto the world coordinate system, the computer device acquires the acquisition pose of the first acquisition device under the first point cloud timestamp, and determines the first position of the first point in the world coordinate system based on the position of the first point in the first point cloud and the acquisition pose of the first acquisition device under the first point cloud timestamp. This first point cloud timestamp is the timestamp for acquiring the first point. During the projection of the second point onto the world coordinate system, the computer device acquires the acquisition pose of the second acquisition device under the second point cloud timestamp, and determines the second position based on the position of the second point in the second point cloud and the acquisition pose of the second acquisition device under the second point cloud timestamp. This second point cloud timestamp is the timestamp for acquiring the second point.

[0145] For example, Figure 10 This is a schematic diagram illustrating the acquisition pose provided in an exemplary embodiment of this application. For example... Figure 10 As shown, the first acquisition device acquires a first point cloud in the uplink trajectory 1001, and the second acquisition device acquires a second point cloud in the downlink trajectory 1002. The uplink and downlink trajectories are opposite travel trajectories. During the acquisition process, both the first and second acquisition devices acquire point clouds corresponding to points P1, P2, and P3. If point P1 acquired in the first and second point clouds is selected as a pair of points with the same name, the computer device will use the acquisition pose of the first acquisition device when acquiring point P1 to project the point corresponding to point P1 in the first point cloud to the world coordinate system, and use the acquisition pose of the second acquisition device when acquiring point P1 to project the point corresponding to point P1 in the second point cloud to the world coordinate system.

[0146] Optionally, since the number of satellites used in measuring the acquisition pose of the acquisition device varies, the acquisition pose at different times corresponds to different confidence levels. Confidence level is used to reflect the reliability of the measurement results. The confidence level of the acquisition pose also needs to be considered during the projection of the first and second points.

[0147] The computer device acquires the confidence level of the acquisition pose of the first acquisition device under the first point cloud timestamp. Based on the position of the first point in the first point cloud and the first weighted pose of the first acquisition device under the first point cloud timestamp, the computer determines the first position. The first weighted pose is determined by multiplying the acquisition pose of the first acquisition device under the first point cloud timestamp with a first weight, and the first weight is positively correlated with the confidence level of the acquisition pose of the first acquisition device under the first point cloud timestamp.

[0148] The computer device acquires the confidence level of the acquisition pose of the second acquisition device under the second point cloud timestamp. Based on the position of the second point in the second point cloud and the second weighted pose of the second acquisition device under the second point cloud timestamp, the second position is determined. The second weighted pose is determined by the product of the acquisition pose of the second acquisition device under the second point cloud timestamp and a second weight, and the second weight is positively correlated with the confidence level of the acquisition pose of the second acquisition device under the second point cloud timestamp.

[0149] For example, Figure 11 This is a schematic diagram illustrating the acquisition pose provided in an exemplary embodiment of this application. For example... Figure 11 As shown, the acquisition pose at time t1 is acquisition pose 1101, and the acquisition pose at time t2 is acquisition pose 1102. The area between acquisition poses 1101 and 1102 represents the observation result 1103 of the point cloud acquisition at times t1 and t2. For scenarios with point cloud degradation (point cloud misalignment), points acquired at different times are considered as pairs of points with the same name. The confidence levels of the acquisition poses at times t1 and t2 may differ. Confidence levels range from 1, 2, 3, to 4, where confidence levels of 1, 2, and 3 represent usable acquisition poses, and a confidence level of 4 represents unusable acquisition poses. The confidence level includes values ​​for all six degrees of freedom, thus a 6-dimensional information matrix of confidence can be constructed. This information matrix represents the uncertainty of the acquisition pose; a larger information matrix carries a higher weight in the optimization process.

[0150] The process of determining the acquisition pose of the acquisition device under the point cloud timestamp:

[0151] The acquisition pose of points in a point cloud acquired by a computer device refers to the acquisition pose of the acquisition device at the point cloud timestamp. The computer device interpolates and determines the acquisition pose of the first acquisition device at the first point cloud timestamp based on the first RTK-calculated pose adjacent to the first point cloud timestamp. Each acquisition pose corresponds to a pose timestamp, and the point cloud corresponds to a point cloud timestamp; these two may not be synchronized. That is, the acquisition pose at the point cloud timestamp can be determined by interpolation based on the acquisition poses of pose timestamps adjacent to the point cloud timestamp. The first RTK-calculated pose is the acquisition pose of the first acquisition device when acquiring the first point cloud, calculated using RTK technology. The computer device interpolates and determines the acquisition pose of the second acquisition device at the second point cloud timestamp based on the second RTK-calculated pose adjacent to the second point cloud timestamp. The second RTK-calculated pose is the acquisition pose of the second acquisition device when acquiring the second point cloud, calculated using RTK technology.

[0152] For example, the expression for interpolation to determine the acquisition pose is as follows:

[0153]

[0154] Where, p L p represents the acquisition pose of the acquisition device under the point cloud timestamp. rtk1 p represents the acquisition pose of the acquisition device that is adjacent to the point cloud timestamp and acquired at the previous pose timestamp. rtk2 This indicates the acquisition pose of the acquisition device at a pose timestamp adjacent to the point cloud timestamp and occurring at a later pose timestamp. t L t1 represents the point cloud timestamp, t2 represents the pose timestamp that is adjacent to and precedes the point cloud timestamp, and t2 represents the pose timestamp that is adjacent to and follows the point cloud timestamp.

[0155] The computer device determines the acquisition pose of the acquisition device under each point cloud timestamp using the above method. In this embodiment, the acquisition pose when acquiring point clouds refers to the acquisition pose of the acquisition device under the point cloud timestamp.

[0156] For example, Figure 12 This is a schematic diagram illustrating interpolation for determining the acquisition pose, provided in an exemplary embodiment of this application. For example... Figure 12 As shown, when determining the acquisition pose of tL in the point cloud timestamp 1202, the computer device will determine t1 and t2, which are adjacent to tL in the pose timestamp 1201, and use the acquisition poses of t1 and t2 to determine the acquisition pose under tL based on interpolation.

[0157] The process of optimizing the acquired pose using the first residual function and the second residual function:

[0158] Optionally, the computer device can determine the second residual function based on the sum of the cross products of the first and second gravity directions at each moment of the first point cloud acquired by the first acquisition device, and the sum of the cross products of the first and second gravity directions at each moment of the second point cloud acquired by the second acquisition device. During the acquisition of the first and second point clouds, the gravity directions measured by the first and second acquisition devices may contain errors, thus affecting the accuracy of the acquired first and second point clouds. The smaller the result of the cross product between vectors, the closer the angles are. Based on minimizing the deviation of the gravity direction at each acquisition moment, the computer device can also construct a residual function. By optimizing the second residual function, the roll and pitch angles can be effectively constrained, thereby effectively reducing the offset of the acquired pose in the z-direction (elevation direction). The computer device can optimize the first and second pose maps by minimizing the sum of the first and second residual functions as the optimization objective. The aforementioned gravity direction can also refer to gravitational acceleration.

[0159] The second residual function is an optimization objective that minimizes the sum of the differences between the first and second gravity directions of the first acquisition device, and also minimizes the sum of the differences between the first and second gravity directions of the second acquisition device. The first gravity direction of the first acquisition device is the measured gravity direction when it acquires the first point cloud, and the second gravity direction is the gravity direction calculated based on the acquisition pose of the first acquisition device when acquiring the first point cloud. Similarly, the first gravity direction of the second acquisition device is the measured gravity direction when it acquires the second point cloud, and the second gravity direction is the gravity direction calculated based on the acquisition pose of the second acquisition device when acquiring the second point cloud.

[0160] Optionally, the first gravity direction of the first acquisition device includes the gravity direction of the first acquisition device at each moment of acquiring the first point cloud, measured based on RTK technology. The second gravity direction of the first acquisition device includes the gravity direction of the first acquisition device at each moment of acquiring the first point cloud, calculated based on a first rotation matrix and the true gravity direction. The first rotation matrix is ​​obtained by post-calculation based on RTK technology, i.e., by post-calculation of the data measured using RTK technology. The first rotation matrix reflects the rotational change of the gravity direction of the first acquisition device relative to the true gravity direction at each moment of acquiring the first point cloud. Similarly, the first gravity direction of the second acquisition device includes the gravity direction of the second acquisition device at each moment of acquiring the second point cloud, measured based on RTK technology. The second gravity direction of the second acquisition device includes the gravity direction of the second acquisition device at each moment of acquiring the second point cloud, calculated based on a second rotation matrix and the true gravity direction. The second rotation matrix is ​​obtained by post-calculation based on RTK technology. The second rotation matrix reflects the rotational change of the gravity direction of the second acquisition device relative to the true gravity direction at each moment of acquiring the second point cloud.

[0161] For example, the expression for calculating the second residual function is as follows:

[0162]

[0163] Among them, g m This indicates the measurement acquisition pose. The table shows the rotation matrix, g, obtained after solving based on RTK technology. w Indicates the actual direction of gravity.

[0164] For example, the expression for the sum of the first residual function and the second residual function is as follows:

[0165]

[0166] The computer device sums the results of the cross product of the first gravity direction and the second gravity direction at each moment based on the first acquisition device for the first point cloud, and the results of the cross product of the first gravity direction and the second gravity direction at each moment based on the second acquisition device for the second point cloud, thereby obtaining the second residual function.

[0167] For example, Figure 13 This is a schematic diagram of a gravity-direction-based constraint-optimized acquisition pose provided by an exemplary embodiment of this application. Figure 13As shown, when the acquisition pose is accurate in the Z direction, the trajectory formed by the acquisition pose is trajectory three (1303). Without optimizing the acquisition pose using the gravity direction, the trajectory formed is trajectory one (1301). With optimization using the gravity direction, the trajectory formed is trajectory two (1301). Combining trajectory one (1301), trajectory two (1302), and trajectory three (1303), it can be seen that optimizing the acquisition pose using the gravity direction makes the acquisition pose closer to the true value. Therefore, it is possible to optimize the acquisition pose.

[0168] Step 808: Align the first point cloud and the second point cloud according to the optimized first acquisition pose and the optimized second pose.

[0169] The computer device projects the first point cloud onto the world coordinate system based on the optimized first acquisition pose, and projects the second point cloud onto the world coordinate system based on the optimized second acquisition pose, thus obtaining an aligned first point cloud and second point cloud.

[0170] For example, Table 1 shows the differences between the point cloud alignment method provided in the embodiments of this application and point cloud alignment using the ICP algorithm:

[0171] Table 1

[0172] Does it depend on the rigid body assumption? yes no Whether the object is stationary depends on the environment. yes no Is it necessary to divide the point cloud into regions? yes no

[0173] As shown in Table 1, the method provided in this application embodiment can support point cloud alignment for non-rigid point clouds. Furthermore, it does not depend on whether objects in the environment are stationary during point cloud acquisition. It can also specify pairs of points with the same name for pose optimization, thereby guiding point cloud alignment and greatly improving the flexibility of point cloud alignment.

[0174] For example, Figure 14 This is a schematic diagram of an optimized acquisition pose provided by an exemplary embodiment of this application. For example... Figure 14 As shown, the trajectory 1401 formed by the unoptimized acquisition pose consists of 5 acquisition poses. The points in the point cloud acquired by the acquisition poses at both ends are labeled as pairs of corresponding points. Optimizing the acquisition pose based on the pairs of corresponding points yields the trajectory 1402 formed by the optimized acquisition pose. Furthermore, the optimized acquisition trajectory maintains smoothness between acquisition poses due to the constraints of other acquisition poses, and the RTK pose (acquisition pose) effectively constrains the optimization of the acquisition pose.

[0175] Step 810: Display the first and second point clouds after point cloud alignment in the user interface.

[0176] After projecting the first point cloud onto the world coordinate system using the optimized first acquisition pose, and the second point cloud onto the world coordinate system using the optimized second acquisition pose, the computer device can obtain the source point cloud after point cloud alignment. The computer device can display the aligned source point cloud in the user interface, and the user can manually determine whether it is necessary to continue calibrating corresponding point pairs to continue point cloud alignment of the source point cloud, thereby improving the accuracy of the source point cloud. The process of performing point cloud alignment again can refer to the above steps, and will not be described in detail here.

[0177] For example, Figure 15 This is a schematic diagram of a user interface provided in an exemplary embodiment of this application. Figure 15 As shown in (a), in the displayed source point cloud 1501, points 15011 in the first point cloud and 15012 in the second point cloud were manually labeled as a pair of corresponding points. Points 15013 in the first point cloud and 15014 in the second point cloud were also labeled as another pair of corresponding points. The labeling of corresponding point pairs is typically done for easily identifiable locations, such as points at the corners of landmarks. Figure 15 As shown in (b), after aligning the source point cloud 1501 based on the corresponding point pairs, the computer device displays the aligned source point cloud 1502. In the displayed source point cloud 1501, the traffic sign has an alignment problem, and the pole on the left is separated. In the displayed aligned source point cloud 1502, the traffic sign and the pole on the left are well aligned. Therefore, the method provided in this application embodiment can not only optimize the acquisition pose of points in the corresponding point pairs, but also optimize the acquisition pose of points outside the corresponding point pairs, thereby improving the point cloud alignment effect.

[0178] After projecting the first and second point clouds onto the world coordinate system according to the optimized acquisition pose, the computer device can obtain a point cloud map. A high-precision map can then be constructed based on this point cloud map.

[0179] For example, Figure 16 This is a schematic diagram of a high-precision map provided in an exemplary embodiment of this application. For example... Figure 16 As shown, the high-precision map 1601 obtained by vectorizing the point cloud includes various ground elements, such as lane lines, geospatial information, traffic signs, and poles. Since point cloud alignment improves the accuracy of the point cloud, it also improves the accuracy of the high-precision map 1601 built based on the point cloud.

[0180] In summary, the method provided in this embodiment optimizes the first acquisition pose of the first acquisition device and the second acquisition pose of the second acquisition device based on the positional differences of the points in the calibrated at least one pair of corresponding points. The optimized first and second acquisition poses enable point cloud alignment of the first and second point clouds. During point cloud alignment, calibrating corresponding point pairs on non-rigid point clouds allows for region division of the non-rigid point clouds based on these pairs, thus enabling point cloud alignment by region. Since local point clouds can be considered rigid, there is no need to consider whether the point clouds are rigid, and the point cloud alignment result is not affected by non-rigid point clouds. Furthermore, even when objects in the acquisition environment move, the calibrated corresponding point pairs enable point cloud alignment for objects that have not moved, thus eliminating the need to consider object movement. This improves the accuracy of the point cloud alignment result.

[0181] The method provided in this application embodiment also optimizes the first acquisition pose and the second acquisition pose by using the minimum positional difference between the first point and the second point in the world coordinate system as a constraint, thus providing a convenient way to optimize the acquisition pose.

[0182] The method provided in this application embodiment also optimizes the first pose and the second pose by using the optimization objective of the first residual function as a constraint, thereby optimizing the first acquisition pose and the second acquisition pose at each acquisition time.

[0183] The method provided in this application embodiment also optimizes the first acquisition trajectory and the second acquisition trajectory, thereby ensuring the smoothness and continuity of the acquisition trajectory and avoiding over-optimization during the process of optimizing the acquisition pose.

[0184] The method provided in this application embodiment also determines the position of a point in the world coordinate system by using the acquisition pose under the point cloud timestamp and the position of the point in the point cloud, providing a convenient way to determine the position of a point in the point cloud in the world coordinate system.

[0185] The method provided in this application embodiment also determines the acquisition pose of the acquisition device under the point cloud timestamp through interpolation, which can improve the accuracy of the acquisition pose used.

[0186] The method provided in this application embodiment also performs weighted processing on the process of determining the position of points in the point cloud in the world coordinate system based on the confidence level of the acquisition pose. This can realize the consideration of the confidence level of different acquisition poses and improve the accuracy of the determined position of points in the point cloud in the world coordinate system.

[0187] The method provided in this application embodiment also optimizes the first pose map and the second pose map through the first residual function and the second residual function, and introduces the constraint of gravity direction to optimize the acquisition pose. It can realize the calibration of the Z direction (gravity direction) of the acquisition pose, thereby correcting the error in the Z direction caused by abnormal displacement of the acquisition device in the Z direction (such as road bumps) during the acquisition of point cloud, and can improve the accuracy of the optimized acquisition pose.

[0188] The method provided in this application embodiment also determines the second residual function by calculating the sum of the cross product results of the first gravity direction and the second gravity direction, thus providing a way to determine the second residual function.

[0189] The method provided in this application embodiment also constructs a second residual function by using the gravity direction measured based on RTK technology and the gravity direction determined based on the rotation matrix, providing a way to optimize the acquisition pose by constraining the gravity direction.

[0190] The method provided in this application embodiment also provides a point cloud alignment method by projecting the point cloud onto the world coordinate system according to the optimized acquisition pose.

[0191] The method provided in this application embodiment also provides a convenient way to calibrate pairs of points by displaying the source point cloud so that the user can perform calibration operations.

[0192] It should be noted that the order of the method steps provided in the embodiments of this application can be appropriately adjusted, and the steps can also be added or removed as appropriate. Any method variations that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the protection scope of this application, and therefore will not be elaborated further.

[0193] Figure 17 This is a schematic diagram of the structure of a point cloud alignment device provided in an exemplary embodiment of this application. Figure 17 As shown, the device includes:

[0194] The acquisition module 1701 is used to acquire the first point cloud and the second point cloud, and the points in the first point cloud and the second point cloud have an intersection.

[0195] The calibration module 1702 is used to calibrate at least one pair of corresponding points in a first point cloud and a second point cloud based on calibration operations. The pair of corresponding points includes a first point in the first point cloud and a second point in the second point cloud.

[0196] The optimization module 1703 is used to optimize the first acquisition pose and the second acquisition pose based on the positional difference between the first point and the second point in at least one pair of corresponding point pairs. The first acquisition pose includes the pose of the first acquisition device when acquiring the first point cloud, and the second acquisition pose includes the pose of the second acquisition device when acquiring the second point cloud.

[0197] Alignment module 1704 is used to align the first point cloud and the second point cloud according to the optimized first acquisition pose and the optimized second optimized pose.

[0198] In an optional design, such as Figure 18 As shown, optimization module 1703 includes:

[0199] Projection submodule 17031 is used to project the first point of at least one pair of corresponding point pairs onto the world coordinate system to obtain the first position of the first point in the world coordinate system. The projection submodule is also used to project the second point of at least one pair of corresponding point pairs onto the world coordinate system to obtain the second position of the second point in the world coordinate system.

[0200] The optimization module 1703 is used to optimize the first acquisition pose and the second acquisition pose with the constraint that the positional difference between the first position and the second position of at least one pair of corresponding point pairs is minimized.

[0201] In an optional design, optimization module 1703 is used for:

[0202] A first pose map of the first acquisition device is constructed based on the first acquisition pose of the first point cloud at each time step. A second pose map of the second acquisition device is constructed based on the second acquisition pose of the second point cloud at each time step. A first residual function is determined based on the difference between the first and second positions of at least one pair of corresponding point pairs. The first and second pose maps are optimized using the optimization objective of the first residual function as a constraint. The first residual function is a residual function whose optimization objective is to minimize the sum of the differences between the first and second positions of at least one pair of corresponding point pairs.

[0203] In an optional design, optimization module 1703 is used for:

[0204] Using the optimization objective of the first residual function as a constraint, the first acquisition trajectory in the first pose graph and the second acquisition trajectory in the second pose graph are optimized. The first acquisition trajectory includes the acquisition pose of the first acquisition device corresponding to the first point, and the trajectory formed by the acquisition poses of the first acquisition device acquiring points other than the first point in the first point cloud, according to a time sequence. The second acquisition trajectory includes the acquisition pose of the second acquisition device corresponding to the second point, and the trajectory formed by the acquisition poses of the second acquisition device acquiring points other than the second point in the second point cloud, according to a time sequence.

[0205] In an optional design, the projection submodule 17031 is used for:

[0206] The acquisition pose of the first acquisition device under the first point cloud timestamp is obtained. The first point cloud timestamp is the timestamp of the acquisition of the first point. Based on the position of the first point in the first point cloud and the acquisition pose of the first acquisition device under the first point cloud timestamp, the first position is determined.

[0207] The acquisition pose of the second acquisition device under the second point cloud timestamp is obtained. The second point cloud timestamp is the timestamp for acquiring the second point. Based on the position of the second point in the second point cloud and the acquisition pose of the second acquisition device under the second point cloud timestamp, the second position is determined.

[0208] In an optional design, the projection submodule 17031 is used for:

[0209] Based on the first RTK calculated pose adjacent to the first point cloud timestamp, the acquisition pose of the first acquisition device under the first point cloud timestamp is determined by interpolation. The first RTK calculated pose is the acquisition pose of the first acquisition device when acquiring the first point cloud, which is obtained by RTK technology.

[0210] Based on the second RTK calculated pose adjacent to the second point cloud timestamp, the acquisition pose of the second acquisition device under the second point cloud timestamp is determined by interpolation. The second RTK calculated pose is the acquisition pose of the second acquisition device when acquiring the second point cloud, which is obtained by RTK technology.

[0211] In an optional design, the projection submodule 17031 is used for:

[0212] Obtain the confidence level of the acquisition pose of the first acquisition device under the first point cloud timestamp. Determine the first position based on the position of the first point in the first point cloud and the first weighted pose of the first acquisition device under the first point cloud timestamp. The first weighted pose is determined by the product of the acquisition pose of the first acquisition device under the first point cloud timestamp and the first weight. The first weight is positively correlated with the confidence level of the acquisition pose of the first acquisition device under the first point cloud timestamp.

[0213] Obtain the confidence level of the acquisition pose of the second acquisition device under the second point cloud timestamp. Determine the second position based on the position of the second point in the second point cloud and the second weighted pose of the second acquisition device under the second point cloud timestamp. The second weighted pose is determined by the product of the acquisition pose of the second acquisition device under the second point cloud timestamp and the second weight. The second weight is positively correlated with the confidence level of the acquisition pose of the second acquisition device under the second point cloud timestamp.

[0214] In an optional design, optimization module 1703 is used for:

[0215] The optimization objective is to minimize the sum of the first residual function and the second residual function, optimizing both the first and second pose maps. The second residual function is defined as minimizing the sum of the differences between the first and second gravity directions of the first acquisition device, and minimizing the sum of the differences between the first and second gravity directions of the second acquisition device. The first gravity direction of the first acquisition device is the measured gravity direction when acquiring the first point cloud, and the second gravity direction is the gravity direction calculated based on the acquisition pose of the first acquisition device when acquiring the first point cloud. Similarly, the first gravity direction of the second acquisition device is the measured gravity direction when acquiring the second point cloud, and the second gravity direction is the gravity direction calculated based on the acquisition pose of the second acquisition device when acquiring the second point cloud.

[0216] In an optional design, such as Figure 19 As shown, the device also includes:

[0217] The determination module 1705 is used to determine the second residual function based on the sum of the cross product results of the first gravity direction and the second gravity direction at each moment of the first point cloud collected by the first acquisition device, and the sum of the cross product results of the first gravity direction and the second gravity direction at each moment of the second point cloud collected by the second acquisition device.

[0218] In an optional design, the first gravity direction of the first acquisition device includes the gravity direction of the first acquisition device at each moment of acquiring the first point cloud, measured based on RTK technology. The second gravity direction of the first acquisition device includes the gravity direction of the first acquisition device at each moment of acquiring the first point cloud, calculated based on the first rotation matrix and the true gravity direction. The first rotation matrix is ​​obtained by solving based on RTK technology and is used to reflect the rotational change of the gravity direction of the first acquisition device relative to the true gravity direction at each moment of acquiring the first point cloud.

[0219] The first gravity direction of the second acquisition device includes the gravity direction of the second acquisition device at each moment of acquiring the second point cloud, measured based on RTK technology. The second gravity direction of the second acquisition device includes the gravity direction of the second acquisition device at each moment of acquiring the second point cloud, calculated based on the second rotation matrix and the true gravity direction. The second rotation matrix is ​​obtained by solving based on RTK technology and is used to reflect the rotational change of the gravity direction of the second acquisition device relative to the true gravity direction at each moment of acquiring the second point cloud.

[0220] In an optional design, alignment module 1704 is used for:

[0221] The first point cloud is projected onto the world coordinate system based on the optimized first acquisition pose, and the second point cloud is projected onto the world coordinate system based on the optimized second acquisition pose, resulting in an aligned first and second point cloud.

[0222] In an optional design, such as Figure 20 As shown, the device also includes:

[0223] Display module 1706 is used to render and display source point cloud in user interface, the source point cloud including first point cloud and second point cloud.

[0224] Calibration module 1702 is used for:

[0225] In response to a calibration operation on points in the displayed source point cloud, obtain the point pairs calibrated by the calibration operation as at least one pair of points with the same name.

[0226] It should be noted that the point cloud alignment device provided in the above embodiments is only an example of the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the point cloud alignment device and the point cloud alignment method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0227] Embodiments of this application also provide a computer device, which includes a processor and a memory. The memory stores at least one instruction, at least one program, code set, or instruction set. The at least one instruction, at least one program, code set, or instruction set is loaded and executed by the processor to implement the point cloud alignment method provided in the above-described method embodiments.

[0228] Alternatively, the computer device is a server. For example, Figure 21 This is a schematic diagram of the structure of a computer device provided in an exemplary embodiment of this application.

[0229] The computer device 2100 includes a central processing unit (CPU) 2101, a system memory 2104 including random access memory (RAM) 2102 and read-only memory (ROM) 2103, and a system bus 2105 connecting the system memory 2104 and the CPU 2101. The computer device 2100 also includes a basic input / output system (I / O system) 2106 to facilitate information transfer between various components within the computer device, and a mass storage device 2107 for storing the operating system 2113, application programs 2114, and other program modules 2115.

[0230] The basic input / output system 2106 includes a display 2108 for displaying information and an input device 2109 for user input, such as a mouse or keyboard. Both the display 2108 and the input device 2109 are connected to the central processing unit 2101 via an input / output controller 2110 connected to the system bus 2105. The basic input / output system 2106 may also include the input / output controller 2110 for receiving and processing input from multiple other devices such as a keyboard, mouse, or electronic stylus. Similarly, the input / output controller 2110 also provides output to a display screen, printer, or other types of output devices.

[0231] The mass storage device 2107 is connected to the central processing unit 2101 via a mass storage controller (not shown) connected to the system bus 2105. The mass storage device 2107 and its associated computer-readable storage media provide non-volatile storage for the computer device 2100. That is, the mass storage device 2107 may include computer-readable storage media (not shown), such as a hard disk or a compact disc read-only memory (CD-ROM) drive.

[0232] Without loss of generality, the computer-readable storage medium may include computer storage media and communication media. Computer storage media include volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information such as computer-readable storage instructions, data structures, program modules, or other data. Computer storage media include RAM, ROM, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other solid-state storage devices, CD-ROM, digital versatile disc (DVD) or other optical storage, magnetic tape cassettes, magnetic tape, disk storage, or other magnetic storage devices. Of course, those skilled in the art will recognize that the computer storage medium is not limited to the above-mentioned types. The system memory 2104 and mass storage device 2107 described above can be collectively referred to as memory.

[0233] The memory stores one or more programs, which are configured to be executed by one or more central processing units 2101. The one or more programs contain instructions for implementing the above method embodiments, and the central processing unit 2101 executes the one or more programs to implement the methods provided by the various method embodiments described above.

[0234] According to various embodiments of this application, the computer device 2100 can also be connected to a remote computer device on a network, such as the Internet. That is, the computer device 2100 can be connected to the network 2112 via the network interface unit 2111 connected to the system bus 2105, or the network interface unit 2111 can be used to connect to other types of networks or remote computer device systems (not shown).

[0235] The memory further includes one or more programs stored in the memory, and the one or more programs include steps performed by a computer device in the methods provided in the embodiments of this application.

[0236] This application also provides a computer-readable storage medium storing at least one instruction, at least one program, code set, or instruction set. When the at least one instruction, at least one program, code set, or instruction set is loaded and executed by the processor of a computer device, the point cloud alignment method provided in the above-described method embodiments is implemented.

[0237] This application also provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the point cloud alignment method provided in the above-described method embodiments.

[0238] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0239] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent switching, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A point cloud alignment method, characterized in that, The method includes: Obtain a first point cloud and a second point cloud, where the points in the first point cloud and the second point cloud intersect; Based on the calibration operation, at least one pair of corresponding points is calibrated in the first point cloud and the second point cloud. The corresponding point pair includes a first point in the first point cloud and a second point in the second point cloud. The points in the corresponding point pair are misaligned points in the first point cloud and the second point cloud, and the points in the corresponding point pair are semantically identical points in the first point cloud and the second point cloud. Obtain the acquisition pose of the first acquisition device at a first point cloud timestamp, where the first point cloud timestamp is the timestamp for acquiring the first point; obtain the confidence level of the acquisition pose of the first acquisition device at the first point cloud timestamp; determine the first position based on the position of the first point in the first point cloud and the first weighted pose of the first acquisition device at the first point cloud timestamp, where the first weighted pose is determined by the product of the acquisition pose of the first acquisition device at the first point cloud timestamp and a first weight, and the first weight is positively correlated with the confidence level of the acquisition pose of the first acquisition device at the first point cloud timestamp; obtain the acquisition pose of the second acquisition device at a second point cloud timestamp, where the second point cloud timestamp is the timestamp for acquiring the second point; obtain the confidence level of the second acquisition device at the first point cloud timestamp; and determine the confidence level of the acquisition pose of the second acquisition device at the first point cloud timestamp. The confidence level of the acquisition pose under the second point cloud timestamp; the second position is determined based on the position of the second point in the second point cloud and the second weighted pose of the second acquisition device under the second point cloud timestamp. The second weighted pose is determined by the product of the acquisition pose of the second acquisition device under the second point cloud timestamp and the second weight. The second weight is positively correlated with the confidence level of the acquisition pose of the second acquisition device under the second point cloud timestamp; the first acquisition pose and the second acquisition pose are optimized with the constraint that the positional difference between the first position and the second position of the at least one pair of corresponding point pairs is minimized. The first acquisition pose includes the pose of the first acquisition device when acquiring the first point cloud, and the second acquisition pose includes the pose of the second acquisition device when acquiring the second point cloud. Based on the optimized first acquisition pose and the optimized second acquisition pose, the first point cloud and the second point cloud are aligned.

2. The method according to claim 1, characterized in that, The optimization of the first and second acquisition poses, using the minimum positional difference between the first and second positions of the at least one pair of corresponding points as a constraint, includes: A first pose map of the first acquisition device is constructed based on the first acquisition pose of the first point cloud at each moment; a second pose map of the second acquisition device is constructed based on the second acquisition pose of the second point cloud at each moment. The first residual function is determined based on the difference between the first position and the second position of the at least one pair of corresponding points; The first pose graph and the second pose graph are optimized using the optimization objective of the first residual function as a constraint. The first residual function is a residual function whose optimization objective is to minimize the sum of the differences between the first position and the second position of the at least one pair of corresponding points.

3. The method according to claim 2, characterized in that, The optimization of the first pose graph and the second pose graph, using the optimization objective of the first residual function as a constraint, includes: Using the optimization objective of the first residual function as a constraint, the first acquisition trajectory in the first pose graph is optimized, and the second acquisition trajectory in the second pose graph is optimized. The first acquisition trajectory includes the acquisition pose of the first acquisition device corresponding to the first point, and the trajectory formed by the acquisition pose of the first acquisition device when acquiring points other than the first point in the first point cloud according to the time sequence; the second acquisition trajectory includes the acquisition pose of the second acquisition device corresponding to the second point, and the trajectory formed by the acquisition pose of the second acquisition device when acquiring points other than the second point in the second point cloud according to the time sequence.

4. The method according to claim 1, characterized in that, The acquisition of the acquisition pose of the first acquisition device under the first point cloud timestamp includes: Based on the first RTK calculated pose adjacent to the first point cloud timestamp, the acquisition pose of the first acquisition device under the first point cloud timestamp is determined by interpolation. The first RTK calculated pose is the acquisition pose of the first acquisition device when acquiring the first point cloud, which is calculated based on RTK technology. The acquisition of the acquisition pose of the second acquisition device under the second point cloud timestamp includes: Based on the second RTK calculated pose adjacent to the second point cloud timestamp, the acquisition pose of the second acquisition device under the second point cloud timestamp is determined by interpolation. The second RTK calculated pose is the acquisition pose of the second acquisition device when acquiring the second point cloud, which is calculated based on the RTK technology.

5. The method according to claim 2, characterized in that, The optimization of the first pose graph and the second pose graph, using the optimization objective of the first residual function as a constraint, includes: The first pose graph and the second pose graph are optimized based on minimizing the sum of the first residual function and the second residual function. The second residual function is an optimization objective that minimizes the sum of the differences between the first gravity direction and the second gravity direction of the first acquisition device, and minimizes the sum of the differences between the first gravity direction and the second gravity direction of the second acquisition device. The first gravity direction of the first acquisition device is the measured gravity direction when the first acquisition device acquires the first point cloud, and the second gravity direction of the first acquisition device is the gravity direction calculated based on the acquisition pose when the first acquisition device acquires the first point cloud. The first gravity direction of the second acquisition device is the measured gravity direction when the second acquisition device acquires the second point cloud, and the second gravity direction of the second acquisition device is the gravity direction calculated based on the acquisition pose when the second acquisition device acquires the second point cloud.

6. The method according to claim 5, characterized in that, The method further includes: The second residual function is determined based on the sum of the cross product results of the first gravity direction and the second gravity direction at each moment of the first point cloud collected by the first acquisition device, and the sum of the cross product results of the first gravity direction and the second gravity direction at each moment of the second point cloud collected by the second acquisition device.

7. The method according to claim 5, characterized in that, The first gravity direction of the first acquisition device includes the gravity direction of the first acquisition device at each moment of acquiring the first point cloud, measured based on RTK technology. The second gravity direction of the first acquisition device includes the gravity direction of the first acquisition device at each moment of acquiring the first point cloud, calculated based on the first rotation matrix and the true gravity direction. The first rotation matrix is ​​obtained by solving based on the RTK technology. The first rotation matrix is ​​used to reflect the rotational change of the gravity direction of the first acquisition device relative to the true gravity direction at each moment of acquiring the first point cloud. The first gravity direction of the second acquisition device includes the gravity direction of the second acquisition device at each moment of acquiring the second point cloud, measured based on the RTK technology. The second gravity direction of the second acquisition device includes the gravity direction of the second acquisition device at each moment of acquiring the second point cloud, calculated based on the second rotation matrix and the true gravity direction. The second rotation matrix is ​​obtained by solving based on the RTK technology and is used to reflect the rotational change of the gravity direction of the second acquisition device relative to the true gravity direction at each moment of acquiring the second point cloud.

8. The method according to any one of claims 1 to 7, characterized in that, The step of aligning the first point cloud and the second point cloud based on the optimized first acquisition pose and the optimized second acquisition pose includes: The first point cloud is projected onto the world coordinate system based on the optimized first acquisition pose, and the second point cloud is projected onto the world coordinate system based on the optimized second acquisition pose, to obtain an aligned first point cloud and second point cloud.

9. The method according to any one of claims 1 to 7, characterized in that, The method further includes: The source point cloud is rendered and displayed in the user interface, the source point cloud including the first point cloud and the second point cloud; The calibration operation, which calibrates at least one pair of points with the same name in the first point cloud and the second point cloud, includes: In response to a calibration operation on points in the displayed source point cloud, the point pairs calibrated by the calibration operation are obtained as the at least one pair of points with the same name.

10. A point cloud alignment device, characterized in that, The device includes: The acquisition module is used to acquire a first point cloud and a second point cloud, wherein the points in the first point cloud and the second point cloud intersect; A calibration module is used to calibrate at least one pair of corresponding points in the first point cloud and the second point cloud based on a calibration operation. The corresponding point pair includes a first point in the first point cloud and a second point in the second point cloud. The points in the corresponding point pair are misaligned points in the first point cloud and the second point cloud, and the points in the corresponding point pair are points with the same semantics in the first point cloud and the second point cloud. The projection submodule is used to obtain the acquisition pose of the first acquisition device under a first point cloud timestamp, where the first point cloud timestamp is the timestamp of acquiring the first point; obtain the confidence level of the acquisition pose of the first acquisition device under the first point cloud timestamp; and determine a first position based on the position of the first point in the first point cloud and the first weighted pose of the first acquisition device under the first point cloud timestamp, where the first weighted pose is determined by the product of the acquisition pose of the first acquisition device under the first point cloud timestamp and a first weight, where the first weight and the confidence level of the acquisition pose of the first acquisition device under the first point cloud timestamp are... Positive correlation; obtain the acquisition pose of the second acquisition device under the second point cloud timestamp, where the second point cloud timestamp is the timestamp for acquiring the second point; obtain the confidence level of the acquisition pose of the second acquisition device under the second point cloud timestamp; determine the second position based on the position of the second point in the second point cloud and the second weighted pose of the second acquisition device under the second point cloud timestamp, where the second weighted pose is determined by the product of the acquisition pose of the second acquisition device under the second point cloud timestamp and the second weight, and the second weight is positively correlated with the confidence level of the acquisition pose of the second acquisition device under the second point cloud timestamp; The optimization module is used to optimize the first acquisition pose and the second acquisition pose with the constraint that the positional difference between the first position and the second position of the at least one pair of corresponding point pairs is minimized. The first acquisition pose includes the pose of the first acquisition device when acquiring the first point cloud, and the second acquisition pose includes the pose of the second acquisition device when acquiring the second point cloud. The alignment module is used to align the first point cloud and the second point cloud based on the optimized first acquisition pose and the optimized second acquisition pose.

11. The apparatus according to claim 10, characterized in that, The optimization module is also used for: A first pose map of the first acquisition device is constructed based on the first acquisition pose of the first point cloud at each moment; a second pose map of the second acquisition device is constructed based on the second acquisition pose of the second point cloud at each moment. The first residual function is determined based on the difference between the first position and the second position of the at least one pair of corresponding points; The first pose graph and the second pose graph are optimized using the optimization objective of the first residual function as a constraint. The first residual function is a residual function whose optimization objective is to minimize the sum of the differences between the first position and the second position of the at least one pair of corresponding points.

12. The apparatus according to claim 11, characterized in that, The optimization module is also used for: Using the optimization objective of the first residual function as a constraint, the first acquisition trajectory in the first pose graph is optimized, and the second acquisition trajectory in the second pose graph is optimized. The first acquisition trajectory includes the acquisition pose of the first acquisition device corresponding to the first point, and the trajectory formed by the acquisition pose of the first acquisition device when acquiring points other than the first point in the first point cloud according to the time sequence; the second acquisition trajectory includes the acquisition pose of the second acquisition device corresponding to the second point, and the trajectory formed by the acquisition pose of the second acquisition device when acquiring points other than the second point in the second point cloud according to the time sequence.

13. The apparatus according to any one of claims 10 to 12, characterized in that, The alignment module is also used for: The first point cloud is projected onto the world coordinate system based on the optimized first acquisition pose, and the second point cloud is projected onto the world coordinate system based on the optimized second acquisition pose, to obtain an aligned first point cloud and second point cloud.

14. The apparatus according to any one of claims 10 to 12, characterized in that, The device further includes: The display module is used to render and display the source point cloud in the user interface, wherein the source point cloud includes the first point cloud and the second point cloud; The calibration module is also used for: In response to a calibration operation on points in the displayed source point cloud, the point pairs calibrated by the calibration operation are obtained as the at least one pair of points with the same name.

15. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing at least one instruction, at least one program, a code set, or an instruction set, the at least one instruction, the at least one program, the code set, or the instruction set being loaded and executed by the processor to implement the point cloud alignment method as described in any one of claims 1 to 9.

16. A computer-readable storage medium, characterized in that, The readable storage medium stores at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the point cloud alignment method as described in any one of claims 1 to 9.

17. A computer program product, characterized in that, The computer program product includes computer instructions stored in a computer-readable storage medium, a processor of a computer device reading the computer instructions from the computer-readable storage medium, and the processor executing the computer instructions to cause the computer device to perform the point cloud alignment method as described in any one of claims 1 to 9.

Images