Point cloud data registration method, device, electronic equipment, system and storage medium

Abstract

Embodiments of the present application provide a point cloud data registration method, device, electronic equipment, system and storage medium, and relate to the field of point cloud calibration. Point cloud data including a plurality of to-be-registered point clouds and a plurality of target point clouds is obtained, and each to-be-registered point cloud and a corresponding target point cloud constitutes a point cloud pair. Then, according to the coordinate information of the to-be-registered point cloud and the target point cloud in each point cloud pair, an error function of the point cloud data is constructed, a transformation matrix of the point cloud data is calculated according to the error function, the coordinate information of each to-be-registered point cloud is updated according to the transformation matrix, and the error function is iterated according to the updated coordinate information of each to-be-registered point cloud until the iteration parameter of the error function meets a preset iteration condition. The method can improve the registration accuracy.

Description

Technical Field

[0001] This application relates to the field of point cloud calibration, and more specifically, to a point cloud data registration method, apparatus, electronic device, system, and storage medium. Background Technology

[0002] Currently, with the development of autonomous driving technology, there are more and more types of equipment on vehicles, and these devices are mainly used for environmental perception and positioning, such as lidar and TOF (Time-of-flight) cameras. In order to provide rich environmental information for autonomous driving, it is necessary to accurately register the images collected by the above devices. However, existing technologies have the problem of poor registration accuracy. Summary of the Invention

[0003] In view of this, the purpose of this application is to provide a point cloud data registration method, apparatus, electronic device, system and storage medium to solve the problem of poor registration accuracy in the prior art.

[0004] To achieve the above objectives, the technical solutions adopted in the embodiments of this application are as follows:

[0005] Firstly, this application provides a point cloud data registration method, the method comprising:

[0006] Acquire point cloud data; the point cloud data includes multiple point clouds to be registered and multiple target point clouds; each point cloud to be registered and a corresponding target point cloud constitute a point cloud pair;

[0007] Based on the coordinate information of the point cloud to be registered and the target point cloud in each point cloud pair, an error function for the point cloud data is constructed; wherein, the error function includes a weight function for the point cloud data, and the weight function is used to characterize the weight of each point cloud pair in the registration process;

[0008] The transformation matrix of the point cloud data is calculated based on the error function, and the coordinate information of each point cloud to be registered is updated based on the transformation matrix.

[0009] The error function is iterated according to the updated coordinate information of each point cloud to be registered until the iteration parameters of the error function meet the preset iteration conditions.

[0010] In an optional implementation, constructing an error function for the point cloud data based on the coordinate information of the point cloud to be registered and the target point cloud in each point cloud pair includes:

[0011] Based on the coordinate information of the point cloud to be registered and the target point cloud in each point cloud pair, an initial error function for the point cloud data is constructed.

[0012] Calculate the variance matrix of the point cloud data based on the initial error function;

[0013] Construct a weighting function for the point cloud data based on the variance matrix;

[0014] An error function for the point cloud data is constructed based on the weight function and the initial error function.

[0015] In an optional implementation, calculating the weight function of the point cloud pair based on the variance matrix includes:

[0016] Obtain the depth parameters of each point cloud pair, and calculate the first weight of each point cloud pair based on the depth parameters;

[0017] The variance matrix is ​​decomposed according to a preset decomposition algorithm to obtain multiple eigenvalues;

[0018] The second weight of each point cloud pair is calculated based on the feature vector corresponding to the minimum value among the plurality of feature values, the coordinate information of the point cloud to be registered and the target point cloud in each point cloud pair;

[0019] A weight function for the point cloud data is constructed based on the first and second weights of each point cloud pair.

[0020] In an optional implementation, calculating the first weight for each point cloud pair based on the depth parameter includes:

[0021] The first weight is calculated according to the following formula:

[0022] W deg =1 / Z i2

[0023] Among them, W deg Z represents the first weight. i The depth parameter represents the i-th point cloud pair.

[0024] In an optional implementation, the step of calculating the second weight of each point cloud pair based on the feature vector corresponding to the minimum value among the plurality of feature values, and the coordinate information of the point cloud to be registered and the target point cloud in each point cloud pair, includes:

[0025] The second weight is calculated using the following formula:

[0026]

[0027] Among them, W sta V represents the second weight. s p represents the eigenvector corresponding to the minimum value among the plurality of eigenvalues. iThe three-dimensional coordinates of the point cloud to be registered in the i-th point cloud pair are represented. It represents the inversion of the normal vector of the i-th point cloud.

[0028] In an optional implementation, constructing a weight function for the point cloud data based on the first and second weights of each point cloud pair includes:

[0029] The weight function is constructed according to the following formula:

[0030] W final =W sta ·W deg

[0031] Among them, W final Characterizing the weight function, W sta Characterizing the second weight, W deg This represents the first weight.

[0032] In an optional implementation, constructing an initial error function for the point cloud data based on the coordinate information of the point cloud to be registered and the target point cloud in each point cloud pair includes:

[0033] The initial error function is constructed according to the following formula:

[0034] E=∑((R·p i +tq i )·n i ) 2

[0035] Where E represents the initial error function, R represents the rotation matrix, and p i The 3D coordinates of the point cloud to be registered in the i-th point cloud pair are represented by t, and the translation parameter is q. i The three-dimensional coordinates of the target point cloud in the i-th point cloud pair are represented by n. i The normal vector representing the target point cloud in the i-th point cloud pair.

[0036] In an optional implementation, calculating the variance matrix of the point cloud data based on the initial error function includes:

[0037] The variance matrix of the point cloud data is calculated using the following formula:

[0038] C = A T A

[0039] A = [a i1 ,a i2 ,a i3 ,n ix ,n iy ,n iz ]

[0040] Where C represents the variance matrix, A T The transpose of matrix A, n ix The normal vector of the target point cloud in the i-th point cloud pair along the x-axis, n iy The normal vector of the target point cloud in the i-th point cloud pair along the y-axis is represented by n. iz p represents the normal vector of the target point cloud in the i-th point cloud pair along the z-axis. ix p represents the x-axis coordinates of the point cloud to be registered in the i-th point cloud pair. iy p represents the y-coordinate of the point cloud to be registered in the i-th point cloud pair. iz The coordinates of the point cloud to be registered in the i-th point cloud pair are represented in the z-axis direction.

[0041] In an optional implementation, constructing the error function of the point cloud data based on the weight function and the initial error function includes:

[0042] The error function for the point cloud data is constructed according to the following formula:

[0043] E′=∑(W final ·(R·p i +tq i )·n i ) 2

[0044] Where E′ represents the error function, W final The weight function is represented by R, the rotation matrix is ​​represented by p. i The 3D coordinates of the point cloud to be registered in the i-th point cloud pair are represented by t, and the translation parameter is q. i The three-dimensional coordinates of the target point cloud in the i-th point cloud pair are represented by n. i The normal vector representing the target point cloud in the i-th point cloud pair.

[0045] In an optional implementation, calculating the transformation matrix of the point cloud data based on the error function includes:

[0046] The error function is minimized to obtain the transformation matrix increment;

[0047] The transformation matrix is ​​calculated based on the initial transformation matrix and the increment of the transformation matrix.

[0048] In an optional implementation, the iteration parameters include the current iteration number, and the step of iterating the error function according to the updated coordinate information of each of the point clouds to be registered until the iteration parameters of the error function meet the preset iteration conditions includes:

[0049] The error function is iterated according to the updated coordinate information of each point cloud to be registered, and it is determined whether the current iteration number has reached the preset iteration number.

[0050] If the current iteration count reaches the preset iteration count, then the iteration parameters of the error function are determined to satisfy the preset iteration conditions.

[0051] In an optional implementation, the iteration parameters include error values, and the step of iterating the error function according to the updated coordinate information of each of the point clouds to be registered until the iteration parameters of the error function satisfy a preset iteration condition includes:

[0052] The error function is iterated according to the updated coordinate information of each point cloud to be registered to obtain the iterated error function, and the error value is calculated according to the iterated error function.

[0053] If the error value is less than a preset error threshold, then the iterative parameters of the error function are determined to satisfy the preset iterative conditions.

[0054] In an optional implementation, before the step of calculating the transformation matrix of the point cloud data according to the error function and updating the coordinate information of each point cloud to be registered according to the transformation matrix, the method further includes:

[0055] If it is determined that the point cloud to be registered in each point cloud pair has not been normalized, normalization is performed on the point cloud to be registered in each point cloud pair.

[0056] In an optional implementation, prior to the step of acquiring point cloud data, the method further includes:

[0057] An initial point cloud is acquired and preprocessed to obtain multiple point clouds to be registered and multiple target point clouds corresponding to the same location at the same time.

[0058] Each of the point clouds to be registered is matched with the target point cloud to obtain multiple point cloud pairs;

[0059] Normalize the point clouds to be registered in all the point cloud pairs.

[0060] In an optional implementation, the normalization process for the point clouds to be registered in all the point cloud pairs includes:

[0061] Calculate the coordinates of the center point based on the initial coordinate information of the point clouds to be registered in all the point cloud pairs;

[0062] Based on the initial coordinate information of the point cloud to be registered in each point cloud pair and the coordinates of the center point, calculate the distance from each point cloud to the center point.

[0063] The distance from the point cloud to be registered in each point cloud pair to the center point is normalized to obtain the coordinate information of the point cloud to be registered in each point cloud pair.

[0064] Secondly, this application provides a point cloud data registration device, the device comprising:

[0065] An acquisition module is used to acquire point cloud data; the point cloud data includes multiple point clouds to be registered and multiple target point clouds; each point cloud to be registered and a corresponding target point cloud constitute a point cloud pair.

[0066] A construction module is used to construct an error function for the point cloud data based on the coordinate information of the point cloud to be registered and the target point cloud in each point cloud pair; wherein, the error function includes a weight function for the point cloud data, and the weight function is used to characterize the weight of each point cloud pair in the registration process;

[0067] The update module is used to calculate the transformation matrix of the point cloud data according to the error function, and update the coordinate information of each point cloud to be registered according to the transformation matrix;

[0068] An iteration module is used to iterate the error function according to the updated coordinate information of each point cloud to be registered until the iteration parameters of the error function meet the preset iteration conditions.

[0069] Thirdly, this application provides an electronic device including a processor and a memory, wherein the memory stores a computer program executable by the processor, and the processor can execute the computer program to implement the method described in any of the foregoing embodiments.

[0070] Fourthly, this application provides a point cloud data registration system, including an electronic device, a device to be registered, and a target device, wherein the device to be registered is a lidar and the target device is a TOF camera.

[0071] Fifthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as described in any of the foregoing embodiments.

[0072] The point cloud data registration method, apparatus, electronic device, system, and storage medium provided in this application acquire point cloud data including multiple point clouds to be registered and multiple target point clouds, with each point cloud to be registered and a corresponding target point cloud forming a point cloud pair. Then, based on the coordinate information of the point clouds to be registered and the target point clouds in each point cloud pair, an error function for the point cloud data is constructed, and a transformation matrix for the point cloud data is calculated based on this error function. The coordinate information of each point cloud to be registered is updated according to the transformation matrix. Simultaneously, the error function is iterated according to the updated coordinate information of each point cloud to be registered until the iteration parameters of the error function satisfy a preset iteration condition. This method generates an error function with a weighted function, which can comprehensively consider the weight of each point cloud pair in the registration process when calculating the transformation matrix of the point cloud data, thereby obtaining a more accurate transformation matrix and improving registration accuracy.

[0073] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0074] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0075] Figure 1 A block diagram of the point cloud data registration system provided in an embodiment of this application is shown; Figure 2 A block diagram of an electronic device provided in an embodiment of this application is shown;

[0076] Figure 3 This paper illustrates a flowchart of a point cloud data registration method provided in an embodiment of this application.

[0077] Figure 4 A schematic diagram of point cloud data registration is shown;

[0078] Figure 5 This illustration shows another flowchart of the point cloud data registration method provided in an embodiment of this application;

[0079] Figure 6 This illustration shows another flowchart of the point cloud data registration method provided in an embodiment of this application;

[0080] Figure 7 This illustration shows another flowchart of the point cloud data registration method provided in an embodiment of this application;

[0081] Figure 8 A functional block diagram of a point cloud data registration device provided in an embodiment of this application is shown.

[0082] Icons: 10-Electronic device; 20-Phase to be registered; 30-Target device; 100-Memory; 110-Processor; 120-Communication module; 200-Acquisition module; 210-Construction module; 220-Update module; 230-Iteration module. Detailed Implementation

[0083] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can be arranged and designed in various different configurations.

[0084] Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0085] It should be noted that relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0086] Figure 1 Please refer to the block diagram of the point cloud data registration system provided in the embodiments of this application. Figure 1 The system includes an electronic device 10, a registration device 20, and a target device 30, and the registration device 20 and the target device 30 are respectively connected to the electronic device 10 for communication.

[0087] Optionally, the electronic device 10 is used to acquire point cloud data sent by the device to be registered 20 and the target device 30, and to perform point cloud data registration based on the point cloud data sent by the device to be registered 20 and the target device 30.

[0088] Understandably, the registration device 20 is used to generate the point cloud to be registered, and the target device 30 is used to generate the target point cloud.

[0089] Optionally, the point cloud data registration system can be mounted on a vehicle. In this embodiment, if the vehicle is equipped with a LiDAR and a TOF camera, in one possible scenario, the device to be registered 20 can be a LiDAR and the target device 30 can be a TOF camera; in another possible scenario, the device to be registered 20 can be a TOF camera and the target device 30 can be a LiDAR.

[0090] Please refer to Figure 2 This is a block diagram of an electronic device 10. The electronic device 10 includes a memory 100, a processor 110, and a communication module 120. The memory 100, processor 110, and communication module 120 are electrically connected to each other directly or indirectly to realize data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines.

[0091] The memory 100 is used to store programs or data. The memory 100 may be, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.

[0092] The processor 110 is used to read / write data or programs stored in memory and to perform corresponding functions.

[0093] The communication module 120 is used to establish a communication connection between the server and other communication terminals through the network, and to send and receive data through the network.

[0094] It should be understood that, Figure 2 The structure shown is only a schematic diagram of the electronic device 10. The electronic device may also include components that are larger than... Figure 2 The more or fewer components shown, or having the same Figure 2 The different configurations shown. Figure 2 The components shown can be implemented using hardware, software, or a combination thereof.

[0095] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, can implement the point cloud data registration method provided in this application.

[0096] The following is based on the above. Figure 1 The electronic device 10 in this application serves as the execution entity. The point cloud data registration method provided in this embodiment is illustrated with a flowchart. Specifically, Figure 3 For a flowchart illustrating the point cloud data registration method provided in this application embodiment, please refer to [link / reference]. Figure 3 The method includes:

[0097] Step S20: Acquire point cloud data;

[0098] The point cloud data includes multiple point clouds to be registered and multiple target point clouds; each point cloud to be registered and a corresponding target point cloud constitute a point cloud pair; understandably, the point cloud data includes multiple point cloud pairs, and each point cloud pair contains a point cloud to be registered and a corresponding target point cloud.

[0099] Optionally, the point cloud to be registered refers to the point cloud data that needs to be registered, while the target point cloud refers to the standard point cloud data. It can be understood that the registration of point cloud data is to register each point cloud to be registered to its corresponding target point cloud.

[0100] In one possible implementation, the vehicle is equipped with a LiDAR device and a TOF camera, with the LiDAR being the device to be registered and the TOF camera being the target device. In this case, the point cloud data can include point cloud data collected by the LiDAR and point cloud data converted from depth maps collected by the TOF camera. The point cloud to be registered is the point cloud data generated by the LiDAR, and the target point cloud is the point cloud data generated by the TOF camera.

[0101] Understandably, in this scenario, electronic devices can accurately calibrate the positional relationship between the LiDAR and TOF cameras by registering the point cloud data generated by the LiDAR with the point cloud data generated by the TOF camera, thus fusing the data acquired by both into the same coordinate system. In one example, the point cloud to be registered can be point cloud data converted from a depth map acquired by the TOF camera, and the target point cloud can be point cloud data acquired by the LiDAR.

[0102] In this embodiment, the electronic device can acquire point cloud data and perform point cloud data registration upon receiving a registration instruction input by the user.

[0103] In one possible implementation, please see Figure 4This is a schematic diagram for point cloud data registration, including the plane to be registered and the target plane. The plane to be registered is a plane composed of multiple point clouds to be registered, and the target plane is a plane composed of multiple target point clouds.

[0104] Understandably, in this embodiment, the electronic device performs point cloud data registration by registering each point cloud to be registered to its corresponding target point cloud, thereby achieving the most accurate possible fit between the plane to be registered and the target plane.

[0105] Step S21: Construct an error function for the point cloud data based on the coordinate information of the point cloud to be registered and the target point cloud in each point cloud pair;

[0106] The error function includes a weight function for the point cloud data, which is used to characterize the weight of each point cloud pair in the registration process.

[0107] Optionally, the coordinate information may include the three-dimensional coordinate information and normal vectors of the point cloud data.

[0108] In this embodiment, after obtaining point cloud data, the electronic device can construct an error function containing a weight function of the point cloud data based on the coordinate information of the point cloud to be registered and the target point cloud in the multiple point cloud pairs contained in the point cloud data.

[0109] Step S22: Calculate the transformation matrix of the point cloud data according to the error function, and update the coordinate information of each point cloud to be registered according to the transformation matrix;

[0110] Step S23: Iterate the error function according to the updated coordinate information of each point cloud to be registered until the iteration parameters of the error function meet the preset iteration conditions.

[0111] Optionally, the transformation matrix of the point cloud data is used to register the point cloud to be registered to the target point cloud in each point cloud pair.

[0112] In this embodiment, the electronic device can calculate the transformation matrix of the point cloud data based on the obtained error function, and update the coordinate information of the point cloud to be registered based on the transformation matrix to obtain the new coordinate information of the point cloud to be registered.

[0113] Optionally, since the error function includes a weight function, the transformation matrix calculated based on the error function can fully take into account the weight of each point cloud pair in the registration process, thereby improving the accuracy of the transformation matrix and achieving accurate registration of point cloud data.

[0114] Furthermore, existing registration methods often suffer from drift when registering planar regions with relatively simple geometric features, such as ground point clouds, resulting in poorer registration accuracy. However, the point cloud data registration method provided in this application calculates the transformation matrix based on an error function that includes a weighted function. This method can fully consider the advantages of the weights of each point cloud pair in the registration process. Therefore, whether it is a region with prominent geometric features or a planar region with relatively simple geometric features, accurate registration can be achieved by fully combining the registration weights of each point cloud pair.

[0115] Optionally, after updating the coordinate information of the point cloud to be registered, the electronic device can iterate the error function based on the updated coordinate information to obtain a new error function, and determine whether the iteration parameters of the error function meet the preset iteration conditions. If they do, the iteration exits and the point cloud data registration process ends; if they do not, the transformation matrix is ​​calculated based on the new error function, the coordinate information of the point cloud to be registered is updated, and then the new error function is iterated again until the iteration parameters of the error function meet the preset iteration conditions.

[0116] Understandably, multiple iterations of registration are needed to achieve the registration of point cloud data.

[0117] The point cloud data registration method provided in this application acquires point cloud data including multiple point clouds to be registered and multiple target point clouds, with each point cloud to be registered and a corresponding target point cloud forming a point cloud pair. Then, based on the coordinate information of the point clouds to be registered and the target point clouds in each point cloud pair, an error function for the point cloud data is constructed, and a transformation matrix of the point cloud data is calculated based on this error function. The coordinate information of each point cloud to be registered is updated according to the transformation matrix. Simultaneously, the error function is iterated according to the updated coordinate information of each point cloud to be registered until the iteration parameters of the error function satisfy a preset iteration condition. This method generates an error function with a weighted function, which can comprehensively consider the weight of each point cloud pair in the registration process when calculating the transformation matrix of the point cloud data, thereby obtaining a more accurate transformation matrix and improving registration accuracy.

[0118] In one possible implementation, the electronic device can process the initial point cloud to obtain point cloud data upon receiving a registration instruction from the user and the initial point cloud, and then perform registration based on the point cloud data. In another possible implementation, the electronic device can obtain the initial point cloud in advance, process it, and save the processed point cloud data. Upon receiving a registration instruction from the user, it can retrieve the saved point cloud data and perform point cloud data registration.

[0119] Optionally, since the initial point cloud may contain multiple points to be registered and the target point cloud at multiple times and locations, which is quite messy, the electronic device needs to first filter out the point cloud data at the same time and location from the initial point cloud, and then process the point cloud data at the same time and location.

[0120] Specifically, in Figure 3 On this basis, Figure 5 For another flowchart illustrating the point cloud data registration method provided in this application embodiment, please refer to [link / reference]. Figure 5 The method also includes:

[0121] Step S10: Obtain the initial point cloud and preprocess the initial point cloud to obtain multiple point clouds to be registered and multiple target point clouds corresponding to the same location at the same time.

[0122] Alternatively, the initial point cloud refers to data imported from equipment mounted on the vehicle.

[0123] In this embodiment, the electronic device can first preprocess the initial point cloud, for example, by selecting the point cloud to be processed and the target point cloud at the same time, removing the background point cloud from the point cloud to be processed and the target point cloud at the same location, and selecting the point cloud to be processed and the target point cloud at the same location.

[0124] Optionally, background point cloud refers to point cloud data other than planar point cloud, such as point clouds reflecting trees, point clouds reflecting houses, etc.

[0125] Optionally, if the data imported from the device mounted on the vehicle is not point cloud data, it needs to be converted into point cloud data. For example, if the data imported from the TOF camera is a depth map, it needs to be converted into point cloud data first before proceeding with the next steps.

[0126] Step S11: Match each point cloud to be registered with the target point cloud to obtain multiple point cloud pairs;

[0127] Optionally, after the electronic device obtains multiple point clouds to be registered and multiple target point clouds corresponding to the same location at the same time, it can determine a target point cloud corresponding to each point cloud to be registered, thereby forming multiple point cloud pairs.

[0128] In one possible implementation, the electronic device can determine the target point cloud with the smallest distance to each point cloud to be registered, and determine whether the distance is less than or equal to a preset distance threshold. If so, the target point cloud is determined to be a point cloud that matches the point cloud to be registered, and the two can form a point cloud pair. In another possible implementation, the electronic device can use the ANN (Approximate Nearest Neighbor) algorithm to determine the corresponding target point cloud for each point cloud to be registered, thereby obtaining multiple point cloud pairs.

[0129] Understandably, if a point cloud to be registered cannot be matched with a corresponding target point cloud, or if a target point cloud does not have a corresponding point cloud to be registered, then that point cloud can be deleted. Optionally, to make the matching process more accurate, the electronic device can also update the coordinate information of each point cloud to be registered using a preset transformation matrix before matching each point cloud to be registered with the target point cloud, and then match the updated coordinate information of each point cloud to be registered with the target point cloud.

[0130] Step S12: Normalize the point clouds to be registered in all point cloud pairs.

[0131] Optionally, the electronic device can also perform normalization processing on the point clouds to be registered contained in all point cloud pairs after obtaining multiple point cloud pairs.

[0132] Optionally, point cloud data can be obtained by normalizing the point clouds to be registered contained in all point cloud pairs. It can be understood that the point cloud to be registered in each point cloud pair in the point cloud data is the normalized point cloud to be registered.

[0133] Optionally, the electronic device can normalize the point clouds to be registered in all point cloud pairs through the following steps, that is, the above step S12 can also be implemented through the following steps:

[0134] Based on the initial coordinate information of the point clouds to be registered in all point cloud pairs, calculate the coordinates of the center point; based on the initial coordinate information of the point clouds to be registered in each point cloud pair and the coordinates of the center point, calculate the distance from each point cloud to be registered to the center point; normalize the distance from each point cloud to be registered to the center point in each point cloud pair to obtain the coordinate information of the point clouds to be registered in each point cloud pair.

[0135] Optionally, the initial coordinate information of the point cloud to be registered is the coordinate information of the point cloud to be registered before normalization processing, including the three-dimensional coordinates of the point cloud to be registered.

[0136] Alternatively, the electronic device can obtain the coordinates of the center point by averaging the three-dimensional coordinates of the point clouds to be registered across all point cloud pairs. Understandably, the coordinates of this center point are also three-dimensional coordinates.

[0137] Optionally, the electronic device can calculate the distance between each point cloud to be registered and the center point. In one possible implementation, the distance includes distances in three dimensions, namely, distances in the x, y, and z directions. That is, the electronic device can obtain the distance from each point cloud to be registered to the center point by subtracting its three-dimensional coordinates.

[0138] Alternatively, the electronic device can obtain the coordinate information of the point cloud to be registered in each point cloud pair by normalizing the distance from the point cloud to the center point.

[0139] Optionally, when performing point cloud data registration, the electronic device can construct an error function for the point cloud data based on the initial error function and weight function of the point cloud data. Specifically, in Figure 3 On this basis, Figure 6 For another flowchart illustrating the point cloud data registration method provided in this application embodiment, please refer to [link / reference]. Figure 6 The above step S21 can also be achieved through the following steps:

[0140] Step S21-1: Based on the coordinate information of the point cloud to be registered and the target point cloud in each point cloud pair, construct the initial error function of the point cloud data;

[0141] Step S21-2: Calculate the variance matrix of the point cloud data based on the initial error function;

[0142] Optionally, the initial error function can characterize the distance between the point cloud to be registered and the tangent plane of the target point cloud in each point cloud pair, or the projection length of the Euclidean distance between the point cloud to be registered and the target point cloud in each point cloud pair onto the normal vector of the target point cloud.

[0143] Optionally, please continue to see Figure 4 For the target point cloud q1, the tangent plane refers to the tangent plane of the target plane at the point where the target point cloud q1 is located. The normal vector n1 of the target point cloud q1 is the normal vector of the tangent plane where the target point cloud is located. When the initial error function is reflected on the target point cloud q1, it can characterize the distance l1 from the point cloud p1 to be registered to the tangent plane where the target point cloud q1 is located.

[0144] In one possible implementation, the initial error function for the point cloud data can be constructed according to the following formula:

[0145] E=∑((R·p i +tqi )·n i ) 2

[0146] Where E represents the initial error function, R represents the rotation matrix, and p i The 3D coordinates of the point cloud to be registered in the i-th point cloud pair are represented by t, and the translation parameter is q. i The three-dimensional coordinates of the target point cloud in the i-th point cloud pair are represented by n. i The normal vector representing the target point cloud in the i-th point cloud pair.

[0147] Understandably, if Euler angles α, β, and γ represent the rotation angles along the x, y, and z axes respectively, then the rotation matrix R can be expressed by the following formula:

[0148]

[0149]

[0150] Optionally, since the rotation angle between the point cloud to be registered and the corresponding target point cloud is very small, the above cos operation can be replaced by 1, and the above sin operation can be replaced by an angle. Based on this, the above rotation matrix R can also be expressed in the following form:

[0151]

[0152] Based on this, in the above initial error function It can also be expressed in the following form:

[0153]

[0154] Where, p ix p represents the x-axis coordinate of the point cloud to be registered in the i-th point cloud pair. iy p represents the y-axis coordinate of the point cloud to be registered in the i-th point cloud pair. iz The z-axis coordinate of the point cloud to be registered in the i-th point cloud pair is q. ix The x-axis coordinate of the target point cloud in the i-th point cloud pair is represented by q. iy The y-axis coordinate of the target point cloud in the i-th point cloud pair is represented by q. iz The z-axis coordinate of the target point cloud in the i-th point cloud pair is represented by n. ix The x-axis coordinate of the normal vector of the target point cloud in the i-th point cloud pair, n iy The y-axis coordinate of the normal vector of the target point cloud in the i-th point cloud pair, n iz The z-axis coordinate of the normal vector of the target point cloud in the i-th point cloud pair, t x The translation parameter t represents the x-axis. yThe translation parameter t that characterizes the y-axis z The translation parameter characterizing the z-axis.

[0155] Alternatively, ideally, the error value of the i-th point cloud pair should be 0. In this case, the transformation matrix x can be represented as x = [Rt]. T =[α,β,γ,t] x ,t y ,t z ] T If the error value of the i-th point cloud pair is 0, it can be represented by a linear form, i.e., by the following formula:

[0156] [a i1 ,a i2 ,a i3 ,n ix ,n iy ,n iz ]·xb i =0

[0157] That is, Ax-b=0.

[0158] Among them, a i1 =n iz p iy -n iy p iz a i2 =n ix p iz -n iz p ix a i3 =n iy p ix -n ix p iy n ix The normal vector of the target point cloud in the i-th point cloud pair along the x-axis, n iy The normal vector of the target point cloud in the i-th point cloud pair along the y-axis is represented by n. iz b represents the normal vector of the target point cloud in the i-th point cloud pair along the z-axis. i This is a constant term.

[0159] Understandably, the formula Ax-b is actually a linear least squares problem, meaning we need to minimize the value of the formula Ax-b, which can be represented by the following formula:

[0160]

[0161] Where, x * This is the minimum value of the formula.

[0162] Based on the above formula, x * The minimum necessary and sufficient condition is that x* For A T Ax = A T The solution to b. Where A T Represents the transpose of matrix A.

[0163] Optionally, A T A is the variance matrix of the point cloud data. Specifically, the electronic device can calculate the variance matrix C of the point cloud data using the following formula:

[0164] C = A T A

[0165] A = [a i1 ,a i2 ,a i3 ,n ix ,n iy ,n iz ]

[0166] Among them, a i1 =n iz p iy -n iy p iz a i2 =n ix p iz -n iz p ix a i3 =n iy p ix -n ix p iy n ix The normal vector of the target point cloud in the i-th point cloud pair along the x-axis, n iy The normal vector of the target point cloud in the i-th point cloud pair along the y-axis is represented by n. iz p represents the normal vector of the target point cloud in the i-th point cloud pair along the z-axis. ix p represents the x-axis coordinates of the point cloud to be registered in the i-th point cloud pair. iy p represents the y-coordinate of the point cloud to be registered in the i-th point cloud pair. iz The coordinates of the point cloud to be registered in the i-th point cloud pair are represented in the z-axis direction.

[0167] Step S21-3: Construct the weight function for the point cloud data based on the variance matrix;

[0168] Step S21-4: Construct the error function for the point cloud data based on the weight function and the initial error function.

[0169] In this embodiment, the electronic device can construct a weight function for the point cloud data based on the variance matrix C, and construct an error function for the point cloud data based on the weight function and the aforementioned initial error function.

[0170] In one possible implementation, the weight function of the point cloud data can be composed of a first weight and a second weight. The electronic device can then calculate the first and second weights respectively and construct the weight function based on these first and second weights. Specifically, in Figure 6 On this basis, Figure 7 For another flowchart illustrating the point cloud data registration method provided in this application embodiment, please refer to [link / reference]. Figure 7 The above steps S21-3 can also be achieved through the following steps:

[0171] Step S21-3-1: Obtain the depth parameters of each point cloud pair, and calculate the first weight of each point cloud pair based on the depth parameters;

[0172] Optionally, the depth parameter refers to the pixel depth captured by the camera. In one possible implementation, if the vehicle is equipped with a TOF camera, then the depth parameter is the pixel depth of each pixel in the image captured by the TOF camera. Optionally, the first weight can be calculated using the following formula:

[0173] W deg =1 / Z i2

[0174] Among them, W deg Representing the first weight, Z i The depth parameter represents the i-th point cloud pair.

[0175] Step S21-3-2: Decompose the variance matrix according to the preset decomposition algorithm to obtain multiple eigenvalues;

[0176] Step S21-3-3: Calculate the second weight of each point cloud pair based on the feature vector corresponding to the minimum value among multiple feature values, the coordinate information of the point cloud to be registered and the target point cloud in each point cloud pair;

[0177] Optionally, the preset decomposition algorithm can be SVD (Singular Value Decomposition).

[0178] In this embodiment, the electronic device can decompose the variance matrix using a preset decomposition algorithm to obtain multiple eigenvalues. Then, the electronic device can determine the smallest eigenvalue from the multiple eigenvalues ​​and determine the eigenvector corresponding to the smallest eigenvalue. Based on the eigenvector, the coordinate information of the point cloud to be registered and the target point cloud in each point cloud pair, the second weight of each point cloud pair is calculated.

[0179] Optionally, the feature vector represents the unstable descent direction of the point cloud data during the registration process; alternatively, the second weight represents the weight of each point cloud pair during the registration process.

[0180] Alternatively, the second weight can be calculated using the following formula:

[0181]

[0182] Among them, W sta Representing the second weight, V s p represents the eigenvector corresponding to the minimum value among multiple eigenvalues. i The three-dimensional coordinates of the point cloud to be registered in the i-th point cloud pair are represented. It represents the inversion of the normal vector of the i-th point cloud.

[0183] Step S21-3-4: Construct a weight function for the point cloud data based on the first and second weights of each point cloud pair.

[0184] Optionally, with the first weight value W deg The second weight is W sta Based on this, the weight function can be constructed using the following formula:

[0185] W final =W sta ·W deg

[0186] Among them, W final Characterize the weighting function.

[0187] Based on this, the electronic device can be based on the weight function W final And the above initial error function E=∑((R·p i +tq i )·n i ) 2 Construct the final error function for this point cloud data, namely:

[0188] E′=∑(W final ·(R·p i +tq i )·n i ) 2

[0189] Here, E′ represents the error function.

[0190] Optionally, after obtaining the error function, the electronic device can obtain the transformation matrix of the point cloud data by minimizing the error function. Specifically, in Figure 3 Based on the above, the calculation of the transformation matrix of the point cloud data according to the error function in step S22 can also be achieved through the following steps:

[0191] Minimize the error function to obtain the transformation matrix increment; calculate the transformation matrix based on the initial transformation matrix and the transformation matrix increment.

[0192] Optionally, the initial transformation matrix refers to the transformation matrix before obtaining the current transformation matrix increment.

[0193] In one possible implementation, if the electronic device calculates the transformation matrix of the point cloud data for the first time based on the error function, the initial transformation matrix can be a pre-stored transformation matrix; if the electronic device calculates the transformation matrix of the point cloud data based on the iterative error function, the initial transformation matrix refers to the transformation matrix obtained in the previous calculation.

[0194] In this embodiment, the electronic device can minimize the error matrix to obtain the transformation matrix increment, and then the electronic device can calculate the current transformation matrix based on the transformation matrix increment and the initial transformation matrix.

[0195] Optionally, after calculating the current transformation matrix, the electronic device can update the coordinate information of the point cloud to be registered in each point cloud pair according to the new information of the transformation matrix.

[0196] Optionally, in order to speed up the iteration rate of the error function, the electronic device can determine whether the point cloud to be registered in each point cloud pair has been normalized before each calculation of the transformation matrix of the point cloud data according to the error function, and normalize the point cloud to be registered in each point cloud pair if it is determined that the point cloud to be registered in each point cloud pair has not been normalized.

[0197] In this embodiment, the electronic device can determine whether the point cloud to be registered in each point cloud pair has undergone normalization processing based on the coordinate information of the point cloud to be registered. For example, for each point cloud to be registered, it can determine whether its three-dimensional coordinates all meet the preset coordinate values. If so, it is determined that it has undergone normalization processing. In this embodiment, each time the electronic device calculates the transformation matrix of the point cloud data according to the error function, it can first determine whether the point cloud to be registered in each point cloud pair has undergone normalization processing. If it is determined that it has not undergone normalization processing, it performs normalization processing on the point cloud to be registered in each point cloud pair to obtain new coordinate information, and then calculates the transformation matrix of the point cloud data based on the new coordinate information.

[0198] Optionally, the normalization process can be the same as the implementation process of step S12 above, which will not be elaborated on here.

[0199] Optionally, to ensure the accuracy of registration, the electronic device can pre-set preset iteration conditions to determine whether to exit the iteration of the error function and end the registration.

[0200] In one possible implementation, the preset iteration condition can be whether the current iteration count has reached a preset iteration count, where the iteration parameter is the current iteration count. In this case, step S23 can also be implemented through the following steps:

[0201] The error function is iterated according to the updated coordinate information of each point cloud to be registered, and it is determined whether the current iteration number has reached the preset iteration number; if the current iteration number has reached the preset iteration number, it is determined that the iteration parameters of the error function meet the preset iteration conditions.

[0202] Optionally, the electronic device can iterate the error function based on the coordinate information updated by the transformation matrix of the point cloud to be registered, and record the current iteration number. If the current iteration number has reached a preset threshold, it is determined that the iteration parameters of the error function meet the preset iteration conditions, and the iteration can be exited and the registration process can be ended. If the current iteration number has not yet reached the preset threshold, it is determined that the iteration parameters of the error function do not meet the preset iteration conditions, and it is necessary to continue iterating to obtain the iterated error function, calculate the transformation matrix, and achieve registration.

[0203] Optionally, the preset number of iterations can be set according to actual needs.

[0204] In another possible implementation, the preset iteration condition can be that the error value is less than a preset error threshold, and the iteration parameter is the error value. Specifically, step S23 above can also be implemented through the following steps:

[0205] The error function is iterated according to the updated coordinate information of each point cloud to be registered to obtain the iterated error function, and the error value is calculated based on the iterated error function; if the error value is less than the preset error threshold, it is determined that the iteration parameters of the error function meet the preset iteration conditions.

[0206] Optionally, the electronic device can iterate the error function based on the coordinate information updated by the transformation matrix of the point cloud to be registered, obtain the iterated error function, and calculate the error value of the iterated error function. If the error value is less than a preset error threshold, it is determined that the iteration parameters of the error function meet the preset iteration conditions, and the iteration can be exited and the registration process can be ended. If the error value is not less than the preset error threshold, it is determined that the iteration parameters of the error function do not meet the preset iteration conditions, and it is necessary to continue iterating to obtain the iterated error function, calculate the transformation matrix, and achieve registration.

[0207] Optionally, the preset error threshold can be set according to actual needs.

[0208] To perform the corresponding steps in the above embodiments and various possible methods, an implementation of a point cloud data registration device is given below. Optionally, the point cloud data registration device can adopt the above-described... Figure 2 The device structure of the electronic device is shown. Further, please refer to... Figure 8 , Figure 8 This is a functional block diagram of a point cloud data registration device provided in an embodiment of this application. It should be noted that the basic principle and technical effects of the point cloud data registration device provided in this embodiment are the same as those in the above embodiments. For the sake of brevity, any parts not mentioned in this embodiment can be referred to the corresponding content in the above embodiments. The point cloud data registration device includes: an acquisition module 200, a construction module 210, an update module 220, and an iteration module 230.

[0209] The acquisition module 200 is used to acquire point cloud data; the point cloud data includes multiple point clouds to be registered and multiple target point clouds; each point cloud to be registered and a corresponding target point cloud constitute a point cloud pair;

[0210] Understandably, the acquisition module 200 can be used to perform the above step S20;

[0211] The construction module 210 is used to construct an error function for the point cloud data based on the coordinate information of the point cloud to be registered and the target point cloud in each point cloud pair; wherein, the error function includes a weight function for the point cloud data, and the weight function is used to characterize the weight of each point cloud pair in the registration process;

[0212] Understandably, this building module 210 can be used to perform the above step S21;

[0213] The update module 220 is used to calculate the transformation matrix of the point cloud data according to the error function, and update the coordinate information of each point cloud to be registered according to the transformation matrix.

[0214] Understandably, the update module 220 can be used to perform the above step S22;

[0215] The iteration module 230 is used to iterate the error function according to the updated coordinate information of each point cloud to be registered until the iteration parameters of the error function meet the preset iteration conditions.

[0216] Understandably, the iterative module 230 can be used to perform the above step S23.

[0217] Optionally, the construction module 210 is further configured to: construct an initial error function for the point cloud data based on the coordinate information of the point cloud to be registered and the target point cloud in each point cloud pair; calculate the variance matrix of the point cloud data based on the initial error function; construct a weight function for the point cloud data based on the variance matrix; and construct an error function for the point cloud data based on the weight function and the initial error function.

[0218] Understandably, this building module 210 can also be used to perform the above steps S21-1 to S21-4.

[0219] Optionally, the construction module 210 is further configured to obtain the depth parameters of each point cloud pair, calculate the first weight of each point cloud pair based on the depth parameters, decompose the variance matrix according to a preset decomposition algorithm to obtain multiple eigenvalues, calculate the second weight of each point cloud pair based on the eigenvector corresponding to the minimum value among the multiple eigenvalues, the coordinate information of the point cloud to be registered and the target point cloud in each point cloud pair, and construct a weight function for the point cloud data based on the first weight and the second weight of each point cloud pair.

[0220] Understandably, this building module 210 can also be used to perform the above steps S21-3-1 to S21-3-4.

[0221] Optionally, the building module 210 is also configured to calculate a first weight according to the following formula:

[0222] W deg =1 / Z i2

[0223] Among them, W deg Representing the first weight, Z i The depth parameter represents the i-th point cloud pair.

[0224] Optionally, the building module 210 is also configured to calculate the second weight according to the following formula:

[0225]

[0226] Among them, W sta Representing the second weight, V s p represents the eigenvector corresponding to the minimum value among multiple eigenvalues. i The three-dimensional coordinates of the point cloud to be registered in the i-th point cloud pair are represented. It represents the inversion of the normal vector of the i-th point cloud.

[0227] Optionally, the building module 210 is also used to construct a weight function according to the following formula:

[0228] W final =W sta ·W deg

[0229] Among them, W final The weighting function, W sta Representing the second weight, W deg The first weight is represented. Optionally, the building module 210 is also used to construct an initial error function according to the following formula:

[0230] E=Σ((R·p i +tq i )·n i ) 2

[0231] Where E represents the initial error function, R represents the rotation matrix, and p i The 3D coordinates of the point cloud to be registered in the i-th point cloud pair are represented by t, and the translation parameter is q. i The three-dimensional coordinates of the target point cloud in the i-th point cloud pair are represented by n. i The normal vector representing the target point cloud in the i-th point cloud pair.

[0232] Optionally, the building module 210 is also used to calculate the variance matrix of the point cloud data according to the following formula:

[0233] C = A T A

[0234] A = [a i1 ,a i2 ,a i3 ,n ix ,n iy ,n iz ]

[0235] a i1 =n iz p iy -n iy p iz

[0236] a i2 =n ix p iz -n iz p ix

[0237] a i3 =n iy p ix -n ix p iy

[0238] Where C represents the variance matrix, A T The transpose of matrix A, n ix The normal vector of the target point cloud in the i-th point cloud pair along the x-axis, n iy The normal vector of the target point cloud in the i-th point cloud pair along the y-axis is represented by n. iz p represents the normal vector of the target point cloud in the i-th point cloud pair along the z-axis. ix p represents the x-axis coordinates of the point cloud to be registered in the i-th point cloud pair. iy p represents the y-coordinate of the point cloud to be registered in the i-th point cloud pair.iz The coordinates of the point cloud to be registered in the i-th point cloud pair are represented in the z-axis direction.

[0239] Optionally, the building module 210 is also used to construct an error function for the point cloud data according to the following formula:

[0240] E′=∑(W final ·(R·p i +tq i )·n i ) 2

[0241] Where E′ represents the error function, W final R represents the weight function, R represents the rotation matrix, and p i The 3D coordinates of the point cloud to be registered in the i-th point cloud pair are represented by t, and the translation parameter is q. i The three-dimensional coordinates of the target point cloud in the i-th point cloud pair are represented by n. i The normal vector representing the target point cloud in the i-th point cloud pair.

[0242] Optionally, the update module 220 is also used to minimize the error function to obtain the transformation matrix increment; and to calculate the transformation matrix based on the initial transformation matrix and the transformation matrix increment.

[0243] Optionally, the iteration module 230 is further configured to iterate the error function according to the updated coordinate information of each point cloud to be registered, and determine whether the current iteration number has reached the preset iteration number; if the current iteration number has reached the preset iteration number, then the iteration parameters of the error function are determined to meet the preset iteration conditions.

[0244] Optionally, the iteration module 230 is further configured to iterate the error function according to the updated coordinate information of each point cloud to be registered, obtain the iterated error function, and calculate the error value based on the iterated error function; if the error value is less than a preset error threshold, then the iteration parameters of the error function are determined to meet the preset iteration conditions.

[0245] Optionally, the update module 220 is further configured to normalize the point cloud to be registered in each point cloud pair if it is determined that the point cloud to be registered in each point cloud pair has not been normalized.

[0246] Optionally, the acquisition module 200 is further configured to acquire an initial point cloud, preprocess the initial point cloud to obtain multiple point clouds to be registered and multiple target point clouds corresponding to the same location at the same time; match each point cloud to be registered with the target point cloud to obtain multiple point cloud pairs; and normalize the point clouds to be registered in all point cloud pairs.

[0247] Understandably, the acquisition module 200 can also be used to perform the above steps S10 to S12;

[0248] Optionally, the acquisition module 200 is further configured to calculate the coordinates of the center point based on the initial coordinate information of the point clouds to be registered in all point cloud pairs; calculate the distance from the center point to each point cloud to be registered based on the initial coordinate information of the point clouds to be registered in each point cloud pair and the coordinates of the center point; and normalize the distance from the point clouds to the center point in each point cloud pair to obtain the coordinate information of the point clouds to be registered in each point cloud pair.

[0249] The point cloud data registration device provided in this application acquires point cloud data through an acquisition module. The point cloud data includes multiple point clouds to be registered and multiple target point clouds. Each point cloud to be registered and its corresponding target point cloud constitute a point cloud pair. A construction module constructs an error function for the point cloud data based on the coordinate information of the point clouds to be registered and the target point clouds in each point cloud pair. The error function includes a weight function for the point cloud data, which characterizes the weight of each point cloud pair in the registration process. An update module calculates the transformation matrix of the point cloud data based on the error function and updates the coordinate information of each point cloud to be registered according to the transformation matrix. An iteration module iterates the error function according to the updated coordinate information of each point cloud to be registered until the iteration parameters of the error function meet preset iteration conditions. By generating an error function with a weight function, this device can comprehensively consider the weight of each point cloud pair in the registration process when calculating the transformation matrix of the point cloud data, thereby obtaining a more accurate transformation matrix and improving registration accuracy.

[0250] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0251] In addition, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0252] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0253] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A point cloud data registration method, characterized in that, The method includes: Acquire point cloud data; the point cloud data includes multiple point clouds to be registered and multiple target point clouds; each point cloud to be registered and a corresponding target point cloud constitute a point cloud pair; Based on the coordinate information of the point cloud to be registered and the target point cloud in each point cloud pair, an error function for the point cloud data is constructed; wherein, the error function includes a weight function for the point cloud data, and the weight function is used to characterize the weight of each point cloud pair in the registration process; The step of constructing an error function for the point cloud data based on the coordinate information of the point cloud to be registered and the target point cloud in each point cloud pair includes: Based on the coordinate information of the point cloud to be registered and the target point cloud in each point cloud pair, an initial error function for the point cloud data is constructed. Calculate the variance matrix of the point cloud data based on the initial error function; Obtain the depth parameters of each point cloud pair, and calculate the first weight of each point cloud pair based on the depth parameters; The variance matrix is ​​decomposed according to a preset decomposition algorithm to obtain multiple eigenvalues; The second weight of each point cloud pair is calculated based on the feature vector corresponding to the minimum value among the plurality of feature values, the coordinate information of the point cloud to be registered and the target point cloud in each point cloud pair; Construct a weight function for the point cloud data based on the first and second weights of each point cloud pair; Based on the weight function and the initial error function, an error function for the point cloud data is constructed. The transformation matrix of the point cloud data is calculated based on the error function, and the coordinate information of each point cloud to be registered is updated based on the transformation matrix. The error function is iterated according to the updated coordinate information of each point cloud to be registered until the iteration parameters of the error function meet the preset iteration conditions.

2. The method according to claim 1, characterized in that, The step of calculating the first weight of each point cloud pair based on the depth parameter includes: The first weight is calculated according to the following formula: in, Characterizing the first weight, The depth parameter represents the i-th point cloud pair.

3. The method according to claim 1, characterized in that, The step of calculating the second weight of each point cloud pair based on the feature vector corresponding to the minimum value among the plurality of feature values, and the coordinate information of the point cloud to be registered and the target point cloud in each point cloud pair, includes: The second weight is calculated using the following formula: in, Characterizing the second weight, The eigenvector representing the minimum value among the plurality of eigenvalues. The three-dimensional coordinates of the point cloud to be registered in the i-th point cloud pair are represented. It represents the inversion of the normal vector of the i-th point cloud.

4. The method according to claim 1, characterized in that, The step of constructing a weight function for the point cloud data based on the first and second weights of each point cloud pair includes: The weight function is constructed according to the following formula: in, Characterizing the weight function, Characterizing the second weight, This represents the first weight.

5. The method according to claim 1, characterized in that, The step of constructing an initial error function for the point cloud data based on the coordinate information of the point cloud to be registered and the target point cloud in each point cloud pair includes: The initial error function is constructed according to the following formula: in, Characterizes the initial error function. Characterizing the rotation matrix, The three-dimensional coordinates of the point cloud to be registered in the i-th point cloud pair are represented. Characterizing translation parameters, The three-dimensional coordinates of the target point cloud in the i-th point cloud pair are represented. The normal vector representing the target point cloud in the i-th point cloud pair.

6. The method according to claim 1, characterized in that, The step of calculating the variance matrix of the point cloud data based on the initial error function includes: The variance matrix of the point cloud data is calculated using the following formula: in, Characterizing the variance matrix, Characterizes the transpose of matrix A. The normal vector representing the target point cloud in the i-th point cloud pair along the x-axis. The normal vector of the target point cloud in the i-th point cloud pair is represented by the y-axis direction. The normal vector representing the target point cloud in the i-th point cloud pair along the z-axis. The coordinates of the point cloud to be registered in the i-th point cloud pair are represented in the x-axis direction. The coordinates of the point cloud to be registered in the i-th point cloud pair are represented in the y-axis direction. The coordinates of the point cloud to be registered in the i-th point cloud pair are represented in the z-axis direction.

7. The method according to claim 1, characterized in that, The step of constructing the error function for the point cloud data based on the weight function and the initial error function includes: The error function for the point cloud data is constructed according to the following formula: in, Characterizing the error function, Characterizing the weight function, Characterizing the rotation matrix, The three-dimensional coordinates of the point cloud to be registered in the i-th point cloud pair are represented. Characterizing translation parameters, The three-dimensional coordinates of the target point cloud in the i-th point cloud pair are represented. The normal vector representing the target point cloud in the i-th point cloud pair.

8. The method according to claim 1, characterized in that, The step of calculating the transformation matrix of the point cloud data based on the error function includes: The error function is minimized to obtain the transformation matrix increment; The transformation matrix is ​​calculated based on the initial transformation matrix and the increment of the transformation matrix.

9. The method according to claim 1, characterized in that, The iteration parameters include the current iteration number. Iterating the error function according to the updated coordinate information of each point cloud to be registered until the iteration parameters of the error function meet preset iteration conditions includes: The error function is iterated according to the updated coordinate information of each point cloud to be registered, and it is determined whether the current iteration number has reached the preset iteration number. If the current iteration count reaches the preset iteration count, then the iteration parameters of the error function are determined to satisfy the preset iteration conditions.

10. The method according to claim 1, characterized in that, The iteration parameters include error values. The step of iterating the error function according to the updated coordinate information of each point cloud to be registered until the iteration parameters of the error function meet preset iteration conditions includes: The error function is iterated according to the updated coordinate information of each point cloud to be registered to obtain the iterated error function, and the error value is calculated according to the iterated error function. If the error value is less than a preset error threshold, then the iterative parameters of the error function are determined to satisfy the preset iterative conditions.

11. The method according to claim 1, characterized in that, Before the steps of calculating the transformation matrix of the point cloud data based on the error function and updating the coordinate information of each point cloud to be registered based on the transformation matrix, the method further includes: If it is determined that the point cloud to be registered in each point cloud pair has not been normalized, normalization processing is performed on the point cloud to be registered in each point cloud pair.

12. The method according to claim 1, characterized in that, Prior to the step of acquiring point cloud data, the method further includes: An initial point cloud is acquired and preprocessed to obtain multiple point clouds to be registered and multiple target point clouds corresponding to the same location at the same time. Each of the point clouds to be registered is matched with the target point cloud to obtain multiple point cloud pairs; Normalize the point clouds to be registered in all the point cloud pairs.

13. The method according to claim 12, characterized in that, The normalization process for the point clouds to be registered in all the point cloud pairs includes: Calculate the coordinates of the center point based on the initial coordinate information of the point clouds to be registered in all the point cloud pairs; Based on the initial coordinate information of the point cloud to be registered in each point cloud pair and the coordinates of the center point, calculate the distance from each point cloud to the center point. The distance from the point cloud to be registered in each point cloud pair to the center point is normalized to obtain the coordinate information of the point cloud to be registered in each point cloud pair.

14. A point cloud data registration device, characterized in that, The device includes: An acquisition module is used to acquire point cloud data; the point cloud data includes multiple point clouds to be registered and multiple target point clouds; each point cloud to be registered and a corresponding target point cloud constitute a point cloud pair. A construction module is used to construct an error function for the point cloud data based on the coordinate information of the point cloud to be registered and the target point cloud in each point cloud pair; wherein, the error function includes a weight function for the point cloud data, and the weight function is used to characterize the weight of each point cloud pair in the registration process; The construction module is further configured to: construct an initial error function for the point cloud data based on the coordinate information of the point cloud to be registered and the target point cloud in each point cloud pair; calculate the variance matrix of the point cloud data based on the initial error function; obtain the depth parameter of each point cloud pair and calculate a first weight for each point cloud pair based on the depth parameter; decompose the variance matrix according to a preset decomposition algorithm to obtain multiple eigenvalues; calculate a second weight for each point cloud pair based on the eigenvector corresponding to the minimum value among the multiple eigenvalues ​​and the coordinate information of the point cloud to be registered and the target point cloud in each point cloud pair; construct a weight function for the point cloud data based on the first weight and the second weight for each point cloud pair; and construct an error function for the point cloud data based on the weight function and the initial error function. The update module is used to calculate the transformation matrix of the point cloud data according to the error function, and update the coordinate information of each point cloud to be registered according to the transformation matrix; An iteration module is used to iterate the error function according to the updated coordinate information of each point cloud to be registered until the iteration parameters of the error function meet the preset iteration conditions.

15. An electronic device, characterized in that, It includes a processor and a memory, the memory storing a computer program executable by the processor, the processor being able to execute the computer program to implement the method of any one of claims 1-13.

16. A point cloud data registration system, characterized in that, The device includes the electronic device, the registration device, and the target device as described in claim 15, wherein the registration device is used to generate a point cloud to be registered, the target device is used to generate a target point cloud, and the registration device is a lidar and the target device is a TOF camera.

17. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-13.

Images