A point cloud registration method, device, electronic equipment and readable storage medium

Abstract

The application provides a point cloud registration method and device, electronic equipment and a readable storage medium. The method comprises: for each z-axis rotation amount in the z-axis rotation set, determining the relevant parameters between the parent point cloud and the first new point cloud after determining the first transformation parameter; updating the initial transformation parameter according to the relevant parameters of each z-axis rotation amount to obtain the second transformation parameter; processing the parent point cloud and the child point cloud according to the second transformation parameter to obtain the transformed parent point cloud and the second new point cloud; for each z-axis translation amount in the z-axis translation set, calculating the minimum distance sum between the edge vector of the transformed parent point cloud and the edge vector of the third new point cloud; updating the second transformation parameter according to the minimum distance sum of each z-axis translation amount to obtain the third transformation parameter; and adjusting the child point cloud according to the third transformation parameter to make the child point cloud and the parent point cloud registered. The application accurately registers the child point cloud and the parent point cloud based on the feature map similarity method and the edge vector of the point cloud, and improves the registration accuracy.

Description

Technical Field

[0001] The present application belongs to the field of spatial positioning technology, and in particular relates to a point cloud registration method, device, electronic device and readable storage medium. Background Art

[0002] Point cloud registration technology is widely used in surveying and mapping and SLAM (Simultaneous Localization And Mapping). In surveying and mapping, a precise model of the target object is established by registering multiple point cloud images. In SLAM, real-time or offline mapping is performed through real-time point cloud registration; the target is self-positioned in real time by matching the offline point cloud map with the point cloud of the target captured in real time. Point cloud registration technology includes coarse matching technology. The coarse matching process roughly matches two or more point cloud images based on points, and the error is controlled at the centimeter or decimeter level. However, in the process of collecting different point clouds, different collection positions and heights occur, which makes the coarse matching technology have the problem of unstable matching accuracy. Summary of the Invention

[0003] The embodiments of the present application provide a point cloud registration method, device, electronic device, readable storage medium and computer program product, which can solve the problem of unstable matching accuracy in coarse matching technology.

[0004] In a first aspect, an embodiment of the present application provides a point cloud registration method, comprising:

[0005] Obtain the parent point cloud and child point cloud of the external environment;

[0006] For each z-axis rotation amount in the z-axis rotation set, after obtaining the corresponding first transformation parameters, determining the correlation parameters between the parent point cloud and the first new point cloud, the z-axis rotation set including a plurality of different z-axis rotation amounts, the first new point cloud being the child point cloud processed by the first transformation parameters;

[0007] updating the initial transformation parameters according to the relevant parameters of each of the z-axis rotation amounts to obtain second transformation parameters;

[0008] Processing the mother point cloud and the child point cloud according to the second transformation parameters to obtain a transformed mother point cloud and a second new point cloud;

[0009] For each z-axis translation in the z-axis translation set, calculating the minimum sum of distances between edge vectors of the transformed mother point cloud and edge vectors of a third new point cloud, wherein the z-axis translation set includes a plurality of different z-axis translations, and the third new point cloud is the second new point cloud processed by the z-axis translations;

[0010] updating the second transformation parameter according to the minimum sum of the z-axis translation amounts to obtain a third transformation parameter;

[0011] Adjusting the sub-point cloud according to the third transformation parameter so that the sub-point cloud is registered with the mother point cloud;

[0012] The transformation parameters include the rotation amount of each axis and the translation amount of each axis.

[0013] In one embodiment, for each z-axis rotation amount in the z-axis rotation set, after obtaining the corresponding first transformation parameter, determining the correlation parameters between the parent point cloud and the first new point cloud includes:

[0014] For each z-axis rotation amount, updating the initial transformation parameter according to the z-axis rotation amount to obtain the corresponding first transformation parameter;

[0015] Adjusting the sub-point cloud according to the first transformation parameter to obtain the first new point cloud;

[0016] Calculating a correlation matrix between the parent point cloud and the first new point cloud, wherein the rows of the correlation matrix are labeled as x-axis coordinates and the columns are labeled as y-axis coordinates;

[0017] Determine the x-axis coordinate and the y-axis coordinate of the extreme value element corresponding to the correlation matrix, wherein the correlation parameters include the x-axis coordinate and the y-axis coordinate corresponding to the extreme value element.

[0018] In one embodiment, calculating the correlation matrix between the parent point cloud and the first new point cloud includes:

[0019] Generating a grid projection map of the mother point cloud and a grid projection map of the child point cloud respectively;

[0020] The correlation matrix between the grid projection image of the mother point cloud and the grid projection image of the child point cloud is calculated.

[0021] In one embodiment, updating the initial transformation parameters according to the parameters related to each of the z-axis rotation amounts includes:

[0022] Determining target related parameters among the related parameters of each of the z-axis rotation amounts, wherein the target related parameters include a target x-axis coordinate and a target y-axis coordinate corresponding to a target extreme value element, wherein the target extreme value element is the largest extreme value element among the extreme value elements;

[0023] The initial transformation parameters are updated according to the target-related parameters and the corresponding target z-axis rotation amount.

[0024] In one embodiment, before calculating the minimum sum of distances between the edge vectors of the transformed mother point cloud and the edge vectors of the third new point cloud, the method further includes:

[0025] Determine the z-axis translation range;

[0026] Based on the preset height resolution and the z-axis translation range, a z-axis translation amount set is established.

[0027] In one embodiment, calculating the minimum sum of distances between edge vectors of the transformed mother point cloud and edge vectors of the third new point cloud for each z-axis translation in the z-axis translation set includes:

[0028] For each of the z-axis translation amounts, translate the second new point cloud and the edge vector of the second new point cloud according to the z-axis translation amount to obtain the third new point cloud and the edge vector of the third new point cloud;

[0029] For the edge vector set of the third new point cloud, the following steps are repeated until the termination condition is met:

[0030] Get the edge vector to be processed;

[0031] Searching for a target edge vector in the edge vectors of the transformed mother point cloud according to the angle of the edge vector to be processed, wherein the difference between the angle of the target edge vector and the angle of the edge vector to be processed is less than a preset angle threshold;

[0032] Calculating the distance between each target edge vector and the edge vector to be processed;

[0033] Determine a target distance value among the distance values, wherein the target distance value is the minimum distance value;

[0034] After the termination condition is satisfied, summing the target distance values ​​of each edge vector of the third new point cloud to obtain the minimum distance sum;

[0035] The termination condition is that all edge vectors of the third new point cloud have been processed.

[0036] In one embodiment, updating the second transformation parameter according to the minimum sum of the distances of the z-axis translations includes:

[0037] Establishing an equation based on each of the z-axis translation amounts and the corresponding minimum distance sum;

[0038] determining a target derivative of the equation, the target derivative being a maximum second-order derivative;

[0039] The second transformation parameter is updated according to the target z-axis translation, where the target z-axis translation is the z-axis translation corresponding to the target derivative.

[0040] In a second aspect, an embodiment of the present application provides a point cloud registration device, comprising:

[0041] The acquisition module is used to obtain the parent point cloud and child point cloud of the external environment;

[0042] a first processing module configured to determine, for each z-axis rotation amount in a z-axis rotation set, a correlation parameter between the parent point cloud and a first new point cloud after obtaining a corresponding first transformation parameter, the z-axis rotation set including a plurality of different z-axis rotation amounts, the first new point cloud being the child point cloud processed by the first transformation parameter;

[0043] further configured to update the first transformation parameters according to the parameters related to the z-axis rotation amounts to obtain second transformation parameters;

[0044] A second processing module is used to process the mother point cloud and the child point cloud according to the second transformation parameters to obtain a transformed mother point cloud and a second new point cloud;

[0045] further configured to calculate, for each z-axis translation in a z-axis translation set, a minimum sum of distances between edge vectors of the transformed mother point cloud and edge vectors of a third new point cloud, wherein the z-axis translation set includes a plurality of different z-axis translations, and the third new point cloud is the second new point cloud processed by the z-axis translations;

[0046] further configured to update the second transformation parameter according to the minimum sum of the distances of the z-axis translation amounts to obtain a third transformation parameter;

[0047] a registration module, configured to adjust the sub-point cloud according to the third transformation parameter so as to align the sub-point cloud with the mother point cloud;

[0048] The transformation parameters include the rotation amount of each axis and the translation amount of each axis.

[0049] In a third aspect, an embodiment of the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method described in any one of the first aspects above is implemented.

[0050] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method as described in any one of the above-mentioned first aspects is implemented.

[0051] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when executed on an electronic device, enables the electronic device to execute any one of the methods described in the first aspect above.

[0052] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0053] The embodiment of the present application determines the relevant parameters between the mother point cloud and the first new point cloud after obtaining the corresponding first transformation parameters for each z-axis rotation amount in the z-axis rotation set; updates the initial transformation parameters according to the relevant parameters of each z-axis rotation amount to obtain the second transformation parameters; processes the mother point cloud and the child point cloud according to the second transformation parameters to obtain the transformed mother point cloud and the second new point cloud; calculates the minimum distance sum between the edge vector of the transformed mother point cloud and the edge vector of the third new point cloud for each z-axis translation amount in the z-axis translation set; updates the second transformation parameters according to the minimum distance sum of each z-axis translation amount to obtain the third transformation parameters; adjusts the child point cloud according to the third transformation parameters to align the child point cloud with the mother point cloud. In this way, based on the feature map similarity method and the edge vector of the point cloud, the child point cloud can be accurately aligned with the mother point cloud, the alignment accuracy is improved, and the problem of unstable alignment accuracy of the coarse matching technology is solved.

[0054] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0056] Figure 1 This is a flowchart of a point cloud registration method provided by an embodiment of the present application;

[0057] Figure 2 Schematic diagram of the structure of a point cloud registration device provided in one embodiment of the present application;

[0058] Figure 3 It is a structural diagram of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0059] In the following description, specific details such as specific system structures and techniques are provided for purposes of illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present application with unnecessary detail.

[0060] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or collections thereof.

[0061] It will also be understood that the term "and / or" used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0062] As used in this specification and the appended claims, the term "if" can be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0063] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0064] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present application. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0065] Figure 1 This is a flow chart of the point cloud registration method provided by an embodiment of the present application. Figure 1 As shown, the method includes:

[0066] S11: Obtain the parent point cloud and child point clouds of the external environment.

[0067] In applications, a point cloud of the external environment is collected using a camera or radar. For example, the camera can be a depth camera, which generates a point cloud based on the depth information output by the depth camera. The radar can be a lidar, which generates a point cloud based on the received beam output by the lidar.

[0068] The parent point cloud and the child point cloud can be adjacent point clouds or point clouds separated by several frames. The acquisition time of the parent point cloud is earlier than the acquisition time of the child point cloud.

[0069] S12: For each z-axis rotation amount in the z-axis rotation set, after obtaining the corresponding first transformation parameters, determining the relevant parameters between the parent point cloud and the first new point cloud.

[0070] The z-axis rotation set includes multiple different z-axis rotation amounts, and the first new point cloud is a sub-point cloud processed by the first transformation parameter.

[0071] In the application, pre-acquire the initial transformation parameters, which include the initial rotation and translation for each axis. The z-axis rotation set includes the initial z-axis rotation and the set z-axis rotation after adding the rotation increment to the initial z-axis rotation.

[0072] For example, a z-axis rotation set is established, including rz, rz+90°, rz+180°, and rz+270°, where rz is the initial rotation of the z-axis, and rz+90°, rz+180°, and rz+270° are the set rotations of the z-axis.

[0073] When the z-axis rotation amount reaches the initial z-axis rotation amount, the initial transformation parameters are obtained as the first transformation parameters. The child point cloud is rotated according to the first transformation parameters to obtain a first new point cloud. The correlation matrix between the parent point cloud and the first new point cloud is calculated, and the coordinate information of the extreme value elements in the correlation matrix is ​​determined as the correlation parameters.

[0074] When the z-axis rotation amount is the set z-axis rotation amount, replace the initial z-axis rotation amount in the initial transformation parameters with the set z-axis rotation amount to obtain the first transformation parameters. Rotate the child point cloud according to the first transformation parameters to obtain the first new point cloud. Calculate the correlation matrix between the parent point cloud and the first new point cloud, and determine the coordinate information of the extreme value elements in the correlation matrix as the correlation parameters.

[0075] S13: Update the initial transformation parameters according to the relevant parameters of each z-axis rotation amount to obtain the second transformation parameters.

[0076] In the application, a target extreme value element is determined based on the extreme value elements in each correlation matrix. The target extreme value element is the largest extreme value element. The coordinate information and z-axis rotation amount corresponding to the target extreme value element are determined. Based on the coordinate information and z-axis rotation amount corresponding to the target extreme value element, the rotation and translation amounts in the initial transformation parameters are updated to obtain the second transformation parameters.

[0077] S14: Processing the mother point cloud and the child point cloud according to the second transformation parameters to obtain a transformed mother point cloud and a second new point cloud.

[0078] In the application, the mother point cloud and the child point cloud are rotated according to the second transformation parameter to obtain the transformed mother point cloud and the second new point cloud.

[0079] S15: For each z-axis translation amount in the z-axis translation set, calculate the minimum sum of distances between the edge vectors of the transformed mother point cloud and the edge vectors of the third new point cloud.

[0080] The z-axis translation set includes a plurality of different z-axis translation amounts, and the third new point cloud is the second new point cloud processed by the z-axis translation amounts.

[0081] In the application, for each edge vector in the third new point cloud, the edge vector of the corresponding transformed mother point cloud is determined according to the edge vector of the third new point cloud, the minimum distance between the edge vector of the third new point cloud and the edge vector of the corresponding transformed mother point cloud is calculated, and all minimum distances are calculated to obtain the minimum distance sum.

[0082] In one possible implementation, establishing a z-axis translation set includes:

[0083] Determine the z-axis translation range; based on the preset height resolution and the z-axis translation range, establish a z-axis translation amount set.

[0084] For example, based on the parent point cloud and child point clouds, the z-axis translation range is determined to be [-5, 5]. The height resolution is preset to 0.2, and the z-axis translation value set {-5, -4.8, -4.6…4.8, 5} is established.

[0085] In one possible implementation, obtaining the edge vector of the transformed mother point cloud includes:

[0086] Obtain an edge vector of the mother point cloud; rotate the edge vector of the mother point cloud according to the second transformation parameter to obtain an edge vector of the transformed mother point cloud.

[0087] Similarly, obtain the edge vector of the third new point cloud.

[0088] In one possible implementation, obtaining the edge vector of the transformed mother point cloud includes:

[0089] Obtain an edge point set of the transformed mother point cloud; and determine an edge vector of the transformed mother point cloud based on the edge point set of the transformed mother point cloud.

[0090] Similarly, obtain the edge vector of the third new point cloud.

[0091] The edge vector is determined based on the edge point set, including:

[0092] Determine the offset rate of the point according to the normal vector and the sum vector of the point; filter points whose offset rate is greater than a preset offset rate to obtain an edge point set, the edge point set including at least one edge point; and determine the edge vector according to the edge point set.

[0093] Specifically, the offset rate calculation formula is: Among them, DivergencyP i Point P i The offset rate, Point P i The normal vector of Point P i The sum vector of the edge points in the edge point set is obtained by performing straight line fitting.

[0094] Get the normal vector of the point, including:

[0095] Obtain multiple neighboring points of a point; analyze the neighboring points to obtain an eigenvalue set and a first vector set. The eigenvalue set includes multiple eigenvalues, and the first vector set includes the first vector of each eigenvalue. Determine the first vector corresponding to the target eigenvalue as the normal vector. The target eigenvalue is the smallest eigenvalue in the eigenvalue set.

[0096] Get the sum of points and vectors, including:

[0097] Acquire multiple neighboring points of the point; establish a second vector set, where the second vector set includes a second vector of the point and each neighboring point; and perform an average calculation on the second vector set to obtain a sum vector.

[0098] S16: Update the second transformation parameter according to the minimum sum of the distances of each z-axis translation to obtain a third transformation parameter.

[0099] In the application, the target z-axis translation is determined based on the minimum sum of the distances of the z-axis translations, and the translation in the second transformation parameter is updated based on the target z-axis translation to obtain the third transformation parameter.

[0100] S17: Adjust the sub-point cloud according to the third transformation parameter to align the sub-point cloud with the mother point cloud.

[0101] In the application, a transformation matrix is ​​determined according to the third transformation parameter, and the sub-point cloud is adjusted according to the transformation matrix.

[0102] The embodiment of the present application determines the relevant parameters between the mother point cloud and the first new point cloud after obtaining the corresponding first transformation parameters for each z-axis rotation amount in the z-axis rotation set; updates the initial transformation parameters according to the relevant parameters of each z-axis rotation amount to obtain the second transformation parameters; processes the mother point cloud and the child point cloud according to the second transformation parameters to obtain the transformed mother point cloud and the second new point cloud; calculates the minimum distance sum between the edge vector of the transformed mother point cloud and the edge vector of the third new point cloud for each z-axis translation amount in the z-axis translation set; updates the second transformation parameters according to the minimum distance sum of each z-axis translation amount to obtain the third transformation parameters; adjusts the child point cloud according to the third transformation parameters to align the child point cloud with the mother point cloud. In this way, based on the feature map similarity method and the edge vector of the point cloud, the child point cloud can be accurately aligned with the mother point cloud, the alignment accuracy is improved, and the problem of unstable alignment accuracy of the coarse matching technology is solved.

[0103] In one embodiment, step S12 includes:

[0104] S21: For each z-axis rotation amount, update the initial transformation parameter according to the z-axis rotation amount to obtain the corresponding first transformation parameter.

[0105] S22: Adjust the sub-point cloud according to the first transformation parameter to obtain a first new point cloud.

[0106] S23: Calculate the correlation matrix between the parent point cloud and the first new point cloud.

[0107] The rows of the correlation matrix are labeled as x-axis coordinates and the columns are labeled as y-axis coordinates.

[0108] In a possible implementation, step S23 includes:

[0109] S231: Generate a grid projection image of the mother point cloud and a grid projection image of the child point cloud respectively.

[0110] Specifically, the parent point cloud and child point cloud are projected onto the xoy plane, with the grid projection image's coordinate system being the xy-axis coordinate system. The parent point cloud projection and child point cloud projection images are then rasterized to obtain the parent point cloud's grid projection and the child point cloud's grid projection. The amplitude of each grid cell is the number of points in the grid, n, or exp(n). The grid size is adjusted based on the actual usage scenario. Generally, densely populated grids have smaller grids, while sparsely populated grids have larger grids.

[0111] S232: Calculate the correlation matrix between the grid projection image of the mother point cloud and the grid projection image of the child point cloud.

[0112] Specifically, a grid matrix of the mother point cloud is generated based on the grid projection of the mother point cloud. A grid matrix of the child point cloud is generated based on the grid projection of the child point cloud. The elements of the grid matrix are amplitudes, the rows are labeled as x-axis coordinates, and the columns are labeled as y-axis coordinates.

[0113] S24: Determine the x-axis coordinate and the y-axis coordinate of the extreme value element corresponding to the correlation matrix, where the relevant parameters include the x-axis coordinate and the y-axis coordinate of the extreme value element.

[0114] Step S13 includes:

[0115] S131: Determine target related parameters among the related parameters of each z-axis rotation amount.

[0116] The target-related parameters include the target x-axis coordinate and the target y-axis coordinate corresponding to the target extreme value element, and the target extreme value element is the largest extreme value element among all extreme value elements.

[0117] In the application, among the parameters related to each z-axis rotation, the x-axis and y-axis coordinates corresponding to the largest extreme value element are selected to obtain the target-related parameter. Simultaneously, the z-axis rotation corresponding to the largest extreme value element is obtained, i.e., the target z-axis rotation. This target z-axis rotation also corresponds to the target-related parameter.

[0118] For example, parameters related to rz, rz+90°, rz+180°, and rz+270° are obtained. rz is the initial z-axis rotation, and rz+90°, rz+180°, and rz+270° are the set z-axis rotations. The extreme element at rz+90° is determined to be the largest extreme element, and the target extreme element is obtained. The x-axis and y-axis coordinates corresponding to the extreme element at rz+90° are selected to obtain the target parameters and the target z-axis rotation, rz+90°.

[0119] S132: Update the initial transformation parameters according to the target-related parameters and the corresponding target z-axis rotation amount.

[0120] In the application, the initial x-axis translation in the initial transformation parameters is replaced with the target x-axis coordinate, the initial y-axis translation in the initial transformation parameters is replaced with the target y-axis coordinate, and the initial z-axis rotation in the initial transformation parameters is replaced with the target z-axis rotation to obtain the second transformation parameters. The second transformation parameters include the initial x-axis rotation, the initial y-axis rotation, the updated z-axis rotation (target z-axis rotation), the updated x-axis translation (target x-axis coordinate), the updated y-axis translation (target y-axis coordinate), and the initial z-axis translation.

[0121] Step S15 includes:

[0122] S151: For each z-axis translation amount, translate the second new point cloud and the edge vector of the second new point cloud according to the z-axis translation amount to obtain a third new point cloud and the edge vector of the third new point cloud.

[0123] S152: Repeat steps S153 to S156 for the edge vector set of the third new point cloud until the termination condition is met:

[0124] S153: Obtain the edge vector to be processed.

[0125] In the application, a to-be-processed edge vector is searched in the edge vector set.

[0126] S154: Searching for a target edge vector in the edge vectors of the transformed mother point cloud according to the angle of the edge vector to be processed.

[0127] The difference between the angle of the target edge vector and the angle of the edge vector to be processed is less than a preset angle threshold.

[0128] In the application, for each edge vector in the transformed mother point cloud, the angle difference between the edge vector of the transformed mother point cloud and the edge vector to be processed is calculated. The edge vectors of the transformed mother point cloud whose angle difference is less than a preset angle threshold are screened to obtain the target edge vector. The target edge vector is approximately parallel to the edge vector to be processed.

[0129] S155: Calculate the distance between each target edge vector and the edge vector to be processed.

[0130] In the application, the vertical distance between the target edge vector and the edge vector to be processed is calculated to obtain a distance value.

[0131] S156: Determine a target distance value among the distance values.

[0132] Among them, the target distance value is the minimum distance value.

[0133] S157: After the termination condition is met, the target distance value of each edge vector of the third new point cloud is summed to obtain the minimum distance sum.

[0134] The termination condition is that all edge vectors of the third new point cloud have been processed.

[0135] In the application, after executing steps S154 to S156 for each edge vector in the third new point cloud, the sum of all target distance values ​​is calculated to obtain the minimum distance sum.

[0136] Step S16 includes:

[0137] S161: Establish an equation based on each z-axis translation amount and the corresponding minimum distance sum.

[0138] In one possible implementation, an equation is obtained by fitting based on the z-axis translation and the minimum distance sum. The specific equation is Dist=Dist(dZ), where Dist is the minimum distance sum and dZ is the z-axis translation.

[0139] S162: Determine the target derivative of the equation.

[0140] Among them, the target derivative is the largest second-order derivative.

[0141] In application, the second derivative function of the equation is d 2 Dist=d 2 Dist(dZ) / d(dZ) 2 Based on the second-order derivative function, the second-order derivative is calculated, and then the maximum second-order derivative is determined to obtain the target derivative.

[0142] S164: Update the second transformation parameter according to the target z-axis translation amount.

[0143] The target z-axis translation is the z-axis translation of the corresponding target derivative.

[0144] In the application, the initial z-axis translation in the second transformation parameters is replaced with the target z-axis translation to obtain the third transformation parameters. The third transformation parameters include the initial x-axis rotation, the initial y-axis rotation, the updated z-axis rotation (target z-axis rotation), the updated x-axis translation (target x-axis coordinate), the updated y-axis translation (target y-axis coordinate), and the updated z-axis translation (target z-axis translation).

[0145] Correspondingly, the transformation matrix is ​​determined according to the third transformation parameter, and the sub-point cloud is adjusted according to the transformation matrix. The transformation matrix is ​​specifically

[0146] Among them, tx is the updated x-axis translation, ty is the updated y-axis translation, and tz is the updated z-axis translation.

[0147]

[0148] rx is the initial rotation of the x-axis, ry is the initial rotation of the y-axis, and r*z is the updated rotation of the z-axis.

[0149] The embodiment of the present application determines the translation of the x and y axes and the rotation of the z axis through the plane feature map similarity method to obtain accurate translation of the x and y axes and rotation of the z axis, and determines the translation of the z axis through the edge vector distance comparison method to obtain accurate translation of the z axis, providing a basis for accurate alignment of the sub-point cloud with the parent point cloud, especially providing a basis for accurate translation alignment of the sub-point cloud with the parent point cloud, and also providing a basis for accurate alignment of targets with repetitive and similar features in height (such as stairs).

[0150] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0151] Corresponding to the method described in the above embodiment, for the sake of convenience of explanation, only the part related to the embodiment of the present application is shown.

[0152] Figure 2 Schematic diagram of the structure of the point cloud registration device provided by an embodiment of the present application. Figure 2 As shown, the device includes:

[0153] An acquisition module 10 is used to acquire a parent point cloud and a child point cloud of an external environment;

[0154] A first processing module 11 is configured to determine, for each z-axis rotation amount in the z-axis rotation set, a correlation parameter between a parent point cloud and a first new point cloud after obtaining a corresponding first transformation parameter, wherein the z-axis rotation set includes a plurality of different z-axis rotation amounts, and the first new point cloud is a child point cloud processed with the first transformation parameter;

[0155] It is also used to update the first transformation parameters according to the relevant parameters of each z-axis rotation amount to obtain the second transformation parameters;

[0156] The second processing module 12 is used to process the mother point cloud and the child point cloud according to the second transformation parameters to obtain a transformed mother point cloud and a second new point cloud;

[0157] further configured to calculate, for each z-axis translation in the z-axis translation set, a minimum sum of distances between edge vectors of the transformed parent point cloud and edge vectors of a third new point cloud, wherein the z-axis translation set includes a plurality of different z-axis translations, and the third new point cloud is the second new point cloud processed by the z-axis translations;

[0158] further configured to update the second transformation parameter according to the minimum sum of the distances of the z-axis translations to obtain a third transformation parameter;

[0159] The registration module 13 is configured to adjust the sub-point cloud according to the third transformation parameter so as to register the sub-point cloud with the mother point cloud.

[0160] In one embodiment, the first processing module is specifically used to update the initial transformation parameters according to the z-axis rotation amount for each z-axis rotation amount to obtain the corresponding first transformation parameters; adjust the sub-point cloud according to the first transformation parameters to obtain the first new point cloud; calculate the correlation matrix between the mother point cloud and the first new point cloud, the rows of the correlation matrix are labeled as x-axis coordinates and the columns are labeled as y-axis coordinates; determine the x-axis coordinates and y-axis coordinates of the extreme elements of the corresponding correlation matrix, and the related parameters include the x-axis coordinates and y-axis coordinates of the corresponding extreme elements.

[0161] In one embodiment, the first processing module is specifically configured to generate a grid projection image of the mother point cloud and a grid projection image of the child point cloud respectively; and calculate a correlation matrix between the grid projection image of the mother point cloud and the grid projection image of the child point cloud.

[0162] In one embodiment, the first processing module is specifically used to determine target-related parameters among the related parameters of each z-axis rotation amount, where the target-related parameters include the target x-axis coordinates and target y-axis coordinates corresponding to the target extreme value elements, and the target extreme value element is the largest extreme value element among all the extreme value elements; and the initial transformation parameters are updated according to the target-related parameters and the corresponding target z-axis rotation amount.

[0163] In one embodiment, the second processing module is further configured to determine a z-axis translation range; and to establish a z-axis translation amount set based on the preset height resolution and the z-axis translation range.

[0164] In one embodiment, the second processing module is specifically used to translate the second new point cloud and the edge vector of the second new point cloud according to the z-axis translation amount for each z-axis translation amount to obtain the third new point cloud and the edge vector of the third new point cloud; for the edge vector set of the third new point cloud, repeatedly perform the following steps until the termination condition is met: obtain the edge vector to be processed; according to the angle of the edge vector to be processed, search for the target edge vector in the edge vector of the transformed mother point cloud, and the difference between the angle of the target edge vector and the angle of the edge vector to be processed is less than a preset angle threshold; calculate the distance value between each target edge vector and the edge vector to be processed; determine the target distance value among each distance value, and the target distance value is the minimum distance value; after the termination condition is met, sum the target distance value of each edge vector of the third new point cloud to obtain the minimum distance sum; wherein the termination condition is that all edge vectors of the third new point cloud have been processed.

[0165] In one embodiment, the second processing module is specifically used to establish an equation based on each z-axis translation and the corresponding minimum distance sum; determine the target derivative of the equation, the target derivative is the largest second-order derivative; according to the target z-axis translation, update the second transformation parameter, the target z-axis translation is the z-axis translation corresponding to the target derivative.

[0166] Figure 3 This is a schematic diagram of the structure of an electronic device provided in one embodiment of the present application. Figure 3 As shown, the electronic device 2 of this embodiment includes: at least one processor 20 ( Figure 3 Only one is shown), a memory 21 and a computer program 22 stored in the memory 21 and executable on the at least one processor 20, wherein the processor 20 implements the steps of any of the above-mentioned method embodiments when executing the computer program 22.

[0167] The electronic device 2 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The electronic device 2 may include, but is not limited to, a processor 20 and a memory 21. Those skilled in the art will understand that Figure 3 This is merely an example of the electronic device 2 and does not constitute a limitation on the electronic device 2 . The electronic device 2 may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device 2 may also include input and output devices, network access devices, etc.

[0168] The processor 20 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor may be a microprocessor or any conventional processor.

[0169] In some embodiments, the memory 21 may be an internal storage unit of the electronic device 2, such as a hard disk or memory of the electronic device 2. In other embodiments, the memory 21 may also be an external storage device of the electronic device 2, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device 2. Furthermore, the memory 21 may also include both an internal storage unit of the electronic device 2 and an external storage device. The memory 21 is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as the program code of the computer program. The memory 21 may also be used to temporarily store data that has been output or is to be output.

[0170] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of this application. Their specific functions and technical effects can be found in the method embodiment section and will not be repeated here.

[0171] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0172] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned various method embodiments can be implemented.

[0173] An embodiment of the present application provides a computer program product. When the computer program product is run on an electronic device, the electronic device can implement the steps of the above-mentioned method embodiments when executing the computer program product.

[0174] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the process of the above-mentioned method embodiment by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, it can implement the steps of each of the above-mentioned method embodiments. The computer program includes computer program code, which can be in source code form, object code form, executable file, or some intermediate form. The computer-readable medium can at least include: any entity or device capable of carrying computer program code to the camera / terminal device, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal, and software distribution medium. For example, a USB flash drive, mobile hard drive, magnetic disk, or optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals or telecommunication signals.

[0175] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0176] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0177] In the embodiments provided in this application, it should be understood that the disclosed devices / network equipment and methods can be implemented in other ways. For example, the device / network equipment embodiments described above are merely illustrative. For example, the division of the modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0178] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0179] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A point cloud registration method, characterized in that: include: Obtain the parent point cloud and child point cloud of the external environment; For each z-axis rotation amount in the z-axis rotation set, after obtaining the corresponding first transformation parameters, determining the correlation parameters between the parent point cloud and the first new point cloud, the z-axis rotation set including a plurality of different z-axis rotation amounts, the first new point cloud being the child point cloud processed by the first transformation parameters; updating the initial transformation parameters according to the relevant parameters of each of the z-axis rotation amounts to obtain second transformation parameters; Processing the mother point cloud and the child point cloud according to the second transformation parameters to obtain a transformed mother point cloud and a second new point cloud; For each z-axis translation in the z-axis translation set, calculating the minimum sum of distances between edge vectors of the transformed mother point cloud and edge vectors of a third new point cloud, wherein the z-axis translation set includes a plurality of different z-axis translations, and the third new point cloud is the second new point cloud processed by the z-axis translations; updating the second transformation parameter according to the minimum sum of the z-axis translation amounts to obtain a third transformation parameter; Adjusting the sub-point cloud according to the third transformation parameter so that the sub-point cloud is registered with the mother point cloud; The transformation parameters include the rotation amount of each axis and the translation amount of each axis.

2. The method according to claim 1, characterized in that After obtaining the corresponding first transformation parameters for each z-axis rotation amount in the z-axis rotation set, determining the correlation parameters between the parent point cloud and the first new point cloud includes: For each z-axis rotation amount, updating the initial transformation parameter according to the z-axis rotation amount to obtain the corresponding first transformation parameter; Adjusting the sub-point cloud according to the first transformation parameter to obtain the first new point cloud; Calculating a correlation matrix between the parent point cloud and the first new point cloud, wherein the rows of the correlation matrix are labeled as x-axis coordinates and the columns are labeled as y-axis coordinates; Determine the x-axis coordinate and the y-axis coordinate of the extreme value element corresponding to the correlation matrix, wherein the correlation parameters include the x-axis coordinate and the y-axis coordinate corresponding to the extreme value element.

3. The method according to claim 2, characterized in that The calculating a correlation matrix between the parent point cloud and the first new point cloud includes: Generating a grid projection map of the mother point cloud and a grid projection map of the child point cloud respectively; The correlation matrix between the grid projection image of the mother point cloud and the grid projection image of the child point cloud is calculated.

4. The method according to claim 2, characterized in that The updating of the initial transformation parameters according to the parameters related to the z-axis rotation amounts includes: Determining target related parameters among the related parameters of each of the z-axis rotation amounts, wherein the target related parameters include a target x-axis coordinate and a target y-axis coordinate corresponding to a target extreme value element, wherein the target extreme value element is the largest extreme value element among the extreme value elements; The initial transformation parameters are updated according to the target-related parameters and the corresponding target z-axis rotation amount.

5. The method according to any one of claims 1 to 4, characterized in that Before calculating the minimum sum of the distances between the edge vectors of the transformed mother point cloud and the edge vectors of the third new point cloud, the method further includes: Determine the z-axis translation range; Based on the preset height resolution and the z-axis translation range, a z-axis translation amount set is established.

6. The method according to claim 5, characterized in that The step of calculating the minimum sum of distances between edge vectors of the transformed mother point cloud and edge vectors of the third new point cloud for each z-axis translation amount in the z-axis translation set includes: For each of the z-axis translation amounts, translate the second new point cloud and the edge vector of the second new point cloud according to the z-axis translation amount to obtain the third new point cloud and the edge vector of the third new point cloud; For the edge vector set of the third new point cloud, the following steps are repeated until the termination condition is met: Get the edge vector to be processed; Searching for a target edge vector in the edge vectors of the transformed mother point cloud according to the angle of the edge vector to be processed, wherein the difference between the angle of the target edge vector and the angle of the edge vector to be processed is less than a preset angle threshold; Calculating the distance between each target edge vector and the edge vector to be processed; Determine a target distance value among the distance values, wherein the target distance value is the minimum distance value; After the termination condition is satisfied, summing the target distance values ​​of each edge vector of the third new point cloud to obtain the minimum distance sum; The termination condition is that all edge vectors of the third new point cloud have been processed.

7. The method according to claim 6, characterized in that The updating of the second transformation parameter according to the minimum sum of the distances of the z-axis translations includes: Establishing an equation based on each of the z-axis translation amounts and the corresponding minimum distance sum; determining a target derivative of the equation, the target derivative being a maximum second-order derivative; The second transformation parameter is updated according to the target z-axis translation, where the target z-axis translation is the z-axis translation corresponding to the target derivative.

8. A point cloud registration device, characterized in that: include: The acquisition module is used to obtain the parent point cloud and child point cloud of the external environment; a first processing module configured to determine, for each z-axis rotation amount in a z-axis rotation set, a correlation parameter between the parent point cloud and a first new point cloud after obtaining a corresponding first transformation parameter, the z-axis rotation set including a plurality of different z-axis rotation amounts, the first new point cloud being the child point cloud processed by the first transformation parameter; further configured to update the first transformation parameters according to the parameters related to the z-axis rotation amounts to obtain second transformation parameters; A second processing module is used to process the mother point cloud and the child point cloud according to the second transformation parameters to obtain a transformed mother point cloud and a second new point cloud; further configured to calculate, for each z-axis translation in a z-axis translation set, a minimum sum of distances between edge vectors of the transformed mother point cloud and edge vectors of a third new point cloud, wherein the z-axis translation set includes a plurality of different z-axis translations, and the third new point cloud is the second new point cloud processed by the z-axis translations; further configured to update the second transformation parameter according to the minimum sum of the distances of the z-axis translation amounts to obtain a third transformation parameter; a registration module, configured to adjust the sub-point cloud according to the third transformation parameter so as to align the sub-point cloud with the mother point cloud; The transformation parameters include the rotation amount of each axis and the translation amount of each axis.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images