Point cloud splicing method and device, electronic equipment and storage medium
Through iterative calculation and update of point cloud intersection methods, the hand-eye calibration error and local optimization problems in point cloud splicing are solved, the splicing accuracy and efficiency are improved, and the splicing error and ghosting are reduced.
Patent Information
- Application Number
- CN202411092349.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-09
- Publication Date
- 2024-12-13
AI Technical Summary
The existing point cloud splicing technology is difficult to eliminate splicing ghosting under the influence of hand-eye calibration errors, and the registration-based method is prone to fall into local optimization when there is less intersection of external point clouds, resulting in splicing error and ghosting problems.
By acquiring and preprocessing the point cloud to be spliced, compute and iteratively update the intersection and transformation matrix between the two point clouds until the preset conditions are met, splicing is performed based on the target transformation matrix, reducing the running time of the ICP algorithm and avoiding local optimization question.
It effectively reduces splicing errors and ghosting, improves the accuracy and efficiency of point cloud splicing, and reduces the errors caused by hand-eye calibration.
Smart Images
Figure CN119130794A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of point cloud splicing, and in particular to a point cloud splicing method, device, electronic device and storage medium. Background Art
[0002] The existing point cloud stitching technology, method 1 follows the stitching paradigm X'=T·H·X, where T represents the TCP coordinates of the end of the robot arm, H represents the hand-eye calibration matrix, and X represents the input raw data. Method 2, based on the registration method, based on the local registration method, registers and stitches the point clouds in the two base coordinate systems obtained in method 1 to obtain the final stitching result.
[0003] Regarding method 1, directly performing hand-eye calibration conversion will be affected by the hand-eye calibration matrix error. When there is a certain error in the hand-eye calibration, the stitching ghosting error is difficult to eliminate.
[0004] Regarding the second method, based on the registration method, when the intersection between the external input point clouds is small, it will fall into a local optimal situation, and stitching errors will occur, resulting in ghosting. Summary of the invention
[0005] The object of the present invention is to provide a point cloud stitching method, device, electronic device and storage medium, which can improve the accuracy of point cloud stitching and avoid stitching errors.
[0006] In order to achieve the above purpose, the technical solution adopted in the embodiment of the present application is as follows:
[0007] In a first aspect, an embodiment of the present application provides a point cloud stitching method, the method comprising:
[0008] Obtain a first point cloud to be stitched and a second point cloud to be stitched;
[0009] Preprocess the first to-be-joined point cloud and the second to-be-joined point cloud respectively to obtain a first target to-be-joined point cloud and a second target to-be-joined point cloud;
[0010] Calculating a first intersection and a second intersection of the first target point cloud to be stitched and the second target point cloud to be stitched;
[0011] Calculating a transformation matrix of the first intersection and the second intersection;
[0012] Return to the step of calculating the first intersection and the second intersection of the first target point cloud to be spliced and the second target point cloud to be spliced to calculate the transformation matrix of the first intersection and the second intersection until a preset condition is met, thereby obtaining a target transformation matrix;
[0013] The first to-be-stitched point cloud and the second to-be-stitched point cloud are stitched together based on the target transformation matrix.
[0014] Optionally, the step of calculating a transformation matrix of the first intersection and the second intersection includes:
[0015] Determine the point pair weights of the first intersection and the second intersection and the centroid of the first intersection and the centroid of the second intersection;
[0016] A transformation matrix of the first intersection and the second intersection is calculated based on the first intersection, the second intersection, the centroid of the first intersection, the centroid of the second intersection, and point pair weights of the first intersection and the second intersection.
[0017] Optionally, the step of stitching the first to-be-stitched point cloud and the second to-be-stitched point cloud based on the target transformation matrix includes:
[0018] Based on the target transformation matrix, transforming the first point cloud to be stitched into the second point cloud to be stitched, to obtain a third point cloud to be stitched;
[0019] Calculating a difference between the second point cloud to be stitched and the third point cloud to be stitched;
[0020] The sum of the difference set and the second point cloud to be stitched is used as a stitched point cloud of the first point cloud to be stitched and the second point cloud to be stitched.
[0021] Optionally, the step of calculating a first intersection and a second intersection of the first target point cloud to be stitched and the second target point cloud to be stitched includes:
[0022] For each point in the first target point cloud to be stitched together, calculating a first distance between the point and each point in the second target point cloud to be stitched together;
[0023] Acquire each first target distance greater than a preset threshold from each first distance;
[0024] Constructing a first set based on points corresponding to each of the first target distances;
[0025] For each point in the second target point cloud to be stitched together, calculate a second distance between the point and each point in the first target point cloud to be stitched together;
[0026] Acquire each second target distance greater than a preset threshold from each second distance;
[0027] A second set is formed based on points corresponding to each of the second target distances.
[0028] Optionally, the step of determining the point pair weights of the first intersection and the second intersection includes:
[0029] For each point in the first intersection, calculate a third distance from a point in the second intersection corresponding to the point;
[0030] Based on the third distance, a weight of the point pair is obtained.
[0031] Optionally, a transformation matrix of the first intersection and the second intersection satisfies the following formula:
[0032]
[0033] Wherein, H is the transformation matrix of the first intersection and the second intersection, w i is the weight of the i-th point pair, n is the number of point pairs, p i is the coordinate of the i-th point in the first set, q i is the coordinate of the i-th point in the second set, centroid source is the centroid of the first set, centroid target is the centroid of the second set.
[0034] Optionally, the step of returning to execute the step of calculating the first intersection and the second intersection of the first target point cloud to be stitched and the second target point cloud to be stitched to calculate the transformation matrix of the first intersection and the second intersection until a preset condition is satisfied to obtain the target transformation matrix includes:
[0035] Returning to the step of calculating the first intersection and the second intersection of the first target point cloud to be stitched together and the second target point cloud to be stitched together to calculating the transformation matrix of the first intersection and the second intersection, and obtaining a first point cloud based on the transformation matrix and the second target point cloud to be stitched together;
[0036] Calculating fourth distances between each point in the first point cloud and each point corresponding to the first target point cloud to be stitched;
[0037] Calculating the sum of the fourth distances;
[0038] Calculating a ratio of a sum of the fourth distances to the number of points in the first point cloud;
[0039] comparing the ratio with a preset value;
[0040] When the ratio is less than or equal to the preset value, it is determined that the preset condition is met, and a target conversion matrix is obtained.
[0041] In a second aspect, an embodiment of the present application provides a point cloud stitching device, the device comprising:
[0042] An acquisition module, used for acquiring a first point cloud to be stitched and a second point cloud to be stitched;
[0043] A preprocessing module, used to preprocess the first to-be-stitched point cloud and the second to-be-stitched point cloud respectively to obtain a first target to-be-stitched point cloud and a second target to-be-stitched point cloud;
[0044] A calculation module, used for calculating the first intersection and the second intersection of the first target point cloud to be spliced and the second target point cloud to be spliced; calculating the transformation matrix of the first intersection and the second intersection; returning to execute the steps of calculating the first intersection and the second intersection of the first target point cloud to be spliced and the second target point cloud to calculate the transformation matrix of the first intersection and the second intersection until a preset condition is met, thereby obtaining a target transformation matrix;
[0045] A stitching module is used to stitch the first target point cloud to be stitched and the point cloud to be stitched based on the target transformation matrix.
[0046] In a third aspect, an embodiment of the present application provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the point cloud stitching method when executing the computer program.
[0047] In a fourth aspect, an embodiment of the present application provides a storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of the point cloud stitching method are implemented.
[0048] This application has the following beneficial effects:
[0049] The present application obtains the first point cloud to be spliced and the second point cloud to be spliced, pre-processes the first point cloud to be spliced and the second point cloud to be spliced, obtains the first target point cloud to be spliced and the second target point cloud to be spliced, calculates the first intersection and the second intersection of the first target point cloud to be spliced and the second target point cloud to be spliced, calculates the transformation matrix of the first intersection and the second intersection, returns to execute the steps of calculating the first intersection and the second intersection of the first target point cloud to be spliced and the second target point cloud to be spliced to calculating the transformation matrix of the first intersection and the second intersection, until the preset conditions are met, obtains the target transformation matrix, and splices the first target point cloud to be spliced and the point cloud to be spliced based on the target transformation matrix. By continuously iteratively calculating the intersection between the two point clouds, the running time of the ICP algorithm can be greatly reduced through the intersection, and at the same time, the problem of ICP registration falling into local optimality can be effectively reduced by calculating the intersection. And through the above-mentioned registration method, the splicing error caused by hand-eye calibration can be reduced. BRIEF DESCRIPTION OF THE DRAWINGS
[0050] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.
[0051] Figure 1 A block diagram of an electronic device provided by an embodiment of the present invention;
[0052] Figure 2 One of the flow charts of a point cloud stitching method provided by an embodiment of the present invention;
[0053] Figure 3 A second flowchart of a point cloud stitching method provided by an embodiment of the present invention;
[0054] Figure 4 A third flowchart of a point cloud stitching method provided by an embodiment of the present invention;
[0055] Figure 5 A fourth flowchart of a point cloud stitching method provided by an embodiment of the present invention;
[0056] Figure 6 A fifth flowchart of a point cloud stitching method provided by an embodiment of the present invention;
[0057] Figure 7 A sixth flowchart of a point cloud stitching method provided by an embodiment of the present invention;
[0058] Figure 8 A structural block diagram of a point cloud stitching device provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0059] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.
[0060] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0061] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.
[0062] In the description of the present invention, it should be noted that if the terms "upper", "lower", "inside", "outside", etc. appear to indicate an orientation or position relationship, they are based on the orientation or position relationship shown in the accompanying drawings, or are the orientation or position relationship in which the product of the invention is usually placed when used. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.
[0063] In addition, the terms “first”, “second”, etc., if used, are merely used to distinguish between the descriptions and should not be understood as indicating or implying relative importance.
[0064] In the description of this application, it should also be noted that, unless otherwise clearly specified and limited, the terms "set", "install", "connect", and "connect" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two elements. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.
[0065] After extensive research, it was found that the existing point cloud stitching technology, method 1 follows the stitching paradigm X'=T·H·X, where T represents the TCP coordinates of the end of the robot arm, H represents the hand-eye calibration matrix, and X represents the input raw data. Method 2, based on the registration method, is based on the local registration method. The point clouds in the two base coordinate systems obtained by method 1 are registered and stitched to obtain the final stitching result.
[0066] Regarding method 1, directly performing hand-eye calibration conversion will be affected by the hand-eye calibration matrix error. When there is a certain error in the hand-eye calibration, the stitching ghosting error is difficult to eliminate.
[0067] Regarding the second method, based on the registration method, when the intersection between the external input point clouds is small, it will fall into a local optimal situation, and stitching errors will occur, resulting in ghosting.
[0068] In view of the discovery of the above problems, the present embodiment provides a point cloud stitching method, device, electronic device and storage medium, which can obtain the first point cloud to be stitched and the second point cloud to be stitched, respectively pre-process the first point cloud to be stitched and the second point cloud to be stitched, obtain the first target point cloud to be stitched and the second target point cloud to be stitched, calculate the first intersection and the second intersection of the first target point cloud to be stitched and the second target point cloud to be stitched, calculate the transformation matrix of the first intersection and the second intersection, return to execute the steps of calculating the first intersection and the second intersection of the first target point cloud to be stitched and the second target point cloud to be stitched to calculate the transformation matrix of the first intersection and the second intersection, until the preset conditions are met, the target transformation matrix is obtained, and the first target point cloud to be stitched and the point cloud to be stitched are stitched based on the target transformation matrix. By continuously iteratively calculating the intersection between the two point clouds, the running time of the ICP algorithm can be greatly reduced through the intersection, and at the same time, the problem of ICP registration falling into a local optimum can be effectively reduced by calculating the intersection. Furthermore, through the above-mentioned registration method, the stitching error caused by hand-eye calibration can be reduced. The solution provided in this embodiment is described in detail below.
[0069] This embodiment provides an electronic device that can stitch point clouds. In a possible implementation, the electronic device can be a user terminal, for example, the electronic device can be, but is not limited to, a server, a smart phone, a personal computer (PC), a tablet computer, a personal digital assistant (PDA), a mobile Internet device (MID), etc.
[0070] Please refer to Figure 1 , Figure 1 1 is a schematic diagram of the structure of the electronic device 100 provided in the embodiment of the present application. The electronic device 100 may also include Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown. Figure 1 Each component shown in the figure can be implemented by hardware, software or a combination thereof.
[0071] The electronic device 100 includes a point cloud stitching device 110 , a memory 120 and a processor 130 .
[0072] The components of the memory 120 and the processor 130 are electrically connected to each other directly or indirectly to realize data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines. The point cloud stitching device 110 includes at least one software function module that can be stored in the memory 120 in the form of software or firmware or solidified in the operating system (OS) of the electronic device 100. The processor 130 is used to execute the executable modules stored in the memory 120, such as the software function modules and computer programs included in the point cloud stitching device 110.
[0073] The memory 120 may be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), etc. The memory 120 is used to store a program, and the processor 130 executes the program after receiving an execution instruction.
[0074] Please refer to Figure 2 , Figure 2 For application Figure 1 A flow chart of a point cloud stitching method of an electronic device 100 is provided, and the method including each step is described in detail below.
[0075] S201: Acquire a first point cloud to be stitched and a second point cloud to be stitched.
[0076] S202: Preprocessing the first to-be-stitched point cloud and the second to-be-stitched point cloud respectively to obtain a first target to-be-stitched point cloud and a second target to-be-stitched point cloud.
[0077] S203: Calculate a first intersection and a second intersection of the first target point cloud to be stitched together and the second target point cloud to be stitched together.
[0078] S204: Calculate a transformation matrix of the first intersection and the second intersection.
[0079] S205: Return to the step of calculating the first intersection and the second intersection of the first target point cloud to be stitched together and the second target point cloud to be stitched together to calculate the transformation matrix of the first intersection and the second intersection until a preset condition is met to obtain a target transformation matrix.
[0080] S206: stitching the first to-be-stitched point cloud and the second to-be-stitched point cloud based on the target transformation matrix.
[0081] It should be noted that there are multiple ways to preprocess the first point cloud to be spliced and the second point cloud to be spliced. In one implementation, in order to improve the point cloud splicing quality of the first point cloud to be spliced and the second point cloud to be spliced, and eliminate the influence of messy noise points, RANSAC plane fitting can be used to remove ground points to eliminate the interference of the ground on point cloud splicing. Statistical filtering can be applied to the first point cloud to be spliced and the second point cloud to be spliced after removing the ground points to remove isolated noise points in the point cloud.
[0082] Subsequently, KD-Tree is used to construct a point cloud index, and the first point cloud to be spliced is divided into multiple connected domains through region growing or Euclidean clustering algorithm. The second point cloud to be spliced is divided into multiple connected domains through region growing or Euclidean clustering algorithm. The largest connected domain with the largest number of points in the multiple connected domains corresponding to the first point cloud to be spliced is retained, and the remaining small connected domains are removed to obtain the first connected domain. The largest connected domain with the largest number of points in the multiple connected domains corresponding to the second point cloud to be spliced is retained, and the remaining small connected domains are removed to obtain the second connected domain. The first connected domain and the second connected domain are downsampled respectively using voxel grid filtering to reduce the amount of data and improve computational efficiency. The point cloud in the downsampled first connected domain is converted to the base coordinate system to obtain the first target point cloud to be spliced. The point cloud in the downsampled second connected domain is converted to the base coordinate system to obtain the second target point cloud to be spliced.
[0083] Calculate the first intersection of the first target point cloud to be stitched on the second target point cloud to be stitched, calculate the second intersection of the second target point cloud to be stitched on the first target point cloud to be stitched, use the first intersection and the second intersection as the initial input of the ICP registration algorithm, execute the ICP registration algorithm, obtain the transformation matrix of the first intersection and the second intersection, calculate the new first intersection of the first intersection on the second intersection, calculate the new second intersection of the second intersection on the first intersection, use the new first intersection and the new second intersection as the initial input of the ICP registration algorithm, execute the ICP configuration algorithm, obtain the transformation matrix of the new first intersection and the new second intersection, repeat the above steps until the preset conditions are met, obtain the latest first intersection and the latest second intersection, and determine the target transformation matrix of the latest first intersection and the latest second intersection. Splice the first target point cloud to be stitched and the point cloud to be stitched based on the target transformation matrix.
[0084] There are many ways to calculate the transformation matrix of the first intersection and the second intersection. In one implementation, for example, Figure 3 As shown, the following steps are included:
[0085] S204-1: Determine the point pair weights of the first intersection and the second intersection and the centroid of the first intersection and the centroid of the second intersection.
[0086] S204-2: Calculate a transformation matrix of the first intersection and the second intersection based on the first intersection, the second intersection, the centroid of the first intersection, the centroid of the second intersection, and point pair weights of the first intersection and the second intersection.
[0087] The transformation matrix of the first intersection and the second intersection satisfies the following formula:
[0088]
[0089] Wherein, H is the transformation matrix of the first intersection and the second intersection, w i is the weight of the i-th point pair, n is the number of point pairs, p i is the coordinate of the i-th point in the first set, q i is the coordinate of the i-th point in the second set, centroid source is the centroid of the first set, centroid target is the centroid of the second set.
[0090] There are many implementation methods for splicing the first to-be-spliced point cloud and the second to-be-spliced point cloud based on the target transformation matrix. In one implementation method, Figure 4 As shown, the following steps are included:
[0091] S206 - 1 : Based on the target transformation matrix, transform the first point cloud to be stitched into the second point cloud to be stitched, to obtain a third point cloud to be stitched.
[0092] S206-2: Calculate the difference between the second to-be-joined point cloud and the third to-be-joined point cloud.
[0093] S206-3: taking the sum of the difference set and the second point cloud to be stitched together as a stitched point cloud of the first point cloud to be stitched together and the second point cloud to be stitched together.
[0094] In order to prevent the problem of ghosting after point cloud stitching, the difference between the second point cloud to be stitched and the third point cloud to be stitched after transformation based on the target transformation matrix is calculated, and the second point cloud to be stitched and the difference are summed to obtain the final stitching result.
[0095] There are many implementation methods for calculating the first intersection and the second intersection of the first target to-be-joined point cloud and the second target to-be-joined point cloud. In one implementation method, for example: Figure 5 As shown, the following steps are included:
[0096] S203-1: For each point in the first target point cloud to be stitched together, calculate a first distance between the point and each point in the second target point cloud to be stitched together.
[0097] S203-2: Acquire each first target distance greater than a preset threshold from each first distance.
[0098] S203-3: Construct a first set based on the points corresponding to each of the first target distances.
[0099] S203-4: For each point in the second target point cloud to be stitched together, calculate a second distance between the point and each point in the first target point cloud to be stitched together.
[0100] S203-5: Acquire each second target distance greater than a preset threshold from each second distance.
[0101] S203-6: Construct a second set based on the points corresponding to each of the second target distances.
[0102] It should be noted that the first target point cloud to be stitched includes point A, point B and point C, and the second target point cloud to be stitched includes point D, point E and point F.
[0103] For point A, the first distance between point A and point D, the first distance between point A and point E, and the first distance between point A and point F are calculated respectively, and the first target distance greater than the preset threshold is determined from the first distance between point A and point D, the first distance between point A and point E, and the first distance between point A and point F. For example, if the first distance between point A and point D is greater than the preset threshold, and the first distance between point A and point E is greater than the preset threshold, then the first distance between point A and point D is greater than the preset threshold and the first distance between point A and point E is taken as the first target distance, and point D and point E are taken as elements of the first set. For point B, the first distance between point B and point D, the first distance between point B and point E, and the first distance between point B and point F are calculated respectively, and the first target distance greater than the preset threshold is determined from the first distance between point B and point D, the first distance between point B and point E, and the first distance between point B and point F. If the first distance between point B and point D is greater than the preset distance, then the first distance between point B and point D is determined as the first target distance, and point D is taken as an element of the first set. For point C, the first distance between point C and point D, the first distance between point C and point E, and the first distance between point C and point F are calculated respectively, and the first target distance greater than the preset threshold is determined from the first distance between point C and point D, the first distance between point C and point E, and the first distance between point C and point F. If the first distance between point C and point D is greater than the preset distance, the first distance between point C and point D is determined as the first target distance, and point D is taken as an element of the first set. Finally, point D and point E constitute the first set.
[0104] The method for determining the second intersection is similar to that for determining the first intersection, and will not be described in detail here.
[0105] There are many ways to determine the weights of the point pairs of the first intersection and the second intersection. In one implementation, Figure 6 As shown, the following steps are included:
[0106] S301: For each point in the first intersection, calculate a third distance from a point in the second intersection corresponding to the point.
[0107] S302: Obtaining a weight of the point pair based on the third distance.
[0108] For example, when the first intersection includes point D and point E, and the second intersection includes point A and point B, and point D and point A form a point pair, and point E and point B form a point pair, the third distance between point A and point D is calculated, and the weight of the point pair between point A and point D is obtained by the formula of 1 / 1+third distance.
[0109] There are many ways to determine whether the preset conditions are met. In one implementation, Figure 7 As shown, the following steps are included:
[0110] S401: Return to the step of calculating the first intersection and the second intersection of the first target point cloud to be stitched together and the second target point cloud to be stitched together to calculate the transformation matrix of the first intersection and the second intersection, and obtain a first point cloud based on the transformation matrix and the second target point cloud to be stitched together.
[0111] S402: Calculate fourth distances between each point in the first point cloud and each point corresponding to the first target point cloud to be stitched.
[0112] S403: Calculate the sum of the fourth distances.
[0113] S404: Calculate the ratio of the sum of the fourth distances to the number of points in the first point cloud.
[0114] S405: Compare the ratio with a preset value.
[0115] S406: When the ratio is less than or equal to the preset value, determine that the preset condition is met, and obtain a target conversion matrix.
[0116] In another implementation of determining whether the preset condition is satisfied, when the number of cycles is reached, it is determined that the preset condition is satisfied, and a target conversion matrix is obtained based on the new first intersection and the new second intersection.
[0117] Please refer to Figure 8 The present application embodiment also provides a method for applying Figure 1 The point cloud stitching device 110 of the electronic device 100 includes:
[0118] An acquisition module 111 is used to acquire a first point cloud to be stitched and a second point cloud to be stitched;
[0119] A preprocessing module 112, used for preprocessing the first to-be-joined point cloud and the second to-be-joined point cloud respectively to obtain a first target to-be-joined point cloud and a second target to-be-joined point cloud;
[0120] The calculation module 113 is used to calculate the first intersection and the second intersection of the first target point cloud to be spliced and the second target point cloud to be spliced; calculate the transformation matrix of the first intersection and the second intersection; return to the step of calculating the first intersection and the second intersection of the first target point cloud to be spliced and the second target point cloud to calculate the transformation matrix of the first intersection and the second intersection until a preset condition is met to obtain a target transformation matrix;
[0121] The stitching module 114 is configured to stitch the first target point cloud to be stitched and the point cloud to be stitched based on the target transformation matrix.
[0122] The present application also provides an electronic device 100, which includes a processor 130 and a memory 120. The memory 120 stores computer executable instructions, and when the computer executable instructions are executed by the processor 130, the point cloud stitching method is implemented.
[0123] The embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by the processor 130, the point cloud stitching method is implemented.
[0124] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to the multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or the flowchart, and the combination of boxes in the block diagram and / or the flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.
[0125] In addition, each functional module in each embodiment of the present application can be integrated together to form an independent part, or each module can exist separately, or two or more modules can be integrated to form an independent part. If the function is implemented in the form of a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a disk or an optical disk.
[0126] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.
[0127] The above are only various implementations of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
Claims
1. A point cloud stitching method, characterized in that: The method comprises: Obtain a first point cloud to be stitched and a second point cloud to be stitched; Preprocess the first to-be-joined point cloud and the second to-be-joined point cloud respectively to obtain a first target to-be-joined point cloud and a second target to-be-joined point cloud; Calculating a first intersection and a second intersection of the first target point cloud to be stitched and the second target point cloud to be stitched; Calculating a transformation matrix of the first intersection and the second intersection; Return to the step of calculating the first intersection and the second intersection of the first target point cloud to be spliced and the second target point cloud to be spliced to calculate the transformation matrix of the first intersection and the second intersection until a preset condition is met, thereby obtaining a target transformation matrix; The first to-be-stitched point cloud and the second to-be-stitched point cloud are stitched together based on the target transformation matrix.
2. The method according to claim 1, characterized in that The step of calculating the transformation matrix of the first intersection and the second intersection comprises: Determine the point pair weights of the first intersection and the second intersection and the centroid of the first intersection and the centroid of the second intersection; A transformation matrix of the first intersection and the second intersection is calculated based on the first intersection, the second intersection, the centroid of the first intersection, the centroid of the second intersection, and point pair weights of the first intersection and the second intersection.
3. The method according to claim 1, characterized in that: The step of stitching the first to-be-stitched point cloud and the second to-be-stitched point cloud based on the target transformation matrix comprises: Based on the target transformation matrix, transforming the first point cloud to be stitched into the second point cloud to be stitched, to obtain a third point cloud to be stitched; Calculating a difference between the second point cloud to be stitched and the third point cloud to be stitched; The sum of the difference set and the second point cloud to be stitched is used as a stitched point cloud of the first point cloud to be stitched and the second point cloud to be stitched.
4. The method according to claim 1, characterized in that: The step of calculating the first intersection and the second intersection of the first target point cloud to be spliced and the second target point cloud to be spliced includes: For each point in the first target point cloud to be stitched together, calculating a first distance between the point and each point in the second target point cloud to be stitched together; Acquire each first target distance greater than a preset threshold from each first distance; Constructing a first set based on points corresponding to each of the first target distances; For each point in the second target point cloud to be stitched together, calculate a second distance between the point and each point in the first target point cloud to be stitched together; Acquire each second target distance greater than a preset threshold from each second distance; A second set is formed based on points corresponding to each of the second target distances.
5. The method according to claim 2, characterized in that: The step of determining the point pair weights of the first intersection and the second intersection comprises: For each point in the first intersection, calculate a third distance from a point in the second intersection corresponding to the point; Based on the third distance, a weight of the point pair is obtained.
6. The method according to claim 2, characterized in that The transformation matrix of the first intersection and the second intersection satisfies the following formula: Wherein, H is the transformation matrix of the first intersection and the second intersection, w i is the weight of the i-th point pair, n is the number of point pairs, p i is the coordinate of the i-th point in the first set, q i is the coordinate of the i-th point in the second set, centroid source is the centroid of the first set, centroid target is the centroid of the second set.
7. The method according to claim 3, characterized in that The step of returning to the step of calculating the first intersection and the second intersection of the first target point cloud to be spliced and the second target point cloud to be spliced to the step of calculating the transformation matrix of the first intersection and the second intersection until a preset condition is met to obtain the target transformation matrix includes: Returning to the step of calculating the first intersection and the second intersection of the first target point cloud to be stitched together and the second target point cloud to be stitched together to calculating the transformation matrix of the first intersection and the second intersection, and obtaining a first point cloud based on the transformation matrix and the second target point cloud to be stitched together; Calculating fourth distances between each point in the first point cloud and each point corresponding to the first target point cloud to be stitched; Calculating the sum of the fourth distances; Calculating a ratio of a sum of the fourth distances to the number of points in the first point cloud; comparing the ratio with a preset value; When the ratio is less than or equal to the preset value, it is determined that the preset condition is met, and a target conversion matrix is obtained.
8. A point cloud stitching device, characterized in that: The device comprises: An acquisition module, used for acquiring a first point cloud to be stitched and a second point cloud to be stitched; A preprocessing module, used to preprocess the first to-be-stitched point cloud and the second to-be-stitched point cloud respectively to obtain a first target to-be-stitched point cloud and a second target to-be-stitched point cloud; A calculation module, used for calculating the first intersection and the second intersection of the first target point cloud to be spliced and the second target point cloud to be spliced; calculating the transformation matrix of the first intersection and the second intersection; returning to execute the steps of calculating the first intersection and the second intersection of the first target point cloud to be spliced and the second target point cloud to calculate the transformation matrix of the first intersection and the second intersection until a preset condition is met, thereby obtaining a target transformation matrix; A stitching module is used to stitch the first target point cloud to be stitched and the point cloud to be stitched based on the target transformation matrix.
9. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method according to any one of claims 1 to 7 when executing the computer program.
10. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.