Point cloud registration method, device, computer equipment and storage medium

Abstract

The present application relates to a point cloud registration method, device, computer equipment and storage medium. The method comprises: using the robotic arm to carry a 3D camera to shoot a target object at multiple angles, and obtaining point cloud data of the target object at different angles; the target object is an object without surface texture features or with a smooth surface; according to the pose matrix between the 3D camera and the end of the robotic arm, the point cloud data is converted to the base coordinate system of the robotic arm to obtain a rough registration result of the point cloud data; using a multi-laser feature projector to project a set number of feature spots in the overlapping area of ​​the point cloud at different angles, and detecting the laser point pairs of the point cloud to be registered from the feature spots according to the set rules; using the singular value decomposition algorithm to solve the optimal rotation matrix and translation matrix of the laser point pair, and performing point cloud registration according to the optimal rotation matrix and translation matrix. The present application effectively eliminates errors and improves the quality and speed of point cloud registration.

Description

Technical Field

[0001] The present application belongs to the field of machine vision technology, and in particular relates to a point cloud registration method, apparatus, computer equipment, and storage medium. Background Art

[0002] Point cloud registration is a key technology for aligning and fusing multiple point cloud datasets or point cloud data from multiple perspectives. It can obtain complete three-dimensional information about the target surface and has broad applications in computer vision and computer graphics, including autonomous driving, robotic navigation, map production, architectural reconstruction, and cultural heritage preservation. Point cloud registration can resolve inconsistencies in position, pose, scale, and other aspects of multiple point cloud datasets. By registering these point cloud datasets from different sensors, at different time points, or from different perspectives into a common coordinate system, it enables precise 3D data fusion and reconstruction for analysis, measurement, and control.

[0003] Currently, the commonly used point cloud registration algorithms include two categories:

[0004] The first category is feature-based matching algorithms; these algorithms include two approaches: one is to use image processing techniques such as SIFT, HOG, and SURF to extract feature points from the image, calculate their positions in three-dimensional space based on camera parameters, and then use methods such as SVD to solve the rotation and translation transformations between the two sets of feature points. This approach fails when the target surface lacks texture information. The other is to directly calculate local or global feature sub-elements such as FPFH and SHOT from the point cloud, match the corresponding relationships based on their similarity, and then use methods such as SVD to solve the rotation and translation transformations between the two sets of feature sub-elements. This approach fails when the target surface is too smooth or noisy.

[0005] The second type is an iterative matching algorithm. This algorithm optimizes the point pair error in the overlapping region, minimizing the error through a continuous iterative optimization process, thereby obtaining the rotational and translational transformations between the two point clouds. This algorithm also requires finding a sufficient number of regions with distinct features and similarities within the point clouds to be registered as overlapping regions. When stitching together flat or smooth surface point clouds without distinct texture features, the stitched surface is prone to "drifting." Summary of the Invention

[0006] The present application provides a point cloud registration method, apparatus, computer device, and storage medium, which aim to solve at least one of the above-mentioned technical problems in the prior art to a certain extent.

[0007] In order to solve the above problems, this application provides the following technical solutions:

[0008] A point cloud registration method, comprising:

[0009] Performing hand-eye calibration on the robotic arm and the 3D camera using a 3D camera and a calibration plate to obtain a pose matrix between the 3D camera and the end of the robotic arm; wherein the 3D camera is fixed to the end of the robotic arm;

[0010] The target object is photographed at multiple angles by the 3D camera carried by the robotic arm to obtain point cloud data of the target object at different angles; the target object is an object without surface texture features or with a smooth surface;

[0011] Converting the point cloud data into a base coordinate system of the robotic arm according to a pose matrix between the 3D camera and the end of the robotic arm to obtain a coarse registration result of the point cloud data;

[0012] Based on the coarse registration result of the point cloud data, a multi-laser feature projector is used to project a set number of feature spots in the overlapping area of ​​the point cloud at different angles, and laser point pairs of the point cloud to be registered are detected from the feature spots according to set rules;

[0013] The optimal rotation matrix and translation matrix of the laser point pair are solved by using a singular value decomposition algorithm, and the point cloud to be registered is registered according to the optimal rotation matrix and translation matrix.

[0014] The technical solution adopted in the embodiment of the present application also includes: using the 3D camera and the calibration plate to perform hand-eye calibration on the robotic arm and the 3D camera, specifically:

[0015] Find the pose of the 3D camera and the center point of the tool at the end of the robotic arm For the case where the camera is fixed at the end of the robotic arm and the eye is on the hand, the relative position of the camera and the end of the robotic arm is Fixed, the relative position of the calibration plate and the robotic arm base According to the fixed relative posture, the conversion matrix between the calibration plate and the camera can be used under different postures of two sets of robotic arms. And the relative position of the end of the robot arm to the base Establish the equation:

[0016]

[0017]

[0018]

[0019] in, Indicates the pose relationship between the calibration object and the base coordinate during the i-th shooting. Indicates the pose relationship between the calibration object and the base coordinate during the j-th shooting;

[0020] The hand-eye calibration problem is organized as follows:

[0021] AX=XB

[0022] Then, based on Tsai's calibration algorithm, the equation AX=XB is solved to complete the hand-eye calibration of the robotic arm and 3D camera.

[0023] The technical solution adopted in the embodiment of the present application further includes: photographing the target object from multiple angles by the robotic arm carrying a 3D camera, and then further includes:

[0024] Acquiring texture images of a target object at different angles using the multi-laser feature projector;

[0025] The center points of the light spots of the multi-laser feature projector are obtained according to the texture image, and the 3D coordinates corresponding to each center point of the light spot are calculated.

[0026] The technical solution adopted in the embodiment of the present application also includes: obtaining the center point of the light spot of the multi-laser feature projector according to the texture image, and calculating the 3D coordinates corresponding to each center point of the light spot, specifically:

[0027] Dedistorting the texture image using camera intrinsic parameters, and calculating the center point of the light spot of the multi-laser feature projector using a Hough circle fitting algorithm;

[0028] The 3D coordinates of the center point of the light spot are obtained through the projection conversion relationship between the point cloud data and the texture image in the 3D camera.

[0029] The technical solution adopted in the embodiment of the present application further includes: converting the point cloud data into the base coordinate system of the robotic arm according to the pose matrix between the 3D camera and the end of the robotic arm to obtain a coarse registration result of the point cloud data, specifically:

[0030] Convert the target object from the world coordinate system to the camera coordinate system;

[0031] The point cloud data of the target object {P i}Convert to the tool center point coordinate system at the end of the robot arm;

[0032] Using the pose relationship between the tool center point at the end of the manipulator and the base coordinate system The point cloud data {P i}Convert to the base coordinate system:

[0033]

[0034] The technical solution adopted by the embodiment of the present application also includes: using a multi-laser feature projector to project a set number of feature spots in the overlapping area of ​​the point cloud at different angles, and detecting laser point pairs of the point cloud to be registered from the feature spots according to set rules, specifically:

[0035] For each characteristic light spot, a sphere is constructed with the accuracy of the robotic arm as the radius, and it is determined whether other characteristic light spots fall within the sphere. If so, the two characteristic light spots within the same sphere are considered to be corresponding points, and the laser point pairs of the point cloud to be aligned are obtained.

[0036] The technical solution adopted in the embodiment of the present application also includes: using the singular value decomposition algorithm to solve the optimal rotation matrix and translation matrix of the laser point pair, and registering the point cloud to be registered according to the optimal rotation matrix and translation matrix, specifically:

[0037] Based on the center point of the light spot and its 3D coordinates, the coordinate matrix of the laser point pair is converted into an orthogonal matrix using a singular value decomposition algorithm, and the product of the orthogonal matrices is calculated to obtain a rotation matrix;

[0038] Calculating the mean difference of the laser point pairs to obtain a translation matrix;

[0039] The point cloud to be registered is converted into the coordinate system of the reference point cloud by using the rotation matrix and the translation matrix, thereby completing the precise registration of the point cloud to be registered.

[0040] Another technical solution adopted in the embodiment of the present application is: a point cloud registration device, comprising:

[0041] Hand-eye calibration module: used to perform hand-eye calibration on the robotic arm and 3D camera using a 3D camera and a calibration plate to obtain a pose matrix between the 3D camera and the end of the robotic arm; wherein the 3D camera is fixed to the end of the robotic arm;

[0042] Point cloud acquisition module: used to use the 3D camera carried by the robotic arm to shoot the target object from multiple angles to obtain point cloud data of the target object at different angles; the target object is an object with no surface texture features or a smooth surface;

[0043] A first registration module is configured to convert the point cloud data into a base coordinate system of the robotic arm according to a pose matrix between the 3D camera and the end of the robotic arm, thereby obtaining a coarse registration result of the point cloud data;

[0044] Feature projection module: used to project a set number of feature light spots in the overlapping area of ​​the point cloud at different angles using a multi-laser feature projector based on the coarse registration result of the point cloud data, and detect laser point pairs of the point cloud to be registered from the feature light spots according to set rules;

[0045] The second registration module uses a singular value decomposition algorithm to solve the optimal rotation matrix and translation matrix of the laser point pair, and registers the point cloud to be registered according to the optimal rotation matrix and translation matrix.

[0046] Another technical solution adopted by the embodiment of the present application is: a computer device, the computer device includes a processor and a memory coupled to the processor, wherein:

[0047] The memory stores program instructions for implementing the point cloud registration method;

[0048] The processor is configured to execute the program instructions stored in the memory to control a point cloud registration method.

[0049] Another technical solution adopted in the embodiment of the present application is: a storage medium storing program instructions executable by a processor, wherein the program instructions are used to execute the point cloud registration method.

[0050] Compared to the prior art, the beneficial effects of the embodiments of the present application are as follows: the point cloud registration method, apparatus, computer equipment, and storage medium of the embodiments of the present application propose a point cloud registration method based on a multi-angle three-dimensional scanning point cloud of a robotic arm and a multi-laser feature projector, and a point cloud registration method based on a virtual projection spot. A rough registration is performed according to the calibration parameters of the robotic arm, and then by finding the laser point pairs of the multi-laser feature projector, the singular value decomposition is used to solve the precise rotation matrix and translation matrix, thereby achieving high-precision point cloud registration. The embodiments of the present application can perform multi-angle three-dimensional scanning and precise registration on the surface of objects without surface texture features and smooth surfaces, eliminating the traditional method of attaching registration auxiliary points, greatly improving scanning efficiency, and effectively eliminating errors, thereby improving the quality and speed of point cloud registration. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 is a flow chart of the point cloud registration method according to an embodiment of the present application;

[0052] Figure 2 Schematic diagram of the structure of the registration system based on the robotic arm and the laser feature projector according to an embodiment of the present application;

[0053] Figure 3 This is a schematic structural diagram of a point cloud registration device according to an embodiment of the present application;

[0054] Figure 4 This is a schematic diagram of the computer device structure according to an embodiment of the present application;

[0055] Figure 5 A schematic diagram of the structure of the storage medium of an embodiment of the present application. DETAILED DESCRIPTION

[0056] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0057] The terms "first," "second," and "third" in this application are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first," "second," or "third" may explicitly or implicitly include at least one of such features. In the description of this application, "multiple" means at least two, for example, two, three, etc., unless otherwise specifically defined. All directional indications in the embodiments of this application (such as up, down, left, right, front, back...) are only used to explain the relative positional relationship, movement, etc. between the components under a specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indications also change accordingly. In addition, the terms "including" and "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or computer device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units inherent to these processes, methods, products, or computer devices.

[0058] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0059] Specifically, see Figure 1 , is a flow chart of the point cloud registration method according to an embodiment of the present application. The point cloud registration method according to an embodiment of the present application comprises the following steps:

[0060] S100: Perform hand-eye calibration using a 3D camera and calibration plate fixed at the end of the robotic arm to obtain a pose matrix between the 3D camera and the end of the robotic arm;

[0061] In this step, please also refer to Figure 2, is a schematic diagram of the structure of a registration system based on a robotic arm and a laser feature projector according to an embodiment of the present application. The system includes a robotic arm, a 3D camera fixed to the end of the robotic arm, and a multi-laser feature projector. The robotic arm includes, but is not limited to, a three-axis robotic arm or a five-axis robotic arm, the 3D camera includes, but is not limited to, a structured light 3D camera or a stereoscopic 3D camera, and the multi-laser feature projector can be a point feature projector or a laser feature projector with a special coded pattern processed by a process such as DOE. The multi-laser feature projector projects a feature light spot with a pattern such as concentric circles or crosshairs onto the surface of the target object, which is used to more accurately locate the center point of the virtual feature light spot.

[0062] Based on the above system, for the case where the camera is fixed at the end of the robotic arm and the eye is on the hand, the relative position of the camera and the end of the robotic arm is Fixed, the relative position of the calibration plate and the robotic arm base According to the fixed relative posture, the conversion matrix between the calibration plate and the camera can be used under different postures of two sets of robotic arms. And the relative position of the end of the robot arm to the base Establish the equation:

[0063]

[0064]

[0065]

[0066] In the above formula, Indicates the pose relationship between the calibration object and the base coordinate during the i-th shooting. It represents the pose relationship between the calibration object and the base coordinates at the jth shooting time.

[0067] The hand-eye calibration problem is organized as follows:

[0068] AX=XB (4)

[0069] Then, based on Tsai's calibration algorithm, the equation AX = XB (i.e., the subscript part in formula (3)) is solved to complete the hand-eye calibration of the robotic arm and the 3D camera.

[0070] S110: Use a 3D camera carried by a robotic arm to shoot the target object from multiple angles to obtain point cloud data of the target object at different angles. After each shot, use a multi-laser feature projector to obtain texture images of the target object at different angles.

[0071] In the embodiment of the present application, the target object is an object without surface texture features or with a smooth surface, such as a car shell, etc. The shooting angle of the point cloud data can be set according to the actual application scenario.

[0072] S120: converting the point cloud data at different angles into the base coordinate system of the robotic arm according to the pose matrix between the 3D camera and the end of the robotic arm, and obtaining a coarse registration result of all the point cloud data;

[0073] In this step, when performing rough registration on the point cloud data, the target object needs to be transferred from the world coordinate system to the camera coordinate system, and then the pose matrix between the 3D camera and the end of the robotic arm is used. The point cloud data of the target object {P i}Convert to the tool center coordinate system at the end of the robot arm, and then use the pose relationship between the tool center point at the end of the robot arm and the base coordinate system Point cloud data {P i}Convert to the base coordinate system:

[0074]

[0075] After all point cloud data are converted to the base coordinate system, the coarse registration results of all point cloud data are obtained. It should be noted that since the base coordinate system of the manipulator and the target object are relatively stationary, and the movement of the manipulator has errors, the registration results obtained after the conversion are only coarse registration results of the point cloud data.

[0076] S130: Obtain the center points of the light spots of the multi-laser feature projector according to the texture image, and calculate the 3D coordinates corresponding to each center point of the light spot;

[0077] In this step, the texture image obtained by the multi-laser feature projector is first dedistorted using the camera intrinsic parameters, and then the Hough circle fitting algorithm is used to calculate the center point of the light spot of each laser feature projector. The 3D coordinates of the center point of the light spot are obtained through the projection transformation relationship between the point cloud data and the texture image in the 3D camera.

[0078] S140: Based on the coarse registration result of the point cloud data, a multi-laser feature projector is used to project a set number of feature spots in the overlapping area of ​​the point cloud at different angles, and laser point pairs of the point cloud to be registered are detected from the feature spots according to the set rules;

[0079] In this step, the laser point pair acquisition method is specifically as follows: based on the coarse registration results of the point cloud data, a set number of characteristic light spots are projected by multiple laser feature projectors at different angles in the overlapping area of ​​the point cloud. For each characteristic light spot, a sphere is constructed with the accuracy of the robotic arm as the radius, and it is determined whether other characteristic light spots fall within the sphere. If so, the two characteristic light spots within the same sphere are considered to be corresponding points, thereby obtaining laser point pairs between the overlapping areas of the point cloud (i.e., the point cloud to be registered). Among them, the embodiment of the present application sets the number of characteristic light spots projected to be 3 or more, and the specific setting can be based on the actual application scenario.

[0080] S150: Based on the center point of the light spot and its 3D coordinates, the singular value decomposition (SVD) algorithm is used to solve the optimal rotation matrix and translation matrix of the laser point pair. The point cloud data is aligned according to the optimal rotation matrix and translation matrix to obtain a high-precision point cloud registration result.

[0081] In this step, the singular value decomposition algorithm specifically involves converting the coordinate matrices of the two laser point pairs into two orthogonal matrices using the SVD decomposition algorithm based on the spot center point and its 3D coordinates. The rotation matrix is ​​then multiplied to obtain the product of the two orthogonal matrices. The translation matrix is ​​then calculated by calculating the mean difference between the two laser point pairs. Using the rotation and translation matrices, the point cloud to be registered is transformed into the coordinate system of the reference point cloud, achieving precise registration of the point cloud to be registered.

[0082] Based on the above, the point cloud registration method of the embodiment of the present application proposes a point cloud registration method based on multi-angle three-dimensional scanning of the robotic arm and multiple laser feature projectors, as well as a point cloud registration method based on a virtual projection spot. A coarse registration is performed according to the calibration parameters of the robotic arm. Then, by finding laser point pairs of the multiple laser feature projectors, singular value decomposition is used to solve the precise rotation matrix and translation matrix, thereby achieving high-precision point cloud registration. The embodiment of the present application can perform multi-angle three-dimensional scanning and precise registration on the surface of objects without surface texture features and smooth surfaces, eliminating the traditional method of attaching registration auxiliary points, greatly improving scanning efficiency, and effectively eliminating errors, improving the quality and speed of point cloud registration.

[0083] See also Figure 3 , is a schematic diagram of the structure of a point cloud registration device according to an embodiment of the present application. The point cloud registration device 40 according to an embodiment of the present application comprises:

[0084] The hand-eye calibration module 41 is used to perform hand-eye calibration on the robotic arm and the 3D camera using a 3D camera and a calibration plate to obtain a pose matrix between the 3D camera and the end of the robotic arm; wherein the 3D camera is fixed to the end of the robotic arm;

[0085] Point cloud acquisition module 42: used to shoot the target object at multiple angles using the 3D camera carried by the robotic arm to obtain point cloud data of the target object at different angles; the target object is an object with no surface texture features or a smooth surface;

[0086] A first registration module 43 is configured to convert the point cloud data into a base coordinate system of the robotic arm according to a pose matrix between the 3D camera and the end of the robotic arm, and obtain a coarse registration result of the point cloud data;

[0087] Feature projection module 44: used to project a set number of feature light spots in the overlapping area of ​​the point cloud at different angles using a multi-laser feature projector based on the coarse registration result of the point cloud data, and detect laser point pairs of the point cloud to be registered from the feature light spots according to set rules;

[0088] The second registration module 45 uses a singular value decomposition algorithm to solve the optimal rotation matrix and translation matrix of the laser point pair, and registers the point cloud to be registered according to the optimal rotation matrix and translation matrix.

[0089] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of this application. Their specific functions and technical effects can be found in the method embodiment section and will not be repeated here.

[0090] The device provided in the embodiment of the present application can be applied in the aforementioned method embodiment. For details, please refer to the description of the aforementioned method embodiment, which will not be repeated here.

[0091] See also Figure 4 , is a schematic diagram of the computer device structure of an embodiment of the present application. The computer device 50 includes:

[0092] A memory 51 storing executable program instructions;

[0093] a processor 52 connected to the memory 51;

[0094] The processor 52 is used to call the executable program instructions stored in the memory 51 and perform the following steps: using a 3D camera and a calibration plate to perform hand-eye calibration on the robotic arm and the 3D camera to obtain a pose matrix between the 3D camera and the end of the robotic arm; wherein the 3D camera is fixed to the end of the robotic arm; the target object is photographed from multiple angles by the robotic arm carrying the 3D camera to obtain point cloud data of the target object at different angles; the target object is an object without surface texture features or with a smooth surface; according to the pose matrix between the 3D camera and the end of the robotic arm, the point cloud data is converted to the base coordinate system of the robotic arm to obtain a coarse alignment result of the point cloud data; based on the coarse alignment result of the point cloud data, a set number of feature spots are projected by a multi-laser feature projector in the overlapping area of ​​the point cloud at different angles, and the laser point pairs of the point cloud to be aligned are detected from the feature spots according to the set rules; the optimal rotation matrix and translation matrix of the laser point pair are solved by the singular value decomposition algorithm, and the point cloud to be aligned is aligned according to the optimal rotation matrix and translation matrix.

[0095] The processor 52 may also be referred to as a CPU (Central Processing Unit). The processor 52 may be an integrated circuit chip having signal processing capabilities. The processor 52 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component. The general-purpose processor may be a microprocessor or any conventional processor.

[0096] See also Figure 5 , is a structural diagram of the storage medium of an embodiment of the present application. The storage medium of the embodiment of the present application stores program instructions 61 capable of implementing the following steps: using a 3D camera and a calibration plate to perform hand-eye calibration on a robotic arm and a 3D camera to obtain a pose matrix between the 3D camera and the end of the robotic arm; wherein the 3D camera is fixed to the end of the robotic arm; the robotic arm carries the 3D camera to shoot a target object from multiple angles to obtain point cloud data of the target object at different angles; the target object is an object without surface texture features or with a smooth surface; according to the pose matrix between the 3D camera and the end of the robotic arm, the point cloud data is converted to the base coordinate system of the robotic arm to obtain a coarse registration result of the point cloud data; based on the coarse registration result of the point cloud data, a set number of feature spots are projected at the overlapping area of ​​the point cloud at different angles using a multi-laser feature projector, and laser point pairs of the point cloud to be registered are detected from the feature spots according to set rules; the optimal rotation matrix and translation matrix of the laser point pairs are solved using a singular value decomposition algorithm, and the point cloud to be registered is registered according to the optimal rotation matrix and translation matrix. Among them, the program instructions 61 can be stored in the above-mentioned storage medium in the form of a software product, including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network computer device, etc.) or a processor to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: various media that can store program instructions, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or a terminal computer device such as a computer, a server, a mobile phone, or a tablet. Among them, the server can be an independent server, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0097] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0098] In addition, each functional unit in each embodiment of the present application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit. The above is only an implementation method of the present application and does not limit the patent scope of the present application. Any equivalent structure or equivalent process conversion made using the contents of the description and drawings of this application, or directly or indirectly used in other related technical fields, is also included in the patent protection scope of this application.

Claims

1. A point cloud registration method, characterized in that: include: Performing hand-eye calibration on the robotic arm and the 3D camera using a 3D camera and a calibration plate to obtain a pose matrix between the 3D camera and the end of the robotic arm; wherein the 3D camera is fixed to the end of the robotic arm; The target object is photographed at multiple angles by the 3D camera carried by the robotic arm to obtain point cloud data of the target object at different angles; the target object is an object without surface texture features or with a smooth surface; Converting the point cloud data into a base coordinate system of the robotic arm according to a pose matrix between the 3D camera and the end of the robotic arm to obtain a coarse registration result of the point cloud data; Based on the coarse registration result of the point cloud data, a multi-laser feature projector is used to project a set number of feature spots in the overlapping area of ​​the point cloud at different angles, and laser point pairs of the point cloud to be registered are detected from the feature spots according to set rules; The optimal rotation matrix and translation matrix of the laser point pair are solved by using a singular value decomposition algorithm, and the point cloud to be registered is registered according to the optimal rotation matrix and translation matrix.

2. The point cloud registration method according to claim 1, characterized in that: The hand-eye calibration of the robotic arm and the 3D camera using the 3D camera and the calibration plate is specifically as follows: Find the pose of the 3D camera and the center point of the tool at the end of the robotic arm For the case where the camera is fixed at the end of the robotic arm and the eye is on the hand, the relative position of the camera and the end of the robotic arm is Fixed, the relative position of the calibration plate and the robotic arm base Also fixed; according to the fixed relative posture, the conversion matrix between the calibration plate and the camera can be used under different postures of two sets of robotic arms And the relative position of the end of the robot arm to the base Establish the equation: in, Indicates the pose relationship between the calibration object and the base coordinate during the i-th shooting. Indicates the pose relationship between the calibration object and the base coordinate during the j-th shooting; The hand-eye calibration problem is organized as follows: AX=XB Then, based on Tsai's calibration algorithm, the equation AX=XB is solved to complete the hand-eye calibration of the robotic arm and 3D camera.

3. The point cloud registration method according to claim 2, characterized in that: The target object is photographed from multiple angles by the robotic arm carrying a 3D camera, and then the method further includes: Acquiring texture images of a target object at different angles using the multi-laser feature projector; The center points of the light spots of the multi-laser feature projector are obtained according to the texture image, and the 3D coordinates corresponding to each center point of the light spot are calculated.

4. The point cloud registration method according to claim 3, characterized in that: The method of obtaining the center points of the light spots of the multi-laser feature projector according to the texture image and calculating the 3D coordinates corresponding to each center point of the light spot is as follows: Dedistorting the texture image using camera intrinsic parameters, and calculating the center point of the light spot of the multi-laser feature projector using a Hough circle fitting algorithm; The 3D coordinates of the center point of the light spot are obtained through the projection conversion relationship between the point cloud data and the texture image in the 3D camera.

5. The point cloud registration method according to claim 4, characterized in that: The point cloud data is converted to the base coordinate system of the robotic arm according to the pose matrix between the 3D camera and the end of the robotic arm to obtain a rough registration result of the point cloud data, specifically: Convert the target object from the world coordinate system to the camera coordinate system; The point cloud data of the target object {P i }Convert to the tool center point coordinate system at the end of the robot arm; Using the pose relationship between the tool center point at the end of the manipulator and the base coordinate system The point cloud data {P i }Convert to the base coordinate system:

6. The point cloud registration method according to claim 5, characterized in that: The method of using a multi-laser feature projector to project a set number of feature light spots in the overlapping area of ​​the point cloud at different angles, and detecting laser point pairs of the point cloud to be registered from the feature light spots according to the set rules, is specifically as follows: For each characteristic light spot, a sphere is constructed with the accuracy of the robotic arm as the radius, and it is determined whether other characteristic light spots fall within the sphere. If so, the two characteristic light spots within the same sphere are considered to be corresponding points, and the laser point pairs of the point cloud to be aligned are obtained.

7. The point cloud registration method according to claim 6, characterized in that: The singular value decomposition algorithm is used to solve the optimal rotation matrix and translation matrix of the laser point pair, and the point cloud to be registered is registered according to the optimal rotation matrix and translation matrix, specifically: Based on the center point of the light spot and its 3D coordinates, the coordinate matrix of the laser point pair is converted into an orthogonal matrix using a singular value decomposition algorithm, and the product of the orthogonal matrices is calculated to obtain a rotation matrix; Calculating the mean difference of the laser point pairs to obtain a translation matrix; The point cloud to be registered is converted into the coordinate system of the reference point cloud by using the rotation matrix and the translation matrix, thereby completing the precise registration of the point cloud to be registered.

8. A point cloud registration device, characterized in that: include: Hand-eye calibration module: used to perform hand-eye calibration on the robotic arm and 3D camera using a 3D camera and a calibration plate to obtain a pose matrix between the 3D camera and the end of the robotic arm; wherein the 3D camera is fixed to the end of the robotic arm; Point cloud acquisition module: used to use the 3D camera carried by the robotic arm to shoot the target object from multiple angles to obtain point cloud data of the target object at different angles; the target object is an object with no surface texture features or a smooth surface; A first registration module is configured to convert the point cloud data into a base coordinate system of the robotic arm according to a pose matrix between the 3D camera and the end of the robotic arm, thereby obtaining a coarse registration result of the point cloud data; Feature projection module: used to project a set number of feature light spots in the overlapping area of ​​the point cloud at different angles using a multi-laser feature projector based on the coarse registration result of the point cloud data, and detect laser point pairs of the point cloud to be registered from the feature light spots according to set rules; The second registration module uses a singular value decomposition algorithm to solve the optimal rotation matrix and translation matrix of the laser point pair, and registers the point cloud to be registered according to the optimal rotation matrix and translation matrix.

9. A computer device, characterized in that: The computer device includes a processor and a memory coupled to the processor, wherein: The memory stores program instructions for implementing the point cloud registration method according to any one of claims 1 to 7; The processor is configured to execute the program instructions stored in the memory to control a point cloud registration method.

10. A storage medium, characterized in that: Program instructions executable by a processor are stored, and the program instructions are used to execute the point cloud registration method according to any one of claims 1 to 7.

Images