Multi-view three-dimensional point cloud imaging method and device for ceramic sanitary ware

By installing a three-dimensional camera on a ceramic sanitary ware and using a rotating platform, combining robot kinematics and iterative nearest point algorithms, the problem that a single-view angle cannot perceive complete information in three-dimensional point cloud imaging of ceramic sanitary ware is solved, and efficient multi-view point cloud is achieved Imaging and precise registration simplifies data processing and improves production line efficiency and quality.

CN118941601AActive Publication Date: 2024-11-12GUANGDONG UNIV OF TECH

Patent Information

Application Number
CN202410999589.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-24
Publication Date
2024-11-12
Estimated Expiration
2044-07-24

AI Technical Summary

Technical Problem

In the three-dimensional point cloud imaging process of ceramic sanitary ware, complete surface information cannot be sensed by a single perspective, and the data processing of multi-view imaging schemes is complex, so the registration algorithm is prone to falling into local optimality, making it difficult to obtain an accurate complete point cloud.

Method used

Using multi-view three-dimensional point cloud imaging method, by installing a three-dimensional camera at the end of the robot and combining a rotating platform, point cloud information at multiple perspectives is obtained, and rough registration and precise registration is performed, using robot positive kinematics and iterative close-point algorithm. Perform point cloud registration to simplify data acquisition and calculation process.

Benefits of technology

It realizes rapid and automated reconstruction of a complete and accurate point cloud model, reduces the demand for imaging equipment, simplifies the system hardware structure, improves work efficiency and quality, and provides accurate spatial position information for the ceramic sanitary ware production line.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118941601A_ABST
    Figure CN118941601A_ABST
Patent Text Reader

Abstract

The invention relates to a ceramic sanitary ware-oriented multi-view three-dimensional point cloud imaging method and device, and the method comprises the following steps: keeping a rotating platform still, moving a robot, and obtaining point cloud information and a transformation relation at each view angle recorded by the surface information of a ceramic sanitary ware biscuit sensed by a three-dimensional camera at a plurality of view angles; the robot keeps still, the rotating platform is rotated, so that the three-dimensional camera senses the surface information of the ceramic sanitary ware biscuit at multiple visual angles, and point cloud information and a transformation relation at each visual angle are obtained; repeating the process until the complete surface information of the ceramic sanitary ware is sensed; and the misaligned point clouds are registered through a machine vision processing system, and redundant point clouds are removed, so that a complete point cloud model can be reconstructed. According to the method, accurate spatial position information can be provided for subsequent operation of the ceramic sanitary ware production line, and a good foundation is laid for segmentation of a grinding area and planning of a grinding path.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the relevant technical field, and in particular to a multi-view three-dimensional point cloud imaging method and device for ceramic sanitary ware. Background Art

[0002] With the rapid development of various automation technologies, robots are gradually replacing manual labor in production lines such as ceramic sanitary ware grinding and spraying. Among them, three-dimensional imaging equipment can provide information such as identification and positioning for robot processing, and is an indispensable component of the production line. Since ceramic sanitary ware is an unstructured object composed of multiple complex free-form surfaces, this special structural composition will cause optical occlusion between different working surfaces, and the viewing angle of the imaging equipment is limited. Therefore, in the process of obtaining the three-dimensional point cloud model of ceramic sanitary ware, it is impossible to reconstruct the complete surface spatial information of ceramic sanitary ware through a single perspective. Therefore, it is necessary to perceive the surface of ceramic sanitary ware from multiple perspectives to obtain complete spatial information, but the spatial information of the surface point cloud obtained by scanning ceramic sanitary ware from multiple perspectives by a three-dimensional camera is misplaced, and accurate registration is required to obtain a complete point cloud. However, the registration algorithm is prone to fall into local optimality, and the registration result is prone to large deviations, making it difficult to obtain an accurate and complete point cloud. At present, most multi-view imaging solutions use array cameras to collect data, and their data processing process is relatively cumbersome, and the computational complexity of fusion data is high. Therefore, it is necessary to develop a simple and efficient multi-view imaging method to simplify the data acquisition method and the calculation process. Summary of the invention

[0003] The purpose of the present invention is to solve at least one of the deficiencies of the prior art and to provide a multi-view three-dimensional point cloud imaging method and device for ceramic sanitary ware.

[0004] In order to achieve the above object, the present invention adopts the following technical solutions:

[0005] Specifically, a multi-view 3D point cloud imaging method for ceramic sanitary ware is proposed, including the following:

[0006] With the robot base as the base coordinate system, a three-dimensional camera is installed on the end flange of the robot, and the ceramic sanitary ware is placed on the rotating platform;

[0007] Step 110, the rotating platform remains stationary, and the moving robot obtains the point cloud information and transformation relationship at each viewing angle recorded by the three-dimensional camera sensing the surface information of the ceramic sanitary ware blank at multiple viewing angles;

[0008] Step 120: The robot remains stationary and rotates the rotating platform so that the three-dimensional camera senses the surface information of the ceramic sanitary ware blank at multiple viewing angles, and obtains the point cloud information and transformation relationship at each viewing angle;

[0009] Step 130, repeating steps 110 to 120 until the complete surface information of the ceramic sanitary ware is sensed;

[0010] Step 140, performing coarse registration of multi-view point clouds on the complete surface information of the ceramic sanitary ware to obtain a coarse registration point cloud;

[0011] Step 150, performing multi-view point cloud fine registration on the coarse registration point cloud to obtain a fine registration point cloud;

[0012] Step 160: Perform point cloud processing on the precisely registered point cloud to obtain a complete model.

[0013] Further, specifically, the complete surface information of the ceramic sanitary ware is subjected to multi-view point cloud coarse registration to obtain a coarse registration point cloud, including:

[0014] A fixed viewing angle is set, and an object is placed on a rotating platform. The imaging device can obtain a point on the surface of the object in the camera coordinate system {C}. If the rotation angle is β i , the current point is Represents a column vector. When the rotating platform rotates, the point The position is always distributed on a space circle. The center O of the current space circle is used as the rotation center. The unit normal vector C of the plane where the space circle is located is v As the rotation axis direction vector, let the plane equation of the current space circle under {C} be Ax+By+Cz+D=0, then:

[0015]

[0016] Substituting a series of observation points into formula (5) we can obtain v = [AB C] T, and after normalizing v we can obtain C v , the chord on the space circle is perpendicularly bisected by the perpendicular line from the center of the circle to the chord, then:

[0017]

[0018] The rotation angle β of a series of observation points i Substituting into formulas (5) and (6) we can find the rotation center O; let point P under {B} B Rotate β around the rotation center O and the rotation axis i Get a new point, which can be transformed into a rotation of β around the origin and the rotation axis under {W} i , this problem can be described by the Rodriguez rotation formula, which can be obtained

[0019]

[0020] T(β) in The good initial pose of the point cloud under multiple perspectives can be obtained, and the rough alignment of the point cloud model under any posture of the robot and any angle of the rotating platform can be achieved.

[0021] Further, specifically, the conversion process of formula (2) includes:

[0022] The base coordinate system of the robot is {B}, the end coordinate system of the robot is {E}, the modeling coordinate system of the imaging device is {C}, the zero point coordinate system of the rotating platform is {R}, and the world coordinate system is {W}. B , can be observed from different perspectives, then:

[0023]

[0024] In formula (1), P C is the point P in the modeling coordinate system of the imaging device at the current viewing angle B The coordinates of is the homogeneous transformation matrix from the robot base coordinate system to the end flange, is the homogeneous transformation matrix from coordinate system {E} to {C}; when the robot posture remains unchanged and the rotating platform rotates β, to correct the imaging under different rotation angles, P B Rotate the rotating platform in the opposite direction of the rotation axis by β to make it located in the zero-point coordinate system {R}. When the robot is in any posture and the rotating platform is at any angle, if the world coordinate system {W} is assumed to coincide with the zero-point coordinate system {R} of the rotating platform, any scanning point P can be obtained by combining formula (1): C Representation in the world coordinate system:

[0025]

[0026] In formula (2), T(β) represents the homogeneous transformation matrix of the rotation of the rotating platform in the opposite direction of β. Through the forward kinematics analysis of the robot, the DH parameters are established according to the model parameters of the robot, and the transformation between its joints is calculated to obtain the transformation matrix from the robot base coordinate system to the end flange.

[0027] Further, specifically, the present invention uses the eye-on-hand method to install the imaging device, and the transformation relationship from the robot end coordinate system {E} to the imaging device coordinate system {C} is called the hand-eye matrix When the rotating platform does not rotate, only the robot moves, and the point clouds imaged under the viewing angles a and b are unified to the base coordinate system {B}. If there is an overlap between the two clusters of point clouds, the overlap satisfies the following formula:

[0028]

[0029] Since the imaging device is fixed to the end flange of the robot, and is the same transformation matrix, denoted as Will Indicates {C a} to {C b}, then formula (3) can be expressed as:

[0030]

[0031] The transformation matrix from coordinate system {B} to coordinate system {E} is obtained through the robot forward kinematics. In formula (4), and The robot joint angles at the corresponding viewing angles can be obtained by substituting them into the robot transformation matrix. The hand-eye matrix can be obtained Solving for rigid body transformations The problem is transformed into a matching problem of point clouds in different coordinate systems; the iterative closest point algorithm ICP is used to calculate the transformation matrix of the two clusters of point clouds to obtain Substituting it into formula (4) we can get the hand-eye matrix

[0032] Further, specifically, performing multi-view point cloud fine registration on the coarse registration point cloud to obtain a fine registration point cloud includes:

[0033] For each point p in the source point cloud P i , find the nearest neighbor point q in its target point cloud Q i , calculate the square of the Euclidean distance as follows:

[0034]

[0035] Assume the number of overlapping point clouds is N overlap , the number of source point clouds is N p ,Will Sort in ascending order and take the first N overlap Calculate the mean square error (MSE) of each point in the iteration process; continuously iterate and update the transformation matrix until the MSE is less than a certain threshold and the algorithm converges, then the accurate rotation matrix R and translation vector t can be obtained;

[0036]

[0037] The multi-view point cloud precise registration process includes the following:

[0038] The point clouds of different viewpoints under the same robot posture are registered. Taking the rotation angle β = 0 as the reference, the point clouds under other rotation angles are registered to this reference. i∈(0,π], align forward to the reference view in sequence; if the rotation angle β i ∈(π,2π], align backward to the reference view in sequence;

[0039] The point clouds under all the angles β = 0 under different robot postures are registered. The point cloud with angle β = 0 in one robot state is selected as the reference, and the point clouds under different robot postures are registered to the reference; and the rotation matrix R of each registration is recorded. i and the translation vector t i ;

[0040] The point cloud under the same robot state except for the view of β = 0 is executed with the corresponding rotation matrix R i and the translation vector t i , the accurate registration of the global multi-view point cloud can be obtained.

[0041] The present invention also proposes a multi-view three-dimensional point cloud imaging device for ceramic sanitary ware, comprising the following:

[0042] With the robot base as the base coordinate system, a three-dimensional camera is installed on the end flange of the robot, and the ceramic sanitary ware is placed on a rotating platform at a preset position;

[0043] The first data acquisition module is used to obtain the point cloud information and transformation relationship at each viewing angle recorded by the three-dimensional camera sensing the surface information of the ceramic sanitary ware blank at multiple viewing angles when the rotating platform remains stationary and the robot moves;

[0044] The second data acquisition module is used for the robot to keep still and rotate the rotating platform so that the three-dimensional camera can sense the surface information of the ceramic sanitary ware blank at multiple viewing angles and obtain the point cloud information and transformation relationship at each viewing angle;

[0045] Repeating the first data acquisition module and the second data acquisition module until the complete surface information of the ceramic sanitary ware is sensed;

[0046] A coarse registration module is used to perform coarse registration of multi-view point clouds on the complete surface information of ceramic sanitary ware to obtain coarse registration point clouds;

[0047] A fine registration module, used for performing multi-view point cloud fine registration on the coarse registration point cloud to obtain a fine registration point cloud;

[0048] The complete model acquisition module is used to perform point cloud processing on the precisely registered point cloud to acquire a complete model.

[0049] The beneficial effects of the present invention are:

[0050] The present invention proposes a multi-view 3D point cloud imaging method and device for ceramic sanitary ware. By fixing a 3D camera on the end flange of a robot and cooperating with the rotation of a rotating platform, the surface information of the ceramic sanitary ware can be fully sensed, solving the problem that the complete surface information of the ceramic sanitary ware cannot be sensed under a single viewing angle. In addition, the present invention can realize multi-view point cloud imaging using only a single imaging device, reducing the demand for imaging devices and simplifying the system hardware structure; and point cloud data collection is performed through automated equipment such as robots and rotating platforms, avoiding complex manual operations, simplifying the process flow, and improving work efficiency and work quality.

[0051] Through the method of this patent, a complete and accurate point cloud model can be reconstructed quickly and automatically, providing accurate spatial position information for subsequent operations of the ceramic sanitary ware production line, and laying a good foundation for the segmentation of the grinding area and the planning of the grinding path. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The above and other features of the present disclosure will become more obvious by describing in detail the embodiments shown in the accompanying drawings. The same reference numerals in the accompanying drawings of the present disclosure represent the same or similar elements. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure. For those skilled in the art, other accompanying drawings can be obtained based on these accompanying drawings without creative work. In the accompanying drawings:

[0053] Figure 1 It is a flowchart of a multi-view three-dimensional point cloud imaging method for ceramic sanitary ware according to the present invention;

[0054] Figure 2 (a) shows a schematic diagram of imaging a ceramic sanitary ware blank from a single viewing angle in one embodiment, and (b) shows a schematic diagram of imaging a ceramic sanitary ware blank from multiple viewing angles in one embodiment;

[0055] Figure 3 The figure shows the working principle diagram of the multi-view 3D point cloud imaging device;

[0056] Figure 4 The figure shows the flow chart of solving coordinate transformation relationship;

[0057] Figure 5 Shown is a schematic diagram of the transformation relationship between two different three-dimensional imaging coordinate systems;

[0058] Figure 6 The figure shows a schematic diagram of the rotation of an object on a rotating platform;

[0059] Figure 7 Shown is a schematic diagram of a multi-view point cloud registration method;

[0060] Figure 8(a) is a point cloud of a ceramic sanitary ware blank in one embodiment, and (b) is a point cloud of a ceramic sanitary ware blank in another embodiment. DETAILED DESCRIPTION

[0061] The following will be combined with the embodiments and drawings to clearly and completely describe the concept, specific structure and technical effects of the present invention, so as to fully understand the purpose, scheme and effect of the present invention. It should be noted that the embodiments in this application and the features in the embodiments can be combined with each other without conflict. The same reference numerals used throughout the drawings indicate the same or similar parts.

[0062] Example 1, reference Figure 1 The present invention proposes a multi-view three-dimensional point cloud imaging method for ceramic sanitary ware, comprising the following:

[0063] With the robot base as the base coordinate system, a three-dimensional camera is installed on the end flange of the robot, and the ceramic sanitary ware is placed on the rotating platform;

[0064] Step 110, the rotating platform remains stationary, and the robot moves to obtain the point cloud information and transformation relationship at each viewing angle recorded by the three-dimensional camera sensing the surface information of the ceramic sanitary ware blank at multiple viewing angles;

[0065] Step 120: The robot remains stationary and rotates the rotating platform so that the three-dimensional camera senses the surface information of the ceramic sanitary ware blank at multiple viewing angles, and obtains the point cloud information and transformation relationship at each viewing angle;

[0066] Step 130, repeating steps 110 to 120 until the complete surface information of the ceramic sanitary ware is sensed;

[0067] Step 140, performing coarse registration of multi-view point clouds on the complete surface information of the ceramic sanitary ware to obtain a coarse registration point cloud;

[0068] Step 150, performing multi-view point cloud fine registration on the coarse registration point cloud to obtain a fine registration point cloud;

[0069] Step 160: Perform point cloud processing on the precisely registered point cloud to obtain a complete model.

[0070] In this embodiment 1, by fixing the three-dimensional camera on the end flange of the robot and cooperating with the rotation of the rotating platform, the surface information of the ceramic sanitary ware can be fully sensed, solving the problem that the complete surface information of the ceramic sanitary ware cannot be sensed from a single perspective. In addition, the present invention can realize multi-perspective point cloud imaging using only a single imaging device, reducing the demand for imaging devices and simplifying the system hardware structure; and point cloud data collection is performed through automated equipment such as robots and rotating platforms, avoiding complex manual operations, simplifying the process flow, and improving work efficiency and work quality.

[0071] Through the method of this patent, a complete and accurate point cloud model can be reconstructed quickly and automatically, providing accurate spatial position information for subsequent operations of the ceramic sanitary ware production line, and laying a good foundation for the segmentation of the grinding area and the planning of the grinding path.

[0072] The implementation process of the present invention mainly includes the following:

[0073] 1. Analysis and construction of 3D imaging system

[0074] Ceramic sanitary ware is an unstructured object composed of multiple complex curved surfaces. Due to its own structural characteristics, it has optical self-occlusion phenomenon. And it is limited by the field of view of the imaging device. It is impossible to fully perceive all information in a single perspective. Figure 2 If you want to perceive the complete surface information of the ceramic sanitary ware blank, you can Figure 2 As shown in (b), complete information is obtained through perception from multiple perspectives, and then the 3D point cloud model is reconstructed based on the relative spatial information of each perspective.

[0075] The present invention fixes the imaging device on the end flange of the robot and cooperates with the rotating platform so that the imaging device can perceive the complete model information, such as Figure 3 As shown in the figure. The base coordinate system of the robot is {B}, the end coordinate system of the robot is {E}, the modeling coordinate system of the imaging device is {C}, the zero point coordinate system of the rotating platform is {R}, and the world coordinate system is {W}. Based on a point P in the base coordinate system of the robot B , can be observed from different perspectives, then:

[0076]

[0077] In formula (1), P C is the point P in the modeling coordinate system of the imaging device at the current viewing angle B The coordinates of is the homogeneous transformation matrix from the robot base coordinate system to the end flange, is the homogeneous transformation matrix from coordinate system {E} to {C}. When the robot posture remains unchanged and the rotating platform rotates β, to correct the imaging under different rotation angles, P B Rotate β in the opposite direction of the rotation axis of the rotating platform to make it located in the zero-point coordinate system {R}. When the robot is in any posture and the rotating platform is at any angle, if the world coordinate system {W} is assumed to coincide with the zero-point coordinate system {R} of the rotating platform, the arbitrary scanning point P can be obtained by combining formula (1): C Representation in the world coordinate system:

[0078]

[0079] In formula (2), T(β) represents the homogeneous transformation matrix of the rotation of the rotating platform in the opposite direction of β. Through the above formula, the point cloud obtained by any scan can be transformed into the world coordinate system {W} to achieve rough registration of the point cloud and obtain the three-dimensional point cloud model of the ceramic sanitary ware blank. For the spatial transformation relationship matrix T(β) in formulas (1) and (2), and Will follow Figure 4 The flowchart shown is used for calculation respectively.

[0080] 2. Robot forward kinematics

[0081] Through the robot forward kinematics analysis, the DH parameters are established according to the robot model parameters, and the transformation between its joints is calculated to obtain the transformation matrix from the robot base coordinate system to the end flange.

[0082] 3. Hand-eye calibration method based on point cloud registration

[0083] like Figure 3 As shown, the present invention uses the eye-on-hand method to install the imaging device, and the transformation relationship from the robot end coordinate system {E} to the imaging device coordinate system {C} is called the hand-eye matrix When the rotating platform does not rotate, only the robot moves, and the point clouds imaged under the viewing angles a and b are unified to the base coordinate system {B}. If there is an overlap between the two clusters of point clouds, the overlap satisfies the following formula:

[0084]

[0085] like Figure 3 As shown in the figure, the imaging device is fixed on the end flange of the robot. and is the same transformation matrix, denoted as Will Indicates {C a} to {C b}, then formula (3) can be expressed as:

[0086]

[0087] The transformation matrix from coordinate system {B} to coordinate system {E} is obtained through the robot forward kinematics. In formula (4), and The robot joint angles at the corresponding viewing angles can be obtained by substituting them into the robot transformation matrix. The hand-eye matrix can be obtained like Figure 5As shown, the rigid body transformation can be solved The problem is transformed into a matching problem of point clouds in different coordinate systems. The present invention uses an iterative closest point algorithm (ICP) to calculate the transformation matrix of two clusters of point clouds, thereby obtaining Substituting it into formula (4) we can get the hand-eye matrix

[0088] 4. Rotating platform calibration method

[0089] like Figure 6 As shown in the figure, a fixed viewing angle is placed on the rotating platform, and a point on the surface of the object in the camera coordinate system {C} can be obtained through the imaging device. If the rotation angle is β i , the current point is represents a column vector. When the rotating platform rotates, the point The position is always distributed on a space circle. The center O of the current space circle is used as the rotation center. The unit normal vector C of the plane where the space circle is located is v As the rotation axis direction vector. Assume that the plane equation of the current space circle under {C} is Ax+By+Cz+D=0, then:

[0090]

[0091] Substituting a series of observation points into formula (5) we can obtain v = [ABC]T, and after normalizing v we can obtain C v .like Figure 6 As shown, the chord on the space circle is perpendicularly bisected by the vertical line from the center of the circle to the chord, then:

[0092]

[0093] The rotation angle β of a series of observation points i Substituting into formulas (5) and (6) we can find the rotation center O. Suppose that at a point P under {B} B Rotate β around the rotation center O and the rotation axis i Get a new point, which can be transformed into a rotation of β around the origin and the rotation axis under {W} i , this problem can be described by the Rodriguez rotation formula, and T(β) in formula (2) can be obtained. The good initial pose of the point cloud under multiple perspectives can be obtained, and the rough alignment of the point cloud model under any posture of the robot and any angle of the rotating platform can be achieved.

[0094] 5. Multi-view point cloud precise registration method

[0095] The above steps have achieved the rough registration of multi-view point clouds, but since the overlapping areas of point clouds under different viewpoints are small, the point cloud clusters in the non-overlapping areas will have a greater impact on the accuracy of the registration results. The present invention uses the TrICP algorithm to solve this problem. For each point p in the source point cloud P i , find the nearest neighbor point q in its target point cloud Q i , calculate the square of the Euclidean distance as follows:

[0096]

[0097] Assume the number of overlapping point clouds is N overlap , the number of source point clouds is N p ,Will Sort in ascending order and take the first N overlap The mean square error (MSE) of each point in the iteration process is calculated. The transformation matrix is ​​updated continuously until the MSE is less than a certain threshold and the algorithm converges. The accurate rotation matrix R and translation vector t can be obtained.

[0098]

[0099] The precise registration process of multi-view point clouds is as follows: Figure 7 As shown, it mainly includes three steps:

[0100] The point clouds of different viewpoints under the same robot posture are registered. Taking the rotation angle β = 0 as the reference, the point clouds under other rotation angles are registered to this reference. i ∈(0,π], align forward to the reference view in sequence; if the rotation angle β i ∈(π,2π], and align backward to the reference view in sequence.

[0101] Register the point clouds under all angles β = 0 in different robot postures. Select a point cloud with angle β = 0 in a robot state as the reference, and register the point clouds under different robot postures to the reference. And record the rotation matrix R of each registration i and the translation vector t i .

[0102] The point cloud under the same robot state except for the view of β = 0 is executed with the corresponding rotation matrix R i and the translation vector t i , the accurate registration of the global multi-view point cloud can be obtained.

[0103] 6. Experimental results

[0104] The imaging device is fixed at the end of the robot, and cooperates with the rotating platform to sense the complete surface information of the ceramic sanitary ware blank. The point cloud information under various viewing angles is fused using the method of the present invention, and after removing redundant information and other processing, the following image can be reconstructed: Figure 8 The complete ceramic sanitary ware blank point cloud model shown in (a) and (b).

[0105] It should be noted that the capital letter subscript under any English letter refers to the meaning in the coordinate system represented by the capital letter, and and Respectively represent the transformation matrix from the robot base coordinate system to the end flange at the a and b perspectives, θ a and θ b Respectively represent the robot joint angle combination in the a and b perspectives.

[0106] like Figure 5 As shown, {C a} and {C b} respectively represent the imaging coordinate system at viewing angles a and b, and Respectively represent the imaging point at {C a} and {C b}Coordinates in the coordinate system.

[0107] The present invention also proposes a multi-view three-dimensional point cloud imaging device for ceramic sanitary ware, comprising the following:

[0108] With the robot base as the base coordinate system, a three-dimensional camera is installed on the end flange of the robot, and the ceramic sanitary ware is placed on a rotating platform at a preset position;

[0109] The first data acquisition module is used to obtain the point cloud information and transformation relationship at each viewing angle recorded by the three-dimensional camera sensing the surface information of the ceramic sanitary ware blank at multiple viewing angles when the rotating platform remains stationary and the robot moves;

[0110] The second data acquisition module is used for the robot to keep still and rotate the rotating platform so that the three-dimensional camera can sense the surface information of the ceramic sanitary ware blank at multiple viewing angles and obtain the point cloud information and transformation relationship at each viewing angle;

[0111] Repeating the first data acquisition module and the second data acquisition module until the complete surface information of the ceramic sanitary ware is sensed;

[0112] A coarse registration module is used to perform coarse registration of multi-view point clouds on the complete surface information of ceramic sanitary ware to obtain coarse registration point clouds;

[0113] A fine registration module, used for performing multi-view point cloud fine registration on the coarse registration point cloud to obtain a fine registration point cloud;

[0114] The complete model acquisition module is used to perform point cloud processing on the precisely registered point cloud to acquire a complete model.

[0115] The basic components of the present invention include: a three-dimensional camera, a six-axis robot, a rotating platform and a machine vision processing system.

[0116] The 3D camera is used to image the 3D point cloud at the current viewing angle;

[0117] The six-axis robot is responsible for moving the three-dimensional camera fixed to the end of the robot to a suitable imaging position;

[0118] The rotating platform is used to rotate the ceramic sanitary ware blank so that it can be completely sensed by the three-dimensional camera;

[0119] The machine vision processing system processes the raw data from the 3D camera and reconstructs a complete point cloud model.

[0120] When applying,

[0121] (i) Using the robot base as the base coordinate system, install a 3D camera on the robot end flange and place the ceramic sanitary ware on the rotating platform;

[0122] (ii) the rotating platform remains stationary and the robot moves, so that the three-dimensional camera senses the surface information of the ceramic sanitary ware blank from multiple viewing angles, and records the point cloud information and transformation relationship from the current viewing angle;

[0123] (3) The robot remains stationary and rotates the rotating platform so that the 3D camera can perceive the surface information of the ceramic sanitary ware in a larger range and record the point cloud information at different rotation angles and the transformation relationship;

[0124] (iv) Steps (ii) and (iii) are performed alternately until the complete surface information of the ceramic sanitary ware is sensed. The misaligned point clouds are registered by the machine vision processing system, and the redundant point clouds are removed, so that a complete point cloud model can be reconstructed.

[0125] In addition, each functional module in each embodiment of the present invention can be integrated into one processing module, each module can exist physically separately, or two or more modules can be integrated into one module. The above integrated modules can be implemented in the form of hardware or software functional modules.

[0126] If the integrated module is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium can include: any entity or system that can carry the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal and software distribution medium, etc.

[0127] Although the description of the present invention has been quite detailed and specifically describes several described embodiments, it is not intended to be limited to any of these details or embodiments or any particular embodiment, but should be regarded as providing a broad possible interpretation of these claims in view of the prior art by reference to the appended claims, thereby effectively covering the intended scope of the present invention. In addition, the above description of the present invention is based on the embodiments foreseeable by the inventor, and its purpose is to provide a useful description, and those non-substantial changes to the present invention that have not yet been foreseen may still represent equivalent changes to the present invention.

[0128] The above is only a preferred embodiment of the present invention. The present invention is not limited to the above implementation. As long as the technical effect of the present invention is achieved by the same means, it should belong to the protection scope of the present invention. Within the protection scope of the present invention, its technical scheme and / or implementation method can have various modifications and changes.

Claims

1. A multi-view three-dimensional point cloud imaging method for ceramic sanitary ware, characterized in that: Includes the following: With the robot base as the base coordinate system, a three-dimensional camera is installed on the end flange of the robot, and the ceramic sanitary ware is placed on the rotating platform; Step 110, the rotating platform remains stationary, and the moving robot obtains the point cloud information and transformation relationship at each viewing angle recorded by the three-dimensional camera sensing the surface information of the ceramic sanitary ware blank at multiple viewing angles; Step 120: The robot remains stationary and rotates the rotating platform so that the three-dimensional camera senses the surface information of the ceramic sanitary ware blank at multiple viewing angles, and obtains the point cloud information and transformation relationship at each viewing angle; Step 130, repeating steps 110 to 120 until the complete surface information of the ceramic sanitary ware is sensed; Step 140, performing coarse registration of multi-view point clouds on the complete surface information of the ceramic sanitary ware to obtain a coarse registration point cloud; Step 150, performing multi-view point cloud fine registration on the coarse registration point cloud to obtain a fine registration point cloud; Step 160: Perform point cloud processing on the precisely registered point cloud to obtain a complete model.

2. The multi-view three-dimensional point cloud imaging method for ceramic sanitary ware according to claim 1, characterized in that: Specifically, the complete surface information of ceramic sanitary ware is roughly registered by multi-view point cloud to obtain the roughly registered point cloud. include, A fixed viewing angle is set, and an object is placed on a rotating platform. The imaging device can obtain a point on the surface of the object in the camera coordinate system {C}. If the rotation angle is β i , the current point is Represents a column vector. When the rotating platform rotates, the point The position is always distributed on a space circle. The center O of the current space circle is used as the rotation center. The unit normal vector C of the plane where the space circle is located is v As the rotation axis direction vector, let the plane equation of the current space circle under {C} be Ax+By+Cz+D=0, then: Substituting a series of observation points into formula (5) we can obtain v = [ABC]T, and after normalizing v we can obtain C v , the chord on the space circle is perpendicularly bisected by the perpendicular line from the center of the circle to the chord, then: The rotation angle β of a series of observation points i Substituting into formulas (5) and (6), the rotation center O can be obtained; Let a point P under {B} B Rotate β around the rotation center O and the rotation axis i Get a new point, which can be transformed into a rotation of β around the origin and the rotation axis under {W} i , this problem can be described by the Rodriguez rotation formula, which can be obtained T(β) in The good initial pose of the point cloud under multiple perspectives can be obtained, and the rough alignment of the point cloud model under any posture of the robot and any angle of the rotating platform can be achieved.

3. The multi-view 3D point cloud imaging method for ceramic sanitary ware according to claim 2, characterized in that: Specifically, the conversion process of formula (2) includes: The base coordinate system of the robot is {B}, the end coordinate system of the robot is {E}, the modeling coordinate system of the imaging device is {C}, the zero point coordinate system of the rotating platform is {R}, and the world coordinate system is {W}. B , can be observed from different perspectives, then: In formula (1), P C is the point P in the modeling coordinate system of the imaging device at the current viewing angle B The coordinates of is the homogeneous transformation matrix from the robot base coordinate system to the end flange, is the homogeneous transformation matrix from coordinate system {E} to {C}; when the robot posture remains unchanged and the rotating platform rotates β, to correct the imaging under different rotation angles, P B Rotate the rotating platform in the opposite direction of the rotation axis by β to make it located in the zero-point coordinate system {R}. When the robot is in any posture and the rotating platform is at any angle, if the world coordinate system {W} is assumed to coincide with the zero-point coordinate system {R} of the rotating platform, any scanning point P can be obtained by combining formula (1): C Representation in the world coordinate system: In formula (2), T(β) represents the homogeneous transformation matrix of the rotation of the rotating platform in the opposite direction of β. Through the forward kinematics analysis of the robot, the DH parameters are established according to the model parameters of the robot, and the transformation between its joints is calculated to obtain the transformation matrix from the robot base coordinate system to the end flange.

4. The multi-view 3D point cloud imaging method for ceramic sanitary ware according to claim 3, characterized in that: Specifically, the present invention uses the eye-on-hand method to install the imaging device, and the transformation relationship from the robot end coordinate system {E} to the imaging device coordinate system {C} is called the hand-eye matrix When the rotating platform does not rotate, only the robot moves, and the point clouds imaged under the viewing angles a and b are unified to the base coordinate system {B}. If there is an overlap between the two clusters of point clouds, the overlap satisfies the following formula: Since the imaging device is fixed to the end flange of the robot, and is the same transformation matrix, denoted as Will Indicates {C a } to {C b }, then formula (3) can be expressed as: The transformation matrix from coordinate system {B} to coordinate system {E} is obtained through the robot forward kinematics. In formula (4), and The robot joint angles at the corresponding viewing angles can be obtained by substituting them into the robot transformation matrix. The hand-eye matrix can be obtained Solving for rigid body transformations The problem is transformed into a matching problem of point clouds in different coordinate systems; the iterative closest point algorithm ICP is used to calculate the transformation matrix of the two clusters of point clouds to obtain Substituting it into formula (4) we can get the hand-eye matrix 5. The multi-view 3D point cloud imaging method for ceramic sanitary ware according to claim 4, characterized in that: Specifically, the coarsely registered point cloud is subjected to multi-view point cloud fine registration to obtain a finely registered point cloud. include, For each point p in the source point cloud P i , find the nearest neighbor point q in its target point cloud Q i , calculate the square of the Euclidean distance as follows: Assume the number of overlapping point clouds is N overlap , the number of source point clouds is N p ,Will Sort in ascending order and take the first N overlap Calculate the mean square error MSE during the iteration process; Continuously iterate and update the transformation matrix until the MSE is less than a certain threshold and the algorithm converges, and then the accurate rotation matrix R and translation vector t can be obtained; The multi-view point cloud precise registration process includes the following: The point clouds of different viewpoints under the same robot posture are registered. Taking the rotation angle β = 0 as the reference, the point clouds under other rotation angles are registered to this reference. i ∈(0,π], align forward to the reference view in sequence; if the rotation angle β i ∈(π,2π], align backward to the reference view in sequence; The point clouds under all the angles β = 0 under different robot postures are registered. The point cloud with angle β = 0 in one robot state is selected as the reference, and the point clouds under different robot postures are registered to the reference; and the rotation matrix R of each registration is recorded. i and the translation vector t i ; The point cloud under the same robot state except for the view of β = 0 is executed with the corresponding rotation matrix R i and the translation vector t i , the accurate registration of the global multi-view point cloud can be obtained.

6. A multi-view three-dimensional point cloud imaging device for ceramic sanitary ware, characterized in that: Includes the following: With the robot base as the base coordinate system, a three-dimensional camera is installed on the end flange of the robot, and the ceramic sanitary ware is placed on a rotating platform at a preset position; The first data acquisition module is used to obtain the point cloud information and transformation relationship at each viewing angle recorded by the three-dimensional camera sensing the surface information of the ceramic sanitary ware blank at multiple viewing angles when the rotating platform remains stationary and the robot moves; The second data acquisition module is used for the robot to keep still and rotate the rotating platform so that the three-dimensional camera can sense the surface information of the ceramic sanitary ware blank at multiple viewing angles and obtain the point cloud information and transformation relationship at each viewing angle; Repeating the first data acquisition module and the second data acquisition module until the complete surface information of the ceramic sanitary ware is sensed; A coarse registration module is used to perform coarse registration of multi-view point clouds on the complete surface information of ceramic sanitary ware to obtain coarse registration point clouds; A fine registration module, used for performing multi-view point cloud fine registration on the coarse registration point cloud to obtain a fine registration point cloud; The complete model acquisition module is used to perform point cloud processing on the precisely registered point cloud to acquire a complete model.

Citation Information

Patent Citations

  • Method and device for jointly calibrating robot and three-dimensional sensing component

    CN108346165A

  • Method for measuring three-dimensional shape of complex structural member based on robot

    CN115546289A

  • Method for visual inspection and apparatus thereof

    WO2019177539A1

  • Robot hand-eye calibration method and apparatus, readable storage medium, and robot

    WO2022160787A1

Cited By

  • Five-axis rotary workbench workpiece positioning method and device

    CN121330050A