System apparatus and registration method for omnidirectional measurement device, electronic device, and storage medium

Abstract

The present invention belongs to the field of three-dimensional measurement, and particularly relates to a system apparatus and registration method for an omnidirectional measurement device comprising multiple lateral work measurement apparatuses, an electronic device and a storage medium. The system apparatus comprises: a motion module: the motion module comprising a plurality of mobile platforms having one to three degrees of freedom, and each mobile platform being provided with a 6-degree-of-freedom serial robot; a measurement module: the tail end of each serial robot being provided with a three-dimensional measurement device, and a measurement direction being a horizontal direction; power drive modules: each movable joint of each 6-degree-of-freedom serial robot and each degree of freedom of each mobile platform being provided with corresponding power drive modules, so as to drive the movable joint or the degree of freedom to achieve single-degree-of-freedom rotation or single-degree-of-freedom translation in an axial direction; and a control module, which can control in a command manner the power drive modules to move to specified positions at a given speed. Without using additional devices, the present apparatus and method can easily compensate the effect of gravity on hand-eye calibration.

Description

Omnidirectional measurement device system, registration method, electronic device and storage medium Technical Field

[0001] The present invention belongs to the field of three-dimensional measurement, and in particular relates to an omnidirectional measurement device system device containing multi-lateral working measurement devices, a registration method, an electronic device and a storage medium. Background Art

[0002] Inspection tasks in manufacturing industries such as automotive, shipbuilding, and aerospace often involve inspecting large workpieces. This often involves both overall topography measurement and local detail measurement. The former is used to confirm whether the workpiece's shape meets design requirements, such as aerodynamic performance and shape and structure, while the latter is used to assess the quality of local structures such as connectors and welds that have a significant impact on strength. Tandem robotic arm systems offer the advantages of flexible workspaces and simple control. However, when operating with large loads or at long working distances, gravity can significantly impact their end-of-line positioning performance. Before measurement begins, the measurement system and the robotic arm's end-of-line coordinates must be calibrated, a process known as hand-eye calibration. The results of this calibration are then used to perform three-dimensional registration of the measurement system's coordinate system with the workpiece's coordinate system. Inadequate end-of-line positioning performance can reduce the accuracy of hand-eye calibration and 3D registration, making it difficult to align and compare the measurement results with the workpiece's digital model, thus impacting measurement accuracy. Therefore, in omnidirectional measurement systems with robotic arms, especially those with sideways-mounted arms, mitigating the effects of gravity during hand-eye calibration and 3D registration can significantly improve registration accuracy.

[0003] Summary of the Invention

[0004] In order to overcome the deficiencies of the prior art, the present application proposes an omnidirectional measurement device system device including a multi-lateral working measurement device, a registration method, an electronic device and a storage medium.

[0005] In order to achieve the above-mentioned invention objectives, the technical solutions provided by this application are as follows:

[0006] An omnidirectional measurement device system device, comprising:

[0007] Motion module: includes multiple 1 to 3 degree-of-freedom mobile platforms, each of which is equipped with one or more 6-degree-of-freedom serial robots;

[0008] Measurement module: A three-dimensional measuring device is installed at the end of each serial robot, and the measurement direction is horizontal;

[0009] Power drive module: Each active joint of the 6-degree-of-freedom serial robot and each degree of freedom of the mobile platform has a corresponding power drive module, which can drive the active joint or degree of freedom to achieve single-degree-of-freedom rotation or single-degree-of-freedom translation along the axial direction; Control module: It can control the power drive module to move to a given position at a given speed in the form of instructions.

[0010] A registration method for an omnidirectional measurement device comprises the following steps:

[0011] Step 1. Determine the positional relationship between the measuring device and the robot end;

[0012] Step 2: Align the workpiece to be measured to the world coordinate system determined by the omnidirectional measurement equipment system.

[0013] Furthermore, the step 1 includes the following steps:

[0014] Step 1). Install the calibration plate vertically within the measurement range of the serial robot and 3D measurement equipment;

[0015] Step 2). Calculate the calibration pose sequence of the serial robot in the simulation program;

[0016] Step 3). Issue motion instructions to the serial robot to make it move to the initial measurement pose, and rotate the robot end joint k angles to measure the calibration plate. The end angle coordinates are marked as θ0, θ1, ..., θ k- 1, the corresponding rotation matrix is ​​recorded as:

[0017] Calculate the rotation matrix between the robot coordinates of the subsequent measurement points and the robot coordinates of the first measurement point:

[0018] Step 4). Issue motion instructions to the serial robot, so that the serial robot passes through the remaining preset nk measurement poses in turn, and calculate the homogeneous transformation matrix H from the serial robot end to the base through the serial robot sensor data g，i , i = k, k+1, ..., n-1; calculate the homogeneous transformation matrix H between the robot coordinates of the subsequent measurement points and the robot coordinates of the k+1th measurement point g，ki , and then obtain the rotation matrix R g，ki and the translation vector t g，ki , i=k, k+1,…,n-1:

[0019] Step 5). Use the measuring device to collect the calibration plate image, calculate the three-dimensional coordinates of each feature point of the calibration plate, and calculate the homogeneous transformation matrix H from the calibration plate coordinate system to the camera coordinate system by the singular value decomposition method. c,i ,i=0,1,…,n-1;

[0020] Calculate the homogeneous transformation matrix H between the camera coordinates of the subsequent measurement points and the camera coordinates of the first measurement point c,0i , and then obtain the rotation matrix R c,0i Translation vector t c，0i , i=1,2,…,k-1:

[0021] Calculate the homogeneous transformation matrix H between the camera coordinates of the subsequent measurement points and the camera coordinates of the kth measurement point c,ki , and then obtain the rotation matrix R c,ki and the translation vector t c,ki , i=k+1,…,n-1:

[0022] Step 6) Use R g，0i , R g，ki ,R c,0i , R c，ki Perform optimization and solve to get the hand-eye rotation vector p cg Multiple sets of constraints, solve the hand-eye rotation vector p through common constraints cg , and then converted into the hand-eye rotation matrix R cg ;

[0023] Step 7). Use R g,0i , R g，ki , R c，0i , R c，ki , t c，0i , t g，0i Perform optimization and solve to get the hand-eye translation vector t cg Multiple sets of constraints, solve the hand-eye rotation vector t through common constraints cg .

[0024] Furthermore, in step 6), the rotation matrix R g，0i ,R g，ki ,R c,0i , R c，ki Expressed as the rotation vector p g，0i , p g，ki , p c，0i , p c，ki ;

[0025] Step 7) is as follows: Select p g，0i , p c，0i , i=1,…,k-1 perform linear least squares solution to obtain the hand-eye rotation vector p cg A set of constraints:

[0026] A1p′ cg =B1

[0027] in

[0028] In the above formula, A1 and B1 are coefficient matrices, and p′ cg is the regularized rotation vector, S(·) is the mapping from vector to antisymmetric matrix;

[0029] Performing singular value decomposition on A1, we can get A1=U1S1V1, where U1 and V1 are orthogonal matrices and S1 is a diagonal matrix, which can be expressed as:

[0030] where λ 11 ,λ 12 ,λ 13 is the eigenvalue of A1. Since A1 is theoretically rank 2, we have λ 13 =0, remember:

[0031] Then we can get p′ cg A constraint:

[0032] H1p′ cg =D1;

[0033] Among them, H1, D1 are coefficient matrices;

[0034] Select p g，ki , p c,ki , i=k+1,…,n-1 perform linear least squares solution to obtain the hand-eye rotation vector p cg Another set of constraints:

[0035] A2p′ cg =B2;

[0036] in

[0037] Among them, A2, B2 are coefficient matrices, p′ cg is the regularized rotation vector;

[0038] Solve using constrained least squares method:

[0039] According to the above formula, ′ cg Calculate the hand-eye rotation vector p cg , and then calculate the hand-eye rotation matrix R cg ;

[0040] Select R g，0i , t c,0i ,t g，0i , i=1,…,k-1 perform linear least squares solution to obtain the hand-eye translation vector t cg A set of constraints:

[0041] A3t cg =B3;

[0042] in

[0043] I is the identity matrix. Performing singular value decomposition on A3, we can get A3=U3S3V3, where U3 and V3 are orthogonal matrices and S3 is a diagonal matrix, which can be expressed as:

[0044] where λ 31 ,λ 32 ,λ 33 is the eigenvalue of A3. Since the theoretical rank of A3 is 2, we have λ 33 ≈0, remember:

[0045] Then we can get t cg A constraint:

[0046] H3t cg =D3;

[0047] Where, is the coefficient matrix;

[0048] Select R g，0i , t c,0i , t g,0i , i=k+1,…,n-1 perform linear least squares solution to obtain the hand-eye translation vector t cg Another set of constraints:

[0049] A4t cg =B4;

[0050] in

[0051] Among them, A4, B4 are coefficient matrices;

[0052] Use constrained least squares method to solve the hand-eye translation vector t cg :

[0053] Furthermore, the steps of step 2 include:

[0054] Step a). Fix the workpiece to be measured to the working position;

[0055] Step b). Selecting several feature areas containing several feature points on the workpiece, and obtaining the coordinates of all feature points in the workpiece coordinate system through the workpiece CAD model;

[0056] Step c) controlling each robot to sequentially pass through the feature area that can be covered, perform three-dimensional measurement on the feature area, and extract the three-dimensional coordinates of the feature points in the coordinate system of the measuring device;

[0057] Step d). For each measurement point, the end of the robot is rotated 90 degrees, 180 degrees, and 270 degrees for three-dimensional measurement, and the three-dimensional coordinates of the feature point in the measuring device coordinate system are extracted;

[0058] Step e). For each measurement result, the three-dimensional coordinates of the feature point in the world coordinate system are calculated by measuring the transformation matrix from the coordinate system of the measuring device to the end of the robot, the forward kinematic model of the robot, and the transformation matrix from the coordinate system of the robot base to the world coordinate system, and the three-dimensional coordinates of the feature point in the world coordinate system and the coordinates of the feature point in the workpiece coordinate system are recorded;

[0059] Step f) using a singular value decomposition method to calculate the transformation matrix from the workpiece coordinate system to the world coordinate system;

[0060] Step g). Calculate the coordinates of each feature point in the world coordinate system using the transformation matrix from the workpiece coordinate system to the world coordinate system, and compare them with the observation results.

[0061] An electronic device, comprising:

[0062] at least one processor; and a memory and a controller communicatively coupled to the at least one processor;

[0063] The memory stores instructions executable by the at least one processor, wherein the instructions are configured to execute the above-mentioned omnidirectional measurement device registration method;

[0064] The controller is capable of sending the motion instruction to the control module.

[0065] A computer-readable storage medium is characterized in that the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute an omnidirectional measurement device registration method.

[0066] The advantages of the present invention are:

[0067] 1. To address the problem of gravity significantly affecting the positioning accuracy of the robot's end-user during sideways operation, resulting in inaccurate hand-eye calibration results and, in turn, insufficient registration accuracy, this application proposes a hand-eye calibration method that changes the measurement direction while maintaining the same measurement position. This method can easily compensate for the influence of gravity on hand-eye calibration without the use of additional equipment.

[0068] 2. To address the low workpiece registration accuracy associated with a single feature point on a single measurement device when hand-eye calibration is inaccurate, a method was proposed that uses multiple systems to measure multiple feature points on the workpiece and then uses singular value decomposition to obtain the registration results. This method effectively compensates for the low hand-eye registration accuracy caused by non-geometric errors such as gravity, which can lead to insufficient registration accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] FIG1 is a schematic diagram of an embodiment of the present invention. DETAILED DESCRIPTION

[0070] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are intended to explain the present invention rather than to limit the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0071] The specific implementation method of the present invention is described below with reference to the accompanying drawings and examples, but the present invention is not limited to this embodiment.

[0072] Example 1

[0073] A system device of an omnidirectional measurement device containing multiple measurement devices, comprising:

[0074] (1) Motion module: Multiple 1 to 3 degree-of-freedom mobile platforms, each with one or more 6-degree-of-freedom serial robots installed.

[0075] (2) Measurement module: A three-dimensional measurement electronic device is installed at the end of each serial robot.

[0076] (3) Power drive module. Each active joint of the 6-DOF serial robot and each degree of freedom of the mobile platform has a corresponding power drive module, which can drive the active joint or degree of freedom to rotate along an axis or translate in a single degree of freedom;

[0077] (4) Control module: It can control the power drive module to move to a given position at a given speed in the form of instructions.

[0078] In one embodiment of the present disclosure, three mobile platforms are included, one of which has x-direction degrees of freedom and two of which have x- and z-direction degrees of freedom in world coordinates. Each mobile platform is equipped with a 6-DOF serial robot, and each robot end is equipped with a measuring device. The core part of the control module is a PLC, which can send motion control instructions to all robots and mobile platforms, so that they move to a given position at a given speed. At the same time, it can accept instructions from the host computer and convert the host computer instructions into motion instructions for the robots and mobile platforms. The following is further explained with reference to the accompanying drawings:

[0079] Figure 1 is a schematic diagram of an embodiment of the present invention. This embodiment primarily comprises an upper single-degree-of-freedom mobile platform 1, an upper collaborative robot 2, an upper measuring device 3, a left two-degree-of-freedom mobile platform 4, a left collaborative robot 5, a left measuring device 6, a right two-degree-of-freedom mobile platform 7, a right collaborative robot 8, and a right measuring device 9. The upper measuring device 3 is mounted on the end of the upper collaborative robot 2, the base of which is mounted on the upper single-degree-of-freedom mobile platform 1; the left measuring device 6 is mounted on the end of the left collaborative robot 5, the base of which is mounted on the left two-degree-of-freedom mobile platform 4; and the right measuring device 9 is mounted on the end of the right collaborative robot 8, the base of which is mounted on the right two-degree-of-freedom mobile platform 7. During measurement, the control system sends independent motion instructions to the upper single-degree-of-freedom mobile platform 1, the upper collaborative robot 2, the left two-degree-of-freedom mobile platform 4, the left collaborative robot 5, the right two-degree-of-freedom mobile platform 7, and the right collaborative robot 8, so that they move to the measurement position, and then sends measurement instructions to the upper measuring device 3, the left measuring device 6, and the right measuring device 9, so that they perform measurements and return the measurement results to the control system.

[0080] Example 2

[0081] The second embodiment disclosed in the present application proposes a three-dimensional registration method for an omnidirectional measurement device including a multi-lateral working measurement device, specifically comprising:

[0082] (1) Using the binocular measuring device of the measuring device to determine the positional relationship between the measuring device and the end of the robot;

[0083] Specifically include

[0084] (1-1) Install the calibration plate vertically within the measurement range of the robot and measuring equipment;

[0085] (1-2) Calculate the robot's calibration pose sequence in the simulation program so that the calibration plate is within the optimal measurement distance;

[0086] (1-3) Issue motion instructions to the robot to make it move to the initial measurement position, and rotate the robot's end joints to multiple angles k to measure the calibration plate. The end angle coordinates are marked. 为 θ0, θ1, …, θ k-1 , the corresponding rotation matrix is ​​recorded as:

[0087] Calculate the rotation matrix between the robot coordinates of the subsequent measurement points and the robot coordinates of the first measurement point:

[0088] (1-4) Issue motion instructions to the robot, so that the robot passes through the remaining preset nk measurement poses in turn, and calculate the homogeneous transformation matrix H from the robot end to the base through the robot sensor data g，i , i=k,k+1,…,n-1. In this embodiment, the robot forward kinematics method is used to solve, that is:

[0089] Further, the homogeneous transformation matrix H between the robot coordinates of the subsequent measurement point and the robot coordinates of the k+1th measurement point is calculated g，ki , and then obtain the rotation matrix R g，ki and the translation vector t g,ki ,i=k,k+1,…,n-1:

[0090] (1-5) Use the measuring device to collect the calibration plate image and calculate the three-dimensional coordinates of each feature point of the calibration plate. Calculate the homogeneous transformation matrix from the calibration plate coordinate system to the camera coordinate system using the singular value decomposition method.

[0091] H c，i , i=0, 1,…, n-1.

[0092] Calculate the homogeneous transformation matrix H between the camera coordinates of the subsequent measurement points and the camera coordinates of the first measurement point c,0i , and then obtain the rotation matrix R c,0i Translation vector t c，0i , i=1,2,…,k-1:

[0093] Calculate the homogeneous transformation matrix H between the camera coordinates of the subsequent measurement points and the camera coordinates of the kth measurement point c,ki , and then obtain the rotation matrix R c，ki and the translation vector t c，ki , i=k+1,…,nc,1:

[0094] (1-6) Using R g，0i , R g，ki , R c，0i , Rc，ki Perform optimization and solve to get the hand-eye rotation vector p cg Multiple sets of constraints, solve the hand-eye rotation vector p through common constraints cg , and then converted into the hand-eye rotation matrix R cg ;

[0095] Rotate the matrix R in steps (1-3) to (1-5) g,0i , R g，ki , R c,0i ,R c，ki Expressed as the rotation vector p g，0i , p g,ki , p c，0i , p c，ki ;

[0096] (1-7) Using R g,0i ,R g,ki , R c，0i ,R c，ki ,t c,0i ,t g,0i Perform optimization and solve to get the hand-eye translation vector t cg Multiple sets of constraints, solve the hand-eye rotation vector t through common constraints cg .

[0097] Specifically: Select p g，0i , p c，0i , i=1,…,k-1 perform linear least squares solution to obtain the hand-eye rotation vector p cg A set of constraints:

[0098] A1p′ cg =B1;

[0099] in

[0100] In the above formula, A1 and B1 are coefficient matrices, and p′ cg is the regularized rotation vector, S(·) is the mapping from vector to antisymmetric matrix;

[0101] Performing singular value decomposition on A1, we can get A1=U1S1V1, where U1 and V1 are orthogonal matrices and S1 is a diagonal matrix, which can be expressed as:

[0102] where λ 11 ,λ 12 ,λ 13 is the eigenvalue of A1. Since A1 is theoretically rank 2, we have λ 13 ≈0, remember:

[0103] Then we can get p′ cgA constraint:

[0104] H1p′ cg =D1;

[0105] Among them, H1, D1 are coefficient matrices;

[0106] (1-8) Select p g，ki , p c，ki , i=k+1,…,n-1 perform linear least squares solution to obtain the hand-eye rotation vector p cg Another set of constraints:

[0107] A2p′ cg =B2

[0108] in

[0109] Among them, A2, B2 are coefficient matrices, p′ cg is the regularized rotation vector;

[0110] Using the constraints in steps (1-7), solve for p′ using constrained least squares cg :

[0111] Then from the above formula p′ cg Calculate the hand-eye rotation vector p cg , and then calculate the hand-eye rotation matrix R cg .

[0112] (1-9) Select R g，0i ,t c,0i ,t g,0i , i=1,…,k-1 perform linear least squares solution to obtain the hand-eye translation vector t cg A set of constraints:

[0113] A3t cg =B3

[0114] in

[0115] I is the identity matrix. Performing singular value decomposition on A3, we can get A3=U3S3V3, where U3 and V3 are orthogonal matrices and S3 is a diagonal matrix, which can be expressed as:

[0116] where λ 31 ,λ 32 ,λ 33 is the eigenvalue of A3. Since the theoretical rank of A3 is 2, we have λ 33 ≈0, remember:

[0117] Then we can get t cg A constraint:

[0118] H3t cg =D3

[0119] Where, is the coefficient matrix.

[0120] (1-10) Select R g，0i , t c,0i ,t g，0i , i=k+1,…,n-1 perform linear least squares solution to obtain the hand-eye translation vector t described in step (1-9) cg Another set of constraints:

[0121] A4t cg =B4

[0122] in

[0123] Using the constraints in steps (1-9), the constrained least squares method is used to solve the hand-eye translation vector t cg :

[0124] and It is an intermediate variable and has no practical meaning.

[0125] (2) Aligning the workpiece to be measured to the world coordinate system determined by the omnidirectional measuring equipment system, specifically including:

[0126] (2-1) Fix the workpiece to be measured to the working position;

[0127] (2-2) Select several feature areas containing several feature points on the workpiece, and obtain the coordinates of all feature points in the workpiece coordinate system through the workpiece CAD model.

[0128] (2-3) controlling each robot to sequentially pass through the feature area that it can cover, performing three-dimensional measurement on the feature area, and extracting the three-dimensional coordinates of the feature points in the coordinate system of the measuring device;

[0129] (2-4) For each measurement point, the end of the robot is rotated 90 degrees, 180 degrees, and 270 degrees respectively for three-dimensional measurement, and the three-dimensional coordinates of the feature point in the measurement equipment coordinate system are extracted.

[0130] (2-5) For each measurement result, calculate the three-dimensional coordinates of the feature point in the world coordinate system using the transformation matrix from the measurement device coordinate system to the robot end point, the robot's forward kinematic model, and the transformation matrix from the robot base coordinate system to the world coordinate system. Record the three-dimensional coordinates of the feature point in the world coordinate system and the coordinates of the feature point in the workpiece coordinate system.

[0131] (2-6) Use the singular value decomposition method to calculate the transformation matrix from the workpiece coordinate system to the world coordinate system.

[0132] (2-7) Use the transformation matrix from the workpiece coordinate system to the world coordinate system to calculate the coordinates of each feature point in the world coordinate system and compare them with the observation results.

[0133] In the embodiment of the present disclosure, three mobile platforms are included. In the world coordinate system, one of them has x-direction freedom and two of them have x- and z-direction freedom. Each mobile platform is equipped with a 6-DOF serial robot, and each robot end is equipped with an active binocular measurement device. The point cloud obtained by the measurement device can be represented by a homogeneous coordinate set Indicates that the coordinates are expressed in the measuring device coordinate system. Through the pre-calibration results, they can be converted to the world coordinate system, that is:

[0134] in From the robot kinematics, R i is the rotation matrix of the installation direction of the i-th robot, is the result of hand-eye calibration, t ji The position of the i-th robot base can be expressed as:

[0135] t j,1 =(x 0，1 ,y 0，1 , z 0，1 ) T +d x，j1 (p x,1 ,q x,1 ,r x，1 ) T

[0136] t ji =(x 0,i ,y 0，i , Z 0,i ) T +d x,ji (p x,i ,q x,i , r x,i ) T +d z，ji (p z，i ,q z，i , r z,i ) T ,i=2,3

[0137] Where (x 0，i ,y 0，i , Z 0,i ), i=1, 2, 3 are the origins of the platform robot base, dx，ji is the movement distance of the i-th mobile platform in the x direction, d z，ji is the movement distance of the i-th mobile platform in the x direction, (p x，i ,q x,i ,r x,i ) T is the movement direction of the ith mobile platform in the x direction in the world coordinates, (p z,i ,q z,i ,r z,i ) T is the movement direction of the i-th mobile platform in the z direction in the world coordinates.

[0138] Example 3

[0139] In order to implement the above embodiments, the third aspect of the present disclosure proposes an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are configured to execute the above method.

[0140] Example 4

[0141] In order to implement the above embodiment, this embodiment provides a computer-readable storage medium on which a computer program is stored. The program is executed by a processor to implement the above embodiment.

[0142] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or component. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0143] The computer-readable medium may be included in the electronic device or may exist independently without being incorporated into the electronic device. The computer-readable medium carries one or more programs, and when the one or more programs are executed by the electronic device, the electronic device executes the above-mentioned embodiments.

[0144] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0145] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0146] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of such features. Throughout the description of this application, "plurality" means at least two, for example, two, three, etc., unless otherwise specifically defined.

[0147] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may be performed out of the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present application belong.

[0148] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk cartridge (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and a portable compact disc read-only memory (CDROM). Furthermore, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or otherwise processing it in a suitable manner if necessary, and then storing it in a computer memory.

[0149] It should be understood that various parts of the present application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0150] Those skilled in the art will appreciate that all or part of the steps in the method for implementing the above-mentioned embodiment can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

[0151] In addition, the functional units in the various embodiments of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into a module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0152] The storage medium mentioned above may be a read-only memory, a magnetic disk, or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present application. Persons skilled in the art may make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present application.

Claims

1. An omnidirectional measurement device system, characterized in that: Includes the following modules Motion module: includes multiple 1 to 3 degree-of-freedom mobile platforms, each of which is equipped with one or more 6-degree-of-freedom serial robots; Measuring module: A three-dimensional measuring device is installed at the end of each serial robot, and the measuring direction is horizontal; Power drive module: Each active joint of the 6-DOF serial robot and each degree of freedom of the mobile platform has a corresponding power drive module, which can drive the active joint or degree of freedom to achieve single-DOF rotation or single-DOF translation along the axial direction; Control module: can control the power drive module to move to a given position at a given speed in the form of instructions.

2. A registration method for an omnidirectional measurement device, characterized in that: The following steps are included Step 1. Determine the positional relationship between the measuring device and the end of the robot; Step 2: Align the workpiece to be measured to the world coordinate system determined by the omnidirectional measuring device system device as claimed in claim 1.

3. The omnidirectional measurement device registration method according to claim 2, characterized in that: The step 1 comprises the following steps: Step 1). Install the calibration plate vertically within the measurement range of the serial robot and the 3D measurement equipment; Step 2). Calculate the calibration pose sequence of the serial robot in the simulation program; Step 3). Send a motion command to the serial robot to make it move to the initial measurement position, and rotate the robot's end joint by k angles to measure the calibration plate. The end angle coordinates are marked as θ0, θ1, ...θ k-1 , the corresponding rotation matrix is ​​recorded as: Calculate the rotation matrix between the robot coordinates of the subsequent measurement points and the robot coordinates of the first measurement point: Step 4). Issue motion instructions to the serial robot, so that the serial robot passes through the remaining preset nk measurement postures in turn, and calculate the homogeneous transformation matrix H from the end of the serial robot to the base through the serial robot sensor data g,i i=k,k+1,…,n-1; calculate the homogeneous transformation matrix H between the robot coordinates of the subsequent measurement points and the robot coordinates of the k+1th measurement point g，ki , and then obtain the rotation matrix R g，ki and the translation vector t g，ki , i=k+1,…,n-1: Step 5). Use the measuring device to collect the calibration plate image, calculate the three-dimensional coordinates of each feature point of the calibration plate, and calculate the homogeneous transformation matrix H from the calibration plate coordinate system to the camera coordinate system by the singular value decomposition method. c，i , i=0,1,…,n-1; Calculate the homogeneous transformation matrix H between the camera coordinates of the subsequent measurement points and the camera coordinates of the first measurement point c，0i , and then obtain the rotation matrix R c，0 i and translation vector t c,0i , i = 1, 2, ..., k-1: Calculate the homogeneous transformation matrix H between the camera coordinates of the subsequent measurement points and the camera coordinates of the kth measurement point c，ki , and then obtain the rotation matrix R c，ki and the translation vector t c,ki ,i=k+1,…,n-1: Step 6). Use R g,0i ,R g,ki ,R c,0i ,R c,ki Perform optimization and solve to obtain the hand-eye rotation vector p cg Multiple sets of constraints, solve the hand-eye rotation vector p through common constraints cg , and then converted into the hand-eye rotation matrix R cg ; Step 7). Use R g,0i ,R g,ki ,R c,0i ,R c,ki , t c，0i , t g，0i Perform optimization and solve to obtain the hand-eye translation vector t cg Multiple sets of constraints, solve the hand-eye rotation vector t through common constraints cg .

4. The registration method of an omnidirectional measurement device according to claim 3, characterized in that: In step 6), the rotation matrix R g,0i ,R g,ki ,R c,0i ,R c,ki Represented as the rotation vector p g,0i ,p g,ki ,p c,0i ,p c,ki .

5. The omnidirectional measurement device registration method according to claim 4, characterized in that: Step 7) is as follows: Select p g,0i ,p c,0i , i=1,…,k-1 perform linear least squares solution to obtain the hand-eye rotation vector p cg A set of constraints: A1p′ cg =B1; in In the above formula, A1 and B1 are coefficient matrices, p′ cg is the regularized rotation vector, S(·) is the mapping from vector to antisymmetric matrix; Performing singular value decomposition on A1, we can get A1=U1S1V1, where U1 and V1 are orthogonal matrices and S1 is a diagonal matrix, which can be expressed as: where λ 11 ,λ 12 ,λ 13 is the eigenvalue of A1. Since A1 is theoretically rank 2, we have λ 13 ≈0, remember: Then we can get p′ cg A constraint: H1p′ cg =D1; Among them, H1, D1 are coefficient matrices; Select p g,ki ,p c,ki , i=k+1,…,n-1 to perform linear least squares solution and obtain the hand-eye rotation direction Quantity p cg Another set of constraints: <h2 style=";text-align:left;direction:ltr">A2p′<h2 style=";text-align:left;direction:ltr"> cg <h2 style=";text-align:left;direction:ltr"> =B2; in Among them, A2, B2 are coefficient matrices, p′ cg is the regularized rotation vector; Solve using constrained least squares method: Then from the above formula p′ cg Calculate the hand-eye rotation vector p cg , and then calculate the hand-eye rotation matrix R cg ; Select R g,0i ,t c,0i ,t g,0i , i=1,…,k-1 perform linear least squares solution to obtain the hand-eye translation vector t cg A set of constraints: <h2 style=";text-align:left;direction:ltr">A3t<h2 style=";text-align:left;direction:ltr"> cg <h2 style=";text-align:left;direction:ltr"> =B3; in I is the unit matrix. Performing singular value decomposition on A3, we can get A3=U3S3V3, where U3, V3 are orthogonal matrices and S3 is a diagonal matrix, which can be expressed as: where λ 31 ,λ 32 ,λ 33 is the eigenvalue of A3. Since the theoretical rank of A3 is 2, we have λ 33 ≈0, remember: Then we can get t cg A constraint: <h2 style=";text-align:left;direction:ltr">H3t<h2 style=";text-align:left;direction:ltr"> cg <h2 style=";text-align:left;direction:ltr"> (D3) Among them, is the coefficient matrix; Select R g,0i ,t c,0i ,t g,0i , i=k+1,…,n-1 perform linear least squares solution to obtain the hand-eye translation vector t cg Another set of constraints: A4t cg =B4; in Among them, A4 and B4 are coefficient matrices; The constrained least squares method is used to solve the hand-eye translation vector t cg :

6. The omnidirectional measurement device registration method according to claim 2, characterized in that: The step 2 comprises: Step a). Fix the workpiece to be tested to the working position; Step b). Selecting several feature areas including several feature points on the workpiece, and obtaining the coordinates of all feature points in the workpiece coordinate system through the workpiece CAD model; Step c) controlling each robot to pass through the feature area that it can cover in turn, performing three-dimensional measurement on the feature area, and extracting the three-dimensional coordinates of the feature points in the coordinate system of the measuring device; Step d). For each measuring point, the end of the robot is rotated 90 degrees, 180 degrees, and 270 degrees respectively for three-dimensional measurement, and the three-dimensional coordinates of the feature point in the coordinate system of the measuring device are extracted; Step e). For each measurement result, the three-dimensional coordinates of the feature point in the world coordinate system are calculated by measuring the transformation matrix from the coordinate system of the device to the end of the robot, the forward kinematics model of the robot, and the transformation matrix from the coordinate system of the robot base to the world coordinate system. Recording the three-dimensional coordinates of the feature points in the world coordinate system and the coordinates of the feature points in the workpiece coordinate system; Step f) using a singular value decomposition method to calculate the transformation matrix from the workpiece coordinate system to the world coordinate system; Step g). Using the transformation matrix from the workpiece coordinate system to the world coordinate system, calculate the coordinates of each feature point in the world coordinate system and compare them with the observation results.

7. An electronic device, characterized in that: comprising at least one processor; and a memory and a controller communicatively connected to the at least one processor; The memory stores instructions executable by the at least one processor, wherein the instructions are configured to execute an omnidirectional measurement device registration method according to any one of claims 2 to 6; The controller is capable of sending the motion instruction to the control module.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute an omnidirectional measurement device registration method as claimed in any one of claims 2 to 6.

Images