A positioning method and device for a motion mechanism

Abstract

This application discloses a positioning method for a motion mechanism. The method includes: on a control side used for controlling the motion mechanism, acquiring a transformation relationship, wherein the transformation relationship characterizes the translational and rotational transformations between the motion mechanism in a first state and the motion mechanism in a second state; using the transformation relationship, determining the spatial position information of the point to be positioned in the motion mechanism in the second state based on the spatial position information of the point to be positioned in the motion mechanism in the first state, wherein the second state is a running state. This application is compatible with various two-dimensional positioning application scenarios, has higher portability and versatility, and possesses better geometric interpretation than planar point operations, thus improving the development efficiency of intelligent control of motion mechanisms.

Description

Technical Field

[0001] This invention relates to the field of industrial control, and in particular, to a positioning method for a motion mechanism. Background Technology

[0002] In industrial production, there are frequent application scenarios involving the manipulation of targets through motion mechanisms. For example, a robotic arm might move a target material from position A to position B, or assemble target material A into target material B. During the manipulation of targets by motion mechanisms, it is often necessary to position the mechanism so that appropriate mechanical control can be performed based on the positioning results.

[0003] Existing positioning methods for motion mechanisms are usually designed based on the operating mode, path, and relationships between the components of the motion mechanism, combined with the specific application scenario. Due to the richness of industrial production and the diversity of application scenarios, the existing positioning methods for motion mechanisms have poor portability. Summary of the Invention

[0004] This invention provides a positioning method for a motion mechanism to improve the portability of motion mechanism positioning methods.

[0005] This invention provides a positioning method for a motion mechanism, the method comprising: on a control side for controlling the motion mechanism,

[0006] Obtain the transformation relationship, wherein the transformation relationship is used to characterize the translational and rotational transformations between the motion mechanism in the first state and the motion mechanism in the second state.

[0007] Using the transformation relationship, and based on the spatial position information of the point to be located in the motion mechanism in the first state, the spatial position information of the point to be located in the motion mechanism in the second state is determined.

[0008] in,

[0009] The second state is the running state.

[0010] Preferably, the transformation relationship is determined as follows:

[0011] First image data and second image data are acquired, wherein the first image data is image data of the motion mechanism in the first state, and the second image data is image data of the motion mechanism in the second state.

[0012] Using the camera calibration transformation relationship of the camera model, the pixel position information of the first pixel in the first image data is converted into the first spatial point position information, and the pixel position information of the second pixel in the second image data is converted into the second spatial point position information. The first pixel and the second pixel correspond to each other, and the first and second spatial point position information are in the same coordinate system.

[0013] Based on the position information of the first spatial point and the position information of the second spatial point, as well as the rotation angle between the first and second spatial points, a translation vector representing the translation transformation and a rotation matrix representing the rotation transformation are determined.

[0014] Based on the translation vector and the rotation matrix, determine the homogeneous transformation matrix used to characterize the transformation relationship.

[0015] Preferably, the first state is the calibration state when the motion mechanism reaches the set position during motion;

[0016] The second state is the current running state.

[0017] The acquisition of the first image data and the second image data includes:

[0018] In the current operating state of the motion mechanism, a current image of the motion mechanism operating the first target in the current operating state is acquired to obtain first image data.

[0019] In the calibration state of the motion mechanism, a current image is acquired when the motion mechanism operates the second target to a set reference pose in the calibration state, thus obtaining second image data.

[0020] The first target and the second target are the same target.

[0021] Preferably, the step of using the camera calibration transformation relationship of the camera model to convert the pixel position information of the first pixel in the first image data into first spatial point position information, and converting the pixel position information of the second pixel in the second image data into second spatial point position information, includes:

[0022] Using the camera parameter matrix obtained from camera calibration, the pixel position information of the first pixel in the first image data is converted into reference point position information, and the pixel position information of the second pixel in the second image data is converted into running point position information.

[0023] The first pixel and the second pixel have the same pixel position information.

[0024] Preferably, the motion mechanism is a coaxial mechanism, wherein the rotation center of the operating part located at the end of the mechanism coincides with the rotation center of the actuator used to control the end of the mechanism.

[0025] The step of determining the translation vector and rotation matrix based on the position information of the first spatial point and the position information of the second spatial point, as well as the rotation angle between the first spatial point and the second spatial point, includes:

[0026] By utilizing the identity that the product of the homogeneous transformation matrix and the spatial vector of the reference point equals the spatial vector of the running point, and based on the position information of the running point and the reference point, the translation vector can be solved.

[0027] Set the rotation matrix to the identity matrix.

[0028] Preferably, the motion mechanism is a non-coaxial mechanism, wherein the rotation center of the operating part located at the end of the mechanism does not coincide with the rotation center of the actuator used to control the end of the mechanism.

[0029] The step of determining the translation vector representing the translation transformation and the rotation matrix representing the rotation transformation based on the position information of the first spatial point and the position information of the second spatial point, as well as the rotation angle between the first spatial point and the second spatial point, includes:

[0030] By utilizing the identity that the product of the homogeneous transformation matrix and the spatial vector of the reference point equals the spatial vector of the running point, and based on the position information of the running point and the reference point, as well as the rotation angle between the running point and the reference point, the rotation matrix and translation vector can be solved.

[0031] Preferably, the operation of the first objective includes a picking operation of picking up the target to be picked up, and the operation of the second objective includes a picking operation of picking up the target to be picked up.

[0032] The point to be located is the gripping point at the end of the motion mechanism.

[0033] The step of using the transformation relationship to determine the spatial position information of the target positioning point in the motion mechanism in the second state, based on the spatial position information of the target positioning point in the motion mechanism in the first state, includes:

[0034] The homogeneous transformation matrix is ​​multiplied by the spatial vector of the gripping point obtained by the motion mechanism in the calibration state to obtain the positioning information of the gripping point in the current operating state.

[0035] Preferably, the first state is the first operating state when the motion mechanism needs to reach a first set position during motion;

[0036] The second state is the second operating state when the motion mechanism reaches the second set position during motion;

[0037] The acquisition of the first image data and the second image data includes:

[0038] In the first operating state, the current image of the motion mechanism operating the first target is acquired to obtain first image data.

[0039] In the second operating state, the current image of the motion mechanism operating the second target is acquired to obtain second image data.

[0040] Wherein, the first target is the target used for assembly, and the second target is the target assembled by the first target.

[0041] Preferably, determining the translation vector characterizing the translation transformation and the rotation matrix characterizing the rotation transformation based on the position information of the first spatial point and the position information of the second spatial point, and the rotation angle between the first spatial point and the second spatial point, includes:

[0042] By utilizing the identity that the product of the homogeneous transformation matrix and the spatial vector of the first spatial point equals the spatial vector of the second spatial point, and based on the position information of the first and second spatial points, as well as the rotation angle between the first and second spatial points, the rotation matrix and translation vector can be solved.

[0043] Preferably, the first objective of the operation includes a picking operation of picking up a first target to be picked up, and the second objective of the operation includes an assembly operation of assembling the picked-up first target with a second target.

[0044] The positioning point to be determined includes the gripping point located at the end of the motion mechanism.

[0045] The step of using the transformation relationship to determine the spatial position information of the target positioning point in the motion mechanism in the second state, based on the spatial position information of the target positioning point in the motion mechanism in the first state, includes:

[0046] The homogeneous transformation matrix is ​​multiplied by the grasping point space vector obtained by the motion mechanism in the first operating state to obtain the positioning information of the grasping point in the second operating state.

[0047] in,

[0048] The spatial vector of the grasping point is obtained in advance through calibration.

[0049] The present invention also provides a positioning device for a motion mechanism, the device comprising:

[0050] The acquisition module is used to acquire transformation relationships, wherein the transformation relationships characterize the translational and rotational transformations between the motion mechanism in the first state and the motion mechanism in the second state.

[0051] The determining module is used to determine the spatial position information of the point to be located in the motion mechanism in the second state, based on the spatial position information of the point to be located in the motion mechanism in the first state, using the transformation relationship.

[0052] in,

[0053] The second state is the running state.

[0054] The positioning method for motion mechanisms provided in this application is based on the more fundamental linear transformation theorem of two-dimensional projective space. It utilizes the transformation relationship between the translation and rotation transformations of the motion mechanism in the first state and the second state to determine the spatial position information of the point to be positioned. This method abandons the method of calculating the rotation and translation offsets of specific spatial points in the plane, making it simpler and reducing computational complexity. The positioning method of this application is independent of the motion mode, components, and specific operations of the motion mechanism, thus it is compatible with various two-dimensional positioning application scenarios, has higher portability and versatility, and has better geometric interpretation than planar point calculations, thereby improving the development efficiency of intelligent control of motion mechanisms. Attached Figure Description

[0055] Figure 1 This is a schematic diagram showing whether the rotation center of the robot arm is coaxial with or not coaxial with the center of the gripper.

[0056] Figure 2 This is a schematic diagram of the teaching posture and the running posture.

[0057] Figure 3 This is a schematic flowchart of a positioning method for a motion mechanism according to an embodiment of this application.

[0058] Figure 4 This is a flowchart illustrating a positioning method for a motion mechanism in the application of a coaxial mechanism for picking up a target, as described in Embodiment 1 of this application.

[0059] Figure 5 This is a flowchart illustrating a positioning method for a motion mechanism in the application of a non-coaxial mechanism for picking up a target, as shown in Embodiment 2 of this application.

[0060] Figure 6 This is a flowchart illustrating a positioning method of the motion mechanism in the application of the motion mechanism assembling a target, as described in Embodiment 3 of this application.

[0061] Figure 7This is a schematic diagram of a process for assembling a circular part into a countersunk hole of a rectangular part according to Embodiment 3 of this application.

[0062] Figure 8 This is a schematic diagram of a motion mechanism assembling a circular part into a countersunk hole of a rectangular part.

[0063] Figure 9 This is another schematic diagram of the positioning device of the motion mechanism in the embodiments of this application.

[0064] Figure 10 This is another schematic diagram of the positioning device of the motion mechanism in the embodiments of this application. Detailed Implementation

[0065] To make the objectives, technical means, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings.

[0066] In industrial production, motion mechanisms include two types: coaxial and non-coaxial. If the rotation center of the operating part located at the end of the motion mechanism body coincides with the rotation center of the actuator used to control the end of the mechanism, then the motion mechanism is a coaxial mechanism. If the rotation center of the operating part located at the end of the motion mechanism body does not coincide with the rotation center of the actuator used to control the end of the mechanism, then the motion mechanism is a non-coaxial mechanism.

[0067] See Figure 1 As shown, Figure 1 This diagram illustrates two scenarios: one where the rotation center of a robotic arm is coaxial with the center of a gripper, and the other where they are not. In the diagram, the gripping point of the robotic arm is used to pick up the target. In the upper diagram, the rotation center of the gripper (the gripping point) coincides with the rotation center of the robotic arm, making this a coaxial mechanism. In the lower diagram, the rotation center 1 of the gripper does not coincide with the rotation center 2 of the robotic arm, thus making this a non-coaxial mechanism. As can be seen from the diagram, in a coaxial mechanism, the end point of the mechanism, such as the gripping point, only undergoes translational changes, without rotational changes. In a non-coaxial mechanism, the end point of the mechanism undergoes not only translational changes but also rotational changes due to the rotation center of the robotic arm.

[0068] In traditional motion mechanism positioning methods, to determine the set position that the motion mechanism needs to reach during operation, such as a target position or a reference position, a reference can be created through teaching. See also Figure 2 As shown, Figure 2 This is a schematic diagram of the teaching posture and the running posture. In the figure, the left image shows the teaching posture of the motion mechanism when it runs to the set position. The image of the motion mechanism with this teaching posture can be used as the reference image. The right image shows the running posture of the motion mechanism when it is running. The image of the motion mechanism with the running posture can be used as the running image.

[0069] by Figure 1 Taking a non-coaxial motion mechanism as an example, in existing positioning methods for non-coaxial motion mechanisms, such as the positioning of the gripping point:

[0070] Based on the reference image, the coordinates 1 of a pixel in the reference image (referred to as the reference point for ease of description) in the world coordinate system are obtained using the calibration matrix. Based on the running image, the coordinates 2 of a pixel in the running image (referred to as the running point for ease of description) in the world coordinate system are obtained using the calibration matrix. The pixel position information of the reference point and the running point correspond.

[0071] The translation deviation is obtained based on the difference between coordinates 1 and 2.

[0072] Based on the sum of the coordinates of the reference point 1 and the reference image capture position coordinates, the reference position coordinates are obtained. The rotational deviation caused by rotating the grasping teaching position around the reference position is determined; for example, the rotational deviation caused by rotating the grasping point teaching position coordinates (obtained during teaching) around the grasping point reference position coordinates. Here, the reference image capture position is the position of the camera when acquiring the reference image.

[0073] The coordinates of the grab point are determined by translational deviation, rotational deviation, and teaching position coordinates.

[0074] Therefore, existing two-dimensional visual positioning mainly adopts planar point manipulation algorithms. The principle is based on the translation and rotation transformation formulas of specific feature points in the Euclidean plane. It involves not only translation and rotation deviations, but also reference position information. It requires calculations of rotating a spatial point around another spatial point. The main drawbacks of this method are: poor portability, as the algorithms often differ for different positioning scenarios; poor abstraction, requiring the definition of a series of concepts such as the rotation center of the mechanism; complex algorithm calculation steps; and poor geometric interpretation.

[0075] In view of this, this application provides a positioning method for a motion mechanism, which reduces the computational complexity of the algorithm without requiring the definition of the mechanism's rotation center, while improving the portability and versatility of the positioning method.

[0076] See Figure 3 As shown, Figure 3 This is a schematic flowchart illustrating a positioning method for a motion mechanism according to an embodiment of this application. On the control side of the motion mechanism, the method includes:

[0077] Step 301: Obtain the transformation relationship, wherein the transformation relationship is used to characterize the translation and rotation transformations between the motion mechanism in the first state and the motion mechanism in the second state.

[0078] Step 302: Using the transformation relationship, based on the spatial position information of the unknown positioning point in the motion mechanism in the first state, determine the spatial position information of the unknown positioning point in the motion mechanism in the second state.

[0079] in,

[0080] The second state is the running state. The transformation relationship can be pre-stored on the control side or determined in real time.

[0081] The transformation relationship can be determined as follows:

[0082] First image data and second image data are acquired, wherein the first image data is image data of the motion mechanism in a first state, and the second image data is image data of the motion mechanism in a second state.

[0083] Using the camera calibration transformation relationship of the camera model, the pixel position information of the first pixel in the first image data is converted into the first spatial point position information, and the pixel position information of the second pixel in the second image data is converted into the second spatial point position information, wherein the first pixel and the second pixel correspond to each other.

[0084] Based on the position information of the first spatial point and the position information of the second spatial point, as well as the rotation angle between the first and second spatial points, a translation vector representing the translation transformation and a rotation matrix representing the rotation transformation are determined.

[0085] Based on the translation vector and the rotation matrix, determine the homogeneous transformation matrix used to characterize the transformation relationship.

[0086] Compared with existing positioning methods based on planar point calculations, the embodiments of this application are simpler to implement, have a wider range of applications, and are more versatile.

[0087] To facilitate understanding of the embodiments of this application, the following description is based on specific application scenarios. It should be understood that this application is not limited to specific mechanism operation applications, and can be applied to the positioning of any motion mechanism.

[0088] Example 1

[0089] See Figure 4 As shown, Figure 4 This is a flowchart illustrating a positioning method for a motion mechanism in the application of a coaxial mechanism for picking up a target, according to an embodiment of this application. On the control side of the motion mechanism, the positioning method includes:

[0090] Step 401, Image Data Acquisition

[0091] When the motion mechanism is in the working state, the current image of the target is captured by the motion mechanism in the second state to obtain the second image data, where the second state is the current operating state.

[0092] For example, during the production process, the current image of the motion mechanism picking up the part or material to be picked up in its current operating state is captured.

[0093] As one implementation method, during the pre-calibration process, the motion mechanism can acquire images of the target in a set reference pose in a first state to obtain first image data. The first state is the calibration state at the set position that the motion mechanism needs to reach during movement. The reference pose can be a taught pose or a set pose, for example... Figure 2 The teaching images in the text.

[0094] All of the above images are two-dimensional images.

[0095] Step 402: Obtain the spatial point position information corresponding to the pixel based on the pixel position information in the image data.

[0096] In this step, the coordinates of the first pixel in the first image are converted to the coordinates of a reference point in the world coordinate system using a calibration matrix.

[0097] The coordinates of the second pixel in the second image are converted to the coordinates of the running point in the world coordinate system using a calibration matrix.

[0098] in,

[0099] The first pixel corresponds to the second pixel; for example, they have the same pixel position information.

[0100] The calibration matrix is ​​the camera parameter matrix, obtained through camera calibration;

[0101] The coordinates of the reference point and the coordinates of the running point are both physical space point coordinates and are in the same coordinate system.

[0102] Step 403, solve for the homogeneous transformation matrix.

[0103] Based on the principle that all transformations of spatial points or coordinate systems in 2D projective space can be represented by homogeneous linear transformations—that is, translation, rotation, shearing, scaling, etc.—of spatial points or coordinate systems can all be represented as a set of homogeneous linear transformations T—when a group of spatial points or coordinate systems satisfy the same linear transformation T, the homogeneous transformation matrix M can be solved using a known set (or more) of corresponding spatial point coordinates. Applying the homogeneous transformation matrix M to the reference coordinates of the point to be located yields the current coordinates of the point in the world coordinate system.

[0104] Homogeneous transformations include planar Euclidean transformations, which can be expressed mathematically as:

[0105]

[0106] In the formula, R is a 2×2 rotation matrix that satisfies R T R = RR T =I is an orthogonal matrix, and t is a 2D translation vector. A planar Euclidean transformation has three degrees of freedom: one rotational degree of freedom and two translational degrees of freedom. Therefore, to determine this planar Euclidean transformation, three unknown parameters must be solved.

[0107] The rotation matrix R is determined by the rotation angle θ from the reference point to the running point, and the translation vector t is determined by the 2D column vector (T). x ,T y ) T Therefore, the general form of the homogeneous transformation matrix M is:

[0108]

[0109] Define a reference point P = (x, y, 1). T The running point is P' = (x', y', 1). T Then, we have the identity that the product of the homogeneous transformation matrix and the spatial vector of the reference point is equal to the spatial vector of the running point, which can be expressed mathematically as:

[0110]

[0111] The following explains the hierarchy of homogeneous transformation matrix transformations.

[0112] Under coaxial constraints, the transformation from the reference point to the running point is a translation transformation, and the rotation matrix is ​​a 2×2 identity matrix. In this case, the homogeneous transformation matrix has 2 degrees of freedom. Therefore, a set of physical coordinates corresponding to the pixel points is needed to solve for the translation vector t, and then construct the transformation matrix M.

[0113] Under non-coaxial constraints, the transformation from the reference point to the running point is a combination of translation and rotation. In this case, the homogeneous transformation matrix has 3 degrees of freedom. Therefore, a set of corresponding point physical coordinates and a rotation angle θ are needed to solve for the rotation matrix R and the translation vector t, thus constructing the homogeneous transformation matrix M. The rotation angle θ can be obtained from the rotation angle between the first and second images.

[0114] In this embodiment, since the motion mechanism is a coaxial structure, there is only translation transformation and no rotation transformation. Therefore, only a set of corresponding reference points and running point coordinates are needed to solve for the translation vector, and then the homogeneous transformation matrix can be solved.

[0115] The rotation matrix is ​​the identity matrix.

[0116] Steps 401 to 403 above are the process of determining the homogeneous transformation matrix in real time during the positioning process. It should be understood that the above steps can also be performed during the calibration process or the teaching process. The obtained homogeneous transformation matrix can be stored. In this way, no matter what kind of target object or what kind of operation, as long as the operation pose remains unchanged, the homogeneous transformation matrix can be used for positioning.

[0117] Step 404: Using the constructed homogeneous transformation matrix, determine the positioning result of the positioning point to be located based on the spatial position information of the positioning point to be located in the motion mechanism in the first state.

[0118] After constructing the transformation matrix M, for any spatial point P of the motion mechanism in the reference pose... base =(x b ,y b ,1) T Then, its current positioning coordinates are: the result of multiplying the homogeneous transformation matrix by the spatial vector obtained by the motion mechanism in the calibration state, expressed mathematically as:

[0119]

[0120] Among them, the spatial coordinates of the motion mechanism under the calibration posture can be obtained through calibration.

[0121] For example, Figure 2 In the process, based on the spatial coordinates of the rotation center under the taught posture, the spatial coordinates of the rotation center under the running posture can be determined using the constructed homogeneous transformation matrix.

[0122] This embodiment utilizes image information from different postures to obtain a homogeneous transformation matrix. Using this matrix, and based on spatial point information from the calibration posture, it determines spatial point information from the running posture, thus obtaining the positioning result of the point to be located in the current running state. This embodiment eliminates the need to process the offset of specific spatial points, such as the rotation center, avoiding complex calculations. It is unaffected by the rotation center or target in the motion mechanism, exhibiting strong versatility and good portability.

[0123] Example 2

[0124] See Figure 5 As shown, Figure 5 This is a flowchart illustrating a positioning method for a motion mechanism in the application of a non-coaxial mechanism for picking up a target, as shown in Embodiment 2 of this application. On the control side of the motion mechanism, the positioning method includes:

[0125] Steps 501 to 502 are the same as steps 401 to 402.

[0126] Step 503, solve for the homogeneous transformation matrix.

[0127] In this embodiment, since the motion mechanism is a non-coaxial structure, the transformation from the reference point to the running point is a combination of translation and rotation. At this time, the homogeneous transformation matrix has 3 degrees of freedom. Therefore, a set of corresponding point physical coordinates and a rotation angle θ are needed to solve for the rotation matrix R and the translation vector t, thereby constructing the homogeneous transformation matrix M. The rotation angle θ can be obtained from the rotation angle between the first and second images, or from the rotation angle between the reference point and the running point.

[0128] Steps 501 to 503 above are the process of determining the homogeneous transformation matrix in real time during the positioning process. It should be understood that the above steps can also be performed during the calibration process or the teaching process. The obtained homogeneous transformation matrix can be stored. In this way, no matter what kind of target object or what kind of operation, as long as the operation pose remains unchanged, the homogeneous transformation matrix can be used for positioning.

[0129] Step 504: Using the constructed homogeneous transformation matrix, determine the positioning result of the positioning point to be located based on the spatial position information of the positioning point to be located in the motion mechanism in the first state.

[0130] In this embodiment, after constructing the transformation matrix M, for any spatial point P of the motion mechanism under the calibration posture... base =(x b ,y b ,1) T Then its current positioning coordinates satisfy:

[0131]

[0132] Among them, the spatial coordinates of the motion mechanism under the calibration posture can be obtained through calibration.

[0133] For example, Figure 1 In the figure below, the spatial coordinates of the rotation center under the teaching posture can be determined by using the constructed homogeneous transformation matrix; similarly, the spatial coordinates of the grasping point under the teaching posture can be determined by using the constructed homogeneous transformation matrix.

[0134] Compared to traditional positioning methods, this embodiment does not require determining the translational offset and rotational deviation of specific operating points in the motion mechanism, nor does it require calculating the rotation of a spatial point around another spatial point, which greatly reduces the computational complexity of positioning and has portability and versatility.

[0135] The basic principles of the above embodiments one and two can be described as follows: Figure 6As shown. During the calibration or teaching process, a reference image is acquired, and during the production process, the current image is acquired. Using the calibration transformation relationship, the pixel position information in the reference image is converted into reference point position information, and the pixels in the current image are converted into running point position information. Using the reference point position information and the running point position information, the homogeneous transformation matrix is ​​solved. Using the homogeneous transformation matrix, the positioning result of the unknown positioning point in the motion structure is obtained.

[0136] Example 3

[0137] See Figure 7 As shown, Figure 7 This is a flowchart illustrating a positioning method for a motion mechanism in the application of the motion mechanism assembling a target, as described in Embodiment 3 of this application. Taking the assembly of a first target to a second target as an example, the positioning method on the control side of the motion mechanism includes:

[0138] Step 601, Image Data Acquisition,

[0139] When the motion mechanism is in working state, the current image of the motion mechanism operating the first target in the first operating state is acquired to obtain the first image data, and the current image of the motion mechanism operating the second target in the second operating state is acquired to obtain the second image data.

[0140] The first target includes a target for assembly, the second target includes a target assembled by the first target, operating the first target in the first operating state includes a picking operation of picking up the first target to be picked up, and operating the first target in the first operating state includes an assembly operation of assembling the picked-up first target with the second target.

[0141] As an example, such as Figure 8 As shown, when a circular part is assembled into the countersunk hole of a rectangular part, the current image of the circular part is acquired as the first image data when the motion mechanism is in the first operating state, and the current image of the rectangular part is acquired as the second image data when the motion mechanism is in the second operating state.

[0142] Step 602: Obtain the spatial point position information corresponding to the pixel based on the pixel position information in the image data.

[0143] The coordinates of the second pixel in the second image are converted into the coordinates of the second spatial point in the world coordinate system using a calibration matrix.

[0144] The coordinates of the first pixel in the first image are converted into the coordinates of the first spatial point in the world coordinate system using a calibration matrix.

[0145] in,

[0146] The first pixel corresponds to the second pixel. For example, the first pixel is the center pixel of the circular part in the first image data, and the second pixel is the center pixel of the assembly position of the circular part in the second image data, such as the center pixel of the countersunk hole in the rectangular part.

[0147] The calibration matrix is ​​the camera parameter matrix, obtained through camera calibration.

[0148] Step 603, solve for the homogeneous transformation matrix.

[0149] By utilizing the identity that the product of the homogeneous transformation matrix and the spatial vector of the first spatial point equals the spatial vector of the second spatial point, and based on the position information of the first and second spatial points, as well as the rotation angle between the first and second spatial points, the rotation matrix and translation vector can be solved.

[0150] In this embodiment, if the motion mechanism is a coaxial mechanism, the rotation angle between the first and second spatial points is 0. Therefore, the translation vector can be solved based solely on the position information of the first and second spatial points, and the rotation matrix can be set as the identity matrix. If the motion mechanism is a non-coaxial mechanism, the translation vector and rotation matrix are solved based on the position information of the first and second spatial points, as well as the rotation angle between them. Then, the homogeneous transformation matrix is ​​determined based on the rotation matrix and the translation vector.

[0151] Steps 601 to 603 above are the process of determining the homogeneous transformation matrix in real time during the positioning process. It should be understood that the above steps can also be performed during the calibration process or the teaching process. The obtained homogeneous transformation matrix can be stored. In this way, no matter what kind of target object or what kind of operation, as long as the operation pose remains unchanged, the homogeneous transformation matrix can be used for positioning.

[0152] Step 604: Using the constructed homogeneous transformation matrix, determine the positioning result of the positioning point to be located in the motion mechanism in the first state based on the spatial position information of the positioning point to be located.

[0153] In this embodiment, the operation of the motion mechanism on the first target in the first operating state is regarded as the teaching pose. For example, the motion mechanism posture of picking up the first target is determined as the teaching pose. For any spatial point coordinate of the motion mechanism in the first operating state, the current operating positioning result of the motion mechanism assembling the first target onto the second target is determined using a homogeneous transformation matrix. For example, Figure 8 The spatial location information of the grasping point is obtained. Among them, the spatial coordinates of the motion mechanism in the first operating state can be obtained by pre-calibration.

[0154] In this embodiment, during the assembly operation of the target part from the first operating state to the second operating state, the homogeneous transformation matrix is ​​solved using image information from both the first and second operating states. The spatial position information of the undetermined positioning point in the moving component is then determined using the homogeneous transformation matrix. This embodiment avoids complex calculations and has strong portability and versatility.

[0155] See Figure 9 As shown, Figure 9 This is a schematic diagram of a positioning device for a motion mechanism according to an embodiment of this application. The positioning device includes:

[0156] The acquisition module is used to acquire transformation relationships, wherein the transformation relationships characterize the translational and rotational transformations between the motion mechanism in the first state and the motion mechanism in the second state.

[0157] The determining module is used to determine the spatial position information of the point to be located in the motion mechanism in the second state, based on the spatial position information of the point to be located in the motion mechanism in the first state, using the transformation relationship.

[0158] in,

[0159] The second state is the running state.

[0160] The acquisition module further includes:

[0161] The image acquisition submodule is used to acquire first image data and second image data, wherein the first image data is image data of the motion mechanism in the first state, and the second image data is image data of the motion mechanism in the second state.

[0162] The transformation relationship calculation submodule uses the camera calibration transformation relationship of the camera model to convert the pixel position information of the first pixel in the first image data into the first spatial point position information, and converts the pixel position information of the second pixel in the second image data into the second spatial point position information, wherein the first pixel and the second pixel correspond to each other.

[0163] Based on the position information of the first spatial point and the position information of the second spatial point, as well as the rotation angle between the first and second spatial points, a translation vector representing the translation transformation and a rotation matrix representing the rotation transformation are determined.

[0164] Based on the translation vector and the rotation matrix, determine the homogeneous transformation matrix used to characterize the transformation relationship.

[0165] See Figure 10 As shown, Figure 10This is another schematic diagram of the positioning device for the motion mechanism according to an embodiment of this application. The positioning device includes a processor, a memory, and an image acquisition device. The memory stores a computer program, and the processor executes the computer program to implement the positioning method for the motion mechanism described in this embodiment of the application.

[0166] The motion mechanism can be a robot, or a combination of mechanisms such as a robotic arm.

[0167] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0168] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0169] This invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the positioning method of the motion mechanism described in this application.

[0170] For the device / network-side equipment / storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and relevant parts can be referred to in the description of the method embodiments.

[0171] In this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, without necessarily requiring or implying any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0172] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A positioning method for a motion mechanism, characterized in that, The method includes: on the control side used for controlling the motion mechanism, Obtain the transformation relationship, wherein the transformation relationship is used to characterize the translational and rotational transformations between the motion mechanism in the first state and the motion mechanism in the second state. Using the transformation relationship, and based on the spatial position information of the point to be located in the motion mechanism in the first state, the spatial position information of the point to be located in the motion mechanism in the second state is determined. in, The second state is the running state. The transformation relationship is determined as follows: First image data and second image data are acquired, wherein the first image data is image data of the motion mechanism in the first state, and the second image data is image data of the motion mechanism in the second state. Using the camera calibration transformation relationship of the camera model, the pixel position information of the first pixel in the first image data is converted into the first spatial point position information, and the pixel position information of the second pixel in the second image data is converted into the second spatial point position information. The first pixel and the second pixel correspond to each other, and the first and second spatial point position information are in the same coordinate system. Based on the position information of the first spatial point and the position information of the second spatial point, as well as the rotation angle between the first spatial point and the second spatial point, a homogeneous transformation matrix is ​​determined to characterize the transformation relationship. The homogeneous transformation matrix includes: a translation vector for characterizing translation transformation and a rotation matrix for characterizing rotation transformation. Under coaxial constraints, the rotation matrix is ​​an identity matrix. Under non-coaxial constraints, the rotation angle in the rotation matrix is ​​the rotation angle between the first image and the second image. The spatial location information of the point to be located is determined in the following manner: Based on the homogeneous transformation matrix and the spatial vector obtained by the motion mechanism under calibration, the positioning information of the target positioning point under the current operating state is determined.

2. The positioning method as described in claim 1, characterized in that, The point to be located includes the rotation center in the motion mechanism.

3. The positioning method as described in claim 2, characterized in that, The first state is the calibration state when the motion mechanism needs to reach a set position during motion; The second state is the current running state, and the acquisition of the first image data and the second image data includes: In the current operating state of the motion mechanism, a current image of the motion mechanism operating the first target in the current operating state is acquired to obtain first image data. In the calibration state of the motion mechanism, a current image is acquired when the motion mechanism operates the second target to a set reference pose in the calibration state, thus obtaining second image data. The first target and the second target are the same target.

4. The positioning method as described in claim 3, characterized in that, The step of using the camera calibration transformation relationship of the camera model to convert the pixel position information of the first pixel in the first image data into the first spatial point position information, and converting the pixel position information of the second pixel in the second image data into the second spatial point position information, includes: Using the camera parameter matrix obtained from camera calibration, the pixel position information of the first pixel in the first image data is converted into reference point position information, and the pixel position information of the second pixel in the second image data is converted into running point position information. The first pixel and the second pixel have the same pixel position information.

5. The positioning method as described in claim 4, characterized in that, The motion mechanism is a coaxial mechanism, in which the rotation center of the operating part located at the end of the mechanism coincides with the rotation center of the actuator used to control the end of the mechanism. The step of determining the homogeneous transformation matrix representing the transformation relationship based on the position information of the first spatial point and the position information of the second spatial point, as well as the rotation angle between the first and second spatial points, includes: By utilizing the identity that the product of the homogeneous transformation matrix and the spatial vector of the reference point equals the spatial vector of the running point, and based on the position information of the running point and the reference point, the translation vector can be solved. Set the rotation matrix to the identity matrix.

6. The positioning method as described in claim 4, characterized in that, The motion mechanism is a non-coaxial mechanism, in which the rotation center of the operating part located at the end of the mechanism does not coincide with the rotation center of the actuator used to control the end of the mechanism. The step of determining the homogeneous transformation matrix representing the transformation relationship based on the position information of the first spatial point and the position information of the second spatial point, as well as the rotation angle between the first and second spatial points, includes: By utilizing the identity that the product of the homogeneous transformation matrix and the spatial vector of the reference point equals the spatial vector of the running point, and based on the position information of the running point and the reference point, as well as the rotation angle between the running point and the reference point, the rotation matrix and translation vector can be solved.

7. The positioning method as described in any one of claims 3 to 6, characterized in that, The first objective of the operation includes a pickup operation to pick up the target to be picked up, and the second objective of the operation includes a pickup operation to pick up the target to be picked up. The point to be located is the gripping point at the end of the motion mechanism. The step of using the transformation relationship to determine the spatial position information of the target positioning point in the motion mechanism in the second state, based on the spatial position information of the target positioning point in the motion mechanism in the first state, includes: The homogeneous transformation matrix is ​​multiplied by the spatial vector of the gripping point obtained by the motion mechanism in the calibration state to obtain the positioning information of the gripping point in the current operating state.

8. The positioning method as described in claim 2, characterized in that, The first state is the first operating state when the motion mechanism needs to reach the first set position during motion; The second state is the second operating state when the motion mechanism reaches the second set position during motion; The acquisition of the first image data and the second image data includes: In the first operating state, the current image of the motion mechanism operating the first target is acquired to obtain first image data. In the second operating state, the current image of the motion mechanism operating the second target is acquired to obtain second image data. Wherein, the first target is the target used for assembly, and the second target is the target assembled by the first target.

9. The positioning method as described in claim 8, characterized in that, The homogeneous transformation matrix used to characterize the transformation relationship, based on the position information of the first spatial point and the position information of the second spatial point, and the rotation angle between the first and second spatial points, includes: By utilizing the identity that the product of the homogeneous transformation matrix and the spatial vector of the first spatial point equals the spatial vector of the second spatial point, and based on the position information of the first and second spatial points, as well as the rotation angle between the first and second spatial points, the rotation matrix and translation vector can be solved.

10. The positioning method as described in claim 8 or 9, characterized in that, The first objective of the operation includes a picking operation of picking up a first target to be picked up, and the second objective of the operation includes an assembly operation of assembling the picked-up first target with a second target. The positioning point to be determined includes the gripping point located at the end of the motion mechanism. The step of using the transformation relationship to determine the spatial position information of the target positioning point in the motion mechanism in the second state, based on the spatial position information of the target positioning point in the motion mechanism in the first state, includes: Multiply the homogeneous transformation matrix with the grasping point space vector obtained by the motion mechanism in the first operating state to obtain the positioning information of the grasping point in the second operating state; in, The spatial vector of the grasping point is obtained in advance through calibration.

11. A positioning device for a motion mechanism, characterized in that, The device includes: The acquisition module is used to acquire transformation relationships, wherein the transformation relationships characterize the translational and rotational transformations between the motion mechanism in the first state and the motion mechanism in the second state. The determining module is used to determine the spatial position information of the point to be located in the motion mechanism in the second state, based on the spatial position information of the point to be located in the motion mechanism in the first state, using the transformation relationship. in, The second state is the running state; The transformation relationship is determined as follows: First image data and second image data are acquired, wherein the first image data is image data of the motion mechanism in the first state, and the second image data is image data of the motion mechanism in the second state. Using the camera calibration transformation relationship of the camera model, the pixel position information of the first pixel in the first image data is converted into the first spatial point position information, and the pixel position information of the second pixel in the second image data is converted into the second spatial point position information. The first pixel and the second pixel correspond to each other, and the first and second spatial point position information are in the same coordinate system. Based on the position information of the first spatial point and the position information of the second spatial point, as well as the rotation angle between the first spatial point and the second spatial point, a homogeneous transformation matrix is ​​determined to characterize the transformation relationship. The homogeneous transformation matrix includes: a translation vector for characterizing translation transformation and a rotation matrix for characterizing rotation transformation. Under coaxial constraints, the rotation matrix is ​​an identity matrix. Under non-coaxial constraints, the rotation angle in the rotation matrix is ​​the rotation angle between the first image and the second image. The spatial location information of the point to be located is determined in the following manner: Based on the homogeneous transformation matrix and the spatial vector obtained by the motion mechanism under calibration, the positioning information of the target positioning point under the current operating state is determined.

Images