A method, apparatus, electronic device, and storage medium for controlling a motion mechanism.

Abstract

This application provides a motion mechanism control method, device, electronic device, and storage medium, relating to the field of robotics. The specific solution is as follows: Obtain the current image pose in the target object image; determine the rotation angle difference between the angle in the current image pose and the angle in the reference image pose; based on the position in the current image pose, the position in the reference image pose, the rotation angle difference, the center position of the motion mechanism's rotation axis, and the center fitting deviation of the rotation axis, obtain the position offset of the motion mechanism moving towards the target object. The center fitting deviation is obtained based on the reference image pose, the angles included in the running image pose of the auxiliary object after rotation relative to the reference position in the image acquisition device, and the re-examination image position of the auxiliary object; control the motion mechanism to move the position offset towards the target object's location. Applying the solution provided in this application can improve the accuracy of controlling the movement of the motion mechanism.

Description

Technical Field

[0001] This application relates to the field of robotics, and in particular to a method, apparatus, electronic device, and storage medium for controlling a motion mechanism. Background Technology

[0002] With the maturity of robotics technology, more and more industrial production scenarios are introducing motion mechanisms such as robotic arms and robotic hands. These motion mechanisms move to the location of workpieces and other objects based on visual positioning results, then grasp the objects and install or place them in designated locations.

[0003] To ensure that the motion mechanism can accurately grasp the aforementioned object, in related technologies, after obtaining the image pose of the object in the image captured by the image acquisition device, the pose of the object in the physical coordinate system of the motion mechanism is determined based on the image pose, which is then used as the running pose of the motion mechanism. Then, assuming that the motion mechanism performs rigid body motion, it is controlled to move from the current pose to the aforementioned running pose.

[0004] However, since the rotation axis of the motion mechanism and the center of mass of the end effector are not perfectly concentric, the rotation of the motion mechanism's rotation axis will cause the end effector to deviate. If the motion mechanism continues to move in the above-mentioned operating posture, the end effector of the motion mechanism will find it difficult to accurately move to the position of the object to be grasped, resulting in low precision in controlling the movement of the motion mechanism, or even failure of the motion mechanism to grasp the object. Summary of the Invention

[0005] The purpose of this application is to provide a motion mechanism control method, device, electronic device, and storage medium to improve the accuracy of controlling the movement of the motion mechanism. The specific technical solution is as follows:

[0006] According to one aspect of the embodiments of this application, a motion mechanism control method is provided, the method comprising:

[0007] Obtain the current image pose of the target object to be grasped by the motion mechanism in the image captured by the image acquisition device;

[0008] Determine the rotation angle difference between the angle in the current image pose and the angle in the reference image pose, wherein the reference image pose is the pose of the object in the image acquired by the image acquisition device when the object is located at the reference position;

[0009] Based on the position in the current image pose, the position in the reference image pose, the rotation angle difference, the center position of the rotation axis of the motion mechanism, and the center fitting deviation of the rotation axis, the position offset of the motion mechanism moving towards the target object is obtained. The center fitting deviation is based on the reference image pose, the angles included in the running image pose in the image acquired by the image acquisition device after the auxiliary object rotates relative to the reference position, and the re-examination image position of the auxiliary object. The re-examination image position is the position of the auxiliary object in the image acquired by the image acquisition device after the motion mechanism carries the auxiliary object's movement pose offset. The pose offset is the offset from the running physical pose to the reference physical pose. The running physical pose is the pose corresponding to the running image pose in the physical space where the motion mechanism is located. The reference physical pose is the pose corresponding to the reference image pose in the physical space. The auxiliary object rotates at least twice relative to the reference position.

[0010] Based on the position offset, the motion mechanism is controlled to move towards the location of the target object.

[0011] In one embodiment of this application, the angular offset in the pose offset is obtained in the following manner:

[0012] Based on the correspondence between image angles and physical angles, the first physical angle corresponding to the angle in the pose of the running image is obtained, wherein the image angle is the angle of the object in the image acquired by the image acquisition device, and the physical angle is the angle of the object in the physical space.

[0013] Based on the correspondence, the second physical angle corresponding to the angle in the pose of the reference image is obtained;

[0014] Calculate the difference between the second physical angle and the first physical angle, and use it as the angle offset.

[0015] In one embodiment of this application, if the auxiliary object rotates more than twice relative to the reference position, the center fitting deviation is obtained as follows:

[0016] For every two rotations, based on the reference image pose, the angles included in the running image pose of the auxiliary object after rotation relative to the reference position in the image acquired by the image acquisition device, and the re-examination image position of the auxiliary object, a set of parameter expressions for the center fitting deviation are constructed.

[0017] The central fitting deviation is obtained by solving the constructed parameter expression relationship based on the least squares method.

[0018] In one embodiment of this application, the center fitting deviation is calculated based on the following expression:

[0019]

[0020] Where Δx and Δy are the deviations along the x-axis and y-axis of the physical coordinate system in the physical space, respectively, in the center fitting deviation; Δθ1 and Δθ2 are the auxiliary angle differences between the angles in the running image pose and the angles in the reference image pose of the auxiliary object during the two rotations; Δx error1 and Δy error1 Δx is the positional difference between the physical position of the auxiliary object after the first rotation and the reference physical position during the two rotations. error2 and Δy error2 The physical position difference between the physical position of the auxiliary object after the second rotation and the reference physical position is the position of the physical image in the physical space, and the reference physical position is the position information in the reference physical pose.

[0021] In one embodiment of this application, the method further includes:

[0022] Based on the correspondence between image angles and physical angles, a third physical angle corresponding to the angle in the reference image pose is obtained, wherein the image angle is the angle of the object in the image acquired by the image acquisition device, and the physical angle is the angle of the object in the physical space.

[0023] Based on the correspondence, the fourth physical angle corresponding to the angle in the current image pose is obtained;

[0024] Calculate the angle difference between the fourth physical angle and the third physical angle;

[0025] The step of controlling the motion mechanism to move towards the location of the object based on the position offset includes:

[0026] Based on the position offset and the angle difference, the motion mechanism is controlled to move towards the location of the object.

[0027] In one embodiment of this application, the correspondence is generated in the following manner:

[0028] The reference image angle of the reference object in the image acquired by the image acquisition device after the motion mechanism carries the reference object to rotate and the reference physical angle of the motion mechanism are obtained, wherein the number of rotations of the reference object carried by the motion mechanism is greater than 2;

[0029] Based on the rotation sequence, and using the reference image angle and reference physical angle obtained after two adjacent rotations, a piecewise linear relationship between the image angle and the physical angle is constructed.

[0030] According to another aspect of the embodiments of this application, a motion mechanism control device is provided, the device comprising:

[0031] The current image pose acquisition module is used to obtain the current image pose of the target object to be grasped by the motion mechanism in the image acquired by the image acquisition device;

[0032] The rotation angle difference determination module is used to determine the rotation angle difference between the angle in the current image pose and the angle in the reference image pose, wherein the reference image pose is the pose of the object in the image acquired by the image acquisition device when the object is located at the reference position.

[0033] The position offset acquisition module is used to obtain the position offset of the motion mechanism moving towards the target object based on the position in the current image pose, the position in the reference image pose, the rotation angle difference, the center position of the rotation axis of the motion mechanism, and the center fitting deviation of the rotation axis. The center fitting deviation is obtained based on the reference image pose, the angle included in the running image pose in the image acquired by the image acquisition device after the auxiliary object rotates relative to the reference position, and the re-examination image position of the auxiliary object. The re-examination image position is the position of the auxiliary object in the image acquired by the image acquisition device after the motion mechanism carries the auxiliary object with the movement pose offset. The pose offset is the offset from the running physical pose to the reference physical pose. The running physical pose is the pose corresponding to the running image pose in the physical space where the motion mechanism is located. The reference physical pose is the pose corresponding to the reference image pose in the physical space. The auxiliary object rotates at least twice relative to the reference position.

[0034] A motion mechanism control module is used to control the motion mechanism to move towards the location of the target object based on the position offset.

[0035] In one embodiment of this application, the angle offset in the pose offset is obtained as follows: based on the correspondence between image angles and physical angles, a first physical angle corresponding to the angle in the running image pose is obtained, wherein the image angle is the angle of the object in the image acquired by the image acquisition device, and the physical angle is the angle of the object in the physical space; based on the correspondence, a second physical angle corresponding to the angle in the reference image pose is obtained; the difference between the second physical angle and the first physical angle is calculated as the angle offset;

[0036] In one embodiment of this application, if the auxiliary object rotates more than twice relative to the reference position, the center fitting deviation is obtained as follows: for each two rotations, based on the reference image pose, the angles included in the running image pose of the auxiliary object after rotation relative to the reference position in the image acquired by the image acquisition device, and the re-examination image position of the auxiliary object, a set of parameter expressions for the center fitting deviation is constructed; the center fitting deviation is obtained by solving the constructed parameter expressions based on the least squares method.

[0037] In one embodiment of this application, the center fitting deviation is calculated based on the following expression:

[0038]

[0039] Where Δx and Δy are the deviations along the x-axis and y-axis of the physical coordinate system in the physical space, respectively, in the center fitting deviation; Δθ1 and Δθ2 are the auxiliary angle differences between the angles in the running image pose and the angles in the reference image pose of the auxiliary object during the two rotations; Δx error1 and Δy error1 Δx is the positional difference between the physical position of the auxiliary object after the first rotation and the reference physical position during the two rotations. error2 and Δy error2 The physical position difference between the re-inspection physical position and the reference physical position after the second rotation of the auxiliary object in the two rotations, wherein the re-inspection physical position is the position of the re-inspection image in the physical space, and the reference physical position is the position information in the reference physical pose;

[0040] In one embodiment of this application, the device further includes: a third physical angle acquisition module, configured to obtain a third physical angle corresponding to the angle in the reference image pose based on the correspondence between image angles and physical angles, wherein the image angle is the angle of the object in the image acquired by the image acquisition device, and the physical angle is the angle of the object in the physical space; a fourth physical angle acquisition module, configured to obtain a fourth physical angle corresponding to the angle in the current image pose based on the correspondence; an angle difference calculation module, configured to calculate the angle difference between the fourth physical angle and the third physical angle; and a motion mechanism control module, specifically configured to control the motion mechanism to move towards the location of the object based on the position offset and the angle difference;

[0041] In one embodiment of this application, the correspondence is generated as follows: obtaining the reference image angle of the reference object in the image acquired by the image acquisition device after the motion mechanism carries the reference object and the reference physical angle of the motion mechanism, wherein the number of rotations of the motion mechanism carrying the reference object is greater than 2; according to the rotation order, based on the reference image angle and reference physical angle obtained after two adjacent rotations, constructing a piecewise linear relationship between the image angle and the physical angle.

[0042] According to another aspect of the embodiments of this application, an electronic device is provided, comprising:

[0043] Memory, used to store computer programs;

[0044] The processor, when executing a program stored in memory, implements any of the motion mechanism control methods described above.

[0045] According to another aspect of the embodiments of this application, a computer-readable storage medium is provided, wherein a computer program is stored therein, and when the computer program is executed by a processor, it implements any of the motion mechanism control methods described above.

[0046] According to another aspect of the embodiments of this application, a computer program product containing instructions is provided, which, when run on a computer, causes the computer to perform any of the motion mechanism control methods described above.

[0047] Beneficial effects of the embodiments in this application:

[0048] As can be seen from the above, in the solution provided by the embodiments of this application, when obtaining the position offset for controlling the motion mechanism to move to the location of the object, in addition to considering the position in the current image pose, the position in the reference image pose, and the difference in rotation angle, the center position of the fitted motion mechanism rotation axis and the center fitting deviation of the actual rotation axis are also considered. The center fitting deviation is obtained based on the running image pose, the re-inspection image position, and the reference image pose obtained by rotating the auxiliary object. The obtained center fitting deviation is more accurate. In this way, the center position of the fitted motion mechanism rotation axis can be corrected more accurately, and thus the accurate position offset can be obtained based on the corrected rotation axis center position, thereby improving the accuracy of controlling the movement of the motion mechanism.

[0049] Of course, implementing any product or method of this application does not necessarily require achieving all of the advantages described above at the same time. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0051] Figure 1 A schematic diagram illustrating the movement error of a motion mechanism provided in an embodiment of this application;

[0052] Figure 2 A flowchart illustrating a motion mechanism control method provided in an embodiment of this application;

[0053] Figure 3 A flowchart illustrating another motion mechanism control method provided in this application embodiment;

[0054] Figure 4 This is a schematic diagram of the structure of a motion mechanism control device provided in an embodiment of this application;

[0055] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0056] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art based on this application are within the scope of protection of this application.

[0057] The background technology of the embodiments of this application will be described below.

[0058] Because the rotation axis of the motion mechanism and the centroid of the end effector of the motion mechanism are difficult to be perfectly concentric, controlling the motion mechanism based on the center of the rotation axis of the motion mechanism to move according to the pose of the object in the physical coordinate system of the motion mechanism will make it difficult for the end effector of the motion mechanism to move accurately to the position of the object to be grasped.

[0059] For example, see Figure 1 A schematic diagram of the movement error of a motion mechanism is provided. Figure 1 In the diagram, point P is the actual position of the object in the physical coordinate system of the motion mechanism, point P' is the position reached after the motion mechanism rotates, and point C is the intersection of line segment OP' and the circle with radius OP. The length of line segment PP' is the overall rotational deviation of the motion mechanism.

[0060] In reality, the rotation axis of a true motion mechanism is a line segment OP, but theoretically it is considered to be a line segment OP'. Therefore, the length difference between line segment OP and line segment OP' represents the deviation of the center position of the rotation axis. Figure 1 The length of the midline segment CP'.

[0061] Furthermore, due to the influence of the manufacturing process and motion parameters, the motion mechanism cannot be rotated to a fixed angle with extremely precise accuracy. This results in the motion mechanism exhibiting issues such as… Figure 1 The rotation angle error ΔR between OP and OP' is the rotation error of the rotating shaft of the motion mechanism, which is the length of line segment CP.

[0062] Therefore, more precise control of the movement of motion mechanisms is required. This application provides a motion mechanism control method to improve the accuracy of controlling the movement of motion mechanisms.

[0063] The execution subject of the embodiments of this application will be described below.

[0064] The solutions provided in this application can be applied to electronic devices such as desktop computers, laptops, tablets, and servers. These electronic devices can control a robotic arm to move. Alternatively, the executing entity of the solutions provided in this application can also be a robotic arm. For ease of description, the executing entity of the motion mechanism control method provided in this application is collectively referred to as a control device.

[0065] The motion mechanism control method provided in this application will be described below with reference to specific embodiments.

[0066] In one embodiment of this application, see [link to embodiment]. Figure 2 A flowchart of a motion mechanism control method is provided, the method including the following steps S201-S204.

[0067] Step S201: Obtain the current image pose of the target object to be grasped by the motion mechanism in the image captured by the image acquisition device.

[0068] The image acquisition device can be installed in a fixed location in the surrounding environment of the motion mechanism, or it can be installed on the robotic arm.

[0069] In one implementation, after the image acquisition device acquires an image of the target object to be captured, the control device performs target recognition on the image of the target object to be captured based on a preset template of the target object to be captured, identifies the position and angle of the target object in the image, and uses the obtained position and angle as the current image pose of the target object to be captured.

[0070] Step S202: Determine the rotation angle difference between the angle in the current image pose and the angle in the reference image pose.

[0071] The reference image pose is the pose of the object in the image acquired by the image acquisition device when the object is located at the reference position.

[0072] The method for obtaining the pose of the reference image is explained below.

[0073] Specifically, the motion mechanism can be controlled to grasp an object placed in a reference pose. Then, the installed image acquisition device can obtain an image of the object grasped by the motion mechanism. The pose of the object on the motion mechanism in the image can be identified to obtain the reference image pose.

[0074] The method for determining the rotation angle difference in step S202 will be explained below.

[0075] In one implementation, the control device can calculate the difference between the angle in the recorded reference image pose and the angle in the current image pose, as the rotation angle difference.

[0076] Step S203: Based on the position in the current image pose, the position in the reference image pose, the rotation angle difference, the center position of the rotation axis of the motion mechanism, and the center fitting deviation of the rotation axis, obtain the position offset of the motion mechanism moving towards the target object.

[0077] The center fitting deviation is derived from the pose of the reference image, the angles included in the pose of the running image captured by the image acquisition device after the auxiliary object rotates relative to the reference position, and the position of the auxiliary object in the re-examination image. Furthermore, the center fitting deviation is the deviation between the center position of the motion mechanism's rotation axis obtained through fitting and the actual center position of the rotation axis. In other words, after obtaining the center position of the motion mechanism's rotation axis, adding the center fitting deviation to the aforementioned center position can correct the center position of the rotation axis.

[0078] The re-inspection image position is: the position of the auxiliary object in the image acquired by the image acquisition device after the motion mechanism carries the auxiliary object to move by a pose offset. The pose offset is: the offset from the running physical pose to the reference physical pose. The running physical pose is: the pose corresponding to the running image pose in the physical space where the motion mechanism is located. The reference physical pose is: the pose corresponding to the reference image pose in the physical space. The auxiliary object rotates at least twice relative to the reference position.

[0079] The following explains how the position offset is obtained in step S203.

[0080] In one implementation, the control device determines the reference physical coordinates of the position in the reference image pose in the physical coordinate system based on the calibration matrix and the position in the reference image pose. Based on the calibration matrix, the position in the current image pose, the rotation angle difference, the center position of the motion mechanism's rotation axis, and the center fitting deviation, the control device determines the running physical coordinates of the position in the current image pose in the physical coordinate system. Then, the control device uses the coordinate difference between the determined running physical coordinates and the determined reference physical coordinates as the position offset by which the motion mechanism moves towards the target object.

[0081] Specifically, the position offset can be obtained using the following expression:

[0082]

[0083] Among them, (x base y base ) represents the position information of the reference image pose in the pixel coordinate system, Δθ is the rotation angle difference, and M HAE For the calibration matrix, (x cur y cur (x) represents the position information of the current image pose in the pixel coordinate system. c y c ) represents the coordinates of the center position of the rotation axis of the motion mechanism in the physical coordinate system, Δx represents the deviation of the center fitting deviation on the x-axis in the physical coordinate system, and Δy represents the deviation of the center fitting deviation on the y-axis in the physical coordinate system.

[0084] The calibration matrix represents the transformation relationship between the pixel coordinate system of the image acquisition device and the physical coordinate system of the motion mechanism, where the physical coordinate system is the coordinate system corresponding to the physical space where the motion mechanism is located.

[0085] The methods for obtaining the center position of the rotation axis of the motion mechanism and the center fitting deviation are described in the following embodiments, and will not be detailed here.

[0086] Step S204: Based on the position offset, control the motion mechanism to move to the location of the object.

[0087] Before moving to the object's current image pose, the motion mechanism can be placed in its home pose or zero pose. During the movement of the motion mechanism towards the object's location, it can be controlled to move from its home pose or zero pose towards the object's location.

[0088] Specifically, the control device can control the operating mechanism to translate the aforementioned position offset distance from the position corresponding to the reference image pose in physical space to the position corresponding to the current image pose in physical space. For example, in step S203, the position offset along the first axis and the offset along the second axis in the physical coordinate system can be obtained, and the control device can control the operating mechanism to translate the aforementioned first axis offset along the direction of the first axis in the physical coordinate system and translate the aforementioned second axis offset along the direction of the second axis in the physical coordinate system, respectively.

[0089] As can be seen from the above, in the solution provided by the embodiments of this application, when obtaining the position offset for controlling the motion mechanism to move to the location of the object, in addition to considering the position in the current image pose, the position in the reference image pose, and the difference in rotation angle, the center position of the fitted motion mechanism rotation axis and the center fitting deviation of the actual rotation axis are also considered. The center fitting deviation is obtained based on the running image pose, the re-inspection image position, and the reference image pose obtained by rotating the auxiliary object. The obtained center fitting deviation is more accurate. In this way, the center position of the fitted motion mechanism rotation axis can be corrected more accurately, and thus the accurate position offset can be obtained based on the corrected rotation axis center position, thereby improving the accuracy of controlling the movement of the motion mechanism.

[0090] In this way, when controlling the motion mechanism to move to the target location, based on a more accurate rotation axis center position, the calculation deviation of the angle difference of the motion mechanism rotating around the rotation axis center position can be reduced, and the accuracy of the motion mechanism movement can also be improved when rotational movement is involved.

[0091] The following explains how to obtain the center-fit bias.

[0092] The following explains the method of obtaining the position of the re-inspection image corresponding to the rotation of the auxiliary object relative to the reference position, and the auxiliary angle difference between the angle in the running image pose and the angle in the reference image pose.

[0093] Specifically, the auxiliary object can be rotated from its physical position corresponding to the reference image pose, either manually or by controlling a robotic arm. The position of the rotated auxiliary object in the image acquired by the image acquisition device is then located and identified as the running image pose. Next, the running physical pose corresponding to the running image pose is calculated. Based on the pre-recorded reference physical pose, the pose offset from the running physical pose to the reference physical pose is calculated. After obtaining the pose offset, the motion mechanism is controlled to move the object from the running physical pose to the reference physical pose by the aforementioned pose offset. Furthermore, the difference between the angles in the obtained running image pose and the angles in the reference image pose is recorded as the auxiliary angle difference.

[0094] Due to system translation and rotation errors, although the pose offset is the difference between the running physical pose and the reference physical pose, after the motion mechanism moves from the running physical pose to the reference physical pose by this pose offset, it cannot actually return to the reference physical pose. Therefore, after the motion mechanism moves the auxiliary object by this pose offset, the image acquisition device re-acquires an image of the auxiliary object and locates and identifies its position in the image to obtain the position of the re-examined image. For example, the system translation error is the repeatability error and absolute positioning error of the motion mechanism, and the system rotation error is the difference between the running physical pose and the reference physical pose. Figure 1 The length of PP' in the equation.

[0095] Then, the auxiliary object is repositioned to the reference physical pose, and the process of obtaining the re-inspection image position corresponding to this rotation and the auxiliary angle difference between the angle in the image pose and the angle in the reference image pose is repeated, so that the auxiliary object is rotated at least twice relative to the reference position, and at least two sets of re-inspection image positions and auxiliary angle differences are obtained.

[0096] Furthermore, regarding the angles at which the auxiliary object rotates twice relative to the reference position, one can select the upper and lower limits of the rotation angle range of the running mechanism, or one can select the upper and lower limits required for the running mechanism to grasp the auxiliary object. Alternatively, the angles at which the auxiliary object rotates twice relative to the reference position should be greater than a preset angle difference. This reduces the impact of errors such as acquisition errors and rounding errors, improving the accuracy and stability of the calculated center fitting deviation. Especially in application scenarios where the motion mechanism performs disordered grasping and has a large rotation angle, it can also improve the accuracy of controlling the movement of the motion mechanism.

[0097] Furthermore, the center fit bias is calculated based on the following expression:

[0098]

[0099] Where Δx and Δy are the deviations along the x-axis and y-axis of the physical coordinate system in physical space, respectively, in the center fitting bias; Δθ1 and Δθ2 are the auxiliary angle differences between the angles in the auxiliary object's pose in the running image during the two rotations and the angles in the reference image pose, respectively. error1 and Δy error1 Δx represents the positional difference between the physical position of the auxiliary object after the first rotation and the reference physical position during the two rotations. error2 and Δy error2To assist in the calculation of the position difference between the re-inspection physical position and the reference physical position after the second rotation of the auxiliary object, the re-inspection physical position is the position corresponding to the re-inspection image position in physical space, and the reference physical position is the position information in the reference physical pose. In this way, the center fitting deviation can be accurately calculated, and thus the position offset between the current physical position corresponding to the current image pose and the reference physical position corresponding to the reference image pose can be obtained, improving the accuracy of controlling the movement of the motion mechanism.

[0100] Assuming the auxiliary object undergoes two rotations, after rotating the auxiliary object relative to the reference position, we can obtain Δθ1 and Δx. error1 and Δy error1 This set of auxiliary rotation results, after the auxiliary object is rotated again relative to the reference position, yields Δθ2 and Δx. error2 and Δy error2 This set of auxiliary rotation results allows us to calculate the center fit bias.

[0101] If the auxiliary object undergoes two or more rotations, then Δθ1 and Δx error1 and Δy error1 This can be the auxiliary rotation result obtained from one rotation in a series of rotations, corresponding to Δθ2 and Δx. error2 and Δy error2 This can be an auxiliary rotation result obtained from another rotation in a series of rotations. The following explains two implementation methods for obtaining the center fitting deviation when the auxiliary object rotates more than twice relative to the reference position:

[0102] Method 1

[0103] For every two rotations, a set of parameter expressions for the center fitting bias is constructed based on the reference image pose, the angles included in the running image pose of the auxiliary object after rotation relative to the reference position in the image acquired by the image acquisition device, and the re-examination image position of the auxiliary object. Then, the constructed parameter expressions are solved using the least squares method to obtain the center fitting bias.

[0104] Specifically, we can construct a parametric relationship between the center fitting deviation in the x-axis direction of the physical coordinate system and the auxiliary angle difference and the position difference of the re-examined image, and a parametric relationship between the center fitting deviation in the y-axis direction of the physical coordinate system and the auxiliary angle difference and the position difference of the re-examined image. Then, we solve for the above parametric relationships using the least squares method. For example, we can use Δx and Δy in the above expression for calculating the center fitting deviation as the parametric relationships corresponding to each two rotations, and then solve for the constructed parametric relationships using the least squares method to obtain the center fitting deviation.

[0105] In this way, by rotating the auxiliary object multiple times to obtain multiple sets of auxiliary rotation results, the stability and accuracy of the calculated center fitting deviation can be improved, the influence of various errors such as acquisition error and rounding error can be reduced, the accuracy of the calculated center fitting deviation can be improved, and based on such center fitting deviation, a more accurate position offset can be obtained, thereby improving the precision of controlling the movement of the motion mechanism.

[0106] Method 2

[0107] In another case, for multiple sets of auxiliary rotation results obtained from multiple rotations, take the combination of two different sets of auxiliary rotation results for each type, and calculate multiple center fitting deviations according to the expression used above for calculating center fitting deviations. The average of the multiple center fitting deviations is taken as the final center fitting deviation.

[0108] The following example, taking the auxiliary object as rotating twice relative to the reference position, illustrates the expressions for Δx and Δy.

[0109] With P check1 This represents the position coordinates of the re-examined image after the first rotation in the pixel coordinate system, P. check2 Δθ1 represents the position coordinates of the re-inspection image after the second rotation in the pixel coordinate system, Δθ2 represents the auxiliary angle difference between the angle in the pose of the running image after the first rotation and the angle in the pose of the reference image, and Δθ3 represents the auxiliary angle difference between the angle in the pose of the running image after the second rotation and the angle in the pose of the reference image.

[0110] So, assuming the hand-eye calibration matrix is In the physical coordinate system, P check1 The deviation P between the corresponding physical position of the re-inspection and the position in the reference physical pose error1 It can be represented as:

[0111]

[0112] Where, x check1 and y check1 P respectively check1 The corresponding x-axis and y-axis coordinates of the physical location for re-inspection, x base and y base These are the x-axis and y-axis coordinates of the position in the reference physical pose, respectively.

[0113] Similarly, in the physical coordinate system, P check2 The deviation P between the corresponding physical position of the re-inspection and the position in the reference physical pose error2 It can be represented as:

[0114]

[0115] Where, xcheck2 and y check2 P respectively check2 The corresponding x-axis and y-axis coordinates of the physical location to be re-inspected.

[0116] Furthermore, P error1 and P error2 P represents the deviation between the re-inspected physical position and the position in the reference physical pose within the physical coordinate system. This deviation can be considered as rotation error and translation error, i.e., P error1 and P error2 It can be represented as:

[0117]

[0118] Where, Δx error1 and Δy error1 P respectively error1 The corresponding x-axis and y-axis errors of the physical location for re-inspection, Δx error2 and Δy error2 P respectively error2 The corresponding x-axis and y-axis errors of the physical location for re-inspection. and These represent the rotation error values ​​along the x-axis and y-axis in the physical coordinate system for the first rotation. and Let Δx represent the rotation error of the second rotation along the x-axis and y-axis in the physical coordinate system. T and Δy T This represents the translation error.

[0119] The above P error1 and P error2 Subtracting the expressions yields:

[0120]

[0121] The following rotation error expression will be used:

[0122]

[0123] In the first rotation, Δx in the above formula... R and Δy R It can be and In this case, Δθ is Δθ1, which leads to the following expression:

[0124]

[0125] During the second rotation, Δx in the above formula R and ΔYR It can be and In this case, Δθ is Δθ², which leads to the following expression:

[0126]

[0127] Subtract P error1 and P error2 Combining the expression with the rotation error expression, we can obtain:

[0128]

[0129] Simplifying the above equation, we get:

[0130]

[0131] The expression for rotational error is explained below.

[0132] Assume the center position of the rotation axis of the motion mechanism is located at the coordinate (x) in the physical coordinate system. c y c If the position offset is accurate, then the position offset (x) can be obtained. offset ,y offset The expression is:

[0133]

[0134] Assuming that the motion mechanism is affected by repeatability and absolute positioning accuracy, we can add the translation error to the position offset to obtain the expressions for translation error and position offset:

[0135]

[0136] If the center position of the motion mechanism's rotation axis obtained by fitting has a center fitting deviation from the actual rotation axis position, and the center fitting deviation causes a rotation error in the position offset, then the rotation error is further added to the position offset and translation error, and the center position of the motion mechanism's rotation axis is corrected based on the center fitting deviation, resulting in the expressions for rotation error, translation error, and position offset:

[0137]

[0138] Within the local motion range of the motion mechanism, the change in the system translational deviation of the motion mechanism can be ignored. Therefore, the expression for the translational error and the expression for the positional offset can be subtracted from the expressions for the rotational error, translational error, and positional offset to obtain the expression for the rotational error:

[0139]

[0140] The following explains how the angular offset is obtained in the pose offset of the aforementioned motion mechanism when it carries the auxiliary object from the running physical pose to the reference physical pose.

[0141] In one embodiment of this application, the angle offset in the pose offset is obtained as follows: Based on the correspondence between image angles and physical angles, a first physical angle corresponding to the angle in the running image pose is obtained. Based on the correspondence, a second physical angle corresponding to the angle in the reference image pose is obtained. The difference between the second physical angle and the first physical angle is calculated as the angle offset.

[0142] Here, the image angle is the angle of the object in the image captured by the image acquisition device, and the physical angle is the angle of the object in physical space. For example, the image angle can be determined as follows: determine the image origin in the image, then determine the straight line between the object's position in the image and the origin, and determine the angle between the above straight line and the origin reference line as the image angle.

[0143] The following explains the correspondence between image angles and physical angles.

[0144] We can set the starting coordinates (θ) of the sampling interval. pix1 ,θ world1 ), coordinates of the endpoint of the interval (θ) pix2 ,θ world2 ), where θ pix1 and θ pix2 θ represents the starting and ending image angles of the sampling interval, respectively. world1 and θ world2 Let θ be the physical angles of the starting and ending points of the sampling interval, respectively. Then, for the sample located at (θ... pix1 ,θ pix2 Any θ within this image angle range pix There exists the following θ pix With θ world The correspondence between them:

[0145]

[0146] Thus, in obtaining (θ) pix1 ,θ pix2 Once the image angles within this range are determined, the corresponding physical angles can be calculated.

[0147] Therefore, after obtaining the angle in the pose of the running image, we can determine the correspondence between the angle in the pose of the running image and the image angle interval to which it belongs. Then, based on the determined correspondence, we can calculate the first physical angle corresponding to the angle in the pose of the running image according to the above formula.

[0148] Similarly, the correspondence between the angles in the reference image pose and the image angle intervals can be determined. Then, based on the determined correspondence, the second physical angle corresponding to the angle in the running image pose can be calculated according to the above formula.

[0149] Then, the difference between the second physical angle and the first physical angle is calculated as the angle offset.

[0150] In this way, based on the correspondence between image angles and physical angles, the angular offset in the pose offset can be obtained more accurately. The position of the re-examined image can be obtained by moving according to the more accurate pose offset, which can reduce the influence of errors on the determined center fitting deviation, thereby obtaining a more accurate position offset and improving the accuracy of controlling the movement of the motion mechanism.

[0151] The following describes another way to implement step S204, which controls the movement of the motion mechanism.

[0152] In one embodiment of this application, see [link to embodiment]. Figure 3 A flowchart of another motion mechanism control method is provided. The method includes steps S301-S307. Step S204 can be implemented by step S307.

[0153] Step S301: Obtain the current image pose of the target object to be grasped by the motion mechanism in the image captured by the image acquisition device.

[0154] Step S302: Determine the rotation angle difference between the angle in the current image pose and the angle in the reference image pose.

[0155] Step S303: Based on the position in the current image pose, the position in the reference image pose, the rotation angle difference, the center position of the rotation axis of the motion mechanism, and the center fitting deviation of the rotation axis, obtain the position offset of the motion mechanism moving towards the target object.

[0156] The steps S301-S303 above are the same as those S201-S203 above, and will not be described in detail here.

[0157] Step S304: Based on the correspondence between image angles and physical angles, obtain the third physical angle corresponding to the angle in the reference image pose.

[0158] Among them, the image angle is the angle of the object in the image captured by the image acquisition device, and the physical angle is the angle of the object in physical space.

[0159] Step S305: Based on the correspondence, obtain the fourth physical angle corresponding to the angle in the current image pose.

[0160] Step S306: Calculate the angle difference between the fourth physical angle and the third physical angle.

[0161] The implementation of steps S304-S306 is similar to the method of obtaining the angle offset described above, the only difference being the name.

[0162] Step S307: Based on the position offset and angle difference, control the motion mechanism to move to the location of the object.

[0163] Specifically, the control device can control the operating mechanism to translate the aforementioned positional offset distance from the position corresponding to the reference image pose in physical space to the position corresponding to the current image pose in physical space. Furthermore, the control device can also control the operating mechanism to rotate the aforementioned angular difference from the angle corresponding to the reference image pose in physical space to the angle corresponding to the current image pose in physical space.

[0164] As can be seen from the above, controlling the motion mechanism to move in this way not only allows for translation based on a determined position offset, but also reduces the impact of the deviation between the rotation angle difference in the image and the actual angle difference in the physical coordinate system on the rotation accuracy of the motion mechanism. This allows for more precise rotation control based on a determined angle difference, enabling more accurate control of the motion mechanism to move from the physical position corresponding to the reference image pose to the physical position corresponding to the current image pose.

[0165] The method for generating the above correspondence is explained below.

[0166] In one embodiment of this application, the above correspondence is generated as follows: The reference image angle and the reference physical angle of the motion mechanism in the image acquired by the image acquisition device after the motion mechanism rotates carrying the reference object are obtained. Based on the reference image angle and the reference physical angle obtained after two adjacent rotations, a piecewise linear relationship between the image angle and the physical angle is constructed according to the rotation sequence.

[0167] Among them, the number of times the motion mechanism carries the reference object to rotate is greater than 2.

[0168] For example, taking the reference physical angle as an example, the motion mechanism can be controlled to rotate 10 degrees, 30 degrees, 50 degrees and 70 degrees in sequence according to the rotation order. Corresponding to the above rotation angles, when the motion mechanism rotates the reference physical angle by 10 degrees, the recorded reference image angle is 20 degrees; when the motion mechanism rotates the reference physical angle by 30 degrees, the recorded reference image angle is 40.3 degrees; when the motion mechanism rotates the reference physical angle by 50 degrees, the recorded reference image angle is 60.8 degrees; and when the motion mechanism rotates the reference physical angle by 70 degrees, the recorded reference image angle is 81.5 degrees.

[0169] After two consecutive rotations, the reference physical angles of 10 degrees and 30 degrees correspond to reference image angles of 20 degrees and 40.3 degrees, respectively. Therefore, according to the formula for the correspondence in the above embodiment, the correspondence within the image angle range (20, 40.3) can be obtained:

[0170]

[0171] After two consecutive rotations, the reference image angles corresponding to the reference physical angles of 30 degrees and 50 degrees are 40.3 degrees and 60.8 degrees, respectively. Therefore, according to the formula for the correspondence in the above embodiment, the correspondence within the image angle range (40.3, 60.8) can be obtained as follows:

[0172]

[0173] As can be seen from the above, it is possible to obtain the reference image angle and reference physical angle after each two adjacent rotations, and construct a piecewise linear relationship between the image angle and physical angle corresponding to each angle interval. Based on this correspondence, the accurate physical angle can be obtained, thereby improving the accuracy of controlling the movement of the motion mechanism.

[0174] This can reduce rotational errors during the movement of the motion mechanism, for example, for Figure 1 In this way, the magnitude of ΔR can be reduced, thereby reducing the error between points P and P'.

[0175] Corresponding to the above-mentioned motion mechanism control method, this application provides a motion mechanism control device.

[0176] In one embodiment of this application, see [link to embodiment]. Figure 4 A schematic diagram of a motion mechanism control device is provided, the device comprising:

[0177] The current image pose acquisition module 401 is used to obtain the current image pose of the target object to be grasped by the motion mechanism in the image acquired by the image acquisition device;

[0178] The rotation angle difference determination module 402 is used to determine the rotation angle difference between the angle in the current image pose and the angle in the reference image pose, wherein the reference image pose is the pose of the object in the image acquired by the image acquisition device when the object is located at the reference position.

[0179] The position offset acquisition module 403 is used to obtain the position offset of the motion mechanism moving towards the target object based on the position in the current image pose, the position in the reference image pose, the rotation angle difference, the center position of the rotation axis of the motion mechanism, and the center fitting deviation of the rotation axis. The center fitting deviation is obtained based on the reference image pose, the angle included in the running image pose in the image acquired by the image acquisition device after the auxiliary object rotates relative to the reference position, and the re-examination image position of the auxiliary object. The re-examination image position is the position of the auxiliary object in the image acquired by the image acquisition device after the motion mechanism carries the auxiliary object with the movement pose offset. The pose offset is the offset from the running physical pose to the reference physical pose. The running physical pose is the pose corresponding to the running image pose in the physical space where the motion mechanism is located. The reference physical pose is the pose corresponding to the reference image pose in the physical space. The auxiliary object rotates at least twice relative to the reference position.

[0180] The motion mechanism control module 404 is used to control the motion mechanism to move towards the location of the target object based on the position offset.

[0181] As can be seen from the above, in the solution provided by the embodiments of this application, when obtaining the position offset for controlling the motion mechanism to move to the location of the object, in addition to considering the position in the current image pose, the position in the reference image pose, and the difference in rotation angle, the center position of the fitted motion mechanism rotation axis and the center fitting deviation of the actual rotation axis are also considered. The center fitting deviation is obtained based on the running image pose, the re-inspection image position, and the reference image pose obtained by rotating the auxiliary object. The obtained center fitting deviation is more accurate. In this way, the center position of the fitted motion mechanism rotation axis can be corrected more accurately, and thus the accurate position offset can be obtained based on the corrected rotation axis center position, thereby improving the accuracy of controlling the movement of the motion mechanism.

[0182] In one embodiment of this application, the angle offset in the pose offset is obtained as follows: based on the correspondence between image angles and physical angles, a first physical angle corresponding to the angle in the running image pose is obtained, wherein the image angle is the angle of the object in the image acquired by the image acquisition device, and the physical angle is the angle of the object in the physical space; based on the correspondence, a second physical angle corresponding to the angle in the reference image pose is obtained; the difference between the second physical angle and the first physical angle is calculated as the angle offset.

[0183] In this way, based on the correspondence between image angles and physical angles, the angular offset in the pose offset can be obtained more accurately. The position of the re-examined image can be obtained by moving according to the more accurate pose offset, which can reduce the influence of errors on the determined center fitting deviation, thereby obtaining a more accurate position offset and improving the accuracy of controlling the movement of the motion mechanism.

[0184] In one embodiment of this application, if the auxiliary object rotates more than twice relative to the reference position, the center fitting deviation is obtained as follows: for each two rotations, based on the reference image pose, the angles included in the running image pose in the image acquired by the image acquisition device after the auxiliary object rotates relative to the reference position, and the re-examination image position of the auxiliary object, a set of parameter expressions for the center fitting deviation is constructed; the constructed parameter expressions are solved using the least squares method to obtain the center fitting deviation.

[0185] In this way, by rotating the auxiliary object multiple times to obtain multiple sets of auxiliary rotation results, the stability and accuracy of the calculated center fitting deviation can be improved, the influence of various errors such as acquisition error and rounding error can be reduced, the accuracy of the calculated center fitting deviation can be improved, and based on such center fitting deviation, a more accurate position offset can be obtained, thereby improving the precision of controlling the movement of the motion mechanism.

[0186] In one embodiment of this application, the center fitting deviation is calculated based on the following expression:

[0187]

[0188] Where Δx and Δy are the deviations along the x-axis and y-axis of the physical coordinate system in the physical space, respectively, in the center fitting deviation; Δθ1 and Δθ2 are the auxiliary angle differences between the angles in the running image pose and the angles in the reference image pose of the auxiliary object during the two rotations; Δx error1 and Δy error1 Δx is the positional difference between the physical position of the auxiliary object after the first rotation and the reference physical position during the two rotations. error2 and Δy error2 The physical position of the auxiliary object after the second rotation is the difference between the physical position of the re-inspection and the physical position of the reference. The physical position of the re-inspection is the position of the re-inspection image in physical space, and the physical position of the reference is the position information in the reference physical pose.

[0189] In this way, the center fitting deviation can be accurately calculated, and the positional offset between the current physical position corresponding to the current image pose and the reference physical position corresponding to the reference image pose can be obtained, thereby improving the accuracy of controlling the movement of the motion mechanism.

[0190] In one embodiment of this application, the device further includes: a third physical angle acquisition module, configured to obtain a third physical angle corresponding to the angle in the reference image pose based on the correspondence between image angles and physical angles, wherein the image angle is the angle of the object in the image acquired by the image acquisition device, and the physical angle is the angle of the object in the physical space; a fourth physical angle acquisition module, configured to obtain a fourth physical angle corresponding to the angle in the current image pose based on the correspondence; an angle difference calculation module, configured to calculate the angle difference between the fourth physical angle and the third physical angle; and a motion mechanism control module, specifically configured to control the motion mechanism to move towards the location of the object based on the position offset and the angle difference.

[0191] As can be seen from the above, controlling the motion mechanism to move in this way not only allows for translation based on a determined position offset, but also reduces the impact of the deviation between the rotation angle difference in the image and the actual angle difference in the physical coordinate system on the rotation accuracy of the motion mechanism. This allows for more precise rotation control based on a determined angle difference, enabling more accurate control of the motion mechanism to move from the physical position corresponding to the reference image pose to the physical position corresponding to the current image pose.

[0192] In one embodiment of this application, the correspondence is generated as follows: obtaining the reference image angle of the reference object in the image acquired by the image acquisition device after the motion mechanism carries the reference object and the reference physical angle of the motion mechanism, wherein the number of rotations of the motion mechanism carrying the reference object is greater than 2; according to the rotation order, based on the reference image angle and reference physical angle obtained after two adjacent rotations, constructing a piecewise linear relationship between the image angle and the physical angle.

[0193] As can be seen from the above, it is possible to obtain the reference image angle and reference physical angle after each two adjacent rotations, and construct a piecewise linear relationship between the image angle and physical angle corresponding to each angle interval. Based on this correspondence, the accurate physical angle can be obtained, thereby improving the accuracy of controlling the movement of the motion mechanism.

[0194] This application also provides an electronic device, such as... Figure 5 As shown, it includes:

[0195] Memory 501 is used to store computer programs;

[0196] The processor 502, when executing the program stored in the memory 501, implements any of the motion mechanism control methods described above.

[0197] Furthermore, the aforementioned electronic device may also include a communication bus and / or a communication interface, with the processor 502, communication interface, and memory 501 communicating with each other via the communication bus.

[0198] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0199] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0200] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0201] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0202] In another embodiment provided in this application, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the steps of any of the above-described motion mechanism control methods.

[0203] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the motion mechanism control methods described above.

[0204] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a solid-state drive (SSD), etc.

[0205] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0206] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for methods, apparatuses, electronic devices, storage media, and computer program products, since they are basically similar to the method embodiments, the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0207] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the scope of protection of this application.

Claims

1. A method for controlling a motion mechanism, characterized in that, The method includes: Obtain the current image pose of the target object to be grasped by the motion mechanism in the image captured by the image acquisition device; Determine the rotation angle difference between the angle in the current image pose and the angle in the reference image pose, wherein the reference image pose is the pose of the object in the image acquired by the image acquisition device when the object is located at the reference position; Based on the position in the current image pose, the position in the reference image pose, the rotation angle difference, the center position of the rotation axis of the motion mechanism, and the center fitting deviation of the rotation axis, the position offset of the motion mechanism moving towards the target object is obtained. The center fitting deviation is based on the reference image pose, the angles included in the running image pose in the image acquired by the image acquisition device after the auxiliary object rotates relative to the reference position, and the re-examination image position of the auxiliary object. The re-examination image position is the position of the auxiliary object in the image acquired by the image acquisition device after the motion mechanism carries the auxiliary object's movement pose offset. The pose offset is the offset from the running physical pose to the reference physical pose. The running physical pose is the pose corresponding to the running image pose in the physical space where the motion mechanism is located. The reference physical pose is the pose corresponding to the reference image pose in the physical space. The auxiliary object rotates at least twice relative to the reference position. Based on the position offset, the motion mechanism is controlled to move towards the location of the target object.

2. The method according to claim 1, characterized in that, The angular offset in the pose offset is obtained in the following manner: Based on the correspondence between image angles and physical angles, the first physical angle corresponding to the angle in the pose of the running image is obtained, wherein the image angle is the angle of the object in the image acquired by the image acquisition device, and the physical angle is the angle of the object in the physical space. Based on the correspondence, the second physical angle corresponding to the angle in the pose of the reference image is obtained; Calculate the difference between the second physical angle and the first physical angle, and use it as the angle offset.

3. The method according to claim 1, characterized in that, If the auxiliary object rotates more than twice relative to the reference position, the center fitting deviation is obtained as follows: For every two rotations, based on the reference image pose, the angles included in the running image pose of the auxiliary object after rotation relative to the reference position in the image acquired by the image acquisition device, and the re-examination image position of the auxiliary object, a set of parameter expressions for the center fitting deviation are constructed. The central fitting deviation is obtained by solving the constructed parameter expression relationship based on the least squares method.

4. The method according to any one of claims 1-3, characterized in that, The center-fit deviation is calculated based on the following expression: Where Δx and Δy are the deviations along the x-axis and y-axis of the physical coordinate system in the physical space, respectively, in the center fitting deviation; Δθ1 and Δθ2 are the auxiliary angle differences between the angles in the running image pose and the angles in the reference image pose of the auxiliary object during the two rotations; Δx error1 and Δy error1 Δx is the positional difference between the physical position of the auxiliary object after the first rotation and the reference physical position during the two rotations. error2 and Δy error2 The physical position difference between the physical position of the auxiliary object after the second rotation and the reference physical position is the position of the physical image in the physical space, and the reference physical position is the position information in the reference physical pose.

5. The method according to any one of claims 1-3, characterized in that, The method further includes: Based on the correspondence between image angles and physical angles, a third physical angle corresponding to the angle in the reference image pose is obtained, wherein the image angle is the angle of the object in the image acquired by the image acquisition device, and the physical angle is the angle of the object in the physical space. Based on the correspondence, the fourth physical angle corresponding to the angle in the current image pose is obtained; Calculate the angle difference between the fourth physical angle and the third physical angle; The step of controlling the motion mechanism to move towards the location of the object based on the position offset includes: Based on the position offset and the angle difference, the motion mechanism is controlled to move towards the location of the object.

6. The method according to claim 5, characterized in that, The correspondence is generated in the following manner: The reference image angle of the reference object in the image acquired by the image acquisition device after the motion mechanism carries the reference object to rotate and the reference physical angle of the motion mechanism are obtained, wherein the number of rotations of the reference object carried by the motion mechanism is greater than 2; Based on the rotation sequence, and using the reference image angle and reference physical angle obtained after two adjacent rotations, a piecewise linear relationship between the image angle and the physical angle is constructed.

7. A motion mechanism control device, characterized in that, The device includes: The current image pose acquisition module is used to obtain the current image pose of the target object to be grasped by the motion mechanism in the image acquired by the image acquisition device; The rotation angle difference determination module is used to determine the rotation angle difference between the angle in the current image pose and the angle in the reference image pose, wherein the reference image pose is the pose of the object in the image acquired by the image acquisition device when the object is located at the reference position. The position offset acquisition module is used to obtain the position offset of the motion mechanism moving towards the target object based on the position in the current image pose, the position in the reference image pose, the rotation angle difference, the center position of the rotation axis of the motion mechanism, and the center fitting deviation of the rotation axis. The center fitting deviation is obtained based on the reference image pose, the angle included in the running image pose in the image acquired by the image acquisition device after the auxiliary object rotates relative to the reference position, and the re-examination image position of the auxiliary object. The re-examination image position is the position of the auxiliary object in the image acquired by the image acquisition device after the motion mechanism carries the auxiliary object with the movement pose offset. The pose offset is the offset from the running physical pose to the reference physical pose. The running physical pose is the pose corresponding to the running image pose in the physical space where the motion mechanism is located. The reference physical pose is the pose corresponding to the reference image pose in the physical space. The auxiliary object rotates at least twice relative to the reference position. A motion mechanism control module is used to control the motion mechanism to move towards the location of the target object based on the position offset.

8. The apparatus according to claim 7, characterized in that, The angle offset in the pose offset is obtained as follows: based on the correspondence between image angles and physical angles, a first physical angle corresponding to the angle in the running image pose is obtained, wherein the image angle is the angle of the object in the image acquired by the image acquisition device, and the physical angle is the angle of the object in the physical space; based on the correspondence, a second physical angle corresponding to the angle in the reference image pose is obtained; the difference between the second physical angle and the first physical angle is calculated as the angle offset.

9. The apparatus according to claim 7, characterized in that, If the auxiliary object rotates more than twice relative to the reference position, the center fitting deviation is obtained as follows: For each two rotations, based on the reference image pose, the angles included in the running image pose of the auxiliary object after rotation relative to the reference position in the image acquired by the image acquisition device, and the re-examination image position of the auxiliary object, a set of parameter expressions for the center fitting deviation is constructed; the center fitting deviation is obtained by solving the constructed parameter expressions based on the least squares method.

10. The apparatus according to any one of claims 7-9, characterized in that, The center-fit deviation is calculated based on the following expression: Where Δx and Δy are the deviations along the x-axis and y-axis of the physical coordinate system in the physical space, respectively, in the center fitting deviation; Δθ1 and Δθ2 are the auxiliary angle differences between the angles in the running image pose and the angles in the reference image pose of the auxiliary object during the two rotations; Δx error1 and Δy error1 Δx is the positional difference between the physical position of the auxiliary object after the first rotation and the reference physical position during the two rotations. error2 and Δy error2 The physical position difference between the physical position of the auxiliary object after the second rotation and the reference physical position is the position of the physical image in the physical space, and the reference physical position is the position information in the reference physical pose.

11. The apparatus according to any one of claims 7-9, characterized in that, The device further includes: a third physical angle acquisition module, used to obtain a third physical angle corresponding to the angle in the reference image pose based on the correspondence between image angles and physical angles, wherein the image angle is the angle of the object in the image acquired by the image acquisition device, and the physical angle is the angle of the object in the physical space; a fourth physical angle acquisition module, used to obtain a fourth physical angle corresponding to the angle in the current image pose based on the correspondence; an angle difference calculation module, used to calculate the angle difference between the fourth physical angle and the third physical angle; and a motion mechanism control module, specifically used to control the motion mechanism to move towards the location of the object based on the position offset and the angle difference.

12. The apparatus according to claim 11, characterized in that, The correspondence is generated as follows: after the motion mechanism carries the reference object and rotates it, the reference image angle and the reference physical angle of the motion mechanism in the image acquired by the image acquisition device are obtained, wherein the number of rotations of the motion mechanism carrying the reference object is greater than 2; according to the rotation order, based on the reference image angle and the reference physical angle obtained after two adjacent rotations, a piecewise linear relationship between the image angle and the physical angle is constructed.

13. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the method of any one of claims 1-6.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method described in any one of claims 1-6.

Images