A pose determination method, apparatus, device and storage medium

Abstract

A pose determination method, device, equipment and storage medium are disclosed. The method comprises: obtaining an expected pose corresponding to a mechanical arm, and obtaining a marker plate image under the expected pose; obtaining a current pose of the mechanical arm, and obtaining a marker plate image under the current pose; determining whether a deviation between the marker plate image under the current pose and the marker plate image under the expected pose is less than a preset threshold; and if the deviation is greater than or equal to the preset threshold, determining a target pose corresponding to the mechanical arm according to the current pose and the expected pose. Through the technical scheme of the present application, the robot can be adaptively servo-positioned by using image information of a 2D camera, and then perform a grabbing work, thereby reducing the cost.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of robotics, and in particular to a pose determination method, apparatus, device, and storage medium. Background Technology

[0002] When using a robot with its eye on the hand for localization and grasping, it is usually necessary to know the pose information of the object to be grasped in three-dimensional space. This information needs to be obtained based on the 3D information of the object to be grasped and the hand-eye conversion relationship.

[0003] However, under many operating conditions, the parameters calibrated by the system may change slowly. For example, camera parameters may degrade under radiation conditions. Furthermore, the system calibration results are only valid under the calibration conditions; even minor structural changes may require recalibration. For instance, when a vision positioning system operates under conditions of significant vibration, it may alter hand-eye coordination. In addition, due to factors such as camera distortion, the camera calibration area is generally limited to a finite region, restricting the robot's working range. Moreover, system calibration is usually quite complex, requiring specialized calibration procedures and personnel, resulting in high calibration costs.

[0004] Therefore, how to use a 2D camera to utilize image information to enable a robot to perform grasping work when the 3D information of the object to be grasped is unavailable and the hand-eye transition relationship is unknown is a direction worth considering and innovating. Summary of the Invention

[0005] This invention provides a pose determination method, apparatus, device, and storage medium to enable a robot to perform adaptive servo positioning and grasping operations using only image information from a 2D camera, thereby reducing costs.

[0006] According to one aspect of the present invention, a pose determination method is provided, comprising:

[0007] Obtain the desired pose of the robotic arm and the image of the marker plate under the desired pose;

[0008] Obtain the current pose of the robotic arm and acquire the image of the marker board in the current pose;

[0009] Determine whether the deviation between the marker image in the current pose and the marker image in the desired pose is less than a preset threshold;

[0010] If the deviation is greater than or equal to the preset threshold, the target pose of the robotic arm is determined based on the current pose and the desired pose.

[0011] According to another aspect of the present invention, a pose determination device is provided, the device comprising:

[0012] The first acquisition module is used to acquire the desired pose of the robotic arm and acquire the image of the marker plate under the desired pose.

[0013] The second acquisition module is used to acquire the current pose of the robotic arm and acquire the image of the marker board in the current pose.

[0014] The judgment module is used to determine whether the deviation between the marker image in the current pose and the marker image in the desired pose is less than a preset threshold.

[0015] The determination module is used to determine the target pose of the robotic arm based on the current pose and the desired pose if the deviation is greater than or equal to the preset threshold.

[0016] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0017] At least one processor; and

[0018] A memory communicatively connected to the at least one processor; wherein,

[0019] The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the pose determination method according to any embodiment of the present invention.

[0020] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the pose determination method according to any embodiment of the present invention.

[0021] This invention obtains the desired pose of the robotic arm and a marker image in that desired pose, then obtains the current pose of the robotic arm and a marker image in that current pose. The two are compared pixel-wise to determine if the deviation between the marker image in the current pose and the marker image in the desired pose is less than a preset threshold. If the deviation is less than the preset threshold, the current pose of the robotic arm is directly determined as the target pose. If the deviation is greater than or equal to the preset threshold, the target pose of the robotic arm is determined based on the current pose and the desired pose. Through this invention, the robot can perform adaptive servo positioning and grasping operations using only image information from a 2D camera, thus reducing costs.

[0022] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0023] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a flowchart of a pose determination method according to an embodiment of the present invention;

[0025] Figure 2 This is a schematic diagram of the structure of a pose determination device according to an embodiment of the present invention;

[0026] Figure 3 This is a schematic diagram of the structure of an electronic device that implements the pose determination method of the present invention. Detailed Implementation

[0027] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0028] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0029] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0030] Example 1

[0031] When performing localization and grasping operations on robots with "eyes on the hand," it is often necessary to locate the object to be grasped. This process includes recognizing the object, calculating its spatial pose and position, converting the object's pose information to the robot's end effector using hand-eye coordination, and then converting it to the robot's base coordinate system so that the robot can grasp the object. The conditions for achieving this series of operations often require a 3D camera and a hand-eye coordination mechanism. The 3D camera can calculate the pose of the object by obtaining point cloud information from object recognition, and the hand-eye coordination can be obtained through camera-robot calibration. However, during use, the connection between the camera and the robot's end effector may deform over time, and the camera's internal firmware may also change. These factors can affect the hand-eye coordination mechanism, thus impacting the grasping operation. If only a 2D camera is used to perform adaptive servo positioning using image information, it reduces costs and avoids the impact of changes in the camera's own parameters. The main design concept of this invention is to enable the robot to perform grasping operations using only a 2D camera and image information.

[0032] This invention proposes a closed-loop visual positioning system. The system requires a robotic arm, a 2D camera, and a marker plate. The robotic arm can be the arm of a robot that grasps objects. The 2D camera can be mounted at the end of the robotic arm. The marker plate can be placed below or to the side of the object to be grasped (this embodiment does not limit this and can be set according to the relative position of the robot and the object to be grasped, as well as the grasping method), serving as a pose reference.

[0033] The main design principle of this invention can be described as follows: When grasping a target object (i.e., the object to be grasped), if the relationship between the target object and the marker plate is fixed, it is only necessary to ensure that the relationship between the camera and the marker plate remains unchanged each time the robot performs the grasping task. In this way, the grasping target pose can be guaranteed to be accurate. Therefore, when the robot performs the grasping task, it is necessary to adjust the relative relationship between the camera (the camera is at the end of the robotic arm, i.e., the moving robotic arm) and the marker plate.

[0034] The main design concept of this invention can be described as follows: Set the desired pose of the robotic arm and record the feature information of the marker plate in the camera coordinate system under the desired pose; this means defining the desired relative relationship between the camera and the marker plate. When the robotic arm (i.e., the camera, since the camera is mounted at the end of the robotic arm) deviates from the set desired state, capture the feature point information of the marker plate, adjust the robotic arm pose to return it to the desired state, and then perform the grasping operation.

[0035] Figure 1This is a flowchart of a pose determination method according to an embodiment of the present invention. This embodiment is applicable to situations where a robot with its eye on its hand needs to determine its pose during a positioning and grasping operation. This method can be executed by the pose determination device described in this embodiment, which can be implemented using software and / or hardware, such as... Figure 1 As shown, the method specifically includes the following steps:

[0036] S101. Obtain the desired pose of the robotic arm and the image of the marker plate under the desired pose.

[0037] The desired pose can be understood as the pose information of the robotic arm when it can grasp the object, that is, the pose of the robotic arm that needs to be corrected.

[0038] The image of the marker in the desired pose can be an image of the marker taken by a camera mounted on the robotic arm when the robotic arm is in the desired pose.

[0039] Specifically, first, a desired state for the robot is set, that is, the desired pose of the robotic arm is set (which can be denoted as Pose1), and the feature point information of the marker board is identified by taking a picture under the desired pose and recorded. At the same time, the marker board image under the desired pose is saved (which can be denoted as GoalImage).

[0040] S102. Obtain the current pose of the robotic arm and the image of the marker board in the current pose.

[0041] The current pose can be the robot's pose information during actual operation.

[0042] In this embodiment, to simulate the deviation between the robot's current pose and the desired pose, random perturbation can be added to the desired pose of the robotic arm to simulate the actual working state of the robotic arm. The perturbation range can be random, that is, random perturbation is added to the desired pose in the X, Y, and Z directions respectively to obtain the current pose (which can be denoted as Pose2), ensuring that the marker board can be captured subsequently.

[0043] The marker image in the current pose can be an image of the marker taken by a camera mounted on the robotic arm when the robotic arm is in the current pose state.

[0044] Specifically, the current pose Pose2 of the robotic arm is obtained, and the feature point information of the marker board is identified by taking a picture in the current pose and recorded. At the same time, the marker board image in the current pose (which can be denoted as GoalImage) is saved.

[0045] S103. Determine whether the deviation between the marker image in the current pose and the marker image in the desired pose is less than a preset threshold.

[0046] Here, deviation can refer to the pixel difference between the marker image captured in the current pose and the marker image captured in the desired pose.

[0047] The preset threshold can be a pixel value pre-set according to actual conditions, and this embodiment does not limit it. For example, the preset threshold can be 2 pixels.

[0048] Specifically, the pixel-by-pixel comparison is performed between the marker image captured in the current pose and the marker image captured in the desired pose, and it is determined whether the deviation between the two is less than a preset threshold.

[0049] If the deviation is less than the preset threshold, it can be determined that the current pose of the robotic arm is close to the desired pose, and the grasping operation can be successfully performed. At this time, the current pose of the robotic arm can be directly determined as the target pose, and the robot can be controlled to grasp the target object.

[0050] S104. If the deviation is greater than or equal to the preset threshold, the target pose of the robotic arm is determined based on the current pose and the desired pose.

[0051] The target pose can be the pose information of the robotic arm when it can grasp an object in actual operation.

[0052] Specifically, if the deviation is greater than or equal to the preset threshold, the robot arm's pose is rotated and translated based on the current pose and the desired pose to obtain the target pose of the robot arm, so that the robot arm can successfully grasp the target object.

[0053] This invention obtains the desired pose of the robotic arm and a marker image in that desired pose, then obtains the current pose of the robotic arm and a marker image in that current pose. The two are compared pixel-wise to determine if the deviation between the marker image in the current pose and the marker image in the desired pose is less than a preset threshold. If the deviation is less than the preset threshold, the current pose of the robotic arm is directly determined as the target pose. If the deviation is greater than or equal to the preset threshold, the target pose of the robotic arm is determined based on the current pose and the desired pose. Through this invention, the robot can perform adaptive servo positioning and grasping operations using only image information from a 2D camera, thus reducing costs.

[0054] Optionally, the target pose of the robotic arm is determined based on the current pose and the desired pose, including:

[0055] Perform a rotation initialization operation on the current pose to obtain the rotation deviation.

[0056] In this embodiment, the rotation initialization operation can be understood as the initial rotation operation performed on the robot in its current pose before image correction. The rotation requires that the camera can capture the marker after each rotation.

[0057] Rotational deviation can be the deviation in rotational angle between the current pose and the desired pose of the robotic arm.

[0058] Specifically, a rotation initialization operation is performed on the robot in its current pose, and the rotation deviation is calculated.

[0059] The robotic arm is rotated and corrected based on the rotational deviation to obtain the first pose.

[0060] Among them, rotational correction can be used to rotate the robotic arm according to the rotational deviation, and correct it to be as close as possible to the desired pose.

[0061] The first pose can be the pose information obtained by rotating the robotic arm according to the rotational deviation.

[0062] Specifically, after calculating the rotational deviation, a rotational control calculation is performed on the current pose of the robotic arm, and the first pose (which can be denoted as Pose3) is obtained after rotational correction.

[0063] Perform a translation initialization operation on the first pose to obtain the translation deviation.

[0064] In this embodiment, the translation initialization operation can be understood as the initial translation operation performed on the robot in the first pose after rotation correction before image correction.

[0065] Translational deviation can be the deviation in translational distance between the first pose and the desired pose of the robotic arm.

[0066] Specifically, a translation initialization operation is performed on the robot in its first pose, and the translation deviation is calculated.

[0067] The translation deviation is used to correct the translation of the robotic arm, thus obtaining the second pose.

[0068] Translation correction can be used to translate the robotic arm according to the translation deviation, and correct it to be as close as possible to the desired pose.

[0069] The second pose can be the pose information obtained by translating the robotic arm according to the translation deviation.

[0070] Specifically, after calculating the translation deviation, a translation control calculation is performed on the first pose of the robotic arm, and the second pose (which can be denoted as Pose4) is obtained after translation correction.

[0071] The rotation control and translation control are determined based on the second pose and the desired pose.

[0072] In this embodiment, the rotation control amount and translation control amount can be obtained by calculating the deviation between the marker image in the current pose and the marker image GoalImage in the desired pose based on the second pose Pose4 taking pictures sequentially.

[0073] Specifically, based on the second pose Pose4, the deviation between the marker image in the current pose and the marker image GoalImage in the desired pose is calculated to obtain the rotation control amount and translation control amount, so that the robotic arm can perform rotation control and translation control.

[0074] The robotic arm is rotated and translated based on the rotation control and translation control values, which serve as the updated current pose of the robotic arm.

[0075] Rotation control can be used to rotate the robotic arm, while translation control can be used to translate the robotic arm.

[0076] Specifically, the robotic arm is rotated and translated based on rotation and translation control values ​​to obtain the updated current pose. The system then determines whether the deviation between the current pose and the desired pose of the marker image is less than a preset threshold. If the deviation is less than the threshold, the updated current pose is directly determined as the target pose. If the deviation is greater than or equal to the threshold, then:

[0077] Return to perform rotation initialization operations on the current pose, obtain rotation deviations, and continue until the target pose is obtained.

[0078] Specifically, the process iterates sequentially until the deviation between the marker image captured in the current pose and the marker image GoalImage in the desired pose meets the requirement of being less than a preset threshold. At this point, the iteration terminates, and the camera is adjusted to the desired shooting position by default. Based on this robot state, the grasping operation can be performed.

[0079] Optionally, perform a rotation initialization operation on the current pose to obtain the rotation deviation, including:

[0080] Perform a rotation initialization operation on the current pose and obtain the corresponding set of rotated images in the rotation initialization operation.

[0081] The rotating image set can be a collection of images of several marker boards captured by the camera during the rotation initialization operation of the current pose.

[0082] Specifically, before image-based correction, rotation initialization is performed. Based on the current pose Pose2, initial rotations are performed around the X, Y, and Z axes respectively (each rotation requires that the marker board can be captured in the image). Images of the marker board captured after three rotations are taken, resulting in a set of rotated images.

[0083] The rotational deviation is determined based on the set of rotated images and the marker image in the desired pose.

[0084] Specifically, the rotation deviation is calculated based on the marker image obtained from three rotational photographs and the desired bit image GoalImage.

[0085] Optionally, a rotation initialization operation is performed on the current pose, and the corresponding set of rotated images in the rotation initialization operation is obtained, including:

[0086] The robotic arm in its current pose is rotated around the X-axis by a preset number of degrees to obtain the first rotation pose, and the first rotation image corresponding to the first rotation pose is obtained.

[0087] The preset degree can be a rotation angle pre-set according to actual conditions, and this embodiment does not limit it. For example, the preset degree can be 10 degrees.

[0088] The first rotational pose can be the pose state of the robotic arm after rotating it around the X-axis by a preset number of degrees from its current pose. The first rotational image can be an image of the marker plate taken by a camera mounted on the robotic arm when the robotic arm is in the first rotational pose.

[0089] Specifically, based on the current pose Pose2, the robotic arm is rotated around the X-axis by a preset number of degrees, and a picture is taken to obtain the image of the marker board.

[0090] The robotic arm in the first rotation pose is rotated around the Y-axis by a preset number of degrees to obtain the second rotation pose, and the second rotation image corresponding to the second rotation pose is obtained.

[0091] The second rotational pose can be the pose state of the robotic arm after rotating it around the Y-axis by a preset number of degrees from its first rotational pose. The second rotational image can be an image of the marker plate taken by a camera mounted on the robotic arm when the robotic arm is in the second rotational pose.

[0092] Specifically, based on the first rotational pose, the robotic arm is rotated around the Y-axis by a preset number of degrees, and an image of the marker board is captured.

[0093] The robotic arm, which is in the second rotational pose, is rotated around the Z-axis by a preset number of degrees to obtain the third rotational pose, and the third rotational image corresponding to the third rotational pose is obtained.

[0094] The third rotational pose can be the pose state of the robotic arm after rotating it around the Z-axis by a preset number of degrees from its second rotational pose. The third rotational image can be an image of the marker plate taken by a camera mounted on the robotic arm when the robotic arm is in the third rotational pose.

[0095] Specifically, based on the second rotational pose, the robotic arm is rotated around the Z-axis by a preset number of degrees, and an image of the marker board is captured.

[0096] Optionally, a translation initialization operation is performed on the first pose to obtain the translation deviation, including:

[0097] Perform translation initialization on the first pose and obtain the corresponding set of translation images in the translation initialization operation.

[0098] The translation image set can be a collection of images of several marker boards captured by the camera during the translation initialization operation of the first pose.

[0099] Specifically, based on the first pose Pose3, a translation initialization operation is performed, translating along the X, Y, and Z axes respectively, and taking pictures of the marker board after the three translations to obtain a set of translation images.

[0100] The translation deviation is determined based on the set of translated images and the marker image in the desired pose.

[0101] Specifically, based on the marker image obtained from three translational photographs and the desired position image GoalImage, feature points are identified and the translational deviation is calculated.

[0102] Optionally, a translation initialization operation is performed on the first pose, and the corresponding set of translation images in the translation initialization operation is obtained, including:

[0103] The robotic arm, which is in the third rotational pose, is translated a preset distance along the X-axis to obtain the first translational pose, and the first translational image corresponding to the first translational pose is obtained.

[0104] The preset distance can be a translation distance pre-set according to the actual situation, and this embodiment does not limit it.

[0105] The first translational pose can be the pose state of the robotic arm after it has been translated a preset distance along the X-axis from its first pose. The first translational image can be an image of the marker plate taken by a camera mounted on the robotic arm when the robotic arm is in the first translational pose.

[0106] Specifically, based on the first pose Pose3, the robotic arm is translated a preset distance along the X-axis to take a picture and obtain the image of the marker board.

[0107] The robotic arm in the first translation pose is translated a preset distance along the Y-axis to obtain the second translation pose, and the second translation image corresponding to the second translation pose is obtained.

[0108] The second translational pose can be the pose state of the robotic arm after it has been translated a preset distance along the Y-axis from its first translational pose. The second translational image can be an image of the marker plate taken by a camera mounted on the robotic arm when the robotic arm is in the second translational pose.

[0109] Specifically, based on the first translational pose, the robotic arm is translated a preset distance along the Y-axis to take a picture and obtain the currently captured image of the marker board.

[0110] The robotic arm in the second translation pose is translated a preset distance along the Z-axis to obtain the third translation pose, and the third translation image corresponding to the third translation pose is obtained.

[0111] The third translation pose can be the pose state of the robotic arm after it has been translated a preset distance along the Z-axis from its second pose. The third translation image can be an image of the marker plate taken by a camera mounted on the robotic arm when the robotic arm is in the third translation pose.

[0112] Specifically, based on the second translational pose, the robotic arm is translated a preset distance along the Z-axis, and an image of the marker board is captured.

[0113] Optionally, the deviation between the marker image in the target pose and the marker image in the desired pose is less than a preset threshold.

[0114] As an overall description of an embodiment of the present invention, the detailed process of the visual positioning method can be described as follows:

[0115] 1. First, set a desired state, that is, set the desired pose of the robot (Pose1, record the current pose information of the robot), and take a picture to identify the feature point information of the marker board under the desired pose, and record it. At the same time, save the marker board image GoalImage under the desired pose.

[0116] 2. To simulate the deviation between the robot's current state and the desired pose, random perturbation is added to the robot's desired pose. The perturbation range is random, that is, random perturbation is added to the desired pose in the X, Y, and Z directions to obtain the current pose state Pose2, ensuring that the marker board can be captured in the future.

[0117] 3. Before image-based correction, rotation initialization is performed. Under Pose2, initial rotations are performed around the X, Y, and Z axes respectively (the rotation requires that the marker board can be captured each time). The rotation method is to first rotate around the X axis, then take a picture to obtain the image of the current calibration board, then rotate around the Y axis, and so on, then rotate around the Z axis. After each rotation, the marker board is photographed. Based on the marker board images obtained from the three rotations, the rotation deviation is calculated with the desired position image GoalImage, and a rotation control calculation is performed to correct the rotation of the current robot to obtain Pose3.

[0118] 4. Next, perform translation initialization. First, translate along the X-axis in Pose3 and take a picture to obtain the image of the marker board. Then, translate along the Y-axis, and so on. Next, translate along the Z-axis. Take a picture of the marker board after each translation. Based on the marker board images obtained from the three translations and the desired position image GoalImage, identify feature points, calculate the translation deviation, and perform a translation control calculation to correct the translation of the robot to obtain Pose4.

[0119] 5. Based on the Pose4, take pictures sequentially and calculate the deviation between the current marker image and the desired image GoalImage to obtain the rotation control amount and translation control amount. Make the robot perform rotation control and translation control. Iterate sequentially. Finally, when the deviation between the current captured image and the desired image GoalImage meets the requirements, the iteration terminates. That is, the camera is adjusted to the desired shooting position by default. Based on this robot state, the grasping operation can be performed.

[0120] In robot visual servoing, the robot's next action is planned based on current image features and computation. Visual servoing trajectory planning methods generate suitable image feature point trajectories, and then use a controller to track these trajectories to complete the visual servoing task. This minimizes image errors in each control cycle, ensuring the stability of the spatial trajectory. Under uncalibrated conditions, camera intrinsic parameters, robot kinematic parameters, and hand-eye relationships are all unknown. In this case, a visual servoing scheme based on a "reference image" guides the robot to localization. This scheme is unaffected by changes in the camera's intrinsic parameters. Furthermore, given a "reference image" taken with a completely different camera, the camera can be positioned relative to non-planar objects.

[0121] Example 2

[0122] Figure 2This is a schematic diagram of a pose determination device according to an embodiment of the present invention. This embodiment is suitable for situations where a robotic arm performing a positioning and grasping operation needs to determine its pose. The device can be implemented using software and / or hardware, and can be integrated into any device that provides pose determination functionality, such as… Figure 2 As shown, the pose determination device specifically includes: a first acquisition module 201, a second acquisition module 202, a judgment module 203, and a determination module 204.

[0123] The first acquisition module 201 is used to acquire the desired pose of the robotic arm and acquire the image of the marker plate under the desired pose.

[0124] The second acquisition module 202 is used to acquire the current pose of the robotic arm and acquire the image of the marker board under the current pose;

[0125] The judgment module 203 is used to determine whether the deviation between the marker image in the current pose and the marker image in the desired pose is less than a preset threshold.

[0126] The determination module 204 is used to determine the target pose of the robotic arm based on the current pose and the desired pose if the deviation is greater than or equal to the preset threshold.

[0127] Optionally, the determining module 204 includes:

[0128] The first operation unit is used to perform a rotation initialization operation on the current pose to obtain the rotation deviation;

[0129] The first correction unit is used to correct the rotation of the robotic arm according to the rotational deviation to obtain the first pose.

[0130] The second operation unit is used to perform a translation initialization operation on the first pose to obtain the translation deviation.

[0131] The second correction unit is used to perform translation correction on the robotic arm based on the translation deviation to obtain the second pose.

[0132] A determining unit is configured to determine rotation control quantities and translation control quantities based on the second pose and the desired pose;

[0133] The control unit is used to perform rotation control and translation control on the robotic arm according to the rotation control amount and the translation control amount, as the updated current pose of the robotic arm;

[0134] The execution unit is used to return to the operation of performing rotation initialization on the current pose to obtain the rotation deviation, until the target pose is obtained.

[0135] Optionally, the first operation unit includes:

[0136] The first operation subunit is used to perform a rotation initialization operation on the current pose and obtain the set of rotation images corresponding to the rotation initialization operation.

[0137] The first determining subunit is used to determine the rotation deviation based on the set of rotated images and the marker image in the desired pose.

[0138] Optionally, the first operation subunit is specifically used for:

[0139] The robotic arm in the current pose is rotated around the X-axis by a preset number of degrees to obtain a first rotation pose, and a first rotation image corresponding to the first rotation pose is obtained.

[0140] The robotic arm in the first rotation pose is rotated around the Y-axis by the preset degree to obtain the second rotation pose, and the second rotation image corresponding to the second rotation pose is obtained.

[0141] The robotic arm, which is in the second rotation pose, is rotated around the Z-axis by the preset degree to obtain the third rotation pose, and the third rotation image corresponding to the third rotation pose is obtained.

[0142] Optionally, a translation initialization operation is performed on the first pose to obtain the translation deviation, including:

[0143] The second operation subunit is used to perform a translation initialization operation on the first pose and obtain the translation image set corresponding to the translation initialization operation.

[0144] The second determining subunit is used to determine the translation deviation based on the set of translation images and the marker image in the desired pose.

[0145] Optionally, the second operation subunit is specifically used for:

[0146] The robotic arm in the third rotational pose is translated a preset distance along the X-axis to obtain the first translational pose, and the first translational image corresponding to the first translational pose is obtained.

[0147] The robotic arm in the first translational pose is translated along the Y-axis by the preset distance to obtain the second translational pose, and the second translational image corresponding to the second translational pose is obtained.

[0148] The robotic arm in the second translation pose is translated along the Z-axis by the preset distance to obtain the third translation pose, and the third translation image corresponding to the third translation pose is obtained.

[0149] Optionally, the deviation between the marker image in the target pose and the marker image in the desired pose is less than the preset threshold.

[0150] The above-mentioned products can execute the pose determination method provided in any embodiment of the present invention, and have the corresponding functional modules and beneficial effects of executing the pose determination method.

[0151] Example 3

[0152] Figure 3 A schematic diagram of an electronic device 30 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0153] like Figure 3 As shown, the electronic device 30 includes at least one processor 31 and a memory, such as a read-only memory (ROM) 32 or a random access memory (RAM) 33, communicatively connected to the at least one processor 31. The memory stores computer programs executable by the at least one processor. The processor 31 can perform various appropriate actions and processes based on the computer program stored in the ROM 32 or loaded from storage unit 38 into the RAM 33. The RAM 33 can also store various programs and data required for the operation of the electronic device 30. The processor 31, ROM 32, and RAM 33 are interconnected via a bus 34. An input / output (I / O) interface 35 is also connected to the bus 34.

[0154] Multiple components in electronic device 30 are connected to I / O interface 35, including: input unit 36, such as keyboard, mouse, etc.; output unit 37, such as various types of monitors, speakers, etc.; storage unit 38, such as disk, optical disk, etc.; and communication unit 39, such as network card, modem, wireless transceiver, etc. Communication unit 39 allows electronic device 30 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0155] Processor 31 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 31 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 31 performs the various methods and processes described above, such as pose determination methods:

[0156] Obtain the desired pose of the robotic arm and the image of the marker plate under the desired pose;

[0157] Obtain the current pose of the robotic arm and acquire the image of the marker board in the current pose;

[0158] Determine whether the deviation between the marker image in the current pose and the marker image in the desired pose is less than a preset threshold;

[0159] If the deviation is greater than or equal to the preset threshold, the target pose of the robotic arm is determined based on the current pose and the desired pose.

[0160] In some embodiments, the pose determination method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 38. In some embodiments, part or all of the computer program may be loaded and / or mounted on electronic device 30 via ROM 32 and / or communication unit 39. When the computer program is loaded into RAM 33 and executed by processor 31, one or more steps of the pose determination method described above may be performed. Alternatively, in other embodiments, processor 31 may be configured to perform the pose determination method by any other suitable means (e.g., by means of firmware).

[0161] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0162] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0163] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0164] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0165] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0166] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0167] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0168] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A pose determination method, characterized in that, include: Obtain the desired pose of the robotic arm and the image of the marker plate under the desired pose; Obtain the current pose of the robotic arm and acquire the image of the marker board in the current pose; Determine whether the deviation between the marker image in the current pose and the marker image in the desired pose is less than a preset threshold; If the deviation is greater than or equal to the preset threshold, then the target pose of the robotic arm is determined based on the current pose and the desired pose. Determining the target pose of the robotic arm based on the current pose and the desired pose includes: Perform a rotation initialization operation on the current pose to obtain the rotation deviation; The robotic arm is rotated and corrected based on the rotational deviation to obtain the first pose. Perform a translation initialization operation on the first pose to obtain the translation deviation; The robotic arm is translated and corrected based on the translation deviation to obtain the second pose. Determine the rotation control amount and translation control amount based on the second pose and the desired pose; The robotic arm is rotated and translated according to the rotation control amount and the translation control amount, which serves as the updated current pose of the robotic arm. Return to perform a rotation initialization operation on the current pose, obtain the rotation deviation, and continue until the target pose is obtained; The rotation initialization operation performed on the current pose to obtain the rotation deviation includes: Perform a rotation initialization operation on the current pose and obtain the set of rotated images corresponding to the rotation initialization operation; The rotation deviation is determined based on the set of rotated images and the marker image in the desired pose. The process includes performing a rotation initialization operation on the current pose and obtaining the corresponding set of rotated images during the rotation initialization operation, including: The robotic arm in the current pose is rotated around the X-axis by a preset number of degrees to obtain a first rotation pose, and a first rotation image corresponding to the first rotation pose is obtained. The robotic arm in the first rotation pose is rotated around the Y-axis by the preset degree to obtain the second rotation pose, and the second rotation image corresponding to the second rotation pose is obtained. The robotic arm, which is in the second rotation pose, is rotated around the Z-axis by the preset degree to obtain the third rotation pose, and the third rotation image corresponding to the third rotation pose is obtained.

2. The method according to claim 1, characterized in that, Perform a translation initialization operation on the first pose to obtain the translation deviation, including: Perform a translation initialization operation on the first pose and obtain the corresponding set of translation images in the translation initialization operation; The translation deviation is determined based on the set of translation images and the marker image in the desired pose.

3. The method according to claim 2, characterized in that, Perform a translation initialization operation on the first pose and obtain the corresponding set of translation images in the translation initialization operation, including: The robotic arm in the third rotational pose is translated a preset distance along the X-axis to obtain the first translational pose, and the first translational image corresponding to the first translational pose is obtained. The robotic arm in the first translational pose is translated along the Y-axis by the preset distance to obtain the second translational pose, and the second translational image corresponding to the second translational pose is obtained. The robotic arm in the second translation pose is translated along the Z-axis by the preset distance to obtain the third translation pose, and the third translation image corresponding to the third translation pose is obtained.

4. The method according to claim 1, characterized in that, The deviation between the marker image in the target pose and the marker image in the desired pose is less than the preset threshold.

5. A pose determination device, characterized in that, include: The first acquisition module is used to acquire the desired pose of the robotic arm and acquire the image of the marker plate under the desired pose. The second acquisition module is used to acquire the current pose of the robotic arm and acquire the image of the marker board in the current pose. The judgment module is used to determine whether the deviation between the marker image in the current pose and the marker image in the desired pose is less than a preset threshold. The determination module is used to determine the target pose of the robotic arm based on the current pose and the desired pose if the deviation is greater than or equal to the preset threshold. The determining module includes: The first operation unit is used to perform a rotation initialization operation on the current pose to obtain the rotation deviation; The first correction unit is used to correct the rotation of the robotic arm according to the rotational deviation to obtain the first pose. The second operation unit is used to perform a translation initialization operation on the first pose to obtain the translation deviation. The second correction unit is used to perform translation correction on the robotic arm based on the translation deviation to obtain the second pose. A determining unit is configured to determine rotation control quantities and translation control quantities based on the second pose and the desired pose; The control unit is used to perform rotation control and translation control on the robotic arm according to the rotation control amount and the translation control amount, as the updated current pose of the robotic arm; The execution unit is used to return to the operation of performing rotation initialization on the current pose to obtain the rotation deviation, until the target pose is obtained; The first operation unit includes: The first operation subunit is used to perform a rotation initialization operation on the current pose and obtain the set of rotation images corresponding to the rotation initialization operation. The first determining subunit is used to determine the rotation deviation based on the set of rotated images and the marker image in the desired pose. Specifically, the first operation subunit is used for: The robotic arm in the current pose is rotated around the X-axis by a preset number of degrees to obtain a first rotation pose, and a first rotation image corresponding to the first rotation pose is obtained. The robotic arm in the first rotation pose is rotated around the Y-axis by the preset degree to obtain the second rotation pose, and the second rotation image corresponding to the second rotation pose is obtained. The robotic arm, which is in the second rotation pose, is rotated around the Z-axis by the preset degree to obtain the third rotation pose, and the third rotation image corresponding to the third rotation pose is obtained.

6. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the pose determination method according to any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the pose determination method according to any one of claims 1-4.

Images