Hand-eye mark positioning posture sampling method and device, equipment, storage medium and program product

Abstract

The invention relates to a hand-eye mark positioning posture sampling method and device, equipment, a storage medium and a program product. Comprising the steps of generating a multi-dimensional state vector of the intelligent robot in a hand-eye calibration simulation scene according to robot state information of the intelligent robot and visual observation state information of a visual camera; inputting the multi-dimensional state vector into a preset pose sampling decision model to obtain a pose adjustment instruction of the intelligent robot; controlling the intelligent robot to move to a candidate sampling pose according to the pose adjustment instruction; determining calibration parameter characteristic values of the intelligent robot at the candidate sampling poses according to calibration plate images shot by the visual camera at the candidate sampling poses; and under the condition that the iteratively updated calibration parameter characteristic value is greater than a preset calibration parameter characteristic threshold value, taking the candidate sampling pose corresponding to the iteratively updated calibration parameter characteristic value as the target sampling pose of the intelligent robot. By adopting the method, the hand-eye mark positioning posture sampling effect of the intelligent robot is improved.

Description

Technical Field

[0001] This application relates to the field of robot vision calibration technology, and in particular to a hand-eye calibration pose sampling method, device, computer equipment, computer-readable storage medium, and computer program product. Background Technology

[0002] Hand-eye calibration is one of the key technologies for intelligent robots. By determining the relative pose relationship between the intelligent robot's coordinate system and the camera's coordinate system through hand-eye calibration, the intelligent robot can accurately perform visual guidance tasks such as grasping, assembly, or inspection. In the process of hand-eye calibration, the accurate planning and reasonable selection of the intelligent robot's sampling pose are crucial.

[0003] Currently, fixed sampling strategies are commonly used for pose sampling, such as sampling methods based on fixed geometric patterns, random sampling methods, and sampling methods based on simple heuristic rules. However, due to the diversity of hand-eye calibration scenarios, the predefined sampling poses cannot be adapted to the sampling requirements of different hand-eye calibration scenarios, which easily leads to poor calibration accuracy and long calibration time. Therefore, the current effect of hand-eye calibration pose sampling for intelligent robots is poor. Summary of the Invention

[0004] Therefore, it is necessary to provide a hand-eye coordinate positioning posture sampling method, device, computer equipment, computer-readable storage medium, and computer program product to address the above-mentioned technical problems and improve the effect of hand-eye coordinate positioning posture sampling for intelligent robots.

[0005] In a first aspect, this application provides a hand-eye marker positioning pose sampling method, applied to an intelligent robot equipped with a vision camera; comprising:

[0006] Based on the robot state information of the intelligent robot and the visual observation state information of the visual camera, a multi-dimensional state vector of the intelligent robot in the hand-eye calibration simulation scenario is generated.

[0007] The multidimensional state vector is input into a preset pose sampling decision model, and the pose sampling decision is performed on the intelligent robot through the preset pose sampling decision model to obtain the pose adjustment command of the intelligent robot.

[0008] According to the pose adjustment command, control the intelligent robot to move to the candidate sampling pose;

[0009] Based on the calibration board image captured by the visual camera at the candidate sampling pose, the calibration parameter feature values ​​of the intelligent robot at the candidate sampling pose are determined, wherein the calibration parameter feature values ​​represent the confidence level of the calibration parameters.

[0010] Based on the calibration board image and the candidate sampling pose, the calibration parameter feature value is iteratively updated. If the iteratively updated calibration parameter feature value is greater than the preset calibration parameter feature threshold, the candidate sampling pose corresponding to the iteratively updated calibration parameter feature value is taken as the target sampling pose of the intelligent robot.

[0011] In one embodiment, generating a multi-dimensional state vector of the intelligent robot in a hand-eye calibration simulation scenario based on the robot state information of the intelligent robot and the visual observation state information of the visual camera includes:

[0012] Based on the robot state information of the intelligent robot and the visual observation state information of the visual camera, a basic state vector of the intelligent robot in the hand-eye calibration simulation scenario is generated.

[0013] Based on the historical action sequence information and constraint risk state information of the intelligent robot, the basic state vector is optimized to obtain the multidimensional state vector.

[0014] In one embodiment, the constraint risk status information includes at least one of the following: a first distance information between the joint position of the intelligent robot and the joint limit boundary, a second distance information between the intelligent robot and the preset collision detection model, and the field of view coverage information of the calibration plate observed by the visual camera.

[0015] In one embodiment, determining the calibration parameter feature values ​​of the intelligent robot in the candidate sampling pose based on the calibration board image captured by the visual camera in the candidate sampling pose includes:

[0016] Based on the calibration board image captured by the vision camera at the candidate sampling pose, the calibration evaluation items of the intelligent robot at the candidate sampling pose are determined, wherein the calibration evaluation items include at least one of four: accuracy evaluation items, field of view quality evaluation items, constraint evaluation items, and efficiency evaluation items.

[0017] Based on the calibration evaluation terms, a target calibration evaluation function is constructed to evaluate the confidence level of the calibration parameters of the intelligent robot;

[0018] By solving the target calibration evaluation function, the calibration parameter feature values ​​of the intelligent robot under the candidate sampling pose are obtained.

[0019] In one embodiment, constructing a target calibration evaluation function for evaluating the confidence level of the intelligent robot calibration parameters based on the calibration evaluation term includes:

[0020] Obtain the first evaluation weight of the accuracy evaluation item, the second evaluation weight of the field of view quality evaluation item, the third evaluation weight of the constraint evaluation item, and the fourth evaluation weight of the efficiency evaluation item;

[0021] The target calibration evaluation function is obtained by weighting and fusing the accuracy evaluation item, the field of view quality evaluation item, the constraint evaluation item, the efficiency evaluation item, the first evaluation weight, the second evaluation weight, the third evaluation weight, and the fourth evaluation weight.

[0022] In one embodiment, before generating the multidimensional state vector of the intelligent robot in the hand-eye calibration simulation scenario based on the robot state information of the intelligent robot and the visual observation state information of the visual camera, the method further includes:

[0023] A hand-eye calibration simulation model of the intelligent robot is constructed on a preset simulation platform. The hand-eye calibration simulation model includes the kinematic model of the intelligent robot, the intrinsic parameter model of the vision camera, and a calibration board.

[0024] Based on the environmental simulation information of the intelligent robot during the hand-eye calibration process, the kinematic model, the intrinsic parameter model, and the calibration board, a hand-eye calibration simulation scenario for the intelligent robot is constructed.

[0025] Secondly, this application also provides a hand-eye marker positioning pose sampling device for use in an intelligent robot, the intelligent robot being equipped with a vision camera; comprising:

[0026] The generation module is used to generate a multi-dimensional state vector of the intelligent robot in the hand-eye calibration simulation scenario based on the robot state information of the intelligent robot and the visual observation state information of the visual camera.

[0027] The decision module is used to input the multidimensional state vector into a preset pose sampling decision model, and to perform pose sampling decision on the intelligent robot through the preset pose sampling decision model to obtain the pose adjustment command of the intelligent robot.

[0028] The control module is used to control the intelligent robot to move to the candidate sampling pose according to the pose adjustment command;

[0029] The determination module is used to determine the calibration parameter feature values ​​of the intelligent robot in the candidate sampling pose based on the calibration board image captured by the vision camera in the candidate sampling pose, wherein the calibration parameter feature values ​​represent the confidence level of the calibration parameters;

[0030] The planning module is used to iteratively update the feature values ​​of the calibration parameters based on the calibration board image and the candidate sampling poses. If the updated feature values ​​of the calibration parameters are greater than a preset feature threshold, the candidate sampling pose corresponding to the updated feature values ​​of the calibration parameters is used as the target sampling pose of the intelligent robot.

[0031] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0032] Based on the robot state information of the intelligent robot and the visual observation state information of the vision camera, a multi-dimensional state vector of the intelligent robot in the hand-eye calibration simulation scenario is generated. The multi-dimensional state vector is input into a preset pose sampling decision model, and the pose sampling decision model is used to make pose sampling decisions for the intelligent robot to obtain the pose adjustment command of the intelligent robot. According to the pose adjustment command, the intelligent robot is controlled to move to a candidate sampling pose. Based on the calibration board image captured by the vision camera in the candidate sampling pose, the calibration parameter feature value of the intelligent robot in the candidate sampling pose is determined, wherein the calibration parameter feature value represents the confidence level of the calibration parameter. Based on the calibration board image and the candidate sampling pose, the calibration parameter feature value is iteratively updated. If the iteratively updated calibration parameter feature value is greater than a preset calibration parameter feature threshold, the candidate sampling pose corresponding to the iteratively updated calibration parameter feature value is taken as the target sampling pose of the intelligent robot.

[0033] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0034] Based on the robot state information of the intelligent robot and the visual observation state information of the vision camera, a multi-dimensional state vector of the intelligent robot in the hand-eye calibration simulation scenario is generated. The multi-dimensional state vector is input into a preset pose sampling decision model, and the pose sampling decision model is used to make pose sampling decisions for the intelligent robot to obtain the pose adjustment command of the intelligent robot. According to the pose adjustment command, the intelligent robot is controlled to move to a candidate sampling pose. Based on the calibration board image captured by the vision camera in the candidate sampling pose, the calibration parameter feature value of the intelligent robot in the candidate sampling pose is determined, wherein the calibration parameter feature value represents the confidence level of the calibration parameter. Based on the calibration board image and the candidate sampling pose, the calibration parameter feature value is iteratively updated. If the iteratively updated calibration parameter feature value is greater than a preset calibration parameter feature threshold, the candidate sampling pose corresponding to the iteratively updated calibration parameter feature value is taken as the target sampling pose of the intelligent robot.

[0035] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:

[0036] Based on the robot state information of the intelligent robot and the visual observation state information of the vision camera, a multi-dimensional state vector of the intelligent robot in the hand-eye calibration simulation scenario is generated. The multi-dimensional state vector is input into a preset pose sampling decision model, and the pose sampling decision model is used to make pose sampling decisions for the intelligent robot to obtain the pose adjustment command of the intelligent robot. According to the pose adjustment command, the intelligent robot is controlled to move to a candidate sampling pose. Based on the calibration board image captured by the vision camera in the candidate sampling pose, the calibration parameter feature value of the intelligent robot in the candidate sampling pose is determined, wherein the calibration parameter feature value represents the confidence level of the calibration parameter. Based on the calibration board image and the candidate sampling pose, the calibration parameter feature value is iteratively updated. If the iteratively updated calibration parameter feature value is greater than a preset calibration parameter feature threshold, the candidate sampling pose corresponding to the iteratively updated calibration parameter feature value is taken as the target sampling pose of the intelligent robot.

[0037] The aforementioned hand-eye calibration pose sampling method, device, computer equipment, computer-readable storage medium, and computer program product first generate a multi-dimensional state vector of the intelligent robot in the hand-eye calibration scenario using the robot state information of the intelligent robot and the visual observation state information of the visual camera. Then, the multi-dimensional state vector is input into a preset pose sampling decision model. The preset pose sampling decision model samples the intelligent robot's hand-eye calibration pose to obtain the robot's pose adjustment command. Next, based on the pose adjustment command, the intelligent robot is controlled to move to a candidate sampling pose. Then, based on the calibration parameter feature values ​​of the visual camera in the candidate sampling pose, where the calibration parameter feature values ​​represent the confidence level of the calibration parameters, the calibration parameter feature values ​​are iteratively updated based on the calibration board image and the candidate sampling pose. If the iteratively updated calibration parameter feature values ​​are greater than the preset calibration parameters, the iteratively updated calibration parameters are... The candidate sampling pose corresponding to the numerical feature value is used as the target sampling pose of the intelligent robot. Since the calibration parameter feature value of the intelligent robot in the target sampling pose is greater than the preset calibration parameter feature threshold, the calibration parameters obtained by the intelligent robot in the target sampling pose are reliable. At the same time, the planning process of the target sampling pose is implemented in the hand-eye calibration simulation scenario, which can get rid of the limitations of scene adaptability, robot limit and reprojection error in the real hand-eye calibration scenario, and ensure the comprehensiveness of the sampling pose planning, rather than relying on a fixed sampling strategy for pose sampling. Therefore, it overcomes the technical defects that the predefined sampling pose cannot be adapted to the sampling requirements of different hand-eye calibration scenarios due to the diversity of hand-eye calibration scenarios, which leads to poor calibration accuracy and long calibration time. Therefore, it improves the effect of hand-eye calibration pose sampling for intelligent robots. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 This is a flowchart illustrating a hand-eye marker positioning pose sampling method in one embodiment;

[0040] Figure 2 This is a flowchart illustrating the hand-eye marker positioning pose sampling method in another embodiment;

[0041] Figure 3 This is a structural block diagram of a hand-eye marker positioning posture sampling device in one embodiment;

[0042] Figure 4This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0044] First, it should be understood that hand-eye calibration is one of the key technologies of robot vision systems. In the hand-eye calibration process, the accurate planning and reasonable selection of the intelligent robot's sampling pose are crucial. Selecting an appropriate sampling pose determines the upper limit of calibration accuracy, ensures data integrity, adapts to environmental interference and equipment characteristics, and is related to the stability of subsequent tasks. Traditional hand-eye calibration methods typically rely on acquiring multiple sets of camera images and robot pose data, and using mathematical optimization algorithms (such as least squares or Singular Value Decomposition) to perform the calibration. Methods such as decomposition (singular value decomposition) are used to solve calibration parameters. However, calibration accuracy largely depends on the quality of data acquisition, especially the planning method of the visual camera sampling pose. In practical applications, the planning of camera sampling pose often faces many challenges. Due to the diversity of working scenarios (such as object shape, lighting conditions, or occlusion), fixed sampling strategies may not be suitable for all situations. For example, in complex environments, the camera may not be able to completely capture calibration targets such as calibration boards, resulting in incomplete feature point extraction. Or, due to robot joint limitations and kinematic constraints, some preset poses cannot be reached, thus affecting the integrity of data acquisition. In addition, even if the reprojection error is small, the actual calibration error may be large because the fixed sampling strategy does not take into account the specific characteristics of the scene and cannot optimize the sensitivity of calibration parameters to noise. Specifically, predefined, fixed sampling strategies mainly include the following categories: 1) Sampling methods based on fixed geometric patterns, such as pyramid uniform sampling, spherical sampling, or mesh sampling methods. This method is based on fixed geometry within the workspace of the intelligent robot. 1) The structure predefines a set of camera poses and ensures that the calibration board can be observed from different directions and distances. Its core idea is to cover sufficient observation diversity through the spatial uniform distribution of poses, thereby providing sufficient constraints for the calibration equation. However, this method completely ignores the adaptability to specific scenes, calibration targets, and robot motion constraints. 2) Based on random sampling, in order to overcome the systematic bias that may be caused by fixed patterns, Monte Carlo random sampling method can be used. This method randomly generates a large number of poses within the reachable workspace of the intelligent robot and then filters them. Although it increases the randomness of the pose set to a certain extent, it is still a targetless sampling method in essence. It cannot guarantee the efficiency of sampling and the quality of the pose sequence. Usually, a very large number of samples are required to achieve acceptable accuracy, which is inefficient. 3) Based on simple heuristic rules, that is, the operator manually sets a few rules based on experience, such as "avoiding the calibration board from being too small in the image" and "ensuring that the calibration board appears in the center of the field of view", and searches or filters based on these rules. However, this method relies heavily on expert knowledge, is difficult to standardize and promote, and has a low degree of automation.

[0045] In summary, the above sampling methods generally suffer from common defects such as staticity, blindness, and high dependence, failing to autonomously and intelligently plan the sampling pose for the "next best viewpoint" based on perceived information. Specifically, this manifests in the following aspects: 1) Lack of scene adaptability: Fixed sampling strategies (such as pyramid uniform sampling) do not consider changes in the actual scene, such as the shape, size, and position of the calibration target, as well as ambient lighting and occlusion. This may lead to the camera failing to capture the complete calibration target in certain scenarios, or inaccurate feature point extraction, thus affecting calibration accuracy; 2) Robot limitation issues: The preset sampling pose may exceed the robot's joint limits or kinematic constraints, making certain poses unreachable. This not only wastes time but may also interrupt the calibration process, requiring manual intervention to replan the pose, reducing the degree of automation; 3) Inconsistency between reprojection error and actual error: Fixed sampling methods typically aim to minimize reprojection error, but a small reprojection error does not necessarily mean a small actual calibration error. This is because the sensitivity of calibration parameters to noise and outliers varies under different poses, and fixed sampling cannot optimize this sensitivity, which may lead to larger errors in practical applications. 4) Low sampling efficiency: Fixed sampling often requires the collection of a large amount of data to ensure coverage, but many poses may not contribute much to the calibration accuracy, resulting in data redundancy. This increases the time for data acquisition and processing, and reduces the overall calibration efficiency. 5) Lack of dynamic adjustment capability: Fixed sampling is planned offline and cannot dynamically adjust subsequent poses based on the data collected in real time. For example, if the quality of the data collected in the early stage is high, the number of subsequent poses can be appropriately reduced; conversely, if it is found that the data in some areas is insufficient, sampling needs to be increased, but fixed sampling cannot achieve this adaptive adjustment. 6) Lack of dynamic decision-making ability from the next best perspective; Reliance on human experience: Many fixed sampling methods require setting sampling parameters (such as grid density, sampling range, etc.) based on human experience, which introduces subjectivity and requires readjustment in different applications, lacking universality; The core reason is that hand-eye calibration scenarios are diverse, which makes the predefined sampling poses unable to adapt to the sampling requirements of different hand-eye calibration scenarios, thus making it easy to have poor calibration accuracy and long calibration time. Therefore, there is an urgent need for a hand-eye calibration pose sampling method that can improve the effect of hand-eye calibration pose sampling for intelligent robots.

[0046] In one embodiment, such as Figure 1As shown, a hand-eye calibration pose sampling method is provided. This embodiment uses the method applied to a terminal as an example. The terminal is equipped with a hand-eye calibration pose sampling device. The terminal includes, but is not limited to, personal computers, laptops, smartphones, and tablets. The hand-eye calibration pose sampling device includes a generation module, a decision module, a control module, a determination module, and a planning module. The generation module generates a multi-dimensional state vector of the intelligent robot in the hand-eye calibration simulation scenario based on the robot state information of the intelligent robot and the visual observation state information of the vision camera. The decision module inputs the multi-dimensional state vector into a preset pose sampling decision model and performs pose sampling decision on the intelligent robot through the preset pose sampling decision model to obtain the pose adjustment command of the intelligent robot. The control module controls the intelligent robot to move to the candidate sampling pose according to the pose adjustment command. The determination module determines the intelligent robot's position in the candidate sampling pose based on the calibration board image captured by the vision camera in the candidate sampling pose. The calibration parameter feature value under the sampling pose, where the calibration parameter feature value represents the confidence level of the calibration parameter; the planning module is used to iteratively update the calibration parameter feature value according to the calibration board image and candidate sampling pose. When the iteratively updated calibration parameter feature value is greater than the preset calibration parameter feature threshold, the candidate sampling pose corresponding to the iteratively updated calibration parameter feature value is used as the target sampling pose of the intelligent robot. Then, through the information interaction between the generation module, decision module, control module, determination module and planning module, it can be ensured that the calibration parameters calibrated by the intelligent robot under the final planned target sampling pose are reliable. Moreover, the planning process of the target sampling pose is implemented in the hand-eye calibration simulation scenario, thus overcoming the limitations of scene adaptability, robot limit and reprojection error in the real hand-eye calibration scenario, thereby ensuring the comprehensiveness of the sampling pose planning. Therefore, it can achieve the technical effect of sampling the hand-eye calibration pose of the intelligent robot. In this embodiment, the method includes the following steps:

[0047] Step 202: Generate a multi-dimensional state vector of the intelligent robot in the hand-eye calibration simulation scenario based on the robot state information of the intelligent robot and the visual observation state information of the visual camera.

[0048] It should be noted that intelligent robots refer to intelligent robots that integrate visual perception systems and motion execution systems, and can complete specified visual guidance tasks through visual information guidance. Specifically, they can include industrial robotic arms, collaborative robots, and humanoid robots. Visual guidance tasks include, but are not limited to, grasping, assembly, inspection, and sorting tasks. Intelligent robots are equipped with vision cameras. Specifically, vision cameras can be installed on the end effector of intelligent robots so that the vision camera can move synchronously with the end effector of intelligent robots, thereby capturing visual information of the calibration board and the surrounding environment in real time, and providing a basis for solving the relative pose relationship in hand-eye calibration. A vision camera refers to a visual perception device with image acquisition and optical imaging functions, which can convert optical signals into digital image signals. Specifically, it can be an industrial CCD camera or a CMOS camera.

[0049] It should be noted that robot state information represents the real-time motion state of the intelligent robot, specifically including joint angles, joint velocities, joint torques, and end effector poses; visual observation information refers to the calibration board image data and its derived information acquired by the visual camera, specifically including the coordinates of the calibration board feature points, environmental features within the field of view, and image grayscale values; it can be understood that by concatenating the first information vector obtained by mapping the robot state information and the second information vector obtained by mapping the visual observation state information, a multi-dimensional state vector of the intelligent robot in the hand-eye calibration simulation scenario can be obtained. The multi-dimensional state vector is used to fully represent the collaborative correlation features between the intelligent robot's motion state and the visual camera's observation state during the hand-eye calibration process, providing comprehensive input data for the preset pose sampling decision model, specifically represented as a concatenation of [first information vector dimension 1, first information vector dimension 2, ..., second information vector dimension 1, second information vector dimension 2, ...].

[0050] The hand-eye calibration simulation scene representation is based on a high-fidelity digital twin environment constructed using a pre-set high-precision physical simulation platform. It is a 1:1 replication and simulation of the real hand-eye calibration scene. Before generating the multi-dimensional state vector of the intelligent robot in the hand-eye calibration simulation scene based on the robot state information of the intelligent robot and the visual observation state information of the visual camera, the method also includes:

[0051] A hand-eye calibration simulation model for an intelligent robot is constructed on a pre-defined simulation platform. The hand-eye calibration simulation model includes the kinematic model of the intelligent robot, the intrinsic parameter model of the vision camera, and a calibration board. Based on the environmental simulation information, kinematic model, intrinsic parameter model, and calibration board of the intelligent robot during the hand-eye calibration process, a hand-eye calibration simulation scenario for the intelligent robot is constructed.

[0052] It should be noted that the preset simulation platform can be a high-precision physical simulation platform, such as Gazebo, NVIDIA Isaac Sim, or CoppeliaSim. The high-precision physical simulation platform can be used to construct the kinematic model of the intelligent robot, which can include the kinematic model of the intelligent robot, the intrinsic parameter model of the vision camera, and the calibration board. The kinematic model includes accurate replication of DH parameters, joint limits, velocity and acceleration limits, and the intrinsic parameter model includes intrinsic parameters such as lens distortion, focal length and resolution. The calibration board can be a checkerboard or Charuco board of a specific size.

[0053] It should be noted that in order for the simulation model of hand-eye calibration obtained from the simulation to approximate the real conditions, it is also necessary to construct the hand-eye calibration simulation scene by combining the environmental simulation information of the vision machine robot in the hand-eye calibration process. The environmental simulation information may include the dynamic lighting changes in the hand-eye calibration scene, possible random occlusions, and the surface reflection characteristics of calibration plates of different materials.

[0054] As an example, step 202 includes: constructing a hand-eye calibration simulation model of the intelligent robot through a high-precision physical simulation platform, wherein the hand-eye calibration simulation model includes the kinematic model of the intelligent robot, the intrinsic parameter model of the vision camera, and a calibration board; then, by combining the kinematic model, the intrinsic parameter model, the calibration board, and the environmental simulation information of the intelligent robot in the hand-eye calibration process, a hand-eye calibration simulation scene of the intelligent robot is constructed; then, the robot state information of the intelligent robot is mapped into a first information vector, and the visual observation state information of the vision camera is mapped into a second information vector; and the first information vector and the second information vector are concatenated to form a multidimensional state vector of the intelligent robot in the hand-eye calibration simulation scene.

[0055] Step 204: Input the multidimensional state vector into the preset pose sampling decision model, and use the preset pose sampling decision model to perform pose sampling decision on the intelligent robot to obtain the pose adjustment command of the intelligent robot.

[0056] It should be noted that the preset pose sampling decision model can make decisions on the sampled pose of the intelligent robot. Specifically, the preset pose sampling decision model can be equipped with a policy gradient algorithm, such as proximal policy optimization or soft speech-critic algorithm. The above algorithms can achieve a good balance between exploration and exploitation, and the training is stable. It can be understood that the preset pose sampling decision model is a trained decision model. Specifically, in a digital twin environment, the agent and the environment can interact millions of times. Through a large number of training rounds, the agent gradually learns how to select actions from any initial state to maximize the cumulative reward. Since this training stage can be carried out on a simulation platform, there is zero hardware risk. Finally, the trained preset pose sampling decision model can be deployed to a hand-eye calibration pose sampling system (real robot system). During the real calibration process, real state-action-reward data continues to be collected, and online learning algorithms (such as online PPO) are used to make small-scale adjustments to the policy to make up for the gap between simulation and reality. The pose adjustment command is used to instruct the intelligent robot to adjust its pose based on the current sampled pose.

[0057] As an example, step 204 includes: inputting a multidimensional state vector into a preset pose sampling decision model, having the preset pose sampling decision model perform pose sampling decisions on the intelligent robot, and outputting pose adjustment instructions for the intelligent robot.

[0058] For example, in one feasible approach, the multidimensional state vector can be represented as s_t, and the pre-defined pose sampling decision model can be represented as... The output pose adjustment command can be Specifically, it can indicate the displacement and rotation of the end effector of the intelligent robot. After the pose adjustment command, the intelligent robot can move to the sampled pose a_t.

[0059] Step 206: According to the pose adjustment command, control the intelligent robot to move to the candidate sampling pose.

[0060] It should be noted that candidate sampling poses refer to sampling poses waiting to be selected as target sampling poses. Whether a pose is selected as the target sampling pose depends on the subsequent iterative verification process. It can be understood that the pose adjustment command can be a continuous vector, which can specifically represent the relative displacement and rotation increments of the intelligent robot's end effector in the task space, such as Δx, Δy, Δz, Δroll, Δpitch, Δyaw, or it can represent the joint angle increments in the joint space, such as Δq. Through pose adjustment commands, smoother and more precise pose control can be achieved.

[0061] As an example, step 206 includes: controlling the intelligent robot to move and rotate from the current sampled pose based on pose adjustment instructions until it moves to a candidate sampled pose.

[0062] Step 208: Based on the calibration board image captured by the visual camera in the candidate sampling pose, determine the calibration parameter feature values ​​of the intelligent robot in the candidate sampling pose, wherein the calibration parameter feature values ​​represent the confidence level of the calibration parameters.

[0063] It should be noted that since the calibration board image captured by the visual camera in the candidate sampling pose contains the visual mapping relationship between the intelligent robot and the calibration board, the calibration parameter feature values ​​can be geometrically calculated by combining the intrinsic parameters of the visual camera and the kinematic model of the intelligent robot. The calibration board image refers to the high-fidelity digital image acquired by the visual camera for the calibration board in the candidate sampling pose. The calibration parameter feature values ​​are used to quantitatively characterize the "relative pose relationship between the visual camera and the intelligent robot" in hand-eye calibration, and can characterize the confidence level of the calibration parameters.

[0064] As an example, step 208 includes: based on the visual mapping relationship between the intelligent robot and the calibration board carried by the calibration board image, and combining the intrinsic parameters of the visual camera and the kinematic model of the intelligent robot, dynamically solving for the calibration parameter feature values ​​of the intelligent robot in the candidate sampling pose.

[0065] Step 210: Based on the calibration board image and candidate sampling poses, iteratively update the feature values ​​of the calibration parameters. If the updated feature values ​​of the calibration parameters are greater than the preset feature threshold of the calibration parameters, the candidate sampling pose corresponding to the updated feature values ​​of the calibration parameters is taken as the target sampling pose of the intelligent robot.

[0066] It should be noted that after obtaining the calibration parameter feature values, the relationship between the calibration parameter feature values ​​and the preset calibration parameter feature threshold can be iteratively compared to determine whether the currently obtained candidate sampling pose can be used as the target sampling pose. It can be understood that if the current calibration parameter feature value is less than or equal to the preset calibration parameter feature threshold, the acquired calibration board image and candidate sampling pose can be used as the input state for the next round of decision loop, thereby realizing the iterative update of the state until the preset termination condition is reached (e.g., the accuracy is achieved, that is, the calibration parameter feature value after iterative update is greater than the preset calibration parameter feature threshold). Finally, all data collected during the entire iterative loop process are used to complete the high-precision calibration.

[0067] As an example, step 210 includes: iteratively updating the feature values ​​of calibration parameters based on the calibration board image and candidate sampling poses; and if the updated feature values ​​of calibration parameters are greater than a preset feature threshold of calibration parameters, using the candidate sampling pose corresponding to the updated feature values ​​of calibration parameters as the target sampling pose of the intelligent robot.

[0068] In the aforementioned hand-eye calibration pose sampling method, firstly, a multi-dimensional state vector of the intelligent robot in the hand-eye calibration scenario is generated using the robot state information of the intelligent robot and the visual observation state information of the visual camera. Then, the multi-dimensional state vector is input into a preset pose sampling decision model, which samples the intelligent robot's hand-eye calibration pose to obtain the robot's pose adjustment command. Next, based on the pose adjustment command, the intelligent robot is controlled to move to a candidate sampling pose. Then, based on the calibration parameter feature values ​​of the visual camera in the candidate sampling pose, where the calibration parameter feature values ​​represent the confidence level of the calibration parameters, the calibration parameter feature values ​​are iteratively updated based on the calibration board image and the candidate sampling pose. If the iteratively updated calibration parameter feature values ​​are greater than the preset calibration parameters, the candidate sampling pose corresponding to the iteratively updated calibration parameter feature values ​​is selected. As the target sampling pose for the intelligent robot, the calibration parameters obtained by the intelligent robot under the target sampling pose are reliable because the feature values ​​of the calibration parameters under the target sampling pose are greater than the preset feature threshold of the calibration parameters. At the same time, the planning process of the target sampling pose is implemented in the hand-eye calibration simulation scenario, which can get rid of the limitations of scene adaptability, robot limit and reprojection error in the real hand-eye calibration scenario, and ensure the comprehensiveness of the sampling pose planning, rather than relying on a fixed sampling strategy for pose sampling. Therefore, it overcomes the technical defects that the predefined sampling pose cannot be adapted to the sampling requirements of different hand-eye calibration scenarios due to the diversity of hand-eye calibration scenarios, which leads to poor calibration accuracy and long calibration time. Therefore, it improves the effect of hand-eye calibration pose sampling for intelligent robots.

[0069] In one embodiment, such as Figure 2 As shown, based on the robot state information of the intelligent robot and the visual observation state information of the vision camera, a multi-dimensional state vector of the intelligent robot in the hand-eye calibration simulation scenario is generated, including:

[0070] Step 302: Generate the basic state vector of the intelligent robot in the hand-eye calibration simulation scenario based on the robot state information of the intelligent robot and the visual observation state information of the visual camera.

[0071] It should be noted that the basic state vector is generated from robot state information and visual observation state information. The robot state information and visual observation state information can be used to characterize the visual mapping relationship between the intelligent robot and the visual camera and the calibration board, respectively.

[0072] As an example, step 302 includes: mapping robot state information to a first information vector, mapping visual observation state information to a second information vector, and concatenating the first information vector and the second information vector into a base state vector.

[0073] Step 304: Optimize the basic state vector based on the historical action sequence information and constraint risk state information of the intelligent robot to obtain the basic state vector.

[0074] It should be noted that, since multidimensional state vectors constructed solely from robot state information and visual observation state information are insufficient to fully cover the contextual relationships and safety constraints of hand-eye calibration, they are prone to biased pose sampling decisions. To improve the completeness and reliability of multidimensional state vectors, historical action sequence information and constraint risk state information can be incorporated when defining multidimensional state vectors. Historical action sequence information refers to a fixed-length queue of camera pose actions performed by the intelligent robot over the past N time steps, used to provide contextual memory for pose decisions and help avoid repeated sampling or unreasonable pose iterations. Constraint risk state information refers to core data characterizing the intelligent robot's motion safety and the effectiveness of visual observation. Specifically, the first information vector obtained by mapping robot state information can be represented as S_robot, which may include the current joint angle vectors q_t and the end effector's pose T_ee_base in the base coordinate system. The second information vector of visual observation state information can be represented as S_vision, which may include feature information extracted from the current virtual camera image, such as the pixel coordinates (u,v) of all corner points of the calibration board. The third information vector, obtained by mapping the set, the number of successfully detected corners, and the uniformity of corner distribution in the image, can be represented as S_history. It is a fixed-length queue that records the camera pose actions a_{tN}, ..., a_{t-1} executed in the past N time steps, providing contextual memory for the agent. The fourth information vector, obtained by mapping the constraint risk state information, can be represented as S_constraint. The constraint risk state information may include at least one of the following: the first distance information between the joint position and the joint limit boundary of the intelligent robot, the second distance information between the intelligent robot and the preset collision detection model, and the field of view coverage information of the calibration board observed by the visual camera. Specifically, it may be the distance between the current joint position and each joint limit boundary, the distance with the boundary of the preset self-collision detection model, and a binary flag indicating whether the current field of view completely includes the calibration board.

[0075] As an example, step 304 includes: mapping historical action sequence information to a third information vector, and mapping constraint risk state information to a fourth information vector, and concatenating the basic state vector, the third information vector, and the fourth information vector together to form a multi-dimensional state vector. This fully covers the contextual relationships and safety constraints of hand-eye calibration, thereby ensuring the comprehensiveness of pose sampling decisions. Therefore, it further lays the foundation for improving the planning effect of pose sampling for intelligent robots.

[0076] In one embodiment, determining the calibration parameter feature values ​​of the intelligent robot in the candidate sampling pose based on the calibration board image captured by the visual camera in the candidate sampling pose includes:

[0077] Based on the calibration board image captured by the vision camera in the candidate sampling pose, the calibration evaluation terms of the intelligent robot in the candidate sampling pose are determined. The calibration evaluation terms include at least one of four: accuracy evaluation term, field of view quality evaluation term, constraint evaluation term, and efficiency evaluation term. Based on the calibration evaluation terms, a target calibration evaluation function is constructed to evaluate the confidence of the calibration parameters of the intelligent robot. By solving the target calibration evaluation function, the feature values ​​of the calibration parameters of the intelligent robot in the candidate sampling pose are obtained.

[0078] It should be noted that, in order to accurately calculate the calibration parameter feature values ​​of the intelligent robot in the candidate sampling pose; the calibration evaluation item refers to the multi-dimensional index extracted based on the calibration board image and associated data, used to quantify the calibration effect and work condition adaptability; the accuracy evaluation item is used to quantify the accuracy of the relative pose relationship between the visual camera and the robot's base coordinate system in hand-eye calibration; the field of view quality evaluation item is used to evaluate whether the candidate sampling pose meets the robot's motion safety and scene constraints; and the efficiency evaluation item is used to evaluate the calibration efficiency adaptability of the candidate sampling pose.

[0079] As an example, based on the calibration board images captured by a vision camera at candidate sampling poses, the accuracy evaluation term, field of view quality evaluation term, constraint evaluation term, and efficiency evaluation term of the intelligent robot at the candidate sampling poses are dynamically solved. The accuracy evaluation term, field of view quality evaluation term, constraint evaluation term, and efficiency evaluation term are then added together to obtain the target calibration evaluation function used to evaluate the confidence level of the intelligent robot's calibration parameters. By solving the target calibration evaluation function, the feature values ​​of the calibration parameters of the intelligent robot at the candidate sampling poses are obtained. In this way, on the one hand, the target calibration evaluation function is constructed by direct addition, reducing the complexity of solving the objective function in the simulation scenario and improving the efficiency of obtaining the feature values ​​of the calibration parameters, thus adapting to the real-time decision-making needs of hand-eye calibration. On the other hand, the complete integration of the four evaluation terms avoids the one-sidedness caused by single-dimensional evaluation, ensuring that the feature values ​​can truly reflect the calibration effect and the adaptability to the working conditions, providing a reliable basis for sampling pose selection. Therefore, it further lays the foundation for improving the effect of hand-eye calibration pose sampling.

[0080] In one embodiment, a target calibration evaluation function is constructed based on the calibration evaluation term to evaluate the confidence level of the calibration parameters of the intelligent robot, including:

[0081] Obtain the first evaluation weight of the accuracy evaluation item, the second evaluation weight of the field of view quality evaluation item, the third evaluation weight of the constraint evaluation item, and the fourth evaluation weight of the efficiency evaluation item; perform weighted fusion on the accuracy evaluation item, the field of view quality evaluation item, the constraint evaluation item, the efficiency evaluation item, the first evaluation weight, the second evaluation weight, the third evaluation weight, and the fourth evaluation weight to obtain the target calibration evaluation function.

[0082] It should be noted that, in order to adapt to the dynamic needs and core objective differences of hand-eye calibration scenarios and highlight the priority weights of different calibration evaluation items, a differentiated evaluation weight system can be set to make the target calibration evaluation function more in line with the actual calibration accuracy, safety and efficiency requirements; the sum of the first evaluation weight, the second evaluation weight, the third evaluation weight and the fourth evaluation weight is 1, and the specific values ​​of different evaluation weights can be the same or different.

[0083] As an example, the first evaluation weight of the accuracy evaluation item, the second evaluation weight of the field of view quality evaluation item, the third evaluation weight of the constraint evaluation item, and the fourth evaluation weight of the efficiency evaluation item are obtained. The accuracy evaluation item, field of view quality evaluation item, constraint evaluation item, efficiency evaluation item, and the first, second, third, and fourth evaluation weights are then weighted and fused to obtain the target calibration evaluation function. This embodiment adaptively sets evaluation weights for different calibration evaluation items, and then weights and fuses these different calibration evaluation items and their corresponding evaluation weights to obtain the target calibration evaluation function. This achieves a dynamic balance between multi-dimensional evaluation requirements and the accurate implementation of core objectives, avoiding the one-sidedness of a single evaluation dimension and adapting to different simulation conditions. Therefore, this method can significantly improve the scenario adaptability of the target calibration evaluation function, making the solved calibration parameter feature values ​​more reflective of the actual calibration value of candidate sampling poses, providing reliable support for the accurate selection of subsequent target sampling poses.

[0084] In one feasible approach, the first evaluation weight can be 0.4, the second evaluation weight can be 0.15, the third evaluation weight can be 0.25, and the fourth evaluation weight can be 0.2.

[0085] In one feasible approach, the specific steps of hand-eye calibration pose sampling can be as follows: First, a hand-eye calibration simulation model of the intelligent robot is constructed on a preset simulation platform. This model includes the robot's kinematic model, the intrinsic parameter model of the vision camera, and a calibration board. Based on the environmental simulation information, kinematic model, intrinsic parameter model, and calibration board used in the hand-eye calibration process, a hand-eye calibration simulation scenario is constructed. Then, based on the robot's state information and the vision camera's visual observation state information, a basic state vector of the intelligent robot in the hand-eye calibration simulation scenario is generated. Based on the robot's historical action sequence information and constraint risk state information, the basic state vector is optimized to obtain a multi-dimensional state vector. This multi-dimensional state vector is then input into a preset pose sampling decision model. The preset pose sampling decision model performs pose sampling decisions on the intelligent robot to obtain pose adjustment commands. Based on these commands, the intelligent robot is controlled to move to a candidate sampling pose. Further, based on... The calibration board image captured by the visual camera at the candidate sampling pose is used to determine the calibration evaluation terms for the intelligent robot at the candidate sampling pose. These evaluation terms include at least one of four: accuracy evaluation, field of view quality evaluation, constraint evaluation, and efficiency evaluation. Then, the first evaluation weight of the accuracy evaluation, the second evaluation weight of the field of view quality evaluation, the third evaluation weight of the constraint evaluation, and the fourth evaluation weight of the efficiency evaluation are obtained. The accuracy evaluation, field of view quality evaluation, constraint evaluation, efficiency evaluation, first evaluation weight, second evaluation weight, third evaluation weight, and fourth evaluation weight are weighted and fused to obtain the target calibration evaluation function. By solving the target calibration evaluation function, the calibration parameter feature values ​​of the intelligent robot at the candidate sampling pose are obtained. Finally, based on the calibration board image and the candidate sampling pose, the calibration parameter feature values ​​are iteratively updated. If the iteratively updated calibration parameter feature values ​​are greater than a preset calibration parameter feature threshold, the candidate sampling pose corresponding to the iteratively updated calibration parameter feature values ​​is taken as the target sampling pose of the intelligent robot.

[0086] Because the characteristic values ​​of the calibration parameters of the intelligent robot in the target sampling pose are greater than the preset characteristic threshold of the calibration parameters, the calibration parameters obtained by the intelligent robot in the target sampling pose are reliable. At the same time, the planning process of the target sampling pose is implemented in the hand-eye calibration simulation scenario, which can get rid of the limitations of scene adaptability, robot limit and reprojection error in the real hand-eye calibration scenario, and ensure the comprehensiveness of the planning of the sampling pose, rather than relying on a fixed sampling strategy for pose sampling. Therefore, it overcomes the technical defects that the predefined sampling pose cannot be adapted to the sampling requirements of different hand-eye calibration scenarios due to the diversity of hand-eye calibration scenarios, which leads to poor calibration accuracy and long calibration time. Therefore, it improves the effect of hand-eye calibration pose sampling for intelligent robots.

[0087] Understandably, the above implementation addresses the shortcomings of existing technologies through a closed-loop system where the agent makes real-time decisions: 1) Overcoming "fixed sampling, ignoring scene changes": By making decisions based on multi-dimensional real-time states, the agent judges each step based on the latest image features, robot pose, and other states, enabling the sampled pose to dynamically adapt to the specific scene; 2) Overcoming "potential robot limit / collision": By adding strong constraint penalties to the reward function, the agent learns during training to actively avoid planning poses that approach joint limits or may cause collisions; 3) Overcoming "low actual calibration accuracy": By using the reward function to reduce the uncertainty of calibration parameters as the core means, the agent is guided to select poses that can fundamentally improve the accuracy of parameter estimation, rather than simply pursuing the minimum image reprojection error; 4) Overcoming "low sampling efficiency and redundancy": Through Next-Best-View active optimization with "information gain" as the goal, the agent strives to bring the maximum amount of new information at each step, thereby achieving the required accuracy with the fewest samplings and eliminating redundancy.

[0088] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0089] Based on the same inventive concept, this application also provides a hand-eye marker positioning posture sampling device for implementing the hand-eye marker positioning posture sampling method described above. The solution provided by this device is similar to the solution described in the above method. Therefore, the specific limitations of one or more hand-eye marker positioning posture sampling device embodiments provided below can be found in the limitations of the hand-eye marker positioning posture sampling method above, and will not be repeated here.

[0090] In one exemplary embodiment, such as Figure 3 As shown, a hand-eye marker positioning posture sampling device is provided, comprising: a generation module 401, a decision module 402, a control module 403, a determination module 404, and a planning module 405, wherein:

[0091] The generation module 401 is used to generate a multi-dimensional state vector of the intelligent robot in the hand-eye calibration simulation scenario based on the robot state information of the intelligent robot and the visual observation state information of the visual camera.

[0092] The decision module 402 is used to input the multi-dimensional state vector into the preset pose sampling decision model, and to perform pose sampling decision on the intelligent robot through the preset pose sampling decision model to obtain the pose adjustment command of the intelligent robot.

[0093] Control module 403 is used to control the intelligent robot to move to the candidate sampling pose according to the pose adjustment command;

[0094] The determination module 404 is used to determine the calibration parameter feature values ​​of the intelligent robot in the candidate sampling pose based on the calibration board image captured by the vision camera in the candidate sampling pose, wherein the calibration parameter feature values ​​represent the confidence level of the calibration parameters.

[0095] The planning module 405 is used to iteratively update the feature values ​​of the calibration parameters based on the calibration board image and the candidate sampling poses. If the updated feature values ​​of the calibration parameters are greater than the preset feature threshold of the calibration parameters, the candidate sampling pose corresponding to the updated feature values ​​of the calibration parameters will be used as the target sampling pose of the intelligent robot.

[0096] In one embodiment, the generation module 401 is further configured to:

[0097] Based on the robot state information of the intelligent robot and the visual observation state information of the vision camera, a basic state vector of the intelligent robot in the hand-eye calibration simulation scenario is generated; based on the historical action sequence information and constraint risk state information of the intelligent robot, the basic state vector is optimized to obtain a multi-dimensional state vector.

[0098] In one embodiment, the constraint risk status information includes at least one of the following: a first distance information between the joint position of the intelligent robot and the joint limit boundary, a second distance information between the intelligent robot and the preset collision detection model, and the field of view coverage information of the calibration plate observed by the visual camera.

[0099] In one embodiment, the determining module 404 is further configured to:

[0100] Based on the calibration board image captured by the vision camera in the candidate sampling pose, the calibration evaluation terms of the intelligent robot in the candidate sampling pose are determined. The calibration evaluation terms include at least one of four: accuracy evaluation term, field of view quality evaluation term, constraint evaluation term, and efficiency evaluation term. Based on the calibration evaluation terms, a target calibration evaluation function is constructed to evaluate the confidence of the calibration parameters of the intelligent robot. By solving the target calibration evaluation function, the feature values ​​of the calibration parameters of the intelligent robot in the candidate sampling pose are obtained.

[0101] In one embodiment, the determining module 404 is further configured to:

[0102] Obtain the first evaluation weight of the accuracy evaluation item, the second evaluation weight of the field of view quality evaluation item, the third evaluation weight of the constraint evaluation item, and the fourth evaluation weight of the efficiency evaluation item; perform weighted fusion on the accuracy evaluation item, the field of view quality evaluation item, the constraint evaluation item, the efficiency evaluation item, the first evaluation weight, the second evaluation weight, the third evaluation weight, and the fourth evaluation weight to obtain the target calibration evaluation function.

[0103] In one embodiment, the hand-eye marker positioning pose sampling device is also used for:

[0104] A hand-eye calibration simulation model for an intelligent robot is constructed on a pre-defined simulation platform. The hand-eye calibration simulation model includes the kinematic model of the intelligent robot, the intrinsic parameter model of the vision camera, and a calibration board. Based on the environmental simulation information, kinematic model, intrinsic parameter model, and calibration board of the intelligent robot during the hand-eye calibration process, a hand-eye calibration simulation scenario for the intelligent robot is constructed.

[0105] Each module in the aforementioned hand-eye marker positioning and posture sampling device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0106] In one exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 4As shown. The computer device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a hand-eye coordinate positioning posture sampling method. Those skilled in the art will understand that... Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0107] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0108] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0109] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0110] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0111] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0112] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A hand-eye calibration pose sampling method, characterized in that, Applied to intelligent robots, the intelligent robots are equipped with visual cameras; the method includes: Based on the robot state information of the intelligent robot and the visual observation state information of the visual camera, a multi-dimensional state vector of the intelligent robot in the hand-eye calibration simulation scenario is generated. The multidimensional state vector is input into a preset pose sampling decision model, and the pose sampling decision is performed on the intelligent robot through the preset pose sampling decision model to obtain the pose adjustment command of the intelligent robot. According to the pose adjustment command, control the intelligent robot to move to the candidate sampling pose; Based on the calibration board image captured by the visual camera at the candidate sampling pose, the calibration parameter feature values ​​of the intelligent robot at the candidate sampling pose are determined, wherein the calibration parameter feature values ​​characterize the confidence level of the calibration parameters. Based on the calibration board image and the candidate sampling pose, the calibration parameter feature value is iteratively updated. If the iteratively updated calibration parameter feature value is greater than the preset calibration parameter feature threshold, the candidate sampling pose corresponding to the iteratively updated calibration parameter feature value is taken as the target sampling pose of the intelligent robot.

2. The method of claim 1, wherein, The step of generating a multi-dimensional state vector for the intelligent robot in a hand-eye calibration simulation scenario based on the robot state information of the intelligent robot and the visual observation state information of the visual camera includes: Based on the robot state information of the intelligent robot and the visual observation state information of the visual camera, a basic state vector of the intelligent robot in the hand-eye calibration simulation scenario is generated. Based on the historical action sequence information and constraint risk state information of the intelligent robot, the basic state vector is optimized to obtain the multidimensional state vector.

3. The method of claim 2, wherein, The constraint risk status information includes at least one of the following: the first distance information between the joint position and the joint limit boundary of the intelligent robot, the second distance information between the intelligent robot and the preset collision detection model, and the field of view coverage information of the calibration plate observed by the visual camera.

4. The method of claim 1, wherein, The step of determining the calibration parameter feature values ​​of the intelligent robot in the candidate sampling pose based on the calibration board image captured by the visual camera includes: Based on the calibration board image captured by the vision camera at the candidate sampling pose, the calibration evaluation items of the intelligent robot at the candidate sampling pose are determined, wherein the calibration evaluation items include at least one of four: accuracy evaluation items, field of view quality evaluation items, constraint evaluation items, and efficiency evaluation items. Based on the calibration evaluation terms, a target calibration evaluation function is constructed to evaluate the confidence level of the calibration parameters of the intelligent robot; By solving the target calibration evaluation function, the calibration parameter feature values ​​of the intelligent robot under the candidate sampling pose are obtained.

5. The method according to claim 4, characterized in that, The step of constructing a target calibration evaluation function for evaluating the confidence level of the calibration parameters of the intelligent robot based on the calibration evaluation term includes: Obtain the first evaluation weight of the accuracy evaluation item, the second evaluation weight of the field of view quality evaluation item, the third evaluation weight of the constraint evaluation item, and the fourth evaluation weight of the efficiency evaluation item; The target calibration evaluation function is obtained by weighting and fusing the accuracy evaluation item, the field of view quality evaluation item, the constraint evaluation item, the efficiency evaluation item, the first evaluation weight, the second evaluation weight, the third evaluation weight, and the fourth evaluation weight.

6. The method according to claim 1, characterized in that, Before generating the multidimensional state vector of the intelligent robot in the hand-eye calibration simulation scenario based on the robot state information of the intelligent robot and the visual observation state information of the visual camera, the method further includes: A hand-eye calibration simulation model of the intelligent robot is constructed on a preset simulation platform. The hand-eye calibration simulation model includes the kinematic model of the intelligent robot, the intrinsic parameter model of the vision camera, and a calibration board. Based on the environmental simulation information of the intelligent robot during the hand-eye calibration process, the kinematic model, the intrinsic parameter model, and the calibration board, a hand-eye calibration simulation scenario for the intelligent robot is constructed.

7. A hand-eye marker positioning posture sampling device, characterized in that, Applied to intelligent robots, the intelligent robot is equipped with a vision camera; the device includes: The generation module is used to generate a multi-dimensional state vector of the intelligent robot in the hand-eye calibration simulation scenario based on the robot state information of the intelligent robot and the visual observation state information of the visual camera. The decision module is used to input the multidimensional state vector into a preset pose sampling decision model, and to perform pose sampling decision on the intelligent robot through the preset pose sampling decision model to obtain the pose adjustment command of the intelligent robot. The control module is used to control the intelligent robot to move to the candidate sampling pose according to the pose adjustment command; The determination module is used to determine the calibration parameter feature values ​​of the intelligent robot in the candidate sampling pose based on the calibration board image captured by the vision camera in the candidate sampling pose, wherein the calibration parameter feature values ​​represent the confidence level of the calibration parameters; The planning module is used to iteratively update the feature values ​​of the calibration parameters based on the calibration board image and the candidate sampling poses. If the updated feature values ​​of the calibration parameters are greater than a preset feature threshold, the candidate sampling pose corresponding to the updated feature values ​​of the calibration parameters is used as the target sampling pose of the intelligent robot.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images