Robotic calibration methods, systems, and devices

Abstract

The present specification provides a robot calibration method, system and device. Based on the method, during specific calibration, the target operation type of a target operation can be determined first; then, according to the target operation type of the target operation, a matched target is selected from a preset target set and is respectively arranged on a first mechanical arm and a second mechanical arm as a first target and a second target; wherein the preset target set contains multiple targets with different structural shapes and provided with identification codes; according to a preset calibration rule, a calibration camera and a robot are used for calibration processing to determine the conversion relationship of the key points of a target operation instrument about the second target coordinate system and the conversion relationship of the image coordinate system of an image acquisition end about the first target coordinate system as a calibration result. Thus, the related calibration can be automatically completed accurately, efficiently and pertinently to obtain a calibration result meeting the requirements and reduce calibration errors.

Description

Technical Field

[0001] This manual pertains to the field of robotics technology, and particularly relates to robot calibration methods, systems, and devices. Background Technology

[0002] With the development and popularization of robotics technology, robotic arms are increasingly being used in more and more application scenarios to control corresponding instruments and complete specific engineering operations.

[0003] Before using a robot, it is usually necessary to calibrate it so that it can automatically locate itself during engineering operations. However, existing calibration methods often suffer from problems such as cumbersome and complex calibration processes, poor calibration results, and the calibration results being easily constrained by the position of the calibration camera, affecting subsequent flexible use.

[0004] There is currently no effective solution to the above problems. Summary of the Invention

[0005] This manual provides a robot calibration method, system, and apparatus that can accurately, efficiently, and purposefully complete relevant calibrations automatically, obtain calibration results that meet requirements, and reduce calibration errors.

[0006] This specification provides a robot calibration method. The robot includes at least a first robotic arm and a second robotic arm. The first robotic arm is equipped with an image acquisition terminal, and the second robotic arm is equipped with a target manipulation device. The method includes: determining the target operation type of the target operation; selecting matching targets from a preset target set according to the target operation type and setting them on the first and second robotic arms respectively, as the first target and the second target; wherein the preset target set includes multiple targets with different structural shapes and identification codes; and performing calibration processing using a calibration camera and the robot according to preset calibration rules to determine the transformation relationship of the key points of the target manipulation device with respect to the second target coordinate system, and the transformation relationship of the image coordinate system of the image acquisition terminal with respect to the first target coordinate system, as the calibration result.

[0007] This specification also provides a robot calibration system, which includes at least: an external optical platform, a robot, and a server; wherein the robot includes a first robotic arm and a second robotic arm, the first robotic arm being equipped with an image acquisition terminal, and the second robotic arm being equipped with a target manipulation device; the external optical platform includes at least a calibration camera; the server uses the robot calibration method to control the robot and the external optical platform to perform specific calibration processing and obtain the corresponding calibration results.

[0008] This specification also provides a robot calibration device, the robot comprising a first robotic arm and a second robotic arm, the first robotic arm being equipped with an image acquisition terminal, and the second robotic arm being equipped with a target manipulation device. The device includes: a determination module for determining the target operation type of the target operation; a filtering module for selecting matching targets from a preset target set according to the target operation type of the target operation and setting them on the first and second robotic arms respectively, as the first target and the second target; wherein the preset target set includes multiple targets with different structural shapes and identification codes; and a calibration module for performing calibration processing using a calibration camera and the robot according to preset calibration rules, determining the transformation relationship of the key points of the target manipulation device with respect to the second target coordinate system, and the transformation relationship of the image coordinate system of the image acquisition terminal with respect to the first target coordinate system, as the calibration result.

[0009] This specification also provides another robot calibration method, wherein the robot includes a target manipulator arm, the target manipulator arm is equipped with a target manipulator, and the method includes: determining the target operation type of the target operation; selecting matching targets from a preset target set according to the target operation type and setting them on the target manipulator arm as target targets; wherein the preset target set includes multiple targets with different structural shapes and identification codes; and performing calibration processing using a calibration camera and the robot according to preset calibration rules to determine the transformation relationship of the key points of the target manipulator with respect to the target coordinate system based on the target targets, as the calibration result.

[0010] This specification also provides a computer-readable storage medium having computer instructions stored thereon, which, when executed, implement the relevant steps of the robot calibration method.

[0011] Based on the robot calibration method, system, and apparatus provided in this specification, during specific calibration, the target operation type of the target operation to be performed by the robot can be determined first. Then, according to the target operation type, matching targets are selected from a preset target set and respectively set on the first and second robotic arms as the first and second targets. The preset target set includes multiple targets with different structural shapes and identification codes. According to preset calibration rules, calibration processing is performed using a calibration camera and the robot to determine the transformation relationship of the key points of the target operating device with respect to the second target coordinate system, and the transformation relationship of the image coordinate system of the image acquisition end with respect to the first target coordinate system, as the calibration result. This effectively simplifies the calibration process of robots and other operating equipment, accurately and efficiently completes the relevant calibration automatically, obtains the required calibration results, and reduces calibration errors. Furthermore, when using the calibration results obtained in the above manner to locate and monitor the key points of the target manipulator, it is no longer constrained by the position of the calibration camera. The position coordinates of the key points of the target manipulator based on the image coordinate system of the image acquisition end can be accurately determined, making the location and monitoring of the key points of the target manipulator more flexible and convenient during specific operations, thus adapting to diverse application scenarios. Further, by first determining the target operation type to be performed by the robot; and based on the target operation type, different situations are distinguished to determine the matching target and / or target movement path; then, based on the matching target and / or target movement path, the robot is calibrated in a targeted manner, making the calibration process more targeted, effectively improving calibration accuracy, and further reducing calibration errors. In addition, by introducing and using relatively small, custom targets with identification codes to replace conventional, large, inconvenient, and homogeneous calibration boards for specific calibration, the calibration effect can be effectively improved, further enhancing calibration accuracy. Attached Figure Description

[0012] To more clearly illustrate the embodiments of this specification, the accompanying drawings used in the embodiments will be briefly introduced below. The drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a flowchart illustrating a robot calibration method provided in one embodiment of this specification;

[0014] Figure 2 This is a schematic diagram of an embodiment of the structure of a robot calibration system that applies the robot calibration method provided in the embodiments of this specification in a scenario example;

[0015] Figure 3 This is a schematic diagram of an embodiment of the structural composition of a robot in a robot calibration system, presented in a scenario example.

[0016] Figure 4 This is a schematic diagram of an embodiment in which the robot calibration method provided in the embodiments of this specification is applied to determine the target operation type of a target operation in a scenario example;

[0017] Figure 5 This is a schematic diagram illustrating an embodiment of the robot calibration method provided in this specification, using various custom targets, within a specific scenario example.

[0018] Figure 6 This is a schematic diagram of an embodiment of determining a target movement path using the robot calibration method provided in the embodiments of this specification in a scenario example;

[0019] Figure 7 This is a schematic diagram of an embodiment of the self-test result interface displayed on a monitor, which applies the robot calibration method provided in the embodiments of this specification in a scenario example.

[0020] Figure 8 This is a schematic diagram of an embodiment of the robot calibration method provided in the embodiments of this specification, which is used to control the movement of a robotic arm while acquiring calibration images and robot motion control terminal parameters in a scenario example.

[0021] Figure 9 This is a schematic diagram of an embodiment of various manipulators involved in the robot calibration method provided in the embodiments of this specification, in a scenario example.

[0022] Figure 10 This is a schematic diagram of an embodiment in which the target instrument involved in the robot calibration method provided in this specification includes multiple key points, as shown in a scenario example.

[0023] Figure 11 This is a schematic diagram of the structural composition of a server provided in one embodiment of this specification;

[0024] Figure 12 This is a schematic diagram of the structural composition of a robot calibration device provided in one embodiment of this specification.

[0025] Figure 13 This is a flowchart illustrating a robot calibration method provided in another embodiment of this specification. Detailed Implementation

[0026] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0027] See Figure 1 As shown in the embodiments of this specification, a robot calibration method is provided. The robot includes a first robotic arm and a second robotic arm. The first robotic arm is equipped with an image acquisition terminal, and the second robotic arm is equipped with a target manipulation device. In specific implementations, this method may include the following:

[0028] S101: Determine the target operation type of the target operation;

[0029] S102: Based on the target operation type of the target operation, select matching targets from a preset target set and set them on the first robotic arm and the second robotic arm respectively, as the first target and the second target; wherein, the preset target set includes multiple targets with different structural shapes and set with identification codes;

[0030] S103: According to the preset calibration rules, calibration is performed using a calibration camera and a robot to determine the transformation relationship of the key points of the target operating instrument with respect to the second target coordinate system, and the transformation relationship of the image coordinate system of the image acquisition end with respect to the first target coordinate system, as the calibration result.

[0031] In some embodiments, the above-described robot calibration method can be specifically applied to the server side. See also... Figure 2 As shown, the server can be connected to the robot and the calibration camera. Before the robot performs a target operation, the server can first calibrate the robot according to the robot calibration method provided in this manual.

[0032] For details, please refer to Figure 3 As shown, the robot described above can include at least two robotic arms: a first robotic arm and a second robotic arm. The first robotic arm is used to mount the robot's image acquisition device, and the second robotic arm is used to mount the target manipulation instrument for specific engineering operations.

[0033] Furthermore, the aforementioned second robotic arm may further include one or more independent robotic arms. The target manipulator mounted on the second robotic arm may specifically include one or more different manipulators.

[0034] Furthermore, the aforementioned robot can be of different types depending on the application scenario. For example, in a construction scenario, the robot can be a construction robot; in an assembly line production scenario, it can be a production robot; in a medical scenario, it can be a surgical robot, etc. It should be noted that the application scenarios and robots listed above are merely illustrative. In specific implementations, other types of robots in other application scenarios may also be included, depending on the specific circumstances. Moreover, the aforementioned robot calibration method can be extended to other operating devices besides robots, including first and second robotic arms, and equipped with image acquisition terminals and target manipulation instruments. This specification does not limit its application in this regard.

[0035] For details, please refer to Figure 2 As shown, the calibration camera mentioned above can specifically include an external camera located outside the robot, such as a camera mounted on an external optical platform. Of course, depending on the specific circumstances, the calibration camera can also be a camera mounted on the robot.

[0036] Specifically, the aforementioned external optical platform may further include a display for user interaction. Furthermore, the external optical platform may also include a target storage area and a server placement area. See also... Figure 2 As shown, the server can be specifically deployed in the server placement area of ​​an external optical platform.

[0037] This manual primarily uses surgical robots applied in medical settings as an example for detailed explanation. For other types of robots in other application scenarios, please refer to the relevant embodiments of surgical robots in medical settings, which will not be elaborated upon in this manual.

[0038] Specifically, in a medical setting, the aforementioned robot can be a patient surgery platform facing the patient awaiting surgery. The image acquisition device mounted on the first robotic arm can be an endoscope, camera, or other related equipment. The target manipulation instrument mounted on the second robotic arm can be a surgical instrument used during the surgical procedure, such as a duckbill forceps, a rat-tooth forceps, or a powerful duckbill forceps.

[0039] In some embodiments, the target operation type of the target operation to be performed by the robot can be determined first.

[0040] For details, please refer to Figure 4 As shown, users can interact with the position optical platform system interface displayed on the external optical platform's monitor to input the target operation type.

[0041] For example, the monitor can display a list of operation type options to the user. The user can then select an operation type from this list based on their specific needs and circumstances. Correspondingly, the server can determine the user's selected target operation type using the operation data collected by the monitor on the external optical platform.

[0042] Taking a medical setting as an example, the target operation can be a target surgical procedure, and the target operation type can be a target surgical procedure type. Accordingly, the aforementioned target operation type can specifically include any one of the following: gallbladder surgery, liver surgery, kidney surgery, prostate surgery, stomach surgery, pancreatic surgery, bladder surgery, uterine surgery, rectal surgery, adrenal gland surgery, renal pelvis and ureter surgery, etc.

[0043] In some embodiments, during specific implementation, matching targets can be selected from a preset target set based on the target operation type of the target operation; see reference Figure 3 As shown, matching targets are then set on the first and second robotic arms respectively to obtain the corresponding first and second targets.

[0044] Specifically, the aforementioned preset target set may include multiple custom targets with different structural shapes and identification codes.

[0045] See Figure 5 As shown, it should be noted that the targets in the above-mentioned preset target set can be custom targets that are different from the conventional calibration boards containing chessboard patterns.

[0046] Specifically, firstly, the aforementioned targets are smaller than conventional calibration plates, allowing for more convenient and flexible placement on the robotic arm. Secondly, the targets can be equipped with one or more identification codes (e.g., QR codes), which are smaller than conventional checkerboard patterns, facilitating accurate identification and determination during calibration. Furthermore, the targets include various structural shapes to accommodate different types of engineering operations, ensuring that the calibration camera can successfully acquire images containing the identification codes during calibration for different operation types, achieving better labeling results.

[0047] In some embodiments, see Figure 5 As shown, the targets in the preset target set may specifically include at least one of the following: umbrella-shaped targets, stepped surround targets, spherical targets, Y-shaped targets, cross-shaped targets, etc. Among them, Y-shaped targets and cross-shaped targets are planar targets; umbrella-shaped targets, stepped surround targets, and spherical targets are three-dimensional targets.

[0048] Before implementation, a large number of operation records of different types of engineering operations can be obtained. Based on the operation records, the movement range and action characteristics of different types of engineering operations can be sorted out and determined. Then, based on the movement range and action characteristics of different types of engineering operations, combined with the structural features of targets with different structural shapes, the appropriate structural shape of the target can be determined when performing calibration of different types of engineering operations, and a preset matching relationship table can be constructed.

[0049] In practice, the server can determine the target that matches the target operation type by querying a preset matching table.

[0050] Specifically, taking medical scenarios as an example, based on a preset matching table, for operation types such as stomach surgery, pancreatic surgery, and liver surgery, the matching target can be a step-shaped target; for operation types such as prostate surgery, bladder surgery, rectal surgery, and uterine surgery, the matching target can be an umbrella-shaped target or a spherical target; for operation types such as kidney surgery, adrenal gland surgery, and renal pelvis and ureter surgery, the matching target can be a Y-shaped target or a cross-shaped target.

[0051] In some embodiments, see Figure 3 As shown, after selecting matching targets from a preset target set, these targets can be respectively set on the mechanical boxes of the first and second robotic arms, serving as the first and second targets. Alternatively, in specific implementations, the targets can be set at other locations on the robotic arms, such as the ends of the first and second robotic arms.

[0052] In some embodiments, the calibration process, performed using a calibration camera and a robot according to preset calibration rules, determines the transformation relationship of the key points of the target operating instrument with respect to the second target coordinate system, and the transformation relationship of the image coordinate system of the image acquisition end with respect to the first target coordinate system. In specific implementations, this may include the following:

[0053] S1: According to the preset calibration rules, control the robot to move the first robotic arm and / or the second robotic arm; and acquire calibration images during the movement process through the calibration camera, and obtain the robot motion control terminal parameters through the robot;

[0054] S2: Based on the calibration image and the robot motion control terminal parameters, determine the transformation relationship of the key points of the target operating instrument with respect to the second target coordinate system, and the transformation relationship of the image coordinate system of the image acquisition terminal with respect to the first target coordinate system.

[0055] In practice, firstly, the server can control the robot's motion control terminal to move the first and / or second robotic arms to change their poses. Simultaneously, the server controls a calibration camera to capture images containing the first and / or second targets at different poses during the movement, serving as calibration images. Furthermore, the server receives robot motion control terminal parameters sent at different poses during the robot's movement. These parameters include at least: robot-related state data, such as the position coordinates of the identifier code in the first target based on the coordinate system of the first robotic arm's tail end, the position coordinates of the identifier code in the second target based on the coordinate system of the second robotic arm's tail end, and parameters characterizing the pose relationship of the first and second robotic arms relative to the robot's base. Then, based on the calibration images and robot motion control terminal parameters, the server can determine the transformation relationship of the key points of the target manipulation device with respect to the second target coordinate system, and the transformation relationship of the image coordinate system of the image acquisition terminal with respect to the first target coordinate system, using calibration processing such as hand-eye calibration, as the required calibration result.

[0056] In some embodiments, the above-mentioned control of the robot to move the first robotic arm and / or the second robotic arm according to a preset calibration rule may specifically include:

[0057] S1: Determine the matching target movement path based on the target operation type;

[0058] S2: Control the robot to move the first robotic arm and / or the second robotic arm according to the target movement path.

[0059] In some embodiments, the target movement path may specifically include a preset movement path or a user-defined movement path. The target movement path is a movement control path that comprehensively covers the poses (including position and attitude) involved in the specific implementation of the corresponding target operation.

[0060] In some embodiments, during implementation, the server can determine a matching preset movement path from a preset movement path set based on the target operation type, and use this as the target movement path. The preset movement path set may contain multiple preset movement paths corresponding to multiple operation types. Specifically, the preset movement paths may be determined by pre-learning and organizing the movement paths in the operation records of different operation types.

[0061] In some embodiments, see Figure 6As shown, the external optical platform's display can also show users a custom path settings interface. Users can use this interface to customize their movement path according to their specific needs. Correspondingly, the server can obtain the user-defined movement path through this interface and use it as the target movement path.

[0062] In addition, in the custom path settings interface mentioned above, users can also select a default path to trigger the display of multiple preset movement paths. Users can select one of the displayed preset movement paths as their target movement path. Users can also select a preset movement path from the displayed preset movement paths and make custom adjustments based on that preset movement path to obtain a target movement path that meets their personalized needs.

[0063] In some embodiments, the server may also adjust the placement of the external optical platform relative to the robot and / or the placement of the calibration camera on the external optical platform according to the target operation type, so as to ensure that the calibration camera can successfully acquire a clear calibration image containing the identification code on the target during the subsequent calibration process.

[0064] In some embodiments, before controlling the robot to move the first and / or second robotic arms, the server can automatically detect the device status of the robot, calibration camera, and other related equipment to determine whether the robot can be normally controlled to move the first and / or second robotic arms for calibration. If the self-detection determines that the device status of the robot, calibration camera, and other related equipment is normal, refer to... Figure 7 As shown, the pre-calibration self-test results interface can be displayed to the user on the monitor to inform the user that the current device status has passed the test and subsequent calibration can proceed normally. In addition, the pre-calibration self-test results interface can also display prompts during the calibration process, such as warnings that the calibration camera should not be moved during calibration, so that the user can complete the subsequent calibration normally.

[0065] In some embodiments, the robot is controlled to move the first robotic arm and / or the second robotic arm; and calibration images are acquired during the movement using a calibration camera to obtain robot motion control parameters. Specifically, this may include the following:

[0066] When the robot moves the first and / or second robotic arms to the current pose according to the target movement path, the robot sends a robot message to the intermediate communication module to trigger the calibration camera to acquire the corresponding calibration image. The intermediate communication module then obtains the corresponding robot motion control parameters. The calibration camera receives and responds to the robot message through the intermediate communication module, acquiring the calibration image corresponding to the current pose. The robot motion control parameters are used to determine the transformation relationship between the robot's base coordinate system and the coordinate system of the first robotic arm's tail, and the transformation relationship between the robot's base coordinate system and the coordinate system of the second robotic arm's tail.

[0067] Specifically, the aforementioned intermediate communication module can be the server's MQTT Broker. Accordingly, the server can use the MQTT Broker to interact with the robot's motion control terminal and the camera of the external optical platform based on the MQTT protocol (Message Queuing Telemetry Transport Protocol). This allows the server to acquire the necessary calibration images and robot motion control terminal parameters during the process of controlling the robot to move the first and / or second robotic arms according to the target movement path.

[0068] For details, please refer to Figure 8 As shown, the robot's motion control terminal can publish robot messages via the MQTT Broker and also subscribe to and obtain path information from the MQTT Broker. The server can publish path information for each step via the MQTT Broker and also subscribe to and obtain robot messages from the MQTT Broker. The robot messages include at least: robot status messages and robot motion control terminal parameters.

[0069] In practice, the server can first connect to the calibration camera and the robot, and initialize the parameters; then start the MQTT service. During movement, the server can determine the path information for the current step (e.g., the nth step) based on the target movement path, and publish the path information for the current step to the MQTT Broker. The robot's motion control terminal can obtain the current path information through the MQTT Broker, and based on the current path information, move the corresponding robotic arm (including the first robotic arm and / or the second robotic arm) to the current pose; then update the robot's state information, and based on the updated robot state information, publish the motion control terminal parameters as a robot message to the MQTT Broker. Correspondingly, the server can obtain the robot message through the MQTT Broker to get the current motion control terminal parameters; and based on the robot message, determine whether the motion control terminal has completed the movement of the current step; if it is determined that the movement of the current step has been completed, control the calibration camera to acquire the current calibration image, and store the current calibration image and the current motion control terminal parameters. Next, the server can check whether the target movement path has been completed. If it is determined that the target movement path has been completed, the first robotic arm and / or the second robotic arm will move, and calibration images and robot motion control parameters at different poses during the movement will be collected and stored. Conversely, if it is determined that the target movement path has not been completed, the above process can be repeated to proceed to the next step (e.g., the n+1th step) until the target movement path is completed.

[0070] In some embodiments, the above-described transformation relationship of the key points of the target manipulator with respect to the second target coordinate system and the transformation relationship of the image coordinate system of the image acquisition terminal with respect to the first target coordinate system, based on the calibration image and the robot motion control terminal parameters, may specifically include the following:

[0071] S1: Based on the calibration images, the intrinsic parameter data of the calibration camera, and the parameters of the robot motion control terminal, determine the first extrinsic parameter data of the calibration camera with respect to the first target, and the second extrinsic parameter data of the calibration camera with respect to the second target;

[0072] S2: Based on the first external parameter data, obtain the transformation relationship between the first target coordinate system and the calibration camera coordinate system (e.g., the coordinate system transformation matrix T). target_endo2cam Based on the second extrinsic parameter data, obtain the transformation relationship between the second target coordinate system and the calibration camera coordinate system (e.g., the coordinate system transformation matrix T). target_tool2cam );

[0073] S3: Determine the transformation relationship between the robot base coordinate system and the first robotic arm tail coordinate system based on the robot motion control parameters (e.g., coordinate transformation matrix T). base2end_endoThe transformation relationship between the robot's base coordinate system and the coordinate system of the tail end of the second robotic arm (e.g., the coordinate transformation matrix T). base2end_tool The transformation relationship between the first target coordinate system and the calibration camera coordinate system, and the transformation relationship between the second target coordinate system and the calibration camera coordinate system are determined. The transformation relationship between the calibration camera coordinate system and the robot base coordinate system is then obtained (e.g., the coordinate transformation matrix T). cam2base The transformation relationship between the first target coordinate system and the tail coordinate system of the first robotic arm (e.g., the coordinate transformation matrix T). target_endo2end_endo The transformation relationship between the second target coordinate system and the tail coordinate system of the second robotic arm (e.g., the coordinate transformation matrix T). target_tool2end_tool );

[0074] S4: Based on the transformation relationship between the calibrated camera coordinate system and the robot base coordinate system, and the transformation relationship between the first target coordinate system and the tail coordinate system of the first robotic arm, determine the transformation relationship of the key points of the target manipulator with respect to the first target coordinate system (e.g., coordinate transformation matrix T). endo2target_endo Based on the transformation relationship between the calibrated camera coordinate system and the robot base coordinate system, and the transformation relationship between the second target coordinate system and the tail coordinate system of the second robotic arm, the transformation relationship of the image coordinate system of the image acquisition end with respect to the second target coordinate system is determined (e.g., transformation matrix P). intarget_tool ).

[0075] Specifically, the first target coordinate system can be understood as a world coordinate system based on the first target, and the second target coordinate system can be understood as a world coordinate system based on the second target.

[0076] In some embodiments, during specific implementation, the position coordinates of the identifier code in the first target based on the coordinate system of the first robotic arm tail end and the position coordinates of the identifier code in the second target based on the coordinate system of the second robotic arm tail end can be obtained according to the robot motion control terminal parameters. At the same time, the transformation relationship between the robot base coordinate system and the coordinate system of the first robotic arm tail end and the transformation relationship between the robot base coordinate system and the coordinate system of the second robotic arm tail end can also be obtained according to the robot motion control terminal parameters.

[0077] In addition, the server can efficiently obtain high-precision intrinsic parameter data of the calibration camera by querying the built-in SDK of the calibration camera.

[0078] In some embodiments, during implementation, the server can first find the corresponding (e.g., corresponding to the same pose) calibration image, the position coordinates of the identifier code in the first target based on the coordinate system of the first robotic arm's tail end, and the position coordinates of the identifier code in the second target based on the coordinate system of the second robotic arm's tail end. The server can first process the calibration image to determine the pixel coordinates of the identifier code in the first target and the pixel coordinates of the identifier code in the second target in the calibration image; then, based on the intrinsic parameter data of the calibration camera, convert the pixel coordinates of the identifier code in the first target and the pixel coordinates of the identifier code in the second target into position coordinates of the identifier code in the first target based on the coordinate system of the calibration camera and position coordinates of the identifier code in the second target based on the coordinate system of the calibration camera.

[0079] Next, the server can combine the corresponding identifier in the first target with the position coordinates of the calibration camera coordinate system and the position coordinates of the identifier in the first target with the position coordinates of the tail end coordinate system of the first robotic arm to obtain the first extrinsic parameter data of the calibration camera with respect to the first target; and combine the corresponding identifier in the second target with the position coordinates of the calibration camera coordinate system and the position coordinates of the identifier in the second target with the position coordinates of the tail end coordinate system of the second robotic arm to obtain the second extrinsic parameter data of the calibration camera with respect to the second target.

[0080] Furthermore, the transformation relationship between the first target coordinate system and the calibration camera coordinate system can be extracted based on the first extrinsic parameter data; and the transformation relationship between the second target coordinate system and the calibration camera coordinate system can be obtained based on the second extrinsic parameter data.

[0081] Then, based on the robot motion control terminal parameters, the transformation relationships between the robot base coordinate system and the first robotic arm tail coordinate system, the robot base coordinate system and the second robotic arm tail coordinate system, the first target coordinate system and the calibration camera coordinate system, and the second target coordinate system and the calibration camera coordinate system can be determined, and corresponding coordinate system transformation formulas can be constructed. Then, based on the coordinate transformation formulas, using the known coordinate system transformation relationships, the transformation relationships between the calibration camera coordinate system and the robot base coordinate system, the first target coordinate system and the first robotic arm tail coordinate system, and the second target coordinate system and the second robotic arm tail coordinate system can be solved.

[0082] Specifically, for example, regarding the first target and the second target, the following coordinate system transformation relationships can be constructed respectively:

[0083] T cam2base =T base2end_endo T target_endo2end_endo T target_endo2cam

[0084] T cam2base =T base2end_tool T target_tool2end_toolT target_tool2cam .

[0085] It should be noted that during the movement of the robotic arm during calibration, the relative positions of the external optical platform and the robot are fixed, as are the relative positions of the first target and the tail end of the first robotic arm, and the relative positions of the second target and the tail end of the second robotic arm. Therefore, T cam2base T target_endo2end_endo T target_tool2end_tool It is also fixed. Therefore, the aforementioned positional relationship characteristics can be utilized, using multiple sets of varying T values. target_endo2cam T base2end_endo T target_tool2cam T base2end_tool Solve the system of equations simultaneously to obtain the corresponding values ​​of T. target_endo2end_endo T target_tool2end_tool , and T cam2base .

[0086] Finally, using the above-mentioned transformation relationships between coordinate systems, the transformation relationships of the image coordinate system of the image acquisition end with respect to the first target coordinate system and the transformation relationships of the key points of the target operating instrument with respect to the second target coordinate system can be determined according to the following formulas, as the final calibration results:

[0087] T endo2target_endo =T target_endo2cam T cam2base T base2endo

[0088] P intarget_tool =T target_tool2cam T cam2base P inbase

[0089] Among them, T base2endo P represents the transformation relationship between the robot's base coordinate system and the image coordinate system. inbase The key points of the target manipulator are represented by their position coordinates based on the robot's base coordinate system.

[0090] Based on the transformation relationships of the key points of the target operating instrument with respect to the second target coordinate system and the transformation relationships of the image coordinate system of the image acquisition end with respect to the first target coordinate system, and utilizing the aforementioned positional relationship characteristics, position coordinates based on the image coordinate system, independent of the calibration camera, can be calculated. This allows for the transformation of the key points of the target operating instrument into the image coordinate system for positioning and monitoring, effectively avoiding the influence and constraints of the calibration camera's position. Correspondingly, after completing the calibration and obtaining the corresponding calibration results in the above manner, even if the position of the calibration camera is moved, it will not affect the positioning and monitoring of the key points of the target operating instrument. Therefore, it can be adapted to diverse application scenarios and meet different user needs.

[0091] After obtaining the corresponding calibration results in the above manner, the calibration results can be stored to complete the relevant calibration.

[0092] In some embodiments, the second robotic arm may specifically include multiple robotic arms; the target operating device may specifically include a variety of operating devices.

[0093] For details, please refer to Figure 9 As shown, the second robotic arm can carry different types of operating instruments.

[0094] Specifically, for example, when performing engineering operations of a target operation type, only one second robotic arm might be used, but different manipulators need to be used at different operation stages. To address this, after calibrating the current target manipulator, the target manipulator mounted on the second robotic arm can be replaced, and the calibration of the next target manipulator can be performed in a similar manner until all target manipulators are calibrated, resulting in multiple calibration results. The instrument identifier corresponding to each calibration result is then labeled and stored.

[0095] For example, in engineering operations involving target manipulation, multiple second robotic arms may be used, each carrying a different target manipulation device. In such cases, calibration can be performed as described above to obtain calibration results for the multiple second robotic arms. After labeling each calibration result with the corresponding robotic arm identifier, the results are stored.

[0096] In some embodiments, after determining the transformation relationship of the key points of the target operating instrument with respect to the second target coordinate system, and the transformation relationship of the image coordinate system of the image acquisition end with respect to the first target coordinate system, the method may further include the following:

[0097] S1: Call the calibration camera to obtain a target image containing the current robot;

[0098] S2: Based on the target image, determine the transformation relationship between the current first target coordinate system and the calibration camera coordinate system, and the transformation relationship between the current second target coordinate system and the calibration camera coordinate system;

[0099] S3: Based on the transformation relationship between the current first target coordinate system and the calibration camera coordinate system, the transformation relationship between the current second target coordinate system and the calibration camera coordinate system, and the calibration results, calculate the current position coordinates of the key points of the target operating instrument based on the image coordinates of the image acquisition end.

[0100] The target image contains at least the identifier code of the current first target and the identifier code of the second target.

[0101] When performing specific engineering operations of the target operation type, the position of the calibration camera can be moved relatively freely to acquire target images. At the same time, the current robot motion control parameters corresponding to the target images can be obtained through the intermediate communication module. Then, based on the corresponding target images and the current robot motion control parameters, combined with the matching calibration results, the current position coordinates of the key points of the target operation instrument based on the image coordinates of the image acquisition terminal can be calculated, realizing real-time positioning and monitoring of the key points of the target instrument in engineering operations.

[0102] Specifically, we can first obtain the current extrinsic parameter data based on the target image; at the same time, based on the currently used robotic arm and the target operating device, we can find the matching calibration results; then, by combining the calibration results, the current extrinsic parameter data, and the current robot motion control terminal parameters, we can calculate the current position coordinates of the key points of the target operating device based on the image coordinates of the image acquisition terminal.

[0103] In some embodiments, the key points of the target operating mechanism may specifically include the joints of the target operating mechanism. Furthermore, the key points of the target operating mechanism may specifically include multiple key points.

[0104] Specifically, for example, see Figure 10 As shown, in a medical setting, the key points for a speckle clamp can include three key points: key point 1, key point 2, and key point 3.

[0105] In some embodiments, after calculating the current position coordinates of the key points of the target manipulator based on the image coordinates of the image acquisition terminal, the method may further include the following: obtaining the current robot motion control terminal parameters; determining the current position coordinates of the key points of the target manipulator based on the tail coordinate system of the first robotic arm according to the current robot motion control terminal parameters; and monitoring whether there is an error based on the current position coordinates of the key points of the target manipulator based on the tail coordinate system of the first robotic arm and the current position coordinates based on the image coordinates of the image acquisition terminal.

[0106] In some embodiments, if an error is detected, a calibration scheme can be triggered to calibrate the calibration results in a timely manner to ensure accurate and reliable positioning of key points of the target operating instrument during the execution of the target operation.

[0107] In some embodiments, for medical scenarios, the image acquisition terminal may specifically include an endoscope. Correspondingly, the image coordinate system may include the endoscope coordinate system.

[0108] As can be seen from the above, the robot calibration method provided in this specification, specifically during calibration, firstly determines the target operation type of the target operation; then, based on the target operation type, matching targets are selected from a preset target set and respectively set on the first and second robotic arms as the first and second targets; wherein, the preset target set includes multiple targets with different structural shapes and identification codes; according to preset calibration rules, calibration processing is performed using a calibration camera and a robot to determine the transformation relationship of the key points of the target operating device with respect to the second target coordinate system, and the transformation relationship of the image coordinate system of the image acquisition end with respect to the first target coordinate system, as the calibration result. This simplifies the calibration process of robots and other operating equipment, accurately and efficiently completes the relevant calibration automatically, obtains the required calibration results, and reduces calibration errors. Furthermore, when using the calibration results obtained in the above manner to locate and monitor the key points of the target manipulator, the position of the key points can be accurately determined based on the image coordinate system of the image acquisition end, no longer constrained by the position of the calibration camera. This makes the location of the key points of the target manipulator more flexible and convenient during specific operations, thus adapting to diverse application scenarios. Further, by first determining the type of target operation to be performed by the robot, and then differentiating between different situations based on the target operation type to determine the matching target and / or target movement path, and then performing targeted calibration based on the matching target and / or target movement path, the calibration process becomes more targeted, effectively improving calibration accuracy and further reducing calibration errors. In addition, by introducing and using relatively small targets with different structural shapes and identification codes to replace the conventional, large, inconvenient, and homogeneous calibration boards for specific calibration, the calibration effect can be effectively improved, further enhancing calibration accuracy.

[0109] See Figure 2 As shown in the specification, this specification also provides a robot calibration system, which may include at least: an external optical platform, a robot, and a server; wherein, the robot includes a first robotic arm and a second robotic arm, the first robotic arm being equipped with an image acquisition terminal, and the second robotic arm being equipped with a target manipulation device; the external optical platform includes at least a calibration camera; the server uses the robot calibration method to control the robot and the external optical platform to perform specific calibration processing and obtain the corresponding calibration results.

[0110] This specification also provides a server, including a processor and a memory for storing processor-executable instructions. Specifically, the processor can perform the following steps according to the instructions: determining the target operation type of the target operation; selecting matching targets from a preset target set and setting them on a first robotic arm and a second robotic arm respectively, as the first target and the second target, according to the target operation type; wherein the preset target set includes multiple targets with different structural shapes and identification codes; and performing calibration processing using a calibration camera and a robot according to preset calibration rules to determine the transformation relationship of the key points of the target operation device with respect to the second target coordinate system, and the transformation relationship of the image coordinate system of the image acquisition end with respect to the first target coordinate system, as the calibration result.

[0111] To execute the above instructions more accurately, please refer to... Figure 11 As shown in the embodiments of this specification, another specific server is also provided, wherein the server includes a network communication port 1101, a processor 1102 and a memory 1103, and the above structures are connected by internal cables so that the various structures can perform specific data interaction.

[0112] Specifically, the network communication port 1101 can be used to receive trigger commands.

[0113] The processor 1102 can specifically be used to respond to trigger commands and determine the target operation type of the target operation; according to the target operation type, it selects matching targets from a preset target set and sets them on the first and second robotic arms respectively, as the first target and the second target; wherein, the preset target set includes multiple targets with different structural shapes and identification codes; according to preset calibration rules, it uses a calibration camera and a robot to perform calibration processing, determines the transformation relationship of the key points of the target operation device with respect to the second target coordinate system, and the transformation relationship of the image coordinate system of the image acquisition end with respect to the first target coordinate system, as the calibration result.

[0114] The memory 1103 can be used to store the corresponding instruction program.

[0115] In this embodiment, the network communication port 1101 can be a virtual port bound to different communication protocols, thereby enabling the sending or receiving of different data. For example, the network communication port can be a port responsible for web data communication, a port responsible for FTP data communication, or a port responsible for email data communication. Furthermore, the network communication port can also be a physical communication interface or communication chip. For example, it can be a wireless mobile network communication chip, such as GSM or CDMA; it can also be a Wi-Fi chip; or it can be a Bluetooth chip.

[0116] In this embodiment, the processor 1102 can be implemented in any suitable manner. For example, the processor can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers, etc. This specification is not limiting.

[0117] In this embodiment, the memory 1103 may include multiple layers. In a digital system, anything that can store binary data can be a memory. In an integrated circuit, a circuit with storage function but no physical form is also called a memory, such as RAM, FIFO, etc. In a system, a storage device with a physical form is also called a memory, such as a memory stick, TF card, etc.

[0118] This specification also provides a computer storage medium based on the above-described robot calibration method. The computer storage medium stores computer program instructions, which, when executed, perform the following: determining the target operation type of the target operation; selecting matching targets from a preset target set according to the target operation type and setting them on the first and second robotic arms respectively, serving as the first and second targets; wherein the preset target set includes multiple targets with different structural shapes and identification codes; and performing calibration processing using a calibration camera and a robot according to preset calibration rules to determine the transformation relationship of the key points of the target operating device with respect to the second target coordinate system, and the transformation relationship of the image coordinate system of the image acquisition end with respect to the first target coordinate system, as the calibration result.

[0119] In this embodiment, the storage medium includes, but is not limited to, Random Access Memory (RAM), Read-Only Memory (ROM), cache, hard disk drive (HDD), or memory card. The memory can be used to store computer program instructions. The network communication unit can be an interface configured according to standards specified in the communication protocol for network connection communication.

[0120] In this embodiment, the specific functions and effects implemented by the program instructions stored in the computer storage medium can be explained in comparison with other implementation methods, and will not be repeated here.

[0121] See Figure 12As shown, at the software level, this specification also provides a robot calibration device, which may specifically include the following structural modules:

[0122] Module 1201 is specifically used to determine the target operation type of the target operation.

[0123] The filtering module 1202 can be used to filter out matching targets from a preset target set according to the target operation type of the target operation, and set them on the first robotic arm and the second robotic arm respectively as the first target and the second target; wherein, the preset target set includes multiple targets with different structural shapes and set with identification codes.

[0124] The calibration module 1203 can be used to perform calibration processing using a calibration camera and a robot according to preset calibration rules, to determine the transformation relationship of the key points of the target operating instrument with respect to the second target coordinate system, and the transformation relationship of the image coordinate system of the image acquisition end with respect to the first target coordinate system, as the calibration result.

[0125] In some embodiments, the targets in the preset target set may specifically include at least one of the following: umbrella-shaped target, stepped surround target, spherical target, Y-shaped target, cross-shaped target, etc.

[0126] In some embodiments, the calibration camera may specifically include a camera disposed on an external optical platform independent of the robot.

[0127] In some embodiments, when the calibration module 1203 is specifically implemented, it can perform calibration processing using a calibration camera and a robot according to preset calibration rules in the following manner, so as to determine the transformation relationship of the key points of the target operating instrument with respect to the second target coordinate system, and the transformation relationship of the image coordinate system of the image acquisition end with respect to the first target coordinate system: according to the preset calibration rules, the robot is controlled to move the first robotic arm and / or the second robotic arm; calibration images are acquired through the calibration camera during the movement process, and robot motion control end parameters are obtained through the robot; based on the calibration images and robot motion control end parameters, the transformation relationship of the key points of the target operating instrument with respect to the second target coordinate system, and the transformation relationship of the image coordinate system of the image acquisition end with respect to the first target coordinate system are determined.

[0128] In some embodiments, when the calibration module 1203 is specifically implemented, it can control the robot to move the first robotic arm and / or the second robotic arm according to the preset calibration rules in the following manner: determine the matching target movement path according to the target operation type; control the robot to move the first robotic arm and / or the second robotic arm according to the target movement path.

[0129] In some embodiments, the target movement path may specifically include a preset movement path, or a user-defined movement path, etc.

[0130] In some embodiments, when the calibration module 1203 is specifically implemented, it can acquire calibration images during the movement process through the calibration camera in the following manner to obtain robot motion control parameters: when controlling the robot to move the first robotic arm and / or the second robotic arm to the current pose according to the target movement path; controlling the robot to send a robot message to the intermediate communication module to trigger the calibration camera to acquire the corresponding calibration image; and acquiring the corresponding robot motion control parameters through the intermediate communication module; wherein, the calibration camera receives and responds to the robot message through the intermediate communication module and acquires the calibration image corresponding to the current pose.

[0131] In some embodiments, when the calibration module 1203 is specifically implemented, it can determine the transformation relationship of the key points of the target operating device with respect to the second target coordinate system and the transformation relationship of the image coordinate system of the image acquisition terminal with respect to the first target coordinate system according to the calibration image and the robot motion control terminal parameters in the following manner: Based on the calibration image, the intrinsic parameter data of the calibration camera, and the robot motion control terminal parameters, determine the first extrinsic parameter data of the calibration camera with respect to the first target and the second extrinsic parameter data of the calibration camera with respect to the second target; based on the first extrinsic parameter data, obtain the transformation relationship between the first target coordinate system and the calibration camera coordinate system; based on the second extrinsic parameter data, obtain the transformation relationship between the second target coordinate system and the calibration camera coordinate system; based on the robot motion control terminal parameters, determine the transformation relationship between the robot base coordinate system and the first robotic arm tail coordinate system, and the robot base coordinate system... The transformation relationships between the coordinate systems of the first target and the second robot arm's tail end, the first target coordinate system and the calibration camera coordinate system, and the second target coordinate system and the calibration camera coordinate system are solved to obtain the transformation relationships between the calibration camera coordinate system and the robot base coordinate system, the first target coordinate system and the first robot arm's tail end coordinate system, and the second target coordinate system and the second robot arm's tail end coordinate system. Based on the transformation relationships between the calibration camera coordinate system and the robot base coordinate system, and the first target coordinate system and the first robot arm's tail end coordinate system, the transformation relationship of the image coordinate system of the image acquisition end with respect to the first target coordinate system is determined. Based on the transformation relationships between the calibration camera coordinate system and the robot base coordinate system, and the second target coordinate system and the second robot arm's tail end coordinate system, the transformation relationship of the key points of the target manipulation device with respect to the second target coordinate system is determined.

[0132] In some embodiments, after determining the transformation relationship of the key points of the target manipulator with respect to the second target coordinate system and the transformation relationship of the image coordinate system of the image acquisition end with respect to the first target coordinate system, the device can also be used to call the calibration camera to acquire a target image containing the current robot; based on the target image, determine the transformation relationship between the current first target coordinate system and the calibration camera coordinate system, and the transformation relationship between the current second target coordinate system and the calibration camera coordinate system; based on the transformation relationship between the current first target coordinate system and the calibration camera coordinate system, the transformation relationship between the current second target coordinate system and the calibration camera coordinate system, and the calibration result, calculate the current position coordinates of the key points of the target manipulator based on the image coordinates of the image acquisition end.

[0133] In some embodiments, the second robotic arm may specifically include multiple robotic arms; the target operating device may specifically include a variety of operating devices.

[0134] In some embodiments, the key points of the target operating machinery may specifically include the joints of the target operating machinery; the key points of the target operating machinery may specifically include multiple key points.

[0135] It should be noted that the units, devices, or modules described in the above embodiments can be implemented by computer chips or physical entities, or by products with certain functions. For ease of description, the above devices are described by dividing them into various modules according to their functions. Of course, in implementing this specification, the functions of each module can be implemented in one or more software and / or hardware, or the module that implements the same function can be implemented by a combination of multiple sub-modules or sub-units, etc. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection between the devices or units shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0136] As can be seen from the above, the robot calibration device provided in the embodiments of this specification can effectively simplify the calibration process, accurately, efficiently, and purposefully complete the relevant calibration automatically, obtain the required calibration results, and reduce calibration errors. Furthermore, when using the calibration results obtained in the above manner to locate and monitor the key points of the target manipulator, it is no longer constrained by the position of the calibration camera, and can accurately determine the position coordinates of the key points of the target manipulator based on the image coordinate system of the image acquisition end. This makes the location of the key points of the target manipulator more flexible and convenient during specific operations, thus adapting to diverse application scenarios.

[0137] See Figure 13 As shown, this specification also provides another robot calibration method, wherein the robot includes a target manipulator arm, the target manipulator arm is equipped with a target manipulator, and the method may include the following steps in its specific implementation:

[0138] S1301: Determine the target operation type of the target operation;

[0139] S1302: Based on the target operation type, select matching targets from a preset target set and set them on the target operation robotic arm as target targets; wherein, the preset target set includes multiple targets with different structural shapes and identification codes.

[0140] S1303: According to the preset calibration rules, calibration is performed using a calibration camera and a robot to determine the transformation relationship of the key points of the target operating instrument with respect to the target coordinate system based on the target target, which is then used as the calibration result.

[0141] In some embodiments, during specific implementation, a matching target movement path can be determined based on the target operation type of the target operation; then, based on the preset calibration rules and the target movement path, a targeted calibration process is performed using a calibration camera and a robot to obtain a calibration result with relatively higher accuracy.

[0142] Based on the above embodiments, different operation types can be distinguished, and matching targets can be selected and used for targeted calibration, thereby effectively improving calibration accuracy and reducing calibration errors.

[0143] While this specification provides the steps of operation for the methods described in the embodiments or flowcharts, more or fewer steps may be included based on conventional or non-inventive means. The order of steps listed in the embodiments is merely one possible order of execution among many steps and does not represent the only possible order. In actual device or client product execution, the methods shown in the embodiments or drawings may be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment, or even a distributed data processing environment). The terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, product, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, product, or apparatus. Without further limitations, the presence of other identical or equivalent elements in a process, method, product, or apparatus that includes said elements is not excluded. The terms "first," "second," etc., are used to denote names and do not indicate any particular order.

[0144] Those skilled in the art will also know that, besides implementing the controller using purely computer-readable program code, the same functions can be achieved by logically programming the method steps, making the controller function as logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers (PLCs), and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the devices within it used to implement various functions can also be considered structures within that hardware component. Alternatively, the devices used to implement various functions can be considered as both software modules implementing the method and structures within a hardware component.

[0145] This specification can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, classes, etc., that perform a specific task or implement a specific abstract data type. This specification can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0146] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this specification can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solutions of this specification can essentially be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, mobile terminal, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments of this specification.

[0147] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on its differences from other embodiments. This specification can be used in numerous general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable electronic devices, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices, etc.

[0148] Although this specification has been described by way of examples, those skilled in the art will recognize that many variations and modifications are possible without departing from the spirit of this specification, and it is intended that the appended claims cover such variations and modifications without departing from the spirit of this specification.

Claims

1. A robot calibration method, characterized in that, The robot includes a first robotic arm and a second robotic arm. The first robotic arm is equipped with an image acquisition device, and the second robotic arm is equipped with a target manipulation instrument. The method includes: Determine the target operation type; Based on the target operation type, a pre-defined matching table is consulted to select matching targets from a pre-defined target set and set them on the first and second robotic arms respectively, serving as the first and second targets. The pre-defined target set includes multiple targets with different structural shapes, each equipped with an identification code and a small calibration plate size. The pre-defined target set includes planar targets and three-dimensional targets. The three-dimensional targets include at least one of the following: umbrella-shaped targets, stepped surround targets, and spherical targets. According to preset calibration rules, calibration is performed using a calibration camera and a robot to determine the transformation relationship of the key points of the target manipulator with respect to the second target coordinate system, and the transformation relationship of the image coordinate system of the image acquisition terminal with respect to the first target coordinate system, as the calibration result; including: determining the matching target movement path according to the target operation type; controlling the robot to move the first robotic arm and / or the second robotic arm according to the target movement path; and obtaining robot motion control terminal parameters through calibration images acquired by the calibration camera; and determining the transformation relationship of the key points of the target manipulator with respect to the second target coordinate system based on the calibration images and robot motion control terminal parameters. The transformation relationship of the second target coordinate system and the transformation relationship of the image coordinate system of the image acquisition end with respect to the first target coordinate system; the target movement path is a movement control path that can cover the poses involved in the specific implementation of the corresponding target operation; wherein, the robot motion control end parameters include at least the position coordinates of the identifier code in the first target based on the first robotic arm tail coordinate system and the position coordinates of the identifier code in the second target based on the second robotic arm tail coordinate system when in different poses during the movement of the target movement path, as well as parameters that can characterize the pose relationship between the first robotic arm and the second robotic arm relative to the robot base.

2. The method according to claim 1, characterized in that, The targets in the preset target set include at least one of the following: Y-shaped targets and cross-shaped targets; The calibration camera includes a camera mounted on an external optical platform independent of the robot.

3. The method according to claim 1, characterized in that, The target movement path includes a preset movement path or a user-defined movement path.

4. The method according to claim 3, characterized in that, By acquiring calibration images during the movement using a calibration camera, parameters from the robot's motion control end are obtained, including: When the robot moves the first robotic arm and / or the second robotic arm to the current pose according to the target movement path, the robot sends a robot message to the intermediate communication module to trigger the calibration camera to acquire the corresponding calibration image; the robot obtains the corresponding robot motion control parameters through the intermediate communication module; wherein, the calibration camera receives and responds to the robot message through the intermediate communication module and acquires the calibration image corresponding to the current pose.

5. The method according to claim 3, characterized in that, Based on the calibration images and robot motion control parameters, the transformation relationships of key points of the target manipulator with respect to the second target coordinate system, and the transformation relationships of the image coordinate system of the image acquisition terminal with respect to the first target coordinate system are determined, including: Based on the calibration images, the intrinsic parameter data of the calibration camera, and the parameters of the robot motion control terminal, the first extrinsic parameter data of the calibration camera with respect to the first target and the second extrinsic parameter data of the calibration camera with respect to the second target are determined. Based on the first extrinsic parameter data, obtain the transformation relationship between the first target coordinate system and the calibration camera coordinate system; based on the second extrinsic parameter data, obtain the transformation relationship between the second target coordinate system and the calibration camera coordinate system. Based on the robot motion control terminal parameters, determine the transformation relationships between the robot base coordinate system and the first robotic arm tail coordinate system, the robot base coordinate system and the second robotic arm tail coordinate system, the first target coordinate system and the calibration camera coordinate system, and the second target coordinate system and the calibration camera coordinate system. Solve for the transformation relationships between the calibration camera coordinate system and the robot base coordinate system, the first target coordinate system and the first robotic arm tail coordinate system, and the second target coordinate system and the second robotic arm tail coordinate system. Based on the transformation relationship between the calibrated camera coordinate system and the robot base coordinate system, and the transformation relationship between the first target coordinate system and the tail coordinate system of the first robotic arm, the transformation relationship of the image coordinate system of the image acquisition end with respect to the first target coordinate system is determined; based on the transformation relationship between the calibrated camera coordinate system and the robot base coordinate system, and the transformation relationship between the second target coordinate system and the tail coordinate system of the second robotic arm, the transformation relationship of the key points of the target manipulation device with respect to the second target coordinate system is determined.

6. The method according to claim 1, characterized in that, After determining the transformation relationship of the key points of the target operating instrument with respect to the second target coordinate system, and the transformation relationship of the image coordinate system of the image acquisition end with respect to the first target coordinate system, the method further includes: Use the calibration camera to acquire a target image containing the current robot; Based on the target image, determine the transformation relationship between the current first target coordinate system and the calibration camera coordinate system, and the transformation relationship between the current second target coordinate system and the calibration camera coordinate system; Based on the transformation relationship between the current first target coordinate system and the calibration camera coordinate system, the transformation relationship between the current second target coordinate system and the calibration camera coordinate system, and the calibration results, the current position coordinates of the key points of the target operating instrument based on the image coordinates of the image acquisition end are calculated.

7. The method according to claim 1, characterized in that, The second robotic arm includes multiple robotic arms; the target operating device includes multiple operating devices; the key points of the target operating device include the joints of the target operating device; the key points of the target operating device include multiple key points.

8. A robot calibration system, characterized in that, At least including: An external optical platform, a robot, and a server; wherein the robot includes a first robotic arm and a second robotic arm, the first robotic arm being equipped with an image acquisition terminal, and the second robotic arm being equipped with a target manipulation device; the external optical platform includes at least a calibration camera; the server uses the robot calibration method according to any one of claims 1 to 7 to perform calibration processing by controlling the robot and the external optical platform to obtain the corresponding calibration results.

9. A robot calibration device, characterized in that, The robot includes at least a first robotic arm and a second robotic arm. The first robotic arm is equipped with an image acquisition device, and the second robotic arm is equipped with a target manipulation instrument. The device includes: The determination module is used to determine the target operation type of the target operation. The filtering module is used to filter out matching targets from a preset target set based on the target operation type of the target operation by querying a preset matching relationship table. These matching targets are then set on the first and second robotic arms respectively, serving as the first and second targets. The preset target set includes multiple targets with different structural shapes, each equipped with an identification code and a small calibration plate size. The preset target set includes planar targets and three-dimensional targets. The three-dimensional targets include at least one of the following: umbrella-shaped targets, stepped surround targets, and spherical targets. The calibration module is used to perform calibration processing using a calibration camera and a robot according to preset calibration rules, and to determine the transformation relationship of the key points of the target operating instrument with respect to the second target coordinate system, as well as the transformation relationship of the image coordinate system of the image acquisition end with respect to the first target coordinate system, as the calibration result; The calibration module is specifically used for: determining a matching target movement path based on the target operation type; controlling the robot to move the first robotic arm and / or the second robotic arm according to the target movement path; acquiring calibration images during the movement process through a calibration camera to obtain robot motion control parameters; determining the transformation relationship of key points of the target operation instrument with respect to the second target coordinate system, and the transformation relationship of the image coordinate system of the image acquisition terminal with respect to the first target coordinate system based on the calibration images and the robot motion control parameters; the target movement path is a movement control path that can cover the poses involved in the specific implementation of the corresponding target operation; wherein, the robot motion control parameters include at least the position coordinates of the identifier code in the first target based on the first robotic arm tail coordinate system, the position coordinates of the identifier code in the second target based on the second robotic arm tail coordinate system, and parameters that can characterize the pose relationship between the first robotic arm and the second robotic arm relative to the robot base when in different poses during the movement of the target movement path.

10. A robot calibration method, characterized in that, The robot includes a target manipulator arm, the target manipulator arm being equipped with a target manipulator, and the method includes: Determine the target operation type; Based on the target operation type, a matching target is selected from a preset target set by querying a preset matching relationship table and set on the target operation robotic arm as the target target; wherein, the preset target set includes multiple targets with different structural shapes, each with an identification code and a small calibration plate size; the preset target set includes planar targets and three-dimensional targets; the three-dimensional targets include at least one of the following: umbrella-shaped target, stepped surround target, and spherical target; According to preset calibration rules, calibration is performed using a calibration camera and a robot to determine the transformation relationship of the key points of the target manipulation device with respect to the target coordinate system based on the target target, which is used as the calibration result. This includes: determining a matching target movement path based on the target operation type; controlling the robot to move the first robotic arm and / or the second robotic arm according to the target movement path; acquiring calibration images during the movement process through the calibration camera to obtain robot motion control parameters; determining the transformation relationship of the key points of the target manipulation device with respect to the second target coordinate system, and the transformation relationship of the image coordinate system of the image acquisition terminal with respect to the first target coordinate system based on the second target coordinate system, according to the calibration images and the robot motion control parameters; the target movement path is a movement control path that can cover the poses involved in the specific implementation of the corresponding target operation; wherein, the robot motion control parameters include at least the position coordinates of the identifier code in the first target based on the first robotic arm tail coordinate system, the position coordinates of the identifier code in the second target based on the second robotic arm tail coordinate system, and parameters that can characterize the pose relationship of the first robotic arm and the second robotic arm relative to the robot base when in different poses during the movement based on the target movement path.

11. A computer-readable storage medium, characterized in that, It stores computer instructions that, when executed, implement the relevant steps of the robot calibration method according to any one of claims 1 to 7 or 10.

Images