Robotic arm control method, electronic device and storage medium

By installing multiple image acquisition devices on the robot chassis, calculating the camera position measurement values ​​of the markers, determining and correcting the robot's position offset, the problem of large positioning error of the robotic arm is solved, and control accuracy and detection efficiency are improved.

CN118357927BActive Publication Date: 2024-11-12SHENZHEN YOUIBOT ROBOTICS CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410764848.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-06-14
Publication Date
2024-11-12
Estimated Expiration
2044-06-14

AI Technical Summary

Technical Problem

In existing technologies, the positioning error of robotic arms is relatively large, resulting in low control precision, especially when the distance between the chassis and the robotic arm is far, making it difficult to accurately grasp or place objects.

Method used

First and second image acquisition devices are installed on the robot's mobile chassis to acquire images of first and second markers. By calculating the camera position measurements of the markers, the robot's position offset is determined and the position deviation is corrected to improve the control accuracy of the robotic arm.

Benefits of technology

By improving the accuracy of robot position offset detection and correcting position deviations, the control precision and detection efficiency of the robotic arm are enhanced, the extension action of the robotic arm to take pictures is reduced, and the accuracy of grasping or placing operations is improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118357927B_ABST
    Figure CN118357927B_ABST
Patent Text Reader

Abstract

The present application provides a robot arm control method, electronic device and computer-readable storage medium. The method includes: when the robot moves to a designated working area, a first image acquisition device installed on the mobile chassis of the robot acquires an image of a first marker, and a second image acquisition device acquires an image of a second marker; based on the image of the first marker and the image of the second marker, a second camera position measurement value of the first marker and a second camera position measurement value of the second marker are acquired; based on the second camera position measurement value of the first marker and the second camera position measurement value of the second marker, the position offset of the robot is determined; based on the position offset of the robot, the robot arm is controlled to complete a grasping or placing operation. The present application can improve the detection accuracy of the robot's position offset, use the high-precision robot's position offset to correct the robot's position deviation, and improve the control accuracy of the robot arm.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of robot control technology, and in particular to a robot arm control method, electronic equipment and computer-readable storage medium. Background Art

[0002] With the rise of artificial intelligence, robots are playing an increasingly important role in all walks of life. For robots, grasping is an essential skill for robots to enter the real world, such as sorting objects in the logistics industry and completing the assembly of parts on industrial production lines. For new robots that integrate the functions of mobile robots and general industrial robots (i.e., robotic arms), in order to ensure that the robotic arms can accurately grasp or place objects, the relevant technology installs the robotic arms on the robot chassis and uses cameras to locate the position of the robot; however, due to the long distance between the chassis and the robotic arms, if a camera is installed on the chassis for positioning, the positioning error of the robotic arms will be large, resulting in low control accuracy of the robotic arms. Summary of the invention

[0003] The present application provides a robot arm control method, electronic device and computer-readable storage medium, which can improve the detection accuracy of the robot's position offset, use the high-precision robot position offset to correct the robot's position deviation, and improve the control accuracy of the robot arm.

[0004] In a first aspect, the present application provides a method for controlling a robot arm of a robot, wherein a first image acquisition device and a second image acquisition device are installed on a mobile chassis of the robot, and the method comprises:

[0005] When the robot moves to a designated working area, an image of a first marker is collected by a first image acquisition device installed on a mobile chassis of the robot, and an image of a second marker is collected by a second image acquisition device installed on the mobile chassis, wherein the first marker and the second marker are arranged in the designated working area;

[0006] Based on the image of the first marker and the image of the second marker, obtaining a second camera position measurement value of the first marker and a second camera position measurement value of the second marker, wherein the second camera position measurement value is a coordinate position in a second camera coordinate system, and the second camera coordinate system is a camera coordinate system of a second image acquisition device;

[0007] Determine a position offset of the robot based on a second camera position measurement value of the first marker and a second camera position measurement value of the second marker, the position offset comprising at least one of an angle offset and a coordinate offset;

[0008] Based on the position offset of the robot, the robot's mechanical arm is controlled to complete a grasping or placing operation.

[0009] In a second aspect, the present application further provides an electronic device, comprising a processor and a memory, wherein the memory stores a computer program, and when the processor calls the computer program in the memory, it executes any robot arm control method provided in the present application.

[0010] In a third aspect, the present application also provides a computer-readable storage medium on which a computer program is stored, and the computer program is loaded by a processor to execute the robot's mechanical arm control method.

[0011] In the present application, in the first aspect, the image of the first marker is collected by the first image acquisition device installed on the mobile chassis of the robot, and the image of the second marker is collected by the second image acquisition device installed on the mobile chassis, and the second camera position measurement value of the first marker and the second camera position measurement value of the second marker are used to determine the position offset of the robot, and the position deviation of the robot can be corrected, thereby improving the control accuracy of the robot arm. In the second aspect, by using the first image acquisition device to collect the image of the first marker and the second image acquisition device to collect the image of the second marker, since the use of two cameras can make a small offset angle be magnified in the offset of the coordinates (for example, even if the robot has only a small offset angle, it will be magnified in the coordinate deviation between the second camera position measurement value of the first marker and the second camera position calibration value of the first marker, or in the coordinate deviation between the second camera position measurement value of the second marker and the second camera position calibration value of the second marker), even a small offset angle can be detected, thereby improving the detection accuracy of the position offset of the robot. Thirdly, by installing two image acquisition devices on the mobile chassis of the robot to collect markers to detect the position offset of the robot, it is possible to avoid installing the image acquisition device on the mechanical arm to use the mechanical arm to take pictures, thereby reducing the action of the mechanical arm extending to the photo taking position to improve the detection efficiency of the robot's position offset. Therefore, this embodiment can improve the detection accuracy of the robot's position offset, use the high-precision robot's position offset to correct the robot's position deviation, improve the control accuracy of the mechanical arm, and improve the detection efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0013] Figure 1 is a schematic block diagram of the structure of an electronic device provided in an embodiment of the present application;

[0014] Figure 2 It is a flow chart of a method for controlling a robot arm of a robot provided in an embodiment of the present application;

[0015] Figure 3 is a schematic diagram of a scene illustrating the stopping position of a robot when controlling a robotic arm to operate provided in an embodiment of the present application;

[0016] Figure 4 is a schematic diagram for explaining the coordinate system involved in the embodiments of the present application;

[0017] Figure 5 is a schematic diagram for explaining the conversion between the image position of the marker and the camera position in the embodiment of the present application;

[0018] Figure 6 is a schematic diagram of an installation description of the first image acquisition device and the second image acquisition device in the embodiment of the present application;

[0019] Figure 7 is a schematic diagram for explaining the influence of the installation distance between the image acquisition devices on the detection accuracy of the offset angle in the embodiment of the present application;

[0020] Figure 8 This is a schematic diagram illustrating a scenario in which the angle offset of the robot is detected by using the marker calibration line L1 and the marker measurement line L2 in an embodiment of the present application;

[0021] Fig. 9 This is a schematic diagram illustrating a scenario in which the angle offset of a robot is detected using a marker calibration line L1 and a marker correction line L3 in an embodiment of the present application. DETAILED DESCRIPTION

[0022] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0023] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0024] In the description of the embodiments of the present application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0025] In order to enable any person skilled in the art to implement and use the present application, the following description is provided. In the following description, details are listed for the purpose of explanation. It should be understood that those of ordinary skill in the art can recognize that the present application can also be implemented without using these specific details. In other examples, the known process will not be elaborated in detail to avoid unnecessary details that make the description of the present application embodiment obscure. Therefore, the present application is not intended to be limited to the embodiments shown, but is consistent with the widest range of principles and features disclosed in accordance with the embodiments of the present application.

[0026] The embodiment of the present application provides a robot arm control method, an electronic device and a computer readable storage medium. The electronic device may be a mobile robot, a mobile phone, a computer, etc.

[0027] Some embodiments of the present application are described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0028] Figure 1 It is a schematic block diagram of the structure of an electronic device provided in an embodiment of the present application.

[0029] like Figure 1 As shown, the electronic device 100 includes a processor 101 and a memory 102 , and the processor 101 and the memory 102 are connected via a bus 103 , such as an I2C (Inter-integrated Circuit) bus.

[0030] Specifically, the processor 101 is used to provide computing and control capabilities to support the operation of the entire electronic device 100. The processor 101 may be a central processing unit (CPU), and the processor 101 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0031] Specifically, the memory 102 may be a Flash chip, a read-only memory (ROM) disk, an optical disk, a USB flash drive, or a mobile hard disk.

[0032] Those skilled in the art will understand that Figure 1 The structure shown in the figure is merely a block diagram of a partial structure related to the embodiment scheme of the present application, and does not constitute a limitation on the electronic device to which the embodiment scheme of the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0033] The processor 101 is used to run the computer program stored in the memory 102, and implements any one of the robot arm control methods provided in the embodiments of the present application when executing the computer program. For example, the processor 101 is used to run the computer program stored in the memory 102, and can implement the following steps when executing the computer program:

[0034] When the robot moves to a designated working area, an image of a first marker is acquired by a first image acquisition device installed on a mobile chassis of the robot, and an image of a second marker is acquired by a second image acquisition device installed on the mobile chassis, wherein a first marker and a second marker are arranged in the designated working area; based on the image of the first marker and the image of the second marker, a second camera position measurement value of the first marker and a second camera position measurement value of the second marker are acquired, wherein the second camera position measurement value is a coordinate position in a second camera coordinate system, and the second camera coordinate system is a camera coordinate system of the second image acquisition device; based on the second camera position measurement value of the first marker and the second camera position measurement value of the second marker, a position offset of the robot is determined, wherein the position offset includes at least one of an angle offset and a coordinate offset; based on the position offset of the robot, a mechanical arm of the robot is controlled to complete a grasping or placing operation.

[0035] It should be noted that technicians in the relevant field can clearly understand that for the convenience and simplicity of description, the specific working process of the electronic device described above can refer to the corresponding process in the following robot arm control method embodiment, and will not be repeated here.

[0036] In the following, Figure 1 The electronic device shown in is used as the execution subject of the robot's mechanical arm control method as an example to introduce the robot's mechanical arm control method provided in the embodiment of the present application in detail. For the sake of simplicity and convenience of description, the execution subject will be omitted in the subsequent method embodiments. Figure 1 The scenario is only used to explain the robot arm control method provided in the embodiment of the present application, but does not constitute a limitation on the application scenario of the robot arm control method provided in the embodiment of the present application.

[0037] See also Figure 2 , Figure 2 201 to 204, wherein:

[0038] 201. When the robot moves to a designated working area, an image of a first marker is acquired by a first image acquisition device installed on a mobile chassis of the robot, and an image of a second marker is acquired by a second image acquisition device installed on the mobile chassis.

[0039] To better understand the embodiments of this application, please refer to Figure 4 , the following first introduces some names involved in this embodiment:

[0040] 1. Robot coordinate system O wX w Y w Z w : A three-dimensional coordinate system used to describe the position of an object (such as a robotic arm) relative to the robot. For example, Figure 4 As shown, the robot coordinate system can be O w X w Y w Z w . Further, to reduce the amount of data processing in the coordinate transformation process, Figure 5 The camera coordinate system of camera 2 (assuming it is O c2 X c2 Y c2 Z c2 ) as the robot coordinate system O w X w Y w Z w .

[0041] 2. Robot arm coordinate system O r X r Y r Z r : A coordinate system used to describe the position of an object relative to the robot arm.

[0042] 3. Camera coordinate system O c X c Y c Z c : A three-dimensional coordinate system established on the camera, with the center of the camera's field of view as the origin of the coordinate system O c .like Figure 4 As shown, the camera coordinate system can be O c X c Y c Z c .

[0043] The first camera coordinate system is the camera coordinate system of the first image acquisition device. In this embodiment, the center of the field of view of the first image acquisition device is used as the coordinate origin of the first camera coordinate system. c1 The first camera position refers to the coordinate position in the first camera coordinate system.

[0044] The second camera coordinate system is the camera coordinate system of the second image acquisition device. In this embodiment, the center of the field of view of the second image acquisition device is used as the coordinate origin of the second camera coordinate system. c2 The second camera position refers to the coordinate position in the second camera coordinate system.

[0045] 4. Image coordinate system O p X p Y p: It belongs to the two-dimensional coordinate system. The origin o is the corner point between the camera optical axis and the imaging plane. The image coordinate system and the camera coordinate system are in a perspective projection relationship. The image coordinate system is a two-dimensional coordinate system obtained by projecting the three-dimensional camera coordinate system. For example, Figure 4 As shown, the image coordinate system can be O p X p Y p .

[0046] When the focal length of the camera is known, the conversion between the image coordinate system and the camera coordinate system can be completed by referring to the conversion relationship between the image coordinate system and the camera coordinate system as shown in the following formula (1):

[0047] Formula (1)

[0048] In formula (1), f represents the focal length of the camera.

[0049] Among them, the robot coordinate system O w X w Y w Z w , Robot arm coordinate system O r X r Y r Z r , camera coordinate system O c X c Y c Z c , image coordinate system O p X p Y p The conversion relationship between them will be pre-calibrated to determine the conversion relationship between a certain point in each coordinate system, such as the robot coordinate system O w X w Y w Z w Point P w (X w ,Y w ,Z w ) and the image coordinate system O p X p Y p The corresponding point p(X p ,Y p ) between them.

[0050] 5. First camera position calibration value of the first marker: the coordinate position of the first marker in the first camera coordinate system obtained during the calibration process. During calibration, the coordinate position of the first marker in the first camera coordinate system when the robot is in the standard position is set as the first camera position calibration value of the first marker.

[0051] The first camera position measurement value (X c1 a ,Y c1 a ): The coordinate position of the first marker in the first camera coordinate system measured in the actual application process. Assume that (X c1 a ,Y c1 a )=(x1,y1).

[0052] 6. First camera position calibration value of the second marker: the coordinate position of the second marker in the first camera coordinate system obtained during the calibration process. During calibration, the coordinate position of the second marker in the first camera coordinate system when the robot is in the standard position is set as the first camera position calibration value of the second marker.

[0053] The first camera position measurement value (X c1 b ,Y c1 b ): The coordinate position of the second marker in the first camera coordinate system calculated during the actual application process.

[0054] 7. The second camera position calibration value of the first marker: the coordinate position of the first marker in the second camera coordinate system obtained during the calibration process. During calibration, the coordinate position of the first marker in the second camera coordinate system when the robot is in the standard position is set as the second camera position calibration value of the first marker.

[0055] The second camera position measurement value (X c2 a ,Y c2 a ): The coordinate position of the first marker in the second camera coordinate system, calculated during the actual application process.

[0056] 8. Second camera position calibration value of the second marker: the coordinate position of the second marker in the second camera coordinate system obtained during the calibration process. During calibration, the coordinate position of the second marker in the second camera coordinate system when the robot is in the standard position is set as the second camera position calibration value of the second marker.

[0057] The second camera position measurement value (X c2 b ,Y c2 b ): The coordinate position of the second marker in the second camera coordinate system measured in the actual application process. Assume that (X c2 b ,Yc2 b )=(x2,y2).

[0058] 9. Marker calibration line L1: is the second camera position calibration value of the first marker (reference Figure 8 a2 point shown in ) and the second camera position calibration value of the second marker (reference Figure 8 The line between the points b2 shown in Figure 8 As shown in L1.

[0059] 10. Marker measurement line L2: is the second camera position measurement value of the first marker (reference Figure 8 a2' shown in ) and the second camera position measurement of the second marker (reference Figure 8 The line between the points b2' shown in Figure 8 As shown in L2.

[0060] 11. Marker correction line L3: It is a straight line obtained by correcting the marker measurement line L2.

[0061] 12. First angle: The angle between the marker calibration line L1 and the marker measurement line L2, such as Figure 8 As shown in θ.

[0062] 13. Second angle: The angle between the marker calibration line L1 and the marker correction line L3, such as Fig. 9 As shown in θ.

[0063] 14. Installation distance k: the center of the field of view of the first image acquisition device O c1 and the center of the field of view of the second image acquisition device O c2 The distance between.

[0064] 15. The first marker (marked as a in this article) and the second marker (marked as b in this article): The first marker and the second marker are set in the designated working area. When calibrating, the first marker will be set at the center of the field of view of the first image acquisition device. c1 , the second marker is set at the center of the field of view of the second image acquisition device O c2 On the one hand, after the first camera position measurement value of the first marker is measured (such as (X c1 a ,Y c1 a )=(x1,y1)), the second camera position measurement value of the first marker can be directly calculated according to the installation distance k (i.e. (X c2 a ,Y c2 a)=(x1+k,y1)); On the other hand, when the robot moves to the designated working area and stops, the first image acquisition device can capture the first marker and the second image acquisition device can capture the second marker to ensure that the deviation of the robot can be detected by using the image of the marker.

[0065] Please refer to Figure 3 In order to improve the efficiency of taking pictures, two image acquisition devices will be installed on the robot chassis; in order to improve the detection accuracy of the robot's offset angle. For example, the operation process of the robot using a mechanical arm to grasp or place is as follows:

[0066] ① The robot navigates to the designated operation area as normal. Figure 3 As shown in (a);

[0067] ② After the robot reaches the designated operation area, it stops and the first image acquisition device (such as Figure 3 camera 1), a second image acquisition device (such as Figure 3 The middle camera 2) takes pictures to obtain image 1 and image 2 respectively;

[0068] ③ According to image 1 and image 2, determine whether the robot is in the calibrated standard position. If it is in the standard position (such as Figure 3 If it is in a non-standard position (such as Figure 3 (c) in the figure) then go to ⑤;

[0069] ④ According to the calibrated target position of the robot arm, control the robot arm to perform grasping or placing operations to ensure that the robot arm can accurately grasp or place objects;

[0070] ⑤ Detect the position offset of the robot and adjust the calibrated target position of the robotic arm according to the position offset; according to the adjusted target position of the robotic arm, control the robotic arm to perform grasping or placing operations to ensure that the robotic arm can accurately grasp or place objects.

[0071] Exemplarily, the robot can be controlled to move to a designated work area according to a pre-planned navigation path; ideally, after the robot navigates to the designated work area and stops at a calibrated standard position, the robot's mechanical arm is controlled to extend to complete the corresponding grasping or placement operation, but the robot may not stop completely at the calibrated standard position, for example, the robot may rotate 0.2 degrees in the Z-axis direction relative to the standard position; therefore, it is necessary to take a photo to check for deviations. Therefore, when the robot moves to the designated work area, the robot is controlled to stop, and the image of the first marker is captured by the first image acquisition device installed on the mobile chassis of the robot, and the image of the second marker is captured by the second image acquisition device installed on the mobile chassis.

[0072] In addition, due to the long distance between the robotic arm and the mobile chassis, a 0.2 degree error on the robot's mobile chassis may be transmitted to the robotic arm with an error of more than 0.5 degrees. However, since it is difficult to accurately control the robot to rotate back 0.2 degrees, the robotic arm may not be able to accurately complete the grasping or placement operation. The position offset of the robot is detected in step 203, and the angle offset and / or coordinate offset of the robotic arm is calculated according to the position offset of the robot in step 204 to update the calibrated target position of the robotic arm. The updated target position is obtained to control the robotic arm of the robot to complete the grasping or placement operation, so that the position offset of the robot can be modified to avoid the problem of inaccurate control of the robotic arm caused by the position offset of the robot, thereby improving the control accuracy of the robotic arm to a certain extent.

[0073] 202. Based on the image of the first marker and the image of the second marker, obtain a second camera position measurement value of the first marker and a second camera position measurement value of the second marker.

[0074] The specific shapes of the first marker and the second marker may be set according to the actual business scenario requirements, for example, the marker may be a circle, a square, a rectangle, etc. In this embodiment, the position of the marker may specifically be the center point position of the marker, for example, the first marker and the second marker are circle 1 and circle 2 respectively, then the second camera position (including the measured value and the calibrated value) of the first marker may be the coordinate position of the center of circle 1 in the second camera coordinate system, and the second camera position (including the measured value and the calibrated value) of the second marker may be the coordinate position of the center of circle 2 in the second camera coordinate system.

[0075] Exemplarily, step 202 may specifically include the following steps 2021 to 2023:

[0076] 2021. Based on the image of the first marker, determine a first camera position measurement value of the first marker.

[0077] For example, Figure 5 As shown in (a), first, the image of the first marker is subjected to marker recognition to obtain the image position of the first marker (such as a0'(X p1 a ,Y p1 a ), wherein the image position of the first marker refers to the coordinates in the first image coordinate system, and the first image coordinate system is the reference coordinate system of the image of the first marker; then, based on the conversion relationship between the first image coordinate system and the first camera coordinate system, and the image position a0'(X p1 a ,Y p1a ), determine the first camera position measurement value of the first marker (such as a1'(X c1 a ,Y c1 a )).

[0078] 2022. Determine a second camera position measurement value of the second marker based on the image of the second marker.

[0079] For example, Figure 5 As shown in (b), first, the image of the second marker is subjected to marker recognition to obtain the image position of the second marker (such as b0'(X p2 b ,Y p2 b ), wherein the image position of the second marker refers to the coordinates in the second image coordinate system, and the second image coordinate system is the reference coordinate system of the image of the second marker; then, based on the conversion relationship between the second image coordinate system and the second camera coordinate system, and the image position b0'(X p2 b ,Y p2 b ), determine the second camera position measurement value of the second marker (such as b2'(X c2 b ,Y c2 b )).

[0080] 2023. Determine a second camera position measurement value of the first marker based on an installation distance between the first image acquisition device and the second image acquisition device and a first camera position measurement value of the first marker.

[0081] For example, the center of the field of view of the second image acquisition device can be used as the origin of the second camera coordinate system. c1 (0,0), assuming that the same marker is in the Y coordinate system of the first camera c1 The value is the same as the Y value in the second camera coordinate system. c2 If the values ​​are the same, then the center of the field of view of the first image acquisition device can be known to be O through calibration or mechanical installation value. c2 (k,0), k is the distance difference between the first image acquisition device and the second image acquisition device in the x-axis direction (i.e., the installation distance k). At this time, if the coordinate position of the first marker a in the first camera coordinate system is known to be (x1, y1), then the coordinate position of the first marker a in the second camera coordinate system is (x1+k, y1).

[0082] 203. Determine a position offset of the robot based on a second camera position measurement value of the first marker and a second camera position measurement value of the second marker.

[0083] The position offset includes at least one of an angle offset and a coordinate offset.

[0084] There are many ways to determine the position offset of the robot in step 203, illustratively, including the following:

[0085] (1) The position offset is an angle offset. In step 204, only the influence of the angle offset of the robot on the offset of the robot is considered to correct the target position of the robot arm.

[0086] ① In some embodiments, Figure 8 As shown in (a) or (b), the slope of the marker calibration line L2 and the slope of the marker measurement line L1 can be calculated respectively to calculate the first angle θ, thereby obtaining the angular offset of the robot. At this time, the angular offset of the robot can be determined by referring to steps 2031A to 2033A. Step 203 includes the following steps 2031A to 2033A:

[0087] 2031A. Obtain the slope of the marker calibration line.

[0088] 2032A. Obtain a slope of a marker measurement line based on the second camera position measurement value of the first marker and the second camera position measurement value of the second marker.

[0089] 2033A. Substitute the slope of the marker calibration line and the slope of the marker measurement line into a first preset formula to calculate a first angle between the marker measurement line and the marker calibration line as the angle offset of the robot.

[0090] The first preset formula is shown in the following formula (2):

[0091] Formula (2)

[0092] In formula (2), a1 and a2 are the slopes of the marker calibration line and the marker measurement line, respectively, and θ is the first angle.

[0093] ② In some embodiments, Figure 8 As shown in (a) in the figure, in order to improve the detection speed of the angle offset, the marker calibration line and the first coordinate axis (such as X c1The first angle θ can be obtained by directly calculating the slope of the marker measurement line. At this time, the angle offset of the robot can be determined by referring to step 2031B. Step 203 includes the following steps 2031B:

[0094] 2031B. Substitute the second camera position measurement value of the first marker and the second camera position measurement value of the second marker into a second preset formula to calculate a first angle between the marker measurement line and the marker calibration line as the angle offset of the robot.

[0095] The second preset formula is shown in the following formula (3):

[0096] Formula (3)

[0097] In formula (3), (x1+k, y1) is the second camera position measurement value of the first marker, (x2, y2) is the second camera position measurement value of the second marker, and θ is the first angle.

[0098] (2) The position offset is the coordinate offset. In step 204, only the influence of the robot's coordinate offset on the robot's offset is considered to correct the robot's mechanical arm target position. At this time, the robot's coordinate offset can be determined by referring to steps 2031C to 2033C. Step 203 includes the following steps 2031C to 2033C:

[0099] 2031C. Determine the coordinates of the center point of the marker measurement line based on the second camera position measurement value of the first marker and the second camera position measurement value of the second marker.

[0100] 2032C. Determine the coordinates of the center point of the marker calibration line based on the second camera position calibration value of the first marker and the second camera position calibration value of the second marker.

[0101] 2031C. Based on the coordinates of the center point of the marker measurement line and the coordinates of the center point of the marker calibration line, determine the coordinate deviation between the center point of the marker calibration line and the center point of the marker measurement line as the coordinate offset of the robot.

[0102] For example, Fig. 9 As shown, Fig. 9Where L1 is the marker calibration line, L2 is the marker measurement line, and L3 is the marker correction line. The second camera position measurement value of the first marker a is the point coordinates of a2', and the second camera position measurement value of the second marker b is the point coordinates of b2'. The center point coordinates C2 of the straight line L2 can be calculated using the point coordinates of a2' and b2', and the center point coordinates C1 of the straight line L1 can be calculated using a2 and b2. Then, the coordinate difference △X between the center point coordinates C2 and the center point coordinates C1 is calculated. c2 , △Y c2 , the coordinate deviation (△X c2 ,△Y c2 ) as the coordinate offset of the robot.

[0103] (3) Considering that the position offset includes the coordinate offset and the angle offset, in step 204, the influence of the coordinate offset and the angle offset of the robot on the robot offset is comprehensively considered to correct the target position of the robot arm, so as to improve the control accuracy of the robot arm.

[0104] ① In some embodiments, Figure 8 (a) Figure 8 (b) or Fig. 9 As shown, the slope of the marker calibration line, the slope of the marker measurement line or the slope of the marker repair line can be calculated respectively to calculate the angle offset of the robot, and then the angle offset of the robot is obtained. At this time, the angle offset of the robot can be determined by referring to steps 2031D~203D. Step 203 includes the following steps 2031D~203D:

[0105] 2031D. Check whether the center point of the marker calibration line is the same as the center point of the marker measurement line;

[0106] 2032D. If the center point of the marker calibration line is the same as the center point of the marker measurement line, substitute the slope of the marker calibration line and the slope of the marker measurement line into the first preset formula to calculate the first angle between the marker measurement line and the marker calibration line as the angle offset of the robot.

[0107] Among them, the first preset formula is , a1 and a2 are the slopes of the marker calibration line and the marker measurement line respectively, and θ is the first angle.

[0108] 2033D. If the center point of the marker calibration line is different from the center point of the marker measurement line, the slope of the marker calibration line and the slope of the marker correction line are substituted into the third preset formula to calculate the second angle between the marker correction line and the marker calibration line as the angle offset of the robot.

[0109] The third preset formula is shown in the following formula (4):

[0110] Formula (4)

[0111] In formula (4), a1 and a2' are the slopes of the marker calibration line and the marker measurement line, respectively, and θ' is the second angle.

[0112] The marker correction connection line can be obtained by referring to the following steps A1-A2. Steps A1-A2 are as follows:

[0113] A1. Obtain the coordinate deviation between the center point of the marker calibration line and the center point of the marker measurement line as the coordinate offset of the robot.

[0114] The implementation of step A1 may refer to steps 2031C to 2033C, which will not be repeated here.

[0115] A2. Based on the coordinate offset of the robot, the marker measurement line is corrected to obtain a marker correction line.

[0116] For example, Fig. 9 As shown, the robot's coordinate offset (△X c2 ,△Y c2 ), place the marker measurement line L2 at X c2 Move △X in the axial direction c2 , in Y c2 Move △Y in the axis direction c2 , until the center point C2 of the moved marker measurement line L2 coincides with the center point C1 of the marker calibration line L1, the marker correction line L3 is obtained.

[0117] When L2 moves to L3, marker a corresponds to point A and marker point B. At this time, the coordinates of point A and point B can be used to obtain the slope of the marker correction line L3.

[0118] It can be seen that, on the first aspect, by calculating the angular offset of the robot and using it to update the calibrated target position, the angular offset of the robot relative to the standard position can be corrected, thereby improving the control accuracy of the robotic arm.

[0119] Secondly, by calculating the coordinate offset of the robot and updating the calibrated target position, the coordinate offset of the robot relative to the standard position can be corrected, thereby improving the control accuracy of the robotic arm.

[0120] Thirdly, when the center point of the marker calibration line is different from the center point of the marker measurement line, the robot's coordinate offset is used to correct the marker measurement line to obtain the marker correction line, and then the robot's angle offset is calculated based on the slope of the marker calibration line and the slope of the marker correction line. This can eliminate the influence of the robot's coordinate offset on the robot's angle offset calculation, avoid the problem of inaccurate angle offset calculation of the robot due to the existence of the coordinate offset, improve the robot's angle offset, and thus improve the control accuracy of the robotic arm.

[0121] Fourthly, by making the center point of the marker calibration line different from the center point of the marker measurement line, the second camera position measurement value of the first marker and the second camera position measurement value of the second marker are corrected using the robot's coordinate offset to obtain the second camera position correction value of the first marker and the second camera position correction value of the second marker, and then the robot's angle offset is calculated based on the second camera position correction value of the first marker and the second camera position correction value of the second marker. This can eliminate the influence of the robot's coordinate offset on the robot's angle offset calculation, avoid the problem of inaccurate angle offset calculation of the robot due to the existence of the coordinate offset, improve the robot's angle offset, and thus improve the control accuracy of the robotic arm.

[0122] ② In some embodiments, in order to improve the detection speed of the angle offset, the marker calibration line and the first coordinate axis (such as X c1 The robot is parallel to or coincides with the axis, so that the slope of the marker measurement line can be directly calculated to obtain the robot's angular offset θ. At this time, the robot's angular offset can be determined by referring to steps 2031E~2033E. Step 203 includes the following steps 2031E~203E:

[0123] 2031E, Check whether the center point of the marker calibration line is the same as the center point of the marker measurement line

[0124] 2032E. If the center point of the marker calibration line is the same as the center point of the marker measurement line, the second camera position measurement value of the first marker and the second camera position measurement value of the second marker are substituted into the second preset formula to calculate the first angle between the marker measurement line and the marker calibration line as the angle offset of the robot.

[0125] Among them, the second preset formula is , (x1+k,y1) is the second camera position measurement value of the first marker, (x2,y2) is the second camera position measurement value of the second marker, and θ is the first angle.

[0126] 2033E. If the center point of the marker calibration line is different from the center point of the marker measurement line, the second camera position correction value of the first marker and the second camera position correction value of the second marker are substituted into the fourth preset formula to calculate the angle offset of the robot.

[0127] The fourth preset formula is shown in the following formula (5):

[0128] Formula (5)

[0129] In formula (5), ((x1+k)', y1') is the second camera position correction value of the first marker, (x2', y2') is the second camera position correction value of the second marker, and θ' is the angular offset of the robot.

[0130] The second camera position correction value of the first marker and the second camera position correction value of the second marker may be determined by referring to the following steps B1 and B2. Steps B1 and B2 are as follows:

[0131] B1. Obtain the coordinate deviation between the center point of the marker calibration line and the center point of the marker measurement line as the coordinate offset of the robot.

[0132] The implementation of step B1 may refer to steps 2031C to 2033C, which will not be repeated here.

[0133] B2. Based on the coordinate offset of the robot, the second camera position measurement value of the first marker and the second camera position measurement value of the second marker are corrected to obtain the second camera position correction value of the first marker and the second camera position correction value of the second marker.

[0134] For example, refer to Fig. 9 , the second camera position measurement value of the first marker is the coordinates of point a2' (X c2 a ,Y c2 a ), the second camera position measurement value of the second marker is the coordinate of point b2' (X c2 b ,Y c2 b ), the robot's coordinate offset (△X c2 ,△Y c2), the second camera position correction value (X) of the first marker can be corrected. c2 a +△X c2 ,Y c2 a +△Y c2 ), the second camera position correction value of the second marker (X c2 b +△X c2 ,Y c2 b +△Y c2 At this time, the second camera position correction value (X c2 a +△X c2 ,Y c2 a +△Y c2 ) corresponds to the coordinates of point A, the second camera position correction value of the second marker (X c2 b +△X c2 ,Y c2 b +△Y c2 ) corresponds to the coordinates of point B.

[0135] It can be seen that considering the coordinate deviation (△x, △y) of the marker (that is, the deviation between the coordinates of the marker when measured and the coordinates of the marker when calibrated), part of it is caused by the rotation of the robot (such as rotation around the coordinate axis) and part of it is caused by the translation of the robot (such as translation along the coordinate axis). By determining the angular offset and coordinate offset of the robot through steps 2031D~2033D or steps 2031E~2033E, the detection accuracy of the robot position offset can be improved, thereby improving the accuracy of the updated target position of the robot arm and improving the control accuracy of the robot arm.

[0136] 204. Based on the position offset of the robot, control the robot arm to complete a grasping or placing operation.

[0137] Specifically, since the robot is offset and performing tasks according to the calibrated target position will cause the robotic arm to complete the grasping or placement operation at a deviated position, the calibrated target position of the robotic arm can be adjusted based on the robot's angular offset. The robot's angular offset is converted to the robotic arm's coordinate system after the hand-eye calibration parameters to correct the robotic arm's angular offset and correct the position deviation of the robotic arm, so that the robotic arm returns to the standard position to complete the grasping or placement operation, ensuring that the robotic arm accurately grasps or places objects.

[0138] There are many ways to control the robot's mechanical arm to complete the grasping or placing operation in step 204, illustratively including:

[0139] (1) After correcting the angle offset of the robot arm, the robot arm is controlled to complete the grabbing or placing operation. At this time, step 204 may specifically include the following steps 2041A to 2044A:

[0140] 2041A. Obtain a calibrated target position of the robotic arm.

[0141] 2042A. Determine the angular offset of the robotic arm based on the angular offset of the robot and a first calibration conversion relationship between the robot and the robotic arm.

[0142] 2043A. Update the calibrated target position according to the angular offset of the robotic arm to obtain an updated target position of the robotic arm.

[0143] 2044A. Based on the updated target position, control the robot's mechanical arm to complete a grasping or placing operation.

[0144] For example, the target position is (X r ,Y r ,Z r ,rx,ry,rz), ​​the angle offset of the robot arm is (△rx,△ry,△rz), then the updated target position of the robot arm can be (X r ,,Y r ,Z r ,rx+△rx,ry+△ry,rz+△rz). According to the updated target position (X r ,,Y r ,Z r ,rx+△rx,ry+△ry,rz+△rz) controls the robot's mechanical arm to extend to accurately complete the grasping or placing operation.

[0145] (2) After correcting the coordinate offset of the robot arm, the robot arm is controlled to complete the grabbing or placing operation. At this time, step 204 may specifically include the following steps 2041B to 2043B:

[0146] 2041B. Determine the coordinate offset of the robot based on the coordinate offset of the robot and a second calibration transformation relationship between the second camera coordinate system and the coordinate system of the robotic arm.

[0147] 2042B. Based on the coordinate offset of the robotic arm, the calibrated target position is updated to obtain an updated target position of the robotic arm.

[0148] 2043B. Based on the updated target position, control the robot's mechanical arm to complete a grasping or placing operation.

[0149] For example, the target position is (X r ,Y r ,Z r ,rx,ry,rz), ​​the coordinate offset of the robot arm is (△X r ,△Y r ,△Z r ), then the updated target position of the robot arm can be (X r +△X r ,,Y r +△Y r ,Z r +△Z r ,rx,ry,rz). According to the updated target position (X r +△X r ,,Y r +△Y r ,Z r +△Z r ,rx,ry,rz) controls the robot's mechanical arm to extend to accurately complete the grasping or placing operation.

[0150] (3) After correcting the angle offset and coordinate offset of the robot arm, the robot arm is controlled to complete the grabbing or placing operation. At this time, step 204 may specifically include the following steps 2041C to 2045C:

[0151] 2041C. Obtain a calibrated target position of the robotic arm.

[0152] 2042C. Determine the angular offset of the robotic arm based on the angular offset of the robot and a first calibration conversion relationship between the robot and the robotic arm.

[0153] 2043C. Determine the coordinate offset of the robot based on the coordinate offset of the robot and a second calibration transformation relationship between the second camera coordinate system and the coordinate system of the robotic arm.

[0154] Among them, the determination of the coordinate offset of the robot can refer to the relevant instructions of steps 2031C~2033C and will not be repeated here.

[0155] 2044C. Based on the angle offset of the robotic arm and the coordinate offset of the robotic arm, the calibrated target position is updated to obtain an updated target position of the robotic arm.

[0156] 2045C. Based on the updated target position, control the robot's mechanical arm to complete the grasping or placing operation.

[0157] For example, the target position is (X r ,Yr ,Z r ,rx,ry,rz), ​​the coordinate offset of the robot arm is (△X r ,△Y r ,△Z r ), the angular offset of the robot arm is (△rx,△ry,△rz), then the updated target position of the robot arm can be (X r +△X r ,,Y r +△Y r ,Z r +△Z r ,rx+△rx,ry+△ry,rz+△rz). According to the updated target position (X r +△X r ,,Y r +△Y r ,Z r +△Z r ,rx+△rx,ry+△ry,rz+△rz) controls the robot's mechanical arm to extend to accurately complete the grasping or placing operation.

[0158] Furthermore, in order to improve the detection accuracy of the position offset (offset angle, offset coordinates) of the robot, the installation distance between the first image acquisition device and the second image acquisition device is as large as possible. Figure 6 As shown in (a) in FIG. 1 , the first image acquisition device and the second marker image acquisition device can be installed at the head and tail of the robot's mobile chassis, respectively, and the line between the camera center of the first image acquisition device and the camera center of the second marker image acquisition device is parallel to the edge of the mobile chassis. Figure 6 As shown in (b), the first image acquisition device and the second image acquisition device of the marker can be installed at the positions on both sides of the diagonal of the mobile chassis of the robot, respectively, so that the installation distance between the first image acquisition device and the second image acquisition device is maximized, thereby maximizing the detection accuracy of the robot's offset angle and offset coordinates. In some embodiments, the installation distance between the first image acquisition device and the second image acquisition device is greater than 1 / 2 of the length of the mobile chassis of the robot, so that the installation distance between the first image acquisition device and the second image acquisition device is far enough (such as more than 500mm) to improve the detection accuracy of the robot's offset angle and offset coordinates (such as the angle detection accuracy is improved to less than 0.1 degrees, and the coordinate detection accuracy is improved to less than 0.2mm), and the angle detection accuracy can exceed the detection accuracy of a single camera.

[0159] In order to better understand how the installation distance affects the detection accuracy of the robot's offset angle, the following Figure 7 For explanation. Figure 7As shown in (b) in the figure, taking the conversion of the marker position to the second camera coordinate system as an example, assuming that the robot stops at the standard position, after the image of the first marker a and the image of the second marker b collected by the camera are converted, the second camera position measurement value of the first marker and the second camera position measurement value of the second marker are respectively a2 and b2 (at this time, the robot arm can accurately complete the grasping or placing operation); assuming that the robot stops at a non-standard position, after the image of the first marker and the image of the second marker collected by the camera are converted, the second camera position measurement value of the first marker and the second camera position measurement value of the second marker are respectively a2' and b2' (at this time, the robot arm cannot accurately complete the grasping or placing operation, and the deviation angle θ of the robot needs to be compensated). When the deviation angle θ is the same, when the distance between camera 1 and camera 2 is larger, the deviation between point a2 and point a2' is also larger (or the deviation between point b2 and point b2' is larger), and the detection accuracy is greater. Even a small change in the deviation angle θ can be reflected in the deviation between point a2 and point a2'. Therefore, in this embodiment, the installation distance between the first image acquisition device and the second image acquisition device is as large as possible to improve the detection accuracy of the robot's offset angle, thereby improving the control accuracy of the robot arm. Figure 7 As shown in (a) in the figure, the conversion of the marker position to the first camera coordinate system is similar. It should be noted that, in this embodiment, the conversion of the marker position to the second camera coordinate system is mainly used as an example to illustrate how to determine the position offset of the robot and update the calibrated target position of the robotic arm; similarly, the marker position can also be converted to the first camera coordinate system to achieve the determination of the robot's position offset and update the calibrated target position of the robotic arm.

[0160] From the above content, it can be seen that, on the first aspect, the image of the first marker is captured by the first image acquisition device installed on the mobile chassis of the robot, and the image of the second marker is captured by the second image acquisition device installed on the mobile chassis. The second camera position measurement value of the first marker and the second camera position measurement value of the second marker are used to determine the position offset of the robot, which can correct the position deviation of the robot and thus improve the control accuracy of the robotic arm.

[0161] Secondly, by using the first image acquisition device to acquire the image of the first marker and the second image acquisition device to acquire the image of the second marker, since the two cameras are used, even very small offset angles will be magnified in the coordinate offset (for example, even if the robot has only a very small offset angle, it will be magnified in the coordinate deviation between the second camera position measurement value of the first marker and the second camera position calibration value of the first marker, or the coordinate deviation between the second camera position measurement value of the second marker and the second camera position calibration value of the second marker). Therefore, even very small offset angles can be detected, thereby improving the detection accuracy of the robot's position offset and making it simple to identify the robot's position offset.

[0162] Thirdly, by installing two image acquisition devices on the robot's mobile chassis to collect markers to detect the robot's position offset, it is possible to avoid installing the image acquisition device on a robotic arm and using the robotic arm to take pictures, thereby reducing the action of the robotic arm extending to the photo taking position to take pictures, thereby improving the detection efficiency of the robot's position offset. For example, if the image acquisition device is installed on a robotic arm for image acquisition, the entire robotic arm control operation process includes: ① the mobile chassis navigates to a certain working point; ② stops and takes pictures; ③ takes pictures; ④ detects the position offset of the robot according to the picture results; ⑤ controls the robot's robotic arm to complete the grabbing or placing operation according to the robot's position offset; In this embodiment, the image acquisition device is installed on a mobile chassis for image acquisition, and the entire robotic arm control operation process includes: ① the mobile chassis navigates to a certain working point; ② stops and takes pictures; ③ detects the robot's position offset according to the picture results; ④ controls the robot's robotic arm to complete the grabbing or placing operation according to the robot's position offset; In this way, this embodiment can omit the action of extending the robotic arm from the initial position to the photo taking position, reduce the time of controlling the robotic arm to extend to the photo taking position, and improve the detection efficiency of the robot's position offset.

[0163] Therefore, this embodiment can improve the detection accuracy of the robot's position offset, use the high-precision robot's position offset to correct the robot's position deviation, and improve the control accuracy of the robot arm.

[0164] A person of ordinary skill in the art can understand that all or part of the steps in the above-mentioned robot arm control method can be completed by instructions, or by controlling related hardware through instructions. The instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0165] To this end, an embodiment of the present application provides a computer-readable storage medium, in which a plurality of computer programs are stored, and the computer program can be loaded by a processor to execute any robot arm control method provided in the embodiment of the present application. For example, the computer program can be loaded by a processor to execute the following steps:

[0166] When the robot moves to a designated working area, an image of a first marker is acquired by a first image acquisition device installed on a mobile chassis of the robot, and an image of a second marker is acquired by a second image acquisition device installed on the mobile chassis, wherein a first marker and a second marker are arranged in the designated working area; based on the image of the first marker and the image of the second marker, a second camera position measurement value of the first marker and a second camera position measurement value of the second marker are acquired, wherein the second camera position measurement value is a coordinate position in a second camera coordinate system, and the second camera coordinate system is a camera coordinate system of the second image acquisition device; based on the second camera position measurement value of the first marker and the second camera position measurement value of the second marker, a position offset of the robot is determined, wherein the position offset includes at least one of an angle offset and a coordinate offset; based on the position offset of the robot, a mechanical arm of the robot is controlled to complete a grasping or placing operation.

[0167] The computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0168] In the above-mentioned robot's mechanical arm control method, computer-readable storage medium, and electronic device embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, refer to the relevant descriptions of other embodiments. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process and beneficial effects of the above-described computer-readable storage medium, electronic device, and corresponding units can refer to the description of the robot's mechanical arm control method in the above embodiment, and will not be repeated here.

[0169] The above is a detailed introduction to a robot arm control method, electronic device and computer-readable storage medium provided in an embodiment of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A robot arm control method, characterized in that: A first image acquisition device and a second image acquisition device are installed on the mobile chassis of the robot, and the method comprises: When the robot moves to a designated working area, an image of a first marker is collected by a first image acquisition device installed on a mobile chassis of the robot, and an image of a second marker is collected by a second image acquisition device installed on the mobile chassis, wherein the first marker and the second marker are arranged in the designated working area; Based on the image of the first marker and the image of the second marker, obtaining a second camera position measurement value of the first marker and a second camera position measurement value of the second marker, wherein the second camera position measurement value is a coordinate position in a second camera coordinate system, and the second camera coordinate system is a camera coordinate system of a second image acquisition device; Determine a position offset of the robot based on a second camera position measurement value of the first marker and a second camera position measurement value of the second marker, the position offset comprising at least one of an angle offset and a coordinate offset; Based on the position offset of the robot, controlling the robot arm to complete a grasping or placing operation; The position offset includes an angle offset and a coordinate offset; the method further includes: Obtaining a coordinate deviation between a center point of a marker calibration line and a center point of a marker measurement line as a coordinate offset of the robot; Based on the coordinate offset of the robot, the marker measurement line is corrected to obtain a marker correction line; Check whether the center point of the marker calibration line is the same as the center point of the marker measurement line; If the center point of the marker calibration line is different from the center point of the marker measurement line, the slope of the marker calibration line and the slope of the marker correction line are substituted into a third preset formula to calculate a second angle between the marker correction line and the marker calibration line as the angle offset of the robot; If the center point of the marker calibration line is the same as the center point of the marker measurement line, the slope of the marker calibration line and the slope of the marker measurement line are substituted into a first preset formula to calculate a first angle between the marker measurement line and the marker calibration line as the angle offset of the robot; The step of correcting the marker measurement line based on the coordinate offset of the robot to obtain the marker correction line includes: According to the robot's coordinate offset (△X c2 ,△Y c2 ), place the marker measurement line L2 at X c2 Move △X in the axial direction c2 , in Y c2 Move △Y in the axis direction c2 , until the center point C2 of the moved marker measurement line L2 coincides with the center point C1 of the marker calibration line L1, to obtain the marker correction line L3; The position offset includes an angle offset, and the controlling the robot arm to complete a grasping or placing operation based on the position offset of the robot includes: Obtaining a calibrated target position of the robotic arm; Determining the angular offset of the robotic arm based on the angular offset of the robot and a first calibration conversion relationship between the robot and the robotic arm; updating the calibrated target position according to the angular offset of the robotic arm to obtain an updated target position of the robotic arm; Based on the updated target position, the robot's mechanical arm is controlled to complete a grasping or placing operation.

2. The robot arm control method according to claim 1, characterized in that: The position offset also includes a coordinate offset, and the calibration target position is updated according to the angle offset of the robotic arm to obtain the updated target position of the robotic arm, including: Determine the coordinate offset of the robot based on the coordinate offset of the robot and a second calibration transformation relationship between the second camera coordinate system and the coordinate system of the robot arm; Based on the angular offset of the robotic arm and the coordinate offset of the robotic arm, the calibrated target position is updated to obtain an updated target position of the robotic arm.

3. The robot arm control method according to claim 1, characterized in that: The position offset includes an angle offset, and determining the position offset of the robot based on a second camera position measurement value of the first marker and a second camera position measurement value of the second marker includes: Check whether the center point of the marker calibration line is the same as the center point of the marker measurement line; If the center point of the marker calibration line is the same as the center point of the marker measurement line, the slope of the marker calibration line and the slope of the marker measurement line are substituted into the first preset formula to calculate the first angle between the marker measurement line and the marker calibration line as the angle offset of the robot, wherein the first preset formula is: , a1 and a2 are the slopes of the marker calibration line and the marker measurement line respectively, and θ is the first angle.

4. The robot arm control method according to claim 3, characterized in that: The method further comprises: If the center point of the marker calibration line is different from the center point of the marker measurement line, then obtaining the coordinate deviation between the center point of the marker calibration line and the center point of the marker measurement line as the coordinate offset of the robot; Based on the coordinate offset of the robot, the marker measurement line is corrected to obtain a marker correction line; Substitute the slope of the marker calibration line and the slope of the marker correction line into the third preset formula to calculate the second angle between the marker correction line and the marker calibration line as the angle offset of the robot, wherein the third preset formula is: , a1 and a2' are the slopes of the marker calibration line and the marker measurement line respectively, and θ' is the second angle.

5. The robot arm control method according to claim 1, characterized in that: The position offset includes an angle offset; The determining the position offset of the robot based on the second camera position measurement value of the first marker and the second camera position measurement value of the second marker comprises: Check whether the center point of the marker calibration line is the same as the center point of the marker measurement line; If the center point of the marker calibration line is the same as the center point of the marker measurement line, the second camera position measurement value of the first marker and the second camera position measurement value of the second marker are substituted into the second preset formula to calculate the first angle between the marker measurement line and the marker calibration line as the angle offset of the robot, wherein the second preset formula is: , (x1+k,y1) is the second camera position measurement value of the first marker, (x2,y2) is the second camera position measurement value of the second marker, and θ is the first angle.

6. The robot arm control method according to claim 5, characterized in that: The method further comprises: If the center point of the marker calibration line is different from the center point of the marker measurement line, then obtaining the coordinate deviation between the center point of the marker calibration line and the center point of the marker measurement line as the coordinate offset of the robot; Based on the coordinate offset of the robot, correct the second camera position measurement value of the first marker and the second camera position measurement value of the second marker to obtain the second camera position correction value of the first marker and the second camera position correction value of the second marker; Substitute the second camera position correction value of the first marker and the second camera position correction value of the second marker into the fourth preset formula to calculate the angle offset of the robot, where the fourth preset formula is: , ((x1+k)', y1') is the second camera position correction value of the first marker, (x2', y2') is the second camera position correction value of the second marker, and θ' is the angular offset of the robot.

7. The robot arm control method according to claim 1, characterized in that: The acquiring, based on the image of the first marker and the image of the second marker, a second camera position measurement value of the first marker and a second camera position measurement value of the second marker comprises: determining a first camera position measurement of the first marker based on the image of the first marker; determining a second camera position measurement of the second marker based on the image of the second marker; A second camera position measurement value of the first marker is determined based on an installation distance between the first image acquisition device and the second image acquisition device and a first camera position measurement value of the first marker.

8. An electronic device, characterized in that: The method comprises a processor and a memory, wherein a computer program is stored in the memory, and when the processor calls the computer program in the memory, the robot arm control method according to any one of claims 1 to 7 is executed.

9. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and the computer program is loaded by a processor to execute the robot arm control method according to any one of claims 1 to 7.