Workpiece grabbing method, device, system, electronic equipment and storage medium

Abstract

Embodiments of the present application provide a workpiece grabbing method, device, system, electronic equipment and storage medium. The workpiece grabbing method comprises: obtaining a first coordinate of a grabbing end of a mechanical arm in a space coordinate system; obtaining a workpiece image collected by a camera; determining a second coordinate of the workpiece in the space coordinate system according to the workpiece image and calibration information; generating control information according to the first coordinate and the second coordinate, and sending the control information to the mechanical arm, so that the mechanical arm grabs the workpiece according to the control information. According to the workpiece image collected by the camera, the position of the workpiece in the space coordinate system can be determined, and then the control information for controlling the mechanical arm is generated in combination with the position of the grabbing end of the mechanical arm. The workpiece at any angle and position can be automatically grabbed, and the pose of the workpiece does not need to be limited when the grabbing end is grabbed, so that the production efficiency is improved.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a workpiece grasping method, apparatus, system, electronic device, and storage medium. Background Technology

[0002] In the production process, to reduce labor costs and improve safety, robotic arms are used to replace manual labor in grasping workpieces for tasks such as transfer, flipping, and welding. The robotic arm simulates how the human eye perceives objects and drives it to grasp them. However, the grasping effect of the robotic arm differs significantly from that of manual grasping.

[0003] Currently, to address the aforementioned differences, a common approach is to place the workpiece directly below the robotic arm and manually position it at a specific angle and location to ensure the robotic arm's gripping effectiveness.

[0004] However, manually placing workpieces at specific angles and positions is time-consuming and results in low production efficiency. Summary of the Invention

[0005] To address the aforementioned technical problems, embodiments of the present invention provide a workpiece gripping method, apparatus, system, electronic device, and storage medium to at least solve or alleviate the problems described above.

[0006] According to a first aspect of the present application, a workpiece gripping method is provided, comprising: acquiring a first coordinate of the gripping end of a robotic arm in a spatial coordinate system; acquiring a workpiece image captured by a camera, wherein the relative positions of the camera and the base of the robotic arm in the spatial coordinate system are fixed, and the workpiece image includes an image of a workpiece located within a gripping area; determining a second coordinate of the workpiece in the spatial coordinate system based on the workpiece image and calibration information, wherein the calibration information is used to indicate the mapping relationship between the pixel coordinate system of the image captured by the camera and the spatial coordinate system; generating control information based on the first coordinate and the second coordinate, and sending the control information to the robotic arm so that the robotic arm grips the workpiece according to the control information.

[0007] According to a second aspect of the present application, a workpiece gripping device is provided, comprising: a first acquisition module, configured to acquire a first coordinate of the gripping end of a robotic arm in a spatial coordinate system; a second acquisition module, configured to acquire a workpiece image captured by a camera, wherein the relative positions of the camera and the base of the robotic arm in the spatial coordinate system are fixed, and the workpiece image includes an image of a workpiece located within a gripping area; a determination module, configured to determine a second coordinate of the workpiece in the spatial coordinate system based on the workpiece image and calibration information, wherein the calibration information is used to indicate the mapping relationship between the pixel coordinate system of the image captured by the camera and the spatial coordinate system; and a control module, configured to generate control information based on the first coordinate and the second coordinate, and send the control information to the robotic arm so that the robotic arm grips the workpiece according to the control information.

[0008] According to a third aspect of the embodiments of this application, a workpiece gripping system is provided, comprising: a robotic arm, a camera, and a workpiece gripping device; the workpiece gripping device is used to execute the workpiece gripping method of the first aspect described above; the camera is used to acquire an image of a workpiece located within a gripping area, obtain a workpiece image, and send the workpiece image to the workpiece gripping device; the robotic arm is used to receive control information sent by the workpiece gripping device and grip the workpiece according to the control information.

[0009] According to a fourth aspect of the present application, an electronic device is provided, including: a processor, a communication interface, a memory, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other through the communication bus; the memory is used to store at least one executable instruction, which causes the processor to perform an operation corresponding to the workpiece gripping method of the first aspect described above.

[0010] According to a fifth aspect of the embodiments of this application, a computer storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the workpiece gripping method of the first aspect described above.

[0011] According to a sixth aspect of the embodiments of this application, a computer program product is provided, including computer instructions that instruct a computing device to perform an operation corresponding to the workpiece gripping method of the first aspect described above.

[0012] As can be seen from the above technical solution, the position of the workpiece in the spatial coordinate system can be determined by the workpiece image captured by the camera. Then, combined with the position of the gripping end of the robotic arm, control information for controlling the robotic arm can be generated. Workpieces at any angle and position can be automatically gripped without limiting the pose of the workpiece when gripping at the gripping end of the robotic arm, thus improving production efficiency. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings.

[0014] Figure 1 This is a flowchart of the steps of a workpiece gripping method according to an embodiment of this application;

[0015] Figure 2 This is a flowchart of the steps of a calibration information determination method according to an embodiment of this application;

[0016] Figure 3 This is a flowchart of the steps of a calibration image acquisition method according to an embodiment of this application;

[0017] Figure 4 This is a schematic diagram showing the position of the calibration plate during image acquisition according to one embodiment of this application;

[0018] Figure 5 This is a flowchart of the steps of a calibration information verification method according to an embodiment of this application;

[0019] Figure 6 This is a flowchart of the steps of a control information generation method according to an embodiment of this application;

[0020] Figure 7 This is a schematic diagram of a workpiece gripping device according to an embodiment of this application;

[0021] Figure 8 This is a schematic diagram of a workpiece gripping system according to an embodiment of this application;

[0022] Figure 9 This is a schematic diagram of an electronic device according to an embodiment of this application.

[0023] List of reference numerals in the attached diagram:

[0024]

[0025] 101: Obtain the first coordinate of the robotic arm's gripping end in the spatial coordinate system. 102: Acquire workpiece images captured by the camera. 103: Based on the workpiece image and calibration information, determine the second coordinate of the workpiece in the spatial coordinate system. 104: Generate control information based on the first and second coordinates, and send the control information to the robotic arm. 201: Based on preset X-axis, Y-axis, and Z-axis translations, as well as rotations around the X-axis, Y-axis, and Z-axis, the robotic arm is controlled to perform multiple translations and / or rotations in the spatial coordinate system. Images of the calibration plate are captured by a camera after each translation and / or rotation, resulting in multiple calibration images. 202: Determine the calibration information based on the position of the calibration board image in the calibration image, and the translation and rotation of the robotic arm when acquiring the calibration board image. 301: Based on the preset X-axis and Y-axis translation amounts, control the robotic arm to perform 5 translations in a first plane perpendicular to the Z-axis, and acquire images of the calibration plate after each translation using a camera, obtaining 5 calibration images captured by the camera. 302: Based on the preset X-axis and Y-axis translation amounts, control the robotic arm to perform four translations in a second plane perpendicular to the Z-axis, and acquire images of the calibration plate after each translation using a camera, obtaining four calibration images captured by the camera. 303: Based on preset X-axis translation, Y-axis translation, rotation around the X-axis, rotation around the Y-axis, and rotation around the Z-axis, the robotic arm is controlled to translate within the first plane and then rotate. A camera captures images of the calibration plate after each rotation of the robotic arm, resulting in four calibration images. 401: Identify multiple calibration points set on the calibration board from the calibration image, and obtain the third coordinates of each calibration point in the pixel coordinate system. 402: Based on the calibration information, convert the third coordinate of the calibration point into the fourth coordinate in the spatial coordinate system. 403: Verify the accuracy of the calibration information based on the fourth coordinates of multiple calibration points. 4031: Obtain the attribute information of the calibration board. 4032: Based on the fourth coordinates of multiple calibration points, determine the first distance between the Nth calibration point in the i-th row of calibration points on the calibration board and the first calibration point in the (i+1)-th row of calibration points. 4033: Based on the attribute information, determine the second distance between the Nth calibration point in the i-th row of calibration points on the calibration board and the first calibration point in the (i+1)-th row of calibration points. 4034: Verify the accuracy of the calibration information based on the difference between the first distance and the second distance. 501: Generate candidate control information based on the first and second coordinates. 502: Construct a virtual safety frame for the workpiece based on the second coordinate. 503: Based on the virtual safety frame, determine whether a collision will occur when the robotic arm grasps the workpiece according to the selected control information: 504: If no collision occurs when the robotic arm grasps the workpiece based on the selected control information, then the selected control information is determined as the control information. Detailed Implementation

[0026] To enable those skilled in the art to better understand the technical solutions in the embodiments of this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art should fall within the protection scope of the embodiments of this application.

[0027] In the development of industrial automation, to improve industrial production efficiency, production modes have gradually shifted to using machines and machine systems to replace manual operation for processing and production. Taking workpiece production as an example, currently, semi-automatic production methods are mainly adopted. The position of the workpiece to be grasped is adjusted to a fixed position manually so that a robotic arm can grasp it, resulting in low production efficiency. To improve the degree of industrial automation and to grasp workpieces through automatic control, this application proposes a workpiece grasping method, device, system, electronic device, and storage medium, the specific implementation of which is as follows:

[0028] Workpiece gripping method

[0029] Figure 1 This is a workpiece gripping method according to one embodiment of this application, such as... Figure 1 As shown, the workpiece gripping method 100 includes the following steps:

[0030] Step 101: Obtain the first coordinate of the gripping end of the robotic arm in the spatial coordinate system.

[0031] To automate the control of the robotic arm, the first step is to obtain the initial coordinates of the gripping end of the robotic arm in a spatial coordinate system. The workpiece coordinate system can be a Cartesian coordinate system or a coordinate system defined by the operator to represent the position and orientation of objects in the production area. Taking a Cartesian coordinate system as an example, the initial coordinates can be (2, 2, 7), representing the distances of the workpiece relative to the origin on the X, Y, and Z axes, respectively.

[0032] Step 102: Acquire the workpiece image captured by the camera.

[0033] A camera for capturing workpiece images is fixedly installed above the gripping area. When the camera is positioned in a fixed location, the workpiece within the gripping area is within the camera's image capture area; therefore, the workpiece image includes the image of the workpiece located within the gripping area. Simultaneously, the relative positions of the camera and the robotic arm's base in the spatial coordinate system are fixed. Since the robotic arm's base is generally fixed in position after installation, except in special circumstances such as disassembly, the camera's position in the spatial coordinate system is also relatively fixed. This allows for minimizing interference, collisions, and field-of-view obstruction factors at the camera's installation location, thereby improving the quality of image acquisition and extending the camera's lifespan. Furthermore, because the camera's position is fixed, its extrinsic parameters are determined, reducing the complexity of calibration calculations.

[0034] After the camera captures an image of the workpiece in the gripping area, the workpiece image is acquired.

[0035] Step 103: Determine the second coordinate of the workpiece in the spatial coordinate system based on the workpiece image and calibration information.

[0036] After acquiring the workpiece image, the coordinates of the workpiece in the pixel coordinate system of the image captured by the camera can be determined first. Then, based on the calibration information, these coordinates are transformed into coordinates in the spatial coordinate system to determine the second coordinate of the workpiece in the spatial coordinate system within the workpiece image. The calibration information indicates the mapping relationship between the pixel coordinate system and the spatial coordinate system of the image captured by the camera.

[0037] Step 104: Generate control information based on the first and second coordinates, and send the control information to the robotic arm.

[0038] After obtaining the first and second coordinates, the displacement can be determined based on the difference between them, thereby generating control information. Alternatively, the displacement vector from the first coordinate to the second coordinate can be determined based on the first and second coordinates, also generating control information. Once the control information is obtained, it is sent to the robotic arm, enabling it to move within the spatial coordinate system according to the control information and grasp the workpiece after the movement is complete.

[0039] In this embodiment, the workpiece image captured by the camera can determine the position of the workpiece in the spatial coordinate system. Then, combined with the position of the gripping end of the robotic arm, control information for controlling the robotic arm is generated. Workpieces at any angle and position can be automatically gripped without limiting the posture of the workpiece when gripping at the gripping end. This eliminates the step of manual placement of the workpiece, improves the degree of industrial automation, and thus improves production efficiency.

[0040] In one possible implementation, image samples of the workpiece can be acquired, and annotation results of feature points within the workpiece can be obtained. Based on the image samples and annotation results, a visual model is built. The workpiece image is then input into the visual model to obtain the recognition result output by the model. If the recognition result indicates the presence of a workpiece in the image, the workpiece in the image is then captured. Feature points can be points on the contour line of the workpiece in the image sample. The visual model can be built by inputting the image samples and annotation results of the workpiece into a deep learning network model for training. The visual model is obtained after the model converges.

[0041] In this embodiment of the application, by establishing a visual model, the workpiece in the gripping area can be identified, avoiding the problem of non-target gripping workpieces being mistakenly transported and then gripped by the gripping section of the robotic arm, thus improving production safety.

[0042] Figure 2 This is a flowchart of the steps of a calibration information determination method according to an embodiment of this application, as follows: Figure 2 As shown, the calibration information determination method 200 includes the following steps:

[0043] Step 201: Based on the preset X-axis translation, Y-axis translation, and Z-axis translation, as well as the rotation around the X-axis, Y-axis, and Z-axis, control the robotic arm to perform multiple translations and / or rotations in the spatial coordinate system, and use a camera to capture images of the calibration plate after each translation and / or rotation of the robotic arm, thereby obtaining multiple calibration images captured by the camera.

[0044] The spatial coordinate system is a three-dimensional Cartesian coordinate system. To determine the transformation relationship between the pixel coordinate system of the image captured by the camera and the three-dimensional Cartesian coordinate system, the camera needs to be calibrated. First, a calibration plate is connected to the gripping end of the robotic arm. Then, based on the preset X-axis translation, Y-axis translation, and Z-axis translation, as well as the rotation around the X-axis, Y-axis, and Z-axis, the robotic arm is controlled to perform multiple translations and / or rotations in the spatial coordinate system. The X-axis, Y-axis, and Z-axis translations can be the same or different, as can the rotations around the X-axis, Y-axis, and Z-axis. For example, the X-axis translation could be 5cm, the Y-axis translation 3cm, and the Z-axis translation 5cm; the rotations around the X-axis could be 5°, the Y-axis 7°, and the Z-axis 7°. However, since the camera is positioned above the grasping area, to avoid the calibration plate being directly below the camera during calibration, which could result in excessive rotation and prevent the camera from correctly displaying the calibration plate's position, the rotations around the X-axis, Y-axis, and Z-axis can all be set to be less than 90°. If a rotation exceeding 90° is required, a marking on the calibration plate to identify the front and back sides is necessary to prevent misjudgment of the calibration plate's rotation when it exceeds 90°. During multiple translations and / or rotations, the camera will capture multiple calibration images. Specifically, the camera will capture one calibration image after each translation and / or rotation. Therefore, the multiple calibration images include images of the calibration plate captured by the camera when the robot arm has offset in the X-axis, Y-axis and Z-axis directions, and images of the calibration plate captured by the camera when the robot arm has rotated around the X-axis, Y-axis and Z-axis directions.

[0045] Step 202: Determine the calibration information based on the position of the calibration plate image in the calibration image and the translation and rotation of the robotic arm when acquiring the calibration plate image.

[0046] After obtaining multiple calibration images, the position of the calibration board in the calibration images is first determined, such as the pixel coordinates of the calibration board in the pixel coordinate system. Then, based on the translation and rotation of the robotic arm when acquiring the calibration board images, the spatial coordinates of the calibration board in the spatial coordinate system when acquiring the calibration board images are determined. Finally, based on the pixel coordinates of multiple sets of calibration boards and their corresponding spatial coordinates, the transformation relationship between the pixel coordinate system and the spatial coordinate system is determined, and this transformation relationship is used as calibration information.

[0047] It should be noted that this application is not limited to using pixel coordinates and spatial coordinates to determine the transformation relationship between the pixel coordinate system and the spatial coordinate system. Other methods that can characterize the position of the calibration board in the pixel coordinate system and the position of the calibration board in the spatial coordinate system can also determine the transformation relationship between the pixel coordinate system and the spatial coordinate system.

[0048] In this embodiment of the application, a calibration plate is translated and rotated by a robotic arm. After each translation and / or rotation of the calibration plate, the camera captures an image of the third workpiece. Based on the translation and / or rotation of the calibration plate and the position of the calibration plate image in the image captured by the camera, the mapping relationship between the pixel coordinate system and the spatial coordinate system of the image captured by the camera is determined, and calibration information reflecting the mapping relationship can be obtained.

[0049] In one possible implementation, the rotation around the X-axis, the rotation around the Y-axis, and the rotation around the Z-axis are all between 25° and 45°.

[0050] When photographing the rotated calibration plate with a camera, due to the principle of perspective, if the rotation angle is too large, it will cause a large deformation of the calibration plate in the image, resulting in errors in the calibration results. At the same time, in order to reduce the computational complexity, the rotation amount is limited to between 25° and 45°, such as 30°, 40°, etc.

[0051] In this embodiment of the application, by setting the rotation amount around the X-axis, the rotation amount around the Y-axis, and the rotation amount around the Z-axis to all be between 25° and 45°, the accuracy of the confirmed calibration information and the efficiency of determining the calibration information can be improved.

[0052] In one possible implementation, the process of generating control information may include: calculating the X-axis displacement offset, Y-axis displacement offset, Z-axis displacement offset, and rotation angles around the X-axis, Y-axis, and Z-axis relative to the second coordinate; and then generating control information based on the X-axis displacement offset, Y-axis displacement offset, Z-axis displacement offset, and rotation angles around the X-axis, Y-axis, and Z-axis.

[0053] To grasp a workpiece, we first calculate the first coordinate representing the position of the robotic arm's grasping end in the spatial coordinate system, and the X-axis, Y-axis, and Z-axis displacements relative to the second coordinate representing the position of the workpiece to be grasped in the spatial coordinate system, as well as the rotation angles around the X-axis, Y-axis, and Z-axis. From this, we can determine that the robotic arm's grasping end will reach the position of the workpiece after moving according to the X-axis, Y-axis, and Z-axis displacements and the rotation angles around the X-axis, Y-axis, and Z-axis. Therefore, control information is generated based on the X-axis, Y-axis, and Z-axis displacements and the rotation angles around the X-axis, Y-axis, and Z-axis. This control information can be a text file containing the above parameters or a control command, etc., without limitation here.

[0054] It should be noted that during the movement of the robotic arm, it can perform only translation, for example, translation based on one to three displacement offsets, or it can perform only rotation, for example, rotation based on one to three rotation angles simultaneously. Furthermore, it can perform both translation and rotation simultaneously; for example, translation based on one to three displacement offsets followed by rotation based on one to three rotation angles. The process of performing both translation and rotation simultaneously can also involve translation based on one to three displacement offsets while simultaneously rotating based on one to three rotation angles; there are no limitations on this.

[0055] The robotic arm adjusts itself based on the displacement offset and rotation angle in the control information to grip the workpiece.

[0056] After determining the control information, the robotic arm adjusts itself according to the displacement offset and rotation angle in the control information, thereby moving the gripping end of the robotic arm to the position of the second coordinate to grip the workpiece.

[0057] In this embodiment of the application, by calculating the X-axis displacement offset, Y-axis displacement offset, Z-axis displacement offset, and rotation angles around the X-axis, Y-axis, and Z-axis relative to the second coordinate in the spatial coordinate system, and then using these as control information for the robotic arm, the robotic arm can translate or rotate relative to the first coordinate according to the control information, so that the robotic arm can grasp the workpiece located in the grasping area.

[0058] Figure 3 This is a flowchart of the steps of a calibration image acquisition method according to an embodiment of this application, as follows: Figure 3 As shown, the calibration image acquisition method 300 includes the following steps:

[0059] Step 301: Based on the preset X-axis translation and Y-axis translation, control the robotic arm to perform 5 translations in the first plane perpendicular to the Z-axis, and use a camera to capture images of the calibration plate after each translation of the robotic arm, thus obtaining 5 calibration images captured by the camera.

[0060] The specific method for obtaining calibration images is to first control the robotic arm to perform 5 translations in a first plane perpendicular to the Z-axis according to the preset X-axis translation and Y-axis translation. After each translation of the robotic arm, the camera will capture an image of the calibration plate, and 5 calibration images will be obtained in this process.

[0061] Step 302: Based on the preset X-axis translation and Y-axis translation, control the robotic arm to perform 4 translations in the second plane perpendicular to the Z-axis, and use a camera to capture images of the calibration plate after each translation of the robotic arm, thus obtaining 4 calibration images captured by the camera.

[0062] Based on the preset X-axis and Y-axis translation amounts, the robotic arm is controlled to perform four translations in a second plane perpendicular to the Z-axis. After each translation, the camera captures an image of the calibration plate, thus obtaining four calibration images in the process.

[0063] The distance between the first plane and the second plane is equal to the preset Z-axis translation. The position of the first plane can be determined according to the focal length of the camera. For example, when the focal length of the camera is 1.25m, the distance between the first plane and the camera is 1.25m. If the Z-axis translation is 5cm, the distance between the second plane and the camera can be 1.2m or 1.3m.

[0064] Step 303: Based on the preset X-axis translation, Y-axis translation, rotation around the X-axis, rotation around the Y-axis, and rotation around the Z-axis, control the robotic arm to translate in the first plane and then rotate. The camera captures images of the calibration plate after each rotation of the robotic arm, obtaining four calibration images captured by the camera.

[0065] Based on the preset X-axis translation, Y-axis translation, rotation around the X-axis, rotation around the Y-axis, and rotation around the Z-axis, the robotic arm is controlled to translate in the first plane and then rotate. Each time the robotic arm completes its movement, the camera will capture an image of the calibration plate, and four calibration images will be obtained in this process.

[0066] It should be noted that during the process of the robotic arm performing 5 translations in the first plane perpendicular to the Z-axis, 4 translations in the second plane perpendicular to the Z-axis, and rotating after translation in the first plane, the translation and rotation amounts can be adjusted to avoid the robotic arm appearing in repeated positions in order to ensure the richness of the calibration image.

[0067] In one example, the position of the calibration plate when acquiring calibration images can be as follows: Figure 4 As shown, the calibration plate is first moved from its initial position to position 1 based on the X-axis and / or Y-axis translation. Then, based on the X-axis translation, it is moved from position 1 to position 2 in the opposite direction of the X-axis. Next, based on the X-axis and Y-axis translation, it is moved from position 2 to position 3 in the positive direction of the X-axis and the opposite direction of the Y-axis. Then, based on the X-axis and Y-axis translation, it is moved from position 3 to position 4 in the positive direction of the X-axis and the positive direction of the Y-axis. Finally, based on the X-axis and Y-axis translation, it is moved from position 4 to position 5 in the opposite direction of the X-axis and the positive direction of the Y-axis. It should be noted that the initial position can also be position 1, in which case the calibration plate directly starts translating from position 1.

[0068] Then, according to the preset Z-axis translation amount, the robotic arm is controlled to move in the Z-axis direction to the second plane perpendicular to the Z-axis. Then, according to the Y-axis translation amount, the calibration plate is moved from position 5 to position 6 in the opposite direction of the Y-axis. Then, according to the X-axis and Y-axis translation amounts, the calibration plate is moved from position 6 to position 7 in the opposite direction of the X-axis and Y-axis. Then, according to the X-axis and Y-axis translation amounts, the calibration plate is moved from position 7 to position 8 in the positive direction of the X-axis and the opposite direction of the Y-axis. Finally, according to the X-axis and Y-axis translation amounts, the calibration plate is moved from position 8 to position 9 in the positive direction of the X-axis and the positive direction of the Y-axis.

[0069] Then, the robotic arm is controlled to move back to the first plane in the Z-axis direction. Based on the X-axis translation, Y-axis translation, rotation around the X-axis, rotation around the Y-axis, and rotation around the Z-axis, the calibration plate is moved in the opposite direction of the X-axis and the positive direction of the Y-axis and then rotated to move from position 9 to position 10. Then, based on the Y-axis translation, rotation around the X-axis, rotation around the Y-axis, and rotation around the Z-axis, the calibration plate is moved in the opposite direction of the Y-axis and then rotated to move from position 10 to position 11. Then, based on the X-axis translation, Y-axis translation, rotation around the X-axis, rotation around the Y-axis, and rotation around the Z-axis, the calibration plate is moved in the positive direction of the X-axis and the positive direction of the Y-axis and then rotated to move from position 11 to position 12. Finally, based on the Y-axis translation, rotation around the X-axis, rotation around the Y-axis, and rotation around the Z-axis, the calibration plate is moved in the opposite direction of the Y-axis and then rotated to move from position 12 to position 13. During this process, the camera acquires images of the calibration plate at positions 1 to 13, obtaining a total of 13 calibration images. This 13-point calibration method allows for the determination of calibration information with relatively simple computation. Furthermore, since the 13-point calibration includes samples of translation around the Z-axis, rotation around the X-axis, and rotation around the Y-axis, the determined calibration information can be used to transform 3D coordinates. This enables the robotic arm to adjust more poses to grasp workpieces with more complex poses within the grasping area.

[0070] It should be noted that the X-axis translation amount during each translation process of the calibration plate can be the same as or different from the X-axis translation amount during other translation processes, and the Y-axis translation amount can be the same as or different.

[0071] In this embodiment, the camera acquires calibration images through different movement methods, which can provide sufficient samples for calculating calibration information, thereby improving the accuracy of the calibration information.

[0072] Figure 5 This is a flowchart of the steps of a calibration information verification method according to an embodiment of this application, as follows: Figure 5 As shown, the calibration information verification method 400 includes the following steps:

[0073] Step 401: Identify multiple calibration points set on the calibration board from the calibration image, and obtain the third coordinates of each calibration point in the pixel coordinate system.

[0074] There can be multiple calibration points on the surface of the calibration plate facing the camera lens. The calibration points are identified from the calibration image, and the third coordinates of each calibration point in the pixel coordinate system are obtained.

[0075] The calibration points can be circular, and the spacing between each calibration point is fixed. The calibration points are distributed in a rectangular array on the calibration plate.

[0076] Step 402: Based on the calibration information, convert the third coordinate of the calibration point into the fourth coordinate in the spatial coordinate system.

[0077] Based on the transformation relationship between the pixel coordinate system and the spatial coordinate system, the third coordinate in the pixel coordinate system will be converted into the fourth coordinate in the spatial coordinate system.

[0078] Step 403: Verify the accuracy of the calibration information based on the fourth coordinates of multiple calibration points.

[0079] For example, it can be determined whether the coordinates of the calibration points in each row of a matrix array lie on the same straight line; if so, the calibration information is verified to be accurate. Alternatively, a virtual model can be built to simulate the positions of the calibration points, and the calibration points can be connected in the model to determine whether the lines connecting the calibration points in each row lie on the same straight line; if so, the calibration information is verified to be accurate.

[0080] In this embodiment of the application, by setting calibration points on the calibration plate, the calibration information can be verified by the image of the calibration points, thereby improving the accuracy of the calibration information.

[0081] In one possible implementation, step 403 may also include the following sub-steps:

[0082] Sub-step 4031: Obtain the attribute information of the calibration board.

[0083] To verify the accuracy of the calibration information, the specific operation can be to first obtain the attribute information of the calibration board. The attribute information is used to indicate the number of rows and columns of the calibration points on the calibration board, as well as the distance between the calibration points.

[0084] Sub-step 4032: Based on the fourth coordinates of multiple calibration points, determine the first distance between the Nth calibration point in the i-th row of calibration points on the calibration board and the first calibration point in the (i+1)-th row of calibration points.

[0085] For example, based on the fourth coordinate in the spatial coordinate system obtained by transforming the third coordinate in the pixel coordinate system according to the calibration relationship of multiple calibration points, N can be 9. In this case, the first distance between the 9th calibration point in the first row and the 1st calibration point in the second row on the calibration board is determined. The calculation method for the first distance between the remaining calibration points is the same as in the example. The calibration board includes M rows × N columns of calibration points, where M and N are both positive integers greater than or equal to 2, and i is a positive integer less than M.

[0086] Sub-step 4033: Based on the attribute information, determine the second distance between the Nth calibration point in the i-th row of calibration points on the calibration board and the first calibration point in the (i+1)-th row of calibration points.

[0087] Based on the obtained attribute information of the calibration board, determine the second distance between the calibration points on the calibration board, corresponding to the first distance. The distances between the calibration points can all be the same. In this case, the second distance is equal to the length of the hypotenuse of a right angle, calculated using the distance between N calibration points and the distance between 1 calibration point as the sides. For example, if N equals 5 and the distance between the calibration points is 1, then the second distance is equal to... Furthermore, the distances between calibration points can also be different.

[0088] Sub-step 4034: Verify the accuracy of the calibration information based on the difference between the first distance and the second distance.

[0089] After obtaining the first distance and the second distance, since the first distance is essentially obtained by transforming the third coordinate in the pixel coordinate system according to the calibration information, the calibration information can be considered accurate when the difference between the first distance and the second distance is within the preset threshold.

[0090] In this embodiment, the first distance between the Nth calibration point in the i-th row and the first calibration point in the (i+1)-th row of calibration points on the calibration board is calculated using the fourth coordinate of the calibration points after calibration information conversion. This first distance is then compared with the second distance between the Nth calibration point in the i-th row and the first calibration point in the (i+1)-th row of calibration points in the actual calibration points. This comparison verifies the calibration information, and the verification result is relatively accurate, ensuring the accuracy of the calibration information.

[0091] In one possible implementation, the accuracy of the calibration information can also be verified in the following way:

[0092] A virtual calibration tool is constructed, and its fifth coordinate is obtained. Then, based on the calibration information, the fifth coordinate of the virtual calibration tool is converted into the sixth coordinate in the spatial coordinate system. Finally, the accuracy of the calibration information is verified based on the sixth coordinate of the virtual calibration tool.

[0093] When the virtual calibration tool is at the position indicated by the fifth coordinate, it is perpendicularly pointed to the center of the verification point on the calibration plate that has not been translated or rotated. Based on the calibration information, the fifth coordinate is converted to the sixth coordinate in the spatial coordinate system to verify the accuracy of the calibration information. For example, the accuracy of the calibration information can be verified by determining whether the virtual calibration tool at the sixth coordinate is perpendicularly pointed to the center of the verification point on the calibration plate that has not been translated or rotated. The verification point can point to any calibration point on the calibration plate.

[0094] In this embodiment of the application, a virtual calibration tool can be used to verify the accuracy of the angle after the calibration information is converted, thereby ensuring the accuracy of the calibration information.

[0095] Figure 6 This is a flowchart of the steps of a control information generation method according to an embodiment of this application, as follows: Figure 6 As shown, the control information generation method 500 includes the following steps:

[0096] Step 501: Generate candidate control information based on the first coordinate and the second coordinate.

[0097] In the process of generating control information, candidate control information can be generated first based on the first coordinate and the second coordinate. The candidate control information can be generated based on the difference between the first coordinate and the second coordinate, or the displacement vector from the first coordinate to the second coordinate can be determined based on the first coordinate and the second coordinate, and then the candidate control information can be generated.

[0098] Step 502: Construct a virtual safety frame for the workpiece based on the second coordinate.

[0099] After generating the candidate control information, to ensure the safety of the gripping process, a virtual safety frame can be generated based on the second coordinate indicating the workpiece position. The virtual safety frame encloses the workpiece and can be a shape such as a cuboid. Under normal circumstances, when the robotic arm grips the workpiece, the gripping end of the robotic arm will not interfere with the object within the virtual safety frame. Interference may occur if the gripping end of the robotic arm collides with other objects while gripping an object within the virtual safety frame, or if an anomaly occurs while gripping an object within the virtual safety frame, preventing the robotic arm from gripping the object within the virtual safety frame.

[0100] Step 503: Based on the virtual safety frame, determine whether a collision will occur when the robotic arm grasps the workpiece according to the selected control information.

[0101] For example, based on the constructed virtual safety frame, a virtual route can be constructed for the robotic arm to grasp the workpiece according to the selected control information, and it can be determined whether the robotic arm will collide with other objects besides the workpiece being grasped along the virtual route.

[0102] Step 504: If the robotic arm will not collide with the workpiece when it grasps it according to the candidate control information, then the candidate control information is determined as the control information.

[0103] When it is determined that no collision will occur when the robotic arm grasps the workpiece according to the selected control information, it proves that the process of the robotic arm moving towards the workpiece is safe. At this time, the selected control information can be determined as the control information.

[0104] In this embodiment of the application, by constructing a virtual security frame, the security of the generated control information can be guaranteed.

[0105] Workpiece gripping device

[0106] Figure 7 This is a schematic diagram of a workpiece gripping device according to an embodiment of this application, as shown below. Figure 7 As shown, the workpiece gripping device 600 includes: a first acquisition module 601, a second acquisition module 602, a determination module 603, and a control module 604.

[0107] The first acquisition module 601 is used to acquire the first coordinates of the gripping end of the robotic arm in the spatial coordinate system.

[0108] To automate the control of the robotic arm, the first acquisition module 601 acquires the first coordinates of the gripping end of the robotic arm in the spatial coordinate system. The workpiece coordinate system can be a Cartesian coordinate system or a coordinate system defined by the operator to represent the position and orientation of objects in the production area. Taking a Cartesian coordinate system as an example, the first coordinates can be (4, 1, 3), representing the distances of the workpiece relative to the origin on the X, Y, and Z axes, respectively.

[0109] The second acquisition module 602 is used to acquire workpiece images captured by the camera, wherein the relative positions of the camera and the base of the robotic arm in the spatial coordinate system are fixed, and the workpiece images include images of the workpiece located within the gripping area.

[0110] A camera for capturing images of the workpiece is fixedly installed above the gripping area. When the camera is set in a fixed position, the workpiece within the gripping area is within the camera's shooting area; therefore, the workpiece image includes the image of the workpiece located within the gripping area. Simultaneously, the relative positions of the camera and the robotic arm's base are fixed in the spatial coordinate system.

[0111] After the camera captures an image of the workpiece in the grasping area, the second acquisition module 602 acquires the workpiece image.

[0112] The determination module 603 is used to determine the second coordinate of the workpiece in the spatial coordinate system based on the workpiece image and calibration information, wherein the calibration information is used to indicate the mapping relationship between the pixel coordinate system and the spatial coordinate system of the image acquired by the camera.

[0113] After the second acquisition module 602 acquires the workpiece image, the determination module 603 can first determine the coordinates of the workpiece in the pixel coordinate system of the image captured by the camera based on the workpiece image, and then transform the coordinates into coordinates in the spatial coordinate system based on the calibration information, so as to determine the second coordinates of the workpiece in the spatial coordinate system in the workpiece image.

[0114] The control module 604 is used to generate control information based on the first coordinate and the second coordinate, and send the control information to the robotic arm so that the robotic arm can grasp the workpiece according to the control information.

[0115] After the first acquisition module 601 obtains the first coordinate and the second acquisition module 602 obtains the second coordinate, the control module 604 can determine the displacement based on the difference between the first and second coordinates, and then generate control information. Alternatively, it can determine the displacement vector from the first coordinate to the second coordinate based on the first and second coordinates, and then generate control information. After obtaining the control information, the control information is sent to the robotic arm, so that the robotic arm can move in the spatial coordinate system according to the control information, and grasp the workpiece after the movement is completed.

[0116] In this embodiment, the second acquisition module 602 acquires the workpiece image captured by the camera, which can determine the position of the workpiece in the spatial coordinate system. Then, combined with the position of the gripping end of the robotic arm acquired by the first acquisition module 601, the control module 604 generates control information to control the robotic arm. Workpieces at any angle and position can be automatically gripped without limiting the pose of the workpiece when gripping at the gripping end of the robotic arm, thus improving production efficiency.

[0117] Workpiece gripping system

[0118] Figure 8 This is a schematic diagram of a workpiece gripping system according to an embodiment of this application, as shown below. Figure 8 As shown, the workpiece gripping system 700 includes: a robotic arm 701, a camera 702, and a workpiece gripping device 600;

[0119] The workpiece gripping device 600 is used to perform the workpiece gripping method as described in the above embodiments.

[0120] Camera 702 is used to acquire images of workpieces located within the gripping area, obtain workpiece images, and send the workpiece images to workpiece gripping device 600.

[0121] A camera 702 for acquiring images of the workpiece is fixedly mounted above the gripping area. When the camera 702 is in a fixed position, the workpiece within the gripping area is within the camera 702's field of view; therefore, the workpiece image includes the image of the workpiece located within the gripping area. Simultaneously, the relative positions of the camera 702 and the base of the robotic arm 701 are fixed in the spatial coordinate system. When the workpiece is transported or placed into the gripping area, the camera 702 acquires an image of the workpiece located within the gripping area and sends the acquired workpiece image to the workpiece gripping device 600.

[0122] The robotic arm 701 is used to receive control information sent by the workpiece gripping device 600 and grip the workpiece according to the control information.

[0123] After the workpiece gripping device 600 generates control information, it sends it to the robotic arm 701. The robotic arm 701 moves in the spatial coordinate system according to the control information and grips the workpiece after the movement is completed.

[0124] In this embodiment, the workpiece image acquired by camera 702 can determine the position of the workpiece in the spatial coordinate system. Then, combined with the position of the gripping end of robotic arm 701, control information for controlling robotic arm 701 is generated. Workpieces at any angle and position can be automatically gripped without limiting the workpiece's pose when gripping at the gripping end. This eliminates the need for manual placement of parts, improves the degree of industrial automation, and thus increases production efficiency.

[0125] In one possible implementation, the workpiece gripping system 700 also includes a laser transceiver.

[0126] During the production of workpieces, due to mass production, the workpieces still need to undergo a cutting and separation process after production. However, this process may result in incomplete cutting or no cutting at all, causing the workpieces to stick together. To prevent incompletely cut or uncut workpieces from being picked up and fed into the finished product packaging area, a laser transceiver can be installed in the workpiece gripping system 700.

[0127] The laser transceiver is located at the gripping end of the robotic arm 701. After the robotic arm 701 grips the workpiece, the gripped workpiece will not block the laser emitted by the laser transceiver.

[0128] The laser transceiver is located at the gripping end of the robotic arm 701, and the laser transceiver can emit laser light. When the laser transceiver emits laser light, the gripped workpiece will not block the laser light emitted by the laser transceiver.

[0129] The laser transceiver is used to emit a laser perpendicular to the gripping surface of the workpiece after the gripping end of the robotic arm 701 grips the workpiece, and to send a first adhesion signal to the workpiece gripping device 600 after the emitted laser is blocked by an object located on the same plane as the workpiece. The first adhesion signal is used to indicate that the workpiece has adhered.

[0130] After the gripping end of the robotic arm 701 grips the workpiece, the laser transceiver emits a laser perpendicular to the gripping surface of the workpiece. For example, there can be two laser transceivers, which are respectively set at both ends of the gripping end of the robotic arm 701. After the laser emitted by the laser transceiver is blocked by an object located on the same plane as the workpiece, it can be determined that the workpiece is stuck and a first adhesion signal is sent to the workpiece gripping device 600 to indicate that the workpiece is stuck.

[0131] The workpiece gripping device 600 is used to issue a first alarm message after receiving the first adhesion signal.

[0132] The first alarm signal can be a buzzer or a light turning on.

[0133] In this embodiment of the application, by setting up a laser transceiver, it is possible to detect whether the workpiece being gripped is stuck together, thereby improving the safety of production.

[0134] In one possible implementation, the workpiece gripping system 700 further includes a conveyor belt. The conveyor belt comprises P sub-conveyor belts arranged along the feeding direction, where P is a positive integer greater than or equal to 2.

[0135] The conveyor belt is used to transport the workpiece to the gripping area along the feeding direction. When the workpiece is at the end of the Pth sub-conveyor belt in the feeding direction, the Pth sub-conveyor belt is controlled to stop running, and the 1st to P-1th sub-conveyor belts are controlled to decelerate or stop running.

[0136] For example, a conveyor belt may include 5 sub-conveyor belts. When a workpiece is transported to the end of the fifth sub-conveyor belt in the feeding direction, it can be determined that there is a stockpiling of workpieces and the robotic arm fails to grab the workpieces in the grabbing area in time. At this time, in order to prevent other sub-conveyor belts from continuing to move workpieces to the grabbing area, resulting in the workpieces being squeezed out or piled up, the first to fourth sub-conveyor belts are controlled to slow down or stop running.

[0137] In this embodiment of the application, by setting up segmented sub-conveyor belts, it is possible to prevent workpieces from accumulating during transport, thereby improving the safety of workpiece gripping.

[0138] In one possible implementation, the workpiece gripping system 700 also includes gratings disposed on both sides of the conveyor belt.

[0139] A grating is used to emit laser light that is parallel to the conveyor belt and perpendicular to the feeding direction. After the workpiece is gripped by the gripping end of the robotic arm 701, the laser light emitted by the grating is blocked. A second adhesion signal is sent to the workpiece gripping device 600. The second adhesion signal is used to indicate that the gripped workpiece has adhered.

[0140] During the production of workpieces, due to mass production, the workpieces still need to undergo a cutting and separation process after production. However, this process may result in incomplete cutting or no cutting at all, leading to adhesion between workpieces. During the workpiece gripping process, workpieces may be gripped at a 90° angle due to low adhesion. To prevent incompletely cut or uncut workpieces from being gripped and flowing into the finished product packaging area, optical gratings can be installed on both sides of the conveyor belt in the workpiece gripping system 700. After the gripping end of the robotic arm 701 grips the workpiece, the optical grating emits a laser parallel to the gripping surface of the workpiece. If the laser emitted by the optical grating is blocked after the workpiece gripped by the robotic arm 701 passes through it, a second adhesion signal is sent to the workpiece gripping device 600.

[0141] The workpiece gripping device 600 is used to issue a second alarm message after receiving the second adhesion signal.

[0142] The second alarm message can be a buzzer, a light, or other similar notification. The first and second alarm messages can be the same or different. When the first and second alarm messages are different, information indicating that a workpiece is stuck at a 90° angle can be added to the second alarm message.

[0143] In this embodiment of the application, by setting a grating, it is possible to detect whether the workpiece being gripped is stuck together, thereby improving the safety of production.

[0144] In one possible implementation, multiple robotic arms 701 can be configured, each corresponding to a different gripping area. After receiving control information from the workpiece gripping device 600, each robotic arm 701 can grip the workpiece within its corresponding gripping area according to the control information. The control information is sent to all robotic arms 701. When the workpiece to be gripped, as indicated by the control information, is not within the gripping area of ​​a robotic arm 701, that robotic arm 701 will not grip the workpiece.

[0145] In this embodiment, by setting up multiple robotic arms 701, the efficiency of workpiece gripping can be improved. By controlling each robotic arm 701 to grip the workpiece within its corresponding gripping area, safety issues caused by multiple robotic arms 701 gripping the same workpiece simultaneously can be avoided.

[0146] electronic devices

[0147] Figure 9 This is a schematic diagram of an electronic device according to one embodiment of this application. The specific embodiments of this application do not limit the specific implementation of the electronic device. Figure 9 As shown, the electronic device 800 may include: a processor 801, a communications interface 802, a memory 803, and a communications bus 804. Wherein:

[0148] The processor 801, communication interface 802, and memory 803 communicate with each other through communication bus 804.

[0149] Communication interface 802 is used to communicate with other electronic devices or servers.

[0150] The processor 801 is used to execute program 805, which can specifically execute the relevant steps in the aforementioned workpiece gripping method embodiment.

[0151] Specifically, program 805 may include program code that includes computer operation instructions.

[0152] The processor 801 may be a CPU, an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application. The smart device includes one or more processors, which may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0153] Memory 803 is used to store program 805. Memory 803 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0154] Specifically, program 805 can be used to cause processor 801 to execute the workpiece gripping method in the foregoing embodiments.

[0155] The specific implementation of each step in program 805 can be found in the corresponding steps and units described in the aforementioned workpiece gripping method embodiments, and will not be repeated here. Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the devices and modules described above can be referred to the corresponding process descriptions in the aforementioned method embodiments, and will not be repeated here.

[0156] The electronic device in this application embodiment can determine the position of the workpiece in the spatial coordinate system by capturing the workpiece image through the camera. Then, combined with the position of the gripping end of the robotic arm, control information for controlling the robotic arm is generated. Workpieces at any angle and position can be automatically gripped without limiting the posture of the workpiece when gripping at the gripping end. This eliminates the step of manual placement of the workpiece, improves the degree of industrial automation, and thus improves production efficiency.

[0157] Computer storage media

[0158] This application also provides a computer-readable storage medium storing instructions for causing a machine to perform the workpiece gripping method as described herein. Specifically, a system or apparatus equipped with a storage medium storing software program code that implements the functions of any of the embodiments described above, and enabling the computer (or CPU or MPU) of the system or apparatus to read and execute the program code stored in the storage medium.

[0159] In this case, the program code read from the storage medium can itself implement the function of any of the above embodiments, and therefore the program code and the storage medium storing the program code constitute part of this application.

[0160] Storage media embodiments for providing program code include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), magnetic tapes, non-volatile memory cards, and ROMs. Alternatively, program code can be downloaded from a server computer via a communication network.

[0161] Computer program products

[0162] This application also provides a computer program product, including computer instructions that instruct a computing device to perform any corresponding operation in the above-described plurality of method embodiments.

[0163] It should be noted that, depending on the implementation needs, the various components / steps described in the embodiments of this application can be broken down into more components / steps, or two or more components / steps or parts of the operation of components / steps can be combined into new components / steps to achieve the purpose of the embodiments of this application.

[0164] The methods described in the embodiments of this application can be implemented in hardware, firmware, or as software or computer code that can be stored in a recording medium (such as a CD-ROM, RAM, floppy disk, hard disk, or magneto-optical disk), or as computer code downloaded over a network that is originally stored in a remote recording medium or a non-transitory machine-readable medium and will be stored in a local recording medium. Thus, the methods described herein can be processed by software stored on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware (such as an ASIC or FPGA). It is understood that the computer, processor, microprocessor controller, or programmable hardware includes storage components (e.g., RAM, ROM, flash memory, etc.) capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods described herein. Furthermore, when a general-purpose computer accesses code used to implement the methods shown herein, the execution of the code transforms the general-purpose computer into a dedicated computer for executing the methods shown herein.

[0165] It should be noted that not all steps and modules in the above processes and system structure diagrams are mandatory; some steps or modules can be omitted as needed. The execution order of each step is not fixed and can be adjusted as required. The system structure described in the above embodiments can be a physical structure or a logical structure. That is, some modules may be implemented by the same physical entity, or some modules may be implemented by multiple physical entities, or they may be jointly implemented by certain components in multiple independent devices.

[0166] In this patent application, nouns and pronouns relating to people are not limited to specific genders.

[0167] In the above embodiments, the hardware modules can be implemented mechanically or electrically. For example, a hardware module may include permanent, dedicated circuitry or logic (such as a dedicated processor, FPGA, or ASIC) to perform the corresponding operations. The hardware module may also include programmable logic or circuitry (such as a general-purpose processor or other programmable processor), which can be temporarily configured by software to perform the corresponding operations. The specific implementation method (mechanical, dedicated, permanent circuitry, or temporarily configured circuitry) can be determined based on cost and time considerations.

[0168] The present invention has been shown and described in detail above with reference to the accompanying drawings and preferred embodiments. However, the present invention is not limited to these disclosed embodiments. Based on the above multiple embodiments, those skilled in the art will know that more embodiments of the present invention can be obtained by combining the code review methods in the different embodiments above. These embodiments are also within the protection scope of the present invention.

Claims

1. A workpiece gripping method (100), comprising: Obtain the first coordinate of the gripping end of the robotic arm in a spatial coordinate system, wherein the spatial coordinate system is a three-dimensional Cartesian coordinate system; The workpiece image is acquired by the camera, wherein the relative position of the camera and the base of the robotic arm is fixed in the spatial coordinate system, and the workpiece image includes the image of the workpiece located within the gripping area; Based on the workpiece image and calibration information, the second coordinates of the workpiece in the spatial coordinate system are determined, wherein the calibration information is used to indicate the mapping relationship between the pixel coordinate system of the image captured by the camera and the spatial coordinate system; Control information is generated based on the first coordinate and the second coordinate, and the control information is sent to the robotic arm so that the robotic arm can grasp the workpiece according to the control information; The calibration information is determined using the following method: 201: Based on preset X-axis translation, Y-axis translation, and Z-axis translation, as well as rotation around the X-axis, Y-axis, and Z-axis, the robotic arm is controlled to perform multiple translations and / or rotations in the spatial coordinate system. Images of the calibration plate are captured by the camera after each translation and / or rotation, resulting in multiple calibration images captured by the camera, including... 301: Based on the preset X-axis translation and Y-axis translation, control the robotic arm to perform 5 translations in a first plane perpendicular to the Z-axis, and use the camera to capture images of the calibration plate after each translation of the robotic arm, thereby obtaining 5 calibration images captured by the camera; 302: Based on the preset X-axis translation and Y-axis translation, control the robotic arm to perform 4 translations in a second plane perpendicular to the Z-axis, and use the camera to capture images of the calibration plate after each translation of the robotic arm, thereby obtaining 4 calibration images captured by the camera, wherein the distance between the first plane and the second plane is equal to the preset Z-axis translation. 303: Based on the preset X-axis translation, Y-axis translation, rotation around the X-axis, rotation around the Y-axis, and rotation around the Z-axis, control the robotic arm to translate and then rotate in the first plane, and use the camera to capture images of the calibration plate after each rotation of the robotic arm, thereby obtaining 4 calibration images captured by the camera; The calibration plate is connected to the gripping end of the robotic arm. The multiple calibration images include images of the calibration plate captured by the camera when the robotic arm is offset in the X-axis, Y-axis and Z-axis directions, and images of the calibration plate captured by the camera when the robotic arm rotates around the X-axis, Y-axis and Z-axis directions. 202: Determine the calibration information based on the position of the calibration plate image in the calibration image, and the translation and rotation of the robotic arm when acquiring the calibration plate image.

2. The method according to claim 1, wherein, The method further includes: Identify multiple calibration points set on the calibration board from the calibration image, and obtain the third coordinates of the multiple calibration points in the pixel coordinate system respectively; Based on the calibration information, the third coordinate of the calibration point is converted into the fourth coordinate in the spatial coordinate system; The accuracy of the calibration information is verified based on the fourth coordinates of the plurality of calibration points.

3. The method according to claim 2, wherein, The step of verifying the accuracy of the calibration information based on the fourth coordinates of the plurality of calibration points includes: Obtain the attribute information of the calibration board, wherein the attribute information is used to indicate the number of rows and columns of calibration points on the calibration board, and the distance between calibration points; Based on the fourth coordinates of the plurality of calibration points, determine the first distance between the Nth calibration point in the i-th row of calibration points on the calibration board and the 1st calibration point in the (i+1)-th row of calibration points, wherein the calibration board includes M rows × N columns of calibration points, where M and N are both positive integers greater than or equal to 2, and i is a positive integer less than M; Based on the attribute information, determine the second distance between the Nth calibration point in the i-th row of calibration points on the calibration board and the first calibration point in the (i+1)-th row of calibration points; The accuracy of the calibration information is verified based on the difference between the first distance and the second distance.

4. The method according to claim 1, wherein, The rotation around the X-axis, the rotation around the Y-axis, and the rotation around the Z-axis are all between 25° and 45°.

5. The method according to claim 1, wherein, The method further includes: Construct a virtual calibration tool and obtain the fifth coordinate of the virtual calibration tool, wherein the virtual calibration tool is perpendicularly pointing to the center of the verification point in the calibration plate without translation or rotation when it is at the position indicated by the fifth coordinate; Based on the calibration information, the fifth coordinate of the virtual calibration tool is converted into the sixth coordinate in the spatial coordinate system; The accuracy of the calibration information is verified based on the sixth coordinate of the virtual calibration tool.

6. The method according to claim 1, wherein, The step of generating control information based on the first coordinate and the second coordinate includes: Calculate the X-axis displacement offset, Y-axis displacement offset, Z-axis displacement offset, and rotation angles around the X-axis, Y-axis, and Z-axis of the first coordinate relative to the second coordinate. The control information is generated based on the X-axis displacement offset, the Y-axis displacement offset, the Z-axis displacement offset, and the rotation angles around the X-axis, the Y-axis, and the Z-axis. The robotic arm adjusts itself according to the displacement offset and rotation angle in the control information to grasp the workpiece.

7. The method according to any one of claims 1-5, wherein, The step of generating control information based on the first coordinate and the second coordinate includes: Generate candidate control information based on the first coordinate and the second coordinate; Based on the second coordinates, a virtual safety frame for the workpiece is constructed, wherein when the robotic arm grasps the workpiece, the grasping end of the robotic arm will not interfere with the object within the virtual safety frame; Based on the virtual safety frame, determine whether a collision will occur when the robotic arm grasps the workpiece according to the candidate control information; If the robotic arm does not collide with the workpiece when grasping it according to the candidate control information, then the candidate control information is determined as the control information.

8. A workpiece gripping device (600), comprising: The first acquisition module (601) is used to acquire the first coordinate of the gripping end of the robotic arm in a spatial coordinate system, wherein the spatial coordinate system is a three-dimensional Cartesian coordinate system; The second acquisition module (602) is used to acquire workpiece images captured by the camera, wherein the relative positions of the camera and the base of the robotic arm are fixed in the spatial coordinate system, and the workpiece images include images of workpieces located within the grasping area. The determining module (603) is used to determine the second coordinates of the workpiece in the spatial coordinate system based on the workpiece image and the calibration information, wherein the calibration information is used to indicate the mapping relationship between the pixel coordinate system of the image captured by the camera and the spatial coordinate system; The control module (604) is used to generate control information based on the first coordinate and the second coordinate, and send the control information to the robotic arm so that the robotic arm can grasp the workpiece according to the control information; The calibration information is determined using the following method: 201: Based on preset X-axis translation, Y-axis translation, and Z-axis translation, as well as rotation around the X-axis, Y-axis, and Z-axis, the robotic arm is controlled to perform multiple translations and / or rotations in the spatial coordinate system. Images of the calibration plate are captured by the camera after each translation and / or rotation, resulting in multiple calibration images captured by the camera, including... 301: Based on the preset X-axis translation and Y-axis translation, control the robotic arm to perform 5 translations in a first plane perpendicular to the Z-axis, and use the camera to capture images of the calibration plate after each translation of the robotic arm, thereby obtaining 5 calibration images captured by the camera; 302: Based on the preset X-axis translation and Y-axis translation, control the robotic arm to perform 4 translations in a second plane perpendicular to the Z-axis, and use the camera to capture images of the calibration plate after each translation of the robotic arm, thereby obtaining 4 calibration images captured by the camera, wherein the distance between the first plane and the second plane is equal to the preset Z-axis translation. 303: Based on the preset X-axis translation, Y-axis translation, rotation around the X-axis, rotation around the Y-axis, and rotation around the Z-axis, control the robotic arm to translate and then rotate in the first plane, and use the camera to capture images of the calibration plate after each rotation of the robotic arm, thereby obtaining 4 calibration images captured by the camera; The calibration plate is connected to the gripping end of the robotic arm. The multiple calibration images include images of the calibration plate captured by the camera when the robotic arm is offset in the X-axis, Y-axis and Z-axis directions, and images of the calibration plate captured by the camera when the robotic arm rotates around the X-axis, Y-axis and Z-axis directions. 202: Determine the calibration information based on the position of the calibration plate image in the calibration image, and the translation and rotation of the robotic arm when acquiring the calibration plate image.

9. A workpiece gripping system (700), comprising: Robotic arm (701), camera (702) and workpiece gripping device (600); The workpiece gripping device (600) is used to perform the method as described in any one of claims 1-7 above; The camera (702) is used to acquire images of the workpiece located within the gripping area, obtain workpiece images, and send the workpiece images to the workpiece gripping device (600). The robotic arm (701) is used to receive control information sent by the workpiece gripping device (600) and grip the workpiece according to the control information.

10. The system according to claim 9, wherein, The system also includes: a laser transceiver; The laser transceiver is disposed at the gripping end of the robotic arm (701). After the gripping end of the robotic arm (701) grips the workpiece, the gripped workpiece will not block the laser emitted by the laser transceiver. The laser transceiver is used to emit a laser perpendicular to the gripping surface of the gripped workpiece after the gripping end of the robotic arm (701) grips the workpiece, and to send a first adhesion signal to the workpiece gripping device (600) after the emitted laser is blocked by an object located on the same plane as the gripped workpiece. The first adhesion signal is used to indicate that the gripped workpiece has become stuck. The workpiece gripping device (600) is used to issue a first alarm message after receiving the first adhesion signal.

11. The system according to claim 9, wherein, The system also includes: a conveyor belt; The conveyor belt includes P sub-conveyor belts arranged along the feeding direction, where P is a positive integer greater than or equal to 2; The conveyor belt is used to transport the workpiece to the gripping area along the feeding direction. When the workpiece is transported to the end of the Pth sub-conveyor belt in the feeding direction, the Pth sub-conveyor belt is controlled to stop running, and the 1st to P-1th sub-conveyor belts are controlled to decelerate or stop running.

12. The system according to claim 11, wherein, The system also includes optical gratings disposed on both sides of the conveyor belt; The grating is used to emit laser light that is parallel to the conveyor belt and perpendicular to the feeding direction. After the workpiece is gripped by the gripping end of the robotic arm (701) and passes through the laser light emitted by the grating, if the laser light emitted by the grating is blocked, a second adhesion signal is sent to the workpiece gripping device (600). The second adhesion signal is used to indicate that the gripped workpiece has become stuck. The workpiece gripping device (600) is used to issue a second alarm message after receiving the second adhesion signal.

13. An electronic device (800), comprising: The processor (801), communication interface (802), memory (803), and communication bus (804) communicate with each other through the communication bus (804). The memory (803) is used to store at least one executable instruction that causes the processor (801) to perform the operation corresponding to the workpiece gripping method as described in any one of claims 1-7.

14. A computer storage medium having a computer program stored thereon, which, when executed by a processor, implements the workpiece gripping method as described in any one of claims 1-7.

15. A computer program product comprising computer instructions that instruct a computing device to perform an operation corresponding to the workpiece gripping method as described in any one of claims 1-7.

Images