Robot grasping method and device based on visual positioning and storage medium

Abstract

The application provides a robot grabbing method and device based on visual positioning and a storage medium. The robot grabbing method based on visual positioning comprises the following steps: obtaining a current spatial coordinate of a to-be-grabbed cargo; guiding the robot to move to an optimal shooting position of a 2D camera according to the current spatial coordinate of the cargo, and obtaining a current planar coordinate of the to-be-grabbed cargo; calculating a coordinate offset according to the current planar coordinate of the cargo and a reference planar coordinate of the cargo; calculating a target spatial coordinate of the cargo according to the current spatial coordinate of the cargo, the coordinate offset and a reference spatial coordinate of the cargo; calculating a grabbing spatial coordinate of the robot according to the target spatial coordinate of the cargo, the reference spatial coordinate of the cargo, a current shooting coordinate of a 3D camera of the robot and a reference shooting coordinate of the 3D camera; and guiding the robot to move to an optimal grabbing position according to the grabbing spatial coordinate, and grabbing the to-be-grabbed cargo. The application can improve the positioning accuracy and reduce the failure rate of grabbing.

Description

Technical Field

[0001] This application relates to the field of positioning technology, specifically to a robot grasping method, device, and storage medium based on vision positioning. Background Technology

[0002] With the rapid development of industrial automation and intelligent manufacturing, robot vision grasping technology has been widely used in logistics sorting, precision assembly, and warehouse management. Visual positioning, as one of the core technologies of robot grasping systems, directly determines the success rate and reliability of grasping operations.

[0003] Currently, vision-based robotic cargo grasping technology mainly employs 3D visual matching methods to acquire the spatial location data of the target cargo. Specifically, this technology uses a 3D camera (such as a structured light camera, binocular vision camera, or ToF camera) to collect point cloud data or depth images of the cargo. Then, the real-time acquired 3D data is registered and matched with a pre-stored 3D model template of the cargo to calculate the six-degree-of-freedom pose information of the cargo in the robot coordinate system.

[0004] However, existing technical solutions rely solely on 3D matching to obtain position data, which can easily lead to large positioning deviations and low accuracy in practical applications. This makes it difficult to meet the requirements of high-precision grasping operations, especially in precision assembly or confined space handling scenarios where strict positional accuracy is required, resulting in a significantly higher grasping failure rate. Summary of the Invention

[0005] In view of this, this application proposes a robot grasping method, device and storage medium based on vision positioning to solve the problem of significantly increased robot grasping failure rate caused by low positioning accuracy in related technologies.

[0006] The first aspect of this application proposes a vision-based robot grasping method, the method comprising:

[0007] The robot's 3D camera obtains the current spatial coordinates of the goods to be grabbed.

[0008] The robot is guided to the optimal shooting position of the 2D camera based on the current spatial coordinates of the cargo, and the current planar coordinates of the cargo to be grasped are obtained through the 2D camera.

[0009] The coordinate offset of the cargo to be grasped on the preset plane is calculated based on the current planar coordinates of the cargo and the reference planar coordinates of the cargo; the reference planar coordinates of the cargo refer to the planar coordinates of the cargo to be grasped obtained by the 2D camera when the robot is first debugging the grasping position;

[0010] The target spatial coordinates of the cargo are calculated based on the current spatial coordinates of the cargo, the coordinate offset, and the reference spatial coordinates of the cargo; the reference spatial coordinates of the cargo refer to the spatial coordinates of the cargo to be grasped obtained by the 3D camera when the robot is first debugging the grasping position.

[0011] The robot's grasping spatial coordinates are calculated based on the target spatial coordinates of the cargo, the reference spatial coordinates of the cargo, the current capture coordinates of the robot's 3D camera, and the reference capture coordinates of the 3D camera. The reference capture coordinates of the 3D camera refer to the spatial position of the robot when the reference plane coordinates of the cargo are obtained through the 3D camera.

[0012] Guided by the grasping space coordinates, the robot moves to the optimal grasping position and grasps the goods to be grasped.

[0013] This application embodiment acquires the current planar coordinates of the goods to be grasped using a 2D camera, calculates the coordinate offset of the goods to be grasped on a preset plane based on the current planar coordinates and the reference planar coordinates of the goods, and calculates the target spatial coordinates of the goods based on the current spatial coordinates, the coordinate offset, and the reference spatial coordinates of the goods. It fully considers that "compared to 3D cameras, 2D cameras have higher accuracy in the XY plane", thus introducing 2D cameras into the 3D positioning system. In this way, the positioning accuracy of the XY plane in the 3D positioning system can be greatly improved, thereby ensuring the accuracy of goods grasping.

[0014] In this embodiment of the application, obtaining the current planar coordinates of the goods to be grabbed using the 2D camera includes:

[0015] Calculate the coordinates of the three positioning holes in the cargo to be grabbed;

[0016] The angle of each corner in the target triangle is calculated based on the coordinates of the three positioning holes; the target triangle is formed based on the three positioning holes.

[0017] Determine whether the angle of each corner is within the corresponding angle range;

[0018] If the angle of each corner is within the corresponding angle range, the current plane coordinates of the goods to be grabbed are determined according to the coordinates of the three positioning holes.

[0019] In this embodiment of the application, the calculation of the target spatial coordinates of the cargo based on the current spatial coordinates of the cargo, the coordinate offset, and the reference spatial coordinates of the cargo includes:

[0020] The actual coordinates of the cargo to be grabbed on the preset plane are calculated based on the cargo reference space coordinates and the coordinate offset.

[0021] The target spatial coordinates of the cargo are obtained by replacing the current spatial coordinates of the cargo with the actual coordinates.

[0022] In this embodiment of the application, the robot's grasping spatial coordinates are calculated based on the target spatial coordinates of the cargo, the reference spatial coordinates of the cargo, the current image coordinates of the robot's 3D camera, and the reference image coordinates of the 3D camera, including:

[0023] Calculate the 3D camera coordinate offset based on the robot's current 3D camera coordinates and the 3D camera reference coordinates.

[0024] Calculate the cargo spatial coordinate offset based on the cargo target spatial coordinates and the cargo reference spatial coordinates;

[0025] The robot's grasping space coordinates are calculated based on the 3D camera's reference capture coordinates, the 3D camera's capture coordinate offset, and the cargo's space coordinate offset.

[0026] In this embodiment, the three positioning holes include a first positioning hole, a second positioning hole, and a third positioning hole; determining the current planar coordinates of the goods to be grasped based on the coordinates of the three positioning holes includes:

[0027] A first line segment is obtained by connecting the first positioning hole and the second positioning hole;

[0028] Draw a perpendicular line to the first line segment through the third positioning hole, and obtain the coordinates of the perpendicular point of the perpendicular line;

[0029] Calculate the angle between the perpendicular line and the preset plane;

[0030] The current planar coordinates of the cargo to be grabbed are obtained based on the coordinates of the perpendicular point and the included angle.

[0031] In this embodiment of the application, before calculating the coordinates of the three positioning holes in the cargo to be grasped, the method further includes:

[0032] The three positioning holes in the cargo to be grabbed are located by geometric matching;

[0033] The target triangle is obtained by connecting every two of the three positioning holes.

[0034] In this embodiment of the application, the cargo to be grabbed is the rocker arm cover of an engine.

[0035] An embodiment of the second aspect of this application provides a vision-based robotic grasping device, comprising:

[0036] The cargo current spatial coordinate acquisition module is used to acquire the current spatial coordinates of the cargo to be grasped through the robot's 3D camera;

[0037] The cargo current planar coordinate acquisition module is used to guide the robot to move to the best shooting position of the 2D camera based on the cargo current spatial coordinates, and to acquire the cargo current planar coordinates of the cargo to be grasped through the 2D camera;

[0038] The coordinate offset calculation module is used to calculate the coordinate offset of the cargo to be grasped on a preset plane based on the current plane coordinates of the cargo and the reference plane coordinates of the cargo; the reference plane coordinates of the cargo refer to the plane coordinates of the cargo to be grasped obtained by the 2D camera when the robot is first debugging the grasping position;

[0039] The cargo target spatial coordinate calculation module is used to calculate the cargo target spatial coordinates based on the cargo's current spatial coordinates, the coordinate offset, and the cargo's reference spatial coordinates; the cargo reference spatial coordinates refer to the spatial coordinates of the cargo to be grasped obtained by the robot through the 3D camera during the initial debugging of the grasping position.

[0040] The grasping spatial coordinate calculation module is used to calculate the robot's grasping spatial coordinates based on the target spatial coordinates of the cargo, the reference spatial coordinates of the cargo, the current capture coordinates of the robot's 3D camera, and the reference capture coordinates of the 3D camera; the reference capture coordinates of the 3D camera refer to the spatial position of the robot when the reference plane coordinates of the cargo are obtained through the 3D camera.

[0041] The cargo grasping module is used to guide the robot to move to the optimal grasping position according to the grasping space coordinates, and to grasp the cargo to be grasped.

[0042] An embodiment of the third aspect of this application provides a computer device including a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to perform the vision-based robot grasping method described in the first aspect.

[0043] An embodiment of the fourth aspect of this application provides a computer-readable storage medium storing computer instructions for causing a computer to execute the vision-based robot grasping method described in the first aspect above.

[0044] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0045] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0046] Figure 1 A flowchart illustrating a vision-based robot grasping method according to an embodiment of this application is shown.

[0047] Figure 2 A schematic diagram is shown of three positioning holes in a cargo to be grasped by geometric matching according to an embodiment of this application;

[0048] Figure 3 This invention provides a schematic flowchart for calculating the offset of cargo in the XY plane according to an embodiment of the present application.

[0049] Figure 4 A schematic diagram of the process for calculating the spatial coordinates of a cargo target according to an embodiment of this application is shown;

[0050] Figure 5 A schematic diagram of the process of a computational robot grasping spatial coordinates according to an embodiment of this application is shown;

[0051] Figure 6 A schematic diagram of the structure of a vision-based robotic grasping device according to an embodiment of this application is shown.

[0052] Figure 7 This illustration shows a schematic diagram of the structure of a computer device according to an embodiment of this application;

[0053] Figure 8 A schematic diagram of a storage medium provided in one embodiment of this application is shown. Detailed Implementation

[0054] Exemplary embodiments of this application will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of this application are shown in the drawings, it should be understood that this application may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of this application and to fully convey the scope of this application to those skilled in the art.

[0055] It should be noted that, unless otherwise stated, the technical or scientific terms used in this application shall have the ordinary meaning as understood by one of ordinary skill in the art to which this application pertains.

[0056] According to an embodiment of this application, a vision-based robot grasping method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Also, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0057] This embodiment provides a vision-based robot grasping method. Figure 1 This is a flowchart of a vision-based robot grasping method according to an embodiment of this application, such as... Figure 1 As shown, the process includes the following steps:

[0058] Step S101: Obtain the current spatial coordinates of the goods to be grabbed using the robot's 3D camera.

[0059] Specifically, the robot is controlled to carry a 3D camera to take pictures of the goods to be grasped, thereby obtaining the current spatial coordinates of the goods (X1, Y1, Z1, RX1, RY1, RZ1); where X1, Y1, and Z1 represent the X-axis coordinates, Y-axis coordinates, and Z-axis coordinates of the goods in three-dimensional space, respectively, and RX1, RY1, and RZ1 represent the rotation angles of the goods around the X-axis, Y-axis, and Z-axis, respectively.

[0060] Step S102: Guide the robot to move to the optimal shooting position of the 2D camera according to the current spatial coordinates of the cargo, and obtain the current planar coordinates of the cargo to be grabbed through the 2D camera.

[0061] Specifically, 3D cameras have the advantages of a large field of view and lower precision, and can be used for preliminary goods locating; 2D cameras have the advantages of a small field of view and high precision, and can be used for precise positioning of the grasping position. That is, the 3D camera is first used to coarsely locate the goods to be grasped, then the robot moves to a specific position, and then the 2D camera is used to precisely locate the goods to be grasped, thereby obtaining more accurate planar coordinates.

[0062] In some specific embodiments, step S102 above includes steps S1021-S1024:

[0063] Step S1021: Calculate the coordinates of the three positioning holes in the cargo to be grabbed.

[0064] Specifically, the three positioning holes are, for example... Figure 2 The letters “A”, “B”, and “C” in the text.

[0065] In some specific embodiments, before step S1021, the method further includes: locating three positioning holes in the cargo to be grasped by geometric matching, and connecting every two positioning holes to obtain the target triangle.

[0066] Step S1022: Calculate the angle of each corner in the target triangle based on the coordinates of the three positioning holes.

[0067] Step S1023: Determine whether the angle of each corner is within the corresponding angle range.

[0068] Specifically, the corresponding angle range can be set according to the actual situation, and no specific limitation is made here.

[0069] Step S1024: If the angle of each corner is within the corresponding angle range, then determine the current plane coordinates of the goods to be grabbed based on the coordinates of the three positioning holes.

[0070] In the embodiments of this application, such as Figure 2 , Figure 3 As shown: The geometric matching function can be used to search and locate the feature positions of region ABC. The follow function can be used to further lock the search area of ​​circles A, B, and C. The coordinates of the center A, center B, and center C can be calculated. Connect AB, AC, and BC to obtain ΔABC. Calculate the angles a, b, and c, and determine whether the angles a, b, and c are within the allowable range. If any angle exceeds the allowable range, the detection is considered to have failed. If none of them exceed the allowable range, the 2D camera is considered to have successfully captured and detected the image. The current planar coordinates of the goods to be grabbed can be determined based on the coordinates of the three positioning holes.

[0071] In some specific embodiments, the three positioning holes include a first positioning hole, a second positioning hole, and a third positioning hole; the above step S1024 includes steps a1-a4:

[0072] Step a1: Connect the first positioning hole and the second positioning hole to obtain the first line segment.

[0073] Step a2: Draw a perpendicular line to the first line segment through the third positioning hole, and obtain the coordinates of the perpendicular point of the perpendicular line.

[0074] Step a3: Calculate the angle between the perpendicular line and the preset plane.

[0075] Step a4: Obtain the current planar coordinates of the goods to be grabbed based on the coordinates of the vertical point and the included angle.

[0076] according to Figure 2 , Figure 3The following explains steps a1-a4: When the first positioning hole is A and the second positioning hole is B, line segment AB is obtained by connecting the first positioning hole A and the second positioning hole B; a perpendicular line is drawn through the second positioning hole C to line segment AB, with the perpendicular point being D(X2, Y2). The angle RZ2 between line segment CD and the horizontal plane (i.e., the XY plane) is calculated to obtain the current plane coordinates (X2, Y2, RZ2) of the goods to be grabbed.

[0077] Step S103: Calculate the coordinate offset of the cargo to be grabbed on the preset plane based on the current plane coordinates of the cargo and the reference plane coordinates of the cargo.

[0078] Specifically, the cargo reference plane coordinates refer to the plane coordinates of the cargo to be grasped obtained by the 2D camera when the robot is first debugging the grasping position.

[0079] More specifically, such as Figure 3 As shown: After obtaining the current planar coordinates of the goods to be grabbed, the coordinate offset of the goods on the preset plane can be calculated using the following formula:

[0080] (dX2, dY2, dRZ2) = (X2-X0, Y2-Y0, RZ2-RZ0)

[0081] Where (dX2, dY2, dRZ2) represents the coordinate offset, (X0, Y0, RZ0) represents the reference plane coordinates of the cargo, and (X2, Y2, RZ2) represents the current plane coordinates of the cargo.

[0082] Step S104: Calculate the target spatial coordinates of the cargo based on the current spatial coordinates of the cargo, the coordinate offset, and the reference spatial coordinates of the cargo.

[0083] Specifically, the cargo reference spatial coordinates refer to the spatial coordinates of the cargo to be grasped obtained by the 3D camera when the robot is first debugging the grasping position.

[0084] In some specific embodiments, step S104 above includes steps S1041-S1042:

[0085] Step S1041: Calculate the actual coordinates of the cargo to be grabbed on the preset plane based on the cargo reference space coordinates and the coordinate offset.

[0086] Specifically, such as Figure 4 As shown: The actual coordinates (X3, Y3, RZ3) of the cargo in the XY plane can be obtained by adding the planar coordinates (X, Y, Z) from the cargo's reference spatial coordinates (X, Y, Z, RX, RY, RZ) to the coordinate offsets (dX2, dY2, dRZ2), as shown in the following formula:

[0087] (X3, Y3, RZ3) = (X+dX2, Y+dY2, Z+dRZ2)

[0088] Step S1042: Replace the current spatial coordinates of the cargo with the actual coordinates to obtain the target spatial coordinates of the cargo.

[0089] Specifically, such as Figure 4 As shown: The target spatial coordinates of the cargo (X3, Y3, Z1, RX1, RY1, RZ3) can be obtained by replacing the corresponding values ​​in the current spatial coordinates (X1, Y1, Z1, RX1, RY1, RZ1) of the cargo with the actual coordinates (X3, Y3, RZ3) of the cargo to be grabbed on the XY plane.

[0090] Step S105: Calculate the robot's grasping space coordinates based on the target space coordinates of the cargo, the reference space coordinates of the cargo, the current photo coordinates of the robot's 3D camera, and the reference photo coordinates of the 3D camera.

[0091] Specifically, the 3D camera reference image coordinates refer to the spatial position of the robot when the reference plane coordinates of the cargo are obtained through the 3D camera.

[0092] In some specific embodiments, step S105 above includes steps S1051-S1053:

[0093] Step S1051: Calculate the 3D camera image coordinate offset based on the current image coordinates of the robot's 3D camera and the reference image coordinates of the 3D camera.

[0094] Specifically, the current image coordinates of the 3D camera can be understood as the actual spatial pose coordinates of the robot's end effector when the robot moves with the 3D camera to photograph the goods to be grasped. The reference image coordinates of the 3D camera can be understood as the standard spatial pose coordinates of the robot's end effector when the robot is photographing a reference goods with the 3D camera during the initial debugging or calibration of the grasping position.

[0095] More specifically, such as Figure 5 As shown: The 3D camera's reference coordinates (X4, Y4, Z4, RX4, RY4, RZ4) can be subtracted from the current 3D camera's capture coordinates (X5, Y5, Z5, RX5, RY5, RZ5) to obtain the 3D camera's capture coordinate offset (dX3, dY3, dZ3, dRX3, dRY3, dRZ3), as shown below:

[0096] (dX3, dY3, dZ3, dRX3, dRY3, dRZ3) = (X5-X4, Y5-Y4, Z5-Z4, RX5-RX4, RY5-RY4, RZ5-RZ4)

[0097] Step S1052: Calculate the cargo spatial coordinate offset based on the cargo target spatial coordinates and the cargo reference spatial coordinates.

[0098] Specifically, such as Figure 5 As shown: The cargo spatial coordinate offset (dX4, dY4, dZ4, dRX4, dRY4, dRZ4) can be obtained by subtracting the cargo reference spatial coordinates (X, Y, Z, RX, RY, RZ3) from the cargo target spatial coordinates (X3, Y3, Z1, RX1, RY1, RZ3), as shown below:

[0099] (dX4, dY4, dZ4, dRX4, dRY4, dRZ4)= (X3-X, Y3-Y, Z1-Z, RX1-RX, RY1-RY, RZ3-RZ)

[0100] Step S1053: Calculate the robot's grasping space coordinates based on the 3D camera reference image coordinates, the 3D camera image coordinate offset, and the cargo space coordinate offset.

[0101] Specifically, such as Figure 5 As shown: The robot's grasping space coordinates can be obtained by adding the 3D camera's reference image coordinates, the 3D camera's image coordinate offset, and the cargo space coordinate offset, as shown below:

[0102] The robot's grasping space coordinates (X6, Y6, Z6, RX6, RY6, RZ6) = 3D camera reference image coordinates (X4, Y4, Z4, RX4, RY4, RZ4) + 3D camera image coordinate offsets (dX3, dY3, dZ3, dRX3, dRY3, dRZ3) + cargo space coordinate offsets (dX4, dY4, dZ4, dRX4, dRY4, dRZ4)

[0103] Step S106: Guide the robot to move to the optimal grasping position according to the grasping space coordinates, and grasp the goods to be grasped.

[0104] Specifically, the grasping space coordinates (X6, Y6, Z6, RX6, RY6, RZ6) are sent to the robot's control system so that the robot can move to the optimal grasping position to grasp the goods to be grasped and assemble them.

[0105] To maximize positioning accuracy, this embodiment incorporates a 2D camera into the 3D positioning system, improving the crucial XY plane positioning accuracy to 0.05mm and ensuring accurate grasping. The grasping mechanism utilizes rubber suction cups to enhance the flexibility of the mechanical system, allowing it to adapt to errors caused by spatial angular offsets of the product to a certain extent.

[0106] In this embodiment, a layered detection mechanism is used. Specifically, a 3D camera is used for coarse positioning to obtain the approximate spatial pose of the goods, and then a 2D camera is used for precise positioning. Based on the 3D positioning, the pose offset of the goods in the XY plane (dX2, dY2, dRZ2) is obtained at close range. The (X, Y, RZ) in the 3D spatial coordinates of the goods obtained by the 3D camera are replaced by the pose offset of the goods in the XY plane. This greatly improves the positioning accuracy of the XY plane in the 3D positioning system, thereby ensuring the accuracy of goods grasping.

[0107] Corresponding to the above implementation methods of the vision-based robot grasping method, this application also provides a vision-based robot grasping device for executing the vision-based robot grasping method described in any of the above embodiments. Figure 6 As shown, the vision-based robotic grasping device includes:

[0108] The cargo current spatial coordinate acquisition module is used to acquire the current spatial coordinates of the cargo to be grasped through the robot's 3D camera;

[0109] The cargo current planar coordinate acquisition module is used to guide the robot to move to the best shooting position of the 2D camera based on the cargo current spatial coordinates, and to acquire the cargo current planar coordinates of the cargo to be grasped through the 2D camera;

[0110] The coordinate offset calculation module is used to calculate the coordinate offset of the cargo to be grasped on a preset plane based on the current plane coordinates of the cargo and the reference plane coordinates of the cargo; the reference plane coordinates of the cargo refer to the plane coordinates of the cargo to be grasped obtained by the 2D camera when the robot is first debugging the grasping position;

[0111] The cargo target spatial coordinate calculation module is used to calculate the cargo target spatial coordinates based on the cargo's current spatial coordinates, the coordinate offset, and the cargo's reference spatial coordinates; the cargo reference spatial coordinates refer to the spatial coordinates of the cargo to be grasped obtained by the robot through the 3D camera during the initial debugging of the grasping position.

[0112] The grasping spatial coordinate calculation module is used to calculate the robot's grasping spatial coordinates based on the target spatial coordinates of the cargo, the reference spatial coordinates of the cargo, the current capture coordinates of the robot's 3D camera, and the reference capture coordinates of the 3D camera; the reference capture coordinates of the 3D camera refer to the spatial position of the robot when the reference plane coordinates of the cargo are obtained through the 3D camera.

[0113] The cargo grasping module is used to guide the robot to move to the optimal grasping position according to the grasping space coordinates, and to grasp the cargo to be grasped.

[0114] Optionally, the cargo current plane coordinate acquisition module is also used to calculate the coordinates of the three positioning holes in the cargo to be grabbed; calculate the angle of each corner of the target triangle based on the coordinates of the three positioning holes; the target triangle is formed based on the three positioning holes; determine whether the angle of each corner is within the corresponding angle range; if the angle of each corner is within the corresponding angle range, then determine the cargo current plane coordinates of the cargo to be grabbed based on the coordinates of the three positioning holes.

[0115] Optionally, the cargo target spatial coordinate calculation module is further configured to calculate the actual coordinates of the cargo to be grabbed on a preset plane based on the cargo reference spatial coordinates and the coordinate offset; and replace the current spatial coordinates of the cargo with the actual coordinates to obtain the cargo target spatial coordinates.

[0116] Optionally, the grasping space coordinate calculation module is further configured to calculate the 3D camera capture coordinate offset based on the current capture coordinates of the robot's 3D camera and the reference capture coordinates of the 3D camera; calculate the cargo space coordinate offset based on the cargo target space coordinates and the cargo reference space coordinates; and calculate the robot's grasping space coordinates based on the 3D camera reference capture coordinates, the 3D camera capture coordinate offset, and the cargo space coordinate offset.

[0117] Optionally, the cargo current plane coordinate acquisition module is further configured to connect the first positioning hole and the second positioning hole to obtain a first line segment; draw a perpendicular line to the first line segment through the third positioning hole and obtain the coordinates of the perpendicular point of the perpendicular line; calculate the angle between the perpendicular line and the preset plane; and obtain the cargo current plane coordinates of the cargo to be grabbed based on the coordinates of the perpendicular point and the angle.

[0118] Optionally, the device further includes: a target triangle construction module for locating three positioning holes in the cargo to be grasped through geometric matching; and connecting every two positioning holes to obtain the target triangle.

[0119] The vision-based robot grasping device and the vision-based robot grasping method provided in the above embodiments of this application are based on the same inventive concept and have the same beneficial effects as the methods used, run or implemented by the applications stored in them.

[0120] This application also provides a computer device for executing the above-described vision-based robot grasping method. Please refer to... Figure 7This illustrates a schematic diagram of a computer device provided by some embodiments of this application. For example... Figure 7 As shown, the computer device 7 includes: a processor 700, a memory 701, a bus 702, and a communication interface 703. The processor 700, the communication interface 703, and the memory 701 are connected via the bus 702. The memory 701 stores a computer program that can run on the processor 700. When the processor 700 runs the computer program, it executes the vision-based robot grasping method provided in any of the foregoing embodiments of this application.

[0121] The memory 701 may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Communication between this system network element and at least one other network element is achieved through at least one communication interface 703 (which can be wired or wireless), such as the Internet, wide area network, local area network, or metropolitan area network.

[0122] Bus 702 can be an ISA bus, PCI bus, or EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. The memory 701 is used to store programs. After receiving an execution instruction, the processor 700 executes the program. The vision-based robot grasping method disclosed in any of the foregoing embodiments can be applied to the processor 700, or implemented by the processor 700.

[0123] The processor 700 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 700 or by instructions in software form. The processor 700 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules may reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 701. Processor 700 reads the information in memory 701 and, in conjunction with its hardware, completes the steps of the above method.

[0124] The computer device provided in this application embodiment and the vision-based robot grasping method provided in this application embodiment are based on the same inventive concept and have the same beneficial effects as the methods they adopt, operate or implement.

[0125] This application also provides a computer-readable storage medium corresponding to the vision-based robot grasping method provided in the foregoing embodiments. Please refer to [link / reference]. Figure 8 The computer-readable storage medium shown is an optical disc 30, on which a computer program (i.e., program cargo) is stored. When the computer program is run by a processor, it executes the vision-based robot grasping method provided in any of the foregoing embodiments.

[0126] It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical and magnetic storage media, which will not be elaborated here.

[0127] The computer-readable storage medium provided in the above embodiments of this application and the robot grasping method based on vision positioning provided in the embodiments of this application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the applications stored therein.

[0128] It should be noted that:

[0129] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of this application may be practiced without these specific details. In some instances, well-known structures and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0130] Similarly, it should be understood that, for the sake of brevity and to aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of this application, various features of this application are sometimes grouped together in a single embodiment, figure, or description thereof. However, this disclosure should not be construed as reflecting a schematic diagram in which the claimed application requires more features than expressly recited in each claim. Rather, as reflected in the following claims, inventive aspects lie in fewer than all features of a single foregoing disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of this application.

[0131] Furthermore, those skilled in the art will understand that although some embodiments described herein include certain features but not others included in other embodiments, combinations of features from different embodiments are intended to be within the scope of this application and form different embodiments. For example, in the following claims, any of the claimed embodiments can be used in any combination.

[0132] The above description is merely a preferred embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A robot grasping method based on vision localization, characterized in that, The method includes: The robot's 3D camera obtains the current spatial coordinates of the goods to be grabbed. The robot is guided to the optimal shooting position of the 2D camera based on the current spatial coordinates of the cargo, and the current planar coordinates of the cargo to be grasped are obtained through the 2D camera. The coordinate offset of the cargo to be grasped on the preset plane is calculated based on the current planar coordinates of the cargo and the reference planar coordinates of the cargo; the reference planar coordinates of the cargo refer to the planar coordinates of the cargo to be grasped obtained by the 2D camera when the robot is first debugging the grasping position; The target spatial coordinates of the cargo are calculated based on the current spatial coordinates of the cargo, the coordinate offset, and the reference spatial coordinates of the cargo; the reference spatial coordinates of the cargo refer to the spatial coordinates of the cargo to be grasped obtained by the 3D camera when the robot is first debugging the grasping position. The robot's grasping spatial coordinates are calculated based on the target spatial coordinates of the cargo, the reference spatial coordinates of the cargo, the current capture coordinates of the robot's 3D camera, and the reference capture coordinates of the 3D camera. The reference capture coordinates of the 3D camera refer to the spatial position of the robot when the reference plane coordinates of the cargo are obtained through the 3D camera. Guided by the grasping space coordinates, the robot moves to the optimal grasping position and grasps the goods to be grasped; The 2D camera is used to obtain the current planar coordinates of the goods to be grabbed, including: Calculate the coordinates of the three positioning holes in the cargo to be grabbed; The angle of each corner in the target triangle is calculated based on the coordinates of the three positioning holes; the target triangle is formed based on the three positioning holes. Determine whether the angle of each corner is within the corresponding angle range; If the angle of each corner is within the corresponding angle range, then the current plane coordinates of the goods to be grabbed are determined according to the coordinates of the three positioning holes; The three positioning holes include a first positioning hole, a second positioning hole, and a third positioning hole; determining the current planar coordinates of the goods to be grasped based on the coordinates of the three positioning holes includes: A first line segment is obtained by connecting the first positioning hole and the second positioning hole; Draw a perpendicular line to the first line segment through the third positioning hole, and obtain the coordinates of the perpendicular point of the perpendicular line; Calculate the angle between the perpendicular line and the preset plane; The current planar coordinates of the cargo to be grabbed are obtained based on the coordinates of the perpendicular point and the included angle.

2. The method according to claim 1, characterized in that, Based on the cargo's current spatial coordinates, the coordinate offset, and the cargo's reference spatial coordinates, the target spatial coordinates of the cargo are calculated, including: The actual coordinates of the cargo to be grabbed on the preset plane are calculated based on the cargo reference space coordinates and the coordinate offset. The target spatial coordinates of the cargo are obtained by replacing the current spatial coordinates of the cargo with the actual coordinates.

3. The method according to claim 1, characterized in that, Based on the target spatial coordinates of the cargo, the reference spatial coordinates of the cargo, the current image coordinates of the robot's 3D camera, and the reference image coordinates of the 3D camera, the robot's grasping spatial coordinates are calculated, including: Calculate the 3D camera coordinate offset based on the robot's current 3D camera coordinates and the 3D camera reference coordinates. Calculate the cargo spatial coordinate offset based on the cargo target spatial coordinates and the cargo reference spatial coordinates; The robot's grasping space coordinates are calculated based on the 3D camera's reference capture coordinates, the 3D camera's capture coordinate offset, and the cargo's space coordinate offset.

4. The method according to claim 1, characterized in that, Before calculating the coordinates of the three positioning holes in the cargo to be grasped, the method further includes: The three positioning holes in the cargo to be grabbed are located by geometric matching; The target triangle is obtained by connecting every two of the three positioning holes.

5. The method according to claim 1, characterized in that, The cargo to be grabbed is the rocker arm cover of an engine.

6. A vision-based robotic grasping device, characterized in that, The device includes: The cargo current spatial coordinate acquisition module is used to acquire the current spatial coordinates of the cargo to be grasped through the robot's 3D camera; The cargo current planar coordinate acquisition module is used to guide the robot to move to the best shooting position of the 2D camera based on the cargo current spatial coordinates, and to acquire the cargo current planar coordinates of the cargo to be grasped through the 2D camera; The coordinate offset calculation module is used to calculate the coordinate offset of the cargo to be grasped on a preset plane based on the current plane coordinates of the cargo and the reference plane coordinates of the cargo; the reference plane coordinates of the cargo refer to the plane coordinates of the cargo to be grasped obtained by the 2D camera when the robot is first debugging the grasping position; The cargo target spatial coordinate calculation module is used to calculate the cargo target spatial coordinates based on the cargo's current spatial coordinates, the coordinate offset, and the cargo's reference spatial coordinates; the cargo reference spatial coordinates refer to the spatial coordinates of the cargo to be grasped obtained by the robot through the 3D camera during the initial debugging of the grasping position. The grasping spatial coordinate calculation module is used to calculate the robot's grasping spatial coordinates based on the target spatial coordinates of the cargo, the reference spatial coordinates of the cargo, the current capture coordinates of the robot's 3D camera, and the reference capture coordinates of the 3D camera; the reference capture coordinates of the 3D camera refer to the spatial position of the robot when the reference plane coordinates of the cargo are obtained through the 3D camera. The cargo grasping module is used to guide the robot to move to the optimal grasping position according to the grasping space coordinates, and grasp the cargo to be grasped; The 2D camera is used to obtain the current planar coordinates of the goods to be grabbed, including: Calculate the coordinates of the three positioning holes in the cargo to be grabbed; The angle of each corner in the target triangle is calculated based on the coordinates of the three positioning holes; the target triangle is formed based on the three positioning holes. Determine whether the angle of each corner is within the corresponding angle range; If the angle of each corner is within the corresponding angle range, then the current plane coordinates of the goods to be grabbed are determined according to the coordinates of the three positioning holes; The three positioning holes include a first positioning hole, a second positioning hole, and a third positioning hole; determining the current planar coordinates of the goods to be grasped based on the coordinates of the three positioning holes includes: A first line segment is obtained by connecting the first positioning hole and the second positioning hole; Draw a perpendicular line to the first line segment through the third positioning hole, and obtain the coordinates of the perpendicular point of the perpendicular line; Calculate the angle between the perpendicular line and the preset plane; The current planar coordinates of the cargo to be grabbed are obtained based on the coordinates of the perpendicular point and the included angle.

7. A computer device, characterized in that, include: A memory and a processor are communicatively connected, the memory stores computer instructions, and the processor executes the computer instructions to perform the vision-based robot grasping method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to execute the vision-based robot grasping method according to any one of claims 1 to 5.

Images