Object grasping method, device, system, electronic equipment and storage medium

Abstract

The embodiment of the present application provides an object grasping method, device, system, electronic device and storage medium, which relates to the field of robot technology. The method includes: controlling a composite robot to move to a target docking point, which corresponds to a target machine; controlling the mechanical arm of the composite robot to reach a first posture, which is a posture corresponding to a teaching action; determining a target compensation posture corresponding to the target docking point and the target machine according to the corresponding relationship between the docking point, the machine and the compensation posture; compensating the posture of the mechanical arm according to the target compensation posture so that the mechanical arm reaches a second posture; positioning the target machine using a visual positioning device and a position recognition plate; and controlling the mechanical arm to grasp an object on the target machine according to the positioning result. This solution can simplify the layout of the docking points, reduce the difficulty of implementing the solution, and improve the scheduling efficiency of the composite robot.

Description

Technical Field

[0001] The present application relates to the field of robotics technology, and in particular to an object grasping method, device, system, electronic device and storage medium. Background Art

[0002] In the application scenario of the composite robot grasping objects, the composite robot controls the robotic arm to perform fixed teaching actions, and then controls the visual positioning device installed at the end of the robotic arm to collect the image of the position identification plate (such as a mark plate), uses the image to locate the machine, and grasps the object on the machine based on the positioning result.

[0003] Since the robotic arm performs fixed teaching actions, it is necessary to layout the docking points corresponding to each machine to control the distance between the machine and the composite robot when the machine is positioned, and to ensure that the visual positioning device captures the image of the complete Mark plate when the machine is positioned.

[0004] In scenarios with complex machine layouts, the use of the above-mentioned object grasping solution will result in a very complex layout of docking points, increasing the difficulty of implementation. In addition, due to the complex layout of the docking points, the walking path of the composite robot also becomes very complex, which in turn leads to low scheduling efficiency of the composite robot. Summary of the invention

[0005] The purpose of the embodiments of the present application is to provide an object grasping method, device, system, electronic device and storage medium to simplify the layout of docking points, reduce the difficulty of implementing the solution, and improve the scheduling efficiency of the composite robot. The specific technical solution is as follows:

[0006] In a first aspect, an embodiment of the present application provides a method for grasping an object, the method comprising:

[0007] Controlling the composite robot to move to a target docking point in the topological map, where the target docking point is a docking point of a target machine;

[0008] Controlling the mechanical arm of the composite robot to reach a first posture, wherein the first posture is a posture corresponding to the teaching action;

[0009] Determine the target compensation posture corresponding to the target docking point and the target machine according to the corresponding relationship between the docking point, the machine and the compensation posture;

[0010] According to the target compensation posture, the posture of the robot arm is compensated so that the robot arm reaches a second posture, where the second posture is the posture of the image of the position recognition plate attached to the target machine platform collected by the visual positioning device on the robot arm;

[0011] Positioning the target machine using the visual positioning device and the position identification plate;

[0012] According to the positioning result, the robot arm is controlled to grab the object on the target platform.

[0013] In some embodiments, the step of positioning the target machine using the visual positioning device and the position identification plate includes:

[0014] Controlling the visual positioning device to collect a first image of the position identification plate;

[0015] If the first image is not acquired, the robotic arm is controlled to move a preset step length in a preset direction, and the step of controlling the visual positioning device to acquire the first image of the position recognition plate is re-executed until the first image is acquired, or the distance from the position corresponding to the current posture to the position corresponding to the first posture is greater than a preset distance;

[0016] If the first image is acquired, the first image is recognized to obtain a positioning result of the target machine.

[0017] In some embodiments, before controlling the composite robot to move to the target docking point in the topological map, the method further includes:

[0018] Controlling the composite robot to move to the target docking point, and controlling the robotic arm to reach the first posture;

[0019] Controlling the visual positioning device to collect a second image of the position identification plate;

[0020] If the second image is acquired, determining a third posture of the robotic arm according to the current pixel position of the position identification plate in the second image and a preset pixel position range, the third posture being the posture of the robotic arm when the current pixel position is within the preset pixel position range;

[0021] The deviation between the third posture and the first posture is calculated to obtain the target compensated posture.

[0022] In some embodiments, the step of determining the third posture of the robotic arm according to the current pixel position of the position recognition plate in the second image and a preset pixel position range includes:

[0023] If the current pixel position is within the preset pixel position range, the current posture of the robotic arm is determined to be the third posture of the robotic arm.

[0024] In some embodiments, the step of determining the third posture of the robotic arm according to the current pixel position of the position recognition plate in the second image and a preset pixel position range includes:

[0025] If the current pixel position is not within the preset pixel position range, determining the adjusted posture of the robotic arm according to the current pixel position and the preset pixel position range, wherein the pixel position adjustment direction corresponding to the adjusted posture matches the direction in which the current pixel position approaches the preset pixel position range;

[0026] According to the posture adjustment, the posture of the robotic arm is adjusted, and the step of controlling the visual positioning device to collect the second image of the position identification plate is re-executed until the current pixel position is within the preset pixel position range, thereby obtaining the third posture of the robotic arm.

[0027] In some embodiments, the method further comprises:

[0028] If the second image is not collected, the robotic arm is controlled to move a preset step length in a preset direction, and the step of controlling the visual positioning device to collect the second image of the position identification plate is re-executed until the second image is collected, or the distance from the position corresponding to the current posture to the position corresponding to the first posture is greater than the preset distance.

[0029] In some embodiments, the preset direction is a clockwise direction or a counterclockwise direction on a horizontal plane, and the preset step length is less than or equal to a width of a field of view of the visual positioning device.

[0030] In some embodiments, the method further comprises:

[0031] If the distance from the position of the robot arm to the position corresponding to the first posture is greater than the preset distance and the image of the position recognition plate is not collected, an error prompt message is output.

[0032] In some embodiments, the method further comprises:

[0033] The corresponding relationship between the target docking point, the target machine and the target compensation posture is synchronized to other compound robots; or, the corresponding relationship between the target docking point, the target machine and the target compensation posture is sent to the scheduling platform of the compound robot.

[0034] In some embodiments, a docking point in the topological map corresponds to at least one machine, and the distance from the at least one machine to the docking point is less than or equal to the operating distance of the robot arm.

[0035] In a second aspect, an embodiment of the present application provides an object grasping system, the system comprising a scheduling platform and at least one composite robot;

[0036] The scheduling platform is used to issue a grasping task to a first composite robot among the at least one composite robot, wherein the grasping task includes an identifier of a target docking point in a topological map and an identifier of a target machine, wherein the target docking point is a docking point of the target machine;

[0037] The first compound robot is used to control the first compound robot to move to the target docking point according to the grasping task; control the first manipulator arm of the first compound robot to reach a first posture, the first posture being the posture corresponding to the teaching action; determine the target compensation posture corresponding to the target docking point and the target machine according to the correspondence between the docking point, the machine and the compensation posture; compensate the posture of the first manipulator arm according to the target compensation posture so that the first manipulator arm reaches a second posture, the second posture being the posture of the image of the position identification plate fitted on the target machine collected by the first visual positioning device on the first manipulator arm; use the first visual positioning device and the position identification plate to locate the target machine; and control the first manipulator arm to grab the object on the target machine according to the positioning result.

[0038] In some embodiments, the first compound robot is specifically used to: control the first visual positioning device to collect the first image of the position identification plate; if the first image is not collected, control the first robotic arm to move a preset step length in a preset direction, and re-execute the step of controlling the first visual positioning device to collect the first image of the position identification plate until the first image is collected, or the distance from the position corresponding to the current posture to the position corresponding to the first posture is greater than a preset distance; if the first image is collected, identify the first image to obtain the positioning result of the target machine.

[0039] In some embodiments, the object grasping system further includes a host computer;

[0040] The host computer is used to obtain the address of the second compound robot in the at least one compound robot through the human-computer interaction interface; connect to the second compound robot using the address; after establishing the connection, obtain the calibration task through the human-computer interaction interface, and issue the calibration task to the second compound robot;

[0041] The second compound robot is used to control the second compound robot to move to the target docking point, and according to the calibration task, control the second mechanical arm of the second compound robot to reach the first posture; control the second visual positioning device on the second mechanical arm to collect the second image of the position identification plate; if the second image is collected, determine the third posture of the second mechanical arm according to the current pixel position of the position identification plate in the second image and the preset pixel position range, and the third posture is the posture of the second mechanical arm when the current pixel position is within the preset pixel position range; calculate the deviation between the third posture and the first posture to obtain the target compensation posture.

[0042] In some embodiments, the second compound robot is further used to control the second robotic arm to move a preset step length in a preset direction if the second image is not collected, and re-execute the step of controlling the second visual positioning device on the second robotic arm to collect the second image of the position identification plate until the second image is collected, or the distance from the position corresponding to the current posture to the position corresponding to the first posture is greater than the preset distance.

[0043] In some embodiments, the second compound robot is further used to synchronize the corresponding relationship between the target docking point, the target platform and the target compensation posture to other compound robots in the at least one compound robot; or

[0044] The second composite robot is further used to send the corresponding relationship between the target docking point, the target machine and the target compensation posture to the scheduling platform; the grasping task also includes the corresponding relationship between the target docking point, the target machine and the target compensation posture.

[0045] In some embodiments, a docking point in the topological map corresponds to at least one machine, and the distance from the at least one machine to the docking point is less than or equal to the operating distance of the robot arm.

[0046] In a third aspect, an embodiment of the present application provides an object grasping device, the device comprising:

[0047] A first control module is used to control the composite robot to move to a target docking point in the topological map, where the target docking point is a docking point of a target machine; and control the manipulator arm of the composite robot to reach a first posture, where the first posture is a posture corresponding to a teaching action;

[0048] A first determination module is used to determine the target compensation posture corresponding to the target docking point and the target machine according to the correspondence between the docking point, the machine and the compensation posture;

[0049] A compensation module, used for compensating the posture of the robot arm according to the target compensation posture, so that the robot arm reaches a second posture, where the second posture is the posture of the image of the position identification plate attached to the target machine platform collected by the visual positioning device on the robot arm;

[0050] A positioning module, used to locate the target machine using the visual positioning device and the position recognition plate;

[0051] The grasping module is used to control the robot arm to grasp the object on the target platform according to the positioning result.

[0052] In some embodiments, the positioning module is specifically used to: control the visual positioning device to capture the first image of the position identification plate; if the first image is not captured, control the robotic arm to move a preset step length in a preset direction, and re-execute the step of controlling the visual positioning device to capture the first image of the position identification plate until the first image is captured, or the distance from the position corresponding to the current posture to the position corresponding to the first posture is greater than a preset distance; if the first image is captured, identify the first image to obtain the positioning result of the target machine.

[0053] In some embodiments, the apparatus further comprises:

[0054] A second control module is used to control the composite robot to move to the target docking point in the topological map and control the robot arm to reach the first posture before controlling the composite robot to move to the target docking point in the topological map;

[0055] An acquisition module, used for controlling the visual positioning device to acquire a second image of the position identification plate;

[0056] a second determination module, configured to determine, if the second image is acquired, a third posture of the robotic arm according to a current pixel position of the position recognition plate and a preset pixel position range in the second image, wherein the third posture is a posture of the robotic arm when the current pixel position is within the preset pixel position range;

[0057] A calculation module is used to calculate the deviation between the third posture and the first posture to obtain the target compensation posture.

[0058] In some embodiments, the second determination module is specifically used to: if the current pixel position is within a preset pixel position range, determine that the current posture of the robotic arm is the third posture of the robotic arm.

[0059] In some embodiments, the second determination module is specifically used to: if the current pixel position is not within the preset pixel position range, determine the adjusted posture of the robotic arm based on the current pixel position and the preset pixel position range, and the pixel position adjustment direction corresponding to the adjusted posture matches the direction in which the current pixel position approaches the preset pixel position range; adjust the posture of the robotic arm according to the adjusted posture, and re-execute the step of controlling the visual positioning device to collect the second image of the position identification plate until the current pixel position is within the preset pixel position range, so as to obtain the third posture of the robotic arm.

[0060] In some embodiments, the acquisition module is also used to: if the second image is not acquired, control the robotic arm to move a preset step length in a preset direction, and re-execute the step of controlling the visual positioning device to acquire the second image of the position identification plate, until the second image is acquired, or the distance from the position corresponding to the current posture to the position corresponding to the first posture is greater than a preset distance.

[0061] In some embodiments, the preset direction is a clockwise direction or a counterclockwise direction on a horizontal plane, and the preset step length is less than or equal to a width of a field of view of the visual positioning device.

[0062] In some embodiments, the apparatus further comprises:

[0063] The output module is used to output an error prompt message if the image of the position recognition plate is not collected when the distance from the position of the robot arm to the position corresponding to the first posture is greater than the preset distance.

[0064] In some embodiments, the apparatus further comprises:

[0065] A synchronization module is used to synchronize the corresponding relationship between the target docking point, the target machine and the target compensation posture to other composite robots; or to send the corresponding relationship between the target docking point, the target machine and the target compensation posture to the scheduling platform of the composite robot.

[0066] In some embodiments, a docking point in the topological map corresponds to at least one machine, and the distance from the at least one machine to the docking point is less than or equal to the operating distance of the robot arm.

[0067] In a fourth aspect, an embodiment of the present application provides an electronic device, including:

[0068] Memory, used to store computer programs;

[0069] The processor is used to implement any of the above-mentioned object grasping methods when executing the program stored in the memory.

[0070] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the object grasping method described above is implemented.

[0071] In a sixth aspect, an embodiment of the present application provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute any of the object grasping methods described above.

[0072] Beneficial effects of the embodiments of the present application:

[0073] In the technical solution provided by the embodiment of the present application, when the composite robot grasps an object, the composite robot performs a fixed teaching action, that is, when the mechanical arm of the composite robot reaches the first posture, the corresponding relationship between the docking point, the machine and the compensation posture is used to determine the target compensation posture corresponding to the target docking point and the target machine, and then the posture of the mechanical arm is compensated to achieve the grasping of the object on the target machine at the target docking point. In this solution, although a fixed teaching action is still used, each docking point and each machine has a corresponding compensation posture. As long as the distance from multiple machines to a docking point is less than or equal to the operating distance of the mechanical arm, the technical solution provided by the embodiment of the present application can be used at the docking point to grasp the objects on these multiple machines. That is, multiple machines can share the same docking point, which simplifies the layout of the docking points, reduces the difficulty of implementing the solution, and improves the scheduling efficiency of the composite robot.

[0074] Of course, implementing any product or method of the present application does not necessarily require achieving all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0076] Figure 1 A structural schematic diagram of a composite robot;

[0077] Figure 2a It is a schematic diagram of the first structure of the position identification plate;

[0078] Figure 2b It is a schematic diagram of the second structure of the position identification plate;

[0079] Figure 3 A schematic diagram of the docking point;

[0080] Figure 4a A schematic diagram of a machine layout scenario;

[0081] Figure 4b Based on Figure 4a A schematic diagram of the docking points of the machine layout scenario shown;

[0082] Figure 4c For composite robots based on Figure 4b The first schematic diagram of performing object grasping at the docking point shown;

[0083] Figure 4d For composite robots based on Figure 4b A second schematic diagram of performing object grasping at the docking point shown;

[0084] Figure 5 A schematic diagram showing that the Mark board is not within the field of view of the visual positioning device;

[0085] Figure 6 A schematic diagram of the structure of an object grasping system provided in an embodiment of the present application;

[0086] Figure 7 A first schematic diagram of the interaction process between the scheduling platform and the composite robot provided in an embodiment of the present application;

[0087] Figure 8 Based on the embodiments of the present application Figure 4a A schematic diagram of the docking points of the machine layout scenario shown;

[0088] Fig. 9 A schematic diagram of the posture of the robotic arm provided in an embodiment of the present application;

[0089] Fig.10 A second schematic diagram of the interaction process between the scheduling platform and the composite robot provided in an embodiment of the present application;

[0090] Fig.11a A first schematic diagram of a preset direction provided in an embodiment of the present application;

[0091] Fig.11b A second schematic diagram of a preset direction provided in an embodiment of the present application;

[0092] Fig.12 A schematic diagram of an image provided in an embodiment of the present application;

[0093] Fig.13A first schematic diagram of the interaction process between the host computer and the composite robot provided in an embodiment of the present application;

[0094] Fig.14 A schematic diagram of a human-computer interaction interface provided in an embodiment of the present application;

[0095] Fig.15 A schematic diagram of a preset pixel position range provided in an embodiment of the present application;

[0096] Fig.16 A schematic diagram for comparing images before and after posture adjustment provided in an embodiment of the present application;

[0097] Fig.17 A schematic diagram of a flow chart of an object grabbing method provided in an embodiment of the present application;

[0098] Fig.18 A partial flow chart of the object grasping method provided in the embodiment of the present application;

[0099] Fig.19 The composite robot provided in the embodiment of the present application is based on Figure 8 A schematic diagram of performing object grasping at the docking point shown;

[0100] Fig. 20 A schematic diagram of the structure of a composite robot provided in an embodiment of the present application;

[0101] Fig.21 A schematic diagram of the structure of an object grasping device provided in an embodiment of the present application;

[0102] Fig. 22 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0103] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field based on the present application belong to the scope of protection of the present application.

[0104] To facilitate understanding, the terms appearing in the embodiments of the present application are explained below.

[0105] A machine is a piece of mechanical equipment used in production or work.

[0106] Composite robot: A new type of robot that integrates the functions of a mobile robot and an industrial robot. The structure of a composite robot can be found in Figure 1The shown automatic guided vehicle (AGV) is equipped with a mechanical arm 111. In the embodiment of the present application, a composite robot may have one or more mechanical arms, which is not limited.

[0107] Position recognition plate (Mark plate): The relative position relationship with the grasped object is fixed. The camera takes an image of the Mark plate and recognizes the features of the Mark plate to compensate for the deviation of the grasping object by the robot arm. The styles of the Mark plate can be found in Figure 2a~Figure 2b As shown, in the embodiment of the present application, the style of the Mark plate is not limited, as long as the characteristics of the Mark plate can be accurately identified and the position of the target object can be determined.

[0108] Topological map: It is composed of points and lines, and is used to describe the layout and connectivity of space. Points are the locations that the composite robot needs to pass through when moving. Connecting multiple points can form the walking path of the composite robot.

[0109] Hand-eye grasping: The target object is located by a visual positioning device, thereby guiding the robot arm to grasp the target object. The visual positioning device may include a camera, a sighting device, a lighting compensation device, etc., which are not limited.

[0110] Posture compensation: Sample the motion compensation parameters and adjust the motion.

[0111] Composite robots are often used in handling, mobile loading and unloading, etc. The visual positioning device of the composite robot is generally installed on the end flange of the robot arm. As the robot arm moves, the position of the visual positioning device will also change.

[0112] Before grabbing the object on the machine, the composite robot teaches the robot arm to move so that the composite robot can obtain the position of the robot arm when the machine is positioned. For the same type of machine, the position of the robot arm when the machine is positioned is fixed. Correspondingly, for the same type of machine, the distance between the point corresponding to each machine in the topological map layout (i.e., the docking point) and the machine (i.e., the docking distance) is fixed. Figure 3 As shown, machine 1 to machine 3 are machines of the same type, the distance between machine 1 and docking point 1 is d1, the distance between machine 2 and docking point 2 is d2, and the distance between machine 3 and docking point 3 is d3, d1=d2=d3.

[0113] When grabbing an object on a machine, the composite robot moves to the docking point corresponding to the machine and controls the robotic arm to perform the teaching action, that is, controls the robotic arm to reach the posture obtained when performing the teaching; collects the image of the Mark plate attached to the machine, uses the image to locate the machine, and then guides the robotic arm to grab the object on the machine.

[0114] For scenarios with complex machine layouts, adopting the above-mentioned object grasping solution will result in a very complex layout of docking points, increasing the difficulty of implementation.

[0115] like Figure 4a In the machine layout scenario shown in FIG. 1 , there are 6 machines arranged on both sides of the aisle, namely, machine 1 to machine 6. The distances from machine 1 to machine 6 to the aisle are different, as follows: Figure 4a a1~a6 shown in the figure can improve the space utilization. Since the robot collects the Mark plate image in a fixed position to locate the machine, docking point 1~docking point 6 are arranged for machine 1~machine 6 in the aisle, and the distances from docking point 1~docking point 6 to machine 1~machine 6 are as follows: Figure 4b This makes the layout of the docking points very complicated, and the docking points are laid out by manual measurement, which makes the implementation of the solution more difficult, thereby extending the project delivery cycle.

[0116] In addition, the layout of the docking points is very complex, and the walking path of the composite robot also becomes very complicated, such as Figure 4b As shown in the figure, the six docking points form two parallel walking paths, namely walking path 1 and walking path 2. The composite robot moves on walking path 1 and walking path 2, and the scheduling efficiency is low when performing the object grasping task.

[0117] For example, in a single composite robot scenario, the composite robot is at position 0, and the object grasping tasks include task 1 of grasping the object on platform 2 and task 2 of grasping the object on platform 5. The composite robot first moves along Figure 4c The trajectory represented by the middle dotted line moves to docking point 2, completes task 1, and then moves to docking point 5, completing task 2.

[0118] In the scenario of multiple compound robots, for example, compound robots 1 and 2 are located at point 0, and the object grasping tasks include task 1 of grasping the object on machine 2 and task 1 of grasping the object on machine 5. Compound robot 1 moves along Figure 4d The track represented by the short dashed line moves to docking point 2, completing task 1. Since the width of the aisle is smaller than the sum of the widths of the two composite robots, that is, the aisle cannot accommodate the two composite robots, the composite robot 1 is locked at docking point 5 while completing task 1. The composite robot 2 moves along the track represented by the short dashed line to docking point 2, completing task 1. Figure 4d The trajectory represented by the middle long dashed line moves to docking point 4 and then stops moving. When the compound robot 1 completes task 1 and leaves docking point 2, after moving to docking point 3, the compound robot 2 moves to docking point 5 and completes task 2.

[0119] It can be seen that when the layout of docking points is complex, the scheduling efficiency of the composite robot will be greatly reduced.

[0120] In addition, in the above-mentioned object grasping scheme, since the visual positioning device collects the image of the Mark plate to position the machine when the robot arm performs a fixed teaching action, it has high requirements for the chassis positioning and walking accuracy of the composite robot. If the chassis positioning and walking accuracy of the composite robot are low, the composite robot will not be able to move accurately to the docking point, and then when the robot arm performs a fixed teaching action, the Mark plate is not within the field of view of the visual positioning device, such as Figure 5 This results in a decrease in the positioning accuracy of the machine and makes it impossible to complete the object grasping task.

[0121] To solve the above problems, the present application provides an object grasping system, such as Figure 6 As shown, it includes a scheduling platform 61 and at least one composite robot 62. The interaction process between the scheduling platform 61 and the composite robot 62 is as follows: Figure 7 shown.

[0122] In step S71, the scheduling platform 61 issues a grasping task to the first composite robot in the at least one composite robot 62. The grasping task includes the identification of the target docking point and the identification of the target machine in the topological map, and the target docking point is the docking point of the target machine.

[0123] In the embodiment of the present application, the scheduling platform 61 is responsible for scheduling each compound robot 62 in the object grasping system to complete the object grasping task. The grasping task is used to instruct the compound robot to perform object grasping. The first compound robot can be any compound robot 62 in the object grasping system. The target machine is the machine where the object to be grasped in this grasping task is located.

[0124] The target docking point is a point in the topological map corresponding to the target machine. In the topological map, a docking point can correspond to at least one machine, and the distance from the at least one machine to a docking point is less than or equal to the operating distance of the robot arm.

[0125] like Figure 4a In the machine layout scenario shown in the figure, considering the working distance d of the robot arm, a docking point 11 is deployed between machine 1 and machine 4, a docking point 12 is deployed between machine 2 and machine 5, and a docking point 13 is deployed between machine 3 and machine 6, as shown in FIG. Figure 8 In this case, machine 1 and machine 4 share docking point 11, machine 2 and machine 5 share docking point 12, and machine 3 and machine 6 share docking point 13. The distances a1' to a6' from machine 1 to machine 6 to their respective corresponding docking points are all less than d.

[0126] In the embodiment of the present application, the layout of the docking points is simplified, reducing the difficulty of implementing the solution. In addition, considering the operating distance of the robot arm, although the distances from each machine to the docking point are not exactly the same, the subsequent compensation posture can make up for the error caused by the distance difference and achieve accurate grasping of objects.

[0127] After acquiring the grasping task, the scheduling platform 61 determines a compound robot 62 that executes the grasping task from at least one compound robot 62 as the first compound robot, and issues the grasping task to the first compound robot.

[0128] Among them, the scheduling platform 61 determines an idle compound robot 62 from at least one compound robot 62 as the first compound robot, or can determine the compound robot 62 closest to the target docking point as the first compound robot. The first compound robot can also be determined by other methods, which are not limited to this.

[0129] Step S72: the first compound robot is controlled to move to a target docking point according to the grasping task.

[0130] After the first composite robot receives the grasping task, it can obtain the identification of the target docking point and the identification of the target machine from the grasping task, and then determine the target docking point according to the identification of the target docking point, and then plan a route from the current position of the first composite robot to the target docking point, and drive its own chassis to move along the planned route to the target docking point.

[0131] Step S73: the first compound robot controls the first mechanical arm of the first compound robot to reach a first posture, wherein the first posture is a posture corresponding to the teaching action.

[0132] In the embodiment of the present application, the teaching action is a manually taught action for grasping an object. Before executing a grasping task, the composite robot learns and obtains the teaching action. The number of teaching actions can be one or more. For example, the composite robot learns one teaching action, and the composite robot executes the teaching action when executing a grasping task; or, the composite robot learns two teaching actions, and the composite robot executes the teaching action corresponding to the moving direction of the composite robot when executing a grasping task. In the embodiment of the present application, the teaching action is stored in the form of a posture (such as the first posture), and the first posture can be carried in the grasping task and sent to the first composite robot by the scheduling platform 61; the first posture can also be pre-stored in each composite robot 62, and there is no limitation on this.

[0133] The above-mentioned robot arm posture can be understood as the posture of the tool center point (TCP) at the end of the robot arm relative to the robot arm base, that is, TCP in Fig. 9The position and posture of the robot arm base coordinate system XYZ shown can be expressed as (x, y, z, rx, ry, rz), where (x, y, z) represents the three-axis coordinate value of TCP in the robot arm base coordinate system, and (rx, ry, rz) represents the rotation angle of TCP around the X, Y, and Z axes.

[0134] The first mechanical arm is a mechanical arm of the first composite robot. After the first composite robot moves to the target docking point, the first mechanical arm is controlled to perform a teaching action for grabbing an object on the target machine, that is, the first mechanical arm is controlled to reach the first posture.

[0135] Step S74: the first composite robot determines a target compensation posture corresponding to a target docking point and a target machine according to the correspondence between the docking point, the machine and the compensation posture.

[0136] In the embodiment of the present application, the correspondence between the docking point, the machine and the compensation posture can be pre-stored in the first composite robot to reduce the information carried by the grasping task and improve the transmission efficiency of the grasping task. The correspondence between the docking point, the machine and the compensation posture can also be carried in the grasping task and sent to the first composite robot to save the storage resources of the first composite robot.

[0137] After the first composite robot receives the grasping task, it can obtain the identification of the target docking point and the identification of the target machine from the grasping task, and then search for the correspondence between the target docking point and the target machine in the correspondence between the docking point, the machine and the compensation posture according to the identification of the target docking point and the identification of the target machine; if found, the compensation posture in the correspondence is used as the target compensation posture.

[0138] In the embodiment of the present application, the execution order of step S72 and step S74 is not limited.

[0139] Step S75: the first compound robot compensates the posture of the first robot arm according to the target compensation posture, so that the first robot arm reaches the second posture.

[0140] The second posture is the posture of the first visual positioning device on the first robot arm collecting the image of the position recognition plate attached to the target machine. The first visual positioning device is a visual positioning device installed on the first robot arm, and the position recognition plate is attached to the target machine. When the first robot arm reaches the first posture, the first composite robot compensates the posture of the first robot arm with the target compensation posture. After compensating the target compensation posture, the posture of the first robot arm is the second posture.

[0141] In step S76 , the first composite robot uses the first visual positioning device and the position recognition plate to position the target machine.

[0142] After the first robot arm reaches the second posture, it indicates that the first robot arm has reached the posture for collecting the image of the position identification plate (such as a Mark plate). The first composite robot controls the first visual positioning device to collect the image of the position identification plate. After collecting the image of the position identification plate, the first composite robot recognizes and analyzes the image to obtain the positioning result of the target machine.

[0143] Step S77: the first composite robot controls the first robotic arm to grab the object on the target platform according to the positioning result.

[0144] According to the positioning result, the first composite robot can guide the first robotic arm to grasp the object on the target machine platform to achieve hand-eye grasping.

[0145] In the technical solution provided by the embodiment of the present application, when the composite robot grasps an object, the composite robot performs a fixed teaching action, that is, when the mechanical arm of the composite robot reaches the first posture, the corresponding relationship between the docking point, the machine and the compensation posture is used to determine the target compensation posture corresponding to the target docking point and the target machine, and then the posture of the mechanical arm is compensated to achieve the grasping of the object on the target machine at the target docking point. In this solution, although a fixed teaching action is still used, each docking point and each machine has a corresponding compensation posture. As long as the distance from multiple machines to a docking point is less than or equal to the operating distance of the mechanical arm, the technical solution provided by the embodiment of the present application can be used at the docking point to grasp the objects on these multiple machines. That is, multiple machines can share the same docking point, which simplifies the layout of the docking points, reduces the difficulty of implementing the solution, and improves the scheduling efficiency of the composite robot.

[0146] In some embodiments, Fig.10 As shown, the interaction process between the scheduling platform 61 and the composite robot 62 is also provided.

[0147] Step S101, the scheduling platform 61 issues a grasping task to the first composite robot in the at least one composite robot 62. The grasping task includes the identification of the target docking point in the topological map and the identification of the target machine, and the target docking point is the docking point of the target machine. This is the same as the above step S71.

[0148] Step S102: the first composite robot is controlled to move to the target docking point according to the grasping task, which is the same as the above step S72.

[0149] Step S103, the first compound robot controls the first mechanical arm of the first compound robot to reach a first posture, wherein the first posture is a posture corresponding to the teaching action, which is the same as the above step S73.

[0150] Step S104, the first composite robot determines the target compensation posture corresponding to the target docking point and the target machine according to the corresponding relationship between the docking point, the machine and the compensation posture, which is the same as the above step S74.

[0151] Step S105: the first compound robot compensates the posture of the first mechanical arm according to the target compensation posture, so that the first mechanical arm reaches the second posture. This is the same as the above step S75.

[0152] Step S106, the first composite robot controls the first visual positioning device to collect a first image of the position identification plate; if the first image is not collected, step S107 is executed; if the first image is collected, step S108 is executed.

[0153] In the embodiment of the present application, after the first robotic arm reaches the second posture, the first compound robot controls the first visual positioning device to capture images and identifies whether the captured image includes a position identification plate; if the position identification plate is not included, it means that the first image is not captured and the position identification plate is not within the field of view of the first visual positioning device, and step S107 is executed; if the position identification plate is included, it means that the position identification plate is within the field of view of the first visual positioning device, and the image is the first image, and step S108 is executed.

[0154] In the embodiment of the present application, the first composite robot may use a convolutional neural network to identify whether the image includes a position identification plate, or may use other algorithms to identify whether the image includes a position identification plate, without limitation.

[0155] Step S107, the first composite robot controls the first mechanical arm to move a preset step length in a preset direction. Then, step S106 is re-executed until the first image is acquired or the distance from the position corresponding to the current posture to the position corresponding to the first posture is greater than the preset distance.

[0156] In the embodiment of the present application, the preset direction may be a clockwise direction on a horizontal plane, such as Fig.11a As shown, Fig.11a The direction indicated by the arrow in the middle is the preset direction, the length of the arrow segment is the preset step length, and the number in the circle indicates the number of moves; the preset direction can also be the counterclockwise direction on the horizontal plane, such as Fig.11b As shown, Fig.11b The direction indicated by the middle arrow is the preset direction, the length of the arrow segment is the preset step length, and the number in the circle indicates the number of moves.

[0157] The preset step length is less than or equal to the width of the field of view of the first visual positioning device. Optionally, the preset step length is half of the width of the field of view of the first visual positioning device to avoid over-adjustment of the position of the robotic arm.

[0158] The position corresponding to the current posture is the position of the first manipulator in the current posture, and the position corresponding to the first posture is the position of the first manipulator in the first posture. Here, the position of the first manipulator can be understood as the position (x, y, z) of the TCP on the first manipulator. The preset distance can be determined according to the operating distance of the first compound robot. For example, in the first posture, the distance from the end of the manipulator (i.e., TCP) to the base of the first manipulator is L1, the operating distance of the first compound robot is L2, and the preset distance is less than L2-L1. In this way, the problem of the first manipulator exceeding the operating distance, resulting in unstable operation of the first compound robot, can be avoided.

[0159] When the first image is not collected, the first compound robot controls the first mechanical arm to move a preset step length in a preset direction, and the first mechanical arm reaches a new posture. In this posture, the first compound robot re-executes step S106 to collect an image.

[0160] If the first image is collected, the first composite robot stops looping through step S106 and proceeds to step S108.

[0161] If the distance from the position corresponding to the current posture to the position corresponding to the first posture is greater than the preset distance, and the first composite robot still has not captured the first image, the first composite robot stops looping step S106, but no longer executes step S108. In addition, in order to ensure the normal execution of the grasping task, if the distance from the position corresponding to the current posture to the position corresponding to the first posture is greater than the preset distance, the first composite robot can output an error message to prompt the user to promptly eliminate the problem of positioning failure.

[0162] In the embodiment of the present application, the first composite robot may be equipped with output devices such as a speaker and a display screen, and the first composite robot directly outputs the error prompt information through its own output device. The first composite robot may also send the error prompt information to the scheduling platform 61, and the scheduling platform 61 displays the error prompt information to realize the output of the error prompt information.

[0163] Step S108: The first composite robot recognizes the first image to obtain a positioning result of the target machine.

[0164] When the first image is collected, the first composite robot recognizes the first image and obtains the pixel position (u, v, θ) of the position recognition plate, such as Fig.12 In the image shown, (u, v) is the pixel coordinate of the position identification plate, and θ is the tilt angle of the position identification plate.

[0165] Based on the conversion relationship between the pixel coordinate system and the first robot arm base coordinate system, the first compound robot can convert the pixel position of the position identification plate to the first robot arm base coordinate system to obtain the position of the position identification plate, and then determine the position of the target machine corresponding to the position of the position identification plate based on the relative position relationship between the target machine and the position identification plate. The position of the target machine is the positioning result of the target machine.

[0166] Step S109: The first composite robot controls the first mechanical arm to grab the object on the target platform according to the positioning result, which is the same as the above step S77.

[0167] In the technical solution provided by the embodiment of the present application, when the compensation posture is used to compensate the posture of the first mechanical arm, if the first composite robot fails to capture the first image once, it will perform secondary image capture, that is, control the first mechanical arm to move a preset step length in a preset direction and then capture the image again to ensure that the first image is captured. This effectively solves the problem that the position recognition plate is not within the field of view of the visual positioning device due to the low chassis positioning and walking accuracy of the composite robot, and improves the efficiency of completing the object grasping task.

[0168] In some embodiments, in order to achieve object grasping, when performing the above Figure 7 or Fig.10 Prior to the scheme shown, the user can control the compound robot 62 to learn the compensation posture. In this case, the above-mentioned object grasping system can also include a host computer 63. The host computer 63 can be integrated into the scheduling platform 61, or it can be deployed in the object grasping system as an independent physical machine. For example, the host computer 63 can be a mobile terminal, a personal computer or a server, etc., which is not limited.

[0169] In the embodiment of the present application, the process of the composite robot 62 learning the compensation posture can be seen in Fig.13 The interaction process between the host computer 63 and the compound robot 62 is shown.

[0170] In step S131 , the host computer 63 obtains the address of the second compound robot in the at least one compound robot 62 through the human-machine interaction interface.

[0171] In the embodiment of the present application, the second compound robot may be any compound robot in the at least one compound robot 62, and the second compound robot may be the same as or different from the first compound robot. The address of the compound robot may be an Internet Protocol (IP) address or a Media Access Control (MAC) address, etc.

[0172] The host computer 63 provides a human-machine interaction interface. The user can input the address of the second compound robot on the human-machine interaction interface. The host computer 63 obtains the address of the second compound robot.

[0173] In step S132, the host computer 63 connects to the second compound robot using the address of the second compound robot.

[0174] After acquiring the address of the second compound robot, the host computer 63 can interact with the second compound robot to establish a communication connection.

[0175] For example Fig.14 In the human-computer interaction interface shown in the figure, the user enters the IP address of the composite robot in the car IP input box, such as Fig.14 In the figure, the user clicks the "Login" button, after which the host computer connects to the composite robot with IP address 1.1.1.1.

[0176] Step S133: After establishing the connection, the host computer 63 obtains the calibration task through the human-machine interaction interface and sends the calibration task to the second compound robot, wherein the calibration task is used to instruct the second compound robot to determine the compensation posture.

[0177] Still Fig.14 For example, after the host computer 63 establishes a connection with the compound robot with the IP address of 1.1.1.1, the user selects the calibration type as the posture compensation type and clicks "Start Calibration". Then the host computer 63 generates a calibration task and sends the calibration task to the compound robot with the IP address of 1.1.1.1.

[0178] In an embodiment of the present application, the calibration task may carry the identification of the docking point and the identification of the machine whose corresponding compensation posture needs to be determined, such as the identification of the target docking point and the identification of the target machine. Subsequently, the second compound robot determines the target compensation posture corresponding to the target docking point and the target machine according to the identification carried by the calibration task.

[0179] The calibration task may not carry the identification of the docking point and the identification of the machine. In this case, the scheduling platform 61 can send a task list to the second compound robot, and the task list includes the identification of the docking point and the identification of the machine for which the corresponding compensation posture needs to be determined, wherein the task list may include the identification of one or more docking points and the identification of one or more machines. Upon receiving the calibration task, the second compound robot can determine that the task list includes each docking point and the compensation posture corresponding to each machine (the target compensation posture is one of the compensation postures).

[0180] If the calibration task does not carry the identification of the docking point and the identification of the machine, the second compound robot can also use the current position of the second compound robot as the target docking point according to the calibration task, and multiple machines corresponding to the target docking point as the target machines, that is, before receiving the calibration task, the second compound robot has moved to the target docking point.

[0181] Step S134: the second compound robot controls the second compound robot to move to the target docking point, and controls the second manipulator arm of the second compound robot to reach the first posture according to the calibration task.

[0182] In the embodiment of the present application, the second mechanical arm is the mechanical arm of the second compound robot. After receiving the calibration task, the second compound robot starts to perform the calibration task, which can be specifically divided into the following two situations:

[0183] Case 1: The calibration task carries the identifier of the target docking point and the identifier of the target machine. The second compound robot determines the target docking point according to the identifier of the target docking point carried by the calibration task, and then controls the second compound robot to move to the target docking point, and controls the second manipulator arm of the second compound robot to reach the first posture.

[0184] Case 2: The calibration task does not carry the identification of the docking point and the identification of the machine. In this case, the second composite robot can implement step S134 in the following two ways to further determine the corresponding compensation posture.

[0185] Method 1: the second compound robot first controls the second compound robot to move to the target docking point; after receiving the calibration task, the second robot arm is controlled to reach the first posture, and subsequent steps S135 to S137 are executed to determine the target compensation posture.

[0186] Method 2: The second compound robot is controlled to move along the route formed by the docking points included in the task list according to the calibration task; each time it moves to a docking point included in the task list, the docking point can be used as the target docking point, and the subsequent steps S135 to S137 are executed to determine the corresponding compensation posture. In this case, the second compound robot determines the compensation postures corresponding to all docking points and machines included in the task list.

[0187] For example Figure 8The docking point layout shown in the figure includes point 11, point 12, point 13 and machine 1 to machine 6 in the task list. After receiving the calibration task, the compound robot 1 moves along the route of point 11, point 12 and point 13. After the compound robot 1 moves to point 11, the manipulator arm is controlled to perform teaching action 1, and steps S135 to S137 are performed to determine the compensation posture 1 corresponding to point 11 and machine 1; the manipulator arm is controlled to perform teaching action 2, and steps S135 to S137 are performed to determine the compensation posture 4 corresponding to point 11 and machine 4. Or, the compound robot 1 controls the compound robot 1 to turn around, controls the manipulator arm to perform teaching action 2, and performs steps S135 to S137 to determine the compensation posture 1' corresponding to point 11 and machine 1; controls the manipulator arm to perform teaching action 1, and performs steps S135 to S137 to determine the compensation posture 4' corresponding to point 11 and machine 4. The collected compensation postures 1 and 1', compensation postures 4 and 4' respectively correspond to the compensation postures under different teaching actions when the composite robot grasps on both sides of the same docking point 11. In this way, the composite robot can grasp on both sides with higher efficiency.

[0188] Afterwards, after the composite robot 1 moves to the point 12, the mechanical arm is controlled to perform the teaching action 1, and steps S135 to S137 are performed to determine the compensation posture 2 corresponding to the point 12 and the machine 2; the mechanical arm is controlled to perform the teaching action 2, and steps S135 to S137 are performed to determine the compensation posture 5 corresponding to the point 12 and the machine 5. Or, the composite robot 1 controls the composite robot 1 to turn around, controls the mechanical arm to perform the teaching action 2, and performs steps S135 to S137 to determine the compensation posture 2' corresponding to the point 11 and the machine 2; controls the composite robot 1 to turn around, controls the mechanical arm to perform the teaching action 1, and performs steps S135 to S137 to determine the compensation posture 5' corresponding to the point 12 and the machine 5. The collected compensation postures 2 and 2', compensation postures 5 and 5' respectively correspond to the compensation postures under different teaching actions when the composite robot grasps on both sides of the same docking point 12. This will enable the composite robot to grasp on both sides, which will be more efficient.

[0189] After that, after the composite robot 1 moves to the point 13, the mechanical arm is controlled to perform the teaching action 1, and steps S135 to S137 are performed to determine the compensation posture 3 corresponding to the point 13 and the machine 3; the mechanical arm is controlled to perform the teaching action 2, and steps S135 to S137 are performed to determine the compensation posture 6 corresponding to the point 13 and the machine 6. Or, the composite robot 1 controls the composite robot 1 to turn around, controls the mechanical arm to perform the teaching action 2, and performs steps S135 to S137 to determine the compensation posture 3' corresponding to the point 11 and the machine 2; controls the composite robot 1 to turn around, controls the mechanical arm to perform the teaching action 1, and performs steps S135 to S137 to determine the compensation posture 6' corresponding to the point 13 and the machine 6. The collected compensation postures 3 and 3', compensation postures 6 and 6' respectively correspond to the compensation postures under different teaching actions when the composite robot grasps on both sides of the same docking point 13. This will enable the composite robot to grasp on both sides, which will be more efficient.

[0190] In the embodiment of the present application, the second composite robot controls the second composite robot to move to the target docking point, and controls the second mechanical arm of the second composite robot to reach the first position, such as Fig.14 The specific implementation method of the robotic arm posture shown can be found in the description of steps S71 to S72 above, which will not be repeated here.

[0191] Step S135, the second composite robot controls the second visual positioning device on the second robotic arm to collect a second image of the position identification plate; if the second image is not collected, step S136 is executed; if the second image is collected, step S137 is executed.

[0192] In the embodiment of the present application, the second visual positioning device is a visual positioning device installed on the second robotic arm. After the second robotic arm reaches the first posture, the second composite robot controls the second visual positioning device to capture an image and identifies whether the captured image includes a position recognition plate; if the position recognition plate is not included, it means that the second image is not captured, and step S136 is executed; if the position recognition plate is included, the image is the second image, and step S137 is executed.

[0193] In the embodiment of the present application, the second composite robot may use a convolutional neural network to identify whether the image includes a position identification plate, or may use other algorithms to identify whether the image includes a position identification plate, without limitation.

[0194] Step S136: the second compound robot controls the second mechanical arm to move a preset step length in a preset direction, and re-executes step S135 until a second image is acquired, or the distance from the position corresponding to the current posture to the position corresponding to the first posture is greater than a preset distance.

[0195] In the embodiment of the present application, the preset direction may be a clockwise direction on a horizontal plane, such as Fig.11a The preset direction can also be counterclockwise on the horizontal plane, such as Fig.11b The preset step length is less than or equal to the width of the field of view of the second visual positioning device, such as Fig.14 The single step distance shown is 10 millimeters (mm). Optionally, the preset step length is half the width of the field of view of the second vision positioning device to avoid over-adjustment of the position of the robot arm.

[0196] The position corresponding to the current posture is the position of the second robot arm in the current posture, and the position corresponding to the first posture is the position of the second robot arm in the first posture. Here, the position of the second robot arm can be understood as the position of the TCP on the second robot arm. The preset distance can be determined according to the working distance of the second composite robot, such as Fig.14 The detection thresholds in the four directions of X+, X-, Y+, and Y- are the preset distances, in mm. The preset distances in different directions may be the same or different.

[0197] When the second image is not collected, the second compound robot controls the second manipulator arm to move a preset step length in a preset direction, and the second manipulator arm reaches a new posture. In this posture, the second compound robot re-executes step S135 to collect an image.

[0198] If the second image is collected, the second composite robot stops looping through step S135 and proceeds to step S137.

[0199] If the distance from the position corresponding to the current posture to the position corresponding to the first posture is greater than the preset distance, and the second composite robot still has not captured the second image, the second composite robot stops looping step S135, but no longer executes step S137. In addition, in order to ensure the normal execution of the grasping task, if the distance from the position corresponding to the current posture to the position corresponding to the first posture is greater than the preset distance, the second composite robot can output an error message to prompt the user to promptly eliminate the problem of positioning failure.

[0200] In the embodiment of the present application, the second composite robot may be equipped with an output device such as a speaker and a display screen, and the second composite robot directly outputs the error prompt information through its own output device. The second composite robot may also send the error prompt information to the scheduling platform 61, and the scheduling platform 61 displays the error prompt information to realize the output of the error prompt information.

[0201] Step S137, the second composite robot determines the third posture of the second robot arm according to the current pixel position of the position recognition plate in the second image and the preset pixel position range, wherein the third posture is the posture of the second robot arm when the current pixel position is within the preset pixel position range.

[0202] In the embodiment of the present application, the preset pixel position range is the central area of ​​the second image, such as Fig.15 The center of the image is a rectangular area of ​​w×h. The values ​​of w and h can be set according to actual needs. When the object (such as the position recognition plate) is located in the preset pixel position range, the image effect is better. The current pixel position of the position recognition plate is the pixel position (u, v, θ) of a certain point (such as the center point) on the position recognition plate at the current moment. Fig.12 shown.

[0203] When the second image is acquired, the second composite robot obtains the current pixel position of the position recognition plate in the second image (eg Fig.14 The camera output shown in FIG. 1 is used to determine the position of the second robot arm when the position recognition plate is located in the preset pixel position range according to the current pixel position and the preset pixel position range, that is, the third position, such as Fig.14 The compensation pose for taking photos is shown.

[0204] For example, when the height h of the second visual positioning device is fixed, the actual movement distance of the camera is f when the position recognition plate moves one pixel in the second image. u and f v , f u 、f v are the focal lengths in the u and v directions respectively, f u 、f v At this time, the position T of the second visual positioning device relative to the position recognition plate MC It can be expressed as:

[0205]

[0206] There is a mapping relationship between the camera coordinate system and the second robot base coordinate system. Using this mapping relationship, T MC And the current pixel position of the position recognition plate can determine the distance that the second robotic arm needs to move, such as the distance moved in the u and v directions respectively, and then determine the third posture of the second robotic arm after moving according to the distance.

[0207] In the embodiment of the present application, the second composite robot can use the mapping relationship between the camera coordinate system and the second robot arm base coordinate system, T MC And the current pixel position of the position recognition board, calculate the third pose, so as to quickly determine the third pose and obtain the compensated pose.

[0208] In order to improve the accuracy of the compensation posture, the second composite robot can actually operate the second mechanical arm to move to determine the third posture, which can be divided into the following two cases:

[0209] Case 1: The current pixel position is within the preset pixel position range. In this case, the second compound robot determines the current posture of the second mechanical arm as the third posture of the mechanical arm.

[0210] Case 2: The current pixel position is not within the preset pixel position range. In this case, the second compound robot determines the adjusted posture of the second manipulator according to the current pixel position and the preset pixel position range, and the pixel position adjustment direction corresponding to the adjusted posture matches the direction in which the current pixel position approaches the preset pixel position range; according to the adjusted posture, the posture of the second manipulator is adjusted, and the step of controlling the second visual positioning device to collect the second image of the position recognition plate is re-executed until the current pixel position is within the preset pixel position range, thereby obtaining the third posture of the manipulator.

[0211] In the embodiment of the present application, if the second composite robot acquires the second image, but the current pixel position of the position recognition plate in the second image is not within the preset pixel position range, in order to ensure that the position recognition plate is located at a better pixel position during actual implementation, the second composite robot can use the mapping relationship between the above-mentioned camera coordinate system and the second robot arm base coordinate system, T MC And the current pixel position of the position recognition plate can determine the distance the second robotic arm needs to move (that is, adjust the posture), and then move the second robotic arm to adjust the posture of the second robotic arm.

[0212] After adjusting the posture, the second composite robot controls the second visual positioning device to continue to collect the second image of the position recognition plate; if the current pixel position of the position recognition plate in the second image is within the preset pixel position range, such as Fig.16 As shown, the adjustment of the posture of the second robotic arm is stopped, and the current posture of the second robotic arm is used as the third posture;

[0213] If the current pixel position of the position recognition plate in the second image is still not within the preset pixel position range, continue to use the mapping relationship between the camera coordinate system and the second robot base coordinate system, T MC And the current pixel position of the position recognition plate can determine the distance the second robotic arm needs to move and adjust the posture of the second robotic arm.

[0214] By adopting the solution provided in the embodiment of the present application, the posture of the second robotic arm is automatically adjusted to ensure that the position recognition plate is located at a better pixel position during actual implementation, without the need for manual intervention, thereby reducing labor costs.

[0215] Step S138: The second compound robot calculates the deviation between the third posture and the first posture to obtain a target compensation posture.

[0216] In the embodiment of the present application, after obtaining the third posture, the second compound robot can determine the target compensation posture ΔT using the following formula.

[0217] ΔT=T1-T2;

[0218] Among them, T1 is the third pose and T2 is the first pose.

[0219] After obtaining the target compensation posture, the second compound robot can directly synchronize the correspondence between the target docking point, the target machine and the target compensation posture to other compound robots, or send it to the scheduling platform 61. When the scheduling platform 61 schedules other compound robots (such as the first compound robot) to perform a grasping task, the correspondence between the target docking point, the target machine and the target compensation posture is carried in the grasping task and sent to the first compound robot, so that the first compound robot uses the correspondence to perform posture compensation on the first robot arm and accurately grasp the object.

[0220] In the technical solution provided by the embodiment of the present application, the second composite robot automatically detects the position recognition plate according to the preset direction and the preset step length, without manual intervention, thereby improving the efficiency of acquiring the image of the position recognition plate, and thereby improving the efficiency of determining the target compensation posture. In addition, the third posture determined by the second composite robot is a posture that makes the current pixel position of the position recognition plate within the preset pixel position range, and this posture is used to determine the target compensation posture, thereby ensuring the image effect of the position recognition plate in subsequent applications, improving the accuracy of machine positioning, and thereby improving the accuracy of object grasping.

[0221] In some embodiments, in the above step S135, after the second robotic arm reaches the first posture, the second compound robot controls the second visual positioning device to capture images, but fails to capture the second image. The second compound robot may input a prompt message, and the prompt message is used to indicate that the second image has not been captured, so that the user can quickly locate the reason why the second image has not been captured, and manually guide the movement of the second robotic arm to quickly determine the third posture.

[0222] Corresponding to the above-mentioned object grasping system, the embodiment of the present application also provides an object grasping method, such as Fig.17 As shown, the following steps are included:

[0223] Step S171, controlling the composite robot to move to a target docking point in the topological map, where the target docking point is a docking point of a target machine;

[0224] Step S172, controlling the manipulator arm of the composite robot to reach a first posture, where the first posture is a posture corresponding to the teaching action;

[0225] Step S173, determining a target compensation posture corresponding to a target docking point and a target machine according to the correspondence between the docking point, the machine and the compensation posture;

[0226] Step S174, compensating the posture of the robot arm according to the target compensation posture so that the robot arm reaches a second posture, where the second posture is the posture of the image of the position recognition plate fitted on the target machine platform collected by the visual positioning device on the robot arm;

[0227] Step S175, using a visual positioning device and a position recognition plate to locate the target machine;

[0228] Step S176, controlling the robot arm to grab the object on the target machine according to the positioning result.

[0229] In the technical solution provided by the embodiment of the present application, when the composite robot grasps an object, the composite robot performs a fixed teaching action, that is, when the mechanical arm of the composite robot reaches the first posture, the corresponding relationship between the docking point, the machine and the compensation posture is used to determine the target compensation posture corresponding to the target docking point and the target machine, and then the posture of the mechanical arm is compensated to achieve the grasping of the object on the target machine at the target docking point. In this solution, although a fixed teaching action is still used, each docking point and each machine has a corresponding compensation posture. As long as the distance from multiple machines to a docking point is less than or equal to the operating distance of the mechanical arm, the technical solution provided by the embodiment of the present application can be used at the docking point to grasp the objects on these multiple machines. That is, multiple machines can share the same docking point, which simplifies the layout of the docking points, reduces the difficulty of implementing the solution, and improves the scheduling efficiency of the composite robot.

[0230] In an embodiment of the present application, the above-mentioned object grasping method can be applied to an electronic device, which may be a composite robot (i.e., a processor of the composite robot) or a device connected to the composite robot (such as a scheduling platform or other device, etc.).

[0231] In some embodiments, the above step S175 may include:

[0232] Controlling the visual positioning device to collect a first image of the position identification plate;

[0233] If the first image is not acquired, the robot arm is controlled to move a preset step length in a preset direction, and the step of controlling the visual positioning device to acquire the first image of the position recognition plate is re-executed until the first image is acquired, or the distance from the position corresponding to the current posture to the position corresponding to the first posture is greater than the preset distance;

[0234] If the first image is acquired, the first image is recognized to obtain a positioning result of the target machine.

[0235] In some embodiments, before controlling the composite robot to move to the target docking point in the topological map, Fig.18 As shown, the object grasping method may further include the following steps:

[0236] Step S181, controlling the composite robot to move to the target docking point, and controlling the robotic arm to reach the first posture;

[0237] Step S182, controlling the visual positioning device to collect a second image of the position recognition plate;

[0238] Step S183, if the second image is acquired, determining the third posture of the robot arm according to the current pixel position of the position recognition plate in the second image and the preset pixel position range, the third posture being the posture of the robot arm when the current pixel position is within the preset pixel position range;

[0239] Step S184, calculating the deviation between the third posture and the first posture to obtain the target compensation posture.

[0240] In some embodiments, in the above step S183, the step of determining the third posture of the robot arm according to the current pixel position of the position recognition plate in the second image and the preset pixel position range may include:

[0241] If the current pixel position is within the preset pixel position range, the current posture of the robotic arm is determined to be the third posture of the robotic arm.

[0242] In some embodiments, in the above step S183, the step of determining the third posture of the robot arm according to the current pixel position of the position recognition plate in the second image and the preset pixel position range may include:

[0243] If the current pixel position is not within the preset pixel position range, the adjustment posture of the robot arm is determined according to the current pixel position and the preset pixel position range, and the pixel position adjustment direction corresponding to the adjustment posture matches the direction in which the current pixel position approaches the preset pixel position range;

[0244] According to the adjusted posture, the posture of the robot arm is adjusted, and the step of controlling the visual positioning device to collect the second image of the position recognition plate is re-executed until the current pixel position is within the preset pixel position range, thereby obtaining the third posture of the robot arm.

[0245] In some embodiments, the object grasping method may further include:

[0246] If the second image is not collected, the robotic arm is controlled to move a preset step length in a preset direction, and the step of controlling the visual positioning device to collect the second image of the position identification plate is re-executed until the second image is collected, or the distance from the position corresponding to the current posture to the position corresponding to the first posture is greater than the preset distance.

[0247] In some embodiments, the preset direction is a clockwise direction or a counterclockwise direction on a horizontal plane, and the preset step length is less than or equal to a width of a field of view of the visual positioning device.

[0248] In some embodiments, the object grasping method may further include:

[0249] If the distance from the position of the robot arm to the position corresponding to the first posture is greater than the preset distance and the image of the position recognition plate is not collected, an error prompt message is output.

[0250] In some embodiments, the object grasping method may further include:

[0251] The corresponding relationship between the target docking point, the target machine and the target compensation posture is synchronized to other composite robots; or the corresponding relationship between the target docking point, the target machine and the target compensation posture is sent to the scheduling platform of the composite robot.

[0252] In some embodiments, a docking point in the topological map corresponds to at least one machine, and the distance from at least one machine to a docking point is less than or equal to the operating distance of the robot arm.

[0253] In the technical solution of this application, the operations involved in obtaining, storing, using, processing, transmitting, providing and disclosing user personal information are all carried out with the user's authorization.

[0254] In conjunction with the following embodiments, the object grasping method provided in the embodiment of the present application is described in detail. In the embodiment of the present application, a topological map is pre-planned according to the operating range of the robot arm, and multiple machines can share a docking point. The following is an example of a visual positioning device with a camera and a position recognition plate Mark board.

[0255] Compensation posture calibration stage:

[0256] In step a, the user completes the action teaching of the robot arm and determines a docking point of a machine platform, such as a target docking point, and instructs the composite robot to move to the target docking point.

[0257] In step b, the user connects to the composite robot through the human-machine interaction interface provided by the scheduling platform, and after the connection, sends a command (such as the calibration task mentioned above) to the composite robot to make the composite robot enter the debugging mode.

[0258] Step c: The user selects the calibration type as "posture compensation" through the human-computer interaction interface provided by the scheduling platform, and clicks "Start calibration". At this time, the manipulator arm of the composite robot performs the teaching action, that is, the manipulator arm moves to the photo taking position corresponding to the target machine during teaching, and then the camera on the manipulator arm takes a photo. Among them, the composite robot records the manipulator arm posture at this time (that is, the first posture).

[0259] If the image obtained by the camera during a photo shoot includes the Mark plate, that is, the Mark plate is within the field of view of the camera, execute step d; if the image obtained by the camera during a photo shoot does not include the Mark plate, that is, the Mark plate is not within the field of view of the camera, execute step e.

[0260] Step d: The camera outputs the pixel position (u, v, θ) of the Mark plate. The composite robot automatically adjusts the action of the robotic arm according to the output of the camera, that is, adjusts the position and posture of the robotic arm until the (u, v, θ) output by the camera meets the preset conditions. The composite robot records the position and posture of the robotic arm at this time (that is, the third position and posture).

[0261] The (u, v, θ) output by the camera satisfies the preset condition that the (u, v, θ) output by the camera is within the preset pixel position range. The method of adjusting the position and posture of the robot arm can refer to the relevant description of step S137.

[0262] In step e, the composite robot uses detection to adjust the Mark plate to the camera's field of view, and then executes the process in step d to automatically adjust the action of the robotic arm until the (u, v, θ) output by the camera meets the preset conditions. The composite robot records the position of the robotic arm at this time (i.e., the third position).

[0263] The detection method can refer to the description of step S136, for example, the composite robot can select one of the four directions (X+, X-, Y+, Y-) of the two-dimensional plane. Fig.11b In the X+ direction shown, follow Fig.11b The robot arm is controlled to move counterclockwise as shown. Each time the robot arm is controlled to move a preset step length, the camera takes a picture and identifies whether the Mark board is within the camera's field of view. In the embodiment of the present application, the human-computer interaction interface provided by the scheduling platform can also display the image obtained by the camera taking a picture once, such as Fig.14 The camera image in .

[0264] If the composite robot completes all detection actions, that is, the distance moved by the robotic arm exceeds the detection threshold, but the MarK board is still not within the field of view of the camera, the composite robot outputs an error prompt and the compensation posture calibration fails.

[0265] In step f, the composite robot calculates the difference between the postures recorded in step c and step d (or the difference between the postures recorded in step c and step e), and obtains the compensation posture of the manipulator in the two-dimensional plane (i.e., the target compensation posture).

[0266] The composite robot can write calculation-related parameters into the configuration file. The calculation-related parameters may include, but are not limited to, the target docking point, the target machine, the target compensation posture, the preset step length, the detection threshold, the detection direction (such as the counterclockwise direction mentioned above), etc. Since the compensation posture is associated with the docking point and the machine, it has nothing to do with the action performed by the composite robot. Therefore, after a single composite robot is calibrated to obtain the compensation posture, the configuration file can be synchronized to other composite robots. Other composite robots directly use this set of parameters without recalibrating the compensation posture, thereby improving the efficiency of task execution.

[0267] In addition, the above-mentioned posture compensation calibration process is all automated, which simplifies the implementation operation to a certain extent and improves the implementation efficiency. Different types of composite robots have different operating ranges. In order to accurately achieve posture compensation, a composite robot of each type can execute the above-mentioned compensation posture calibration process respectively, determine the configuration file corresponding to each type of composite robot, and achieve accurate posture compensation for each type of composite robot.

[0268] Compensation posture application stage:

[0269] In step a, the composite robot moves to the target docking point, and based on the target compensation posture, the robotic arm reaches the photographing posture (i.e., the second posture).

[0270] Due to positioning deviation, the composite robot may not move accurately to the target docking point, resulting in the Mark plate not being within the camera's field of view, the camera output is abnormal, and then step b is executed. If the camera output is normal (that is, the Mark plate is within the camera's field of view), skip step b and go directly to step c.

[0271] Step b: The robot uses detection to adjust the Mark plate to the camera's field of view. For details, please refer to the description of step e in the calibration phase, that is:

[0272] The composite robot can choose any one of the four directions (X+, X-, Y+, Y-) in the two-dimensional plane. Fig.11b In the X+ direction shown, follow Fig.11b The robot arm is controlled to move in the counterclockwise direction as shown. Each time the robot arm is controlled to move a preset step length, the camera takes a picture and identifies whether the Mark board is within the camera's field of view.

[0273] Step c: When the Mark plate is adjusted to the field of view of the camera and the camera output is normal, stop detecting; and perform hand-eye positioning based on the camera output (x, y, θ) at this time to guide the robotic arm to grasp the object.

[0274] Combination Figure 8 For example, the composite robot is located at point 10, and the object grasping tasks include task 1 of grasping the object on platform 2 and task 2 of grasping the object on platform 5. The composite robot first moves along Fig.19 The track indicated by the dotted line moves to the docking point 12. After that, the composite robot uses the manipulator to perform the teaching action and performs the corresponding posture compensation to complete task 1; the composite robot turns around, uses the manipulator to perform the teaching action, and performs the corresponding posture compensation to complete task 2.

[0275] Or, if Fig. 20 The composite robot shown uses the robot arm 201 to perform teaching action 1 and performs corresponding posture compensation to complete task 1; at the same time, the composite robot uses the robot arm 202 to perform teaching action 2 and performs corresponding posture compensation to complete task 2.

[0276] The specific way in which the composite robot grasps an object can be implemented by the scheduling platform according to the structural scheduling of the composite robot, and there is no limitation on this.

[0277] In the embodiment of the present application, the calibration scheme of the compensation posture realizes the unbinding of the point layout in the topological map and the machine, reduces the docking points, simplifies the layout of the docking points, reduces the difficulty of implementing the on-site plan, achieves the purpose of rapid project deployment and acceptance, and improves the overall scheduling efficiency. In addition, when grabbing an object, if the first photo fails (that is, the Mark plate is not within the field of view of the camera), the composite robot will use detection to control the mechanical arm to take photos again until the Mark plate is within the field of view of the camera, thereby overcoming the positioning deviation problem of the composite robot, reducing the on-site failure rate, improving the overall operation stability of the composite robot, and reducing the workload of project maintenance.

[0278] Corresponding to the above-mentioned object grasping method, the embodiment of the present application also provides an object grasping device, such as Fig.21 As shown, the device comprises:

[0279] The first control module 211 is used to control the composite robot to move to a target docking point in the topological map, where the target docking point is a docking point of a target machine; and control the manipulator arm of the composite robot to reach a first posture, where the first posture is a posture corresponding to the teaching action;

[0280] A first determination module 212 is used to determine a target compensation posture corresponding to a target docking point and a target machine according to the correspondence between the docking point, the machine and the compensation posture;

[0281] The compensation module 213 is used to compensate the posture of the robot arm according to the target compensation posture, so that the robot arm reaches a second posture, and the second posture is the posture of the image of the position recognition plate attached to the target machine platform collected by the visual positioning device on the robot arm;

[0282] A positioning module 214 is used to locate the target machine using a visual positioning device and a position recognition plate;

[0283] The grasping module 215 is used to control the robot arm to grasp the object on the target platform according to the positioning result.

[0284] In the technical solution provided by the embodiment of the present application, when the composite robot grasps an object, the composite robot performs a fixed teaching action, that is, when the robot arm reaches the first posture, the corresponding relationship between the docking point, the machine and the compensation posture is used to determine the target compensation posture corresponding to the target docking point and the target machine, and then the posture of the robot arm is compensated to achieve the grasping of the object on the target machine at the target docking point. In this solution, although a fixed teaching action is still used, each docking point and each machine has a corresponding compensation posture. As long as the distance from multiple machines to a docking point is less than or equal to the operating distance of the robot arm, the technical solution provided by the embodiment of the present application can be used at the docking point to grasp the objects on these multiple machines. That is, multiple machines can share the same docking point, which simplifies the layout of the docking points, reduces the difficulty of implementing the solution, and improves the scheduling efficiency of the composite robot.

[0285] In some embodiments, the positioning module 214 can be specifically used to: control the visual positioning device to collect the first image of the position identification plate; if the first image is not collected, control the robotic arm to move a preset step length in a preset direction, and re-execute the step of controlling the visual positioning device to collect the first image of the position identification plate until the first image is collected, or the distance from the position corresponding to the current posture to the position corresponding to the first posture is greater than the preset distance; if the first image is collected, identify the first image to obtain the positioning result of the target machine.

[0286] In some embodiments, the object grasping device may further include:

[0287] The second control module is used to control the composite robot to move to the target docking point and control the manipulator to reach the first posture before controlling the composite robot to move to the target docking point in the topological map;

[0288] An acquisition module, used for controlling the visual positioning device to acquire a second image of the position recognition plate;

[0289] A second determination module is used to determine a third posture of the robotic arm according to a current pixel position of the position recognition plate and a preset pixel position range in the second image if a second image is acquired, the third posture being the posture of the robotic arm when the current pixel position is within the preset pixel position range;

[0290] The calculation module is used to calculate the deviation between the third posture and the first posture to obtain the target compensation posture.

[0291] In some embodiments, the second determination module may be specifically used to: if the current pixel position is within a preset pixel position range, determine that the current posture of the robotic arm is the third posture of the robotic arm.

[0292] In some embodiments, the second determination module can be specifically used for: if the current pixel position is not within the preset pixel position range, then determining the adjusted posture of the robot arm based on the current pixel position and the preset pixel position range, and the pixel position adjustment direction corresponding to the adjusted posture matches the direction in which the current pixel position approaches the preset pixel position range; adjusting the posture of the robot arm according to the adjusted posture, and re-executing the step of controlling the visual positioning device to collect the second image of the position identification plate until the current pixel position is within the preset pixel position range, thereby obtaining the third posture of the robot arm.

[0293] In some embodiments, the acquisition module can also be used to: if the second image is not acquired, control the robotic arm to move a preset step length in a preset direction, and re-execute the step of controlling the visual positioning device to acquire the second image of the position identification plate until the second image is acquired, or the distance from the position corresponding to the current posture to the position corresponding to the first posture is greater than the preset distance.

[0294] In some embodiments, the preset direction is a clockwise direction or a counterclockwise direction on a horizontal plane, and the preset step length is less than or equal to a width of a field of view of the visual positioning device.

[0295] In some embodiments, the object grasping device may further include:

[0296] The output module is used to output an error prompt message if the image of the position recognition plate is not collected when the distance from the position of the robot arm to the position corresponding to the first posture is greater than a preset distance.

[0297] In some embodiments, the object grasping device may further include:

[0298] The synchronization module is used to synchronize the correspondence between the target docking point, the target machine and the target compensation posture to other composite robots; or, to send the correspondence between the target docking point, the target machine and the target compensation posture to the scheduling platform of the composite robot.

[0299] In some embodiments, a docking point in the topological map corresponds to at least one machine, and the distance from at least one machine to a docking point is less than or equal to the operating distance of the robot arm.

[0300] Corresponding to the above-mentioned object grasping method, the embodiment of the present application also provides an electronic device, such as Fig. 22 As shown, including:

[0301] Memory 221, used for storing computer programs;

[0302] The processor 222 is used to implement any of the above-mentioned object grasping methods when executing the program stored in the memory 221.

[0303] Furthermore, the electronic device may further include a communication bus and / or a communication interface, and the processor 222, the communication interface, and the memory 221 communicate with each other via the communication bus.

[0304] The communication bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0305] The communication interface is used for communication between the above electronic device and other devices.

[0306] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0307] The processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components.

[0308] In another embodiment provided in the present application, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, any of the above-mentioned object grasping methods is implemented.

[0309] In another embodiment provided by the present application, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute any object grasping method in the above embodiments.

[0310] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a solid-state hard disk (SSD), etc.

[0311] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0312] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the method, device, electronic device, storage medium and program product embodiments, since they are basically similar to the system embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the system embodiment.

[0313] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application are included in the protection scope of the present application.

Claims

1. A method for grasping an object, characterized in that: The method comprises: Control the composite robot to move to a target docking point in a topological map, wherein the target docking point is a docking point of a target machine, wherein one docking point in the topological map corresponds to at least one machine, and when one docking point corresponds to multiple machines, the multiple machines share one docking point, and the distance from the at least one machine to the one docking point is less than or equal to the operating distance of the robot arm; Controlling the mechanical arm of the composite robot to reach a first posture, wherein the first posture is a posture corresponding to the teaching action; Determine the target compensation posture corresponding to the target docking point and the target machine according to the corresponding relationship between the docking point, the machine and the compensation posture; According to the target compensation posture, the posture of the robot arm is compensated so that the robot arm reaches a second posture, where the second posture is the posture of the image of the position recognition plate attached to the target machine platform collected by the visual positioning device on the robot arm; Positioning the target machine using the visual positioning device and the position identification plate; According to the positioning result, the robot arm is controlled to grab the object on the target platform.

2. The method according to claim 1, characterized in that The step of positioning the target machine using the visual positioning device and the position identification plate includes: Controlling the visual positioning device to collect a first image of the position identification plate; If the first image is not acquired, the robotic arm is controlled to move a preset step length in a preset direction, and the step of controlling the visual positioning device to acquire the first image of the position recognition plate is re-executed until the first image is acquired, or the distance from the position corresponding to the current posture to the position corresponding to the first posture is greater than a preset distance; If the first image is acquired, the first image is recognized to obtain a positioning result of the target machine.

3. The method according to claim 1, characterized in that Before controlling the composite robot to move to the target docking point in the topological map, the method further includes: Controlling the composite robot to move to the target docking point, and controlling the robotic arm to reach the first posture; Controlling the visual positioning device to collect a second image of the position identification plate; If the second image is acquired, determining a third posture of the robotic arm according to the current pixel position of the position identification plate in the second image and a preset pixel position range, the third posture being the posture of the robotic arm when the current pixel position is within the preset pixel position range; The deviation between the third posture and the first posture is calculated to obtain the target compensated posture.

4. The method according to claim 3, characterized in that The step of determining the third posture of the robotic arm according to the current pixel position of the position recognition plate in the second image and the preset pixel position range comprises: If the current pixel position is within the preset pixel position range, the current posture of the robotic arm is determined to be the third posture of the robotic arm.

5. The method according to claim 3, characterized in that: The step of determining the third posture of the robotic arm according to the current pixel position of the position recognition plate in the second image and the preset pixel position range comprises: If the current pixel position is not within the preset pixel position range, determining the adjusted posture of the robotic arm according to the current pixel position and the preset pixel position range, wherein the pixel position adjustment direction corresponding to the adjusted posture matches the direction in which the current pixel position approaches the preset pixel position range; According to the posture adjustment, the posture of the robotic arm is adjusted, and the step of controlling the visual positioning device to collect the second image of the position identification plate is re-executed until the current pixel position is within the preset pixel position range, thereby obtaining the third posture of the robotic arm.

6. The method according to claim 3, characterized in that: The method further comprises: If the second image is not collected, the robotic arm is controlled to move a preset step length in a preset direction, and the step of controlling the visual positioning device to collect the second image of the position identification plate is re-executed until the second image is collected, or the distance from the position corresponding to the current posture to the position corresponding to the first posture is greater than the preset distance.

7. The method according to claim 2 or 6, characterized in that: The preset direction is a clockwise direction or a counterclockwise direction on a horizontal plane, and the preset step length is less than or equal to a width of a field of view of the visual positioning device.

8. The method according to claim 2 or 6, characterized in that: The method further comprises: If the distance from the position of the robot arm to the position corresponding to the first posture is greater than the preset distance and the image of the position recognition plate is not collected, an error prompt message is output.

9. The method according to any one of claims 3 to 6, characterized in that: The method further comprises: The corresponding relationship between the target docking point, the target machine and the target compensation posture is synchronized to other compound robots; or, the corresponding relationship between the target docking point, the target machine and the target compensation posture is sent to the scheduling platform of the compound robot.

10. An object grasping system, characterized in that: The system includes a scheduling platform and at least one composite robot; The scheduling platform is used to issue a grasping task to a first composite robot among the at least one composite robot, wherein the grasping task includes an identifier of a target docking point and an identifier of a target machine in a topological map, wherein the target docking point is a docking point of a target machine, wherein one docking point in the topological map corresponds to at least one machine, and when one docking point corresponds to multiple machines, the multiple machines share one docking point, and a distance from the at least one machine to the one docking point is less than or equal to an operating distance of the robotic arm; The first compound robot is used to control the first compound robot to move to the target docking point according to the grasping task; control the first manipulator arm of the first compound robot to reach a first posture, the first posture being the posture corresponding to the teaching action; determine the target compensation posture corresponding to the target docking point and the target machine according to the correspondence between the docking point, the machine and the compensation posture; compensate the posture of the first manipulator arm according to the target compensation posture so that the first manipulator arm reaches a second posture, the second posture being the posture of the image of the position identification plate fitted on the target machine collected by the first visual positioning device on the first manipulator arm; use the first visual positioning device and the position identification plate to locate the target machine; and control the first manipulator arm to grab the object on the target machine according to the positioning result.

11. The system according to claim 10, characterized in that The first composite robot is specifically used for: Controlling the first visual positioning device to collect a first image of the position identification plate; If the first image is not acquired, the first robotic arm is controlled to move a preset step length in a preset direction, and the step of controlling the first visual positioning device to acquire the first image of the position recognition plate is re-executed until the first image is acquired, or the distance from the position corresponding to the current posture to the position corresponding to the first posture is greater than a preset distance; If the first image is acquired, the first image is recognized to obtain a positioning result of the target machine.

12. The system according to claim 10, characterized in that The object grasping system also includes a host computer; The host computer is used to obtain the address of the second compound robot in the at least one compound robot through the human-computer interaction interface; connect to the second compound robot using the address; after establishing the connection, obtain the calibration task through the human-computer interaction interface, and issue the calibration task to the second compound robot; The second compound robot is used to control the second compound robot to move to the target docking point, and control the second mechanical arm of the second compound robot to reach the first posture according to the calibration task; control the second visual positioning device on the second mechanical arm to collect the second image of the position recognition plate; if the second image is collected, determine the third posture of the second mechanical arm according to the current pixel position of the position recognition plate in the second image and the preset pixel position range, the third posture being the posture of the second mechanical arm when the current pixel position is within the preset pixel position range; The deviation between the third posture and the first posture is calculated to obtain the target compensated posture.

13. The system according to claim 12, characterized in that The second compound robot is also used to control the second robotic arm to move a preset step length in a preset direction if the second image is not collected, and re-execute the step of controlling the second visual positioning device on the second robotic arm to collect the second image of the position identification plate until the second image is collected, or the distance from the position corresponding to the current posture to the position corresponding to the first posture is greater than the preset distance.

14. The system according to claim 12 or 13, characterized in that The second compound robot is further used to synchronize the corresponding relationship between the target docking point, the target platform and the target compensation posture to other compound robots in the at least one compound robot; or The second composite robot is further used to send the corresponding relationship between the target docking point, the target machine and the target compensation posture to the scheduling platform; the grasping task also includes the corresponding relationship between the target docking point, the target machine and the target compensation posture.

15. An object grasping device, characterized in that: The device comprises: A first control module is used to control the composite robot to move to a target docking point in a topological map, where the target docking point is a docking point of a target machine; control the manipulator arm of the composite robot to reach a first posture, where the first posture is a posture corresponding to a teaching action, where one docking point in the topological map corresponds to at least one machine, and when one docking point corresponds to multiple machines, the multiple machines share one docking point, and the distance from the at least one machine to the one docking point is less than or equal to the operating distance of the manipulator arm; A first determination module is used to determine the target compensation posture corresponding to the target docking point and the target machine according to the correspondence between the docking point, the machine and the compensation posture; A compensation module, used for compensating the posture of the robot arm according to the target compensation posture, so that the robot arm reaches a second posture, where the second posture is the posture of the image of the position identification plate attached to the target machine platform collected by the visual positioning device on the robot arm; A positioning module, used to locate the target machine using the visual positioning device and the position recognition plate; The grasping module is used to control the robot arm to grasp the object on the target platform according to the positioning result.

16. The device according to claim 15, characterized in that The positioning module is specifically used to: control the visual positioning device to collect the first image of the position recognition plate; if the first image is not collected, control the mechanical arm to move a preset step length in a preset direction, and re-execute the step of controlling the visual positioning device to collect the first image of the position recognition plate until the first image is collected, or the distance from the position corresponding to the current posture to the position corresponding to the first posture is greater than a preset distance; If the first image is acquired, the first image is recognized to obtain a positioning result of the target machine; or, The device also includes: A second control module is used to control the composite robot to move to the target docking point in the topological map and control the robot arm to reach the first posture before controlling the composite robot to move to the target docking point in the topological map; An acquisition module, used for controlling the visual positioning device to acquire a second image of the position identification plate; a second determination module, configured to determine, if the second image is acquired, a third posture of the robotic arm according to a current pixel position of the position recognition plate and a preset pixel position range in the second image, wherein the third posture is a posture of the robotic arm when the current pixel position is within the preset pixel position range; a calculation module, used for calculating the deviation between the third posture and the first posture to obtain the target compensation posture; or The second determination module is specifically configured to: if the current pixel position is within a preset pixel position range, determine that the current posture of the robotic arm is the third posture of the robotic arm; or, The second determination module is specifically used to: if the current pixel position is not within the preset pixel position range, determine the adjusted posture of the robot arm according to the current pixel position and the preset pixel position range, and the pixel position adjustment direction corresponding to the adjusted posture matches the direction in which the current pixel position approaches the preset pixel position range; adjust the posture of the robot arm according to the adjusted posture, and re-execute the step of controlling the visual positioning device to collect the second image of the position identification plate until the current pixel position is within the preset pixel position range, so as to obtain the third posture of the robot arm; or, The acquisition module is further used for: if the second image is not acquired, controlling the robot arm to move a preset step length in a preset direction, and re-executing the step of controlling the visual positioning device to acquire the second image of the position identification plate, until the second image is acquired, or the distance from the position corresponding to the current posture to the position corresponding to the first posture is greater than a preset distance; or, The preset direction is a clockwise direction or a counterclockwise direction on a horizontal plane, and the preset step length is less than or equal to the width of the field of view of the visual positioning device; or, The device also includes: an output module, configured to output an error message if the image of the position recognition plate is not acquired when the distance from the position of the robot arm to the position corresponding to the first posture is greater than the preset distance; or The device also includes: A synchronization module is used to synchronize the corresponding relationship between the target docking point, the target machine and the target compensation posture to other composite robots; or to send the corresponding relationship between the target docking point, the target machine and the target compensation posture to the scheduling platform of the composite robot.

17. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, for implementing any of the methods described in claims 1-9 when executing a program stored in a memory.

18. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images