Multi-mechanical-arm combined lapping control method and system

Abstract

The invention provides a multi-mechanical-arm combined net laying control method and system.The method comprises the steps that the anchor hole position of a previous net piece is obtained through a machine vision system, and the installation position of a current to-be-installed net piece is determined based on the anchor hole position of the previous net piece; on the basis of the installation position of the current to-be-installed mesh, the grabbing positions of the tail ends of a plurality of mechanical arms when the net laying task is completed are determined; the grabbing positions of the tail ends of the multiple mechanical arms serve as task targets to be distributed to multiple target mechanical arms in the multiple mechanical arms; and for each target mechanical arm, an expected posture is generated based on the grabbing position of the tail end of the mechanical arm corresponding to the target mechanical arm, the expected posture is input into the target mechanical arm, and the target mechanical arm is controlled to complete the net laying operation, so that the operation efficiency is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of fully mechanized coal face netting, and in particular to a multi-robot arm combined netting control method and system. BACKGROUND

[0002] In related technologies, anchor rod jumbo is often used for rapid anchor rod support in the roadway. In the anchor rod support process, the netting process is essential. Although the drilling and fastening actions of the current anchor rod jumbo have realized electro-hydraulic control automatic operation, in the entire process of roadway netting, the workers usually manually operate to fix the netting on the top and side of the roadway to achieve the support of the roadway, which leads to the problems of high labor cost, low operation efficiency, and poor operation safety in the traditional roadway netting operation.

[0003] Although the existing single-robot arm assisted netting system can reduce manual operation and make the transportation process automatic, its operation efficiency is still not high, especially for large-size netting operation, which has limitations and cannot be separated from manual intervention. SUMMARY

[0004] The present application provides a multi-robot arm combined netting control method and system to at least partially solve one of the technical problems in related technologies. The technical solution of the present disclosure is as follows.

[0005] In a first aspect, the present application provides a multi-robot arm combined netting control method, comprising: obtaining the anchor hole positions of the previous netting through a machine vision system, and determining the installation position of the current netting to be installed based on the anchor hole positions of the previous netting; determining the end-of-arm grasping positions of multiple robots when the netting task is completed based on the installation position of the current netting to be installed; distributing the end-of-arm grasping positions of the multiple robots to multiple target robots in the multiple robots as task targets; for each target robot, generating a desired pose based on the end-of-arm grasping position corresponding to the target robot and inputting the desired pose into the target robot to control the target robot to complete the netting operation.

[0006] In some implementations, the determination of the installation position of the current netting to be installed based on the anchor hole positions of the previous netting comprises: determining the target anchor hole position closest to the current netting to be installed from the anchor hole positions of the previous netting; determining the installation position of the current netting to be installed based on the target anchor hole position and the fixed distance between the anchor hole position and the edge of the netting.

[0007] In some implementations, the method further comprises: generating a three-dimensional model of the current net sheet based on the installation position of the current net sheet; determining the plurality of end-of-arm grasping positions of the plurality of robots in the three-dimensional model.

[0008] In some implementations, the method further comprises: allocating a group of end-of-arm grasping positions belonging to the same direction and parallel to each other in the plurality of end-of-arm grasping positions as a same task target to a group of target robots in the same direction; wherein the group of target robots comprises a leading robot and a following robot, and the following robot receives a desired pose of the leading robot and follows the leading robot to change the pose.

[0009] In some implementations, the method further comprises: controlling the robots to complete the net laying task through an event-triggered mechanism for a plurality of groups of target robots with different task targets; and a condition of the event-triggered mechanism is that an error between desired poses of the plurality of groups of target robots is greater than an error threshold.

[0010] In some implementations, the method further comprises: evaluating whether there is a collision risk based on distances between a plurality of groups of target robots with different task targets; if there is a collision risk, excluding a group of robots with the collision risk from the plurality of robots, and determining remaining robots; allocating the plurality of end-of-arm grasping positions as task targets to a plurality of target robots in the remaining robots.

[0011] In a second aspect, an embodiment of the present application proposes a multi-robot combined net laying control system, comprising: an image processing module configured to acquire anchor hole positions of a previous net sheet through a machine vision system, and determine an installation position of a current net sheet based on the anchor hole positions of the previous net sheet; a position acquisition module configured to determine a plurality of end-of-arm grasping positions of a plurality of robots in completing a net laying task based on the installation position of the current net sheet; a task allocation module configured to allocate the plurality of end-of-arm grasping positions as task targets to a plurality of target robots in the plurality of robots. The operation control module is configured to, for each target robot arm, generate a desired pose based on the robot end gripping position corresponding to the target robot arm and input the desired pose into the target robot arm, and control the target robot arm to complete the web laying operation.

[0012] In some implementations, the image processing module is specifically configured to: determine a target anchor hole position closest to the current web to be installed from the anchor hole positions of the last web; determine the installation position of the current web to be installed based on the target anchor hole position and a fixed distance between the anchor hole position and the web edge.

[0013] In some implementations, the position acquisition module is specifically configured to: generate a three-dimensional model of the current web to be installed based on the installation position of the current web to be installed; determine a plurality of robot end gripping positions when the web laying task is completed in the three-dimensional model.

[0014] In some implementations, the task allocation module is specifically configured to: allocate a group of robot end gripping positions belonging to the same direction and parallel to each other in the plurality of robot end gripping positions as a same task target to a group of target robot arms in the same direction; wherein the group of target robot arms includes a leading robot arm and a following robot arm, and the following robot arm receives a desired pose of the leading robot arm and follows the leading robot arm to change the pose.

[0015] In some implementations, the operation control module is further configured to: control the robot arms to complete the web laying operation through an event triggering mechanism for a plurality of groups of target robot arms with different task targets; and a condition of the event triggering mechanism is that an error between desired poses of different groups of target robot arms is greater than an error threshold.

[0016] In some implementations, the task allocation module is further configured to: evaluate whether there is a collision risk based on distances between a plurality of groups of target robot arms with different task targets; if there is a collision risk, exclude a group of robot arms with the collision risk from the plurality of robot arms, and determine remaining robot arms; allocate the plurality of robot end gripping positions as task targets to a plurality of target robot arms in the remaining robot arms. In a third aspect, an electronic device is provided, which includes a processor and a memory connected with the processor in communication; the memory stores computer execution instructions; and the processor executes the computer execution instructions stored in the memory to implement the method in the first aspect.

[0017] In a fourth aspect, a computer readable storage medium is provided. The computer readable storage medium stores computer executable instructions that, when executed by a processor, implement the method of the first aspect.

[0018] In a fifth aspect, a computer program product is provided. The computer program product includes a computer program that, when executed by a processor, implements the method of the first aspect.

[0019] The multi-robot combined laying control method and system provided by the application can obtain the installation position of the mesh through a machine vision system, and can complete automatic laying through multi-robot combined control, thereby improving the laying operation efficiency; and the event triggering mechanism is adopted to maintain the consistency of the collaborative control of multiple groups of robots.

[0020] Additional aspects and advantages of the application will be described in part in the description that follows, and in part will become apparent to those skilled in the art from the description, or by practicing the application. BRIEF DESCRIPTION OF DRAWINGS

[0021] The above and / or additional aspects and advantages of the application will become apparent and be readily appreciated from the following description, including the accompanying drawings, in which: Figure 1 A flowchart of a multi-robot combined laying control method provided by an embodiment of the application; Figure 2 A system configuration schematic diagram of a group of robots provided by an embodiment of the application; Figure 3 A simulation control schematic diagram of a group of robots provided by an embodiment of the application; Figure 4 A block diagram of a multi-robot combined laying control system provided by an embodiment of the application; Figure 5 A block diagram of an electronic device provided by an embodiment of the application. DETAILED DESCRIPTION

[0022] The embodiments of the application are described in detail below, examples of which are shown in the accompanying drawings, in which the same or similar reference signs represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by reference to the accompanying drawings are exemplary and are intended to explain the application, and cannot be understood as limiting the application.

[0023] The multi-robot combined laying control method, device and equipment of the embodiments of the application are described below with reference to the accompanying drawings.

[0024] Figure 1A flowchart of a multi-robot combined laying control method provided by an embodiment of the application is shown.

[0025] It should be noted that the execution subject of the multi-robot combined laying control method of the embodiment of the application is the multi-robot combined laying control system of the embodiment of the application, which can be configured in an electronic device so that the electronic device can perform a multi-robot combined laying control function.

[0026] As shown in Figure 1 The multi-robot combined laying control method comprises the following steps: Step S101: The anchor hole position of the last net piece is obtained through a machine vision system, and the installation position of the current net piece to be installed is determined based on the anchor hole position of the last net piece.

[0027] In some embodiments, the target anchor hole position closest to the current net piece to be installed is determined from the anchor hole position of the last net piece; and the installation position of the current net piece to be installed is determined based on the target anchor hole position and the fixed distance between the anchor hole position and the edge of the net piece.

[0028] The anchor hole position in this embodiment refers to the position of the anchor rod tray in the image collected by the machine vision system. The anchor rod tray is identified by a high-precision machine vision system, and the distance between the anchor rod tray and the edge of the net piece is fixed. The coordinates of the anchor rod tray on the last installed net piece are obtained by identifying the anchor rod tray, and the coordinates of the anchor rod tray of the current net piece to be installed are determined accordingly, and the coordinates of the installation position of the current net piece to be installed are determined.

[0029] Step S102: Based on the installation position of the current net piece to be installed, the end-of-arm grasping positions of the plurality of robots when the laying task is completed are determined.

[0030] In some embodiments, a three-dimensional model of the current net piece to be installed is generated based on the installation position of the current net piece to be installed; and the end-of-arm grasping positions of the plurality of robots when the laying task is completed are determined in the three-dimensional model.

[0031] After the installation position of the entire net piece to be installed is determined, the positions of the net pieces grasped by the robots when the laying is completed, i.e., the end-of-arm grasping positions, can be determined accordingly, and the end poses of the robots, i.e., the desired poses of the robots, can be determined based on this.

[0032] Step S103: The end-of-arm grasping positions of the plurality of robots are assigned to a plurality of target robots in the plurality of robots as task targets.

[0033] In some embodiments, a group of end-effector grasping positions belonging to the same direction and parallel to each other among the plurality of end-effector grasping positions are assigned to a group of target arms in the same direction as a same task target; wherein the group of target arms comprises a leading arm and a following arm, the following arm receives a desired pose of the leading arm and follows the leading arm to change the pose.

[0034] In one example, the plurality of arms comprises four groups of arms, each group of arms is placed in parallel, and the end-effector grasping positions in the same direction and parallel to each other are assigned to a group of arms in the same direction as a same task; then the desired poses of the group of arms are consistent, the following arm receives the desired pose of the leading arm and follows the leading arm to change the pose, which can improve the robustness of control.

[0035] In one example, the simulation model of a group of arms completing the same task target is shown in FIG. 1, wherein the desired pose model determined by the S function design and the square wave signal is taken as the input of the leading arm J0, and the pose models determined by the S functions attitude1, attitude2, and attitude3 represent the following arms J1, J2, and J3 in FIG. 1, respectively. Figure 3 Figure 2 Among them, J1 and J2 directly receive the pose information of the leading arm and take it as the desired value, and J3 only communicates with J2, takes the pose information of J2 as the desired information of itself and follows it to change the pose, which has the advantage of improving the robustness of the distributed multi-arm system.

[0036] In some embodiments, after assigning the plurality of end-effector grasping positions to a plurality of target arms in the plurality of arms as task targets, the method further comprises: based on the distance between a plurality of groups of target arms with different task targets, evaluating whether there is a collision risk; if there is a collision risk, excluding a group of arms with a collision risk from the plurality of arms to determine remaining arms; and assigning the plurality of end-effector grasping positions to a plurality of target arms in the remaining arms as task targets.

[0037] In this embodiment, after assigning the task target, it is evaluated whether there is a collision risk between the arms, and if there is, the task target assignment is performed again.

[0038] In some embodiments, the reachability of the arms can also be evaluated, such as according to the correspondence between the end coordinates of the arms and the coordinates of the positions to be installed on the mesh, the correspondence between the end pose of the arms and the desired pose, etc. to evaluate the reachability of the arms, and if the evaluation is not reachable, the task target assignment is performed again.

[0039] ​Step S104, for each target robot arm, generating a desired pose based on the robot arm end gripping position corresponding thereto and inputting the desired pose into the target robot arm, and controlling the target robot arm to complete the web laying operation.

[0040] In some embodiments, for multiple groups of target robot arms with different task targets, the robot arms are controlled to complete the web laying operation through an event triggering mechanism, and a condition of the event triggering mechanism is that an error between the desired poses of the different groups of target robot arms is greater than an error threshold.

[0041] It should be noted that the pose information of the robot arm can be obtained through inverse kinematics based on the robot arm end gripping position, which is prior art and will not be described herein.

[0042] In the multi-robot arm combined web laying control in the embodiment, the installation position of the web is obtained through the machine vision system, the automatic web laying is completed through the multi-robot arm combined control, and the web laying operation efficiency is improved; the event triggering mechanism is adopted to maintain the consistency of the collaborative control of the multiple groups of robot arms.

[0043] In one example, for multiple groups of robot arms with different task targets, the spatiotemporal consistency thereof is ensured through an event triggering mechanism, and an event triggering condition is as follows:

[0044] Wherein, represents an error between the desired poses of different groups of robot arms, the subscript i represents the group number of the group of robot arms, and the subscript j represents a position of the robot arm in the group, represents a set error threshold.

[0045] In order to realize the above-mentioned embodiments, the application further provides a multi-robot arm combined web laying control system. Figure 4 A structural schematic diagram of a multi-robot arm combined web laying control system provided by the embodiment of the application is shown in FIG. 4. Figure 4 As shown in the figure, the multi-robot arm combined web laying control system can include an image processing module 401, a position acquisition module 402, a task allocation module 403, and an operation control module 404.

[0046] The image processing module 401 is configured to obtain the anchor hole position of the last web through the machine vision system, and determine the installation position of the current web to be installed based on the anchor hole position of the last web. The position acquisition module 402 is configured to determine the robot arm end gripping positions when the web laying task is completed based on the installation position of the current web to be installed. The task allocation module 403 is configured to allocate the robot arm end gripping positions as task targets to multiple target robot arms in the multiple robot arms. The operation control module 404 is configured to generate a desired pose for each target robot arm based on the robot end gripping position corresponding to the target robot arm and input the desired pose into the target robot arm, so that the target robot arm completes the web laying operation.

[0047] In some implementations, the image processing module 401 is specifically configured to: determine a target anchor hole position closest to the current web to be installed from the anchor hole positions of the last web; determine the installation position of the current web to be installed based on the target anchor hole position and the fixed distance between the anchor hole position and the web edge.

[0048] In some implementations, the position acquisition module 402 is specifically configured to: generate a three-dimensional model of the current web to be installed based on the installation position of the current web to be installed; determine a plurality of robot end gripping positions when the web laying task is completed in the three-dimensional model.

[0049] In some implementations, the task allocation module 403 is specifically configured to: allocate a group of robot end gripping positions belonging to the same direction and parallel to each other in the plurality of robot end gripping positions as a same task target to a group of target robot arms in the same direction; wherein the target robot arms in the same group include a leading robot arm and a following robot arm, and the following robot arm receives the desired pose of the leading robot arm and follows the leading robot arm to change the pose.

[0050] In some implementations, the operation control module 404 is further configured to: control the robot arms to complete the web laying operation through an event triggering mechanism for a plurality of groups of target robot arms with different task targets; the condition of the event triggering mechanism is that the error between the desired poses of the different groups of target robot arms is greater than an error threshold.

[0051] In some implementations, the task allocation module 403 is further configured to: evaluate whether there is a collision risk based on the distance between the plurality of groups of target robot arms with different task targets; if there is a collision risk, exclude a group of robot arms with a collision risk from the plurality of robot arms, and determine remaining robot arms; allocate the plurality of robot end gripping positions as task targets to a plurality of target robot arms in the remaining robot arms. It should be noted that the foregoing explanation and description of the embodiment of the multi-robot arm combined web laying control method also applies to the embodiment of the multi-robot arm combined web laying control system, which will not be described here again.

[0052] In order to realize the above-mentioned embodiments, an electronic device is further provided. Please refer to Figure 5 ,Figure 5 is a structural schematic diagram of an electronic device provided by an embodiment of the present application. As shown in Figure 5 the electronic device 500 includes a processor 501 and a memory 502 connected with the processor 501 in communication; the memory 502 stores computer execution instructions; and the processor 501 executes the computer execution instructions stored in the memory to implement the method provided by the foregoing embodiments.

[0053] To implement the above-described embodiments, the present application further provides a computer readable storage medium, which stores computer execution instructions, and the computer execution instructions are executed by a processor to implement the method provided by the foregoing embodiments.

[0054] To implement the above-described embodiments, the present application further provides a computer program product, which includes a computer program, and the computer program is executed by a processor to implement the method provided by the foregoing embodiments.

[0055] In the foregoing embodiment description, the description of the terms “one embodiment”, “some embodiments”, “an example”, “a specific example”, or “some examples” and the like means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present description, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any suitable manner in any one or more embodiments or examples. In addition, the person skilled in the art can combine and combine the different embodiments or examples described in the present description and the features of the different embodiments or examples without contradiction.

[0056] In addition, the terms “first”, “second” are only for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with “first”, “second” can explicitly or implicitly include at least one of the features. In the description of the present application, the meaning of “a plurality of” is at least two, for example, two, three, etc., unless otherwise specifically limited.

[0057] Any process or method descriptions in flow charts or otherwise described herein can be understood as representing code modules, segments, or portions of code that include one or more executable instructions for implementing the specified logical functions or steps, and the preferred embodiments of the present application include additional or fewer functions, steps, or codes, and the functions, steps or codes can be performed in the order shown or discussed, including substantially concurrently or in reverse order, as will be apparent to those having ordinary skill in the art.

[0058] The logic and / or steps represented in flow diagrams or otherwise described herein, for example, can be considered as a sequence of executable instructions, and can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For purposes of this specification, a "computer-readable medium" can be any apparatus that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. The computer-readable medium can be a product of the manufacturing and / or processing. The computer-readable medium can include, but is not limited to, the following: an electronic connection (an electronic device having one or more wires), a portable computer diskette (a magnetic device), a RAM (random access memory), a ROM (read-only memory), an EPROM (erasable programmable ROM) or Flash memory, an optical fiber device, and a portable CD ROM. Additionally, the computer-readable medium can be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, via, for example, an optical scanner, then compiled, interpreted, or otherwise processed, and stored in a computer memory in order to be executed.

[0059] It should be understood that aspects of the application can be implemented in hardware, software, firmware or combinations thereof. In the above embodiments, various steps or methods can be implemented in software or firmware that is stored in memory and executed by a suitable instruction execution system. As such, in some embodiments, specifically configured hardware can be used to implement at least some of the functionality described herein. For example, if implemented in hardware, the hardware can include any or a combination of the following: a discrete logic circuit having logic gates for implementing logic functions upon data signals, an application specific integrated circuit having appropriate combinational logic gates, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0060] Those of skill in the art would understand that information and signals can be represented using any of a variety of technologies and techniques. For example, data, instructions, commands, information, signals, bits, symbols, and chips that can be referenced throughout the above description can be represented by voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields or particles, or any combination thereof.

[0061] In addition, each of the functional units in the various embodiments of the present application can be integrated in one processing module, or each of the units can be physically present separately, or two or more units can be integrated in one module. The integrated module can be implemented in the form of hardware or in the form of a software functional module. When the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer readable storage medium.

[0062] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it should be understood that the above embodiments are exemplary and should not be construed as limiting the present application, and those skilled in the art can make changes, modifications, replacements and variations to the above embodiments within the scope of the present application.

Claims

1. A multi-robot arm combined laying control method, characterized by, The method comprises the following steps: obtaining the anchor hole position of the previous mesh through a machine vision system, and determining the installation position of the current mesh to be installed based on the anchor hole position of the previous mesh; determining the plurality of mechanical arm end gripping positions when the mesh laying task is completed based on the installation position of the current mesh to be installed; distributing the plurality of mechanical arm end gripping positions as task targets to a plurality of target mechanical arms in the plurality of mechanical arms; for each target mechanical arm, generating an expected pose based on the corresponding mechanical arm end gripping position and inputting the expected pose into the target mechanical arm to control the target mechanical arm to complete the mesh laying work.

2. The method of claim 1, wherein, The determination of the installation position of the current mesh to be installed based on the anchor hole position of the previous mesh comprises: determining the target anchor hole position closest to the current mesh to be installed from the anchor hole position of the previous mesh; determining the installation position of the current mesh to be installed based on the target anchor hole position and the fixed distance between the anchor hole position and the mesh edge.

3. The method according to claim 1 or 2, characterized in that, The determination of the plurality of mechanical arm end gripping positions when the mesh laying task is completed based on the installation position of the current mesh to be installed comprises: generating a three-dimensional model of the current mesh to be installed based on the installation position of the current mesh to be installed; determining the plurality of mechanical arm end gripping positions when the mesh laying task is completed in the three-dimensional model.

4. The method of claim 1, wherein, The distribution of the plurality of mechanical arm end gripping positions as task targets to a plurality of target mechanical arms in the plurality of mechanical arms comprises: distributing a group of mechanical arm end gripping positions belonging to the same direction and parallel in the plurality of mechanical arm end gripping positions as the same task target to a group of target mechanical arms in the direction; wherein the target mechanical arms in the same group include a leading mechanical arm and a following mechanical arm, and the following mechanical arm receives the expected pose of the leading mechanical arm and follows the leading mechanical arm to change the pose.

5. The method of claim 4, wherein, The method further comprises: for a plurality of groups of target mechanical arms with different task targets, controlling the mechanical arms to complete the mesh laying work through an event triggering mechanism; the condition of the event triggering mechanism is that the error between the expected poses of the different groups of target mechanical arms is greater than an error threshold.

6. The method of claim 5, wherein, After the distribution of the plurality of mechanical arm end gripping positions as task targets to a plurality of target mechanical arms in the plurality of mechanical arms, the method further comprises: based on the distance between the plurality of groups of target mechanical arms with different task targets, evaluating whether there is a collision risk; if there is a collision risk, excluding a group of mechanical arms with a collision risk from the plurality of mechanical arms, and determining the remaining mechanical arms; distributing the plurality of mechanical arm end gripping positions as task targets to a plurality of target mechanical arms in the remaining mechanical arms.

7. A multi-robot arm combined laying control system, characterized by, It comprises: an image processing module for obtaining the anchor hole position of the previous mesh through a machine vision system, and determining the installation position of the current mesh to be installed based on the anchor hole position of the previous mesh; a position acquisition module for determining the plurality of mechanical arm end gripping positions when the mesh laying task is completed based on the installation position of the current mesh to be installed; a task distribution module for distributing the plurality of mechanical arm end gripping positions as task targets to a plurality of target mechanical arms in the plurality of mechanical arms. The operation control module is configured to, for each target mechanical arm, generate a desired pose based on the corresponding mechanical arm end gripping position and input the desired pose into the target mechanical arm, and control the target mechanical arm to complete the tarpaulin laying operation.

8. An electronic device, comprising: The method comprises: a processor, and a memory connected to the processor in communication; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement the method of any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are executed by the processor to implement the method of any one of claims 1-6.

10. A computer program product, characterised in that, The computer program is executed by the processor to implement the method of any one of claims 1-6.

Images