Robot control method, device, equipment and storage medium
Through a robot control method based on non-grabbing operations, the end effector synchronizes the position and attitude with the dynamic target at the interception moment and performs flexible control, the problem of difficulty in grasping large-size or large-load dynamic targets is solved, and the problem of the existing technology is difficult to grasp large-size or large-load dynamic targets is improved. The stability and success rate of crawling are suitable for a wider range of application scenarios.
Patent Information
- Application Number
- CN202210266707.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-03-17
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2042-03-17
AI Technical Summary
Existing robot crawling technologies are difficult to achieve efficient and accurate crawling of dynamic targets, especially large-size or large-load targets, and traditional methods rely on the crawling load and size of end-effectors, limiting their application scenarios.
Using a robot control method based on non-grabbing operation, the interception time and flexibility control information are determined by obtaining the motion information of the target object, and the robot's end effector synchronizes the position and posture with the target object at the interception time, and smoothes after the interception. Control to achieve stable capture of dynamic goals.
It realizes automatic and stable crawling of dynamic targets without crawling points, large size or large loads, improves crawling success rate and stability, is suitable for a wider application scenario, and does not rely on end effector capture. Take load and size.
Smart Images

Figure CN116803625B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the fields of artificial intelligence and robots, and more specifically, to a robot control method, device, equipment and storage medium. Background Art
[0002] As a key technology in the field of robotics, robot grasping plays a decisive role in realizing various operations of robots. For industrial robots used for spraying, assembly, welding, handling and other purposes, the grasped objects are usually static, and relatively mature technologies have been developed. However, as robots begin to be applied to other fields such as scientific experiments, home services, and medical health, the application scenarios are more complex and changeable, requiring robots to have stronger interactive capabilities, breaking the original mode of robots repeating operations according to fixed procedures in a known environment. Grasping dynamic objects has also become one of the robot's work tasks. Researching the robot's dynamic target grasping operation can not only promote the development of the robot's core grasping technology and improve the robot's intelligence level, but also promote the progress of humanoid robots. Therefore, studying the robot's grasping operation of dynamic targets is of great significance and value.
[0003] Therefore, an efficient robot control method is needed so that the robot can quickly and accurately grasp dynamic targets. Summary of the invention
[0004] In order to solve the above problems, the present invention adopts a grasping method based on non-grasping operation to dynamically intercept and compliantly control the target object when the robot's palm is not closed, thereby achieving fast and accurate dynamic target grasping.
[0005] Embodiments of the present disclosure provide a robot control method, apparatus, device, and computer-readable storage medium.
[0006] An embodiment of the present disclosure provides a robot control method, comprising: acquiring current motion information of a target object at a current moment, the current motion information including the position and posture of the target object; determining grasping information of the target object based on the current motion information of the target object, the grasping information including an interception moment of the target object and compliance control information of the target object; and determining motion control information of the robot based on the determined grasping information of the target; wherein, at the interception moment, the end position and end posture of the robot are respectively the same as the position and posture of the target object, and the compliance control information is used to reduce the speed and rotation of the target object to zero after the interception moment.
[0007] An embodiment of the present disclosure provides a robot control device, comprising: an information acquisition module, configured to acquire current motion information of a target object at a current moment, the current motion information including the position and posture of the target object; a grasping determination module, configured to determine grasping information of the target object based on the current motion information of the target object, the grasping information including an interception moment of the target object and compliance control information of the target object; and a motion control module, configured to determine motion control information of the robot based on the determined grasping information of the target, wherein, at the interception moment, the end position and end posture of the robot are respectively the same as the position and posture of the target object, and the compliance control information is used to reduce the speed and rotation of the target object to zero after the interception moment.
[0008] An embodiment of the present disclosure provides a robot control device, comprising: one or more processors; and one or more memories, wherein a computer executable program is stored in the one or more memories, and when the computer executable program is executed by the processor, the robot control method as described above is executed.
[0009] An embodiment of the present disclosure provides a computer-readable storage medium having computer-executable instructions stored thereon, which are used to implement the robot control method as described above when executed by a processor.
[0010] The embodiment of the present disclosure provides a computer program product or a computer program, which includes computer instructions, which are stored in a computer-readable storage medium. The processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the robot control method according to the embodiment of the present disclosure.
[0011] Compared with traditional robot grasping methods, the method provided by the embodiments of the present disclosure does not need to perform envelope grasping, nor does it need to rely on the grasping load and grasping size of the end effector. It can use the robot's simple end effector (palm part, such as a tablet) to achieve the grasping of dynamic target objects with no grasping points, large size or large load.
[0012] The method provided by the embodiment of the present disclosure uses the end effector of the robot to intercept the dynamic target for the grasping method based on non-grasping operation, and performs compliant control on the dynamic target after intercepting the dynamic target, so as to achieve the stable state of the end effector and the dynamic target, thereby realizing the automatic and stable grasping of the dynamic target object. The method of the embodiment of the present disclosure can determine the integrated optimal grasping algorithm for the entire dynamic target grasping process, thereby automatically generating the optimal grasping trajectory and the optimal interception time of the robot, so that the robot can reasonably adjust the grasping trajectory in combination with its own motion state before grasping the target, thereby improving the success rate of target grasping, and after intercepting the dynamic target, the contact force in the grasping process is relieved by compliant control, so that the impact force in the contact process becomes smaller, and the stability of the dynamic target grasping is improved. In addition, the method provided by the embodiment of the present disclosure can realize the grasping of objects without grasping points and dynamic targets of large size and large load by adopting the grasping method based on non-grasping operation, thereby being suitable for a wider range of application scenarios and practical uses. BRIEF DESCRIPTION OF THE DRAWINGS
[0013] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some exemplary embodiments of the present disclosure, and a person of ordinary skill in the art can obtain other drawings based on these drawings without creative work.
[0014] Figure 1A is a schematic diagram showing a scenario of controlling a robot using a computing device according to an embodiment of the present disclosure;
[0015] Figure 1B is a schematic diagram illustrating an example dynamic target grabbing according to an embodiment of the present disclosure;
[0016] Figure 2A is a flowchart illustrating a robot control method according to an embodiment of the present disclosure;
[0017] Figure 2B is a schematic flowchart showing a dynamic target object grasping process according to an embodiment of the present disclosure;
[0018] Figure 2C is a schematic diagram showing a dynamic target object grasping process according to an embodiment of the present disclosure;
[0019] Figure 3 is a schematic diagram showing the association between multiple performance indicators in an integrated optimal crawling algorithm according to an embodiment of the present disclosure;
[0020] Figure 4A is a schematic diagram illustrating a target object pre-interception operation according to an embodiment of the present disclosure;
[0021] Figure 4B is a schematic diagram showing a phase trajectory curve of the XY direction de-racing and de-velocity control of the target object according to an embodiment of the present disclosure;
[0022] Figure 5 is a schematic diagram showing the establishment of an integrated optimal grasping problem according to an embodiment of the present disclosure;
[0023] Figure 6 is a flowchart illustrating an example solution to an integrated optimal grasping problem according to an embodiment of the present disclosure;
[0024] Figure 7 is a schematic diagram showing a robot control device according to an embodiment of the present disclosure;
[0025] Figure 8 A schematic diagram of a robot control device according to an embodiment of the present disclosure is shown;
[0026] Fig. 9 A schematic diagram illustrating the architecture of an exemplary computing device according to an embodiment of the present disclosure; and
[0027] Fig.10 A schematic diagram of a storage medium according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION
[0028] In order to make the purpose, technical solution and advantages of the present disclosure more obvious, the exemplary embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all the embodiments of the present disclosure, and it should be understood that the present disclosure is not limited to the exemplary embodiments described here.
[0029] In this specification and the accompanying drawings, substantially the same or similar steps and elements are represented by the same or similar reference numerals, and repeated descriptions of these steps and elements will be omitted. At the same time, in the description of the present disclosure, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance or ranking.
[0030] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present disclosure pertains. The terms used herein are only for the purpose of describing the embodiments of the present invention and are not intended to limit the present invention.
[0031] To facilitate description of the present disclosure, concepts related to the present disclosure are introduced below.
[0032] The robot control method disclosed in the present invention may be based on artificial intelligence (AI). Artificial intelligence is a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. For example, for a robot control method based on artificial intelligence, it can make the robot reasonably adjust the grasping trajectory in combination with its own motion state before grasping the target in a manner similar to that of humans by predicting the motion information of the dynamic target to be grasped and making instant grasping behavior decisions, so as to plan the optimal grasping trajectory, thereby achieving the grasping of the dynamic target. By studying the design principles and implementation methods of various intelligent machines, artificial intelligence enables the robot control method disclosed in the present invention to have the function of real-time and accurate identification of multiple objects in the environment that meet the contour features of pre-set markers and automatically extracting their position features to guide the positioning of the mobile robot.
[0033] For example, the robot control method disclosed in the present invention can be based on the robot dynamic target grasping technology. Due to the limitations of robots in perception and control, the dynamic target grasping of robots faces many challenges in multiple key technologies. Precision, autonomy, and safety are the three core issues that should be considered when the robot performs dynamic target grasping operations. The grasping process of dynamic targets is very different from that of static targets. Static targets are fixed grasped and released, and the initial and final positions of the target objects have never changed, so the actual operation process is relatively simple. The grasping of dynamic targets requires the synchronous movement of the end effector of the robot, and its position changes throughout the process. Therefore, when performing the grasping operation of the dynamic target, it is necessary to predict the motion of the dynamic target to obtain the prediction information of the target for the grasping operation. Among them, the robot control method disclosed in the present invention mainly focuses on the part of performing the dynamic target grasping operation based on the obtained motion information of the target object, so there is no restriction on the acquisition method of these motion information, such as motion prediction based on the target area, motion prediction based on the target feature, motion prediction based on the target deformation template, and motion prediction based on the target model. The target object trajectory acquisition method can be applied to the robot control method disclosed in the present invention.
[0034] In summary, the solutions provided by the embodiments of the present disclosure involve technologies such as artificial intelligence and robot dynamic target grasping. The embodiments of the present disclosure will be further described below in conjunction with the accompanying drawings.
[0035] Figure 1A It is a schematic diagram showing a scenario of controlling a robot using a computing device according to an embodiment of the present disclosure. Figure 1Bis a schematic diagram illustrating an example dynamic target grabbing according to an embodiment of the present disclosure.
[0036] like Figure 1A As shown, the robot can be controlled through various computing devices, wherein the computing device can obtain the real-time motion information of the robot from each motor of the robot through the network, and after a series of data processing for a specific task based on the real-time motion information, control information of the motion parameters of the robot can be generated, and the control information can be returned to the robot again through the network in the form of a control signal to control the operation of the motors at each joint of the robot.
[0037] Optionally, the computing device may specifically include, for example, a smart phone, a tablet computer, a laptop, a vehicle-mounted terminal, a wearable device, etc. The network may be an Internet of Things (IoT) based on the Internet and / or telecommunication network, which may be a wired network or a wireless network, for example, it may be a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a cellular data communication network, or other electronic network capable of realizing information exchange functions.
[0038] Optionally, the robot may be Figure 1B The robot shown. For example, the robot may include three parts: a base, a robot body, and an end effector, wherein the end effector may be designed as a flat plate to achieve non-grasping grasping of the target. For example, for a dynamic target object moving in the direction of the arrow in the figure, the end effector of the robot may be controlled to dynamically grasp it, and the dynamic grasping may include interception and stabilization operations, that is, firstly achieving pre-contact with the dynamic target through an interception operation, and then compliantly controlling the target that is still in motion through a stabilization operation to achieve static balance between the end effector and the target object.
[0039] At present, robot grasping, as a key technology in the field of robotics, plays a decisive role in realizing various operations of robots. For industrial robots used for spraying, assembly, welding, handling and other purposes, the grasping objects are usually static, and relatively mature technologies have been developed. However, as robots begin to be applied to other fields such as scientific experiments, home services, and medical health, the application scenarios are more complex and changeable, requiring robots to have stronger interactive capabilities, breaking the original mode of robots repeating operations according to fixed procedures in a known environment. Grasping dynamic objects has also become one of the robot's work tasks. Researching the robot's dynamic target grasping operation can not only promote the development of the robot's core grasping technology and improve the robot's intelligence level, but also promote the progress of humanoid robots. Therefore, studying the robot's grasping operation of dynamic targets is of great significance and value.
[0040] For the problem of grasping dynamic targets, existing solutions are usually based on a grasping grasping method, which uses the palm and fingers of the robot's end effector to jointly grasp the target. These dynamic target grasping methods usually directly intercept the target object and directly grasp the target object after accompanying the flight. Among them, the target object is intercepted by the end effector by predicting the trajectory of the target object, and the target object is accompanied by the flight, so that the target object is grasped with the fingers when the target object decelerates to 0. However, since these dynamic target grasping methods need to use the palm and fingers of the end effector to grasp the target at the same time, they usually rely on the grasping load and grasping size of the end effector, and may not be able to grasp dynamic targets of large size or large load. In addition, due to the envelope grasping method, the existing methods also have high requirements for the design and control of the robot's fingers.
[0041] Based on this, the present disclosure provides a robot control method, which dynamically intercepts and flexibly controls the target object when the robot's palm is not closed, thereby achieving fast and accurate dynamic target grasping.
[0042] Compared with traditional robot grasping methods, the method provided by the embodiments of the present disclosure does not need to perform envelope grasping, nor does it need to rely on the grasping load and grasping size of the end effector. It can use the robot's simple end effector (palm part, such as a tablet) to achieve the grasping of dynamic target objects with no grasping points, large size or large load.
[0043] Considering that the dynamic target object grasping problem is an open problem, which involves optimal control in multiple stages and multiple directions, therefore, in the robot control method disclosed in the present invention, the dynamic target object grasping problem can be integratedly modeled and optimized to design an integrated optimal grasping algorithm that includes multiple performance indicators and multiple boundary constraints, thereby automatically generating the grasping trajectory of the robot during the entire grasping process.
[0044] The method provided in the embodiments of the present disclosure uses the robot's end effector to intercept a dynamic target in a grasping method based on non-grasping operations, and flexibly controls the dynamic target after intercepting the dynamic target to achieve a stable state of the end effector and the dynamic target, thereby realizing automatic and stable grasping of the dynamic target object.
[0045] The method of the embodiment of the present disclosure can determine the integrated optimal grasping algorithm for the entire dynamic target grasping process, thereby automatically generating the optimal grasping trajectory and optimal interception time of the robot, so that the robot can reasonably adjust the grasping trajectory in combination with its own motion state before grasping the target, thereby improving the success rate of target grasping, and after intercepting the dynamic target, the contact force in the grasping process is alleviated through compliant control, so that the impact force in the contact process becomes smaller, thereby improving the stability of dynamic target grasping. In addition, the method provided by the embodiment of the present disclosure can realize the grasping of objects without grasping points and dynamic targets of large size and large load by adopting a grasping method based on non-grasping operation, thereby being suitable for a wider range of application scenarios and practical uses.
[0046] Figure 2A is a flowchart illustrating a robot control method 200 according to an embodiment of the present disclosure. Figure 2B It is a schematic flowchart showing a dynamic target object grasping process according to an embodiment of the present disclosure. Figure 2C is a schematic diagram illustrating a dynamic target object grasping process according to an embodiment of the present disclosure.
[0047] In step 201, current motion information of a target object at a current moment may be acquired, where the current motion information includes a position and a posture of the target object.
[0048] like Figure 2B As shown, before executing dynamic target object grasping, target object recognition and trajectory prediction can be performed to obtain the current motion information of the target object, such as the position, speed and posture information of the target object at the current moment, so as to guide the robot to grasp the target object.
[0049] Optionally, considering that the movement of the target object is in a gravitational field, it can be considered that its motion trajectory satisfies the parabolic trajectory principle in the absence of external forces, that is, there is only gravitational acceleration in the vertical downward direction. Therefore, the initial position, initial velocity and initial posture of the target object (that is, the current motion information corresponding to the current moment) can be identified based on vision (or motion capture system), and the motion trajectory of the target object can be predicted based on the kinematic and dynamic equations of the target object, so that the prediction can be used for the subsequent target grasping operation of the robot control method disclosed in the present invention. The embodiments of the present invention do not limit the methods of target object identification and trajectory prediction, and any method that can achieve the acquisition of the current motion information of the target object and trajectory prediction can be used in the robot control method disclosed in the present invention.
[0050] Optionally, the complete grasping process of the target object in the method of the present disclosure may include two stages: pre-interception and post-interception. Figure 2B and 2CAs shown. In the pre-interception stage, there is no contact between the target object and the robot's end effector, and the robot needs to reach the specified position and posture at the specified time (i.e., the interception time) to intercept the target object. In the post-interception stage, the robot's end effector contacts the target object to perform compliant control of the target object (e.g., de-rotation and de-speed operation), thereby achieving static balance of the target object on the end effector by completing dynamic balance with the target object, i.e. Figure 2C Crawl completion status shown.
[0051] It should be understood that in the complete grasping process of the target object, considering the maximum reachable range (maximum motion range) of the robot's end effector, the selection of the interception moment and the compliant control process in the post-interception stage will affect each other. The selection of the interception moment will directly determine the motion state of the target object at the end of the pre-interception stage (i.e., at the end of the position and posture prediction stage of the target object), and the motion state will be the initial value of the next stage (post-interception stage), which will determine the boundary values of the de-rotation and de-speed operations in the post-interception stage, including the final docking position of the end effector, and the final docking position of the end effector needs to meet the robot's reachable workspace requirements. For example, the earlier the interception moment is selected, the larger the motion range and more motion time may be required for the compliant control process to achieve static balance with the target object, and the later the interception moment is selected, the smaller the motion range required for the compliant control process may be, but in both cases the final docking position of the end effector may exceed the maximum motion range of the end effector. Therefore, the selection of the interception moment needs to be comprehensively considered in combination with each stage in the grasping process.
[0052] In step 202, the grasping information of the target object may be determined based on the current motion information of the target object, where the grasping information includes the interception moment of the target object and the compliance control information of the target object.
[0053] As mentioned above, the grasping information of the target object can be integrated based on the various stages of the grasping process, including the task analysis involving the pre-interception and post-interception stages and the establishment of an integrated optimal grasping problem.
[0054] According to an embodiment of the present disclosure, determining the grasping information of the target object based on the current motion information of the target object may include: determining a first performance indicator, a second performance indicator, a third performance indicator and a fourth performance indicator of the robot based on the current motion information of the target object, the first performance indicator being related to a first time and a terminal velocity and a terminal acceleration of the robot, the second performance indicator being related to a second time and a terminal angular velocity and a terminal angular acceleration of the robot, the third performance indicator and the fourth performance indicator being related to a third time and a fourth time, respectively; and jointly determining the grasping information of the target object based on the first performance indicator, the second performance indicator, the third performance indicator and the fourth performance indicator of the robot. Wherein, according to an embodiment of the present disclosure, jointly determining the grasping information of the target object based on the first performance indicator, the second performance indicator, the third performance indicator and the fourth performance indicator of the robot may include: jointly determining the grasping information of the target object based on the first performance indicator, the second performance indicator, the third performance indicator and the fourth performance indicator of the robot, and based on the first constraint condition, the second constraint condition, the third constraint condition and the fourth constraint condition corresponding to the first performance indicator, the second performance indicator, the third performance indicator and the fourth performance indicator respectively.
[0055] Figure 3 is a schematic diagram showing the association between multiple performance indicators in an integrated optimal crawling algorithm according to an embodiment of the present disclosure.
[0056] According to an embodiment of the present disclosure, at the interception moment, the end position and end posture of the robot are respectively the same as the position and posture of the target object.
[0057] For the pre-interception stage, the robot needs to adjust the motion state of its end effector before the interception moment, including adjusting the position, speed and posture of its end effector (i.e., the robot's end position, end speed and end posture) to reach the position, speed and posture of the target object at the interception moment and achieve smooth pre-contact with the target object.
[0058] Optionally, in the motion state adjustment of the above-mentioned robot in the pre-interception stage, the optimization of the first performance indicator and the second performance indicator as described above may be involved to obtain the influence of the grasping information in the pre-interception stage on the motion state of the end effector and these performance indicators.
[0059] Figure 4A is a schematic diagram showing the operation before interception of a target object according to an embodiment of the present disclosure. Figure 4A As shown, the motion trajectory of the intercepted point on the target object is represented as r c (t), the trajectory r c(t) can be obtained in real time according to the trajectory prediction as described above, and the motion trajectory of the interception point on the end effector of the robot is represented as r m (t).
[0060] In the following example, assume that the current time (i.e., the initial time) is t0, and the interception time is make Where u represents the acceleration level control quantity of the end effector. Therefore, a dynamic model of the end effector of the robot can be established, and the state equation of the end effector can be expressed as:
[0061]
[0062] in, represents the state variable of the end effector, Represents the speed level control of the end effector.
[0063] Therefore, as an example, the following weighted first performance index J1 regarding the time, speed and acceleration of the end effector can be designed:
[0064]
[0065] Among them, weights ω1, ω2>0, The time from the current moment to the moment when the end position and end velocity of the robot are consistent with the position and velocity of the target object may be represented as the first time. According to an embodiment of the present disclosure, the first time may be the time from the current moment to the interception moment.
[0066] The first performance indicator J1 as shown in formula (2) may include performance measurements of the first time, the energy consumed during the first time, and the inertial force. Therefore, the optimization of the first performance indicator J1 may be regarded as a joint optimization of the first time length and the energy consumed during the first time length and the magnitude of the inertial force.
[0067] Optionally, in the motion state adjustment of the robot in the pre-interception stage, a two-step matching operation with the target object may be involved. In the first matching operation associated with the first time (i.e., associated with the first performance indicator), the position of the robot's end effector may be made consistent with the position of the target object at the interception moment, and the speed of the end effector may be further made consistent with the speed of the target object.
[0068] According to an embodiment of the present disclosure, the first constraint may include a first equation constraint related to the first time, and the first equation constraint may indicate that the end position and end velocity of the robot are the same as the position and velocity of the target object at the interception moment.
[0069] Therefore, based on the above first matching operation, the first constraint condition ψ1 can be expressed as:
[0070]
[0071] Optionally, the first constraint condition ψ1 may represent the grasping trajectory of the robot's end effector and the motion trajectory of the target object at the interception time. Intersection (i.e. Figure 4A It should be noted that in the first constraint condition shown in equation (3), the interception time It is still an undetermined independent variable.
[0072] Therefore, the Hamiltonian function can be introduced into equation (2):
[0073]
[0074] in, The co-state vector of the above system can be represented.
[0075] Alternatively, the co-state equation and the governing equation can be obtained according to the variational principle of functional extrema:
[0076]
[0077] Substituting it into the Hamiltonian function, we can get
[0078]
[0079]
[0080]
[0081] Therefore, integrating the above formula, we can get
[0082]
[0083] make but Therefore, the optimal interception time can be obtained The analytical solution of the motion trajectory of the robot's end effector is:
[0084]
[0085] Where e is Euler's constant, The optimal interception time The coefficient of correlation.
[0086] Assumptions Among them, cosh represents the hyperbolic cosine function, so according to the boundary conditions, we can get as follows:
[0087]
[0088]
[0089] k0=r m (t0)-k2-k3 (13)
[0090]
[0091] As mentioned above, the above calculation can obtain the analytical expression of the optimal motion trajectory of the robot's end effector, but its boundary condition value is not determined, that is, the optimal interception time Not yet determined. Due to the optimal interception time It will directly determine the boundary value of the above-mentioned interception system, as well as the speed of the target object at the end of the position and posture prediction stage before interception, and the optimal solution of the pre-interception stage will be the initial value of the next stage (post-interception stage). It is necessary to comprehensively consider the relationship between the grasping information and performance indicators of each stage to determine the optimal interception time of the pre-interception stage.
[0092] As described above, the optimization of the first performance index under the first constraint condition realizes the matching of the position and velocity of the end effector with the target object at the interception time. Similarly, the matching of the posture of the end effector with the target object at the interception time can be realized by the second performance index and the second constraint condition as described below.
[0093] Alternatively, assume that the optimal termination time of the robot's end effector in the posture direction is Considering that the robot's terminal posture adjustment needs to be completed before the position prediction interception (that is, it needs to be consistent with the target object's posture before the target interception is implemented, and it can be completed before the posture adjustment termination time To intercept time Keep the posture consistent with the target object), so we can get (For example, in Figure 3 In the example shown, ), and the boundary value of the end posture is given by Decide.
[0094] Alternatively, assume that the robot's end posture is adjusted continuously and smoothly from the end posture Ψ0(t0) at the initial time to the optimal termination time The terminal posture Right now
[0095] Ψ0=[α0,β0,γ0]→Ψ f =[α f , β f , γ f ] (15)
[0096] Among them, α0, β0, γ0 and α f , β f , γ f They represent the Euler angles corresponding to the postures at the initial and final moments respectively.
[0097] Optionally, assume that the rotation matrices A0 and A of the robot’s end effector at the initial moment and the optimal termination time are f They are
[0098]
[0099] Then the pointing deviation of the end effector can be expressed as:
[0100]
[0101] The above formula (17) can express that after the coordinate system of the robot's end effector rotates around the unit vector r by an angle of φ0, it is transformed from the rotation matrix A0 at the initial moment to the rotation matrix A at the optimal termination time: f .
[0102] According to formula (17), r and φ0 can be expressed as:
[0103]
[0104] Therefore, by selecting an appropriate rotation trajectory φ(t) of the end effector's coordinate system, the end effector of the robot can be moved continuously and smoothly to the desired posture, where the end angular velocity of the robot can be expressed as
[0105]
[0106] As described above, let ω be the end angular velocity of the robot. As an example, the following weighted second performance index J2 regarding the time, angular velocity and angular acceleration of the end effector can be designed:
[0107]
[0108] Among them, the weight ω 11 ,ω 12>0, The time from the current moment to the time when the end posture of the robot reaches consistency with the posture of the target object can be represented as the second time. According to an embodiment of the present disclosure, the second time is not greater than the first time.
[0109] Therefore, according to the terminal angular velocity shown in equation (19), equation (20) can be converted into:
[0110]
[0111] Through the above transformation, the three-dimensional optimization problem in the posture direction as shown in formula (20) can be transformed into a one-dimensional rotation angle optimization problem as shown in formula (21).
[0112] Optionally, in the second matching operation associated with the second time (ie, associated with the second performance indicator) in the above two-step matching operation, the posture of the robot's end effector can be made consistent with the posture of the target object at the interception moment.
[0113] According to an embodiment of the present disclosure, the second constraint condition includes a second equality constraint associated with the second time, and the second equality constraint indicates that the end posture of the robot is the same as the posture of the target object at the interception moment.
[0114] Therefore, based on the above second matching operation, the second constraint condition ψ2 can be expressed as:
[0115]
[0116] Optionally, the second constraint ψ2 may represent the rotation angle and rotation angular velocity of the robot's end effector around the unit vector r. The moment is the same as the rotation angle and angular velocity of the target object around the unit vector r. It should be noted that in the second constraint condition shown in the above equation (22), It is also still an independent variable that has not yet been determined.
[0117] Therefore, let According to the optimal control solution method with free termination time and constrained termination state, the optimal termination time can be obtained The analytical expression of the robot's end posture trajectory is as follows:
[0118]
[0119] Where e is Euler's constant, The optimal termination time The coefficient of correlation.
[0120] Assumptions Therefore, according to the boundary conditions, we can get as follows:
[0121]
[0122]
[0123] k 10 =φ m (t0)-k 12 -k 13 (26)
[0124]
[0125] As mentioned above, by determining the end position r of the robot from the initial time t0 m (t0), terminal speed and terminal attitude Ψ0(t0) to interception time The end position Terminal speed and terminal posture The optimal posture planning can make the robot's end effector at the moment of interception Achieve a consistent posture with the target object to achieve a smooth interception of the target object.
[0126] Next, after achieving a smooth interception of the target object, the grasping of the target object can enter the post-interception stage. In this post-interception stage, the robot's end effector can make further contact with the target object to perform compliant control of the target object, including de-rotation and de-velocity processing, such as eliminating the target object's velocity in three-dimensional space and rotation in the XOZ and YOZ planes, so that the target object reaches static balance.
[0127] According to an embodiment of the present disclosure, the compliance control information can be used to reduce the speed and rotation of the target object to zero after the interception moment. The above-mentioned reducing the rotation of the target object to zero may include reducing the rotation angle and rotational angular velocity of the target object to zero. Considering that the robot end and the target object need to reach static balance in the post-interception stage, it is necessary to eliminate the relative motion between the target object and the end effector of the robot, including displacement and rotation.
[0128] According to an embodiment of the present disclosure, the compliance control information may include terminal position planning information of the robot in the X, Y and Z directions after the interception moment, which is used to indicate the motion trajectory of the robot in the X, Y and Z directions after the interception moment. Optionally, the above-mentioned de-rotation and de-speed operation can be achieved by appropriately designing the motion trajectory of the robot terminal to achieve static balance.
[0129] Optionally, in the embodiments of the present disclosure, the grasping problem of the block target object (such as Figure 1B , 2C , 3 and 4A), and the self-rotation of the target object is not considered. Therefore, optionally, the balance control problem of the target object in the post-interception stage can be decomposed into the dynamic balance problem of the XOZ plane and the YOZ plane, including the elimination speed processing for the Z direction, and the elimination speed and elimination rotation processing for the X and Y directions, so as to finally restore the target object to a static equilibrium state. The following will be described in two parts based on the above description, namely, the Z-direction processing and the XY-direction processing.
[0130] Alternatively, if Figure 3 As shown, the Z-direction processing and the XY-direction processing may correspond to the optimization of the third performance index and the optimization of the fourth performance index respectively.
[0131] (1) Z-direction processing
[0132] Optionally, assume that the target object is at the interception time The Z-direction velocity is The system state equation can be expressed as:
[0133]
[0134] in, represents the acceleration of the target object in the Z direction, m represents the mass of the target object, g is the gravitational acceleration, and F z It is the support force of the robot's end effector on the target object in the Z direction.
[0135] Therefore, as an example, the third performance indicator J3 including only time may be designed as follows:
[0136]
[0137] According to an embodiment of the present disclosure, the third time may be the time from the interception moment to the time when the speed of the target object along the Z direction decreases to zero, that is, That is J3.
[0138] As mentioned above, the third performance indicator J3 shown in formula (29) can indicate the Z-direction velocity of the target object from the interception moment The time required for the third performance index J3 to decrease to zero can be regarded as the optimization of the third performance index J3. Time required Optimization.
[0139] According to an embodiment of the present disclosure, the third constraint condition may include a third equality constraint and a third inequality constraint. The third equality constraint may indicate that the speed of the target object along the Z direction decreases to zero within the third time, and the third inequality constraint may indicate that the torque applied by the robot to the target object is not greater than the maximum torque that the robot can apply to the target object.
[0140] As mentioned above, in Time to reach It can be used as the boundary condition for optimizing the third performance index, that is, the third equality constraint. Considering the maximum torque that the robot's end effector can apply to the target object, the limitation of the maximum torque can be used as the inequality constraint for optimizing the third performance index.
[0141] For example, suppose the maximum torque that the robot's end effector can exert on the target object in the Z direction is Then the third inequality constraint can be expressed as:
[0142]
[0143] Among them, ||F z || represents the torque applied by the robot's end effector to the target object in the Z direction. By introducing the Lagrange multiplier The Hamiltonian function of this system can be expressed as:
[0144]
[0145] According to the optimal control theory, the co-state equation can be expressed as:
[0146]
[0147] According to Pontryagin's minimum principle, the optimal solution F z * For any satisfy:
[0148]
[0149] Can get
[0150]
[0151] Therefore, the end position planning of the robot's end effector in the Z direction (that is, the motion trajectory of the target object in the Z direction) can be obtained:
[0152]
[0153] In this case, the Z-axis stabilization time It can be expressed as:
[0154]
[0155] (2) XY processing
[0156] Optionally, assume that the target object is at the interception time The XY velocity under in, and Respectively represent the target object at the interception moment The X- and Y-direction velocities of the target object at the time of interception The initial rotation angle under 0x ,θ 0y ) T , where θ 0x ,θ 0y Respectively represent the target object at the interception moment The initial rotation angles around the X and Y axes, and Indicates the target object at the interception moment The initial rotation angular velocity under .
[0157] Therefore, by analyzing the force on the end effector and the target object, we can get:
[0158]
[0159] Among them, a x represents the acceleration of the target object in the X direction, F represents the support force of the robot's end effector on the target object, and θ represents the rotation angle of the target object. Consider the maximum torque that the end effector can exert on the target object in the Z direction. Right now You can get:
[0160]
[0161]
[0162] Among them, a y Indicates the acceleration of the target object in the Y direction, θ x and θ y Represent the rotation angles of the target object around the X-axis and Y-axis respectively. Consider the state equation of the double integral system:
[0163]
[0164] Where (x1, x2) represents the state of the system (including the target object and the robot's end effector), where x1 represents the rotation angle of the system, and x2 represents the rotation angular velocity of the system. Therefore, as an example, consider the system from the initial state The shortest time to reach the terminal state where both the rotation angle and the rotation angular velocity are 0 can be designed as the fourth performance indicator J4 containing only time as follows:
[0165]
[0166] According to an embodiment of the present disclosure, the fourth time may be the time from the interception moment to the time when the speed of the target object along the X and Y directions decreases to zero, and the rotation angle and rotation angular velocity of the target object around the X and Y axes decrease to zero, that is, the above That is J4.
[0167] As described above, the fourth performance indicator J4 shown in formula (41) can indicate the X and Y speeds of the target object from the interception moment The time required for the target object's rotation angle and rotation angular velocity around the X-axis and Y-axis to decrease to zero, therefore, the optimization of the fourth performance indicator J4 can be regarded as the optimization of achieving Time required Optimization.
[0168] Therefore, on the basis of ensuring that the target object does not slip off the end effector of the robot, the speed of the system (including the end speed of the robot and the speed of the target object) v xy Need to becomes 0, and the rotation angle θ changes from θ0 to 0.
[0169] According to an embodiment of the present disclosure, the fourth constraint condition may include a fourth equality constraint and a fourth inequality constraint, wherein the fourth equality constraint may indicate that the speed of the target object along the X and Y directions and the rotation angle and rotation angular velocity around the X and Y axes decrease to zero within the fourth time, and the fourth inequality constraint may indicate that the joint angular acceleration of the robot is not greater than the maximum joint angular acceleration of the robot. Optionally, the system speed v xy from Becoming 0 and the rotation angle θ changing from θ0 to 0 can be used as the fourth equality constraint.
[0170] As mentioned above, in order to prevent the robot's joint motion from exceeding its motion limit and causing damage to the joint due to sudden changes in joint motion, the fourth inequality constraint can be expressed as:
[0171]
[0172] Similarly, by introducing the Lagrange multipliers λ1 and λ2, the Hamiltonian function of the system can be expressed as
[0173] H4=1+λ1 T ω+λ2 T u (43)
[0174] According to the optimal control theory, the co-state equation can be expressed as:
[0175]
[0176]
[0177] Therefore, by integrating the above equations (44) and (45), we can obtain λ1 and λ2 as follows:
[0178] λ1(t)=c1 (46)
[0179] λ2(t)=c2-c1t (47)
[0180] Among them, c1 and c2 are integration constants.
[0181] As mentioned above, if we need to choose u(t) so that H4 takes a minimum value, we only need λ2 T (t)u(t) takes a minimum value, because Then when And u(t) and λ2 T When the sign of (t) is opposite, H4 can take the minimum value, so the optimal control of u(t) can be expressed as:
[0182]
[0183] Therefore, let According to the state equation of formula (40), we can get:
[0184] x2=u+x 20 (49)
[0185]
[0186] Therefore, based on equations (49) and (50), we can obtain:
[0187]
[0188] Figure 4B is a schematic diagram showing a phase trajectory curve of XY-direction de-racing and de-velocity control of a target object according to an embodiment of the present disclosure.
[0189] like Figure 4B As shown, the phase trajectory curve (switching curve) can be expressed as:
[0190]
[0191]
[0192] Right now:
[0193]
[0194] The following can be divided into three cases to prove the shortest fourth time
[0195] (a) When the state point is to the right of the switching curve, that is, When (such as Figure 4B Point A in the Steering Point B is the transition point, and
[0196] t f =t AB +t BO (55)
[0197] Among them, t AB represents the time from point A to point B, and other similar reference numerals have similar meanings, which will not be explained below. Therefore, it can be obtained that:
[0198]
[0199] Right now By solving, we can get:
[0200]
[0201] Therefore, the value of x2 at point B is x 2B It can be expressed as:
[0202]
[0203] When the status point changes from Figure 4B B along γ + When moving to the origin, but
[0204]
[0205]
[0206] (b) When the state point is to the left of the switching curve, that is, When (such as Figure 4B Point C in the figure), u is given by Steering Likewise, in Figure 4B Point D in the
[0207]
[0208] By solving it, we can get
[0209]
[0210] And the x2 value x at point D 2D It can be expressed as
[0211]
[0212] The state point is from D along γ - Move towards the origin, and t Do =x 2D ,therefore
[0213]
[0214] (c) When the state point is at γ + hour, Then Δt f =-x 20 (x 20 <0); when the state point is in γ - hour, Similarly, Δt f =x 20 , so we can get
[0215]
[0216] Therefore, the XY stabilization time can be determined for
[0217]
[0218] Similarly, according to the above XY stabilization time The corresponding XY end position planning can be determined. It should be understood that the above Z-axis stabilization time and XY direction stabilization time The relative size of is not fixed and will be determined by the actual motion of the target object and the robot's end effector (e.g. Figure 3 As shown, and situations are possible), and and The larger stabilization time can be used as the optimal terminal time of the robot control method disclosed in the present invention.
[0219] Based on the above analysis of the two stages before and after interception, it can be understood that the determination of the interception moment tf1 will determine the entire grasping process, and the smooth control in the post-interception stage is directly affected by it.
[0220] According to an embodiment of the present disclosure, the grasping information corresponds to the optimal grasping information that optimizes the first performance indicator, the second performance indicator, the third performance indicator, and the fourth performance indicator while satisfying the first constraint condition, the second constraint condition, the third constraint condition, and the fourth constraint condition. As described above, the determination of the grasping information needs to comprehensively consider the states of the target object and the end effector at each stage of the entire grasping process, so as to determine the optimal grasping information by combining all performance indicators and boundary constraints.
[0221] Figure 5 is a schematic diagram showing the establishment of an integrated optimal grasping problem according to an embodiment of the present disclosure.
[0222] As described above, by combining the above-mentioned first performance indicator, second performance indicator, third performance indicator and fourth performance indicator, as well as the first constraint condition, second constraint condition, third constraint condition and fourth constraint condition corresponding to these performance indicators respectively, the integrated optimal grasping problem of the present invention can be established, and the integrated optimal grasping problem includes a comprehensive analysis of each stage of the entire grasping process.
[0223] Optionally, considering that the robot's end effector grasping a dynamic target is a typical optimal control problem, and the operation process mainly involves multiple optimization indicators as described above, according to multi-phase optimal control, the integrated performance indicators of the robot control method disclosed in the present invention can be established as follows:
[0224] J=h1J1+h2J2+h3J3+h4J4 (67)
[0225] Among them, h i , i=1, 2, 3, 4 are weight coefficients of the corresponding performance indicators, which are used to weigh the impact of different position and posture unit quantities on the integrated performance indicator, which can be determined based on the priority and importance of each performance indicator.
[0226] In addition, the control of the robot also needs to take into account the constraints of the robot's accessible workspace, that is, during the entire grasping process, the constraints described below also need to be met.
[0227] According to an embodiment of the present disclosure, determining the grasping information of the target object based on the current motion information of the target object may also include: determining the grasping information of the target object based on the current motion information of the target object and the motion constraint conditions of the robot, the grasping information being used to enable the robot to satisfy the motion constraint conditions of the robot during the grasping process of the target object.
[0228] Optionally, the determination of the above grasping information may also be limited by the actual motion constraints of the robot's end effector, that is, the movement of the robot's end effector needs to be within the maximum reachable space of the robot, otherwise the robot may be damaged if the movement of the robot exceeds its motion limit.
[0229] According to an embodiment of the present disclosure, the motion constraint condition of the robot may include that the motion range of the end effector of the robot during the grasping process of the target object does not exceed the maximum motion range of the end effector.
[0230] For example, the maximum motion range of the end effector can be expressed as Figure 4A Inequality constraints at the intercept points shown:
[0231]
[0232] in, and || r m || respectively represent the upper and lower boundaries of the robot's maximum reachable space.
[0233] Therefore, by combining the above integrated optimal grasping problem and the inequality constraint of formula (68), we can establish Figure 5 The integrated optimal grasping problem shown in FIG. By solving the integrated problem, the grasping information can be determined. The example solution method of the integrated problem can be found in the following reference Figure 6 described.
[0234] Next, return to Figure 2A In step 203, motion control information for the robot may be determined based on the determined grasping information for the target.
[0235] According to an embodiment of the present disclosure, determining the motion control information of the robot based on the determined grasping information of the target may include: determining the first time, the second time, the third time and the fourth time based on the grasping information; and determining control information of at least one of the joint position, joint angular velocity or joint angular acceleration of the robot based on the first time, the second time, the third time and the fourth time, for controlling the robot to grasp the target object.
[0236] As mentioned above, by determining the captured information of the target, the interception time can be determined Know Then determine the Z-axis stabilization time and XY direction stabilization time Thereby, control information on at least one of the joint position, joint angular velocity or joint angular acceleration of the robot is determined.
[0237] Figure 6 is a flowchart illustrating an example solution to an integrated optimal grasping problem according to an embodiment of the present disclosure.
[0238] Considering that the interception point of the robot's end effector on the target object is the intersection of the target object's motion trajectory and the working ball in the reachable space of the robot's end effector (i.e., the working ball of any radius that satisfies the reachable space), and as mentioned above, the determination of the optimal solution for the entire grasping process can be regarded as based on the interception moment Therefore, optionally, Figure 6 As shown in FIG. 1 , the optimal analytical solution of the motion trajectory of the robot's end effector can be combined, and the accessible working ball space can be discretized (for example, the working ball corresponding to the maximum accessible space can be divided into N equal parts, each with a radius of Where n=0, 1, 2, ..., N-1).
[0239] Optionally, according to the radius of each equally divided sphere The corresponding equally divided ball and the trajectory of the target object || r C The intersection point of || can be used to solve the end time of the pre-interception stage corresponding to each equally divided ball (i.e. the interception moment)
[0240] Therefore, each interception moment can be calculated The corresponding integrated performance index J(n) of the entire grasping process, n = 0, 1, 2, ..., N-1, the final interception time t stop (n), n = 0, 1, 2, ..., N-1, and the actual terminal position of the robot after interception is r stop (n), n = 0, 1, 2, ..., N-1, where
[0241] As mentioned above, by removing the r that exceeds the maximum motion range of the robot, the above constraints are met. stop (n) (i.e. removing these r stop (n) corresponding to the working ball and interception time options), the optimal value J(k) of the integrated performance index of the above-mentioned integrated optimal grasping problem and the interception time corresponding to the optimal value J(k) can be determined Actual terminal position Therefore, based on the determined grasping information, an integrated optimal grasping trajectory can be generated to control the subsequent grasping operation of the robot.
[0242] In addition, optionally, when a feasible solution that satisfies the above-mentioned integrated optimal grasping problem cannot be obtained, the robot can determine that its own motion state cannot meet the grasping requirements and actively abandon the grasping task. This can be achieved by setting global constraints related to the robot's motion limits for the entire grasping process through the method provided in the embodiments of the present disclosure, thereby avoiding meaningless interception operations and saving energy consumption.
[0243] Figure 7 is a schematic diagram showing a robot control device 700 according to an embodiment of the present disclosure.
[0244] The robot control device 700 may include an information acquisition module 701 , a grasping determination module 702 and a motion control module 703 .
[0245] According to an embodiment of the present disclosure, the information acquisition module 701 may be configured to acquire current motion information of the target object at a current moment, where the current motion information includes the position and posture of the target object.
[0246] Optionally, before executing dynamic target object grasping, target object recognition and trajectory prediction can be performed to obtain current motion information of the target object, such as predicted information of the position, speed and posture of the target object at the current moment, for the robot to grasp the target object.
[0247] Optionally, the complete grasping process of the target object in the method of the present disclosure may include: Figure 2B and 2C There are two stages, pre-interception and post-interception, as shown. In the pre-interception stage, there is no contact between the target object and the robot's end effector, and the robot needs to reach the specified position and posture at the specified time (i.e., the interception time) to intercept the target object. In the post-interception stage, the robot's end effector contacts the target object to perform compliant control of the target object (e.g., de-rotation and de-speed operation), thereby achieving static balance of the target object on the end effector by completing dynamic balance with the target object, i.e. Figure 2C Crawl completion status shown.
[0248] The grabbing determination module 702 may be configured to determine grabbing information of the target object based on current motion information of the target object, wherein the grabbing information includes an interception moment of the target object and compliance control information of the target object.
[0249] Optionally, the grasping determination module 702 may perform the operations described above with reference to step 202. The grasping information of the target object may be determined based on an integrated consideration of each stage of the grasping process, including task analysis involving the pre-interception and post-interception stages and the establishment of an integrated optimal grasping problem.
[0250] In the pre-interception stage, the robot needs to adjust the motion state of its end effector before the interception moment, including adjusting the position, speed and posture of its end effector (i.e., the robot's end position, end speed and end posture) to reach the position, speed and posture of the target object at the interception moment and achieve smooth pre-contact with the target object.
[0251] Optionally, in the motion state adjustment of the above-mentioned robot in the pre-interception stage, the optimization of the first performance indicator and the second performance indicator as described above may be involved to obtain the influence of the grasping information in the pre-interception stage on the motion state of the end effector and these performance indicators.
[0252] After achieving a smooth interception of the target object, the grasping of the target object can enter the post-interception stage. In the post-interception stage, the robot's end effector can make further contact with the target object to perform compliant control of the target object, including de-rotation and de-velocity processing, such as eliminating the target object's velocity in three-dimensional space and rotation in the XOZ and YOZ planes, so that the target object reaches static balance.
[0253] Optionally, the balance control problem of the target object in the post-interception stage can be decomposed into the dynamic balance problem of the XOZ plane and the YOZ plane, including the elimination speed processing in the Z direction, and the elimination speed and elimination rotation processing in the X and Y directions, so as to finally restore the target object to a static balance state. Among them, the Z direction processing and the XY direction processing can correspond to the optimization of the third performance index and the optimization of the fourth performance index respectively.
[0254] Therefore, by combining the above-mentioned first performance indicator, second performance indicator, third performance indicator and fourth performance indicator, as well as the first constraint condition, second constraint condition, third constraint condition and fourth constraint condition corresponding to these performance indicators respectively, and taking into account the constraints of the robot's reachable workspace, the integrated optimal grasping problem disclosed in the present invention can be established, and the above-mentioned grasping information can be determined by solving the integrated problem. The integrated optimal grasping problem includes a comprehensive analysis of each stage of the entire grasping process, and the grasping information is the optimal grasping information determined by combining all performance indicators and boundary constraints.
[0255] The motion control module 703 may be configured to determine motion control information for the robot based on the determined grasping information of the target.
[0256] Optionally, by determining the capture information of the target, the interception time can be determined and Then determine the Z-axis stabilization time and XY direction stabilization time Thereby, the motion control information of the robot is determined, including the control information of at least one of the joint position, joint angular velocity or joint angular acceleration of the robot.
[0257] According to yet another aspect of the present disclosure, a robot control device is also provided. Figure 8 A schematic diagram of a robot control device 2000 according to an embodiment of the present disclosure is shown.
[0258] like Figure 8 As shown, the robot control device 2000 may include one or more processors 2010 and one or more memories 2020. The memory 2020 stores a computer readable code, and when the computer readable code is run by the one or more processors 2010, the robot control method described above may be executed.
[0259] The processor in the embodiments of the present disclosure may be an integrated circuit chip having signal processing capabilities. The processor may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application may be implemented or executed. The general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc., and may be an X86 architecture or an ARM architecture.
[0260] In general, various example embodiments of the present disclosure may be implemented in hardware or dedicated circuits, software, firmware, logic, or any combination thereof. Certain aspects may be implemented in hardware, while other aspects may be implemented in firmware or software that may be executed by a controller, microprocessor, or other computing device. When various aspects of the embodiments of the present disclosure are illustrated or described as block diagrams, flow charts, or using some other graphical representation, it will be understood that the blocks, devices, systems, techniques, or methods described herein may be implemented in hardware, software, firmware, dedicated circuits or logic, general purpose hardware or controllers or other computing devices, or some combination thereof as non-limiting examples.
[0261] For example, the method or device according to the embodiment of the present disclosure may also be implemented by Fig. 9 The architecture of the computing device 3000 shown in FIG. Fig. 9As shown, the computing device 3000 may include a bus 3010, one or more CPUs 3020, a read-only memory (ROM) 3030, a random access memory (RAM) 3040, a communication port 3050 connected to a network, an input / output component 3060, a hard disk 3070, etc. The storage device in the computing device 3000, such as the ROM 3030 or the hard disk 3070, may store various data or files used for processing and / or communication of the robot control method provided in the present disclosure and program instructions executed by the CPU. The computing device 3000 may also include a user interface 3080. Of course, Figure 8 The architecture shown is only exemplary and can be omitted according to actual needs when implementing different devices. Fig. 9 One or more components of a computing device are shown.
[0262] According to yet another aspect of the present disclosure, a computer-readable storage medium is provided. Fig.10 A schematic diagram 4000 of a storage medium according to the present disclosure is shown.
[0263] like Fig.10 As shown, the computer storage medium 4020 stores computer readable instructions 4010. When the computer readable instructions 4010 are executed by the processor, the robot control method according to the embodiment of the present disclosure described with reference to the above figures can be executed. The computer readable storage medium in the embodiment of the present disclosure can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. The non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM) or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous connection dynamic random access memory (SLDRAM) and direct memory bus random access memory (DR RAM). It should be noted that memory of the methods described herein is intended to include, but is not limited to, these and any other suitable types of memory. It should be noted that memory of the methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.
[0264] The embodiments of the present disclosure also provide a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the robot control method according to the embodiments of the present disclosure.
[0265] Embodiments of the present disclosure provide a robot control method, apparatus, device, and computer-readable storage medium.
[0266] Compared with traditional robot grasping methods, the method provided by the embodiments of the present disclosure does not need to perform envelope grasping, nor does it need to rely on the grasping load and grasping size of the end effector. It can use the robot's simple end effector (palm part, such as a tablet) to achieve the grasping of dynamic target objects with no grasping points, large size or large load.
[0267] The method provided by the embodiment of the present disclosure uses the end effector of the robot to intercept the dynamic target for the grasping method based on non-grasping operation, and performs compliant control on the dynamic target after intercepting the dynamic target, so as to achieve the stable state of the end effector and the dynamic target, thereby realizing the automatic and stable grasping of the dynamic target object. The method of the embodiment of the present disclosure can determine the integrated optimal grasping algorithm for the entire dynamic target grasping process, thereby automatically generating the optimal grasping trajectory and the optimal interception time of the robot, so that the robot can reasonably adjust the grasping trajectory in combination with its own motion state before grasping the target, thereby improving the success rate of target grasping, and after intercepting the dynamic target, the contact force in the grasping process is relieved by compliant control, so that the impact force in the contact process becomes smaller, and the stability of the dynamic target grasping is improved. In addition, the method provided by the embodiment of the present disclosure can realize the grasping of objects without grasping points and dynamic targets of large size and large load by adopting the grasping method based on non-grasping operation, thereby being suitable for a wider range of application scenarios and practical uses.
[0268] It should be noted that the flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment, or a part of a code, and the module, program segment, or a part of the code contains at least one executable instruction for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0269] In general, various example embodiments of the present disclosure may be implemented in hardware or dedicated circuits, software, firmware, logic, or any combination thereof. Certain aspects may be implemented in hardware, while other aspects may be implemented in firmware or software that may be executed by a controller, microprocessor, or other computing device. When various aspects of the embodiments of the present disclosure are illustrated or described as block diagrams, flow charts, or using some other graphical representation, it will be understood that the blocks, devices, systems, techniques, or methods described herein may be implemented in hardware, software, firmware, dedicated circuits or logic, general purpose hardware or controllers or other computing devices, or some combination thereof as non-limiting examples.
[0270] The exemplary embodiments of the present disclosure described in detail above are merely illustrative and not restrictive. It should be understood by those skilled in the art that various modifications and combinations may be made to these embodiments or their features without departing from the principles and spirit of the present disclosure, and such modifications should fall within the scope of the present disclosure.
Claims
1. A robot control method, wherein the end effector of the robot is a flat plate for non-grasping grasping of a target object, the method comprising: Acquire current motion information of the target object at a current moment, wherein the current motion information includes the position and posture of the target object; Based on the current motion information of the target object, determining the grasping information of the target object, the grasping information including the interception moment of the target object and the compliance control information of the target object; as well as Determining motion control information for the robot based on the determined grasping information of the target; Among them, at the interception moment, the end position and end posture of the robot are respectively the same as the position and posture of the target object, and after the interception moment, the end effector of the robot contacts the target object to perform compliant control of the target object based on the compliant control information, so that the speed and rotation of the target object are reduced to zero, so as to achieve static balance between the end effector of the robot and the target object.
2. The method of claim 1, wherein: Based on the current motion information of the target object, determining the grasping information of the target object includes: Based on the current motion information of the target object, determine a first performance indicator, a second performance indicator, a third performance indicator, and a fourth performance indicator of the robot, wherein the first performance indicator is related to a first time and a terminal velocity and a terminal acceleration of the robot, the second performance indicator is related to a second time and a terminal angular velocity and a terminal angular acceleration of the robot, and the third performance indicator and the fourth performance indicator are respectively related to the third time and the fourth time; and jointly determining grasping information of the target object based on the first performance indicator, the second performance indicator, the third performance indicator, and the fourth performance indicator of the robot; Wherein, reducing the rotation of the target object to zero includes reducing the rotation angle and the rotation angular velocity of the target object to zero; The first time is the time from the current moment to the time when the end position and end speed of the robot are consistent with the position and speed of the target object; the second time is the time from the current moment to the time when the end posture of the robot is consistent with the posture of the target object; the third time is the time from the interception moment to the time when the speed of the target object along the Z direction decreases to zero; the fourth time is the time from the interception moment to the time when the speed of the target object along the X and Y directions decreases to zero, and the rotation angle and rotation angular velocity of the target object around the X axis and the Y axis decrease to zero. The first time is the time from the current time to the interception time, and the second time is not greater than the first time.
3. The method of claim 2, wherein: Based on the first performance indicator, the second performance indicator, the third performance indicator and the fourth performance indicator of the robot, jointly determining the grasping information of the target object includes: Based on the first performance indicator, the second performance indicator, the third performance indicator and the fourth performance indicator of the robot, and based on the first constraint condition, the second constraint condition, the third constraint condition and the fourth constraint condition corresponding to the first performance indicator, the second performance indicator, the third performance indicator and the fourth performance indicator respectively, jointly determine the grasping information of the target object, The crawling information corresponds to the optimal crawling information that optimizes the first performance indicator, the second performance indicator, the third performance indicator and the fourth performance indicator while satisfying the first constraint condition, the second constraint condition, the third constraint condition and the fourth constraint condition.
4. The method of claim 3, wherein: The first constraint condition includes a first equality constraint associated with the first time, the first equality constraint indicating that the end position and the end velocity of the robot at the interception moment are the same as the position and velocity of the target object; The second constraint condition includes a second equality constraint associated with the second time, the second equality constraint indicating that the end posture of the robot is the same as the posture of the target object at the interception moment; The third constraint condition includes a third equality constraint and a third inequality constraint, wherein the third equality constraint indicates that the speed of the target object along the Z direction decreases to zero within the third time, and the third inequality constraint indicates that the torque applied by the robot to the target object is not greater than the maximum torque that the robot can apply to the target object; The fourth constraint condition includes a fourth equality constraint and a fourth inequality constraint. The fourth equality constraint indicates that the speed of the target object along the X and Y directions and the rotation angle and rotation angular velocity around the X and Y axes decrease to zero within the fourth time. The fourth inequality constraint indicates that the joint angular acceleration of the robot is not greater than the maximum joint angular acceleration of the robot.
5. The method of claim 2, wherein: Based on the determined grasping information of the target, determining the motion control information of the robot includes: Based on the captured information, determining the first time, the second time, the third time, and the fourth time; and Based on the first time, the second time, the third time and the fourth time, control information of at least one of the joint position, joint angular velocity or joint angular acceleration of the robot is determined to control the robot to grasp the target object.
6. The method of claim 2, wherein: Based on the current motion information of the target object, determining the grasping information of the target object further includes: Based on the current motion information of the target object and the motion constraint conditions of the robot, grasping information of the target object is determined, and the grasping information is used to enable the robot to meet the motion constraint conditions of the robot during the grasping process of the target object.
7. The method of claim 6, wherein: The motion constraint condition of the robot includes that the motion range of the end effector of the robot during the grasping process of the target object does not exceed the maximum motion range of the end effector.
8. The method of claim 1, wherein: The compliance control information includes terminal position planning information of the robot along the X, Y and Z directions after the interception moment, which is used to indicate the movement trajectory of the robot along the X, Y and Z directions after the interception moment.
9. A robot control device, wherein the end effector of the robot is a flat plate, used for non-grasping grasping of a target object, the device comprising: An information acquisition module is configured to acquire current motion information of the target object at a current moment, wherein the current motion information includes a position and a posture of the target object; A grasping determination module is configured to determine grasping information of the target object based on current motion information of the target object, wherein the grasping information includes an interception moment of the target object and compliance control information of the target object; as well as a motion control module, configured to determine motion control information for the robot based on the determined grasping information of the target, Among them, at the interception moment, the end position and end posture of the robot are respectively the same as the position and posture of the target object, and after the interception moment, the end effector of the robot contacts the target object to perform compliant control of the target object based on the compliant control information, so that the speed and rotation of the target object are reduced to zero, so as to achieve static balance between the end effector of the robot and the target object.
10. The device of claim 9, wherein: Based on the current motion information of the target object, determining the grasping information of the target object includes: Based on the current motion information of the target object, determine a first performance indicator, a second performance indicator, a third performance indicator, and a fourth performance indicator of the robot, wherein the first performance indicator is related to a first time and a terminal velocity and a terminal acceleration of the robot, the second performance indicator is related to a second time and a terminal angular velocity and a terminal angular acceleration of the robot, and the third performance indicator and the fourth performance indicator are respectively related to the third time and the fourth time; and jointly determining grasping information of the target object based on the first performance indicator, the second performance indicator, the third performance indicator, and the fourth performance indicator of the robot; Wherein, reducing the rotation of the target object to zero includes reducing the rotation angle and the rotation angular velocity of the target object to zero; The first time is the time from the current moment to the time when the end position and end speed of the robot are consistent with the position and speed of the target object; the second time is the time from the current moment to the time when the end posture of the robot is consistent with the posture of the target object; the third time is the time from the interception moment to the time when the speed of the target object along the Z direction decreases to zero; the fourth time is the time from the interception moment to the time when the speed of the target object along the X and Y directions decreases to zero, and the rotation angle and rotation angular velocity of the target object around the X axis and the Y axis decrease to zero. The first time is the time from the current time to the interception time, and the second time is not greater than the first time.
11. The device of claim 10, wherein: Based on the first performance indicator, the second performance indicator, the third performance indicator and the fourth performance indicator of the robot, jointly determining the grasping information of the target object includes: Based on the first performance indicator, the second performance indicator, the third performance indicator and the fourth performance indicator of the robot, and based on the first constraint condition, the second constraint condition, the third constraint condition and the fourth constraint condition corresponding to the first performance indicator, the second performance indicator, the third performance indicator and the fourth performance indicator respectively, jointly determine the grasping information of the target object, The crawling information corresponds to the optimal crawling information that optimizes the first performance indicator, the second performance indicator, the third performance indicator and the fourth performance indicator while satisfying the first constraint condition, the second constraint condition, the third constraint condition and the fourth constraint condition.
12. A robot control device, comprising: one or more processors; as well as One or more memories storing a computer executable program, wherein when the computer executable program is executed by the processor, the method according to any one of claims 1 to 8 is performed.
13. A computer program product stored on a computer-readable storage medium and comprising computer instructions which, when executed by a processor, cause a computer device to perform the method of any one of claims 1 to 8.
14. A computer-readable storage medium having computer-executable instructions stored thereon, wherein the instructions are used to implement the method according to any one of claims 1 to 8 when executed by a processor.
