A robot control method, device, equipment and medium

Abstract

The embodiment of the present invention discloses a robot control method, device, equipment and medium. The method includes: receiving decision information; wherein the decision information is generated based on the robot's state information and decision instructions; generating unconstrained trajectory planning information for controlling the robot based on the decision information; wherein the unconstrained trajectory planning information includes planning information for at least two motion dimensions of the robot, and each motion dimension is pre-associated with a moving part of the robot; obtaining the constraints of each motion dimension, and determining the constrained trajectory planning information based on the constraints and the unconstrained trajectory planning information; generating control instructions for each motion dimension of the robot based on the constrained trajectory planning information to control the robot. This technical solution can control multiple degrees of freedom of the robot at different levels and can interact with people in a rich and diverse manner.

Description

Technical Field

[0001] The present invention relates to the field of robotics technology, and in particular to a robot control method, device, equipment and medium. Background Art

[0002] With the rapid development of computer technology, sensor technology, artificial intelligence and other technologies, robotics technology has become increasingly mature. Among them, mobile robots are the most widely used and play an increasingly important role in many industries such as home services, aerospace, and industry. These various robots can perform their tasks well in specific environments.

[0003] Most existing robots are used for industrial production or to complete specific purposes, such as collaborative robotic arms, food delivery robots, and coffee robots. These robots were not designed with human interaction in mind. Some other toy robots or pet robots, although designed for interaction, are limited by sensors and intelligence levels, or by actuators, resulting in a lack of expressiveness or richness in expression. Summary of the Invention

[0004] The present invention provides a robot control method, device, equipment and medium, which can control multiple levels of freedom of the robot and enable rich and diverse interactions with humans.

[0005] According to one aspect of the present invention, a method for controlling a robot is provided, comprising:

[0006] Receiving decision information; wherein the decision information is generated based on the robot's state information and decision instructions;

[0007] generating, based on the decision information, unconstrained trajectory planning information for controlling the robot; wherein the unconstrained trajectory planning information includes planning information for at least two motion dimensions of the robot, each motion dimension being pre-associated with a moving component of the robot;

[0008] Obtaining constraints for each motion dimension, and determining constrained trajectory planning information based on the constraints and the unconstrained trajectory planning information;

[0009] Based on the constrained trajectory planning information, control instructions for each motion dimension of the robot are generated to control the robot.

[0010] Optionally, the decision information includes expected behavior information generated by the decision and decision behavior history information;

[0011] Accordingly, generating unconstrained trajectory planning information for controlling the robot based on the decision information includes:

[0012] Unconstrained trajectory planning information for controlling the robot is generated based on expected behavior information generated by the robot's decision and decision behavior history information.

[0013] Optionally, after generating unconstrained trajectory planning information for controlling the robot according to the decision information, the method further includes:

[0014] Based on the generated unconstrained trajectory planning information, different trajectories are scored according to a scoring strategy to obtain optimal trajectory information.

[0015] Optionally, the method further includes:

[0016] According to the triggering conditions, unconstrained trajectory planning information for controlling the robot is generated; wherein the triggering conditions include timed start.

[0017] Optionally, the constraint conditions include position constraints, velocity constraints, acceleration constraints, collision constraints, and joint limit constraints.

[0018] Optionally, the control instructions include at least eye movement control instructions, head movement control instructions, body movement control instructions and foot movement control instructions.

[0019] Optionally, generating eye movement control instructions for the robot based on the constrained trajectory planning information includes:

[0020] The position of the eye focus point is generated according to the eye movement dimension, the head movement control command and the body movement control command.

[0021] According to another aspect of the present invention, there is provided a control device for a robot, comprising:

[0022] An information receiving module, configured to receive decision information; wherein the decision information is generated based on the robot's state information and decision instructions;

[0023] a trajectory planning information generation module, configured to generate unconstrained trajectory planning information for controlling the robot based on the decision information; wherein the unconstrained trajectory planning information includes planning information for at least two motion dimensions of the robot, each motion dimension being pre-associated with a moving component of the robot;

[0024] A trajectory planning information determination module is used to obtain the constraint conditions of each motion dimension and determine the constrained trajectory planning information based on the constraint conditions and the unconstrained trajectory planning information;

[0025] A control module is used to generate control instructions for each motion dimension of the robot based on the constrained trajectory planning information to control the robot.

[0026] According to another aspect of the present invention, an electronic device is provided, comprising:

[0027] at least one processor; and

[0028] a memory communicatively connected to the at least one processor; wherein,

[0029] The memory stores a computer program that can be executed by the at least one processor. The computer program is executed by the at least one processor so that the at least one processor can execute the robot control method described in any embodiment of the present invention.

[0030] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the robot control method described in any embodiment of the present invention when executed.

[0031] The technical solution of the embodiment of the present invention is to receive decision information; wherein the decision information is generated based on the robot's state information and decision instructions; based on the decision information, generate unconstrained trajectory planning information for controlling the robot; wherein the unconstrained trajectory planning information includes planning information for at least two motion dimensions of the robot, and each motion dimension is pre-associated with a moving part of the robot; obtain the constraints of each motion dimension, and determine the constrained trajectory planning information based on the constraints and the unconstrained trajectory planning information; based on the constrained trajectory planning information, generate control instructions for each motion dimension of the robot to control the robot. This technical solution can control multiple different levels of freedom of the robot and can interact with people in a rich and diverse manner.

[0032] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0034] Figure 1This is a flowchart of a robot control method provided in accordance with the first embodiment of the present invention;

[0035] Figure 2 This is an overall schematic diagram of a planning system for a robot control method provided in Embodiment 1 of the present invention;

[0036] Figure 3 This is a flow chart of a robot control method provided according to the second embodiment of the present invention;

[0037] Figure 4 This is a schematic diagram of solving motion control instructions for a robot provided in accordance with the second embodiment of the present invention;

[0038] Figure 5 is a schematic diagram of an eye control unit of a robot provided according to a second embodiment of the present invention;

[0039] Figure 6 This is a schematic structural diagram of a robot control device provided according to a third embodiment of the present invention;

[0040] Figure 7 It is a structural diagram of an electronic device provided according to the fourth embodiment of the present invention. DETAILED DESCRIPTION

[0041] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0042] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0043] Example 1

[0044] Figure 1This is a flow chart of a robot control method provided according to the first embodiment of the present invention. This embodiment is applicable to robot control. The method can be executed by a robot control device. The robot control device can be implemented in the form of hardware and / or software. The robot control device can be configured in an electronic device with data processing capabilities. Figure 1 As shown, the method includes:

[0045] S110, receiving decision information; wherein, the decision information is generated based on the robot's state information and decision instructions.

[0046] The technical solution in this embodiment can be implemented by the robot's decision-making system. The robot in this embodiment can be composed of several important parts: a perception system, a decision-making system, a planning system, and an execution system. The perception system can use different sensors distributed throughout the robot (such as RGB cameras, infrared cameras, microphone arrays, and touch sensors) to obtain and integrate a variety of different information to provide basic services for other systems. The decision-making system can generate different decisions based on information from multiple different dimensions, such as the emotional system, memory system, growth system, and perception system. The planning system can accept decisions generated by the decision-making system and dynamically generate commands or motion trajectories for different control units. To ensure the executable nature of the trajectories generated by the planning system, the planning system will subscribe to information from multiple other systems and can continuously update or iteratively optimize trajectory commands using optimization or reinforcement learning. The hardware carrier of the robot in this embodiment may include multiple degrees of freedom (for example, the degrees of freedom of leg and foot movement, the degrees of freedom of spatial movement of the head, the degrees of freedom of robot eye rotation, the degrees of freedom of the tongue and tail of a pet robot, etc.). In addition, the robot of this application can be a legged robot or a wheeled robot. The technical solution of this embodiment can control the robot's multiple levels of freedom, allowing for rich and diverse interactions with people.

[0047] The decision information may be information on the motion dimensions of the various components that control the robot. The decision information may be generated based on the robot's state information and decision instructions. The robot's state may include the current state of each part of the robot and the state information of the surrounding environment. For example, the angle of each joint of the current robot, the position of the line of sight, the surrounding state information, and the state information of people. The robot's state can be used as an output to provide a basis for the spatial coordinate system conversion in trajectory planning, etc., to ensure that fixed sequence actions are not executed mechanically. The decision instructions may be that the robot's decision system generates different decision instructions based on information from multiple different dimensions such as the robot's emotional system, memory system, growth system, and perception system.

[0048] In this embodiment, the planning system can receive information about the motion dimensions of various components of the control robot.

[0049] S120. Generate unconstrained trajectory planning information for controlling the robot based on the decision information; wherein the unconstrained trajectory planning information includes planning information for at least two motion dimensions of the robot, and each motion dimension is pre-associated with a moving component of the robot.

[0050] Among them, the unconstrained trajectory planning information can control the motion dimensions of each moving part of the robot. The unconstrained trajectory planning information can be a simple online dynamic planning, which does not consider the mutual influence of the motion between different joints, nor does it consider the environment and its own collision information to perform robot trajectory planning information. In this embodiment, the unconstrained trajectory planning can be to ignore the various physical constraints existing in the real world and convert the planning problem into a feasible solution search problem within a discrete interval. Among them, the unconstrained trajectory planning information can include planning information for at least two motion dimensions of the robot, and each motion dimension is pre-associated with a moving part of the robot. The motion dimension can be understood as the angle of motion of the moving part of the robot. The planning information can be information for planning the operating dimensions of the moving part of the robot. The moving parts may include moving parts such as eyes, head, body and feet. Each motion dimension in this embodiment can be pre-associated with a moving part of the robot.

[0051] Illustratively, the decision-making system of the robot in this embodiment generates an expected behavior, that is, decision information such as the eyes always looking in the direction of the owner, even if the owner is always moving; at this time, the planning system plans the movement trajectories of the eyes, tail, head movement, body movement, and leg and foot movement that change over time.

[0052] In this embodiment, the planning system may generate planning information for controlling the robot in at least two motion dimensions that are pre-associated with the robot's motion components based on the robot's decision information.

[0053] S130 : Obtain constraint conditions for each motion dimension, and determine constrained trajectory planning information based on the constraint conditions and the unconstrained trajectory planning information.

[0054] The motion dimension can be understood as the angle of motion of the robot's moving parts. Constraints can include position constraints, velocity constraints, acceleration constraints, collision constraints, and joint limit constraints. Constrained trajectory planning information can be understood as trajectory information for the motion dimension of the robot's moving parts, with constraints added for planning.

[0055] The planning system in this embodiment obtains the constraints of each dimension of the robot and determines the constrained trajectory planning information based on the constraints and the unconstrained planning information.

[0056] For example, in this embodiment, the trajectory generated based on the unconstrained trajectory planning information and the constrained dynamic trajectory planning portion can be used to generate a more reasonable trajectory by taking various constraints into account. The advantage of this is that the unconstrained trajectory planning information has essentially searched for a solution close to the global optimal solution. Searching near this solution greatly accelerates the search and convergence speed, while also avoiding falling into local optimal solutions at this step. This is usually achieved through a secondary optimization process.

[0057] Quadratic optimization usually has the following form

[0058]

[0059] Ax≤b

[0060] Ex=d

[0061] Where x represents the variable to be optimized, which can represent information such as the rotation angle at each level. Q is a weight matrix, representing the importance of different variables. C(x) represents the optimization function or penalty function, and the optimization goal is to minimize C(x). Ax≤b represents a linear inequality constraint. A is usually a matrix, representing the combination of multiple inequality constraints. Ex=d represents an equality constraint. E is usually also a matrix, representing the combination of multiple equality constraints.

[0062] The first function, C(x), can generally be interpreted as an optimization objective or penalty function, where the optimization goal is to minimize its value. The second and third functions are inequality and equality constraints, respectively. Quadratic optimization also features fast solution speed and the ability to incorporate both soft and hard constraints. Soft constraints are partially violated, while hard constraints are non-violated. Hard constraints are often expressed as equality or inequality constraints. In this step, soft constraints can be included as a penalty function in the optimization calculation, while hard constraints are included as inequality constraints. Various robot parameter configurations can also be included as penalty functions. By configuring weights for different execution units or modules within the same execution unit, the generated trajectory can exhibit a variety of different behaviors. Some constraints are still nonlinear; to speed up the solution, they can be linearized. For example, a joint limit is a nonlinear constraint. This can be linearized in the current state and generalized coordinates, using the Jacobian matrix to approximate the trajectory for a short period in the future. Because this linearization method is only valid for a short period of time (e.g., 0.1 seconds), continuous iteration is required to continuously update the trajectory to ensure that the constraints are not violated. To address the nonlinearity of joint constraints, you can also first create a hypothetical trajectory and perform linearization along this trajectory. After obtaining the result, linearize on the resulting trajectory, and iterate until the result converges.

[0063] S140. Generate control instructions for each motion dimension of the robot based on the constrained trajectory planning information to control the robot.

[0064] The control instructions can be understood as instructions for controlling the motion dimensions of the robot's moving parts. The control instructions can include at least eye movement control instructions, head movement control instructions, body movement control instructions, and foot movement control instructions.

[0065] In this embodiment, the planning system generates control instructions for each motion dimension of the robot based on the constrained trajectory planning information, thereby controlling the movement of the robot at multiple levels of freedom.

[0066] The technical solution of the embodiment of the present invention is to receive decision information; wherein the decision information is generated based on the robot's state information and decision instructions; based on the decision information, generate unconstrained trajectory planning information for controlling the robot; wherein the unconstrained trajectory planning information includes planning information for at least two motion dimensions of the robot, and each motion dimension is pre-associated with a moving part of the robot; obtain the constraints of each motion dimension, and determine the constrained trajectory planning information based on the constraints and the unconstrained trajectory planning information; based on the constrained trajectory planning information, generate control instructions for each motion dimension of the robot to control the robot. This technical solution can control multiple different levels of freedom of the robot and can interact with people in a rich and diverse manner.

[0067] In this embodiment, optionally, the decision information includes expected behavior information generated by the decision and decision behavior history information; accordingly, generating unconstrained trajectory planning information for controlling the robot based on the decision information includes: generating unconstrained trajectory planning information for controlling the robot based on the expected behavior information generated by the decision of the robot and decision behavior history information.

[0068] The decision information may include expected behavior information generated by the decision and historical information about the decision behavior. The expected behavior information generated by the decision may be information about the expected robot behavior generated by the robot's decision system. For example, the expected behavior information generated by the decision may be information about the robot's behavior when it looks at a person. The historical information about the decision behavior may be information about the decision behavior history obtained from the decision system's memory system. In this embodiment, different information plays different roles in this unconstrained trajectory planning process.

[0069] In this embodiment, the planning system can generate unconstrained trajectory planning information for controlling the robot based on the expected behavior information generated by the robot's decision and the decision behavior history information.

[0070] This solution generates unconstrained trajectory planning information based on decision information. The decision history determines the continuity and consistency of trajectory planning in space and time, ensuring that multiple completely unrelated trajectories are not generated in a short period of time, which could affect the interactive experience. The expected behavior generated by the decision guides the direction of action in the next time period, ensuring timely feedback.

[0071] For example, the overall schematic diagram of the planning system is as follows Figure 2As shown, in addition, the decision system will also generate unconstrained trajectory information based on the different parameter configuration information of the robot; wherein the parameter configuration information may include the initial value of the robot, the maximum value of the angle / acceleration, etc., and parameter information of certain moving parts (such as the range of movement of the eyes) can be configured.

[0072] In this embodiment, unconstrained trajectory planning information is generated based on the robot's state, the expected behavior information generated by the decision, the decision behavior history information, and the various parameter configuration service information of the robot. In addition, the unconstrained trajectory planning information can also be generated by triggering a calculation trigger. Then, based on the unconstrained trajectory planning information and the constraints of each motion dimension of the robot, constrained moving trajectory planning information is generated; wherein the constraints may include position constraints, speed constraints, acceleration constraints, collision constraints, and joint limit constraints. In this example, based on the constrained trajectory planning information, control instructions for each motion dimension of the robot can be generated, for example, the eye unit time trajectory, the head unit time trajectory, the body unit command trajectory, and the leg / wheel unit command trajectory, thereby controlling the various moving parts of the robot.

[0073] Example 2

[0074] Figure 3 This is a flowchart of a robot control method according to a second embodiment of the present invention. This embodiment is optimized based on the above embodiment. Specifically, the optimization comprises: after generating unconstrained trajectory planning information for controlling the robot based on the decision information, the optimization comprises: scoring different trajectories according to a scoring strategy based on the generated unconstrained trajectory planning information to obtain optimal trajectory information.

[0075] like Figure 3 As shown, the method of this embodiment specifically includes the following steps:

[0076] S310, receiving decision information; wherein, the decision information is generated based on the robot's state information and decision instructions.

[0077] S320. Generate unconstrained trajectory planning information for controlling the robot based on the decision information; wherein the unconstrained trajectory planning information includes planning information for at least two motion dimensions of the robot, and each motion dimension is pre-associated with a moving component of the robot.

[0078] S330 : Based on the generated unconstrained trajectory planning information, score different trajectories according to a scoring strategy to obtain optimal trajectory information.

[0079] Among them, the scoring strategy can be based on the degree of violation of constraints of different trajectories, or it can be scored from the perspective of the robot's consumed capabilities, or it can be scored according to the robot's performance level, or it can be scored by aligning other configurable scoring strategies. The scoring strategy can be pre-configured and can be configured according to actual needs. In this embodiment, the trajectory with the best score can be selected based on the scores of different trajectories and passed to the constrained trajectory planning to facilitate constrained trajectory planning. The best trajectory information can be the trajectory information with the highest score after scoring different trajectories according to the scoring strategy.

[0080] In this embodiment, the planning system can score different trajectories according to a scoring strategy based on the generated unconstrained trajectory planning information to obtain the optimal trajectory information.

[0081] S340: Obtain constraints for each motion dimension, and determine constrained trajectory planning information based on the constraints and the unconstrained trajectory planning information.

[0082] S350: Based on the constrained trajectory planning information, generate control instructions for each motion dimension of the robot to control the robot.

[0083] The technical solution of the embodiment of the present invention is to receive decision information; wherein, the decision information is generated based on the state information and decision instructions of the robot; based on the decision information, generate unconstrained trajectory planning information for controlling the robot; wherein, the unconstrained trajectory planning information includes planning information for at least two motion dimensions of the robot, and each motion dimension is pre-associated with a moving part of the robot; based on the generated unconstrained trajectory planning information, score different trajectories according to a scoring strategy to obtain the best trajectory information. Obtain the constraints of each motion dimension, and determine the constrained trajectory planning information based on the constraints and the unconstrained trajectory planning information; based on the constrained trajectory planning information, generate control instructions for each motion dimension of the robot to control the robot. This technical solution can control multiple degrees of freedom of the robot at different levels and can interact with people in a rich and diverse manner.

[0084] In this embodiment, optionally, the method further includes: generating unconstrained trajectory planning information for controlling the robot according to a trigger condition; wherein the trigger condition includes a timed start.

[0085] The triggering condition may include a timed start, which may also be set according to actual needs. In this embodiment, when the calculation trigger reaches a triggering condition such as a timed start or when new decision information is received, the unconstrained trajectory planning operation will be triggered to start.

[0086] In this embodiment, the planning system can trigger the unconstrained trajectory planning information according to the trigger condition, and the computer trigger will generate the unconstrained trajectory planning information for controlling the robot.

[0087] Through such a setting, this solution can set trigger conditions according to needs and generate unconstrained trajectory planning information for controlling the robot, which is more flexible.

[0088] In this embodiment, optionally, the constraint conditions include position constraints, velocity constraints, acceleration constraints, collision constraints, and joint limit constraints.

[0089] Among them, position constraints can be understood as constraints on the positions of the robot's various moving parts. Speed constraints can be understood as constraints on the movement speeds of the robot's various moving parts. Acceleration constraints can be understood as constraints on the accelerations achieved by the movement speeds of the robot's various moving parts. Collision constraints can be understood as collisions between the robot's various moving parts and collisions with other objects. Joint limit constraints can be understood as constraints on the joint limits of the robot's various moving parts, such as the limitation that the movement dimension of a joint can rotate at most 280 degrees.

[0090] The constraints of this embodiment may include position constraints, velocity constraints, acceleration constraints, collision constraints, and joint limit constraints.

[0091] Through such a setting, this solution can calculate the reasonable operation trajectory of the robot according to the constraint conditions.

[0092] In this embodiment, optionally, the control instructions include at least eye movement control instructions, head movement control instructions, body movement control instructions, and foot movement control instructions.

[0093] Among them, eye movement control instructions can be understood as instructions for controlling the movement of the robot's eyes. Head movement control instructions can be understood as instructions for controlling the movement of the robot's head. Body movement control instructions can be understood as instructions for controlling the movement of the robot's body. Foot movement control instructions can be understood as instructions for controlling the movement of the robot's feet.

[0094] The control instructions in this embodiment may include at least eye movement control instructions, head movement control instructions, body movement control instructions, and foot movement control instructions.

[0095] For example, the robot's motion control instruction solution diagram is as follows: Figure 4As shown. In this embodiment, the robot can be controlled to move according to the received head motion control instructions, body and foot motion control instructions. The head and body leg and foot control units can also use a pre-set unified optimization framework to solve the commands of each joint actuator. The advantage of this is that the instructions of different parts can be processed in a unified manner to compensate for each other to the greatest extent. For example, high-speed rotation of the head will affect the balance of the body, and unified optimization can take into account the impact of head movement on the body, so that the legs and feet can produce a corresponding reaction to compensate. Secondly, the received six-degree-of-freedom movement or rotation command of the body may conflict with the gait movement of the legs and feet. In this case, considering the commands in a unified manner can most reasonably track the received commands.

[0096] In this example, the command-comprehensive solution system calculates control command information based on instructions from the planning module, as well as from the external force detection module, the flexible control module, and the protection module. While prioritizing robot balance, it tracks the planning module's real-time spatial trajectory to the greatest extent possible. During execution, to maintain robot balance, the command-comprehensive solution system fine-tunes the robot's body movement, or the foot placement and timing when involved, to ensure balance is maintained. The solution system continuously calculates the desired torque, velocity, and position of each joint actuator at a high control frequency.

[0097] Another possible implementation is to separate the control units for the head and legs during the solution, treating the effects of their motion as interference between the two systems. This has the advantage of separating the two systems and reducing control complexity, but the disadvantage is that it reduces the motion space, as both the head and the body need to leave some space for each other.

[0098] Through such a setting, this solution can control the motion dimensions of each moving part of the robot according to the robot's motion control instructions, thereby controlling multiple degrees of freedom of the robot.

[0099] In this embodiment, optionally, eye movement control instructions for the robot are generated based on the constrained trajectory planning information, including: generating the position of the eye focus point according to the eye movement dimension, head movement control instructions and body movement control instructions.

[0100] The eye movement dimension may include movement dimensions such as eye rotation speed, etc. The eye focus point may be the position of the focus of the eye.

[0101] In this embodiment, the planning system can generate a focus point command in real time according to the rotation speed of the eye and the relative position relationship between the eye, the body and the head, thereby determining the position of the eye focus point.

[0102] Through such a setting, this solution can generate the robot's eye movement control instructions based on the trajectory planning information, making the position of the eye focus point more accurate.

[0103] In addition, the schematic diagram of the eye control unit of this embodiment is as follows Figure 5 As shown, the eye control unit may also include a real-time facial animation rendering engine. The real-time facial animation rendering engine can obtain motion templates from different material library template libraries based on the information of the emotional system and environmental information, and manipulate different control points to generate a three-dimensional space model in real time and project it onto a two-dimensional screen. Alternatively, different control points may be manipulated to directly generate a two-dimensional image and display it on the screen. Alternatively, it is also possible to imitate the movements of the eyes, eyebrows, eyelashes, eyeballs, etc. with the help of a mechanical module instead of a screen. The role of the bionic micro-motion generator is to make the movements of the eyes more natural. For example, the eyes of humans and animals will still have tiny movements or blinking even when they are looking at an object.

[0104] The real-time facial animation rendering engine in this example can also render real-time animations of eyes, pupils, eyelashes, and even tongues as needed. Based on the time trajectory information of the planning system module, the rendering engine is responsible for rendering the three-dimensional animation of the spatial position of each part and projecting it into two-dimensional space, or directly rendering the two-dimensional animation based on the trajectory information of the planning system module.

[0105] Example 3

[0106] Figure 6 This is a schematic diagram of the structure of a robot control device provided by the third embodiment of the present invention. The device can execute the robot control method provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method. Figure 6 As shown, the device includes:

[0107] An information receiving module 610 is configured to receive decision information, wherein the decision information is generated based on the robot's state information and decision instructions;

[0108] a trajectory planning information generation module 620 for generating unconstrained trajectory planning information for controlling the robot based on the decision information; wherein the unconstrained trajectory planning information includes planning information for at least two motion dimensions of the robot, each motion dimension being pre-associated with a moving component of the robot;

[0109] A trajectory planning information determination module 630 is configured to obtain constraints for each motion dimension and determine constrained trajectory planning information based on the constraints and the unconstrained trajectory planning information;

[0110] The control module 640 is configured to generate control instructions for each motion dimension of the robot based on the constrained trajectory planning information, so as to control the robot.

[0111] Optionally, the decision information includes expected behavior information generated by the decision and decision behavior history information;

[0112] Accordingly, the trajectory planning information generating module 620 is specifically configured to:

[0113] Unconstrained trajectory planning information for controlling the robot is generated based on expected behavior information generated by the robot's decision and decision behavior history information.

[0114] Optionally, a trajectory scoring module is used to score different trajectories according to a scoring strategy based on the generated unconstrained trajectory planning information after generating unconstrained trajectory planning information for controlling the robot according to the decision information, so as to obtain optimal trajectory information.

[0115] Optionally, the device further includes:

[0116] A trigger generation module is used to generate unconstrained trajectory planning information for controlling the robot according to trigger conditions; wherein the trigger conditions include timed start.

[0117] Optionally, the constraint conditions include position constraints, velocity constraints, acceleration constraints, collision constraints, and joint limit constraints.

[0118] Optionally, the control instructions include at least eye movement control instructions, head movement control instructions, body movement control instructions and foot movement control instructions.

[0119] Optionally, the eye movement control module is specifically used to generate the position of the eye focus point according to the eye movement dimension, the head movement control instruction and the body movement control instruction.

[0120] A robot control device provided by an embodiment of the present invention can execute a robot control method provided by any embodiment of the present invention, and has functional modules and beneficial effects corresponding to the execution method.

[0121] Example 4

[0122] Figure 7is a structural diagram of an electronic device 10 provided according to a fourth embodiment of the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0123] like Figure 7 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., which is communicatively connected to the at least one processor 11. The memory stores a computer program that can be executed by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. Various programs and data required for the operation of the electronic device 10 can also be stored in the RAM 13. The processor 11, ROM 12, and RAM 13 are connected to each other via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0124] Multiple components in the electronic device 10 are connected to the I / O interface 15, including an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0125] The processor 11 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors that run machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the robot control method.

[0126] In some embodiments, the robot control method can be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the robot control method described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to execute the robot control method in any other suitable manner (e.g., via firmware).

[0127] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0128] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the computer program is executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer program may be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0129] In the context of the present invention, computer-readable storage media can be tangible media that can contain or store a computer program for use with an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. Computer-readable storage media can include but are not limited to electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, computer-readable storage media can be machine-readable signal media. More specific examples of machine-readable storage media can include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0130] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0131] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0132] A computing system may include clients and servers. The clients and servers are typically remote from each other and typically interact via a communication network. This client-server relationship arises through computer programs running on the respective computers, creating a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host. This server is a hosting product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosting and VPS services.

[0133] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the present invention can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved. This is not limited herein.

[0134] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A robot control method, characterized in that: include: Receiving decision information; wherein the decision information is generated based on the robot's state information and decision instructions; generating, based on the decision information, unconstrained trajectory planning information for controlling the robot; wherein the unconstrained trajectory planning information includes planning information for at least two motion dimensions of the robot, each motion dimension being pre-associated with a moving component of the robot; Obtaining constraints for each motion dimension, and determining constrained trajectory planning information based on the constraints and the unconstrained trajectory planning information; wherein the motion dimension is the motion angle of the moving part of the robot; generating control instructions for each motion dimension of the robot based on the constrained trajectory planning information to control the robot; The decision information includes expected behavior information generated by the decision and decision behavior history information; Accordingly, generating unconstrained trajectory planning information for controlling the robot based on the decision information includes: Unconstrained trajectory planning information for controlling the robot is generated based on expected behavior information generated by the robot's decision and decision behavior history information.

2. The method according to claim 1, characterized in that After generating unconstrained trajectory planning information for controlling the robot according to the decision information, the method further comprises: Based on the generated unconstrained trajectory planning information, different trajectories are scored according to a scoring strategy to obtain optimal trajectory information.

3. The method according to claim 1, characterized in that The method further comprises: According to the triggering conditions, unconstrained trajectory planning information for controlling the robot is generated; wherein the triggering conditions include timed start.

4. The method according to claim 1, wherein The constraints include position constraints, velocity constraints, acceleration constraints, collision constraints, and joint limit constraints.

5. The method according to claim 1, characterized in that The control instructions at least include eye movement control instructions, head movement control instructions, body movement control instructions and foot movement control instructions.

6. The method according to claim 5, characterized in that Based on the constrained trajectory planning information, generating eye movement control instructions for the robot, including: The position of the eye focus point is generated according to the eye movement dimension, the head movement control command and the body movement control command.

7. A robot control device, characterized in that: include: An information receiving module, configured to receive decision information; wherein the decision information is generated based on the robot's state information and decision instructions; a trajectory planning information generation module, configured to generate unconstrained trajectory planning information for controlling the robot based on the decision information; wherein the unconstrained trajectory planning information includes planning information for at least two motion dimensions of the robot, each motion dimension being pre-associated with a moving component of the robot; a trajectory planning information determination module, configured to obtain constraints for each motion dimension and determine constrained trajectory planning information based on the constraints and the unconstrained trajectory planning information; wherein the motion dimension is the motion angle of the robot's moving parts; a control module, configured to generate control instructions for each motion dimension of the robot based on the constrained trajectory planning information, so as to control the robot; The decision information includes expected behavior information generated by the decision and decision behavior history information; Accordingly, the trajectory planning information generation module is specifically used to: Unconstrained trajectory planning information for controlling the robot is generated based on expected behavior information generated by the robot's decision and decision behavior history information.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform the robot control method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the robot control method according to any one of claims 1 to 6 when executed.

Images