Method for controlling and evaluating robot and robot

Abstract

Embodiments of the present disclosure provide a method of controlling a robot, a method of evaluating a robot, a robot, a device of controlling a robot, and a non-volatile computer readable storage medium. The robot comprises a body assembly, upper limb assemblies and leg assemblies, and the method comprises the steps that in a simulation space, a virtual body corresponding to the robot and a virtual body corresponding to a to-be-transferred object are generated; determining a first expected trajectory of movement of a virtual body corresponding to the to-be-transferred object; on the basis of the first expected trajectory, a second expected trajectory of the movement is determined, and in the process that a virtual body corresponding to the robot moves along the second expected trajectory, virtual acting force is applied to the virtual body corresponding to the to-be-transferred object; based on the second expected trajectory, first control information used for controlling a virtual body corresponding to the robot in the simulation space is determined; and determining second control information for controlling the robot in the physical space based on the first control information.

Description

Technical Field

[0001] The present application relates to the field of robotics technology, specifically to the field of robot planning and control technology, and in particular to a method for controlling a robot, a method for evaluating a robot, a robot, a device for controlling a robot, and a non-volatile computer-readable storage medium. Background Art

[0002] With the development of society, it is becoming increasingly important to develop advanced human living environment robots to help disabled elderly people. Compared with modifying the environment such as beds or using equipment with specific functions, the use of humanoid general-purpose robots has unique advantages in this scenario. Compared with relying solely on modifying the environment or special equipment, the use of general-purpose robots has the advantages of strong autonomy, good versatility, and more humanization. Robots can autonomously identify needs and respond. Using humanoid robots with joints such as arms and waists, they can flexibly complete various tasks without the need to design special equipment. The movement of robots can refer to human habits, making the elderly feel more natural. In addition, robots can also take on more additional functions such as communication and companionship in the future, and have stronger scalability.

[0003] However, due to the complex physical conditions of disabled elderly people, using robots to help disabled elderly people also presents challenges in kinematics and dynamics. Therefore, further improvements to robots are needed. Summary of the invention

[0004] In view of the above problems, the present disclosure provides a method and apparatus for controlling a robot, a robot, a device for controlling a robot, and a non-volatile computer-readable storage medium.

[0005] According to one aspect of the present disclosure, a method for controlling a robot is proposed, wherein the robot includes a body component, an upper limb component, and a leg component, and the method includes: generating a virtual body corresponding to the robot and a virtual body corresponding to an object to be transferred in a simulation space; determining a first expected trajectory of movement of the virtual body corresponding to the object to be transferred; based on the first expected trajectory, determining a second expected trajectory of the movement, wherein, during the movement of the virtual body corresponding to the robot along the second expected trajectory, a virtual force is applied to the virtual body corresponding to the object to be transferred so that the virtual body corresponding to the object to be transferred moves along the first expected trajectory; based on the second expected trajectory, determining first control information for controlling the virtual body corresponding to the robot in the simulation space; and based on the first control information, determining second control information for controlling the robot in a physical space.

[0006] According to another aspect of the present disclosure, a method for evaluating a robot is proposed, wherein the robot includes a body component, an upper limb component, and a leg component, and the method includes: generating a virtual body corresponding to the robot and a virtual body corresponding to an object to be transferred in a simulation space; determining a first expected trajectory of movement of the virtual body corresponding to the object to be transferred; based on the first expected trajectory, determining a second expected trajectory of the movement, wherein, during the movement of the virtual body corresponding to the robot along the second expected trajectory, a virtual force is applied to the virtual body corresponding to the object to be transferred, so that the virtual body corresponding to the object to be transferred moves along the first expected trajectory; based on the second expected trajectory, determining first control information for controlling the virtual body corresponding to the robot in the simulation space; and evaluating the robot based on the first control information.

[0007] According to another aspect of the present disclosure, a robot is proposed, comprising: a body component, an upper limb component and a leg component; a controller, wherein the controller is arranged on the robot and is capable of executing the above-mentioned method.

[0008] According to another aspect of the present disclosure, a device for controlling a robot is provided, comprising: a processor; and a memory, wherein the memory stores a computer executable code, and when the computer executable code is run by the processor, the above method is executed.

[0009] According to another aspect of the present disclosure, a non-volatile computer-readable storage medium is provided, on which executable codes are stored. When the executable codes are executed by a processor, the processor executes the above method.

[0010] Compared with the traditional robot scheme for assisting the transfer of disabled elderly people, the robot control method provided by the embodiment of the present disclosure enables the robot to autonomously transfer the disabled elderly from the bed to the wheelchair. In addition, the embodiment of the present disclosure fully considers the configuration of the robot and the physical data of the disabled elderly, and provides a control method to control the humanoid robot to assist the disabled elderly to transfer from the bedridden state to the wheelchair. By simulating the robot and the disabled elderly in the simulation space, and simulating the entire process, the safety of the entire transfer process is ensured, so that the humanoid robot can autonomously complete the task of assisting the elderly in position transfer without human intervention or operation (but someone needs to monitor to ensure safety when necessary), and improve the stability and comfort of the humanoid robot holding the disabled elderly. Compared with other control schemes, the control scheme provided by this palace has significantly excellent control effects. Compared with traditional evaluation schemes, the evaluation scheme of the robot control method provided by the embodiment of the present disclosure can intuitively determine whether the performance of the robot can meet the task of assisting the transfer of disabled elderly people. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some exemplary embodiments of the present disclosure. For ordinary technicians in this field, other drawings can also be obtained based on these drawings without creative work. The following drawings are not deliberately scaled according to the actual size, and the focus is on illustrating the main purpose of the present disclosure.

[0012] Figure 1 is a schematic diagram illustrating a robot according to an embodiment of the present disclosure.

[0013] Figure 2 A schematic diagram showing the process of a real nurse picking up a disabled elderly person.

[0014] Figure 3 is a flowchart illustrating a control method for a robot according to an embodiment of the present disclosure.

[0015] Figure 4 is a schematic diagram showing a virtual body corresponding to a robot according to an embodiment of the present disclosure next to a virtual body corresponding to a first plane.

[0016] Figure 5 is a schematic diagram showing a virtual body corresponding to an object to be transferred according to an embodiment of the present disclosure.

[0017] Figure 6 is a schematic diagram showing the contact virtual bodies of the virtual body corresponding to the object to be transferred according to an embodiment of the present disclosure.

[0018] Figure 7 The diagram is a schematic diagram showing a process in which the elderly virtual body goes from lying flat on the virtual body corresponding to the first plane to sitting vertically on the virtual body corresponding to the first plane according to an embodiment of the present disclosure. The process shows how to determine the first half of the first desired trajectory of the elderly virtual body movement.

[0019] Figure 8 The present invention is a schematic diagram showing the force analysis of the elderly virtual body from sitting vertically on the virtual body corresponding to the first plane to sitting vertically on the virtual body corresponding to the second plane according to an embodiment of the present invention. The process shows how to determine the second half of the first desired trajectory of the elderly virtual body movement.

[0020] Fig. 9 is a schematic diagram showing that the end effector of the robot virtual body according to an embodiment of the present disclosure applies a force and follows the trajectory movement of the point where the force is applied.

[0021] Fig.10It is a schematic diagram showing the complete process of interaction between a robot virtual body and an elderly virtual body according to an embodiment of the present disclosure.

[0022] Fig.11 The schematic diagram shows the variation of the driving torque required for each joint of the robot virtual body over time during the process in which the robot virtual body rotates the elderly virtual body from a supine position to a side-lying position in the simulation space.

[0023] Fig.12 A schematic diagram showing the variation of the driving torque required for each joint of the robot virtual body over time during the process in which the robot virtual body rotates the elderly virtual body from a side-lying posture to a sitting posture in the simulation space.

[0024] Fig.13 The schematic diagram shows the variation of the driving torque required for each joint of the robot virtual body over time in the process of the robot virtual body carrying the elderly virtual body from the bed virtual body to the wheelchair virtual body in the simulation space. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solution and advantages of the present disclosure more obvious, the exemplary embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all the embodiments of the present disclosure, and it should be understood that the present disclosure is not limited to the exemplary embodiments described here.

[0026] As shown in the present disclosure and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0027] Although the present disclosure makes various references to certain modules in the apparatus according to the embodiments of the present disclosure, any number of different modules can be used and run on the user terminal and / or server. The modules are only illustrative, and different aspects of the apparatus and method can use different modules.

[0028] Flowcharts are used in this disclosure to illustrate the operations performed by the methods and devices according to the embodiments of the present disclosure. It should be understood that the preceding or following operations are not necessarily performed precisely in order. On the contrary, various steps may be processed in reverse order or simultaneously as required. At the same time, other operations may also be added to these processes, or one or more operations may be removed from these processes.

[0029] To facilitate description of the present disclosure, concepts related to the present disclosure are introduced below.

[0030] The robot disclosed in the present invention is a robot that uses legs to move. It uses animals as bionic objects, and its purpose is to simulate the movement form of animals and replicate the movement ability of animals based on engineering technology and scientific research results. The robot has a strong adaptability to various environments (including structured environments (such as roads, railways, treated flat roads, etc.) and unstructured environments (such as mountains, swamps, rugged roads, etc.). It can adapt to various changes in terrain, climb over higher obstacles, and can effectively reduce loads and improve the energy utilization efficiency of the system. Robots can be divided into single-legged, bipedal, quadrupedal, hexapod, octapodal, etc. according to the number of legs. Among them, humanoid robots have super strong movement ability, better static stability than bipedal robots, and simpler and more flexible movement than hexapod and octapod robots. Therefore, humanoid robots are a common choice for studying robots. The gait of a humanoid robot is the coordination relationship of its four legs in time and space in order for the humanoid robot to be able to move continuously. The gait of a humanoid robot is derived from the gait of a quadrupedal mammal (eg, a puppy), which may include but is not limited to the following simplified three forms: walk, trot, and bound.

[0031] The method for controlling a robot disclosed in the present invention may be based on artificial intelligence (AI). Artificial intelligence is a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. For example, for a method for controlling a robot based on artificial intelligence, it can plan the robot's motion trajectory and gait in a way similar to how humans guide the movement of living animals, making the robot's movement more flexible and bionic. By studying the design principles and implementation methods of various intelligent machines, artificial intelligence enables the method for controlling a robot disclosed in the present invention to automatically and efficiently design the robot's subsequent motion trajectory and gait based on the robot's current motion state.

[0032] In summary, the solutions provided by the embodiments of the present disclosure involve technologies such as artificial intelligence and machine learning. The embodiments of the present disclosure will be further described below in conjunction with the accompanying drawings.

[0033] Figure 1 is a schematic diagram illustrating a robot according to an embodiment of the present disclosure.

[0034] like Figure 1As shown, the quadruped humanoid robot is an intelligent device that can approximately imitate animals. It has the ability to walk and move flexibly in complex terrains and environments, so it has a wide range of applications in many application scenarios. For example, in an emergency, the humanoid robot can be used for tasks such as search and rescue, detection, and demolition. It can work in places that are difficult for humans to reach, such as mountains, deserts, and forests. In daily life, the humanoid robot can be used as an intelligent companion. It can interact with humans in the home environment and can also adapt to different terrains of the home, such as slopes, steps, etc. The robot is not only highly intelligent and flexible, but also highly adaptable and practical.

[0035] The exemplary robot includes multiple components, such as a head component (optional), a body component, and a leg component, etc. Of course, the present disclosure is not limited thereto.

[0036] The exemplary head assembly may be equipped with a visual camera, a voice interaction system and other perception components for environmental interaction and human-computer interaction. In some examples, the head assembly also includes a neck rotation assembly to perform pitch and left-right rotation of the head to obtain a wider field of view. Of course, the present disclosure is not limited to this.

[0037] One end of the head component is connected to the body component of the robot. The exemplary body component may contain components such as a battery, a computing system, and a control system to provide energy and computing support for the robot's movement.

[0038] In addition, exemplary body components include but are not limited to upper limb components, waist components, and hip components. Of course, the present disclosure is not limited to this.

[0039] The left and right ends of an exemplary torso assembly include symmetrical upper limb assemblies. An exemplary upper limb assembly includes a shoulder joint assembly, an arm assembly, and an end effector. The shoulder joint assembly has six degrees of freedom to enable complex movements of the arm assembly, such as rotation and lifting in all directions. One end of the arm assembly is connected to the shoulder joint assembly, and the other end is connected to the end effector. Optionally, the connection between the arm assembly and the end effector includes a motor to enable the end effector to move along four degrees of freedom. Among them, the optional end effector can be a manipulator of any form, which has a rich degree of freedom to imitate human movements such as grasping, pushing, and supporting objects of various shapes.

[0040] The waist component and the hip component are used to connect the leg component and the torso component. A motor is installed inside the waist component to enable the torso component to pitch and rotate so that the robot can imitate the human bending action. A motor is installed inside the hip component to rotate the leg component. The posture of the leg component can be changed by controlling the motor.

[0041] The leg assembly includes four mechanical legs, and the example robot can move based on the four mechanical legs. Wherein, the four mechanical legs are two inner legs (shown in gray) and two outer legs (shown in white). Each mechanical leg includes a retractable rigid assembly and a driving wheel, one end of the retractable rigid assembly of the inner leg is connected to the body assembly of the robot, such as being connected to the hip, and the other end is connected to the driving wheel. The optional inner leg and the outer leg are controlled by different motors respectively, so that the relative positions of the inner leg and the outer leg can be changed to be more suitable for the living environment. The retractable rigid assembly can be extended and shortened. The leg motor is used to drive the mechanical leg to walk, and when encountering an obstacle during walking, the retractable rigid assembly can cross the obstacle by extending or shortening. The driving wheel is used to carry out wheeled motion.

[0042] The telescopic rigid component includes a main leg segment, a telescopic leg segment and a telescopic driving mechanism. The main leg segment is connected to the leg motor. The telescopic leg segment is slidably connected to the main leg segment, and one end of the telescopic leg segment away from the leg motor is connected to the driving wheel assembly. The telescopic driving mechanism is connected to the main leg segment and the telescopic leg segment respectively, and the telescopic driving mechanism is used to drive the telescopic leg segment to slide. When the telescopic driving mechanism drives the telescopic leg segment to slide in a direction away from the leg motor, the mechanical leg is extended; when the telescopic driving mechanism drives the telescopic leg segment to slide in a direction close to the leg motor, the mechanical leg is shortened. The embodiments of the present disclosure do not limit the relative positional relationship between the main leg segment and the telescopic leg segment. In some examples, one side of the main leg segment is slidably connected to one side of the telescopic leg segment. In other examples, the main leg segment has a receiving cavity, a part of the telescopic leg segment is located in the receiving cavity, and the telescopic leg segment can be telescoped relative to the receiving cavity. The embodiments of the present disclosure do not limit the type of the telescopic driving mechanism. In some examples, the telescopic driving mechanism is a screw nut mechanism, a synchronous belt mechanism, a gear rack mechanism, a hydraulic rod mechanism or an electric push rod mechanism.

[0043] It should be noted that the robot can also be equipped with a variety of sensors, such as IMU (Inertial Measurement Unit) sensors and joint angle encoders, etc. Among them, the IMU sensor can provide the robot's acceleration and posture information in real time, and the joint angle encoder can provide the joint angle information of each joint of the robot in real time (such as the angle of the joint angle, angular velocity feedback value, etc.). Under the control of the multiple motors mentioned above, the example robot can already simulate real human actions such as running, jumping, climbing stairs, etc.

[0044] Figure 1 The humanoid robot in the video can autonomously identify the specific needs of disabled elderly people and respond flexibly. Compared with modifying the environment such as beds or using functionally specific equipment, the use of humanoid general-purpose robots has unique advantages in this scenario. Figure 1By configuring the motors of each joint in the humanoid robot, Figure 1 The humanoid robot in the system can complete various types of actions, such as delivering items, pushing wheelchairs, assisting walking and going up and down stairs, preparing meals, opening doors, and other types of nursing tasks, without the need to design special equipment for each function and scenario, which improves the living environment and saves costs. In addition, the joint motion curve, speed change, and force output of the humanoid robot can be optimized through algorithms to approach the normal human motion pattern, so that it can refer to human habits and make the elderly feel more natural and comfortable.

[0045] Specifically, Figure 2 The figure shows the process of a real nurse picking up a disabled elderly person. Figure 2 As shown in the figure, the nurse's arms pass through the elderly's armpits, hug the elderly, and then use her arms to apply an oblique upward force to the elderly to lift the elderly off the bed, and then carry the elderly to the wheelchair. The humanoid robot attempts to simulate the entire process of a nurse holding and supporting a disabled elderly person in order to transfer the bedridden disabled elderly person to a wheelchair.

[0046] However, due to the complex physical conditions of disabled elderly people, Figure 1 Humanoid robot simulation Figure 2 The process of holding and supporting disabled elderly people also faces many challenges in kinematics and dynamics. In order to improve the stability, comfort and safety of humanoid robots holding and supporting disabled elderly people, the embodiments of the present disclosure fully consider the physical data of disabled elderly people, and provide a control method to control the humanoid robot to assist disabled elderly people to transfer from bedridden state to wheelchair, so that the humanoid robot can autonomously complete the task of assisting the elderly in position transfer without manual intervention or operation (but someone needs to monitor to ensure safety when necessary). The embodiments of the present disclosure also provide an evaluation method to determine whether the humanoid robot can complete the task, such as evaluating whether the parameters such as the degree of freedom configuration, joint size, joint motion range, workspace, interference, joint motion speed, joint drive torque of the humanoid robot meet the requirements of the task of assisting the elderly in position transfer, so as to provide a reference for the design of the humanoid robot. Through the evaluation method according to the embodiments of the present disclosure, it can also be determined whether the joint force output of the humanoid robot is suitable for the elderly, whether the holding and supporting process is smooth, whether the movement is in line with the habits of the elderly, etc., so as to avoid causing harm to the elderly or damage to the robot during use, thereby ensuring the health and comfort of the elderly.

[0047] Therefore, in response to the above problems, according to one aspect of the present disclosure, a method for controlling a robot is proposed, wherein the robot includes a body component, an upper limb component and a leg component, and the method includes: generating a virtual body corresponding to the robot and a virtual body corresponding to an object to be transferred in a simulation space; determining a first expected trajectory of the movement of the virtual body corresponding to the object to be transferred; based on the first expected trajectory, determining a second expected trajectory of the movement, wherein, during the movement of the virtual body corresponding to the robot along the second expected trajectory, a virtual force is applied to the virtual body corresponding to the object to be transferred, so that the virtual body corresponding to the object to be transferred moves along the first expected trajectory; based on the second expected trajectory, determining first control information for controlling the virtual body corresponding to the robot in the simulation space; and based on the first control information, determining second control information for controlling the robot in a physical space.

[0048] According to another aspect of the present disclosure, a method for evaluating a robot is proposed, wherein the robot includes a body component, an upper limb component, and a leg component, and the method includes: generating a virtual body corresponding to the robot and a virtual body corresponding to an object to be transferred in a simulation space; determining a first expected trajectory of movement of the virtual body corresponding to the object to be transferred; based on the first expected trajectory, determining a second expected trajectory of the movement, wherein, during the movement of the virtual body corresponding to the robot along the second expected trajectory, a virtual force is applied to the virtual body corresponding to the object to be transferred, so that the virtual body corresponding to the object to be transferred moves along the first expected trajectory; based on the second expected trajectory, determining first control information for controlling the virtual body corresponding to the robot in the simulation space; and evaluating the robot based on the first control information.

[0049] According to another aspect of the present disclosure, a robot is proposed, comprising: a body component, an upper limb component and a leg component; a controller, wherein the controller is arranged on the robot and is capable of executing the above-mentioned method.

[0050] Next, refer to Figure 3 The various embodiments of the present disclosure are described in an integrated manner.

[0051] Figure 3 is a flow chart illustrating a control method 30 for a robot according to an embodiment of the present disclosure.

[0052] The control method 30 for a robot according to an embodiment of the present disclosure may include: Figure 3As described above, the robot includes a body component, an upper limb component and a leg component. The body component, the upper limb component and the leg component can refer to Figure 1 The relevant description will not be repeated in this disclosure.

[0053] In step S301, a virtual body corresponding to the robot and a virtual body corresponding to the object to be transferred are generated in a simulation space.

[0054] For example, a simulation space is a virtual environment simulated by a computer. The simulation space is usually a three-dimensional space, which provides a motion environment for a virtual body by providing three-dimensional coordinates corresponding to the physical space. The physical space described in the embodiment of the present disclosure is a three-dimensional space where matter objectively exists. It has continuity and uniformity, but also contains diversity and complexity, which makes it possible for a real robot to transfer a disabled elderly person from a bed to a wheelchair.

[0055] Optionally, in some embodiments, the simulation space may also set virtual constraints such as gravity acceleration that correspond to the constraints of the real physical space to conform to the real physical environment. A virtual body refers to a virtual model used to describe a real physical object. In the simulation space, real objects can be simulated and emulated by virtual bodies. For example, elements such as rigid body joints can be used to describe the various components that make up a robot. In order to make the virtual body as consistent as possible with the behavior of the real object, real objects such as objects, human bodies, animals in the physical space can be mapped into the simulation space to obtain virtual bodies corresponding to these real objects. Of course, the present disclosure is not limited to this.

[0056] For example, a virtual body corresponding to a robot can be used for simulation and Figure 1 The robot shown in FIG. The virtual body corresponding to the robot includes Figure 1The robot has the same or similar geometric features or dynamic features. Hereinafter, the virtual body corresponding to the robot may be referred to as the robot virtual body. For example, the body component of the robot can be described by a non-deformable geometric body, which is also called a rigid geometric body. A rigid body geometric body can have a corresponding motion constraint relationship with other rigid body geometric bodies, which is also called a rigid body constraint. In some embodiments, the shoulder joint component can be described by a rigid sphere to simulate the relative rotation of the six degrees of freedom of the geometric body corresponding to the upper arm relative to the geometric body corresponding to the trunk. In other embodiments, in the simulation space, the geometric body corresponding to the shoulder joint component can also form a rotation pair with the geometric body corresponding to the trunk. In addition, the waist component and the hip component can also be described by a geometric body forming a rotation pair, wherein the geometric body corresponding to the waist component can rotate relative to the geometric body corresponding to the hip component in two degrees of freedom of pitch and yaw. For another example, the leg component of the robot can be described by a geometric body that is both rigid and retractable, which is also called a retractable rigid geometric body. Thus, the inner and outer legs of the virtual body corresponding to the robot in the simulation space can achieve a certain degree of contraction or extension. Optionally, in order to ensure the authenticity of the simulation, the geometric structure, mass, moment of inertia and other parameters corresponding to the above geometric body can be highly consistent with the corresponding parts of the robot in the real physical space, so that more accurate expected trajectory and control information can be obtained in the subsequent process. Of course, the present disclosure is not limited to this.

[0057] For another example, the virtual body corresponding to the object to be transferred can be used to simulate a disabled elderly person or patient. Hereinafter, the virtual body corresponding to the object to be transferred may be referred to as an elderly virtual body. Figure 1 Similar to the simulation of the robot in the figure, the virtual body of the elderly can also be described by the connection and combination of rigid geometric bodies or retractable rigid geometric bodies. The joint configuration of the virtual body of the elderly is the same or similar to the configuration of the various components of the virtual body corresponding to the robot. However, considering the characteristics of the disabled elderly or patients, some specific joints will be limited to be movable within a certain small range to simulate the disabled state. Since the joint geometry of the virtual body of the elderly can only move within a given reasonable range rather than completely free movement, the virtual body of the elderly can only remain stable in some postures. For example, the rotation angle of the waist geometry, hip geometry, and knee joint geometry of the virtual body of the elderly can be limited or fixed between 0 and 90 degrees, so that the virtual body of the elderly can simulate the lying posture and vertical sitting posture of the elderly and remain stable in such postures. Optionally, the relevant parameters of the height, weight, and mass distribution of the virtual body of the elderly also need to be adjusted accordingly to simulate the state of the disabled elderly as realistically as possible. Of course, the present disclosure is not limited to this.

[0058] In addition, some environmental models can be further arranged in the simulation space. The environmental model includes a virtual body corresponding to the first plane and a virtual body corresponding to the second plane. The virtual body corresponding to the first plane can be used to simulate a bed, and the virtual body corresponding to the second plane can be used to simulate a wheelchair. At the same time, the virtual body corresponding to the first plane and the virtual body corresponding to the second plane can be placed on the virtual body corresponding to the third plane. The virtual body corresponding to the third plane can be used to simulate the ground. In the initial state, the virtual body corresponding to the object to be transferred will be placed on the virtual body corresponding to the first plane. In an ideal situation, the virtual body corresponding to the object to be transferred will be transferred from the virtual body corresponding to the first plane to the virtual body corresponding to the second plane, and during the entire transfer process, the virtual body corresponding to the object to be transferred will not be subjected to virtual forces beyond its tolerance range. Of course, the present disclosure is not limited to this.

[0059] In step S302, a first expected trajectory of movement of the virtual body corresponding to the object to be transferred is determined.

[0060] Optionally, the first expected trajectory refers to the motion trajectory of the virtual body corresponding to the object to be transferred under the action of one or more virtual forces, which include but are not limited to gravity, the contact force applied by the virtual body corresponding to the first plane to the object to be transferred, the contact force applied by the virtual body corresponding to the robot to the object to be transferred, the contact force applied by the virtual body corresponding to the second plane to the object to be transferred, and the contact force applied by the virtual body corresponding to the third plane to the object to be transferred, etc. The first expected trajectory may include at least one of the position information, velocity information, acceleration information, angular velocity information, and angular acceleration information of each joint or foot end of the virtual body corresponding to the object to be transferred at each time step. Of course, the present disclosure is not limited to this.

[0061] Optionally, the first desired trajectory may be determined so that the virtual body corresponding to the object to be transferred is transferred from the virtual body corresponding to the first plane to the virtual body corresponding to the second plane. That is, the object to be transferred will be transferred from the virtual body corresponding to the first plane to the virtual body corresponding to the second plane under the action of the virtual forces provided by each virtual body during the movement along the first desired trajectory. Of course, the present disclosure is not limited to this.

[0062] In one example, the first expected trajectory can be represented by a time series numerical sequence composed of the position information of the reference points corresponding to each time step. Optionally, the above-mentioned reference points include the center of the left and right shoulder joints, the center of the hip joint, the center of mass, etc. of the real disabled elderly, which are the positions where the real disabled elderly may contact the end effector of the robot, the bed surface, the surface of the wheelchair, etc. Optionally, each element in the time series numerical sequence can have 3 dimensions, which respectively represent the position of the reference point corresponding to a certain time step in the x-axis direction, the position of the joint in the y-axis direction, and the position of the joint in the z direction (gravity direction). Of course, the first expected trajectory can also be represented by other data structures, and the present disclosure is not limited to this.

[0063] In step S303, based on the first expected trajectory, a second expected trajectory of the movement of the virtual body corresponding to the robot is determined, wherein, during the movement of the virtual body corresponding to the robot along the second expected trajectory, a virtual force is applied to the virtual body corresponding to the object to be transferred, so that the virtual body corresponding to the object to be transferred moves along the first expected trajectory.

[0064] Optionally, the second expected trajectory refers to the expected motion trajectory of the robot when performing a task in the simulation space. The expected trajectory may include at least one of the position information (e.g., three-dimensional coordinate information in the simulation space) expected to be possessed by each joint, foot end, and / or end effector of the robot at each time step, velocity information, acceleration information, angular velocity information, and angular acceleration information. Similarly, the second expected trajectory can be represented by a time sequence composed of relevant information of each joint corresponding to each time step. Of course, the second expected trajectory can also be represented by other data structures, and the present disclosure is not limited thereto.

[0065] Optionally, in the above assumptions, the contact position of the robot with the object to be transferred in the simulation space is mostly represented by the contact of two rigid geometric bodies. The rigid geometric bodies do not deform, so the contact effect between the rigid geometric bodies will be simplified to the virtual repulsive force generated by the tiny penetration between the two rigid geometric bodies. That is, the two rigid geometric bodies will only generate a virtual repulsive force when there is a tiny penetration between each other, and when there is no penetration, the virtual contact force between the two is considered to be zero. Based on the above assumptions, the second expected trajectory can be derived from the first expected trajectory.

[0066] It is worth noting that the time period corresponding to the first desired trajectory and the second desired trajectory will be as long as possible, corresponding to the robot picking up the elderly as slowly as possible in the real physical space to avoid causing harm to the robot or the elderly. The process of determining the first desired trajectory and the second desired trajectory will be described in detail with reference to the various figures, and this disclosure will not be repeated here.

[0067] In step S304, based on the second expected trajectory, first control information for controlling a virtual body corresponding to the robot in the simulation space is determined.

[0068] Optionally, after the second desired trajectory is determined, the control information of each component of the robot (i.e., the output torque of each joint motor in the simulation space) can be planned by the robot inverse kinematics method, thereby obtaining the first control information for controlling the virtual body corresponding to the robot in the simulation space. Inverse kinematics is a method for calculating the joint angle of the robot according to the desired position and posture of the foot end of the robot. Using inverse kinematics, the target motion control of the robot can be achieved.

[0069] Optionally, the control information for controlling the joint of the virtual body corresponding to the robot includes but is not limited to all information that can control the joint motor of the robot in the simulation space. It can be the acceleration of each joint motor or the torque of the joint motor. Although mathematically speaking, these two physical quantities do not have much difference as the control information for controlling the rotation of the joint motor, in the actual physical system, it is not that these two physical quantities can be accurately measured. Therefore, those skilled in the art can select data test results better according to the specific circumstances in the experiment, and the physical quantity that is more in line with the model is calculated later.

[0070] The process of planning the control information of each component of the robot through the method of robot kinematics can be mathematically described by the following formula (1).

[0071] q=IK(robot,p) (1)

[0072] Where q is the vector of the joint angle values ​​of the virtual body corresponding to the robot in the simulation space, and p is the position vector of the geometric body corresponding to the end effector in the simulation space. Formula (1) establishes a mapping from the space corresponding to each virtual joint to the Cartesian space. For example, the joint angles that meet the target position and posture can be solved based on the dynamic model corresponding to the robot by using numerical methods or analytical methods, and then the control information for controlling each joint can be determined using the joint angle information so that the robot can perform the target action.

[0073] Optionally, when there is no analytical solution for the inverse kinematics mapping, a numerical approximation method can be used. Given the position vector p of the geometric body corresponding to the end effector in the simulation space, an iterative algorithm is used to search for the corresponding joint angle solution q that satisfies the constraints. Many kinematic simulation software have built-in inverse kinematics solvers that can automatically perform iterative calculations without manual derivation and programming. After obtaining the inverse kinematics solution, the kinematic solution q in the joint space corresponds to the joint control amount that enables the end effector to reach the target posture p. From this, dynamic analysis can be continued, the joint driving force / torque can be calculated, the motion control can be completed, and the first control information in the simulation space can be obtained. Of course, the present disclosure is not limited to this.

[0074] In step S305, second control information for controlling the robot in a physical space is determined based on the first control information.

[0075] Optionally, the mapping relationship between the simulation space and the physical space can be considered to map the first control information into the second control information. In this process, it may be necessary to further consider the dynamic model, inertial characteristics, mechanical characteristics, etc. of the actual robot, and adjust the second control information during the conversion process to obtain more accurate motor speed and torque. Optionally, in some embodiments, the motor speed and torque will be further converted into standard drive signals such as PWM, and these drive signals will be used as the final second control information. In some embodiments, it is also necessary to consider some control factors unique to the physical space, such as closed-loop control and other constraints, to make up for the deviation between the simulation space and the physical space, so that the robot can eventually move according to a predetermined trajectory and complete the task of transferring disabled elderly people.

[0076] The motion trajectory of the real robot controlled by the second control information and moving is roughly consistent with the second expected trajectory. It is worth noting that the "roughly consistent" described in the present disclosure means that when the real machine is actually tested, the actual trajectory of the end effector of the robot is very close to or even the same as the second expected trajectory in the simulation space. Due to the limitations of the joint motor performance and the inaccuracy of the simplified model, the joint motor often cannot output the ideal torque. At the same time, considering the changes in the external environment (for example, the sudden appearance of wind and other disturbances), it is often difficult to control the robot to completely comply with the second expected trajectory. Therefore, the embodiments of the present disclosure only require that the difference between the mapping of the actual trajectory and the second expected trajectory in the physical space is small enough.

[0077] Compared with the traditional robot scheme for assisting the transfer of disabled elderly people, the robot control method provided by the embodiment of the present disclosure enables the robot to autonomously transfer the disabled elderly from the bed to the wheelchair. In addition, the embodiment of the present disclosure fully considers the configuration of the robot and the physical data of the disabled elderly, and provides a control method to control the humanoid robot to assist the disabled elderly to transfer from the bedridden state to the wheelchair. By simulating the robot and the disabled elderly in the simulation space and simulating the entire process, the safety of the entire transfer process is ensured, so that the humanoid robot can autonomously complete the task of assisting the elderly in position transfer without human intervention or operation (but someone needs to monitor to ensure safety when necessary), and improve the stability and comfort of the humanoid robot holding the disabled elderly. Compared with other control schemes, the control scheme provided by this palace has a significantly excellent control effect.

[0078] Next, refer to Figures 4 to 13 Some details of various embodiments of the present disclosure are further described.

[0079] First, refer to Figures 4 to 6 Some details of implementing step S301 are further described, that is, an example process of how to generate a virtual body corresponding to the robot and a virtual body corresponding to the object to be transferred in the simulation space.

[0080] Figure 4 is a schematic diagram showing a virtual body corresponding to a robot according to an embodiment of the present disclosure next to a virtual body corresponding to a first plane. Figure 5 is a schematic diagram showing a virtual body corresponding to an object to be transferred according to an embodiment of the present disclosure. Figure 6 is a schematic diagram showing the contact virtual bodies of the virtual body corresponding to the object to be transferred according to an embodiment of the present disclosure.

[0081] like Figure 4 As shown, in step S301, the three-dimensional model corresponding to the robot can be imported into the simulation space to generate a virtual body corresponding to the robot. Specifically, the three-dimensional model of the robot can be constructed by professional three-dimensional CAD modeling software, so that its geometric shape and even internal structure are highly restored to the physical robot system. Then export the general three-dimensional model file format, such as STL format.

[0082] Furthermore, a virtual body corresponding to the bed in the real physical space can also be set in the simulation space. Hereinafter, the virtual body corresponding to the bed is referred to as the virtual body corresponding to the first plane (or simply referred to as the bed virtual body). Optionally, there is a certain space under the virtual body corresponding to the first plane, which is convenient for placing the inner leg or outer leg of the robot. Further, a virtual body (not shown) corresponding to the ground of the real physical space and a virtual body (not shown) corresponding to the wheelchair can also be set in the simulation space. Hereinafter, the virtual body corresponding to the wheelchair is referred to as the virtual body corresponding to the second plane (or simply referred to as the wheelchair virtual body), and the virtual body corresponding to the ground is referred to as the virtual body corresponding to the third plane (or simply referred to as the ground virtual body). Optionally, the lower side of the virtual body corresponding to the second plane also has a certain space to facilitate the movement and stability of the robot. At the same time, the virtual body corresponding to the second plane can also include an armrest assembly. The armrest assembly can be raised and lowered as needed to avoid inconvenience to the robot and the elderly during the transfer process. Of course, the present disclosure is not limited to this.

[0083] exist Figure 4 In the figure, the outer leg of the virtual body corresponding to the robot is in contact with the edge of the virtual body corresponding to the bed and the driving wheel of the outer leg is located at the bottom of the bed, and the inner leg is at an angle of about 90 degrees to the outer leg and is located behind the robot. Later, the outer leg can be called the front leg, and the inner leg can be called the rear leg. As a result, an interactive virtual force is formed between the virtual body corresponding to the first plane and the virtual body corresponding to the robot to prevent the robot from falling forward. In addition, since the inner leg of the virtual body corresponding to the robot is at the back, the space between the outer legs of the robot is larger, which is more conducive to placing the leg components of the subsequent object to be transferred. Figure 4 Only one relative position between the virtual body corresponding to the robot and the virtual body corresponding to the first plane is shown. Those skilled in the art should understand that the present disclosure is not limited thereto.

[0084] like Figure 5 As shown, in step S301, the three-dimensional model corresponding to the elderly can be imported into the simulation space to generate a virtual body corresponding to the object to be transferred. Optionally, the three-dimensional model corresponding to the elderly may include but is not limited to a three-dimensional model consistent with the anatomical structure of a real disabled elderly. For example, the skull model of the elderly may include a skull, and the torso model may include a rib skeleton and a shoulder joint skeleton, etc. Thus, the virtual body corresponding to the object to be transferred can include the rigid structure of the head, torso, upper and lower limbs. And the joint components of the virtual body corresponding to the object to be transferred can include: a spherical shoulder joint, a uniaxial elbow joint, a spherical wrist joint, and a uniaxial hip joint, knee joint, ankle joint, etc.

[0085] like Figure 5As shown, the virtual body corresponding to the object to be transferred has an overall structure similar to that of a real disabled elderly person. At the same time. The design of the virtual body corresponding to the object to be transferred can also refer to the physiological data of the disabled elderly, and then set the range of motion constraints of each joint to simulate the situation of the elderly's inconvenience in movement. For example, the waist joint of a disabled elderly may only be allowed to move within the range of 0 to 90 degrees, and then only part of the center of gravity support function can be guaranteed. In addition, the overall height, weight, and mass distribution parameters of the virtual body of the object to be transferred can also be adjusted according to the parameter statistics of the real disabled elderly. In a specific implementation scenario, the three-dimensional model can also be adjusted for a specific disabled elderly or patient, and the virtual body of the object to be transferred can be correspondingly generated based on the adjusted three-dimensional model, thereby realizing individualized modeling. Those skilled in the art should understand that the present disclosure is not limited to this.

[0086] In some embodiments, the Figure 6 The virtual body corresponding to the object to be transferred is replaced by the contact virtual body shown. Each component of the contact virtual body can be a rigid geometric body in the simulation space. Compared with the irregular geometric body components in the virtual body corresponding to the object to be transferred, the shapes of the various components of the contact virtual body can be regular, for example, a sphere, a cylinder, a cube, etc. The various rigid geometric bodies of the contact virtual body can be filled with each component in the virtual body corresponding to the object to be transferred by a certain filling rule to obtain the corresponding rigid geometric body. Optionally, the size of these rigid geometric bodies can be roughly equal to or slightly larger than the size of the corresponding component of the virtual body corresponding to the object to be transferred, so that the rigid geometric body can roughly cover the surface of the relevant component of the virtual body corresponding to the object to be transferred.

[0087] The contact virtual body can be used to detect the contact effect when the virtual body corresponding to the object to be transferred contacts other virtual bodies. The contact effect can be simplified as the repulsive force generated by the slight penetration of the two objects. The two objects only generate repulsive force when they penetrate each other, and there is no contact force when they do not penetrate. Therefore, when using the contact virtual body, the subsequent calculation can include two parts, namely, determining whether there is penetration between rigid bodies and calculating the penetration depth between rigid bodies. If there is penetration between rigid bodies, it proves that there is a contact force between the two contact surfaces. If there is no penetration between rigid bodies, it proves that there is no contact force between the two contact surfaces. The penetration depth between rigid bodies can indicate the size of the force between rigid bodies.

[0088] Specifically, in step S303, if the penetration between the virtual body corresponding to the object to be transferred and the virtual body corresponding to the robot is directly used to determine the second expected trajectory, a large number of surface feature points may be required for calculation. In contrast, since the shapes of the components of the contact virtual body are regular, it can be determined whether penetration occurs and the depth of penetration, which can significantly improve the calculation efficiency. Although only the contact virtual body of the virtual body corresponding to the object to be transferred is shown, the contact virtual body of the virtual body corresponding to the robot can also be set accordingly, and the present disclosure is not limited to this.

[0089] Next, refer to Figures 7 and 8 Let's further describe some details of implementing step S302, that is, how to determine the first expected trajectory of the elderly virtual body movement.

[0090] Figure 7 The diagram is a schematic diagram showing a process in which the elderly virtual body goes from lying flat on the virtual body corresponding to the first plane to sitting vertically on the virtual body corresponding to the first plane according to an embodiment of the present disclosure. The process shows how to determine the first half of the first desired trajectory of the elderly virtual body movement. Figure 8 The present invention is a schematic diagram showing the force analysis of the elderly virtual body from sitting vertically on the virtual body corresponding to the first plane to sitting vertically on the virtual body corresponding to the second plane according to an embodiment of the present invention. The process shows how to determine the second half of the first desired trajectory of the elderly virtual body movement.

[0091] Considering the limited range of activities of the virtual body of the elderly, the virtual body of the elderly can be assisted in adjusting its posture from lying flat to sitting up in stages. At this time, the first expected trajectory can indicate at least one or a combination of the following items: the center of mass of the virtual body corresponding to the object to be transferred takes the center of one side of the trunk component as the center of the circle and half of the width of the trunk component as the radius, and rotates counterclockwise on the vertical plane, and stops after rotating 1 / 4 of the circle; the center of mass of the virtual body corresponding to the object to be transferred takes the bottom center of the trunk component as the center of the circle and half of the length of the trunk of the trunk component as the radius, and rotates counterclockwise on the vertical plane, and stops after rotating 1 / 4 of the circle; the center of mass of the virtual body corresponding to the object to be transferred moves diagonally upward for a distance, then rotates on the horizontal plane, and then moves diagonally downward for a distance.

[0092] First, the robot virtual body applies appropriate force to the elderly virtual body to enable it to complete the transition from lying flat (i.e., Figure 7 from position a) to side-lying (i.e., Figure 7 The first stage of posture change in posture b) is shown in Figure 1. At this time, the robot virtual body is needed to assist the elderly virtual body in rotating the shoulder and hip joints.

[0093] like Figure 7 As shown in the figure, when the old man's virtual body is lying completely flat, the end effector of the robot virtual body will apply a force diagonally upward to the left relative to the old man's virtual body on the right shoulder joint and right hip joint of the old man's virtual body. The diagonally upward straight line in the schematic diagram corresponding to posture a shows the point of action of the force exerted by the end effector of the robot virtual body on the old man's virtual body and the direction of the force exerted on the old man's virtual body when the old man's virtual body is lying completely flat.

[0094] As the posture of the old man's virtual body changes during the rotation process, the direction of the force applied by the end effector of the robot virtual body and the absolute position of the point of action will also change accordingly. The arc line diagonally pointing to the upper left in the schematic diagram corresponding to posture b shows the trajectory of the force points of the end effector of the robot virtual body on the right shoulder joint and right hip joint of the old man's virtual body during the whole process.

[0095] Next, based on the side-lying posture of the elderly virtual body, the end effector of the robot virtual body will continue to apply force to the left shoulder joint (lower shoulder) and right hip joint (upper hip) of the elderly virtual body to assist the elderly virtual body to change its posture from side-lying (i.e., Figure 7 c) to side-lying (i.e., Figure 7 The second stage posture change of the posture in d).

[0096] exist Figure 7 In the case of posture c in the figure, the end effector of the robot virtual body will apply a combined force upward and toward the end of the bed to the left shoulder joint of the elderly virtual body to drive the rotation of the shoulder and upper torso of the elderly virtual body. At the same time, the other end effector of the robot virtual body will apply a combined force downward and toward the end of the bed to the right hip joint of the elderly virtual body to drive the rotation of the lower limbs of the elderly virtual body. The straight line with an arrow in the schematic diagram corresponding to posture c shows the point of action of the force exerted by the end effector of the robot virtual body on the elderly virtual body and the direction of the force exerted on the elderly virtual body when the elderly virtual body is lying on its side. The arc with an arrow in the schematic diagram corresponding to postures c and d shows the trajectory of the point of action of the force exerted by the end effector of the robot virtual body on the elderly virtual body when the elderly virtual body changes from lying on its side to sitting up.

[0097] Although the above only describes the entire process of the elderly virtual body turning over to the left and then sitting up, those skilled in the art can also determine the entire process of the elderly virtual body turning over to the right and sitting up based on the same method.

[0098] The whole process of the old man's virtual body from lying flat to sitting up has been described in detail above. Based on this process, the motion trajectory of the old man's virtual body from lying flat to sitting up can be determined as the first half of the first expected trajectory of the virtual body movement corresponding to the object to be transferred in step S302. Specifically, considering that the posture of the old man's virtual body itself has not changed during the whole process, it has only been displaced and rotated as a whole. Therefore, by determining the change in the center of mass of the old man's virtual body during the whole process, the first half of the first expected trajectory of the old man's virtual body can be directly determined.

[0099] As Figure 7 As shown, the center of mass of the old man's virtual body first rotates counterclockwise on a vertical plane with the center of the left side of the old man's virtual body as the center of the circle and half of the width of the old man's virtual body as the radius, and stops after rotating 1 / 4 of a circle. Then, the center of mass of the old man's virtual body rotates counterclockwise on a vertical plane with the center of the hip joint of the old man's virtual body as the center of the circle and half of the length of the torso of the old man's virtual body as the radius, and stops after rotating 1 / 4 of a circle. Of course, the present disclosure is not limited to this.

[0100] Next, refer to Figure 8 . Figure 8 The process can be briefly described as follows: the robot virtual body picks up the old man's virtual body from the bed virtual body, then the robot virtual body turns to the wheelchair virtual body, and then places the old man's virtual body on the wheelchair virtual body. In the whole process, in addition to the force exerted by the robot virtual body on the old man's virtual body, it also includes the support force exerted by the bed virtual body on the old man's virtual body, the support force exerted on the old man's virtual body, and the support force exerted by the wheelchair virtual body on the old man's virtual body.

[0101] Taking into account the physical condition and comfort of the disabled elderly, the end effector of the robot virtual body can use the armpit of the elderly virtual body as the force point and apply force to the elderly virtual body. Figure 8 As shown in the figure, the end effector of the robot virtual body can provide a vertical upward support force Fv to offset part of the influence of gravity, and at the same time provide the necessary forces Fx and Fy in the front and back and left and right directions to hold the elderly virtual body tightly to ensure the stability of the elderly virtual body's posture. In order to reduce Fx and Fy and achieve a more comfortable hold, the end effector of the robot virtual body selects the center area of ​​the elderly virtual body's armpit as the force point, making the force arm in the front and back and left and right directions as small as possible to reduce the torque.

[0102] The whole process of the old man's virtual body from sitting vertically on the bed virtual body to being transferred to sitting vertically on the wheelchair virtual body has been described in detail above. Based on this process, the movement trajectory of the old man's virtual body from sitting vertically on the bed virtual body to sitting vertically on the wheelchair virtual body can be determined as the second half of the first expected trajectory of the virtual body movement corresponding to the object to be transferred in step S302. Specifically, considering that in the whole process, the posture of the old man's virtual body itself has not changed significantly but has only been displaced as a whole. Therefore, by determining the change in the center of mass of the old man's virtual body in the whole process, the second half of the first expected trajectory of the old man's virtual body can be directly determined. Optionally, the center of mass of the old man's virtual body is first displaced obliquely upward for a distance, then rotated on the horizontal plane, and then displaced obliquely downward for a distance. Of course, the present disclosure is not limited to this.

[0103] Next, refer to Figures 9 and 10 Some details of implementing step S303 are described, that is, how to determine the second expected trajectory according to the first expected trajectory.

[0104] Fig. 9 1 is a schematic diagram showing that the end effector of the robot virtual body according to an embodiment of the present disclosure applies a force and follows the trajectory of the point of application of the force. The arc line with an arrow indicates the movement trajectory of the point of application of the force applied by the end effector of the robot virtual body. The straight line with an arrow indicates the direction of the force applied by the end effector of the robot virtual body at two positions. Fig.10 It is a schematic diagram showing the complete process of interaction between a robot virtual body and an elderly virtual body according to an embodiment of the present disclosure.

[0105] according to Figure 7 and Figure 8 The first expected trajectory can be determined based on the description. In step S303, it is assumed that the old man's virtual body does not need to provide power autonomously, and the force is completely provided by the robot virtual body, the bed virtual body, and the ground virtual body. In such an assumption, it can be imagined that a virtual force directly acts on the center of mass of the old man's virtual body, so that the center of mass of the old man's virtual body moves along the first expected trajectory. The virtual force is the resultant force of all the forces provided by the robot virtual body, the bed virtual body, and the ground virtual body.

[0106] During the whole process, the second expected trajectory can instruct the end effector of the virtual body corresponding to the robot to perform at least one or a combination of more of the following items: taking the upper corner of one side of the trunk component of the virtual body corresponding to the object to be transferred as the center of the circle, and the width of the trunk component as the radius, rotating counterclockwise on a vertical plane, and stopping after rotating 1 / 4 of a circle; and / or taking the lower corner of the same side of the trunk component of the virtual body corresponding to the object to be transferred as the center of the circle, and the width of the trunk component as the radius, rotating counterclockwise on a vertical plane, and stopping after rotating 1 / 4 of a circle.

[0107] For another example, the second expected trajectory may also instruct the end effector of the virtual body corresponding to the robot to perform at least one or a combination of more of the following items: taking the lower corner of one side of the trunk component of the virtual body corresponding to the object to be transferred as the center of the circle, taking the length of the trunk component as the radius, rotating counterclockwise on a vertical plane, and stopping after rotating 1 / 4 of a circle; taking the lower corner of the same side of the trunk component of the virtual body corresponding to the object to be transferred as the center of the circle, taking the width of the trunk component as the radius, rotating counterclockwise on a vertical plane, and stopping after rotating 1 / 4 of a circle.

[0108] For another example, the second expected trajectory instructs the end effector of the virtual body corresponding to the robot to first move diagonally upward for a distance, then rotate on a horizontal plane, and then move diagonally downward for a distance.

[0109] As mentioned above, the center of mass of the old man's virtual body first rotates counterclockwise on the vertical plane with the left center of the old man's virtual body as the center of the circle and half of the width of the old man's virtual body as the radius, and stops after rotating 1 / 4 of the circle. This process is mainly provided by the robot virtual body and the bed virtual body. Fig.10 The three figures in the first row assume that during the whole process, the position of the end effector of the robot virtual body relative to the old man's virtual body remains unchanged, that is, it always keeps in contact with the center point of the right shoulder joint and the center point of the right hip joint of the old man's virtual body. Therefore, in this process, the expected trajectory of one end effector of the robot virtual body can be described as: taking the center point of the left shoulder joint of the old man's virtual body as the center of the circle, the width of the old man's virtual body as the radius, rotating counterclockwise on a vertical plane, and stopping after rotating 1 / 4 of a circle. The expected trajectory of another end effector of the robot virtual body can be described as: taking the center point of the left hip joint of the old man's virtual body as the center of the circle, the width of the old man's virtual body as the radius, rotating counterclockwise on a vertical plane, and stopping after rotating 1 / 4 of a circle. Of course, the present disclosure is not limited to this.

[0110] Next, the center of mass of the elderly virtual body rotates counterclockwise on the vertical plane with the center of the elderly virtual body's hip joint as the center of the circle and half the length of the elderly virtual body's torso as the radius, and stops after rotating 1 / 4 of the circle. This process is mainly provided by the robot virtual body and the bed virtual body. Fig.10 The first two figures in the second row assume that the position of the end effector of the robot virtual body relative to the old man's virtual body remains unchanged during the whole process, that is, it always keeps in contact with the center point of the left shoulder joint and the center point of the right hip joint of the old man's virtual body. In this process, the expected trajectory of an end effector of the robot virtual body (the end effector is in contact with the center point of the left shoulder joint of the old man's virtual body) can be described as: with the center point of the left hip joint of the old man's virtual body as the center of the circle and the length of the old man's virtual body's torso as the radius, it rotates counterclockwise on the vertical plane and stops after rotating 1 / 4 of the circle. Fig. 9 The figure shows the positions of the end effector at two moments in this process, and the forces applied by the end effector at these two moments. Fig. 9 As shown, the direction of the applied force is directly toward the direction of force application, that is, perpendicular to the surface of the end effector of the robot virtual body and outward. In contrast, the expected trajectory of the other end effector of the robot virtual body (the end effector is in contact with the center point of the right hip joint of the old man's virtual body) can be described as: taking the center point of the left hip joint of the old man's virtual body as the center of the circle and the width of the old man's virtual body as the radius, rotating counterclockwise on the vertical plane and stopping after rotating 1 / 4 of the circle. In this way, the first half of the second expected trajectory of the virtual body movement corresponding to the robot can be determined. Of course, the present disclosure is not limited to this.

[0111] Next, the center of mass of the old man's virtual body first moves diagonally upward for a distance, then rotates on the horizontal plane, and then moves diagonally downward for a distance. This process is mainly provided by the robot virtual body, the bed virtual body and the bottom virtual body. Then refer to the last figure in the second row and the first figure in the third row of the figure, assuming that during the whole process, the position of the end effector of the robot virtual body relative to the old man's virtual body remains unchanged, that is, it always keeps in contact with the center point of the left armpit and the center point of the right armpit of the old man's virtual body. Therefore, in this process, the expected trajectory of the end effector of the robot virtual body can be described as: first move diagonally upward for a distance, then rotate on the horizontal plane, and then move diagonally downward for a distance. In this way, the second half of the second expected trajectory of the virtual body movement corresponding to the robot can be determined. Of course, the present disclosure is not limited to this.

[0112] Continue to refer Fig.10, the above has described in detail how to determine the second desired trajectory of the end effector of the robot virtual body. However, in the whole process, not only the end effector of the robot virtual body is moving, but other components of the robot virtual body are also moving. The motion trajectories of other components of the robot virtual body can be solved accordingly according to the trajectory of the end effector of the robot virtual body. It can be understood that there is more than one way to make the end effector of the robot virtual body achieve the second desired trajectory described in detail above.

[0113] Therefore, some constraints can be provided to make the other components of the robot as smooth and efficient as possible so that the end effector of the robot virtual body moves in a manner that substantially conforms to the second desired trajectory described in detail above.

[0114] For example, these constraints include but are not limited to: the robot should be as close as possible to the bed virtual body and the elderly virtual body to minimize the distance between the robot's force point and the robot's virtual body trunk component, thereby improving the stability of the robot's virtual body, reducing the lever arm, reducing the driving torque required for the joints, and ensuring the safety of the elderly. In the process of the virtual robot rotating the elderly virtual body from lying flat to lying on the side, the inner and outer legs of the virtual robot remain motionless, and the movement of the end effector of the robot virtual body is mainly achieved by rotating the waist in the pitch direction, rotating the shoulder joint, and rotating the end effector relative to the upper arm joint to make the movement of the end effector of the robot virtual body as close to the second desired trajectory as possible.

[0115] These constraints also include but are not limited to: when the virtual robot rotates the old man's virtual body from lying on his side to sitting vertically on the bed virtual body, the left and right shoulder joints are rotated so that the end effector on the left side of the virtual robot contacts the right hip joint of the old man's virtual body, and the end effector on the right side of the robot's virtual body contacts the left shoulder joint of the old man's virtual body. Optionally, the end effector on the left side of the robot's virtual body will extend under the left shoulder joint of the old man's virtual body to provide better support.

[0116] In the second half of the second expected trajectory, that is, when the robot virtual body carries the old man's virtual body from the bed virtual body to the wheelchair virtual body, the following constraints can be further set: the robot virtual body is close to the bed virtual body and the old man's virtual body and the waist of the robot virtual body is pitched downward at a certain angle; the end effector of the robot virtual body extends to the left and right armpits of the old man's virtual body; the arm of the robot virtual body is raised to contact the center point of the old man's armpit; the end effector of the robot virtual body is maintained in contact with the center point of the old man's armpit, and the hip of the old man's virtual body leaves the bed virtual body through changes in the shoulder joint and the pitch angle of the waist component; the torso and upper arms of the robot virtual body are maintained motionless, and the center of mass of the robot virtual body and the old man's virtual body rotate on the horizontal plane only through the rotation of the driving wheels of the inner and outer legs of the robot virtual body; the pitch angle of the robot torso is adjusted by changing the pitch angle of the waist component of the robot virtual body so that the hip of the old man's virtual body contacts the wheelchair virtual body; the arm of the robot virtual body descends, leaves the armpit of the old man's virtual body and is then pulled out.

[0117] The above simply lists some constraints. Those skilled in the art will understand that there may be more constraints. For example, the time period corresponding to the second expected trajectory should be long enough to ensure the comfort and safety of the elderly during the actual operation; the changes in all joint angles of the robot virtual body should not exceed the limitations brought by the configuration of the robot virtual body, etc.

[0118] Next, refer to Figures 11 to 13 Some details of implementing step S304 and step S305 are described, that is, how to determine the first control information and the second control information. Fig.11 The schematic diagram shows the variation of the driving torque required for each joint of the robot virtual body over time during the process in which the robot virtual body rotates the elderly virtual body from a supine position to a side-lying position in the simulation space. Fig.12 A schematic diagram showing the variation of the driving torque required for each joint of the robot virtual body over time during the process in which the robot virtual body rotates the elderly virtual body from a side-lying posture to a sitting posture in the simulation space. Fig.13 The schematic diagram shows the variation of the driving torque required for each joint of the robot virtual body over time in the process of the robot virtual body carrying the elderly virtual body from the bed virtual body to the wheelchair virtual body in the simulation space.

[0119] In the simulation space, the simulation software will calculate the completion of the simulation based on the mass and moment of inertia of the virtual body of the gravity robot and the mass and moment of inertia of the virtual body of the elderly according to some physical constraints in the simulation space. Fig.10 The driving force required for each action shown. For ease of calculation, it can be based on Figure 6The first control information for controlling the virtual body corresponding to the robot in the simulation space is determined based on the collision information between the contact virtual body and the virtual body corresponding to the robot. The first control information is as follows: Figures 11 to 13 shown.

[0120] Fig.11 The figure shows the driving torque required for each joint of the robot virtual body when the robot virtual body rotates the elderly virtual body from a supine position to a side-lying position in the simulation space. Fig.11 As shown in the first figure on the left, during the whole process, the pitch angle of the waist joint of the robot virtual body changes greatly, and the motor used to control the rotation of the waist pitch direction of the robot virtual body will gradually output a larger torque, and then gradually reduce the output torque. Fig.11 As shown in the middle figure, the shoulder joint outputs a larger torque in the direction of forward and backward rotation. At the same time, in order to maintain the contact force provided by the end effector to the elderly virtual body, Fig.11 As shown in the figure on the right, the motor of the end effector will also output a certain torque to keep the end effector from leaving the virtual body of the elderly.

[0121] Fig.12 The figure shows the driving force required for each joint of the robot virtual body when the robot virtual body rotates the elderly virtual body from a side-lying posture to a sitting posture in the simulation space. Fig.12 As shown in the first figure on the left, the rotation angle of the end effector of the robot virtual body changes greatly during the whole process. The motor used to control the end effector of the robot virtual body to rotate in the pitch and yaw directions will gradually output a larger torque, and then gradually reduce the output torque. Fig.12 As shown in the middle figure, the shoulder joint can output large torque in all three degrees of freedom. At the same time, in order to maintain the contact force provided by the end effector to the elderly virtual body, Fig.12 As shown in the figure on the right, the motor of the end effector will also output a certain torque to keep the end effector from leaving the virtual body of the elderly.

[0122] Fig.13 The figure shows the driving torque required by each joint of the robot virtual body when the robot virtual body carries the old man's virtual body from the bed virtual body to the wheelchair virtual body in the simulation space. During the whole process, each joint needs to output a certain torque to maintain the stability of the robot virtual body and the old man's virtual body.

[0123] The mapping relationship between the simulation space and the physical space can be considered to map the first control information into the second control information. In this process, it may be necessary to further consider the dynamic model, inertial characteristics, mechanical characteristics, etc. of the actual robot, and adjust the second control information during the conversion process to obtain more accurate motor speed and torque. Optionally, in some embodiments, the motor speed and torque will be further converted into standard drive signals such as PWM, and these drive signals will be used as the final second control information. In some embodiments, it is also necessary to consider some control factors unique to the physical space, such as closed-loop control and other constraints, to make up for the deviation between the simulation space and the physical space, so that the robot can eventually move according to a predetermined trajectory and complete the task of transferring disabled elderly people.

[0124] Compared with the traditional robot scheme for assisting the transfer of disabled elderly people, the robot control method provided by the embodiment of the present disclosure enables the robot to autonomously transfer the disabled elderly from the bed to the wheelchair. In addition, the embodiment of the present disclosure fully considers the configuration of the robot and the physical data of the disabled elderly, and provides a control method to control the humanoid robot to assist the disabled elderly to transfer from the bedridden state to the wheelchair. By simulating the robot and the disabled elderly in the simulation space and simulating the entire process, the safety of the entire transfer process is ensured, so that the humanoid robot can autonomously complete the task of assisting the elderly in position transfer without human intervention or operation (but someone needs to monitor to ensure safety when necessary), and improve the stability and comfort of the humanoid robot holding the disabled elderly. Compared with other control schemes, the control scheme provided by this palace has a significantly excellent control effect.

[0125] According to another aspect of the present disclosure, a method for evaluating a robot is also proposed, wherein the robot includes a body component, an upper limb component, and a leg component, and the method includes: generating a virtual body corresponding to the robot and a virtual body corresponding to an object to be transferred in a simulation space; determining a first expected trajectory of movement of the virtual body corresponding to the object to be transferred; based on the first expected trajectory, determining a second expected trajectory of the movement, wherein, in the process of the virtual body corresponding to the robot moving along the second expected trajectory, a virtual force is applied to the virtual body corresponding to the object to be transferred so that the virtual body corresponding to the object to be transferred moves along the first expected trajectory; based on the second expected trajectory, determining first control information for controlling the virtual body corresponding to the robot in the simulation space; and evaluating the robot based on the first control information. Wherein, the evaluation of the robot includes: determining whether the performance of the robot can satisfy the requirement of making the virtual body corresponding to the robot move along the second expected trajectory. The performance of the robot includes, but is not limited to, degree of freedom configuration, joint size, joint motion range, workspace, interference, joint motion speed, joint driving torque, etc.

[0126] According to another aspect of the present disclosure, a robot is also provided. The robot may include a body component, an upper limb component, and a leg component.

[0127] The leg assembly refers to the foot-like component of the robot for realizing movement, which includes, for example, a mechanical leg and a motor connecting the mechanical leg to a base and for realizing movement control of the mechanical leg. The embodiments of the present disclosure are not limited by the specific composition type of the lower limb part and the number of the lower limbs.

[0128] The body component refers to the main body part of the robot, for example, it can be the torso of the robot. The embodiments of the present disclosure are not limited by the specific shape and composition of the base part.

[0129] In some embodiments, the body component may include, for example, 2 spinal joints, and the leg component may include, for example, 8 lower limb joints. The embodiments of the present disclosure are not limited by the specific number of joints included in the body component and the leg component, nor are they limited by the specific joint configuration of the robot.

[0130] The robot may further include a controller, which is disposed on the robot and is capable of executing the motion control method as described above and has the functions as described above.

[0131] The controller includes, for example, a processing device. The processing device may include a microprocessor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array, a state machine, or other processing device for processing electrical signals received from the sensor line. Such a processing device may include a programmable electronic device, such as a PLC, a programmable interrupt controller (PIC), a programmable logic device (PLD), a programmable read-only memory (PROM), an electronic programmable read-only memory, etc.

[0132] In addition, the robot may also include a bus, a memory, a sensor component, a communication module, an input and output device, etc.

[0133] The bus may be a circuit that interconnects the components of the robot and passes communication information (eg, control messages or data) among the components.

[0134] The sensor component can be used to perceive the physical world, such as a camera, an infrared sensor, an ultrasonic sensor, etc. In addition, the sensor component can also include a device for measuring the current operation and motion state of the robot, such as a Hall sensor, a laser position sensor, or a strain sensor.

[0135] The communication module can be connected to a network, for example, by wire or invalid, so as to communicate with the physical world (e.g., a server). The communication module can be wireless and can include a wireless interface, such as IEEE 802.11, Bluetooth, wireless local area network (WLAN) transceiver, or a radio interface for accessing a cellular phone network (e.g., a transceiver / antenna for accessing a CDMA, GSM, UMTS or other mobile communication network). In another example, the communication module can be wired and can include an interface such as Ethernet, USB or IEEE 1394.

[0136] The input-output device may transmit commands or data input, for example, from a user or any other external device to one or more other components of the robot, or may output commands or data received from one or more other components of the robot to the user or other external device.

[0137] A plurality of robots may form a robot system to collaboratively complete a task, the plurality of robots being communicatively connected to a server and receiving collaborative robot instructions from the server.

[0138] According to one aspect of an embodiment of the present disclosure, a device for controlling a robot is provided, the device comprising a processor and a memory, the memory storing at least one instruction, at least one program, a code set or an instruction set, the at least one instruction, the at least one program, the code set or the instruction set being loaded and executed by the processor to implement the above method.

[0139] According to one aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, in which at least one instruction, at least one program, a code set or an instruction set is stored, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement the above method.

[0140] The program part of the technology can be considered as a "product" or "manufactured product" in the form of executable code and / or related data, which is participated in or realized by computer-readable media. Tangible and permanent storage media can include any memory or storage used by any computer, processor, or similar device or related module. For example, various semiconductor memories, tape drives, disk drives or any similar devices that can provide storage functions for software.

[0141] All software or parts of it may sometimes be communicated over a network, such as the Internet or other communications network. Such communications can load software from one computer device or processor to another. Therefore, another medium capable of transmitting software elements may also be used as a physical connection between local devices, such as light waves, radio waves, electromagnetic waves, etc., transmitted through cables, optical cables or air. Physical media used to carry carriers, such as cables, wireless connections or optical cables and the like, can also be considered as media that carry software. As used herein, unless limited to tangible "storage" media, other terms referring to computer or machine "readable media" refer to media that participate in the process of executing any instructions by the processor.

[0142] The present application uses specific words to describe the embodiments of the present application. For example, "first / second embodiment", "one embodiment", and / or "some embodiments" refer to a certain feature, structure or characteristic related to at least one embodiment of the present application. Therefore, it should be emphasized and noted that "one embodiment" or "an embodiment" or "an alternative embodiment" mentioned twice or multiple times in different positions in this specification does not necessarily refer to the same embodiment. In addition, some features, structures or characteristics in one or more embodiments of the present application can be appropriately combined.

[0143] In addition, it will be appreciated by those skilled in the art that various aspects of the present application may be illustrated and described by a number of patentable categories or situations, including any new and useful process, machine, product or combination of substances, or any new and useful improvements thereto. Accordingly, various aspects of the present application may be performed entirely by hardware, entirely by software (including firmware, resident software, microcode, etc.), or by a combination of hardware and software. The above hardware or software may all be referred to as "data blocks", "modules", "engines", "units", "components" or "systems". In addition, various aspects of the present application may be represented as a computer product located in one or more computer-readable media, which includes computer-readable program code.

[0144] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the invention belongs. It should also be understood that terms such as those defined in common dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology and should not be interpreted in an idealized or extremely formal sense, unless explicitly defined as such herein.

[0145] The above is an explanation of the present invention and should not be considered as a limitation thereof. Although several exemplary embodiments of the present invention have been described, it will be readily appreciated by those skilled in the art that many modifications may be made to the exemplary embodiments without departing from the novel teachings and advantages of the present invention. Therefore, all such modifications are intended to be included within the scope of the present invention as defined in the claims. It should be understood that the above is an explanation of the present invention and should not be considered as being limited to the specific embodiments disclosed, and modifications to the disclosed embodiments and other embodiments are intended to be included within the scope of the appended claims. The present invention is defined by the claims and their equivalents.

Claims

1. A method for controlling a robot, the robot comprising a body component, an upper limb component and a leg component, the method comprising: In the simulation space, a virtual body corresponding to the robot and a virtual body corresponding to the object to be transferred are generated; Determining a first expected trajectory of movement of the virtual body corresponding to the object to be transferred; Based on the first expected trajectory, determining a second expected trajectory of the movement, wherein, during the movement of the virtual body corresponding to the robot along the second expected trajectory, a virtual force is applied to the virtual body corresponding to the object to be transferred, so that the virtual body corresponding to the object to be transferred moves along the first expected trajectory; Based on the second expected trajectory, determining first control information for controlling a virtual body corresponding to the robot in the simulation space; and Based on the first control information, second control information for controlling the robot in a physical space is determined.

2. The method of claim 1, wherein: The method further comprises: generating, in the simulation space, a virtual body corresponding to the first plane and a virtual body corresponding to the second plane, Wherein, determining the first expected trajectory further includes: The first expected trajectory is determined so that the virtual body corresponding to the object to be transferred is transferred from the virtual body corresponding to the first plane to the virtual body corresponding to the second plane.

3. The method of claim 1, wherein: The method also includes: in the simulation space, generating a contact virtual body associated with the virtual body corresponding to the object to be transferred, wherein each component of the contact virtual body is a rigid geometric body in the simulation space, and each component of the contact virtual body covers the surface of the relevant component of the virtual body corresponding to the object to be transferred.

4. The method of claim 3, wherein: The determining, based on the second expected trajectory, first control information for controlling a virtual body corresponding to the robot in the simulation space further includes: Based on the collision information between the contact virtual body and the virtual body corresponding to the robot, first control information for controlling the virtual body corresponding to the robot in the simulation space is determined.

5. The method of claim 1, wherein: The first desired trajectory indicates at least one or a combination of the following: The center of mass of the virtual body corresponding to the object to be transferred is rotated counterclockwise on a vertical plane with the center of one side of the trunk component as the center of the circle and half the width of the trunk component as the radius, and stops after rotating 1 / 4 of the circle; The center of mass of the virtual body corresponding to the object to be transferred is rotated counterclockwise on a vertical plane with the bottom center of the trunk component as the center of the circle and half the length of the trunk of the trunk component as the radius, and stops after rotating 1 / 4 of the circle; The center of mass of the virtual body corresponding to the object to be transferred is displaced diagonally upward for a distance, then rotated on the horizontal plane, and then displaced diagonally downward for a distance.

6. The method of claim 1, wherein: The second expected trajectory instructs the end effector of the virtual body corresponding to the robot to perform at least one or a combination of multiple of the following: Taking the upper corner of one side of the trunk component of the virtual body corresponding to the object to be transferred as the center of the circle and the width of the trunk component as the radius, rotate counterclockwise on the vertical plane and stop after rotating 1 / 4 of the circle; Take the lower corner of the same side of the trunk component of the virtual body corresponding to the object to be transferred as the center of the circle, take the width of the trunk component as the radius, rotate counterclockwise on the vertical plane, and stop after rotating 1 / 4 of the circle.

7. The method of claim 1, wherein: The second expected trajectory instructs the end effector of the virtual body corresponding to the robot to perform at least one or a combination of multiple of the following: Taking the lower corner of one side of the trunk component of the virtual body corresponding to the object to be transferred as the center of the circle and the length of the trunk component as the radius, rotate counterclockwise on the vertical plane and stop after rotating 1 / 4 of the circle; Take the lower corner of the same side of the trunk component of the virtual body corresponding to the object to be transferred as the center of the circle, take the width of the trunk component as the radius, rotate counterclockwise on the vertical plane, and stop after rotating 1 / 4 of the circle.

8. The method of claim 1, wherein: The second expected trajectory instructs the end effector of the virtual body corresponding to the robot to first move diagonally upward for a distance, then rotate on a horizontal plane, and then move diagonally downward for a distance.

9. A method for evaluating a robot, the robot comprising a body component, an upper limb component, and a leg component, the method comprising: In the simulation space, a virtual body corresponding to the robot and a virtual body corresponding to the object to be transferred are generated; Determining a first expected trajectory of movement of the virtual body corresponding to the object to be transferred; Based on the first expected trajectory, determining a second expected trajectory of the movement, wherein, during the movement of the virtual body corresponding to the robot along the second expected trajectory, a virtual force is applied to the virtual body corresponding to the object to be transferred, so that the virtual body corresponding to the object to be transferred moves along the first expected trajectory; Based on the second expected trajectory, determining first control information for controlling a virtual body corresponding to the robot in the simulation space; and Based on the first control information, the robot is evaluated.

10. The method of claim 9, wherein: The evaluating of the robot includes: determining whether the performance of the robot can satisfy the requirement of causing a virtual body corresponding to the robot to move along a second expected trajectory.

11. A robot comprising: torso assembly, upper limb assembly, and leg assembly; A controller, wherein the controller is arranged on the robot and is capable of executing the method according to any one of claims 1 to 10.

12. A device for controlling a robot, comprising: processor; and A memory, wherein the memory stores computer executable codes, and when the computer executable codes are executed by the processor, the method according to any one of claims 1 to 10 is executed.

13. A non-volatile computer-readable storage medium having executable codes stored thereon, wherein when the executable codes are executed by a processor, the processor is caused to execute the method according to any one of claims 1 to 10.

Images