Robot control method, robot and electronic equipment

Abstract

The invention provides a robot control method, a robot and electronic equipment, and the embodiment of the invention provides the robot control method, the robot and the electronic equipment, and the method comprises the steps: determining the state information of the robot and the upper limb action information of the robot; and the state information and the upper limb action information are input into a prediction model obtained through training based on sample upper limb information subjected to dynamics enhancement, lower limb action information which is output by the prediction model and used for representing the angles of all joints in the lower limbs of the robot is obtained, and the robot is controlled according to the lower limb action information. Therefore, the prediction model obtained by training the sample upper limb information after dynamics enhancement fully considers the movement condition of the upper limbs of the robot. Therefore, the model can still accurately determine the lower limb action information capable of ensuring that the robot is kept stable even under the complex condition that the upper limbs of the robot violently move, so that the safety and the reliability of the robot in the actual use process are fundamentally guaranteed.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of robots, in particular to a robot control method, a robot and an electronic device. BACKGROUND

[0002] With the continuous development of technology and the acceleration of intelligent process, robots are increasingly popular in people's lives. As one of the common types of robots, humanoid foot robots have been widely used in various scenes.

[0003] The current robot control method usually obtains the state information of the robot, and inputs the state information into a pre-trained model to output the control instruction of the lower limbs of the robot.

[0004] However, the current technical solution does not fully consider the influence of the violent movement of the upper limbs of the robot on the stability of the lower limbs when training the model, resulting in low accuracy of the control instruction output by the model. When the control instruction output by the model is used to control the robot, the robot cannot stand or walk stably, which poses a safety hazard. SUMMARY

[0005] Therefore, the embodiments of the present application provide a robot control method, a robot and a robot electronic device, which can determine the lower limb action information of the robot based on the upper limb action information of the robot to ensure the safety of the robot during use.

[0006] In a first aspect, the embodiments of the present application provide a robot control method, comprising: determining state information of a robot and upper limb action information of the robot; inputting the state information and the upper limb action information into a trained prediction model to obtain lower limb action information of the robot output by the prediction model, the prediction model being trained based on sample upper limb information enhanced by dynamics, the lower limb action information being used to represent angles of joints in the lower limbs of the robot; and controlling the robot according to the lower limb action information.

[0007] In a second aspect, the embodiments of the present application provide a robot control device, comprising: a determination module configured to determine state information of a robot and upper limb action information of the robot; a prediction module configured to input the state information and the upper limb action information into a trained prediction model to obtain lower limb action information of the robot output by the prediction model, the prediction model being trained based on sample upper limb information enhanced by dynamics, the lower limb action information being used to represent angles of joints in the lower limbs of the robot; and a control module configured to control the robot according to the lower limb action information.

[0008] In a third aspect, an embodiment of the present application provides a robot, comprising a control module, the control module being configured to execute the robot control method of the first aspect.

[0009] In a fourth aspect, an embodiment of the present application provides an electronic device, comprising: a processor; and a memory configured to store processor-executable instructions, wherein the processor is configured to execute the robot control method of the first aspect.

[0010] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, the storage medium storing a computer program, the computer program being configured to execute the robot control method of the first aspect.

[0011] In a sixth aspect, an embodiment of the present application provides a computer program product, the computer program product comprising a computer program, the computer program being configured to cause a computer device to execute the robot control method of the first aspect when the computer program is executed by a processor of the computer device.

[0012] In a seventh aspect, an embodiment of the present application provides a chip, comprising: a processor; and a memory configured to store processor-executable instructions, wherein the processor is configured to execute the robot control method of the first aspect.

[0013] The robot control method, the robot, and the electronic device provided in the embodiments of the present application determine state information of the robot and upper limb action information of the robot, input the state information and the upper limb action information into a prediction model trained based on sample upper limb information enhanced by dynamics, obtain lower limb action information output by the prediction model and used to represent angles of joints in a lower limb of the robot, and control the robot according to the lower limb action information.

[0014] In this way, the prediction model trained based on the sample upper limb information enhanced by dynamics fully considers the motion of the upper limb of the robot. This enables the prediction model to accurately determine lower limb action information that can ensure that the robot maintains stability even in a complex situation where the upper limb of the robot moves violently, thereby guaranteeing the safety and reliability of the robot in actual use. BRIEF DESCRIPTION OF DRAWINGS

[0015] Figure 1 FIG. 1 shows a system architecture schematic diagram of a robot control system provided by an example embodiment of the present application.

[0016] Figure 2 FIG. 2 shows a flow schematic diagram of a robot control method provided by an example embodiment of the present application.

[0017] Figure 3 FIG. 3 shows a flow schematic diagram of a model training method provided by an example embodiment of the present application.

[0018] Figure 4 Fig. 1 shows a structural schematic diagram of a robot control device according to an example embodiment of the present application.

[0019] Figure 5 Fig. 2 shows a block diagram of an electronic device for executing a robot control method according to an example embodiment of the present application. DETAILED DESCRIPTION

[0020] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0021] SUMMARY

[0022] With the continuous development of technology and the acceleration of intelligentization process, humanoid foot-type robots have been widely used in service, medical treatment, industry and other fields due to their humanoid appearance and adaptability to non-structured environments. In the process of executing tasks, the stability of motion control is particularly crucial.

[0023] The current robot control method usually obtains state information of the robot, and inputs the state information into a pre-trained prediction model to output control instructions of the lower limbs of the robot. These state information usually includes parameters for describing the current motion state of the robot, such as the angles, speeds, accelerations of the joints of the robot, and the overall position, attitude and other data.

[0024] However, in the actual execution process, the motion of the upper limbs of the humanoid foot-type robot is extremely complex and variable. For example, when performing tasks such as carrying and operating tools, the upper limbs will perform large-scale stretching, twisting, swinging and other violent actions. These violent movements of the upper limbs will inevitably cause rapid shift of the overall center of gravity of the robot and changes in inertial force, thereby having a significant impact on the stability of the lower limbs. However, the current prediction model does not consider this key factor during training, which leads to the fact that the prediction model trained at present has low accuracy in predicting the lower limb action information of the robot, and the robot is usually difficult to maintain a stable standing or walking posture when controlling itself through the lower limb action information.

[0025] In summary, the technical problem of "not fully considering the dynamic disturbance of violent motion of the upper limbs when determining the lower limb action information of the robot, resulting in insufficient stability of the lower limbs".

[0026] The application provides a robot control method, state information of a robot and upper limb action information of the robot are determined, the state information and the upper limb action information are input into a prediction model trained based on sample upper limb information enhanced by dynamics, lower limb action information output by the prediction model and used for representing angles of each joint in a lower limb of the robot is obtained, and the robot is controlled according to the lower limb action information. In this way, the prediction model trained based on the sample upper limb information enhanced by dynamics fully considers the motion of the upper limb of the robot. This enables the prediction model to accurately determine lower limb action information that can ensure that the robot maintains stability even in complex situations where the upper limb of the robot moves violently, thereby guaranteeing the safety and reliability of the robot in actual use.

[0027] Exemplary system

[0028] Figure 1 Fig. 1 shows a system architecture schematic diagram of a robot control system provided by an exemplary embodiment of the application. Figure 1 As shown, the robot control system 100 can include a robot 110, which includes an upper limb 111 and a lower limb 112. The robot 110 is a robot with autonomous motion capability, including the upper limb 111 and the lower limb 112, and the motion of the upper limb 111 affects the stability of the lower limb 112, such as a carrying robot, a humanoid foot robot, etc. The robot 110 can be installed with a controller 113. Optionally, the controller 113 can be located outside the robot 110, such as a device independent of the robot 110.

[0029] Exemplarily, the controller 113 can include at least one of a logic controller, a machine vision controller, a motion controller, etc.

[0030] In an application scenario example, the controller 113 can determine state information of the robot and upper limb action information. After obtaining the state information of the robot and the upper limb action information, the controller 113 can input the state information and the upper limb action information as input into the trained prediction model, to obtain lower limb action information of the robot output by the prediction model. Finally, the controller 113 can control each joint of the lower limb according to the lower limb action information. The prediction model is trained based on sample upper limb information enhanced by dynamics. In this way, the controller 113 can accurately determine lower limb action information that can ensure that the robot maintains stability based on the upper limb action information and the state information of the robot.

[0031] In other embodiments, the prediction model can be deployed by other electronic devices to determine the lower limb action information, and then the lower limb action information is transmitted to the controller 113, so that the controller 113 can control the joints in the lower limbs of the robot to move based on the lower limb action information, so as to ensure the stability in the robot control process.

[0032] It should be understood that the above application scenario examples are only shown for the purpose of facilitating understanding of the spirit and principles of the present application, and the embodiments of the present application are not limited thereto. On the contrary, the embodiments of the present application can be applied to any scenario that can be applicable.

[0033] Exemplary method

[0034] Figure 2 A flowchart of a robot control method provided by an exemplary embodiment of the present application is shown. Figure 2 The method can be executed by a robot 110 in Figure 1 , or can be executed by other electronic devices that can communicate with the robot 110. As shown in Figure 2 , the robot control method can include the following contents.

[0035] 210: determining state information of the robot and upper limb action information of the robot.

[0036] In an example, the state information is data used to represent the current motion state of the robot, which can include at least one of a trunk angular velocity, a trunk velocity, a gravity vector of a trunk coordinate system, a velocity command, a position of each joint of the robot at the current time, and other parameters used for the motion state of the robot.

[0037] In an example, the upper limb action information is action data of the upper limb of the robot, which can include at least one of a joint position (angle), an angular velocity, a joint position at the current time, and other parameters used to represent the action trajectory of the upper limb of the robot.

[0038] In an example, the upper limb action information can include action data of each joint of the upper limb of the robot at multiple time points.

[0039] In an example, the state information refers to data reflecting the motion state of the robot, such as a trunk angular velocity reflecting the trunk rotation state, a gravity vector of a trunk coordinate system reflecting the trunk tilt direction, etc.; and the upper limb action information refers to data reflecting the motion history of the upper limb, such as the upper limb joint position, angular velocity, target position, etc.

[0040] In an example, a sensor can be arranged on the robot, and the robot can collect the parameters for representing the current motion state of the robot through the sensor arranged on the robot, and integrate the parameters to obtain the equipment information. The sensor can be at least one of a gyroscope IMU, a joint encoder, and the like.

[0041] In an example, a motion controller can be arranged on the robot, and the robot can extract upper limb joint data at multiple time points (such as a current frame and 5 past frames) from the motion controller, and integrate the upper limb joint data into upper limb motion information.

[0042] 220: inputting the state information and the upper limb motion information into the trained prediction model to obtain lower limb motion information output by the prediction model, the prediction model being trained based on the sample upper limb information enhanced by dynamics, and the lower limb motion information being used to represent angles of joints in the lower limbs of the robot.

[0043] In an example, the dynamics enhancement refers to a processing manner of expanding a disturbance scene covered by samples by diversifying adjustment of dynamics parameters and trajectory playing manners of the upper limbs of the legged robot during training of the upper limb disturbance of the legged robot.

[0044] In an example, the dynamics enhancement manner can include at least one of adjustment of mass of a joint of the upper limb of the robot, adjustment of a control coefficient of the joint of the upper limb, adjustment of a motion speed of the joint of the upper limb, switching of a motion trajectory of the upper limb, and the like.

[0045] In an example, the dynamics enhancement manner can further include adjustment of an angle of the joint of the upper limb of the robot.

[0046] In an example, the sample upper limb information enhanced by dynamics refers to upper limb trajectory data processed by the above dynamics enhancement. The sample upper limb information enhanced by dynamics represents various upper limb disturbance situations that the legged robot can encounter in an actual scene, such as complex dynamics scenes of different load bearing (change of joint mass), different control response characteristics (change of control coefficient), different motion speeds, and sudden and violent swinging (switching of motion trajectory), and covers various disturbance factors such as simulation model error, operation load change, and motion speed difference.

[0047] In an example, the prediction model is a motion control neural network trained based on reinforcement learning, the input of the prediction model is the state information and the upper limb motion information, and the output of the prediction model is lower limb motion information used to represent target angles of joints in the lower limbs.

[0048] In one example, the robot can use the collected status information and upper limb motion information as input into a trained prediction model. The model predicts the lower limb motion information based on the robot's current status information and upper limb motion information, and finally obtains the lower limb motion information output by the prediction model.

[0049] In one example, the prediction model can also infer the robot's lower limb motion information at a frequency of 100 times per second, where 100 frames per second means that the robot's lower limb motion information is output once every 1 / 100 second.

[0050] In one example, when the output frequency is high (e.g., 100 frames of lower limb motion information are output per second), the prediction model can update the robot's status information based on the output lower limb motion information after outputting the robot's lower limb motion information, and continue to predict the robot's lower limb motion information based on the updated status information and upper limb motion information.

[0051] 230: Control the robot based on the lower limb movement information.

[0052] In one example, the robot can adjust the angles of the joints in the robot's lower limbs according to the lower limb movement information, thereby driving the lower limb movement and achieving stable control of walking or standing.

[0053] In one example, the joints of the robot are usually controlled by motors. Therefore, the robot can also determine the torque corresponding to each joint in the robot's lower limbs based on the lower limb movement information, and control the motors of each joint in the lower limbs to adjust their torques based on the torque corresponding to each joint in the robot's lower limbs, so as to adjust each joint in the robot's lower limbs to the angle corresponding to the lower limb movement information, thereby driving the lower limbs to move.

[0054] The present application provides a robot control method, which determines the robot's state information and upper limb motion information, inputs the state information and upper limb motion information into a prediction model obtained by training based on sample upper limb information after dynamic enhancement, obtains lower limb motion information output by the prediction model to characterize the angles of each joint in the robot's lower limbs, and controls the robot based on the lower limb motion information. In this way, the prediction model obtained by training with sample upper limb information after dynamic enhancement fully takes into account the movement of the robot's upper limbs. This ensures that even in complex situations where the robot's upper limbs undergo violent movements, the prediction model can still accurately determine the lower limb motion information that can ensure that the robot maintains stability, thereby fundamentally ensuring the safety and reliability of the robot during actual use.

[0055] Further, for the problem that the prediction model is "unseen" for most upper limb motion trajectories due to limited sample coverage, and has poor adaptability to large-scale and severe upper limb disturbance, and is prone to abnormal results due to "unseen" trajectories, the method trains the prediction model through dynamically enhanced sample upper limb information, significantly expands the coverage of sample upper limb information, makes it contain more diversified dynamic disturbance scenes, reduces the probability of "unseen" trajectories, reduces the risk of overfitting from the source, and enables the model to accurately predict lower limb actions to resist upper limb disturbance in actual scenarios and maintain robot stability. The upper limb motion trajectory can be upper limb action information, and the upper limb action information can be part of the upper limb motion trajectory.

[0056] According to an embodiment of the present application, the training process of the prediction model can include: determining sample upper limb information, and dynamically enhancing the sample upper limb information to obtain dynamically enhanced sample upper limb information, determining sample state information, and determining a plurality of sample data according to the sample upper limb information and the dynamically enhanced sample upper limb information, and the sample state information, inputting the plurality of sample data into the prediction model to be trained to obtain a plurality of sample lower limb information output by the prediction model, and training the prediction model in a reinforcement learning manner based on the stability of the plurality of sample lower limb information.

[0057] In an example, the sample upper limb information refers to upper limb trajectory data for training. The upper limb trajectory data is simulation data. The sample upper limb trajectory can be path data of upper limb motion, including positions of each joint of the upper limb of the robot at each time, and the like, for representing an action trajectory of the upper limb of the robot.

[0058] In an example, the upper limb trajectory data can be a motion trajectory of the upper limb of a target robot after remapping trajectories in an open source trajectory data set to the target robot, or can be a trajectory of the target robot collected in a process of controlling the target robot to perform a task by teleoperation or the like. The target robot is a robot for deploying the prediction model.

[0059] In an example, the robot has different models, and different prediction models can be trained for different models of robots.

[0060] In an example, dynamic enhancement refers to expanding the disturbance scene covered by the sample by adjusting the dynamic parameters of the upper limb motion.

[0061] In an example, the sample state information can be state data of the robot in a simulation environment.

[0062] In an example, the robot control method is applied to a scenario of training the prediction model through sample upper limb information in a simulation environment, and deploying the prediction model on the robot after training is completed.

[0063] In an example, the sample upper limb information can be collected first and is subjected to dynamics enhancement to obtain dynamics-enhanced sample upper limb information. Then, the robot can collect sample state information corresponding to the sample upper limb information in a simulation environment, and combine the two to obtain sample data. Then, the sample data is input into the prediction model to be trained to obtain sample lower limb information. Finally, based on the robot stability corresponding to the sample lower limb information, the model parameters are iterated through reinforcement learning until the model converges.

[0064] In an example, the training process of the prediction model can be performed by the robot or other electronic devices (such as a server) that can communicate with the robot.

[0065] In the embodiments of the present application, the disturbance scenarios of the sample are expanded through dynamics enhancement, and the generalization ability of the prediction model to unobserved upper limb disturbances is greatly improved through reinforcement learning training, and the risk of overfitting is reduced.

[0066] According to an embodiment of the present application, the step of determining the dynamics-enhanced sample upper limb information can include: determining a plurality of enhanced joints from the joints included in the robot, adjusting the angle of each enhanced joint in the plurality of enhanced joints to obtain an adjusted joint angle, and determining the dynamics-enhanced sample upper limb information according to the plurality of enhanced joints after adjusting the joint angle.

[0067] In an example, the enhanced joints refer to a part of the joints in the upper limb.

[0068] In an example, the enhanced joints can be joints in the upper limb that are significantly affected by disturbances, such as shoulder joints, elbow joints, etc.

[0069] In an example, adjusting the joint angle refers to adding an added angle within a certain range based on the original joint angle, so that a similar but not identical angle is generated. The added angle can be randomly generated

[0070] In an example, the enhanced joints can be determined from the joints included in the upper limb of the robot, and the angle of each enhanced joint can be adjusted to obtain the enhanced joint after adjusting the angle. Finally, according to the enhanced joint after adjusting the angle and other joints in the upper limb of the robot, the motion trajectory of the upper limb of the robot in the simulation environment can be obtained, so that the dynamics-enhanced sample upper limb information is obtained.

[0071] In the embodiment of the present application, on the basis of the sample upper limb information, the angle of the enhanced joint in the upper limb joint is fine-tuned to generate the sample upper limb information after dynamics enhancement, so that the sample upper limb information and the sample upper limb information after dynamics enhancement cover different information, and then the prediction model is trained by the sample upper limb information and the sample upper limb information after dynamics enhancement, thereby enhancing the coverage of the small range motion deviation of the upper limb in the training sample, and improving the adaptability of the model to the angle error in the actual motion.

[0072] According to an embodiment of the present application, the step of determining the sample upper limb information after dynamics enhancement can include: determining a plurality of enhanced joints from each joint included in the robot; adjusting the quality of each enhanced joint in the plurality of enhanced joints to obtain an adjusted joint quality; and determining the sample upper limb information after dynamics enhancement according to the plurality of enhanced joints after adjusting the joint quality.

[0073] In an example, the enhanced joint can be each joint in the upper limb joints of the robot for performing the dynamics enhancement operation.

[0074] In an example, adjusting the joint quality means adding an enhanced quality within a certain range to the quality of the original joint, and the enhanced quality can be a positive number or a complex number.

[0075] In an example, adjusting the joint quality can be multiplying the quality of the original joint by a first enhancement coefficient to obtain a quality used for simulating different load bearing, and the first enhancement coefficient is a positive number.

[0076] In an example, the enhanced joint can be determined from the upper limb joints of the robot, and for each enhanced joint, the quality of the enhanced joint is multiplied by a first enhancement coefficient to obtain a new joint quality. Finally, according to the plurality of enhanced joints after adjusting the joint quality, the dynamics characteristics of the upper limb motion (such as the degree of change of the center of gravity of the robot with the amplitude of the upper limb motion) can be determined in the simulation environment. Finally, based on the enhanced joints after adjusting the joint quality, the sample upper limb information after dynamics enhancement can be formed.

[0077] In an example, the sample upper limb information after dynamics enhancement can be determined according to the determined dynamics characteristics, or can be determined according to the simulation results of each enhanced joint and each other joint in the upper limb of the robot in the simulation environment after adjusting the joint quality.

[0078] In an example, the first enhancement coefficient can be a random value within its corresponding range, and the range of the first enhancement coefficient is 0.9-1.1. Of course, with the accumulation of the number of model training, the range of the first enhancement coefficient can be expanded to 0.9-1.5.

[0079] In the embodiment of the present application, the sample upper limb information after dynamics enhancement can cover the mass changes of the upper limb caused by weight bearing or simulation model errors, and therefore the trained prediction model can improve its stability control capability for upper limb disturbance under different loads.

[0080] According to an embodiment of the present application, the step of determining the sample upper limb information after dynamics enhancement can include: determining a plurality of enhanced joints from the joints included in the robot; adjusting the control coefficient of each enhanced joint in the plurality of enhanced joints to obtain an adjusted control coefficient, the control coefficient being used to represent the response weight of the joint to the angle error; and determining the sample upper limb information after dynamics enhancement according to the plurality of enhanced joints after adjusting the control coefficient.

[0081] In an example, the control coefficient can be a proportional coefficient and a differential coefficient in proportional differential control, or other parameters used to control the response speed of the upper limb joints of the robot to the angle error. For the same angle error, the response speed of the joint to the error is different under different control coefficients, and the response degree is also different. For example, in the case of an angle error of 0.5 degrees, the joint with a low control coefficient can not be sensitive to the error, and even the case that the joint itself is considered to have no error and thus does not need to be adjusted can occur, while the joint with a high control coefficient can sensitively perceive the error and respond to it at a faster speed to adjust its position (angle) to eliminate the error.

[0082] In an example, adjusting the control coefficient can be multiplying the original control coefficient of the joint by a second enhancement coefficient to obtain a control parameter used to simulate the difference in control response.

[0083] In an example, adjusting the control parameter can be adding an enhanced control parameter within a certain range to the original control parameter of the joint, and the enhanced control parameter can be a positive number or a complex number.

[0084] In an example, the second enhancement coefficient can be a random value within its corresponding range, and the range of the second enhancement coefficient is 0.9-1.1. Of course, as the number of model training increases, the range of the second enhancement coefficient can be expanded to 0.9-1.5.

[0085] In an example, the enhanced joint can be determined from the upper limb joints of the robot, and for each enhanced joint, the original control parameter of the enhanced joint is multiplied by the second enhancement coefficient to obtain the adjusted control coefficient. Finally, based on the adjusted control coefficient, different response characteristics of the upper limb when tracking the trajectory are simulated to form the sample upper limb information after dynamics enhancement.

[0086] In the embodiment of the present application, the sample upper limb information after dynamics enhancement can cover the response deviation of the upper limb movement caused by the difference of the control parameters, so as to further improve the robustness of the model to the parameter fluctuation in the actual control.

[0087] According to an embodiment of the present application, the step of determining the sample upper limb information after dynamics enhancement can include: determining a plurality of enhanced joints from each joint included in the robot; adjusting the movement speed of each enhanced joint in the plurality of enhanced joints to obtain an adjusted movement speed, the movement speed being used to represent the playing speed of the sample upper limb trajectory; and determining the sample upper limb information after dynamics enhancement according to the plurality of enhanced joints after adjusting the movement speed.

[0088] In an example, the sample upper limb information is a sample upper limb trajectory of the robot, and the movement speed of the joint is used to represent the playing speed of the sample upper limb trajectory.

[0089] In an example, the adjusted movement speed can be the movement speed obtained by multiplying the third enhancement coefficient on the basis of the original movement speed of the joint, which is used to simulate different movement rates.

[0090] In an example, the adjustment of the control parameter can be adding an enhanced movement speed within a certain range to the original movement speed of the joint, and the enhanced movement speed can be a positive number or a complex number.

[0091] In an example, the third enhancement coefficient can be a random value within its corresponding range, and the range of the third enhancement coefficient is 1.0-3.0. Of course, with the accumulation of the number of model training times, the range of the third enhancement coefficient can be further expanded.

[0092] In an example, the robot can determine the enhanced joint from each joint included in the upper limb, and after determining the enhanced joint, for each enhanced joint, multiply the movement speed (playing speed of the trajectory) of the enhanced joint corresponding to the trajectory in the sample upper limb information by the third enhancement coefficient. Then, based on the adjusted movement speed, regenerate the trajectory sequence (such as the position of each joint at different time points) corresponding to each joint of the upper limb of the robot respectively to form the sample upper limb information after dynamics enhancement.

[0093] In the embodiment of the present application, the sample upper limb information after dynamics enhancement can cover the disturbance scene of different movement rates of the upper limb, so as to further improve the adaptability of the model to the fast or slow upper limb movement.

[0094] According to an embodiment of the present application, the step of determining the sample upper limb information with enhanced dynamics can include: determining specific sample information from other sample upper limb information, controlling each joint of the robot to stop playing the sample upper limb trajectory of the sample upper limb information of the robot, and playing the sample upper limb trajectory corresponding to the specific sample information, and determining the sample upper limb information with enhanced dynamics according to the action of the robot.

[0095] In an example, the specific sample information can be any other trajectory in the upper limb trajectory data set except the sample upper limb information.

[0096] In an example, the sample upper limb trajectory refers to the upper limb movement path data used for training. The sample upper limb information with enhanced dynamics refers to the upper limb movement data with frequent and violent swings formed by random switching of trajectories.

[0097] In an example, the trajectory switching interval can be preset. When the robot upper limb plays the current sample upper limb trajectory, after reaching the switching interval, the current trajectory is stopped, another sample upper limb information is randomly selected from the upper limb trajectory data set, and playing starts from a random frame thereof. The upper limb movement trajectory after switching is recorded to form the sample upper limb information with enhanced dynamics. For example, in a simulation environment, the robot is controlled to simulate according to the sample upper limb trajectory corresponding to the sample upper limb information A, and when the time reaches 3s, the robot is controlled to simulate according to the sample upper limb trajectory corresponding to the sample upper limb information B.

[0098] In an example, the upper limb trajectory data set can be a set of sample upper limb information used for training the prediction model.

[0099] In an embodiment of the present application, violent and discontinuous upper limb movements are generated by random switching of trajectories, so as to generate the sample upper limb information with enhanced dynamics including violent and discontinuous upper limb movements, and further generate training samples including violent and discontinuous upper limb movements, thereby further enhancing the stability control capability of the model to sudden and large disturbances.

[0100] According to an embodiment of the present application, the training process further includes: deploying the prediction model on the robot and testing the robot in the case of meeting the test condition; collecting upper limb action information corresponding to the unstable state in the case of existing unstable state in the robot control process, and re-determining the sample upper limb information according to the upper limb action information corresponding to the unstable state, and training the prediction model based on the re-determined sample upper limb information.

[0101] In an example, the test condition can be that the prediction model reaches preliminary convergence after a certain number of iterations in simulation, or that the stability of the sample lower limb information output by the prediction model is higher than a stability threshold.

[0102] In an example, the unstable state refers to a state in which the robot cannot maintain stability in real machine testing, such as falling down, trunk tilting beyond a threshold, and the like.

[0103] In an example, when the prediction model meets the test condition, it can be deployed on an actual robot. Then, the upper limbs of the robot are controlled to make various movements through a teleoperation module or other electronic devices, and the stability of the robot is tested. During the robot testing, if an unstable state occurs, the upper limb trajectory data corresponding to the unstable state can be recorded. Then, the newly collected upper limb trajectory data can be supplemented to the upper limb trajectory data set, and the prediction model parameters are retrained based on the updated upper limb trajectory data set.

[0104] In the embodiments of the present application, the complex working conditions (unstable states) not covered by the real machine feedback are supplemented, and the prediction model is continuously iterated based on the complex working conditions, so that stable control under all target working conditions can be finally realized, further ensuring the stability of the output results of the prediction model.

[0105] According to an embodiment of the present application, the step of determining the training sample includes: sampling the sample upper limb information and the sample upper limb information enhanced by dynamics respectively to obtain a plurality of sample upper limb positions, and combining the plurality of sample upper limb positions with the sample state information respectively to obtain a plurality of sample data.

[0106] In an example, the sample state information can be state data of the robot in the simulation environment.

[0107] In an example, the sampling can be extracting the upper limb positions at multiple time points from the sample state information.

[0108] In an example, the sample data refers to the combination of the state information and the upper limb position, which is used for model training.

[0109] In an example, a plurality of upper limb positions can be sampled at time intervals from the sample upper limb information and the sample upper limb information enhanced by dynamics, and each sample upper limb position can be combined with the corresponding sample state information to generate a plurality of sample data.

[0110] In an example, the sample upper limb position can include an upper limb position at one time point, or can include upper limb positions at a plurality of continuous time points.

[0111] In an example, after generating the sample upper limb information enhanced by dynamics, the sample upper limb information can be added to the upper limb trajectory data set, and then each sample upper limb information in the upper limb trajectory data set can be sampled to obtain a plurality of upper limb positions, and a plurality of sample data can be generated according to the plurality of upper limb positions.

[0112] In an embodiment of the present application, the sample upper limb information and the sample upper limb information enhanced by dynamics are respectively sampled, and combined with the sample state information to generate sample data, which enriches the diversity of the training data, improves the generalization ability of the model, enhances the adaptability of the model to the complex motion of the upper limb of the robot, optimizes the model training effect, enables the model to more accurately learn the motion relationship of each part of the robot, and improves the motion performance of the robot in actual application.

[0113] According to an embodiment of the present application, the step of determining the training sample includes: sampling the sample upper limb information to obtain a plurality of initial upper limb positions, and adjusting parameters of at least part of the initial upper limb positions in the plurality of initial upper limb positions to obtain a plurality of sample upper limb positions.

[0114] In an example, the initial upper limb position refers to the original upper limb position obtained by sampling, which includes the positions (angles) of the joints in the upper limb of the robot at a certain time.

[0115] In an example, adjusting the parameters of the initial upper limb position can be adding noise to the specific values corresponding to the initial upper limb position, for example, the angle of joint A in the initial upper limb position is 1 degree, and after adding noise, the corresponding angle can be 1.1 degrees, so that the sample upper limb position after adjusting the parameters is obtained.

[0116] In an example, the noise refers to a random disturbance value.

[0117] In an example, the sample upper limb position refers to the initial upper limb position after adding noise.

[0118] Therefore, the robot can sample the sample state information to obtain a plurality of initial upper limb positions, and then add noise to the selected initial upper limb position, and take the position containing noise as the sample upper limb position.

[0119] In an embodiment of the present application, by adding noise to the upper limb position instruction, the instruction error in actual control is simulated, the overfitting of the model is further reduced, and the adaptability to instruction disturbance is improved.

[0120] According to an embodiment of the present application, the model training process can be as shown in the following figure. Figure 3 Fig. 2 shows a flowchart of a model training method provided by another example embodiment of the present application. Figure 3 The embodiments are Figure 2 The example of the model training process corresponding to the embodiments is described below. For the same parts, reference can be made to the description in the above embodiments, which will not be described here again. For example, Figure 3 As shown in the figure, the robot control method can include the following contents.

[0121] 310: Collect an open-source trajectory dataset, adapt it to the upper limb structure of the target robot through a remapping module, and collect a small amount of real machine upper limb trajectory data of the target robot using a teleoperation module, and generate an upper limb trajectory dataset according to the open-source trajectory dataset and the real machine upper limb trajectory data.

[0122] 320: Run multiple robot simulation environments in parallel, each environment randomly samples sample upper limb information from the upper limb trajectory dataset as a reference motion trajectory, and performs dynamics enhancement on the sample upper limb information to obtain dynamics-enhanced sample upper limb information.

[0123] In an example, the dynamics enhancement on the sample upper limb information can be adjusting the mass of each joint in the upper limb of the target robot.

[0124] In an example, the dynamics enhancement on the sample upper limb information can be adjusting the control coefficient of each joint in the upper limb of the target robot.

[0125] In an example, the dynamics enhancement on the sample upper limb information can be adjusting the angle of each joint in the upper limb of the target robot.

[0126] In an example, the dynamics enhancement on the sample upper limb information can be adjusting the motion speed of each joint in the upper limb of the target robot.

[0127] In an example, the dynamics enhancement on the sample upper limb information can be randomly switching the reference motion trajectory to a random frame of other sample upper limb information in the upper limb trajectory dataset to generate discontinuous and violent swing interference.

[0128] 330: Add the dynamics-enhanced sample upper limb information to the upper limb trajectory dataset.

[0129] 340: Sample the sample upper limb information in the upper limb trajectory dataset to obtain multiple initial upper limb positions.

[0130] 350: Add noise to the sampled initial upper limb positions to obtain sample upper limb positions, and combine the sample state information to serve as the input part of the prediction model to obtain the sample lower limb information output by the prediction model.

[0131] The input of the prediction model includes: torso angular velocity, gravity vector of the torso coordinate system, velocity command, position and angular velocity of all joints, and parameters reflecting the current motion state of the robot, and sample upper limb position. The prediction model can infer sample lower limb information at a preset frequency, i.e., the position (angle) of each joint in the lower limb of the robot, and convert it into joint torque.

[0132] 360: Simulate the sample lower limb information, determine the stability of the sample lower limb information, and adjust the model parameters based on the stability of each sample lower limb information using reinforcement learning.

[0133] Wherein, the stability can be measured by using the center of gravity, attitude angle change, joint force fluctuation and other parameters.

[0134] In an example, the center of gravity offset can be the offset distance and direction of the robot's center of gravity during simulation. Excessive center of gravity offset can cause the robot to lose balance.

[0135] In an example, the attitude angle change can be used to monitor the change of the attitude angle (such as pitch angle, roll angle, yaw angle) of the robot's torso or key parts. Sharp attitude angle change can indicate instability.

[0136] In an example, the joint force fluctuation can be the fluctuation range of the force and torque borne by each joint of the lower limb. Abnormal force fluctuation can affect the normal movement and stability of the robot.

[0137] In an example, suitable reinforcement learning algorithms can be selected according to the task characteristics and model structure, such as deep Q network (DQN), proximal policy optimization algorithm (PPO), etc. Different algorithms have their own advantages and disadvantages in handling complex problems and convergence speed, and need to be selected according to the actual situation.

[0138] In the reinforcement learning process, the model parameters can be initialized first, and the reinforcement learning hyperparameters are set. Then, the reward function is defined according to the stability evaluation results of the sample lower limb information. The design of the reward function should encourage the model output to make the robot move more stably. For example, if the simulation results of the sample lower limb information show that the robot moves stably and all stability indicators are within a reasonable range, a positive reward value is given; otherwise, if the movement is unstable, a negative reward value is given. The size of the reward value can be adjusted according to the stability deviation to more accurately guide the model learning. Then, the model parameters can be adjusted based on the reward function to complete the reinforcement learning training process.

[0139] 370: The simulation environment collects data for a certain period of time, and uses the collected data, i.e. the upper limb trajectory data set, to iterate the prediction model, and uses the updated prediction model for the next stage of simulation.

[0140] 380: When the test conditions are met, deploy the prediction model to the target robot of the real machine, control the upper limb to make various movements (such as twisting, high-frequency shaking) through the teleoperation module, and test its stability when walking and standing.

[0141] 390: If there is an unstable state (such as the robot falling over or abnormal joint movement), collect the upper limb trajectory data in the unstable state and add it to the upper limb trajectory dataset.

[0142] In this embodiment, through the above steps, the reinforcement learning model can learn whole-body control strategies in a variety of disturbance scenarios, and gradually improve the adaptability and stability to complex upper limb disturbances.

[0143] It should be understood that the execution order of the above steps can be adjusted according to actual needs.

[0144] Exemplary devices

[0145] The embodiment of the present application also provides a robot, which includes a control module, which is used to execute the above Figure 2 ,and Figure 3 The robot control method provided in any one of the embodiments.

[0146] The specific functions and effects of the robot provided in the embodiments of the present application can be referred to the description in the above method embodiments. In order to avoid repetition, they will not be described here.

[0147] Figure 4 The figure shows a schematic diagram of the structure of a robot control device provided by an exemplary embodiment of the present application. Figure 4 As shown, the robot control device 400 includes a determination module 410 , a prediction module 420 and a control module 430 .

[0148] The determination module 410 is used to determine the robot's state information and the robot's upper limb motion information.

[0149] The prediction module 420 is used to input the state information and upper limb motion information into the trained prediction model to obtain the robot's lower limb motion information output by the prediction model. The prediction model is trained based on the sample upper limb information after dynamic enhancement, and the lower limb motion information is used to characterize the angles of each joint in the robot's lower limbs.

[0150] The control module 430 is used to control the robot according to the lower limb movement information.

[0151] The robot control device 400 further includes a training module 440 .

[0152] Optionally, the training module 440 is configured to: determine sample upper limb information, and perform dynamics enhancement on the sample upper limb information to obtain dynamics-enhanced sample upper limb information; determine sample state information, and determine a plurality of sample data according to the sample upper limb information and the dynamics-enhanced sample upper limb information and the sample state information; input the plurality of sample data into the to-be-trained prediction model to obtain a plurality of sample lower limb information output by the prediction model; and train the prediction model in a reinforcement learning manner based on stability of the plurality of sample lower limb information.

[0153] Optionally, the training module 440 is configured to: determine a plurality of enhanced joints from joints included in an upper limb of the robot, adjust an angle of each enhanced joint in the plurality of enhanced joints to obtain an adjusted joint angle, and determine the dynamics-enhanced sample upper limb information according to the plurality of enhanced joints after the joint angle is adjusted.

[0154] Optionally, the training module 440 is configured to: determine a plurality of enhanced joints from joints included in an upper limb of the robot, adjust a mass of each enhanced joint in the plurality of enhanced joints to obtain an adjusted joint mass, and determine the dynamics-enhanced sample upper limb information according to the plurality of enhanced joints after the joint mass is adjusted.

[0155] Optionally, the training module 440 is configured to: determine a plurality of enhanced joints from joints included in an upper limb of the robot, adjust a control coefficient of each enhanced joint in the plurality of enhanced joints to obtain an adjusted control coefficient, the control coefficient being used to represent a response speed of the joint to an angle error, and determine the dynamics-enhanced sample upper limb information according to the plurality of enhanced joints after the control coefficient is adjusted.

[0156] Optionally, the sample upper limb information is a sample upper limb trajectory of the robot, and the training module 440 is configured to: determine a plurality of enhanced joints from joints included in the robot, adjust a motion speed of each enhanced joint in the plurality of enhanced joints to obtain an adjusted motion speed, the motion speed being used to represent a playing speed of the sample upper limb trajectory, and determine the dynamics-enhanced sample upper limb information according to the plurality of enhanced joints after the motion speed is adjusted.

[0157] Optionally, the sample upper limb information is a sample upper limb trajectory of the robot, and the training module 440 is configured to: determine specific sample information from other sample upper limb information, control joints of the robot to stop playing the sample upper limb trajectory of the sample upper limb information, and play a sample upper limb trajectory corresponding to the specific sample information, and determine the dynamics-enhanced sample upper limb information according to an action of the robot.

[0158] Optionally, the training module 440 is configured to deploy the prediction model on the robot in a case where the test condition is met, test the robot, in a case where there is an unstable state during the testing, collect upper limb action information corresponding to the unstable state, and re-determine the sample upper limb information according to the upper limb action information corresponding to the unstable state, and train the prediction model based on the re-determined sample upper limb information.

[0159] Optionally, the training module 440 is configured to sample the sample upper limb information and the sample upper limb information enhanced by dynamics respectively to obtain a plurality of sample upper limb positions, and combine the plurality of sample upper limb positions with the sample state information respectively to obtain a plurality of sample data.

[0160] Optionally, the training module 440 is configured to sample the sample upper limb information to obtain a plurality of initial upper limb positions, and adjust parameters of at least part of the initial upper limb positions in the plurality of initial upper limb positions to obtain a plurality of sample upper limb positions.

[0161] It should be understood that the operations and functions of the determination module 410, the prediction module 420, the control module 430 and the training module 440 in the above embodiments can refer to the descriptions of the robot control method provided in the above embodiments. For the sake of brevity, they will not be repeated here. Figure 2

[0162] Figure 5 Fig. 5 shows a block diagram of an electronic device 500 for executing the robot control method according to an example embodiment of the present application. The electronic device 500 can be specifically a robot, a server or other device interacting with the robot.

[0163] Referring to Figure 5 , the electronic device 500 includes a processing component 510, which further includes one or more processors, and a memory resource represented by a memory 520, for storing instructions executable by the processing component 510, such as an application program. The application program stored in the memory 520 can include one or more than one module each corresponding to a set of instructions. In addition, the processing component 510 is configured to execute the instructions to perform the robot control method described above.

[0164] The electronic device 500 can further include a power supply component configured to perform power management of the electronic device 500, a wired or wireless network interface configured to connect the electronic device 500 to a network, and an input / output (I / O) interface. The electronic device 500 can be operated based on an operating system stored in the memory 520, such as Windows Server TM , MacOSX TM , Unix TM , Linux TM ​FreeBSD TM or the like.

[0165] A non-transitory computer-readable storage medium, when the instructions in the storage medium are executed by the processor of the electronic device 500, enable the electronic device 500 to perform a robot control method.

[0166] A computer program product, the computer program product comprising a computer program, the computer program being executed by a processor of a computer device, enabling the computer device to perform the robot control method provided in any of the above embodiments.

[0167] All the optional technical solutions described above can be combined to form optional embodiments of the present application, which will not be described one by one here.

[0168] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in connection with the embodiments disclosed herein can be realized by electronic hardware, or a combination of computer software and electronic hardware. Whether the functions are performed by hardware or software depends on the specific application and design constraints of the technical solutions. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0169] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described system, device and unit can refer to the corresponding processes in the foregoing method embodiments, which will not be described here.

[0170] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interface, device or unit, and can be electrical, mechanical or other forms.

[0171] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., they can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0172] In addition, each function unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit.

[0173] The functions, if implemented in the form of software function units and sold or used as independent products, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product, and the computer software product is stored in a storage medium, and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various program code storage media.

[0174] It should be noted that in the description of the present application, the terms "first", "second", "third" and the like are only for the purpose of description, and cannot be understood as indicating or implying relative importance. In addition, in the description of the present application, unless otherwise specified, the meaning of "a plurality of" is two or more.

[0175] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation portal for user to choose authorization or refusal.

[0176] The above only describes the preferred embodiments of the present application and does not limit the present application. Any modification, equivalent replacement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A robot control method, characterized in that: The robot control method comprises: Determining state information of a robot and upper limb motion information of the robot; Inputting the state information and the upper limb motion information into a trained prediction model to obtain lower limb motion information of the robot output by the prediction model, wherein the prediction model is trained based on the dynamically enhanced sample upper limb information, and the lower limb motion information is used to represent the angles of each joint in the lower limb of the robot; The robot is controlled according to the lower limb movement information.

2. The robot control method according to claim 1, characterized in that: The prediction model is trained in the following way: Determining sample upper limb information, and dynamically enhancing the sample upper limb information to obtain dynamically enhanced sample upper limb information; Determining sample state information, and determining a plurality of sample data based on the sample upper limb information, the dynamically enhanced sample upper limb information, and the sample state information; Inputting the plurality of sample data into the prediction model to be trained to obtain a plurality of sample lower limb information output by the prediction model; Based on the stability of the lower limb information of the multiple samples, the prediction model is trained using reinforcement learning.

3. The robot control method according to claim 2, characterized in that: The step of dynamically enhancing the sample upper limb information to obtain dynamically enhanced sample upper limb information includes: determining a plurality of enhanced joints from among the joints included in the upper limb of the robot; For each of the plurality of enhanced joints, adjusting the angle of the enhanced joint to obtain an adjusted joint angle; According to the multiple enhanced joints after adjusting the joint angles, the sample upper limb information after dynamic enhancement is determined.

4. The robot control method according to claim 2, characterized in that: The step of dynamically enhancing the sample upper limb information to obtain dynamically enhanced sample upper limb information includes: determining a plurality of enhanced joints from among the joints included in the upper limb of the robot; For each of the plurality of enhanced joints, adjusting the mass of the enhanced joint to obtain an adjusted joint mass; According to the plurality of enhanced joints after adjusting the joint masses, the sample upper limb information after dynamic enhancement is determined.

5. The robot control method according to claim 2, wherein: The step of dynamically enhancing the sample upper limb information to obtain dynamically enhanced sample upper limb information includes: determining a plurality of enhanced joints from among the joints included in the upper limb of the robot; For each of the plurality of enhanced joints, adjusting a control coefficient of the enhanced joint to obtain an adjusted control coefficient, wherein the control coefficient is used to characterize a response speed of the joint to an angle error; According to the multiple enhanced joints after adjusting the control coefficients, the sample upper limb information after dynamic enhancement is determined.

6. The robot control method according to claim 2, characterized in that: The sample upper limb information is a sample upper limb trajectory of the robot; The step of dynamically enhancing the sample upper limb information to obtain dynamically enhanced sample upper limb information includes: Determining a plurality of enhanced joints from among the joints included in the robot; For each of the plurality of enhanced joints, adjusting a movement speed of the enhanced joint to obtain an adjusted movement speed, wherein the movement speed is used to represent a playback speed of the sample upper limb trajectory; The sample upper limb information after dynamic enhancement is determined based on the multiple enhanced joints after the movement speed is adjusted.

7. The robot control method according to claim 2, characterized in that: The sample upper limb information is a sample upper limb trajectory of the robot; The step of dynamically enhancing the sample upper limb information to obtain dynamically enhanced sample upper limb information includes: Determine specific sample information from other sample upper limb information; Controlling each joint of the robot to stop playing the sample upper limb trajectory of the sample upper limb information and play the sample upper limb trajectory corresponding to the specific sample information; According to the movement of the robot, the sample upper limb information after dynamic enhancement is determined.

8. The robot control method according to claim 2, characterized in that: The training process of the prediction model also includes: When the test conditions are met, deploying the prediction model on the robot and testing the robot; When an unstable state exists during the test, the upper limb movement information corresponding to the unstable state is collected, and the sample upper limb information is re-determined based on the upper limb movement information corresponding to the unstable state, and the prediction model is trained based on the re-determined sample upper limb information.

9. The robot control method according to claim 2, characterized in that: The determining of a plurality of sample data according to the sample upper limb information, the dynamically enhanced sample upper limb information, and the sample state information includes: Sampling the sample upper limb information and the dynamically enhanced sample upper limb information respectively to obtain a plurality of sample upper limb positions; The plurality of sample upper limb positions are respectively combined with the sample state information to obtain a plurality of sample data.

10. The robot control method according to claim 9, characterized in that: The sample upper limb information is sampled to obtain multiple sample upper limb positions, including: Sampling the sample upper limb information to obtain multiple initial upper limb positions; Parameters of at least some of the multiple initial upper limb positions are adjusted to obtain the multiple sample upper limb positions.

11. A robot, characterized in that: The robot comprises a control module, wherein the control module is used to execute the robot control method according to any one of claims 1 to 10.

12. An electronic device, characterized in that: The electronic device includes: a processor; and a memory for storing processor-executable instructions, wherein the processor is configured to execute the robot control method according to any one of claims 1 to 10.

13. A computer program product, characterized in that The computer program product includes a computer program, and when the computer program is executed by a processor of a computer device, the computer device is enabled to execute the robot control method according to any one of claims 1 to 10.

Images