Robot control method, device, storage medium and robot

Abstract

The present disclosure relates to a robot control method and device, a storage medium and a robot, and relates to the field of terminal control. The method comprises: obtaining control information corresponding to a current task scene, the control information comprising a control sequence of forward kinematics control and inverse kinematics control, and first control information corresponding to the forward kinematics control and second control information corresponding to the inverse kinematics control. A digital twin model of the robot is controlled to perform a target action in a virtual scene according to the control information. The robot is controlled to perform the target action according to motion information of the digital twin model when performing the target action. The present disclosure combines the control method of forward kinematics and inverse kinematics to control the robot, and can more flexibly and accurately control the robot.

Description

Technical Field

[0001] This disclosure relates to the field of terminal technology, specifically to a robot control method, device, storage medium, and robot. Background Technology

[0002] With the rapid development of artificial intelligence, robots are increasingly being used in people's daily lives. Robot kinematics includes forward kinematics and inverse kinematics. Forward kinematics calculates the position and orientation of the robot's end effector given the joint variables of each joint, while inverse kinematics calculates the joint variables at the corresponding position of the robot's end effector, given its position and orientation. However, forward kinematics makes it difficult to calculate full-arm control data based on the target end effector position, while inverse kinematics makes it difficult to effectively control the orientation of individual joints of the entire arm, hindering flexible robot control. Summary of the Invention

[0003] The purpose of this disclosure is to provide a robot control method, device, storage medium, and robot to improve the flexibility of robot control.

[0004] According to a first aspect of the present disclosure, a robot control method is provided, the method comprising:

[0005] Obtain control information corresponding to the current task scenario. The control information includes the control sequence of forward kinematic control and inverse kinematic control, as well as the first control information corresponding to forward kinematic control and the second control information corresponding to inverse kinematic control.

[0006] The digital twin model of the robot is controlled to perform target actions in a virtual scene based on the control information.

[0007] The robot is controlled to perform the target action based on the motion information of the digital twin model when performing the target action.

[0008] Optionally, obtaining the control information corresponding to the current task scenario includes:

[0009] Obtain the task category corresponding to the task scenario;

[0010] Obtain the environmental information of the task scenario;

[0011] The control information is determined based on the task category and the environmental information.

[0012] Optionally, controlling the digital twin model of the robot to perform the target action in the virtual scene according to the control information includes:

[0013] The digital twin model is controlled to perform the target action in a virtual scene according to the control sequence, the first control information, and the second control information.

[0014] Optionally, the first control information includes a first target angle of the first target joint of the digital twin model; the second control information includes the target position of the target control point of the digital twin model; controlling the digital twin model to perform the target action in the virtual scene according to the control sequence, the first control information, and the second control information includes:

[0015] According to the control sequence and the first control information, the first target joint is controlled to rotate to the first target angle in the virtual scene;

[0016] According to the control sequence and the second control information, the target control point is controlled to move to the target location in the virtual scene.

[0017] Optionally, controlling the target control point to move to the target location in the virtual scene according to the control sequence and the second control information includes:

[0018] Determine the second target angle of the second target joint of the digital twin model based on the target position;

[0019] Control the second target joint to rotate to the second target angle so that the target control point reaches the target position.

[0020] Optionally, the motion information includes the angle information of each joint of the digital twin model; controlling the robot to perform the target action based on the motion information of the digital twin model when performing the target action includes:

[0021] The angle information of each joint of the digital twin model is collected at a preset frequency when the target action is performed.

[0022] The robot is controlled based on the angle information so that it performs the target action.

[0023] According to a second aspect of the present disclosure, a robot control device is provided, the device comprising:

[0024] The acquisition module is configured to acquire control information corresponding to the current task scenario. The control information includes the control sequence of forward kinematic control and inverse kinematic control, as well as the first control information corresponding to forward kinematic control and the second control information corresponding to inverse kinematic control.

[0025] The first control module is configured to control the robot's digital twin model to perform target actions in a virtual scene based on the control information.

[0026] The second control module is configured to control the robot to perform the target action based on the motion information when the digital twin model performs the target action.

[0027] Optionally, the acquisition module is configured as follows:

[0028] Obtain the task category corresponding to the task scenario;

[0029] Obtain the environmental information of the task scenario;

[0030] The control information is determined based on the task category and the environmental information.

[0031] Optionally, the first control module is configured to:

[0032] The digital twin model is controlled to perform the target action in a virtual scene according to the control sequence, the first control information, and the second control information.

[0033] Optionally, the first control information includes a first target angle of the first target joint of the digital twin model; the second control information includes the target position of the target control point of the digital twin model; the first control module includes:

[0034] The first control submodule is configured to control the first target joint to rotate to the first target angle in the virtual scene according to the control sequence and the first control information;

[0035] The second control submodule is configured to control the target control point to move to the target position in the virtual scene according to the control sequence and the second control information.

[0036] Optionally, the second control submodule is configured as follows:

[0037] Determine the second target angle of the second target joint of the digital twin model based on the target position;

[0038] Control the second target joint to rotate to the second target angle so that the target control point reaches the target position.

[0039] Optionally, the motion information includes the angle information of each joint of the digital twin model; the second control module includes:

[0040] The acquisition submodule is configured to acquire angle information of each joint of the digital twin model when it performs the target action at a preset frequency;

[0041] The third control submodule is configured to control the robot based on the angle information so that the robot performs the target action.

[0042] According to a third aspect of the present disclosure, a non-transitory computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps of the method described in the first aspect of the present disclosure.

[0043] According to a fourth aspect of the present disclosure, a robot is provided, comprising:

[0044] A memory on which computer programs are stored;

[0045] A processor for executing the computer program in the memory to implement the steps of the method described in the first aspect of this disclosure.

[0046] Through the above technical solution, this disclosure first obtains the control information corresponding to the current task scenario. This control information includes the control sequence of forward kinematics control and inverse kinematics control, as well as the first control information corresponding to forward kinematics control and the second control information corresponding to inverse kinematics control. Then, based on the control information, the robot's digital twin model is controlled to execute the target action in the virtual scene. Furthermore, based on the motion information of the digital twin model when executing the target action, the robot is controlled to perform the target action. This disclosure combines forward and inverse kinematics control methods to control the robot, enabling more flexible and precise robot control.

[0047] Other features and advantages of this disclosure will be described in detail in the following detailed description section. Attached Figure Description

[0048] The accompanying drawings are provided to further illustrate the present disclosure and form part of the specification. They are used together with the following detailed description to explain the present disclosure, but do not constitute a limitation thereof. In the drawings:

[0049] Figure 1 This is a flowchart illustrating a robot control method according to an exemplary embodiment;

[0050] Figure 2 This is a flowchart illustrating another robot control method according to an exemplary embodiment;

[0051] Figure 3 This is a flowchart illustrating another robot control method according to an exemplary embodiment;

[0052] Figure 4 This is a flowchart illustrating another robot control method according to an exemplary embodiment;

[0053] Figure 5 This is a block diagram illustrating a robot control device according to an exemplary embodiment;

[0054] Figure 6 This is a block diagram illustrating another robot control device according to an exemplary embodiment;

[0055] Figure 7 This is a block diagram illustrating another robot control device according to an exemplary embodiment;

[0056] Figure 8 This is a block diagram illustrating a robot according to an exemplary embodiment. Detailed Implementation

[0057] The specific embodiments of this disclosure will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit this disclosure.

[0058] Before introducing the robot control method, device, storage medium, and robot shown in the embodiments of this disclosure, the application scenarios involved in the embodiments of this disclosure will first be introduced. The robot in this disclosure can be equipped with UE5 (Unreal Engine 5). A digital twin model of the robot can be built in UE5. This digital twin model has a skinned skeleton system, and an animation blueprint using the robot skeleton can be created. Control of the robot is achieved by controlling the digital twin model. The motion control of a robot arm generally refers to how to control the robot arm to reach a target state from its current state. The target state can be described using the joint states of the robot arm (i.e., the angle of each joint) or the three-dimensional spatial coordinates of the position of the robot arm's end effector. When using joint states to describe the target state of the robot arm, transformBone can be used in UE5 to implement motion control of each joint of the robot arm (i.e., forward kinematics control); when using the position of the robot arm's end effector to describe the target state of the robot arm, fullbody-IK can be used in UE5 to implement motion control of each joint of the robot arm (i.e., inverse kinematics control).

[0059] Figure 1 This is a flowchart illustrating a robot control method according to an exemplary embodiment, such as... Figure 1 As shown, the method may include:

[0060] Step 101: Obtain the control information corresponding to the current task scenario. The control information includes the control sequence of forward kinematics control and inverse kinematics control, as well as the first control information corresponding to forward kinematics control and the second control information corresponding to inverse kinematics control.

[0061] For example, a robot may encounter multiple task scenarios when performing a task. These scenarios could include "moving to the target location," "avoiding obstacles," or "grabbing an apple from a table." The robot can acquire control information corresponding to the current task scenario. This control information includes the control sequence of forward and inverse kinematics control, as well as the first control information corresponding to forward kinematics control and the second control information corresponding to inverse kinematics control. In other words, forward and inverse kinematics control can be combined to control the robot, enabling it to perform related actions more flexibly.

[0062] In some embodiments, the control order of forward kinematics control and inverse kinematics control can be pre-defined for each task scenario. For example, for a task scenario of "moving to the target location," only forward kinematics control can be used; for a task scenario of "avoiding obstacles in front," inverse kinematics control can be used first, followed by forward kinematics control; and for a task scenario of "grabbing an apple from the table in front," forward kinematics control can be used first, followed by inverse kinematics control. After determining the current task scenario, the control order corresponding to the task scenario can be obtained.

[0063] In other embodiments, the first control information may include a first target angle corresponding to a first target joint of the digital twin model, and the second control information may include a target position corresponding to a target control point of the digital twin model. After determining the task scenario, the first and second control information can be determined based on the task category and environmental information corresponding to the task scenario.

[0064] Step 102: Control the robot's digital twin model to perform the target action in the virtual scene according to the control information.

[0065] For example, a digital twin model can be controlled to perform target actions in a virtual scene according to the control sequence, first control information, and second control information. The virtual scene can be understood as a scene built in UE5. Taking the control sequence of first using forward kinematics control and then inverse kinematics control as an example, forward kinematics control can be executed according to the first control information to adjust the first target joint of the digital twin model to the first target angle. Then, inverse kinematics control can be executed according to the second control information to move the target control point of the digital twin model to the target position. For example, if the first target joints are the joints of the left arm, the first target angle is 0 degrees, the target control point is the end of the left arm, and the target position is position A, then the joints of the left arm can be rotated to 0 degrees using transformBone, and then the end of the left arm can be moved to position A using fullbodyIK.

[0066] Taking the control sequence of first employing inverse kinematics control and then forward kinematics control as an example, inverse kinematics control can be executed according to the second control information to move the target control point of the digital twin model to the target position. Then, forward kinematics control can be executed according to the first control information to adjust the first target joint of the digital twin model to the first target angle. For example, if the first target joints are the joints of the left arm, the first target angle is 0 degrees, the target control point is the end of the left arm, and the target position is position A, then the end of the left arm can first be controlled to move to position A via fullbodyIK, and then the joints of the left arm can be rotated to 0 degrees via transformBone.

[0067] Step 103: Control the robot to perform the target action based on the motion information of the digital twin model when performing the target action.

[0068] For example, when a digital twin model performs a target action, motion information of the digital twin model can be collected at a preset frequency. This motion information can include the angle information of each joint, thus obtaining multiple angle information of each joint when the digital twin model performs the target action. Then, the robot can be controlled sequentially according to the multiple angle information of each joint, so that the robot performs the same target action.

[0069] In summary, this disclosure first obtains the control information corresponding to the current task scenario. This control information includes the control sequence of forward kinematics control and inverse kinematics control, as well as the first control information corresponding to forward kinematics control and the second control information corresponding to inverse kinematics control. Then, based on the control information, the robot's digital twin model is controlled to execute the target action in the virtual scene. Furthermore, based on the motion information of the digital twin model when executing the target action, the robot is controlled to perform the target action. This disclosure combines forward and inverse kinematics control methods to control the robot, enabling more flexible and precise robot control.

[0070] Figure 2 This is a flowchart illustrating another robot control method according to an exemplary embodiment, such as... Figure 2 As shown, step 101 can be achieved through the following steps:

[0071] Step 1011: Obtain the task category corresponding to the task scenario.

[0072] Step 1012: Obtain environmental information for the task scenario.

[0073] Step 1013: Determine control information based on task category and environmental information.

[0074] For example, during the robot's task execution, it can first obtain the task category corresponding to the current task scenario. For instance, if the current task scenario is "grab the apple on the table in front", the corresponding task category can be a grasping task. Or, if the current task scenario is "avoid the obstacle in front", the corresponding task category can be an obstacle avoidance task. Or, if the current task scenario is "move to the target location", the corresponding task category can be a moving task.

[0075] In some embodiments, environmental information of the current task scenario can also be obtained. For example, if the current task scenario is "grabbing an apple from the table in front", then the position coordinates of the apple can be obtained using computer vision algorithms as environmental information. If the current task scenario is "avoiding obstacles in front", then the position and outline information of the obstacles can be obtained using computer vision algorithms as environmental information. If the current task scenario is "moving to the target location", then the distance from the robot to the target location can be obtained as environmental information.

[0076] In other embodiments, the robot's control information can be determined based on the task category and environmental information. For example, a control sequence of forward kinematics control and inverse kinematics control can be pre-defined for each task category to obtain a preset correspondence between task categories and control sequences. After obtaining the task category corresponding to the current task scenario, the control sequence corresponding to that task category can be determined based on the preset correspondence. Then, by combining the control sequence and environmental information with a preset control algorithm, the first control information corresponding to forward kinematics control and the second control information corresponding to inverse kinematics control can be obtained.

[0077] Figure 3 This is a flowchart illustrating another robot control method according to an exemplary embodiment, such as... Figure 3 As shown, step 102 can be achieved through the following steps:

[0078] Step 1021: According to the control sequence and the first control information, control the first target joint to rotate to the first target angle in the virtual scene.

[0079] For example, the first control information may include the first target angle corresponding to the first target joint of the digital twin model. Following the sequence of forward kinematic control as indicated by the control sequence, and the first control information for forward kinematic control, the first target joint of the digital twin model can be controlled to rotate to the first target angle. For instance, taking a control sequence of first performing inverse kinematic control and then forward kinematic control, with the first target joint including all joints of the left arm, and the first target angle being 150 degrees, after completing the inverse kinematic control, the various joints of the left arm can be controlled to rotate to 150 degrees.

[0080] In some embodiments, forward kinematic control can be implemented in UE5 using transformBone. Taking a robot including a six-axis robotic arm as an example, an array of length 6 can be used to control the motion posture of a digital twin model of a six-axis robotic arm. The i-th value in the array is used to generate a rotation angle around the rotation axis, and then the rotation angle is used as a parameter of the transformBone method to set the rotation angle of the i-th joint of the robotic arm, thereby rotating each joint to the corresponding angle.

[0081] Step 1022: According to the control sequence and the second control information, control the target control point to move to the target position in the virtual scene.

[0082] For example, the second control information may include the target position of the target control point in the digital twin model. The target control point can be moved to the target position in the virtual scene according to the sequence of inverse kinematics control indicated by the control sequence, and the second control information of the inverse kinematics control.

[0083] In another embodiment, step 1022 can be implemented in the following way:

[0084] The second target angle of the second target joint of the digital twin model is determined based on the target location.

[0085] Control the second target joint to rotate to the second target angle so that the target control point reaches the target position.

[0086] For example, taking a control sequence of first performing forward kinematics control and then inverse kinematics control, with the target control point being the left end of the robotic arm and the target position being the robot's chest position, after completing the inverse kinematics control, the second target angle corresponding to the second target joint of the digital twin model can be obtained based on the coordinates of the chest position using a preset calculation algorithm. The second target joint includes the joints that require the left end of the robotic arm to rotate to reach the chest position. Then, the second target joint is controlled to rotate to the second target angle, causing the left end of the robotic arm to reach the chest position.

[0087] For example, inverse kinematics control can be implemented in UE5 using FullBodyIK. Taking the target control point as the end effector of the arm as an example, the solver of FullBodyIK can be used to calculate the target position of the end effector of the arm indicated by the second control information, and obtain the angle data of each joint of the robot arm. When each joint of the robot arm is assigned the corresponding angle data, the end effector of the arm will reach the target position.

[0088] Figure 4 This is a flowchart illustrating another robot control method according to an exemplary embodiment, such as... Figure 4 As shown, step 103 can be achieved through the following steps:

[0089] Step 1031: Collect angle information of each joint of the digital twin model when it performs the target action at a preset frequency.

[0090] Step 1032: Control the robot according to the angle information so that the robot can perform the target action.

[0091] For example, the motion information of a digital twin model when performing a target action includes the angle information of each joint of the digital twin model. This angle information can be collected at a preset frequency. For instance, the preset frequency could be 40 times per second, meaning 40 angle measurements of each joint are collected per second, thus obtaining multiple consecutive angle measurements of each joint when the digital twin model performs the target action. Then, the robot's joints can be controlled sequentially according to the collected angle information, enabling the robot to perform the same target action as the digital twin model.

[0092] Taking a preset frequency of 40 times / second and a target action execution time of 5 seconds as an example, the digital twin model can collect angle information of each joint 200 times when executing the target action, with a total of 200 angle information points collected for each joint. Then, according to the time of collecting each angle information, the angle of each joint of the robot can be controlled sequentially, so that each joint of the robot is adjusted 200 times to complete the target action.

[0093] In summary, this disclosure first obtains the control information corresponding to the current task scenario. This control information includes the control sequence of forward kinematics control and inverse kinematics control, as well as the first control information corresponding to forward kinematics control and the second control information corresponding to inverse kinematics control. Then, based on the control information, the robot's digital twin model is controlled to execute the target action in the virtual scene. Furthermore, based on the motion information of the digital twin model when executing the target action, the robot is controlled to perform the target action. This disclosure combines forward and inverse kinematics control methods to control the robot, enabling more flexible and precise robot control.

[0094] Figure 5 This is a block diagram illustrating a robot control device according to an exemplary embodiment, such as... Figure 5 As shown, the device 200 includes:

[0095] The acquisition module 201 is configured to acquire control information corresponding to the current task scenario. The control information includes the control sequence of forward kinematic control and inverse kinematic control, as well as the first control information corresponding to forward kinematic control and the second control information corresponding to inverse kinematic control.

[0096] The first control module 202 is configured to control the robot's digital twin model to perform target actions in a virtual scene based on control information.

[0097] The second control module 203 is configured to control the robot to perform the target action based on the motion information of the target action performed by the digital twin model.

[0098] In some embodiments, the acquisition module 201 is configured to:

[0099] Obtain the task category corresponding to the task scenario.

[0100] Obtain environmental information about the task scenario.

[0101] Control information is determined based on task category and environmental information.

[0102] In other embodiments, the first control module 202 is configured to:

[0103] According to the control sequence, the first control information, and the second control information, the digital twin model is controlled to perform the target action in the virtual scene.

[0104] Figure 6 This is a block diagram illustrating another robot control device according to an exemplary embodiment, such as... Figure 6 As shown, the first control information includes the first target angle of the first target joint of the digital twin model. The second control information includes the target position of the target control point of the digital twin model. The first control module 202 includes:

[0105] The first control submodule 2021 is configured to control the first target joint to rotate to the first target angle in the virtual scene according to the control sequence and the first control information.

[0106] The second control submodule 2022 is configured to control the target control point to move to the target position in the virtual scene according to the control sequence and the second control information.

[0107] In other embodiments, the second control submodule 2022 is configured as follows:

[0108] The second target angle of the second target joint of the digital twin model is determined based on the target location.

[0109] Control the second target joint to rotate to the second target angle so that the target control point reaches the target position.

[0110] Figure 7 This is a block diagram illustrating another robot control device according to an exemplary embodiment, such as... Figure 7 As shown, the motion information includes the angle information of each joint in the digital twin model. The second control module 203 includes:

[0111] The acquisition submodule 2031 is configured to acquire the angle information of each joint of the digital twin model when it performs the target action at a preset frequency.

[0112] The third control submodule 2032 is configured to control the robot based on angle information so that the robot can perform the target action.

[0113] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0114] In summary, this disclosure first obtains the control information corresponding to the current task scenario. This control information includes the control sequence of forward kinematics control and inverse kinematics control, as well as the first control information corresponding to forward kinematics control and the second control information corresponding to inverse kinematics control. Then, based on the control information, the robot's digital twin model is controlled to execute the target action in the virtual scene. Furthermore, based on the motion information of the digital twin model when executing the target action, the robot is controlled to perform the target action. This disclosure combines forward and inverse kinematics control methods to control the robot, enabling more flexible and precise robot control.

[0115] Figure 8 This is a block diagram illustrating a robot according to an exemplary embodiment. Figure 8 As shown, the robot 300 may include a processor 301 and a memory 302. The robot 300 may also include one or more of a multimedia component 303, an input / output (I / O) interface 304, and a communication component 305.

[0116] The processor 301 controls the overall operation of the robot 300 to complete all or part of the steps in the robot control method described above. The memory 302 stores various types of data to support the operation of the robot 300. This data may include, for example, instructions for any application or method used to operate on the robot 300, and application-related data such as contact data, sent and received messages, images, audio, video, etc. The memory 302 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. Multimedia component 303 may include a screen and an audio component. The screen may be, for example, a touchscreen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signals may be further stored in memory 302 or transmitted via communication component 305. The audio component also includes at least one speaker for outputting audio signals. I / O interface 304 provides an interface between processor 301 and other interface modules, such as a keyboard, mouse, buttons, etc. These buttons may be virtual or physical buttons. Communication component 305 is used for wired or wireless communication between the robot 300 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, 4G, NB-IoT, eMTC, or other 5G technologies, or combinations thereof, is not limited here. Therefore, the corresponding communication component 305 may include: a Wi-Fi module, a Bluetooth module, an NFC module, etc.

[0117] In an exemplary embodiment, the robot 300 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to execute the robot control method described above.

[0118] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, which, when executed by a processor, implement the steps of the robot control method described above. For example, the computer-readable storage medium may be the memory 302 including program instructions, which may be executed by the processor 301 of the robot 300 to complete the robot control method described above.

[0119] In another exemplary embodiment, a computer program product is also provided, the computer program product comprising a computer program executable by a programmable device, the computer program having a code portion for performing the above-described robot control method when executed by the programmable device.

[0120] The preferred embodiments of this disclosure have been described in detail above with reference to the accompanying drawings. However, this disclosure is not limited to the specific details of the above embodiments. Within the scope of the technical concept of this disclosure, various simple modifications can be made to the technical solutions of this disclosure, and these simple modifications all fall within the protection scope of this disclosure.

[0121] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, this disclosure will not describe the various possible combinations separately.

[0122] Furthermore, various different embodiments of this disclosure can be combined in any way, as long as they do not violate the spirit of this disclosure, they should also be regarded as the content disclosed in this disclosure.

Claims

1. A method for controlling a robot, characterized in that, The method includes: Obtain control information corresponding to the current task scenario. The control information includes the control sequence of forward kinematic control and inverse kinematic control, as well as the first control information corresponding to forward kinematic control and the second control information corresponding to inverse kinematic control. The digital twin model of the robot is controlled to perform target actions in a virtual scene based on the control information. Based on the motion information of the digital twin model when performing the target action, the robot is controlled to perform the target action; The step of controlling the digital twin model of the robot to perform the target action in the virtual scene according to the control information includes: controlling the digital twin model to perform the target action in the virtual scene according to the control sequence, the first control information and the second control information; Wherein, the first control information includes the first target angle of the first target joint of the digital twin model; the second control information includes the target position of the target control point of the digital twin model; controlling the digital twin model to perform the target action in the virtual scene according to the control sequence, the first control information, and the second control information includes: According to the control sequence and the first control information, the first target joint is controlled to rotate to the first target angle in the virtual scene; According to the control sequence and the second control information, the target control point is controlled to move to the target position in the virtual scene; The motion information includes the angle information of each joint of the digital twin model; controlling the robot to perform the target action based on the motion information of the digital twin model when performing the target action includes: The angle information of each joint of the digital twin model is collected at a preset frequency when the target action is performed. Based on the time it takes to collect information from each angle, the angle of each joint of the robot is controlled sequentially, so that the robot performs the same target action as the digital twin model.

2. The method according to claim 1, characterized in that, The process of obtaining the control information corresponding to the current task scenario includes: Obtain the task category corresponding to the task scenario; Obtain the environmental information of the task scenario; The control information is determined based on the task category and the environmental information.

3. The method according to claim 1, characterized in that, The step of controlling the target control point to move to the target position in the virtual scene according to the control sequence and the second control information includes: Determine the second target angle of the second target joint of the digital twin model based on the target position; Control the second target joint to rotate to the second target angle so that the target control point reaches the target position.

4. A control device for a robot, characterized in that, The device includes: The acquisition module is configured to acquire control information corresponding to the current task scenario. The control information includes the control sequence of forward kinematic control and inverse kinematic control, as well as the first control information corresponding to forward kinematic control and the second control information corresponding to inverse kinematic control. The first control module is configured to control the robot's digital twin model to perform target actions in a virtual scene based on the control information. The second control module is configured to control the robot to perform the target action based on the motion information of the digital twin model when performing the target action; The first control module is configured to control the digital twin model to perform the target action in a virtual scene according to the control sequence, the first control information, and the second control information. Wherein, the first control information includes the first target angle of the first target joint of the digital twin model; the second control information includes the target position of the target control point of the digital twin model; the first control module includes: The first control submodule is configured to control the first target joint to rotate to the first target angle in the virtual scene according to the control sequence and the first control information; The second control submodule is configured to control the target control point to move to the target position in the virtual scene according to the control sequence and the second control information; The second control module includes: The acquisition submodule is configured to acquire angle information of each joint of the digital twin model when it performs the target action at a preset frequency; The third control submodule is configured to control the angle of each joint of the robot sequentially according to the time of collecting each angle information, so that the robot performs the same target action as the digital twin model.

5. The apparatus according to claim 4, characterized in that, The acquisition module is configured as follows: Obtain the task category corresponding to the task scenario; Obtain the environmental information of the task scenario; The control information is determined based on the task category and the environmental information.

6. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the steps of the method described in any one of claims 1-3.

7. A robot, characterized in that, include: A memory on which computer programs are stored; A processor for executing the computer program in the memory to implement the steps of the method according to any one of claims 1-3.

Images