Robot action track execution method and device, readable storage medium and robot
By generating and transmitting motion trajectories through an external workstation, the problem of insufficient real-time execution of robot motion trajectories is solved, enabling efficient and real-time execution of robot motion trajectories.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- UBTECH ROBOTICS CORP LTD
- Filing Date
- 2025-12-19
- Publication Date
- 2026-04-21
AI Technical Summary
Existing methods for executing robot motion trajectories suffer from poor real-time performance, primarily due to the large computational load of visual language motion models and the limited computing power of edge computing units on the robot.
By deploying a visual language action model on an external workstation, the computing power of the external workstation is used to generate the robot's target action trajectory, which is then executed by the robot. This achieves decoupling of reasoning and execution and data synchronization, ensuring the real-time performance of the action trajectory.
It effectively overcomes the limitations of robot computing power, meets the real-time requirements of robot motion trajectory execution, and improves the efficiency and accuracy of robot motion trajectory execution.
Smart Images
Figure CN121893244A_ABST
Abstract
Description
Technical Field
[0001] This application belongs to the field of robotics technology, and in particular relates to a method, apparatus, computer-readable storage medium, and robot for executing robot motion trajectories. Background Technology
[0002] Vision-Language-Action (VLA) models are a type of multimodal artificial intelligence system that aims to integrate visual perception, natural language understanding and generation, and action decision-making or execution to achieve more intelligent and generalized embodied intelligence.
[0003] In existing technologies, visual language action models can be deployed in the edge computing unit on the robot, using local inference to generate and execute the robot's motion trajectory. However, the computational load of visual language action models is large, while the edge computing unit on the robot often has limited computing power, making it difficult to meet the real-time requirements of robot motion trajectory execution. Summary of the Invention
[0004] In view of this, embodiments of this application provide a robot motion trajectory execution method, apparatus, computer-readable storage medium, and robot to solve the problem of poor real-time performance in existing motion trajectory execution methods.
[0005] A first aspect of this application provides a robot motion trajectory execution method, applied in a robot, wherein the robot establishes a communication connection with an external workstation; the method includes:
[0006] Obtain the target motion trajectory generated by the external workstation; wherein, the target motion trajectory is the motion trajectory to be executed by the robot;
[0007] According to a preset trajectory execution frequency, each trajectory point in the target motion trajectory is sequentially sent to each joint controller of the robot, so that each joint controller of the robot executes the target motion trajectory.
[0008] In one specific implementation of the first aspect, the method may further include:
[0009] During the process of sequentially sending each trajectory point in the target motion trajectory to each joint controller of the robot according to the preset trajectory execution frequency, the preset trajectory execution index is updated; wherein, the trajectory execution index is used to indicate the number of trajectory points that have been executed;
[0010] If the trajectory execution index is greater than or equal to the preset maximum trajectory length, it is determined that the target action trajectory has been completed.
[0011] In one specific implementation of the first aspect, after determining that the target action trajectory has been completed, it may further include:
[0012] Obtain the target image of the robot, and the image timestamp corresponding to the target image; wherein, the target image is the latest image in the robot's image buffer;
[0013] The target joint state data of the robot is determined based on the image timestamp; wherein, the target joint state data is the joint state data synchronized with the target image in the robot's joint state buffer;
[0014] Perform data integrity verification on the target joint status data;
[0015] If the data integrity verification is successful, the target image and the target joint status data are sent to the external workstation so that the external workstation can generate the next round of motion trajectory based on the target image and the target joint status data.
[0016] A second aspect of this application provides a robot motion trajectory execution method, applied in an external workstation, wherein a communication connection is established between the external workstation and the robot; the method may include:
[0017] Acquire the target image and target joint state data sent by the robot; wherein the target image is the latest image in the robot's image buffer, and the target joint state data is the joint state data in the robot's joint state buffer that is synchronized with the target image;
[0018] A motion trajectory is generated based on the target image and the target joint state data to obtain the target motion trajectory; wherein, the target motion trajectory is the motion trajectory to be executed by the robot.
[0019] The target motion trajectory is sent to the robot so that the robot executes the target motion trajectory.
[0020] In one specific implementation of the second aspect, the step of generating a motion trajectory based on the target image and the target joint state data to obtain the target motion trajectory may include:
[0021] Based on the target image and the target joint state data, a preset visual language action model is invoked to perform model reasoning, and the original reasoning trajectory output by the visual language action model is obtained.
[0022] The original inference trajectory is interpolated according to the preset maximum trajectory length to obtain the target action trajectory.
[0023] In one specific implementation of the second aspect, the method may further include:
[0024] During the trajectory interpolation process of the original inference trajectory, the velocity and acceleration corresponding to the trajectory start point are both set to zero, and the velocity and acceleration corresponding to the trajectory end point are both set to zero.
[0025] A third aspect of this application provides a robot motion trajectory execution device, applied in a robot, wherein the robot establishes a communication connection with an external workstation; the device may include:
[0026] The motion trajectory acquisition module is used to acquire the target motion trajectory generated by the external workstation; wherein, the target motion trajectory is the motion trajectory to be executed by the robot;
[0027] The motion trajectory execution module is used to send each trajectory point in the target motion trajectory to each joint controller of the robot in sequence according to a preset trajectory execution frequency, so that each joint controller of the robot executes the target motion trajectory.
[0028] In one specific implementation of the third aspect, the motion trajectory execution module may include:
[0029] The trajectory execution completion determination unit is used to update a preset trajectory execution index during the process of sequentially sending each trajectory point in the target motion trajectory to each joint controller of the robot according to a preset trajectory execution frequency; wherein, the trajectory execution index is used to indicate the number of executed trajectory points; and if the trajectory execution index is greater than or equal to the preset maximum trajectory length, the target motion trajectory is determined to be completed.
[0030] In one specific implementation of the third aspect, the apparatus may further include:
[0031] A synchronous data acquisition module is used to acquire the target image of the robot and the image timestamp corresponding to the target image; wherein the target image is the latest image in the robot's image buffer; determine the target joint state data of the robot based on the image timestamp; wherein the target joint state data is the joint state data synchronized with the target image in the robot's joint state buffer; perform data integrity verification on the target joint state data; if the data integrity verification is successful, send the target image and the target joint state data to the external workstation so that the external workstation can generate the next round of motion trajectory based on the target image and the target joint state data.
[0032] A fourth aspect of this application provides a robot motion trajectory execution device, applied in an external workstation, wherein the external workstation and the robot have a communication connection; the device may include:
[0033] A synchronous data acquisition module is used to acquire the target image and target joint state data sent by the robot; wherein the target image is the latest image in the robot's image buffer, and the target joint state data is the joint state data in the robot's joint state buffer that is closest to the target image in time;
[0034] The motion trajectory generation module is used to generate a motion trajectory based on the target image and the target joint state data to obtain a target motion trajectory; wherein, the target motion trajectory is the motion trajectory to be executed by the robot;
[0035] The motion trajectory sending module is used to send the target motion trajectory to the robot so that the robot can execute the target motion trajectory.
[0036] In one specific implementation of the fourth aspect, the motion trajectory generation module may include:
[0037] The model inference unit is used to call a preset visual language action model to perform model inference based on the target image and the target joint state data, and obtain the original inference trajectory output by the visual language action model.
[0038] The trajectory interpolation unit is used to interpolate the original inference trajectory according to the preset maximum trajectory length to obtain the target action trajectory.
[0039] In one specific implementation of the fourth aspect, the motion trajectory generation module may further include:
[0040] The endpoint zero-velocity and zero-acceleration setting unit is used to set the velocity and acceleration corresponding to the starting point of the trajectory to zero and the velocity and acceleration corresponding to the ending point of the trajectory to zero during the trajectory interpolation process of the original inference trajectory.
[0041] A fifth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of any of the above-described robot motion trajectory execution methods applied to a robot.
[0042] A sixth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of any of the above-described robot motion trajectory execution methods applied to an external workstation.
[0043] A seventh aspect of this application provides a robot, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any of the robot motion trajectory execution methods described above.
[0044] An eighth aspect of this application provides an external workstation, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of any of the robot motion trajectory execution methods described above for use in an external workstation.
[0045] A ninth aspect of this application provides a computer program product that, when run on a robot, causes the robot to execute the steps of any of the robot motion trajectory execution methods described above.
[0046] The tenth aspect of this application provides a computer program product that, when run on an external workstation, causes the external workstation to execute the steps of any of the robot motion trajectory execution methods applied to the external workstation.
[0047] The beneficial effects of this application embodiment compared with the prior art are as follows: A communication connection is established between the robot and the external workstation in this application embodiment; the robot acquires the target motion trajectory generated by the external workstation; wherein, the target motion trajectory is the motion trajectory to be executed by the robot; each trajectory point in the target motion trajectory is sequentially sent to each joint controller of the robot according to a preset trajectory execution frequency, so that each joint controller of the robot executes the target motion trajectory. Through this application embodiment, the computing power resources of the external workstation can be fully utilized, with the external workstation generating the robot's target motion trajectory, and the robot only needs to execute the target motion trajectory generated by the external workstation, thereby effectively overcoming the robot's computing power limitations and meeting the real-time requirements of robot motion trajectory execution. Attached Figure Description
[0048] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0049] Figure 1 This is a schematic diagram of a robot motion trajectory execution system according to an embodiment of this application;
[0050] Figure 2 This is a flowchart of one embodiment of a robot motion trajectory execution method applied in a robot, as described in this application.
[0051] Figure 3 A schematic flowchart illustrating the robot's synchronous data acquisition and transmission process:
[0052] Figure 4 This is a flowchart of one embodiment of a robot motion trajectory execution method applied in an external workstation according to the present application;
[0053] Figure 5 This is a timing diagram of a robot motion trajectory execution method according to an embodiment of this application;
[0054] Figure 6 This is a structural diagram of one embodiment of a robot motion trajectory execution device applied in a robot, as described in this application.
[0055] Figure 7 This is a structural diagram of one embodiment of a robot motion trajectory execution device applied in an external workstation according to the present application.
[0056] Figure 8This is a schematic block diagram of a robot according to an embodiment of this application;
[0057] Figure 9 This is a schematic block diagram of an external workstation according to an embodiment of this application. Detailed Implementation
[0058] To make the inventive objectives, features, and advantages of this application more apparent and understandable, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described below are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0059] It should be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.
[0060] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.
[0061] It should also be further understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.
[0062] As used in this specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if [the described condition or event] is detected" may be interpreted, depending on the context, as "once determined," "in response to determination," "once [the described condition or event] is detected," or "in response to detection of [the described condition or event]."
[0063] Furthermore, in the description of this application, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.
[0064] Vision-Language-Action (VLA) models are a type of multimodal artificial intelligence system that aims to integrate visual perception, natural language understanding and generation, and action decision-making or execution to achieve more intelligent and generalized embodied intelligence.
[0065] In existing technologies, visual language action models can be deployed in the edge computing unit on the robot, using local inference to generate and execute the robot's motion trajectory. However, the computational load of visual language action models is large, while the edge computing unit on the robot often has limited computing power, making it difficult to meet the real-time requirements of robot motion trajectory execution.
[0066] In view of this, embodiments of this application provide a robot motion trajectory execution method, apparatus, computer-readable storage medium, and robot to solve the problem of poor real-time performance in existing motion trajectory execution methods.
[0067] In this embodiment, the computing power of the external workstation can be fully utilized to generate the target motion trajectory of the robot. The robot only needs to execute the target motion trajectory generated by the external workstation, thereby effectively getting rid of the robot's computing power limitation and meeting the real-time requirements of robot motion trajectory execution.
[0068] Figure 1 The figure shows a schematic diagram of a robot motion trajectory execution system according to an embodiment of this application. As shown, the system may include, but is not limited to, a robot and an external workstation, with a communication connection established between the robot and the external workstation. The external workstation can be an electronic device independent of the robot, and a visual language motion model can be deployed in the external workstation to generate the robot's motion trajectory.
[0069] Please see Figure 2 One embodiment of a robot motion trajectory execution method applied in a robot, as described in this application, may include:
[0070] Step S201: Obtain the target motion trajectory generated by the external workstation.
[0071] The target motion trajectory is the motion trajectory to be executed by the robot. The total number of trajectory points contained therein (denoted as the maximum trajectory length, max_trajectory_length) can be flexibly set according to the actual situation, and this embodiment does not impose a specific limitation on it. As an example, the maximum trajectory length can be set to 300 points.
[0072] Step S202: Send each trajectory point in the target action trajectory to each joint controller of the robot in sequence according to a preset trajectory execution frequency, so that each joint controller of the robot executes the target action trajectory.
[0073] Among them, the specific value of the trajectory execution frequency can be flexibly set according to the actual situation, and the embodiments of the present application do not make specific limitations on this. As an example, the trajectory execution frequency can be set to 100 Hertz (Hz).
[0074] If the reasoning and execution are completely decoupled, that is, the external workstation continuously outputs the action trajectory and the robot blindly executes it, it may occur that the previous action trajectory has not been executed yet, and the next action trajectory starts to be executed, resulting in the overlap of the new and old trajectories or the misalignment of the states.
[0075] Therefore, in a specific implementation manner of the embodiments of the present application, the reasoning and execution can be organically coupled, and the next action trajectory is generated after the previous action trajectory is executed.
[0076] If relying on a fixed time interval to trigger the reasoning, it may be impossible to accurately determine whether the action trajectory has been executed due to interference factors such as network delay and middleware scheduling jitter (for example, the actual triggering time of the trajectory execution may be later than the set period).
[0077] Therefore, in a specific implementation manner of the embodiments of the present application, a trajectory execution index (current_trajectory_index) can be introduced to indicate the number of executed trajectory points, and its initial value is 0. During the process of sending each trajectory point in the target action trajectory to each joint controller of the robot in sequence according to the preset trajectory execution frequency, each time a trajectory point is sent, the trajectory execution index can be updated by increasing it by one counting unit: current_trajectory_index = current_trajectory_index + 1.
[0078] When the trajectory execution index is less than the maximum trajectory length (i.e., current_trajectory_index < max_trajectory_length), it can be determined that the target action trajectory has not been executed yet, and the subsequent trajectory points continue to be sent. When the trajectory execution index is greater than or equal to the maximum trajectory length (i.e., current_trajectory_index ≥ max_trajectory_length), it can be determined that the target action trajectory has been executed, and there is no need to send trajectory points to the joint controller anymore.
[0079] After determining that the target action trajectory has been executed, the robot can pass through such as Figure 3 The process shown involves synchronous data acquisition and transmission:
[0080] Step S301: Obtain the target image of the robot and the image timestamp corresponding to the target image.
[0081] The robot can acquire images using image acquisition devices such as cameras at a preset image acquisition frequency, and store the acquired images and corresponding image timestamps in the robot's image buffer. The specific value of the image acquisition frequency can be flexibly set according to actual conditions, and this embodiment does not impose a specific limitation on it. As an example, the image acquisition frequency can be set to 10Hz.
[0082] Once the target motion trajectory is determined to be completed, a synchronous data acquisition operation can be triggered. At this time, the robot can acquire the latest image in the image buffer, record it as the target image, and obtain its image timestamp.
[0083] Step S302: Determine the target joint state data of the robot based on the image timestamp.
[0084] The robot can collect joint states using preset joint sensors at a preset joint state acquisition frequency, and store the collected joint states and corresponding timestamps in the robot's joint state buffer. The joint states may include, but are not limited to, joint states corresponding to the left arm, right arm, head, and waist, respectively. Correspondingly, the joint state buffers may include, but are not limited to, joint state buffers corresponding to the left arm, right arm, head, and waist, respectively. The specific value of the joint state acquisition frequency can be flexibly set according to actual conditions, and this embodiment does not impose a specific limitation on it. As an example, the joint state acquisition frequency can be set to 50Hz.
[0085] The target joint state data is the joint state data synchronized with the target image in the joint state buffer. In this embodiment, the joint state data that is closest to the image timestamp of the target image can be found in the joint state buffer and determined as the robot's target joint state data.
[0086] Step S303: Perform data integrity verification on the target joint status data.
[0087] If the target joint state data is missing, or if the time deviation between the target joint state data and the target image exceeds a preset time deviation threshold, the data integrity verification can be determined to have failed. In this case, the current inference can be abandoned, and the process can wait for the next iteration. The specific value of the time deviation threshold can be flexibly set according to the actual situation, and this application embodiment does not impose a specific limitation on it.
[0088] If the target joint state data is missing and the time deviation between the target joint state data and the target image is less than or equal to the time deviation threshold, the data integrity verification can be determined to be successful, and step S304 can be continued.
[0089] Step S304: Send the target image and target joint state data to the external workstation so that the external workstation can generate the next round of motion trajectory based on the target image and target joint state data.
[0090] It is easy to understand that, for a robot, the above Figure 2 and Figure 3 The process shown is a continuous loop. After the robot sends the target image and target joint state data to the external workstation, the external workstation can perform the next round of motion trajectory generation and send the generated new target motion trajectory to the robot, which can then follow the trajectory. Figure 2 and Figure 3 The process shown continues to execute the new target motion trajectory, then performs synchronous data acquisition, and sends the new data to the external workstation. This cycle continues until the task is completed or a stop command is received from the user.
[0091] Please see Figure 4 One embodiment of a robot motion trajectory execution method applied in an external workstation, as described in this application, may include:
[0092] Step S401: Obtain the target image and target joint state data sent by the robot.
[0093] The target image is the latest image in the robot's image buffer, and the target joint state data is the joint state data in the robot's joint state buffer that is synchronized with the target image.
[0094] Step S402: Generate the motion trajectory based on the target image and target joint state data to obtain the target motion trajectory.
[0095] The target motion trajectory is the motion trajectory to be executed by the robot.
[0096] In this embodiment, the external workstation can invoke a preset visual language action model to perform model inference based on the target image and target joint state data, thereby obtaining the original inference trajectory output by the visual language action model. The total number of trajectory points included in the original inference trajectory can be flexibly set according to actual conditions, and this embodiment does not impose a specific limitation on this. As an example, it can be set to 10 points.
[0097] To obtain a denser trajectory, the original inference trajectory can be interpolated according to a preset maximum trajectory length (e.g., 300 points) to obtain the target action trajectory.
[0098] During trajectory interpolation of the original inference trajectory, the velocity and acceleration corresponding to the trajectory start point and the trajectory end point can both be set to zero, achieving zero velocity and zero acceleration at the endpoints. This constraint improves the physical smoothness of the trajectory, significantly reduces start-stop shock, extends mechanical life, and is more suitable for precision operation scenarios.
[0099] Step S403: Send the target motion trajectory to the robot so that the robot can execute the target motion trajectory.
[0100] It is easy to understand that, for external workstations, the above... Figure 4 The process shown is a continuous loop. After the external workstation sends the target motion trajectory to the robot, the robot can perform the next round of motion trajectory execution and synchronous data acquisition, and send the new data to the external workstation. The external workstation can then process the data according to the specified parameters. Figure 4 The process shown continues to generate new target motion trajectories and sends them to the robot, repeating this cycle until the task is completed or a stop command is received from the user.
[0101] In summary, considering the robot and external workstation as a whole system, Figure 5 A timing diagram of a robot motion trajectory execution method according to an embodiment of this application is shown. As shown, the robot acquires the target motion trajectory generated by an external workstation and sequentially sends each trajectory point in the target motion trajectory to the robot's joint controllers according to a preset trajectory execution frequency, so that the robot's joint controllers execute the target motion trajectory. After determining that the target motion trajectory has been executed, synchronous data acquisition is triggered. The robot acquires the target image and the image timestamp corresponding to the target image. Based on the image timestamp, the robot determines the target joint state data and performs data integrity verification on the target joint state data. If the data integrity verification is successful, the robot sends the synchronous data (target image and target joint state data) to the external workstation. The external workstation acquires the target image and target joint state data sent by the robot, calls the visual language action model to perform model inference, and obtains the original inferred trajectory. The external workstation performs trajectory interpolation on the original inferred trajectory, setting the velocity and acceleration corresponding to the trajectory endpoints (start and end points) to zero, thereby obtaining a new target motion trajectory, and sends the new target motion trajectory to the robot so that the robot executes the new target motion trajectory. This process continues until the task is completed or a stop command is received from the user.
[0102] In one specific implementation of this application, all messages can be deeply copied using a message deep cloning unit, thereby avoiding middleware memory management conflicts; mutual exclusion callback groups can be used between inference and control tasks to prevent data races; and in the event of an exception, complete stack information can be recorded for remote debugging. This application provides a compatible control architecture that is compatible with both general-purpose middleware and enterprise-developed middleware, allowing seamless operation between the two and offering good deployment flexibility.
[0103] In summary, through the embodiments of this application, the computing power resources of the external workstation can be fully utilized, and the external workstation can generate the target motion trajectory of the robot. The robot only needs to execute the target motion trajectory generated by the external workstation, thereby effectively getting rid of the robot's computing power limitation and meeting the real-time requirements of robot motion trajectory execution.
[0104] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.
[0105] Corresponding to the robot motion trajectory execution method applied to a robot described in the above embodiments, Figure 6 This illustration shows a structural diagram of an embodiment of a robot motion trajectory execution device applied in a robot, provided by an embodiment of this application.
[0106] In this embodiment, a robot motion trajectory execution device may include:
[0107] The motion trajectory acquisition module 601 is used to acquire the target motion trajectory generated by the external workstation; wherein, the target motion trajectory is the motion trajectory to be executed by the robot.
[0108] The motion trajectory execution module 602 is used to send each trajectory point in the target motion trajectory to each joint controller of the robot in sequence according to a preset trajectory execution frequency, so that each joint controller of the robot executes the target motion trajectory.
[0109] In one specific implementation of this application embodiment, the motion trajectory execution module may include:
[0110] The trajectory execution completion determination unit is used to update a preset trajectory execution index during the process of sequentially sending each trajectory point in the target motion trajectory to each joint controller of the robot according to a preset trajectory execution frequency; wherein, the trajectory execution index is used to indicate the number of executed trajectory points; and if the trajectory execution index is greater than or equal to the preset maximum trajectory length, the target motion trajectory is determined to be completed.
[0111] In one specific implementation of this application embodiment, the apparatus may further include:
[0112] A synchronous data acquisition module is used to acquire the target image of the robot and the image timestamp corresponding to the target image; wherein the target image is the latest image in the robot's image buffer; determine the target joint state data of the robot based on the image timestamp; wherein the target joint state data is the joint state data synchronized with the target image in the robot's joint state buffer; perform data integrity verification on the target joint state data; if the data integrity verification is successful, send the target image and the target joint state data to the external workstation so that the external workstation can generate the next round of motion trajectory based on the target image and the target joint state data.
[0113] Corresponding to the robot motion trajectory execution method applied to an external workstation described in the above embodiments, Figure 7 This illustration shows a structural diagram of an embodiment of a robot motion trajectory execution device applied in an external workstation, as provided in this application.
[0114] In this embodiment, a robot motion trajectory execution device may include:
[0115] The synchronous data acquisition module 701 is used to acquire the target image and target joint state data sent by the robot; wherein, the target image is the latest image in the robot's image buffer, and the target joint state data is the joint state data in the robot's joint state buffer that is closest to the target image in time;
[0116] The motion trajectory generation module 702 is used to generate a motion trajectory based on the target image and the target joint state data to obtain a target motion trajectory; wherein, the target motion trajectory is the motion trajectory to be executed by the robot;
[0117] The motion trajectory sending module 703 is used to send the target motion trajectory to the robot so that the robot executes the target motion trajectory.
[0118] In one specific implementation of this application embodiment, the motion trajectory generation module may include:
[0119] The model inference unit is used to call a preset visual language action model to perform model inference based on the target image and the target joint state data, and obtain the original inference trajectory output by the visual language action model.
[0120] The trajectory interpolation unit is used to interpolate the original inference trajectory according to the preset maximum trajectory length to obtain the target action trajectory.
[0121] In one specific implementation of this application embodiment, the motion trajectory generation module may further include:
[0122] The endpoint zero-velocity and zero-acceleration setting unit is used to set the velocity and acceleration corresponding to the starting point of the trajectory to zero and the velocity and acceleration corresponding to the ending point of the trajectory to zero during the trajectory interpolation process of the original inference trajectory.
[0123] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.
[0124] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.
[0125] Figure 8 A schematic block diagram of a robot provided in an embodiment of this application is shown. For ease of explanation, only the parts related to the embodiment of this application are shown.
[0126] like Figure 8 As shown, the robot 8 in this embodiment includes: a processor 80, a memory 81, and a computer program 82 stored in the memory 81 and executable on the processor 80. When the processor 80 executes the computer program 82, it implements the steps in the various robot motion trajectory execution method embodiments described above, for example... Figure 2 Steps S201 to S202 are shown. Alternatively, when the processor 80 executes the computer program 82, it implements the functions of each module / unit in the above-described device embodiments, for example... Figure 6 The functions of modules 601 to 602 shown.
[0127] For example, the computer program 82 may be divided into one or more modules / units, which are stored in the memory 81 and executed by the processor 80 to complete this application. The one or more modules / units may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program 82 in the robot 8.
[0128] Those skilled in the art will understand that Figure 8 This is merely an example of robot 8 and does not constitute a limitation on robot 8. It may include more or fewer components than shown, or combine certain components, or different components. For example, robot 8 may also include input / output devices, network access devices, buses, etc.
[0129] The processor 80 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.
[0130] The memory 81 can be an internal storage unit of the robot 8, such as a hard drive or memory. The memory 81 can also be an external storage device of the robot 8, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, the memory 81 can include both internal and external storage units of the robot 8. The memory 81 is used to store the computer program and other programs and data required by the robot 8. The memory 81 can also be used to temporarily store data that has been output or will be output.
[0131] Figure 9 A schematic block diagram of an external workstation provided in an embodiment of this application is shown. For ease of explanation, only the parts related to the embodiment of this application are shown.
[0132] like Figure 9As shown, the external workstation 9 in this embodiment includes: a processor 90, a memory 91, and a computer program 92 stored in the memory 91 and executable on the processor 90. When the processor 90 executes the computer program 92, it implements the steps in the various external workstation motion trajectory execution method embodiments described above, for example... Figure 4 Steps S401 to S403 are shown. Alternatively, when the processor 90 executes the computer program 92, it implements the functions of each module / unit in the above-described device embodiments, for example... Figure 7 The functions of modules 701 to 703 are shown.
[0133] For example, the computer program 92 may be divided into one or more modules / units, which are stored in the memory 91 and executed by the processor 90 to complete this application. The one or more modules / units may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program 92 in the external workstation 9.
[0134] Those skilled in the art will understand that Figure 9 This is merely an example of an external workstation 9 and does not constitute a limitation on the external workstation 9. It may include more or fewer components than shown, or combine certain components, or different components. For example, the external workstation 9 may also include input / output devices, network access devices, buses, etc.
[0135] The processor 90 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.
[0136] The memory 91 can be an internal storage unit of the external workstation 9, such as a hard disk or memory of the external workstation 9. The memory 91 can also be an external storage device of the external workstation 9, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the external workstation 9. Furthermore, the memory 91 can include both internal storage units and external storage devices of the external workstation 9. The memory 91 is used to store the computer program and other programs and data required by the external workstation 9. The memory 91 can also be used to temporarily store data that has been output or will be output.
[0137] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.
[0138] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.
[0139] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0140] In the embodiments provided in this application, it should be understood that the disclosed devices / robots and methods can be implemented in other ways. For example, the device / robot embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.
[0141] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0142] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0143] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable storage medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc. It should be noted that the content included in the computer-readable storage medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable storage medium does not include electrical carrier signals and telecommunication signals.
[0144] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.
Claims
1. A method for executing robot motion trajectories, characterized in that, Applied in robots, the robot establishes a communication connection with an external workstation; the method includes: Obtain the target motion trajectory generated by the external workstation; wherein, the target motion trajectory is the motion trajectory to be executed by the robot; According to a preset trajectory execution frequency, each trajectory point in the target motion trajectory is sequentially sent to each joint controller of the robot, so that each joint controller of the robot executes the target motion trajectory.
2. The robot motion trajectory execution method according to claim 1, characterized in that, Also includes: During the process of sequentially sending each trajectory point in the target motion trajectory to each joint controller of the robot according to the preset trajectory execution frequency, the preset trajectory execution index is updated; wherein, the trajectory execution index is used to indicate the number of trajectory points that have been executed; If the trajectory execution index is greater than or equal to the preset maximum trajectory length, it is determined that the target action trajectory has been completed.
3. The robot motion trajectory execution method according to claim 2, characterized in that, After determining that the target action trajectory has been completed, the process also includes: Obtain the target image of the robot, and the image timestamp corresponding to the target image; wherein, the target image is the latest image in the robot's image buffer; The target joint state data of the robot is determined based on the image timestamp; wherein, the target joint state data is the joint state data synchronized with the target image in the robot's joint state buffer; Perform data integrity verification on the target joint status data; If the data integrity verification is successful, the target image and the target joint status data are sent to the external workstation so that the external workstation can generate the next round of motion trajectory based on the target image and the target joint status data.
4. A method for executing robot motion trajectories, characterized in that, The method is applied in an external workstation, wherein a communication connection is established between the external workstation and the robot; the method includes: Acquire the target image and target joint state data sent by the robot; wherein the target image is the latest image in the robot's image buffer, and the target joint state data is the joint state data in the robot's joint state buffer that is synchronized with the target image; A motion trajectory is generated based on the target image and the target joint state data to obtain the target motion trajectory; wherein, the target motion trajectory is the motion trajectory to be executed by the robot. The target motion trajectory is sent to the robot so that the robot executes the target motion trajectory.
5. The robot motion trajectory execution method according to claim 4, characterized in that, The step of generating a motion trajectory based on the target image and the target joint state data to obtain the target motion trajectory includes: Based on the target image and the target joint state data, a preset visual language action model is invoked to perform model reasoning, and the original reasoning trajectory output by the visual language action model is obtained. The original inference trajectory is interpolated according to the preset maximum trajectory length to obtain the target action trajectory.
6. The robot motion trajectory execution method according to claim 5, characterized in that, Also includes: During the trajectory interpolation process of the original inference trajectory, the velocity and acceleration corresponding to the trajectory start point are both set to zero, and the velocity and acceleration corresponding to the trajectory end point are both set to zero.
7. A robot motion trajectory execution device, characterized in that, Applied in robots, the robot establishes a communication connection with an external workstation; the device includes: The motion trajectory acquisition module is used to acquire the target motion trajectory generated by the external workstation; wherein, the target motion trajectory is the motion trajectory to be executed by the robot; The motion trajectory execution module is used to send each trajectory point in the target motion trajectory to each joint controller of the robot in sequence according to a preset trajectory execution frequency, so that each joint controller of the robot executes the target motion trajectory.
8. A robot motion trajectory execution device, characterized in that, Applied in an external workstation, wherein a communication connection is established between the external workstation and the robot; the device includes: A synchronous data acquisition module is used to acquire the target image and target joint state data sent by the robot; wherein the target image is the latest image in the robot's image buffer, and the target joint state data is the joint state data in the robot's joint state buffer that is closest to the target image in time; The motion trajectory generation module is used to generate a motion trajectory based on the target image and the target joint state data to obtain a target motion trajectory; wherein, the target motion trajectory is the motion trajectory to be executed by the robot; The motion trajectory sending module is used to send the target motion trajectory to the robot so that the robot can execute the target motion trajectory.
9. A robot comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the robot motion trajectory execution method as described in any one of claims 1 to 3.
10. An external workstation, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the robot motion trajectory execution method as described in any one of claims 4 to 6.