Robot remote driving method, device, equipment and medium

Abstract

The invention discloses a robot remote driving method, device and equipment and a medium. The method comprises the steps that to-be-executed action content and an action instruction generation model of a to-be-executed task corresponding to a to-be-controlled robot are obtained, and the action instruction generation model is used for inputting the to-be-executed action content and outputting an executable instruction of a motor in the to-be-controlled robot; environment downloading indication information corresponding to the action instruction generation model is obtained; and sending the to-be-executed action content of the to-be-executed task, the action instruction generation model and the environment downloading indication information to the to-be-controlled robot, so that the to-be-controlled robot executes a corresponding action according to the to-be-executed action content, the action instruction generation model and the environment downloading indication information. According to the embodiment of the invention, the remote driving accuracy of the robot can be improved.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method, apparatus, device, and medium for remotely driving a robot. Background Technology

[0002] With the rapid development of technology, the types of robots are gradually increasing and they are being applied to all aspects of people's lives. Robot models can be configured on robots and specific instructions can be sent to the robots to drive their movement.

[0003] Currently, robot models are manually configured onto the robots.

[0004] However, there are many types and a large number of robots, and manually configuring robot models is inefficient. Summary of the Invention

[0005] This invention provides a method, apparatus, device, and medium for remotely driving a robot, thereby improving the accuracy of remote robot driving.

[0006] In a first aspect, embodiments of the present invention provide a robot remote driving method applied to a remote control system, the method comprising:

[0007] Obtain the action content to be executed and the action instruction generation model of the task to be executed for the robot to be controlled. The action instruction generation model is used to input the action content to be executed and output the executable instructions of the motors in the robot to be controlled.

[0008] Obtain the environment download instruction information corresponding to the action command generation model;

[0009] Send the content of the action to be executed, the action instruction generation model, and the environment download instruction information of the task to be controlled to the robot to be controlled, so that the robot to be controlled can execute the corresponding action according to the content of the action to be executed, the action instruction generation model, and the environment download instruction information.

[0010] Secondly, embodiments of the present invention provide a remote robot driving method, applied to a robot to be controlled, the method comprising:

[0011] Obtain the content of the actions to be executed, the action instruction generation model, and the environment download instruction information of the tasks to be executed issued by the remote control system;

[0012] Execute the corresponding action based on the content of the action to be executed, the action instruction generation model, and the environment download instruction information.

[0013] Thirdly, embodiments of the present invention also provide a robot remote drive device, the robot remote drive device comprising:

[0014] At least one processor; and

[0015] A memory that is communicatively connected to at least one processor; wherein,

[0016] The memory stores a computer program that can be executed by at least one processor, such that the at least one processor is able to perform the robot remote drive method according to any embodiment of the present invention.

[0017] According to another aspect of the present invention, a computer-readable storage medium is provided, which stores computer instructions for causing a processor to execute and implement the robot remote driving method of any embodiment of the present invention.

[0018] The technical solution of this invention obtains the content of the action to be executed and the action instruction generation model of the task to be executed corresponding to the robot to be controlled. The action instruction generation model is used to input the content of the action to be executed and output the executable instructions of the motors in the robot to be controlled. It also obtains the environment download indication information corresponding to the action instruction generation model and sends the content of the action to be executed, the action instruction generation model and the environment download indication information of the task to be executed to the robot to be controlled, so that the robot to be controlled can execute the corresponding action according to the content of the action to be executed, the action instruction generation model and the environment download indication information. The operating environment and instruction generation model of the robot to be controlled can be configured remotely without manual configuration, which improves the efficiency of remote robot driving.

[0019] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a flowchart of a robot remote driving method provided according to Embodiment 1 of the present invention;

[0022] Figure 2 This is a flowchart of a robot remote driving method provided according to Embodiment 2 of the present invention;

[0023] Figure 3 This is a flowchart of a robot remote driving method provided according to Embodiment 3 of the present invention;

[0024] Figure 4 This is a structural diagram of a robot remote drive device according to an embodiment of the present invention;

[0025] Figure 5 This is a structural diagram of a robot remote drive device according to an embodiment of the present invention;

[0026] Figure 6 This is a structural schematic diagram of a robot remote drive device provided in an embodiment of the present invention. Detailed Implementation

[0027] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0028] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0029] In the technical solutions of the embodiments of the present invention, the acquisition, storage and application of tasks to be executed, etc., all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0030] Example 1

[0031] Figure 1 This is a flowchart illustrating a robot remote driving method according to Embodiment 1 of the present invention. This embodiment of the invention is applicable to situations involving remote robot driving. The method can be executed by a robot remote driving device, which can be implemented in hardware and / or software.

[0032] See Figure 1 The robot remote drive method shown is applied to a remote control system and includes:

[0033] S101. Obtain the action content to be executed and the action instruction generation model of the task to be executed corresponding to the robot to be controlled. The action instruction generation model is used to input the action content to be executed and output the executable instructions of the motor in the robot to be controlled.

[0034] The robot to be controlled can be a remotely controlled robot entity waiting to receive external control commands and execute corresponding tasks; it is the ultimate executor of actions, such as an inspection robot in a factory or a handling robot in a warehouse. The task to be executed can be generated based on the description of the specific work task the robot needs to complete. The action content to be executed can be a set of commands for a series of specific actions the robot needs to perform sequentially to complete the task. The action command generation model can be a trained algorithm model. Executable commands can be standardized commands conforming to motor control protocols, such as motor rotation angle, speed, or start / stop signals.

[0035] Specifically, the remote control system provides a visual interface for users to interact with and obtain the tasks to be executed. Simultaneously, users can pre-train a command generation model and upload it to the remote control system. The remote control system acquires the robot's task and breaks it down into multiple actions, each corresponding to a different communication command. Based on the communication commands for each action, the system integrates these actions to form a complete set of actions. For example, if the task is to move parts to a designated platform, the actions might include movement, grasping, and placement. The system then integrates these actions based on their corresponding communication commands. The command generation model is then input into the action content, and model matching is performed to determine if the model can handle the action content, ensuring that the model can output executable commands for the corresponding motors.

[0036] S102. Obtain the environment download instruction information corresponding to the action instruction generation model.

[0037] Among them, the environment download instruction information can be used to indicate which environmental configuration information the robot to be controlled needs to download.

[0038] Specifically, different motion command generation models rely on different runtime environment configurations. Based on the motion command generation model, the runtime environment configuration information can be determined, and corresponding environment download instructions can be generated. This allows the robot to receive the environment download instructions and download the appropriate runtime environment configuration information. For example, environment download prompts may include algorithm libraries adapted to the motion command generation model, data parsing modules, system configuration parameters, or data packages. For instance, some models may require a specific version of the robot control algorithm package to operate correctly.

[0039] S103. Send the content of the action to be executed, the action instruction generation model, and the environment download instruction information of the task to be controlled to the robot to be controlled, so that the robot to be controlled can execute the corresponding action according to the content of the action to be executed, the action instruction generation model, and the environment download instruction information.

[0040] Specifically, the remote control system first generates the content of the action to be executed, and simultaneously confirms the integrity of the action instruction generation model. For example, it checks whether each module of the action instruction generation model can correctly process the remotely issued action content as driving instructions and correctly drive the robot to be controlled to execute the corresponding action. It also needs to verify whether the data packet corresponding to the environmental download instruction information is accurate to avoid missing or incorrect configuration data. The remote control system establishes a stable communication link with the robot to be controlled through a wireless network to ensure uninterrupted and undistorted data transmission. The smaller environmental download instruction information is transmitted first, followed by the action instruction generation model, and finally the content of the action to be executed. A verification mechanism can be included during transmission. If data packet loss occurs, the system automatically retransmits, for example, by confirming the integrity of the received data through a data packet checksum. After the robot to be controlled receives the three types of data, it sends a feedback signal of successful reception to the remote control system. If the control system does not receive feedback in time, it will initiate a transmission request again until it confirms that all data has been received.

[0041] The technical solution of this invention obtains the content of the action to be executed and the action instruction generation model of the task to be executed corresponding to the robot to be controlled. The action instruction generation model is used to input the content of the action to be executed and output the executable instructions of the motors in the robot to be controlled. It also obtains the environment download indication information corresponding to the action instruction generation model and sends the content of the action to be executed, the action instruction generation model and the environment download indication information of the task to be executed to the robot to be controlled, so that the robot to be controlled can execute the corresponding action according to the content of the action to be executed, the action instruction generation model and the environment download indication information. The operating environment and instruction generation model of the robot to be controlled can be configured remotely without manual configuration, which improves the efficiency of remote robot driving.

[0042] Example 2

[0043] Figure 2 This is a flowchart illustrating a remote robot driving method according to Embodiment 2 of the present invention. Based on the above embodiments, this embodiment optimizes and improves the remote robot driving operation.

[0044] Furthermore, the process of "obtaining the environment download instruction information corresponding to the motion instruction generation model" is further refined into "obtaining the current environment information of the robot to be controlled; obtaining the target environment information corresponding to the motion instruction generation model; comparing the target environment information with the current environment information to obtain the information comparison result; determining the download environment information required by the robot to be controlled based on the information comparison result; and generating environment download instruction information based on the download environment information," in order to improve the operation of remote robot driving.

[0045] It should be noted that for parts not described in detail in the embodiments of the present invention, please refer to the descriptions in other embodiments.

[0046] See Figure 2 The robot remote drive method shown includes:

[0047] S201. Obtain the content of the action to be executed and the action instruction generation model of the task to be executed corresponding to the robot to be controlled. The action instruction generation model is used to input the content of the action to be executed and output the executable instructions of the motor in the robot to be controlled.

[0048] S202. Obtain the current environmental information of the robot to be controlled.

[0049] Among them, the current environment information can be the operating environment configuration information of the robot to be controlled at the current moment.

[0050] Specifically, the remote control system can send standardized environmental information query commands to the robot being controlled via its communication module. For example, in the ROS2 system, a GetCurrentEnv.srv request can be sent via the srv service, specifying the environmental category to be queried (such as "software dependencies" or "hardware configuration"). After receiving the request, the robot being controlled starts its local information acquisition program and obtains the version information of the installed environment configuration data package through system commands. The robot then organizes the collected version information of the environment configuration data package into structured data, including key fields such as "environment category - specific parameters - version number," and then feeds the structured data back to the remote control system through the communication channel. After receiving the data, the remote control system verifies the integrity of the format (such as whether key fields are missing) and stores it in its local database.

[0051] S203. Obtain the target environment information corresponding to the action instruction generation model.

[0052] Among them, the target environment information can be the standard configuration information of the environment necessary for the normal operation of the action instruction generation model.

[0053] Specifically, during the development of the action command generation model, a configuration file for the model's runtime environment is preset. Different action command generation models correspond to different target environment information. A mapping relationship exists between the action command generation model and the target environment information. The action command generation model to be searched can be determined by identifying the model features corresponding to the action command generation model. Based on the preset mapping relationship, the target environment information corresponding to the action command generation model to be searched is determined. Search methods include, but are not limited to, sequential search algorithms, binary search algorithms, hash search algorithms, or tree table search algorithms, etc., and this embodiment of the invention does not impose any limitations on these methods.

[0054] S204. Compare the target environment information with the current environment information to obtain the information comparison result.

[0055] Specifically, a comparison framework can be established, dividing environmental information into three core dimensions: software dependencies, hardware compatibility, and data resources. Each dimension can be further subdivided into specific sub-items (e.g., software dependencies include operating systems, framework versions, and library files) to ensure no omissions. Item-by-item matching is performed. If a sub-item of the target environment information matches a sub-item of the current environment information (e.g., the target requires "Python 3.10," while the current environment uses "Python 3.10.12"), it is considered a match. If a sub-item of the target environment information does not match a sub-item of the current environment information, it is further categorized as follows: Missing: The current environment completely lacks the components required by the target; Version mismatch: The component exists but the version is incompatible (e.g., the target requires "PyTorch 2.0," while the current environment uses "PyTorch 1.13"); Hardware incompatibility: The hardware type does not meet the requirements (e.g., the target requires a "depth camera," but only a regular RGB camera is available). Structured comparison results are generated: The matched and differing items are compiled into a readable results report, clearly indicating the type, current value, and target value of each differing item.

[0056] S205. Based on the information comparison results, determine the download environment information required for the robot to be controlled.

[0057] Among them, the downloaded environment information can be the environmental configuration information that the robot to be controlled needs to obtain through the network or other means.

[0058] Specifically, irrelevant differences (such as minor software version differences that do not affect model operation) can be eliminated from the information comparison results first. Focus on the key differences, and for each type of key difference, determine the appropriate download content and key attributes. For software differences, the specific version should be specified; for example, PyTorch needs to match the target requirement version 2.0. In addition to key differences, relevant auxiliary information can also be downloaded, such as the cloud storage address of the resources, verification codes (to prevent corrupted downloaded files), or installation priorities. Organize the above content into clear text, clearly specifying the resource name, version, download address, and installation order of the download environment information, to facilitate automatic parsing by the robot to be controlled.

[0059] S206. Generate environment download instruction information based on the download environment information.

[0060] Specifically, from the established download environment information, key attributes of each resource are extracted, such as resource ID / name, version number, download address, storage directory, verification rules, or installation priority. These are the core parameters of the commands. Based on the robot's operating system or control framework, an appropriate command format is selected. To avoid download failures, additional parameters can be added, such as skipping invalid licenses, setting timeout retries, or overwriting old files. If multiple resources need to be downloaded, the commands are sorted by installation priority, generating an ordered command set and generating environment download instruction information to ensure the robot executes them in the correct order.

[0061] S207. Send the content of the action to be executed, the action instruction generation model, and the environment download instruction information of the task to be controlled to the robot to be controlled, so that the robot to be controlled can execute the corresponding action according to the content of the action to be executed, the action instruction generation model, and the environment download instruction information.

[0062] This invention improves the efficiency of environment configuration by acquiring the current environment information of the robot to be controlled; acquiring the target environment information corresponding to the action command generation model; comparing the target environment information with the current environment information to obtain the information comparison result; determining the download environment information required by the robot to be controlled based on the information comparison result; and generating environment download instruction information based on the download environment information. This eliminates the need for manual configuration of download information and automatically generates download prompt information for resource download, thus improving the efficiency of environment configuration.

[0063] Optionally, based on the information comparison results, the download environment information required by the robot to be controlled is determined, including: when the information comparison result is determined to be consistent with the target environment information and the current environment information, the download environment information required by the robot to be controlled is empty; when the information comparison result is determined to be inconsistent with the target environment information and the current environment information, at least one associated data packet in the target environment information is obtained, and the associated data packet not included in the current environment information is determined as the download environment information required by the robot to be controlled.

[0064] Specifically, first, set the criteria for determining environmental consistency. For each item to be compared in the target environment information, compare the item to be compared with the corresponding item in the current environment information. If the comparison result of the item to be compared is consistent, it means that the environment information corresponding to the item to be compared in the current environment has met the task requirements, and the download environment information of the item to be compared is directly determined to be empty, and the environment information corresponding to the item to be compared does not need to be downloaded again. If the determination result is inconsistent, it means that the environment information corresponding to the item to be compared in the current environment does not meet the task requirements, and the download environment information of the item to be compared is directly determined to be the environment information corresponding to the item to be compared, or other available versions of the environment information of the item to be compared can be determined as the download environment information.

[0065] When the comparison result of the determination information shows that the target environment information is consistent with the current environment information, the download environment information required by the robot to be controlled is empty; when the comparison result of the determination information shows that the target environment information is inconsistent with the current environment information, at least one associated data packet in the target environment information is obtained, and the associated data packet not included in the current environment information is determined as the download environment information required by the robot to be controlled. This can avoid repeatedly downloading existing data and save the memory space of the robot to be controlled.

[0066] Optionally, obtaining the content of the actions to be executed for the task to be executed corresponding to the robot to be controlled includes: determining the attribute information of at least one subtask and the execution order of each subtask according to the task to be executed corresponding to the robot to be controlled; the attributes include: at least one task attribute and the attribute value corresponding to each task attribute; for each subtask, adding the attribute value of each task attribute of the subtask to the corresponding command template to determine the sub-action content corresponding to the subtask; sorting the sub-action content according to the execution order to generate the content of actions to be executed.

[0067] Subtasks can be small, independently operable task units obtained by breaking down the overall task to be executed. They are the steps to complete the overall task; for example, a handling task can be broken down into subtasks such as moving to shelf A and picking up parts. Attribute information can be feature data used to define the execution details of a subtask, providing key parameters to support its execution. Task attributes can be categories describing key features of a subtask, such as target location, end effector, and action duration, used to define the execution specifications of the subtask. Attribute values ​​can be the specific data content corresponding to the task attributes, a quantitative expression of the task attributes; for example, the attribute value for target location might be shelf A, level 3. Execution order can be the logical relationship between subtasks, determined by task dependencies; for example, picking up parts must precede moving them. Command templates can be pre-defined, fixed-format instruction frameworks adapted to the robot's execution logic, containing attribute value placeholders to facilitate parameter filling and specific instruction generation. Sub-action content can be a single action instruction that the robot can directly execute after filling the subtask attribute values ​​into the command template. The action content to be executed can be a complete sequence of actions for the robot to complete the overall task by integrating all sub-action content according to the execution order of sub-tasks.

[0068] Specifically, first, clearly define the task to be performed by the robot to be controlled. Break down the overall task into several operable subtasks, which need to be refined to the point that the robot can complete them independently. Assign at least one task attribute to each subtask and determine the corresponding attribute value. Analyze the dependencies between subtasks and determine the execution order. For example, subtasks with sequential dependencies should be logically ordered, while those without dependencies can be flexibly adjusted, but must conform to the logic of the actual scenario. For each decomposed subtask, retrieve the preset command template adapted to that type of action. Fill the corresponding placeholders in the template with the attribute values ​​corresponding to each task attribute of the subtask, generating sub-action content that can directly guide the robot's operation. For example, the template for a grasping subtask is "Use the [execution tool] to grasp the [target object], and the grasping force is the [force value]", which, after filling in, will give the specific sub-action. Arrange all the generated sub-action content in sequence according to the determined subtask execution order, remove duplicate or conflicting action instructions, and finally form a coherent and complete set of actions to be executed, ensuring that the robot can complete the overall task by executing in this sequence.

[0069] By determining the attribute information of at least one subtask and the execution order of each subtask based on the task to be executed for the robot to be controlled, the attributes include: at least one task attribute and the attribute value corresponding to each task attribute; for each subtask, the attribute value of each task attribute of the subtask is added to the corresponding command template to determine the sub-action content corresponding to the subtask; the sub-action content is sorted according to the execution order to generate the action content to be executed. The accuracy of determining the action content to be executed is improved by generating the action content to be executed through multi-dimensional data.

[0070] Optional attribute information includes: action object, control granularity type, model type, and action type.

[0071] The action object can be a specific part of the robot that is directly manipulated in the subtask. The control granularity type can be the precision control level when the robot performs the action. The model type can be the algorithm architecture type written in the action instruction model. The action type can be the end effector gripping operation type when performing the subtask.

[0072] Specifically, users can upload task information to the remote control system. A task includes multiple subtasks, each with multiple task attributes. Users can pre-configure attribute information, including: action object, control granularity type, model type, and action type. The action object can be a joint identifier. The control granularity type can be incremental or non-incremental. Incremental refers to the robot model's output action, non-incremental model outputs the robot's next state, and incremental model outputs the difference between the current and next states. The action type can be a gripper or a dexterous hand. Users can also upload an initial position packet, a rosbag file that records the robot's progress from its initial state to the start state of the specified task. Users need to upload the appropriate rosbag file based on the desired task to help the robot move from the powered-on state to the task-ready state. The model type can be a model constructed using different algorithms.

[0073] The diversity of attribute information acquisition is determined by using multidimensional data, including: action object, control granularity type, model type, and action type.

[0074] The remote control system also provides version control functionality, allowing users to add, delete, modify, and query all versions of the model generated from user-uploaded action commands, facilitating version viewing, selection, and modification. The remote control system also provides a model inference interface, where users can select the model version and the target robot they wish to control. Clicking the connect button allows monitoring of the target robot's status. The selected model is then sent to the target robot, and the system checks the deployment environment. If the robot's environment does not meet the model's deployment requirements, it automatically generates a suitable environment. Clicking the "Start Inference" button causes the robot to execute actions inferred by the model based on the task scenario, gradually completing the task objective. "Pause Inference" and "Resume Inference" buttons allow users to remotely pause and resume the target robot's current actions during monitoring. The "Start Recording" option provides the option to retain robot sensor information during robot movement. "Stop Inference" provides the function to completely shut down the inference process and end the task. The inference frequency and maximum execution steps determine the model's inference speed and time. The model outputs the next state at a certain frequency to control the robot's movement; each output is one execution. Inference automatically stops after reaching the maximum number of execution steps. The inference progress is displayed visually to the user. The inference log displays the robot's running status log during the inference process, facilitating user monitoring of the model's operation. The interface visualizes information from the robot's main sensors, with three windows displaying real-time video from the head and hand cameras, and multiple smaller windows providing real-time line graphs showing the robot's motor status. Through these visualization windows and the real-time display of inference logs, users can remotely monitor the robot's movement.

[0075] Example 3

[0076] Figure 3 This is a flowchart illustrating a robot remote driving method according to Embodiment 1 of the present invention. This embodiment of the invention is applicable to situations involving remote robot driving. The method can be executed by a robot remote driving device, which can be implemented in hardware and / or software.

[0077] See Figure 3 The robot remote drive method shown is applied to the robot to be controlled and includes:

[0078] S301. Obtain the content of the action to be executed, the action instruction generation model, and the environment download instruction information of the task to be executed issued by the remote control system.

[0079] Specifically, after the robot to be controlled starts, it automatically establishes an encrypted communication link with the remote control system (to prevent data leakage or tampering). The remote control system sends the content of the actions to be performed, the action instruction generation model, and the environment download instruction information to the robot to be controlled. After the robot to be controlled confirms, it enters the receiving state. Data can be received in a pre-set priority order to avoid waiting for critical data. After receiving each type of data, the robot can use a preset checksum to verify the data's integrity (no packet loss and no damage). If the verification fails, a retransmission request is sent to the remote control system; if the verification succeeds, feedback indicates reception completion. The three types of data are stored in a designated directory on the robot, and the storage path is recorded for easy retrieval later.

[0080] S302. Execute the corresponding action based on the content of the action to be executed, the action instruction generation model, and the environment download instruction information.

[0081] Specifically, the robot to be controlled reads the environment download instruction information, identifies the dependencies to be downloaded (such as software libraries, drivers, or data packages); automatically downloads and installs missing resources, skipping existing resources; configures environment variables (such as specifying the model running path or driver interface), and performs a self-check to ensure the environment meets the requirements (such as checking the library version and whether the driver is loaded correctly). It retrieves the motion instruction model from the local storage directory and loads it into the robot's computing unit (CPU / GPU), completing the matching connection between the model port and the robot's environment port. It initializes model parameters (such as setting inference accuracy and motion output frequency) to ensure the model is in a ready state. The robot to be controlled breaks down the actions to be executed into sub-tasks, inputting motion instructions one by one to generate a model; the model converts instructions according to its own type (incremental / non-incremental). For example, an incremental model outputs "the difference between the current state and the target state" (such as "mechanical arm joint 1 rotates another 5°"), while a non-incremental model outputs "the target state" (such as "mechanical arm joint 1 jumps to 30°"). It integrates the instructions of all sub-tasks to form a coherent sequence of motor instructions (including execution order, speed, and force parameters). The robot under control drives its components to work together according to the command sequence (such as chassis movement, gripper opening and closing, or joint rotation); during the execution, sensor data (such as position sensor and force sensor) is collected in real time to monitor whether the action deviates from the expectation; if a deviation occurs (such as insufficient gripping force), the model dynamically adjusts the command based on the feedback; if the deviation is too large, the execution is suspended and an alarm is sent to the remote control system, and the real-time recorded sensor information and deviation analysis information are sent back to the remote control system and displayed visually.

[0082] The technical solution of this invention obtains the content of the action to be executed, the action instruction generation model, and the environment download indication information of the task to be executed issued by the remote control system; and executes the corresponding action according to the content of the action to be executed, the action instruction generation model, and the environment download indication information, without the need for manual intervention, thereby improving the efficiency of robot control.

[0083] Optionally, based on the content of the action to be executed, the action instruction generation model, and the environment download instruction information, the corresponding action is executed, including: downloading the data package of the runtime environment corresponding to the environment download instruction information and loading it to form a model runtime environment; loading the action instruction generation model in the model runtime environment; inputting the content of the action to be executed into the action instruction generation model to obtain executable instructions for at least one motor; sending each executable instruction to the corresponding motor and executing each executable instruction to perform the corresponding action of the task to be executed.

[0084] Specifically, the robot parses the environment download instruction information, extracts the download address, storage path, and installation requirements of the data package (e.g., "Download address http: / / xxx, store to / usr / lib, installation order: driver first, then algorithm library"). It automatically deploys according to the installation requirements, overwriting old versions of the driver, configuring environment variables for the algorithm library, and importing data files into the specified directory. It retrieves the complete file (including configuration file config.yaml and weight file model.pth) of the motion instruction generation model from the robot's local specified directory (e.g., / model / ). According to the model configuration file requirements, it initializes the model in the built runtime environment (e.g., setting inference accuracy, motion output frequency, and port mapping rules). It completes the binding of model ports to environment ports (e.g., connecting the model's "motor control output port" to the robot's "joint motor input interface"), ensuring that the model's output instructions can be transmitted to the motors. It runs test cases (e.g., inputting a simple motion description "gripper closes" to verify if the model can output valid instructions); if the self-check passes, the model is ready. The robot breaks down the action content to be executed into sub-tasks, and according to the execution order of the sub-tasks, it inputs the sub-action content into motion instructions one by one to generate the model, forming a coherent sequence of motor instructions. The robot sends each instruction to the corresponding motor according to the instruction sequence. After receiving the instruction, the motor operates according to the parameters (such as the joint motor rotating at a specified angle or speed, and the gripper motor opening and closing with a specified force).

[0085] The system downloads the data package corresponding to the instruction information of the running environment and loads it to form the model running environment; it loads the action instruction generation model in the model running environment; it inputs the action content to be executed into the action instruction generation model to obtain at least one executable instruction for a motor; it sends each executable instruction to the corresponding motor and executes each executable instruction to perform the corresponding action of the task to be executed. This automated deployment of the running environment improves the efficiency of environment deployment.

[0086] Example 4

[0087] Figure 4This is a schematic diagram of a robot remote drive device according to Embodiment 3 of the present invention. This embodiment of the present invention is applicable to situations involving remote robot drive; the device can execute a robot remote drive method, and the device can be implemented in hardware and / or software.

[0088] See Figure 4 The robot remote drive device shown is used in a remote control system and includes: a task information acquisition module 401, a download information determination module 402, and a robot drive module 403.

[0089] The task information acquisition module 401 is used to acquire the content of the actions to be executed and the action instruction generation model of the task to be executed corresponding to the robot to be controlled. The action instruction generation model is used to input the content of the actions to be executed and output the executable instructions of the motors in the robot to be controlled.

[0090] The download information determination module 402 is used to obtain the environmental download indication information corresponding to the action instruction generation model;

[0091] The robot drive module 403 is used to send the content of the action to be executed, the action instruction generation model, and the environment download indication information of the task to be executed to the robot to be controlled, so that the robot to be controlled can execute the corresponding action according to the content of the action to be executed, the action instruction generation model, and the environment download indication information.

[0092] The technical solution of this invention obtains the content of the action to be executed and the action instruction generation model of the task to be executed corresponding to the robot to be controlled. The action instruction generation model is used to input the content of the action to be executed and output the executable instructions of the motors in the robot to be controlled. It also obtains the environment download indication information corresponding to the action instruction generation model and sends the content of the action to be executed, the action instruction generation model and the environment download indication information of the task to be executed to the robot to be controlled, so that the robot to be controlled can execute the corresponding action according to the content of the action to be executed, the action instruction generation model and the environment download indication information. The operating environment and instruction generation model of the robot to be controlled can be configured remotely without manual configuration, which improves the efficiency of remote robot driving.

[0093] Optionally, the download information determination module 402 includes:

[0094] The current information acquisition unit is used to acquire the current environmental information of the robot to be controlled;

[0095] The environment information determination unit is used to obtain the target environment information corresponding to the action instruction generation model;

[0096] The information comparison unit is used to compare the target environment information with the current environment information to obtain the information comparison result.

[0097] The download data determination unit is used to determine the download environment information required by the robot to be controlled based on the information comparison results.

[0098] The information generation unit is used to generate environment download instruction information based on the download environment information.

[0099] Optionally, the data download determination unit is specifically used for:

[0100] When the information comparison result indicates that the target environment information is consistent with the current environment information, the download environment information required by the robot to be controlled is empty;

[0101] When the information comparison result indicates that the target environment information is inconsistent with the current environment information, at least one associated data packet in the target environment information is obtained, and the associated data packet not included in the current environment information is determined as the download environment information required by the robot to be controlled.

[0102] Optionally, the task information acquisition module 401 is specifically used for:

[0103] Based on the task to be executed corresponding to the robot to be controlled, determine the attribute information of at least one subtask and the execution order of each subtask. The attributes include: at least one task attribute and the attribute value corresponding to each task attribute.

[0104] For each subtask, add the attribute values ​​of each task attribute of the subtask to the corresponding command template to determine the sub-action content corresponding to the subtask;

[0105] The content of each sub-action is sorted according to the execution order to generate the content of actions to be executed.

[0106] Optional attribute information includes: action object, control granularity type, model type, and action type.

[0107] The robot remote drive device provided in the embodiments of the present invention can execute the robot remote drive method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of executing the robot remote drive method.

[0108] Example 5

[0109] Figure 5 This is a schematic diagram of a robot remote drive device according to Embodiment 3 of the present invention. This embodiment of the present invention is applicable to situations involving remote robot drive; the device can execute a robot remote drive method, and the device can be implemented in hardware and / or software.

[0110] See Figure 5 The robot remote drive device shown is configured on the robot to be controlled and includes: a content acquisition module 501 and an action execution module 502, wherein,

[0111] The content acquisition module 501 is used to acquire the content of the action to be executed, the action instruction generation model, and the environment download instruction information of the task to be executed issued by the remote control system;

[0112] The action execution module 502 is used to execute the corresponding action based on the content of the action to be executed, the action instruction generation model, and the environment download instruction information.

[0113] The technical solution of this invention involves acquiring the content of the action to be executed, the action instruction generation model, and the environment download indication information of the task to be executed issued by the remote control system; and executing the corresponding action according to the content of the action to be executed, the action instruction generation model, and the environment download indication information.

[0114] Optionally, the action execution module 502 is specifically used for:

[0115] Download the data package corresponding to the download environment instruction information and load it to form the model's runtime environment;

[0116] Load action instructions into the model runtime environment to generate the model;

[0117] The action to be performed is input into the action instruction generation model to obtain at least one executable instruction for a motor.

[0118] Each executable instruction is sent to the corresponding motor and executed to perform the corresponding action of the task to be performed.

[0119] The robot remote drive device provided in the embodiments of the present invention can execute the robot remote drive method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of executing the robot remote drive method.

[0120] Example 6

[0121] Figure 6 A schematic diagram of a robot remote drive device 600, which can be used to implement embodiments of the present invention, is shown. The robot remote drive device 600 may be a server belonging to a remote control system or a robot to be controlled.

[0122] like Figure 6As shown, the robot remote drive device 600 includes at least one processor 601 and a memory, such as a read-only memory (ROM) 602 and a random access memory (RAM) 603, communicatively connected to the at least one processor 601. The memory stores computer programs executable by the at least one processor. The processor 601 can perform various appropriate actions and processes based on the computer program stored in the ROM 602 or loaded into the RAM 603 from storage unit 608. The RAM 603 can also store various programs and data required for the operation of the robot remote drive device 600. The processor 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0123] Multiple components in the robot remote drive device 600 are connected to the I / O interface 605, including: an input unit 606, such as a keyboard, mouse, etc.; an output unit 607, such as various types of displays, speakers, etc.; a storage unit 608, such as a disk, optical disk, etc.; and a communication unit 609, such as a network card, modem, wireless transceiver, etc. The communication unit 609 allows the robot remote drive device 600 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0124] Processor 601 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. Processor 601 performs the various methods and processes described above, such as remote robot actuation methods.

[0125] In some embodiments, the robot remote drive method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed onto the robot remote drive device 600 via ROM 602 and / or communication unit 609. When the computer program is loaded into RAM 603 and executed by processor 601, one or more steps of the robot remote drive method described above may be performed. Alternatively, in other embodiments, processor 601 may be configured to execute the robot remote drive method by any other suitable means (e.g., by means of firmware).

[0126] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include: implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0127] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0128] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0129] To provide interaction with a user, the systems and techniques described herein can be implemented on a remotely driven robot device, which includes: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the remotely driven robot device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0130] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0131] A computing system can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system. It addresses the shortcomings of traditional physical hosts and VPS (Virtual Private Server) services, such as high management difficulty and weak business scalability.

[0132] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0133] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for remotely driving a robot, characterized in that, Applied to a remote control system, the method includes: Obtain the action content to be executed and the action instruction generation model of the task to be executed corresponding to the robot to be controlled. The action instruction generation model is used to input the action content to be executed and output the executable instructions of the motors in the robot to be controlled. Obtain the environment download indication information corresponding to the action instruction generation model; The robot to be controlled is sent with the action content to be executed for the task to be executed, the action instruction generation model, and the environment download indication information, so that the robot to be controlled can execute the corresponding action according to the action content to be executed, the action instruction generation model, and the environment download indication information.

2. The method according to claim 1, characterized in that, The step of obtaining the environment download indication information corresponding to the action instruction generation model includes: Obtain the current environmental information of the robot to be controlled; Obtain the target environment information corresponding to the action instruction generation model; The target environment information is compared with the current environment information to obtain the information comparison result; Based on the information comparison results, the required download environment information for the robot to be controlled is determined; Based on the download environment information, generate environment download instruction information.

3. The method according to claim 2, characterized in that, The step of determining the download environment information required by the robot to be controlled based on the information comparison results includes: When it is determined that the information comparison result is consistent with the target environment information and the current environment information, the download environment information required by the robot to be controlled is empty; When it is determined that the information comparison result is inconsistent between the target environment information and the current environment information, at least one associated data packet in the target environment information is obtained, and the associated data packet not included in the current environment information is determined as the download environment information required by the robot to be controlled.

4. The method according to claim 1, characterized in that, The process of obtaining the action content to be executed for the task corresponding to the robot to be controlled includes: Based on the task to be executed corresponding to the robot to be controlled, determine the attribute information of at least one subtask and the execution order of each subtask. The attributes include: at least one task attribute and the attribute value corresponding to each task attribute. For each subtask, the attribute values ​​of each task attribute of the subtask are added to the corresponding command template to determine the sub-action content corresponding to the subtask; The sub-actions are sorted according to the execution order to generate action content to be executed.

5. The method according to claim 4, characterized in that, The attribute information includes: action object, control granularity type, model type, and action type.

6. A method for remotely driving a robot, characterized in that, The method, applied to a robot to be controlled, includes: Obtain the content of the action to be executed for the task to be executed issued by the remote control system, the action instruction generation model, and the environment download indication information; The corresponding action is executed based on the content of the action to be executed, the action instruction generation model, and the environment download instruction information.

7. The method according to claim 8, characterized in that, The step of executing the corresponding action based on the content of the action to be executed, the action instruction generation model, and the environment download indication information includes: Download the data package of the runtime environment corresponding to the environment download instruction information, and load it to form the model runtime environment; The action instructions are loaded into the model runtime environment to generate the model; The action to be executed is input into the action instruction generation model to obtain at least one executable instruction for a motor; Each of the executable instructions is sent to the corresponding motor and executed to perform the corresponding action of the task to be performed.

8. The method according to claim 7, characterized in that, The process of loading the action instructions to generate the model in the model runtime environment includes: Generate a model based on the action instructions, and determine the model ports and port connection rules; Based on the model port, search among the environment ports in the model running environment to determine the environment port corresponding to the model port; According to the port connection rules, the model port is connected to the environment port.

9. A robot remote drive device, characterized in that, The robot remote drive device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor to enable the at least one processor to perform the robot remote drive method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the robot remote drive method according to any one of claims 1-7.

Images