Robot control language generation method, system and apparatus, and storage medium

Abstract

Disclosed in the present application are a robot control language generation method, system and apparatus, and a storage medium. The method comprises: on the basis of a user task description and system information, constructing target prompt information, and inputting the target prompt information into a large language model, so as to obtain a robot skill language; performing lexical analysis, syntax analysis and a semantic check on the robot skill language by means of a robot skill language compiler, so as to determine whether a robot skill language rule is met; when the robot skill language rule is not met, generating error feedback information by means of a robot skill language debugger, updating the target prompt information on the basis of the error feedback information, and returning to the step of inputting the target prompt information into a large language model; and when the robot skill language rule is met, compiling the robot skill language by means of the robot skill language compiler, so as to generate a robot control language. The present application improves the accuracy and executability of a robot control language, and can be applied to the technical field of robot control.

Description

Robot control language generation method, system, device and storage medium Technical Field

[0001] This application relates to the field of robot control technology, and in particular to a method, system, device and storage medium for generating robot control language. Background Technology

[0002] In the field of modern robotics, with the increasing prevalence of intelligent robots, improving the efficiency, accuracy, and debuggability of robot programming has become a crucial research topic. As the application of robotics technology expands in homes, manufacturing, healthcare, and service industries, users require more intuitive and convenient ways to program and operate robots. This has created a demand for direct conversion from natural language (such as user-provided task descriptions) to robot programming languages ​​(such as C++, Python, etc.). However, this conversion process faces the following challenges:

[0003] 1) Conversion from Natural Language to Robot Programming Language: Users typically describe tasks in natural language, such as "grab the bottle on the table" or "move around the obstacle." It is extremely challenging for robots to understand these natural language descriptions and generate corresponding robot code. Existing robot control languages ​​have complex syntax and require users to have a programming background, which most users lack. Therefore, how to utilize artificial intelligence to directly convert natural language into robot-understandable instructions has become an important research direction.

[0004] 2) Correctness and Debugging of Generated Programs: Even if a robot program is successfully generated, its correctness is still difficult to guarantee. Errors in robot programs may not only be syntax errors but also semantic errors caused by inaccurate task decomposition. Furthermore, debugging robot programs is a cumbersome process involving in-depth analysis of code logic, especially in complex scenarios, making it almost impossible for non-professional users to complete. Therefore, providing effective tools to verify the syntax and semantics of generated code is a pressing issue that needs to be addressed.

[0005] There are two main language generation methods in existing technologies for automatically generating robot programs: structure-based methods and semantic-based methods. Structure-based methods use programming constructs (such as Python) to guide the LLM (Language Model) to directly generate robot programs; semantic-based methods decompose user tasks into actions and generate code by mapping these actions to predefined executable functions. However, the former method is prone to errors due to the complexity of robot systems, while the latter suffers from semantic inconsistencies in the two-step generation process. Therefore, neither method can guarantee the correctness of the generated output. Furthermore, traditional debugging tools, such as robot operating system (ROS) debuggers and common compilers / debuggers, are primarily used to check for syntax and logic errors in robot programs. They typically rely on rule-based or simulation-based methods, discovering errors by stepping through program execution. While these tools perform well in detecting syntax errors, they fall short in providing semantic feedback, especially when robot programs involve complex task decomposition and execution. Traditional debugging tools cannot effectively provide feedback on task understanding and semantic consistency, limiting their application in complex tasks.

[0006] In summary, the existing technology has the following drawbacks:

[0007] 1) Low accuracy of generated programs: Existing methods such as ProgPrompt rely heavily on the design and quality of code generation prompts, which can easily lead to syntax or logic errors in the generated programs. Especially when dealing with complex tasks, the generated code is often incomplete or inaccurate.

[0008] 2) Lack of semantic feedback mechanism: While traditional debugging tools can detect syntax errors, they are insufficient in verifying semantic consistency. The correctness of robot programs is not only reflected in syntax, but also requires semantic understanding and correct decomposition of the task. Existing technologies lack effective semantic feedback mechanisms, making it difficult to perform further semantic-level debugging and optimization after code generation.

[0009] 3) Complex debugging and high user requirements: Existing debugging tools rely on rules and simulations, making the debugging process cumbersome and complex, and requiring a high level of programming background from users. For non-professional users, especially those lacking programming experience, existing tools are difficult to use effectively, seriously affecting the popularization and usability of robot programming. Summary of the Invention

[0010] The purpose of this application is to at least partially solve one of the technical problems existing in the prior art.

[0011] Therefore, one objective of this application is to provide a robot control language generation method, which improves the accuracy and executability of robot control language by introducing Robot Skill Language (RSL) and RSL middleware to achieve the conversion between natural language and executable robot programs.

[0012] Another objective of this application is to provide a robot control language generation system.

[0013] To achieve the above-mentioned technical objectives, the technical solutions adopted in the embodiments of this application include:

[0014] In a first aspect, embodiments of this application provide a method for generating robot control language, comprising the following steps:

[0015] Obtain user task description and system information, construct target prompt information based on the user task description and system information, input the target prompt information into a large language model to obtain robot skill language;

[0016] The robot skill language is subjected to lexical analysis, syntax analysis and semantic checking by a robot skill language compiler to determine whether the robot skill language conforms to preset robot skill language rules;

[0017] When the robot skill language does not conform to the robot skill language rules, an error feedback message is generated by the robot skill language debugger, the target prompt message is updated according to the error feedback message, and the process returns to the step of inputting the target prompt message into the large language model.

[0018] When the robot skill language conforms to the robot skill language rules, the robot skill language is compiled by the robot skill language compiler to generate the robot control language;

[0019] The robot control language is the underlying Python code for robot control, and the robot skill language is a higher-level abstract description of the underlying Python code for robot control.

[0020] Furthermore, in one embodiment of this application, the step of constructing target prompt information based on the user task description and the system information specifically includes:

[0021] The system information is used to determine the basic context required to generate the robot's skill language;

[0022] The target prompt information is obtained by concatenating the user task description and the basic context according to the preset prompt template.

[0023] Furthermore, in one embodiment of this application, the step of performing lexical analysis, syntactic analysis, and semantic checking on the robot skill language to determine whether the robot skill language conforms to preset robot skill language rules specifically includes:

[0024] Traverse each character of the robot skill language, decompose the robot skill language into multiple basic lexical units, and determine the lexical unit type of each basic lexical unit;

[0025] Once all characters of the robot skill language have been successfully decomposed, a corresponding word sequence is generated according to the character arrangement order. An abstract syntax tree is constructed based on the word sequence, and the structural relationship between each basic word is verified based on the abstract syntax tree.

[0026] When the structural relationships of each of the basic lexical units are correct, the abstract syntax tree is semantically verified.

[0027] When the abstract syntax tree passes semantic verification, it is determined that the robot skill language conforms to the robot skill language rules;

[0028] If a character in the robot skill language is not successfully decomposed, or if the structural relationship of the basic lexical units is incorrect, or if the abstract syntax tree fails semantic verification, it is determined that the robot skill language does not conform to the robot skill language rules.

[0029] Furthermore, in one embodiment of this application, the step of generating error feedback information through a robot skill language debugger specifically includes:

[0030] When a character in the robot skill language fails to be successfully decomposed, the robot skill language compiler generates lexical analysis error information based on the undecomposed character segment and sends the lexical analysis error information to the robot skill language debugger.

[0031] When the structural relationship of the basic lexical is incorrect, the robot skill language compiler generates syntax analysis error information based on the basic lexical with incorrect structural relationship, and sends the syntax analysis error information to the robot skill language debugger.

[0032] When the abstract syntax tree fails semantic verification, the robot skill language compiler generates semantic verification error information based on the subtree that failed semantic verification, and sends the semantic verification error information to the robot skill language debugger.

[0033] The robot skill language compiler generates corresponding error feedback information based on the lexical analysis error information, the syntax analysis error information, or the semantic verification error information.

[0034] Furthermore, in one embodiment of this application, updating the target prompt information based on the error feedback information specifically includes:

[0035] Generate feedback description information based on the error feedback information;

[0036] The feedback description information is inserted into the target prompt information according to the preset prompt template to obtain the updated target prompt information.

[0037] Furthermore, in one embodiment of this application, the step of compiling the robot skill language using the robot skill language compiler to generate the robot control language specifically includes:

[0038] A depth-first traversal is performed on the abstract syntax tree, and corresponding robot code is generated according to the type of each subtree.

[0039] The robot code is integrated according to the syntactic relationships of each subtree to obtain the robot control language.

[0040] Furthermore, in one embodiment of this application, the robot control language generation method further includes the following steps:

[0041] The robot control language is sent to the target robot's programming interface, so that the target robot converts the robot control language into robot action instructions and executes the corresponding user tasks according to the robot action instructions.

[0042] Secondly, embodiments of this application provide a robot control language generation system, including:

[0043] The robot skill language generation module is used to acquire user task descriptions and system information, construct target prompt information based on the user task descriptions and system information, and input the target prompt information into a large language model to obtain robot skill language;

[0044] The robot skill language analysis module is used to perform lexical analysis, syntactic analysis and semantic checks on the robot skill language through a robot skill language compiler, and to determine whether the robot skill language conforms to preset robot skill language rules.

[0045] The robot skill language fine-tuning module is used to generate error feedback information through the robot skill language debugger when the robot skill language does not conform to the robot skill language rules, update the target prompt information according to the error feedback information, and return to the step of inputting the target prompt information into the large language model;

[0046] A robot control language generation module is used to compile the robot skill language using the robot skill language compiler to generate a robot control language when the robot skill language conforms to the robot skill language rules.

[0047] The robot control language is the underlying Python code for robot control, and the robot skill language is a higher-level abstract description of the underlying Python code for robot control.

[0048] Thirdly, embodiments of this application provide a robot control language generation apparatus, comprising:

[0049] At least one processor;

[0050] At least one memory for storing at least one program;

[0051] When the at least one program is executed by the at least one processor, the at least one processor implements the robot control language generation method described above.

[0052] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a processor-executable program, which, when executed by a processor, is used to perform the aforementioned robot control language generation method.

[0053] The advantages and beneficial effects of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application:

[0054] This application embodiment obtains user task description and system information, constructs target prompt information based on the user task description and system information, inputs the target prompt information into a large language model to obtain robot skill language, performs lexical analysis, syntax analysis and semantic checking on the robot skill language through a robot skill language compiler to determine whether the robot skill language conforms to preset robot skill language rules. If the robot skill language does not conform to the robot skill language rules, an error feedback message is generated through a robot skill language debugger, the target prompt information is updated based on the error feedback message, and the process returns to the step of inputting the target prompt information into the large language model. If the robot skill language conforms to the robot skill language rules, the robot skill language is compiled through a robot skill language compiler to generate robot control language. This application's embodiments achieve the conversion between the natural language used to describe user tasks and the executable robot control language by introducing Robot Skill Language (RSL) and RSL middleware (RSL compiler, RSL debugger). Complex robot program details are abstracted based on the RSL language, and the RSL compiler verifies whether the generated RSL program meets syntactic and semantic requirements, improving the accuracy and executability of the robot control language. The RSL debugger generates semantically intuitive error feedback information, enabling the large language model to iteratively fine-tune the RSL program based on the error feedback, improving the semantic consistency of the generated RSL program and further enhancing the accuracy and executability of the robot control language. By automatically generating and debugging robot programs using the large language model, the requirements for user programming skills are reduced, allowing non-professional users to describe tasks using natural language and generate compliant robot code, thus improving the accessibility and usability of robot programming. Attached Figure Description

[0055] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the embodiments of this application are described below. It should be understood that the drawings described below are only for the purpose of clearly illustrating some embodiments of the technical solutions in this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0056] Figure 1 is a flowchart of a robot control language generation method provided in an embodiment of this application;

[0057] Figure 2 is a logic block diagram of a robot control language generation method provided in an embodiment of this application;

[0058] Figure 3 is a logic block diagram of robot skill language feedback fine-tuning provided in an embodiment of this application;

[0059] Figure 4 is a structural block diagram of a robot control language generation system provided in an embodiment of this application;

[0060] Figure 5 is a structural block diagram of a robot control language generation device provided in an embodiment of this application. Detailed Implementation

[0061] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application. The step numbers in the following embodiments are set only for ease of explanation, and there is no limitation on the order between the steps. The execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0062] In the description of this application, "multiple" means two or more. The use of "first" and "second" is for the purpose of distinguishing technical features and should not be construed as indicating or implying relative importance, or implicitly indicating the number of technical features indicated, or the order in which the technical features are indicated. Furthermore, unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art.

[0063] First, let me explain the relevant terms used in this application.

[0064] RSL (Robot Skill Language): The abstract programming language proposed in this application aims to convert natural language descriptions into machine-executable code by abstracting the details of robot operations.

[0065] Compiler: A compiler is a tool that translates high-level languages ​​(such as RSL) into low-level machine instructions that a computer can understand and execute. In this application, the role of the RSL compiler is to perform syntactic analysis and semantic verification on the input RSL program, ensuring that the program conforms to predetermined syntactic and semantic rules, and converting it into instruction code that a robot can execute. The compiler is part of the middleware; it translates human-readable RSL programs into machine-executable code through steps such as lexical analysis, syntax analysis, semantic verification, and machine instruction generation.

[0066] Debugger: A debugger is a tool used to detect errors in a program and help developers fix them. In this application, the RSL debugger not only detects program errors but also helps the Large Language Model (LLM) understand the type and location of errors by generating intuitive error feedback messages. The debugger is designed based on the lexical and syntactic rules of RSL, enabling it to convert complex technical error information into more easily understood natural language prompts. These prompts allow the LLM to adjust and optimize the code, thereby improving the program's correctness.

[0067] Large Language Models (LLMs): Large Language Models (LLMs) are a class of deep learning-based natural language processing models, typically containing billions or even hundreds of billions of parameters, capable of processing and generating complex language tasks. LLMs can generate corresponding programs or responses based on a user-provided natural language task description. In this application, the LLM generates corresponding RSL programs by inputting system messages, task descriptions (such as "grab the bottle on the table"), and sample data. It is a key component in this application, responsible for converting the user's task's natural language into RSL language. The LLM's generation capabilities are further optimized through a feedback fine-tuning mechanism to ensure that the generated RSL programs meet the robot's execution requirements.

[0068] Referring to Figure 1, this application embodiment provides a robot control language generation method, which specifically includes the following steps:

[0069] S101. Obtain user task description and system information, construct target prompt information based on user task description and system information, input target prompt information into large language model to obtain robot skill language;

[0070] S102. Perform lexical analysis, syntax analysis and semantic checks on the robot skill language through the robot skill language compiler to determine whether the robot skill language conforms to the preset robot skill language rules.

[0071] S103. When the robot skill language does not conform to the robot skill language rules, generate error feedback information through the robot skill language debugger, update the target prompt information according to the error feedback information, and return to the step of inputting the target prompt information into the large language model.

[0072] S104. When the robot skill language conforms to the robot skill language rules, the robot skill language is compiled by the robot skill language compiler to generate the robot control language.

[0073] The robot control language is the underlying Python code for robot control, and the robot skill language is a higher-level abstract description of the underlying Python code for robot control.

[0074] Figure 2 shows a logic block diagram of a robot control language generation method provided in an embodiment of this application, which mainly includes an LLM large language model, RSL middleware, and a robot, wherein:

[0075] The LLM Large Language Model accepts user tasks, system messages, and prompt templates (optional) as input to generate corresponding RSL programs. The system messages and prompt templates specify the role of the LLM, the target language, and the output template, forming prompts for effective guidance of language generation.

[0076] The RSL middleware consists of an RSL compiler and an RSL debugger. The RSL compiler verifies the syntactic and semantic correctness of the RSL program and converts it into executable code for the underlying robot. If an error is found, the RSL debugger generates error feedback information for the LLM to adjust the RSL program.

[0077] The robot includes the target robot and its programming interface, and provides the robot's skill set to execute generated code to complete user tasks.

[0078] Referring to Figure 2, a complete process of an embodiment of this application is as follows:

[0079] 1) Constructing the Prompt Message: Based on the system information, prompt template, and user task, a prompt message is constructed. The system information provides the basic context required to generate the task, while the sample provides a reference pattern for the LLM large language model to ensure that the generated program conforms to the expected format. The user task is the actual operation that the robot needs to perform (e.g., "grab the bottle on the table"). The user task description is concatenated with the system information to form the final prompt message, which serves as the input for the LLM large language model to generate the robot program.

[0080] 2) Generating RSL Programs (i.e., Robot Skill Language): After receiving the prompts, the LLM Large Language Model generates a preliminary RSL program based on those prompts. This program translates the user's natural language task into an abstract language for the robot, describing the robot's behavior and operations. The generated program may consist of instructions or statements (e.g., "Approach the table; Grab the bottle"), which instruct the robot to perform specific actions.

[0081] 3) RSL Program Compilation and Verification: The generated RSL program is then passed to the RSL compiler for compilation. At this stage, the RSL compiler checks whether the program's syntax and semantics conform to the rules of the RSL programming language. If the RSL program is correct and conforms to the standard, the RSL compiler successfully generates an executable robot program and prepares it for transfer to the robot actuator for actual operation. If the RSL program contains syntax errors or logical problems, the RSL compiler generates error feedback information indicating the problems in the RSL program. The purpose of this stage is to ensure that the generated RSL program is syntactically and semantically valid and can be executed by the robot.

[0082] 4) Error Handling and Feedback Loop: If an error is detected in the RSL program during compilation, the RSL debugger generates an error message and sends it along with the original error message to the LLM Large Language Model. The LLM Large Language Model regenerates the RSL program based on the new error message (i.e., the combination of the original error message and the error message) to correct the previous error. This process repeats until the generated RSL program is error-free or the maximum number of iterations is reached. In each iteration, the LLM Large Language Model adjusts the program through a feedback mechanism to ensure that the syntax and semantics of the RSL program fully meet the requirements, thereby improving the correctness of the generated RSL program.

[0083] 5) Task Execution: After successful compilation, the RSL program generates an executable program, which the robot module receives. The robot module then translates this robot control language into actual robot actions through its internal interface and control system, and begins executing the user's task.

[0084] It can be recognized that the embodiments of this application realize the conversion between the natural language of user task description and executable robot control language by introducing Robot Skill Language (RSL) and RSL middleware (RSL compiler, RSL debugger); the complex robot program details are abstracted based on the RSL language, and the RSL compiler is used to verify whether the generated RSL program meets the syntactic and semantic requirements, thereby improving the accuracy and executability of the robot control language; the RSL debugger generates semantically intuitive error feedback information, enabling the large language model to iteratively fine-tune the RSL program based on the error feedback information, improving the semantic consistency of the generated RSL program, and further improving the accuracy and executability of the robot control language; by using the large language model to automatically generate and debug robot programs, the requirements for user programming ability are reduced, enabling non-professional users to describe tasks in natural language and generate robot code that meets the requirements, thereby improving the popularization and usability of robot programming.

[0085] As a further optional implementation, target prompt information is constructed based on the user task description and system information, specifically including:

[0086] S1011. Determine the basic context required to generate robot skill language based on system information;

[0087] S1012. Based on the preset prompt template, the user task description and basic context are concatenated to obtain the target prompt information.

[0088] As an optional implementation, the robot skill language is further subjected to lexical analysis, syntactic analysis, and semantic checking to determine whether the robot skill language conforms to preset robot skill language rules. Specifically, this includes:

[0089] S1021. Traverse each character of the robot skill language, decompose the robot skill language into multiple basic words, and determine the word type of each basic word.

[0090] S1022. When all characters of the robot skill language are successfully decomposed, the corresponding word sequence is generated according to the character arrangement order. An abstract syntax tree is constructed based on the word sequence, and the structural relationship between each basic word is verified based on the abstract syntax tree.

[0091] S1023. When the structural relationships of each basic word are correct, perform semantic verification on the abstract syntax tree.

[0092] S1024. When the abstract syntax tree passes semantic verification, it is determined that the robot skill language conforms to the robot skill language rules.

[0093] S1025. When a robot skill language has characters that have not been successfully decomposed, or when the structural relationship of basic lexical units is incorrect, or when the abstract syntax tree fails semantic verification, it is determined that the robot skill language does not conform to the robot skill language rules.

[0094] Specifically, the process by which the RSL compiler determines whether a robot's skill language conforms to the preset RSL language rules is as follows:

[0095] 1) Lexical Analysis: The RSL code is broken down into basic units (Tokens) using spaces or newlines, such as keywords, variable names, operators, and symbols. The lexical analyzer categorizes each Token according to language rules, generating a Token sequence and recording its type and order. Parts that do not match the language rules are marked as errors, preventing further parsing.

[0096] 2) Syntax Analysis: The LL(1) syntax analysis method is adopted. The token sequence is parsed through the predictive parsing table to gradually build the abstract syntax tree (AST). The parser verifies the structural relationships between tokens, such as whether there are valid operands on both sides of the operator and whether the control structure is closed. If a part that does not conform to the grammar rules is found, the compiler records the specific error and stops parsing.

[0097] 3) Semantic checking: After the syntax analysis passes, the abstract syntax tree undergoes semantic verification. The compiler checks semantics such as whether variables have been declared to ensure logical consistency. After the semantic check passes, the syntax tree is optimized to remove redundant parts, preparing it for code generation.

[0098] As a further optional implementation, error feedback information is generated through a robot skill language debugger, specifically including:

[0099] S1031. When there are characters in the robot skill language that have not been successfully decomposed, the robot skill language compiler generates lexical analysis error information based on the undecomposed character segments and sends the lexical analysis error information to the robot skill language debugger.

[0100] S1032. When there is an incorrect structural relationship between basic lexical units, the robot skill language compiler generates syntax analysis error information based on the basic lexical units with incorrect structural relationships and sends the syntax analysis error information to the robot skill language debugger.

[0101] S1033. When the abstract syntax tree fails semantic verification, the robot skill language compiler generates semantic verification error information based on the subtree that failed semantic verification and sends the semantic verification error information to the robot skill language debugger.

[0102] S1034. Generate corresponding error feedback information based on lexical analysis error information, syntax analysis error information, or semantic verification error information through the robot skill language compiler.

[0103] As a further optional implementation, the target prompt information is updated based on the error feedback information, specifically including:

[0104] S1035. Generate feedback description information based on error feedback information;

[0105] S1036. Insert the feedback description information into the target prompt information according to the preset prompt template to obtain the updated target prompt information.

[0106] Figure 3 shows the logic block diagram of robot skill language feedback fine-tuning provided in the embodiment of this application. The specific process of RSL program feedback fine-tuning is as follows:

[0107] 1) Collecting initial prompts: The user-inputted task (e.g., "grab the bottle on the table") and the initial information provided by the system (including LLM role identity and RSL syntax) together constitute the initial prompts. This prompt clarifies the task the robot needs to complete, serves as the input for the LLM to generate the initial RSL program, and is also the basis for subsequent feedback iterations.

[0108] 2) Error Analysis and Feedback Generation: The RSL compiler performs syntax and semantic checks on the generated initial RSL program. During the check, if problems are found, such as "keywords need to be lowercase" or "statements need to end with a semicolon," the debugger generates explicit error feedback.

[0109] 3) Combining error feedback with original prompts: The system integrates the user's initial prompts with the error feedback generated by the debugger to generate new prompts. These new prompts clearly indicate the specific areas needing improvement, providing clear guidance for LLM to rebuild the program.

[0110] 4) Generate the revised RSL program: LLM regenerates the RSL program based on the new prompts. Combining the original task description and error feedback, LLM improves the code to meet the compiler's requirements, such as correcting keyword capitalization or adding necessary semicolons.

[0111] 5) Iteratively verify and execute the generated program: The compiler checks the corrected code again. If the code fails syntax and semantic verification, steps 2) through 4) are repeated, iterating until verification is passed or the maximum number of repetitions is reached; if the code passes syntax and semantic verification, it is compiled into executable code for the robot. The executable code is passed to the robot module, and the robot performs tasks according to the code instructions, such as approaching a target object and performing a grasping operation.

[0112] As a further optional implementation, the robot skill language is compiled using a robot skill language compiler to generate a robot control language, which specifically includes:

[0113] S1041. Perform a depth-first traversal of the abstract syntax tree and generate corresponding robot code based on the type of each subtree.

[0114] S1042. Integrate the robot code according to the syntactic relationship of each subtree to obtain the robot control language.

[0115] Specifically, the compiler performs a depth-first traversal of the abstract syntax tree, generating corresponding robot code based on the type of each subtree. For example, the conditional subtree generates branch instructions, the loop subtree generates repetitive instructions, and the action subtree generates specific robot control commands, such as "approach object" or "grasp item." The generated code is directly mapped to the robot's interface commands, ensuring it can be loaded into the robot system for execution. Through this stage, the RSL program is ultimately transformed into an executable robot control language.

[0116] As an optional implementation, the robot control language generation method further includes the following steps:

[0117] S105. Send the robot control language to the target robot's programming interface, so that the target robot converts the robot control language into robot action instructions and executes the corresponding user task according to the robot action instructions.

[0118] The method steps of the embodiments of this application have been described above. It can be understood that the embodiments of this application achieve the conversion between the natural language describing user tasks and the executable robot control language by introducing Robot Skill Language (RSL) and RSL middleware (RSL compiler, RSL debugger); by abstracting complex robot program details based on the RSL language and using the RSL compiler to verify whether the generated RSL program conforms to syntactic and semantic requirements, the accuracy and executability of the robot control language are improved; by using the RSL debugger to generate semantically intuitive error feedback information, the large language model can iteratively fine-tune the RSL program based on the error feedback information, improving the semantic consistency of the generated RSL program and further improving the accuracy and executability of the robot control language; by using the large language model to automatically generate and debug robot programs, the requirements for user programming skills are reduced, enabling non-professional users to describe tasks in natural language and generate compliant robot code, thus improving the accessibility and usability of robot programming.

[0119] Compared with the prior art, this application also has the following advantages:

[0120] 1) This application achieves an efficient conversion between natural language task descriptions and robot execution code through Robot Skill Language (RSL). By abstracting the low-level operational details, the RSL language provides a concise set of instructions and syntax rules, enabling complex robot tasks to be easily mapped to robot execution code through natural language descriptions and the RSL programming language.

[0121] 2) This application verifies the syntax and semantics of the RSL program through the RSL compiler to ensure that the generated code meets the requirements of the robot task and can be executed successfully; the RSL debugger provides semantic-based error feedback, which allows the LLM to adjust the program based on the feedback, thereby promoting code optimization and ensuring the correctness and executability of the generated program.

[0122] 3) This application proposes a feedback-based optimization mechanism that assists the LLM in optimizing code by generating detailed error feedback during the program generation process. This mechanism guides the LLM to fine-tune the generated program based on the error feedback, effectively improving the correctness and execution performance of the robot program. This mechanism ensures that the robot task can be optimized through multiple iterations before execution, avoiding task failures caused by erroneous code.

[0123] 4) This application can handle complex, multi-step or fuzzy robot tasks, improving the feasibility and accuracy of task generation; it can avoid the need for a large amount of computing resources and dedicated corpora for model training, and improve its applicability in different robots and application scenarios.

[0124] Referring to Figure 4, an embodiment of this application provides a robot control language generation system, including:

[0125] The robot skill language generation module is used to obtain user task descriptions and system information, construct target prompt information based on user task descriptions and system information, input the target prompt information into the large language model, and obtain robot skill language;

[0126] The robot skill language analysis module is used to perform lexical analysis, syntax analysis and semantic checks on the robot skill language through the robot skill language compiler to determine whether the robot skill language conforms to the preset robot skill language rules.

[0127] The robot skill language fine-tuning module is used to generate error feedback information through the robot skill language debugger when the robot skill language does not conform to the robot skill language rules, update the target prompt information according to the error feedback information, and return to the step of inputting the target prompt information into the large language model;

[0128] The robot control language generation module is used to compile the robot skill language into a robot control language by means of a robot skill language compiler when the robot skill language conforms to the robot skill language rules.

[0129] The robot control language is the underlying Python code for robot control, and the robot skill language is a higher-level abstract description of the underlying Python code for robot control.

[0130] The content of the above method embodiments is applicable to this system embodiment. The specific functions implemented in this system embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0131] Referring to Figure 5, an embodiment of this application provides a robot control language generation device, including:

[0132] At least one processor;

[0133] At least one memory for storing at least one program;

[0134] When the above-mentioned at least one program is executed by the above-mentioned at least one processor, the above-mentioned at least one processor implements the above-mentioned robot control language generation method.

[0135] The content of the above method embodiments is applicable to the device embodiments. The specific functions implemented by the device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0136] This application also provides a computer-readable storage medium storing a processor-executable program that, when executed by a processor, performs the aforementioned robot control language generation method.

[0137] This application provides a computer-readable storage medium that can execute a robot control language generation method provided in the method embodiments of this application. It can execute any combination of the implementation steps of the method embodiments and has the corresponding functions and beneficial effects of the method.

[0138] This application also discloses a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium and execute the computer instructions, causing the computer device to perform the method shown in FIG1.

[0139] In some alternative embodiments, the functions / operations mentioned in the block diagrams may not occur in the order shown in the operation diagrams. For example, depending on the functions / operations involved, two consecutively shown blocks may actually be executed substantially simultaneously, or the aforementioned blocks may sometimes be executed in reverse order. Furthermore, the embodiments presented and described in the flowcharts of this application are provided by way of example to provide a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of various operations is changed and sub-operations described as part of a larger operation are executed independently.

[0140] Furthermore, although this application is described in the context of functional modules, it should be understood that, unless otherwise stated to the contrary, one or more of the aforementioned functions and / or features may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding this application. Rather, given the properties, functions, and internal relationships of the various functional modules in the apparatus disclosed herein, the actual implementation of the module will be understood within the realm of conventional skill for an engineer. Therefore, those skilled in the art can implement the application set forth in the claims using ordinary techniques without excessive experimentation. It is also understood that the specific concepts disclosed are merely illustrative and not intended to limit the scope of this application, which is determined by the full scope of the appended claims and their equivalents.

[0141] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0142] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0143] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the aforementioned program can be printed, because the aforementioned program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or, if necessary, processing in other suitable ways, and then stored in computer memory.

[0144] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0145] In the foregoing description of this specification, the references to terms such as "one embodiment," "another embodiment," or "some embodiments," etc., indicate that a specific feature, structure, material, or characteristic described in connection with an embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0146] Although embodiments of this application have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the claims and their equivalents.

[0147] The above is a detailed description of the preferred embodiments of this application, but this application is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of this application, and these equivalent modifications or substitutions are all included within the scope defined by the claims of this application.

Claims

1. A robot control language generation method characterized by comprising: The method comprises the following steps: obtaining a user task description and system information, constructing target prompt information according to the user task description and the system information, inputting the target prompt information into a large language model to obtain robot skill language; performing lexical analysis, syntax analysis and semantic checking on the robot skill language through a robot skill language compiler to determine whether the robot skill language conforms to a preset robot skill language rule; when the robot skill language does not conform to the robot skill language rule, generating error feedback information through a robot skill language debugger, updating the target prompt information according to the error feedback information, and returning to the step of inputting the target prompt information into the large language model; when the robot skill language conforms to the robot skill language rule, compiling the robot skill language through the robot skill language compiler to generate robot control language; wherein the robot control language is robot bottom-layer control Python code, and the robot skill language is an upper-layer abstract description of the robot bottom-layer control Python code.

2. The method of claim 1, wherein, The constructing of the target prompt information according to the user task description and the system information specifically comprises: determining a basic context required for generating the robot skill language according to the system information; splicing the user task description and the basic context according to a preset prompt template to obtain the target prompt information.

3. The method of claim 1, wherein, The lexical analysis, syntax analysis and semantic checking on the robot skill language to determine whether the robot skill language conforms to the preset robot skill language rule specifically comprises: traversing each character of the robot skill language, disassembling the robot skill language into a plurality of basic tokens, and determining a token type of each basic token; when all characters of the robot skill language are successfully disassembled, generating a corresponding token sequence according to the character arrangement order, constructing an abstract syntax tree according to the token sequence, and verifying whether the structural relationship between the basic tokens is correct according to the abstract syntax tree; when the structural relationship between the basic tokens is correct, performing semantic verification on the abstract syntax tree; when the abstract syntax tree passes the semantic verification, it is determined that the robot skill language conforms to the robot skill language rule; when there are characters of the robot skill language that are not successfully disassembled, or when the structural relationship between the basic tokens is incorrect, or when the abstract syntax tree does not pass the semantic verification, it is determined that the robot skill language does not conform to the robot skill language rule.

4. The method of claim 3, wherein, The generating of error feedback information through the robot skill language debugger specifically comprises: when there are characters of the robot skill language that are not successfully disassembled, generating lexical analysis error information according to the character segments that are not successfully disassembled through the robot skill language compiler, and sending the lexical analysis error information to the robot skill language debugger; When the structural relationship of the basic word element is incorrect, the robot skill language compiler generates syntax error information according to the basic word element with incorrect structural relationship, and sends the syntax error information to the robot skill language debugger; When the abstract syntax tree fails the semantic verification, the robot skill language compiler generates semantic verification error information according to the sub-tree that fails the semantic verification, and sends the semantic verification error information to the robot skill language debugger; The robot skill language compiler generates corresponding error feedback information according to the lexical analysis error information, the syntax error information, or the semantic verification error information.

5. The method of claim 1, wherein, The updating of the target prompt information according to the error feedback information specifically includes: generating feedback description information according to the error feedback information; inserting the feedback description information into the target prompt information according to a preset prompt template to obtain updated target prompt information.

6. The method of claim 3, wherein, The compiling of the robot skill language by the robot skill language compiler to generate a robot control language specifically includes: performing depth-first traversal on the abstract syntax tree to generate corresponding robot code according to the type of each sub-tree; integrating the robot code according to the syntax relationship of each sub-tree to obtain the robot control language.

7. The method of generating a robot control language according to any one of claims 1 to 6, wherein, The robot control language generation method further includes the following steps: sending the robot control language to a programming interface of a target robot, so that the target robot converts the robot control language into robot action instructions and performs corresponding user tasks according to the robot action instructions.

8. A robot control language generation system characterized by comprising: It includes: a robot skill language generation module configured to obtain user task description and system information, construct target prompt information according to the user task description and the system information, input the target prompt information into a large language model, and obtain robot skill language; a robot skill language analysis module configured to perform lexical analysis, syntax analysis, and semantic checking on the robot skill language by a robot skill language compiler, and determine whether the robot skill language conforms to a preset robot skill language rule; a robot skill language fine-tuning module configured to, when the robot skill language does not conform to the robot skill language rule, generate error feedback information by a robot skill language debugger, update the target prompt information according to the error feedback information, and return to the step of inputting the target prompt information into the large language model; a robot control language generation module configured to, when the robot skill language conforms to the robot skill language rule, compile the robot skill language by the robot skill language compiler to generate a robot control language; The robot control language is a robot bottom-layer control Python code, and the robot skill language is an upper-layer abstract description of the robot bottom-layer control Python code.

9. A robot control language generation apparatus characterized by comprising: It includes: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor is caused to realize the robot control language generation method according to any one of claims 1 to 7.

10. A computer readable storage medium having stored therein a program which is executable by a processor, characterized in that, The program executable by the processor, when executed by the processor, is for performing the robot control language generation method according to any one of claims 1 to 7.

Images