Robot control language generation method, system, device and storage medium

Abstract

The application discloses a robot control language generation method, system and device and a storage medium, and comprises the following steps: constructing target prompt information according to a user task description and system information, inputting the target prompt information into a large language model to obtain robot skill language; performing lexical analysis, syntax analysis and semantic checking on the robot skill language through a robot skill language compiler to determine whether the robot skill language conforms to robot skill language rules; when the robot skill language does not conform to the robot skill language rules, generating error feedback information through a robot skill language debugger, updating the target prompt information according to the error feedback information, and returning to the step of inputting the target prompt information into the large language model; and when the robot skill language conforms to the robot skill language rules, compiling the robot skill language through the robot skill language compiler to generate robot control language. The application improves the accuracy and executability of the robot control language and can be applied to the technical field of robot control.

Description

Technical Field

[0001] This invention relates to the field of robot control technology, and in particular to a method, system, device, and storage medium for generating robot control language. Background Technology

[0002] In the field of modern robotics, with the increasing prevalence of intelligent robots, improving the efficiency, accuracy, and debuggability of robot programming has become a crucial research topic. As the application of robotics technology expands in homes, manufacturing, healthcare, and service industries, users require more intuitive and convenient ways to program and operate robots. This has created a demand for direct conversion from natural language (such as user-provided task descriptions) to robot programming languages ​​(such as C++, Python, etc.). However, this conversion process faces the following challenges:

[0003] 1) Conversion from Natural Language to Robot Programming Language: Users typically describe tasks in natural language, such as "grab the bottle on the table" or "move around the obstacle." It is extremely challenging for robots to understand these natural language descriptions and generate corresponding robot code. Existing robot control languages ​​have complex syntax and require users to have a programming background, which most users lack. Therefore, how to utilize artificial intelligence to directly convert natural language into robot-understandable instructions has become an important research direction.

[0004] 2) Correctness and Debugging of Generated Programs: Even if a robot program is successfully generated, its correctness is still difficult to guarantee. Errors in robot programs may not only be syntax errors but also semantic errors caused by inaccurate task decomposition. Furthermore, debugging robot programs is a cumbersome process involving in-depth analysis of code logic, especially in complex scenarios, making it almost impossible for non-professional users to complete. Therefore, providing effective tools to verify the syntax and semantics of generated code is a pressing issue that needs to be addressed.

[0005] There are two main language generation methods in existing technologies for automatically generating robot programs: structure-based methods and semantic-based methods. Structure-based methods use programming constructs (such as Python) to guide the LLM (Language Model) to directly generate robot programs; semantic-based methods decompose user tasks into actions and generate code by mapping these actions to predefined executable functions. However, the former method is prone to errors due to the complexity of robot systems, while the latter suffers from semantic inconsistencies in the two-step generation process. Therefore, neither method can guarantee the correctness of the generated output. Furthermore, traditional debugging tools, such as robot operating system (ROS) debuggers and common compilers / debuggers, are primarily used to check for syntax and logic errors in robot programs. They typically rely on rule-based or simulation-based methods, discovering errors by stepping through program execution. While these tools perform well in detecting syntax errors, they fall short in providing semantic feedback, especially when robot programs involve complex task decomposition and execution. Traditional debugging tools cannot effectively provide feedback on task understanding and semantic consistency, limiting their application in complex tasks.

[0006] In summary, the existing technology has the following drawbacks:

[0007] 1) Low accuracy of generated programs: Existing methods such as ProgPrompt rely heavily on the design and quality of code generation prompts, which can easily lead to syntax or logic errors in the generated programs. Especially when dealing with complex tasks, the generated code is often incomplete or inaccurate.

[0008] 2) Lack of semantic feedback mechanism: While traditional debugging tools can detect syntax errors, they are insufficient in verifying semantic consistency. The correctness of robot programs is not only reflected in syntax, but also requires semantic understanding and correct decomposition of the task. Existing technologies lack effective semantic feedback mechanisms, making it difficult to perform further semantic-level debugging and optimization after code generation.

[0009] 3) Complex debugging and high user requirements: Existing debugging tools rely on rules and simulations, making the debugging process cumbersome and complex, and requiring a high level of programming background from users. For non-professional users, especially those lacking programming experience, existing tools are difficult to use effectively, seriously affecting the popularization and usability of robot programming. Summary of the Invention

[0010] The purpose of this invention is to at least partially solve one of the technical problems existing in the prior art.

[0011] Therefore, one objective of this invention is to provide a robot control language generation method. This method improves the accuracy and executability of robot control language by introducing Robot Skill Language (RSL) and RSL middleware to achieve the conversion between natural language and executable robot programs.

[0012] Another objective of this invention is to provide a robot control language generation system.

[0013] To achieve the above-mentioned technical objectives, the technical solutions adopted in the embodiments of the present invention include:

[0014] In a first aspect, embodiments of the present invention provide a method for generating robot control language, comprising the following steps:

[0015] Obtain user task description and system information, construct target prompt information based on the user task description and system information, input the target prompt information into a large language model to obtain robot skill language;

[0016] The robot skill language is subjected to lexical analysis, syntax analysis and semantic checking by a robot skill language compiler to determine whether the robot skill language conforms to preset robot skill language rules;

[0017] When the robot skill language does not conform to the robot skill language rules, an error feedback message is generated by the robot skill language debugger, the target prompt message is updated according to the error feedback message, and the process returns to the step of inputting the target prompt message into the large language model.

[0018] When the robot skill language conforms to the robot skill language rules, the robot skill language is compiled by the robot skill language compiler to generate the robot control language;

[0019] The robot control language is the underlying Python code for robot control, and the robot skill language is a higher-level abstract description of the underlying Python code for robot control.

[0020] Furthermore, in one embodiment of the present invention, the step of constructing target prompt information based on the user task description and the system information specifically includes:

[0021] The system information is used to determine the basic context required to generate the robot's skill language;

[0022] The target prompt information is obtained by concatenating the user task description and the basic context according to the preset prompt template.

[0023] Furthermore, in one embodiment of the present invention, the step of performing lexical analysis, syntactic analysis, and semantic checking on the robot skill language to determine whether the robot skill language conforms to preset robot skill language rules specifically includes:

[0024] Iterate through each character of the robot skill language, decompose the robot skill language into multiple basic lexical units, and determine the lexical unit type of each basic lexical unit;

[0025] Once all characters of the robot skill language have been successfully decomposed, a corresponding word sequence is generated according to the character arrangement order. An abstract syntax tree is constructed based on the word sequence, and the structural relationship between each basic word is verified based on the abstract syntax tree.

[0026] When the structural relationships of each of the basic lexical units are correct, the abstract syntax tree is semantically verified.

[0027] When the abstract syntax tree passes semantic verification, it is determined that the robot skill language conforms to the robot skill language rules;

[0028] If a character in the robot skill language is not successfully decomposed, or if the structural relationship of the basic lexical units is incorrect, or if the abstract syntax tree fails semantic verification, it is determined that the robot skill language does not conform to the robot skill language rules.

[0029] Furthermore, in one embodiment of the present invention, the generation of error feedback information through the robot skill language debugger specifically includes:

[0030] When a character in the robot skill language fails to be successfully decomposed, the robot skill language compiler generates lexical analysis error information based on the undecomposed character segment and sends the lexical analysis error information to the robot skill language debugger.

[0031] When the structural relationship of the basic lexical unit is incorrect, the robot skill language compiler generates a syntax analysis error message based on the basic lexical unit with the incorrect structural relationship, and sends the syntax analysis error message to the robot skill language debugger.

[0032] When the abstract syntax tree fails semantic verification, the robot skill language compiler generates semantic verification error information based on the subtree that failed semantic verification, and sends the semantic verification error information to the robot skill language debugger.

[0033] The robot skill language compiler generates corresponding error feedback information based on the lexical analysis error information, the syntax analysis error information, or the semantic verification error information.

[0034] Furthermore, in one embodiment of the present invention, updating the target prompt information based on the error feedback information specifically includes:

[0035] Generate feedback description information based on the error feedback information;

[0036] The feedback description information is inserted into the target prompt information according to the preset prompt template to obtain the updated target prompt information.

[0037] Furthermore, in one embodiment of the present invention, the step of compiling the robot skill language using the robot skill language compiler to generate the robot control language specifically includes:

[0038] A depth-first traversal is performed on the abstract syntax tree, and corresponding robot code is generated according to the type of each subtree.

[0039] The robot code is integrated according to the syntactic relationships of each subtree to obtain the robot control language.

[0040] Furthermore, in one embodiment of the present invention, the robot control language generation method further includes the following steps:

[0041] The robot control language is sent to the target robot's programming interface, so that the target robot converts the robot control language into robot action instructions and executes the corresponding user tasks according to the robot action instructions.

[0042] Secondly, embodiments of the present invention provide a robot control language generation system, comprising:

[0043] The robot skill language generation module is used to obtain user task description and system information, construct target prompt information based on the user task description and system information, and input the target prompt information into a large language model to obtain robot skill language;

[0044] The robot skill language analysis module is used to perform lexical analysis, syntactic analysis and semantic checks on the robot skill language through a robot skill language compiler, and to determine whether the robot skill language conforms to preset robot skill language rules.

[0045] The robot skill language fine-tuning module is used to generate error feedback information through the robot skill language debugger when the robot skill language does not conform to the robot skill language rules, update the target prompt information according to the error feedback information, and return to the step of inputting the target prompt information into the large language model;

[0046] A robot control language generation module is used to compile the robot skill language using the robot skill language compiler to generate a robot control language when the robot skill language conforms to the robot skill language rules.

[0047] The robot control language is the underlying Python code for robot control, and the robot skill language is a higher-level abstract description of the underlying Python code for robot control.

[0048] Thirdly, embodiments of the present invention provide a robot control language generation device, comprising:

[0049] At least one processor;

[0050] At least one memory for storing at least one program;

[0051] When the at least one program is executed by the at least one processor, the at least one processor implements the robot control language generation method described above.

[0052] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing a processor-executable program, which, when executed by a processor, is used to perform the above-described robot control language generation method.

[0053] The advantages and beneficial effects of the present invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention:

[0054] This invention obtains user task description and system information, constructs target prompt information based on the user task description and system information, inputs the target prompt information into a large language model to obtain robot skill language, and performs lexical analysis, syntactic analysis and semantic checking on the robot skill language through a robot skill language compiler to determine whether the robot skill language conforms to preset robot skill language rules. If the robot skill language does not conform to the robot skill language rules, an error feedback message is generated through a robot skill language debugger, the target prompt information is updated based on the error feedback message, and the process returns to the step of inputting the target prompt information into the large language model. If the robot skill language conforms to the robot skill language rules, the robot skill language is compiled through a robot skill language compiler to generate robot control language. This invention achieves the conversion between the natural language used to describe user tasks and the executable robot control language by introducing Robot Skill Language (RSL) and RSL middleware (RSL compiler, RSL debugger). It abstracts complex robot program details based on the RSL language and uses the RSL compiler to verify whether the generated RSL program meets syntactic and semantic requirements, improving the accuracy and executability of the robot control language. The RSL debugger generates semantically intuitive error feedback information, enabling the large language model to iteratively fine-tune the RSL program based on the error feedback, improving the semantic consistency of the generated RSL program and further enhancing the accuracy and executability of the robot control language. By automatically generating and debugging robot programs using the large language model, the requirements for user programming skills are reduced, allowing non-professional users to describe tasks using natural language and generate compliant robot code, thus improving the accessibility and usability of robot programming. Attached Figure Description

[0055] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the embodiments of the present invention are described below. It should be understood that the drawings described below are only for the convenience of clearly describing some embodiments of the technical solutions of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0056] Figure 1 A flowchart illustrating the steps of a robot control language generation method provided in an embodiment of the present invention;

[0057] Figure 2 A logic block diagram of a robot control language generation method provided in an embodiment of the present invention;

[0058] Figure 3 A logic block diagram for fine-tuning robot skill language feedback provided in an embodiment of the present invention;

[0059] Figure 4 A structural block diagram of a robot control language generation system provided in an embodiment of the present invention;

[0060] Figure 5 This is a structural block diagram of a robot control language generation device provided in an embodiment of the present invention. Detailed Implementation

[0061] The embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention. The step numbers in the following embodiments are set only for ease of explanation, and there is no limitation on the order between the steps. The execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0062] In the description of this invention, "multiple" means two or more. The use of "first" and "second" is for distinguishing technical features only and should not be construed as indicating or implying relative importance, or implicitly indicating the number of indicated technical features, or the order of the indicated technical features. Furthermore, unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art.

[0063] First, the relevant terminology involved in this invention will be explained.

[0064] RSL (Robot Skill Language): This invention proposes an abstract programming language that aims to convert natural language descriptions into machine-executable code by abstracting the details of robot operations.

[0065] Compiler: A compiler is a tool that translates high-level languages ​​(such as RSL) into low-level machine instructions that a computer can understand and execute. In this invention, the role of the RSL compiler is to perform syntactic analysis and semantic verification on the input RSL program, ensuring that the program conforms to predetermined syntactic and semantic rules, and converting it into instruction code that a robot can execute. The compiler is part of the middleware; it translates human-readable RSL programs into machine-executable code through steps such as lexical analysis, syntax analysis, semantic verification, and machine instruction generation.

[0066] Debugger: A debugger is a tool used to detect errors in a program and help developers fix them. In this invention, the RSL debugger not only detects program errors but also helps the Large Language Model (LLM) understand the type and location of errors by generating intuitive error feedback messages. The debugger is designed based on the lexical and syntactic rules of RSL, enabling it to convert complex technical error information into more easily understood natural language prompts. These prompts allow the LLM to adjust and optimize the code, thereby improving program correctness.

[0067] Large Language Models (LLMs): Large Language Models (LLMs) are a class of deep learning-based natural language processing models, typically containing billions or even hundreds of billions of parameters, capable of processing and generating complex language tasks. LLMs can generate corresponding programs or responses based on a user-provided natural language task description. In this invention, the LLM generates corresponding RSL programs by inputting system messages, task descriptions (such as "grab the bottle on the table"), and sample data. It is a key component in this invention, responsible for converting the user's task's natural language into RSL language. The LLM's generation capabilities are further optimized through a feedback fine-tuning mechanism to ensure that the generated RSL programs meet the robot's execution requirements.

[0068] Reference Figure 1 This invention provides a method for generating robot control language, which specifically includes the following steps:

[0069] S101. Obtain user task description and system information, construct target prompt information based on user task description and system information, input target prompt information into large language model to obtain robot skill language;

[0070] S102. Perform lexical analysis, syntax analysis and semantic checks on the robot skill language through the robot skill language compiler to determine whether the robot skill language conforms to the preset robot skill language rules.

[0071] S103. When the robot skill language does not conform to the robot skill language rules, generate error feedback information through the robot skill language debugger, update the target prompt information according to the error feedback information, and return to the step of inputting the target prompt information into the large language model.

[0072] S104. When the robot skill language conforms to the robot skill language rules, the robot skill language is compiled by the robot skill language compiler to generate the robot control language.

[0073] The robot control language is the underlying Python code for robot control, and the robot skill language is a higher-level abstract description of the underlying Python code for robot control.

[0074] like Figure 2 The diagram shown is a logical block diagram of a robot control language generation method provided by an embodiment of the present invention, mainly including an LLM large language model, RSL middleware, and a robot, wherein:

[0075] The LLM Large Language Model accepts user tasks, system messages, and prompt templates (optional) as input to generate corresponding RSL programs. The system messages and prompt templates specify the role of the LLM, the target language, and the output template, forming prompts for effective guidance of language generation.

[0076] The RSL middleware consists of an RSL compiler and an RSL debugger. The RSL compiler verifies the syntactic and semantic correctness of the RSL program and converts it into executable code for the underlying robot. If an error is found, the RSL debugger generates error feedback information for the LLM to adjust the RSL program.

[0077] The robot includes the target robot and its programming interface, and provides the robot's skill set to execute generated code to complete user tasks.

[0078] Reference Figure 2 A complete process of an embodiment of the present invention is as follows:

[0079] 1) Constructing the Prompt Message: Based on the system information, prompt template, and user task, a prompt message is constructed. The system information provides the basic context required to generate the task, while the sample provides a reference pattern for the LLM large language model to ensure that the generated program conforms to the expected format. The user task is the actual operation that the robot needs to perform (e.g., "grab the bottle on the table"). The user task description is concatenated with the system information to form the final prompt message, which serves as the input for the LLM large language model to generate the robot program.

[0080] 2) Generating RSL Programs (i.e., Robot Skill Language): After receiving the prompts, the LLM Large Language Model generates a preliminary RSL program based on those prompts. This program translates the user's natural language task into an abstract language for the robot, describing the robot's behavior and operations. The generated program may consist of instructions or statements (e.g., "Approach the table; Grab the bottle"), which instruct the robot to perform specific actions.

[0081] 3) RSL Program Compilation and Verification: The generated RSL program is then passed to the RSL compiler for compilation. At this stage, the RSL compiler checks whether the program's syntax and semantics conform to the rules of the RSL programming language. If the RSL program is correct and conforms to the standard, the RSL compiler successfully generates an executable robot program and prepares it for transfer to the robot actuator for actual operation. If the RSL program contains syntax errors or logical problems, the RSL compiler generates error feedback information indicating the problems in the RSL program. The purpose of this stage is to ensure that the generated RSL program is syntactically and semantically valid and can be executed by the robot.

[0082] 4) Error Handling and Feedback Loop: If an error is detected in the RSL program during compilation, the RSL debugger generates an error message and sends it along with the original error message to the LLM Large Language Model. The LLM Large Language Model regenerates the RSL program based on the new error message (i.e., the combination of the original error message and the error message) to correct the previous error. This process repeats until the generated RSL program is error-free or the maximum number of iterations is reached. In each iteration, the LLM Large Language Model adjusts the program through a feedback mechanism to ensure that the syntax and semantics of the RSL program fully meet the requirements, thereby improving the correctness of the generated RSL program.

[0083] 5) Task Execution: After successful compilation, the RSL program generates an executable program, which the robot module receives. The robot module then translates this robot control language into actual robot actions through its internal interface and control system, and begins executing the user's task.

[0084] It can be recognized that the embodiments of the present invention achieve the conversion between the natural language of user task description and executable robot control language by introducing Robot Skill Language (RSL) and RSL middleware (RSL compiler, RSL debugger); the complex robot program details are abstracted based on the RSL language, and the RSL compiler is used to verify whether the generated RSL program meets the syntactic and semantic requirements, thereby improving the accuracy and executability of the robot control language; the RSL debugger generates semantically intuitive error feedback information, enabling the large language model to iteratively fine-tune the RSL program based on the error feedback information, improving the semantic consistency of the generated RSL program, and further improving the accuracy and executability of the robot control language; by using the large language model to automatically generate and debug robot programs, the requirements for user programming ability are reduced, enabling non-professional users to describe tasks in natural language and generate robot code that meets the requirements, thereby improving the popularization and usability of robot programming.

[0085] As a further optional implementation, target prompt information is constructed based on the user task description and system information, specifically including:

[0086] S1011. Determine the basic context required to generate robot skill language based on system information;

[0087] S1012. Based on the preset prompt template, the user task description and basic context are concatenated to obtain the target prompt information.

[0088] As an optional implementation, the robot skill language is further subjected to lexical analysis, syntactic analysis, and semantic checking to determine whether the robot skill language conforms to preset robot skill language rules. Specifically, this includes:

[0089] S1021. Traverse each character of the robot skill language, decompose the robot skill language into multiple basic words, and determine the word type of each basic word.

[0090] S1022. When all characters of the robot skill language are successfully decomposed, the corresponding word sequence is generated according to the character arrangement order. An abstract syntax tree is constructed based on the word sequence, and the structural relationship between each basic word is verified based on the abstract syntax tree.

[0091] S1023. When the structural relationships of each basic word are correct, perform semantic verification on the abstract syntax tree.

[0092] S1024. When the abstract syntax tree passes semantic verification, it is determined that the robot skill language conforms to the robot skill language rules.

[0093] S1025. When a robot skill language has characters that have not been successfully decomposed, or when the structural relationship of basic lexical units is incorrect, or when the abstract syntax tree fails semantic verification, it is determined that the robot skill language does not conform to the robot skill language rules.

[0094] Specifically, the process by which the RSL compiler determines whether a robot's skill language conforms to the preset RSL language rules is as follows:

[0095] 1) Lexical Analysis: The RSL code is broken down into basic units (Tokens) using spaces or newlines, such as keywords, variable names, operators, and symbols. The lexical analyzer categorizes each Token according to language rules, generating a Token sequence and recording its type and order. Parts that do not match the language rules are marked as errors, preventing further parsing.

[0096] 2) Syntax Analysis: The LL(1) syntax analysis method is adopted. The token sequence is parsed through the predictive parsing table to gradually build the abstract syntax tree (AST). The parser verifies the structural relationships between tokens, such as whether there are valid operands on both sides of the operator and whether the control structure is closed. If a part that does not conform to the grammar rules is found, the compiler records the specific error and stops parsing.

[0097] 3) Semantic checking: After the syntax analysis passes, the abstract syntax tree undergoes semantic verification. The compiler checks semantics such as whether variables have been declared to ensure logical consistency. After the semantic check passes, the syntax tree is optimized to remove redundant parts, preparing it for code generation.

[0098] As a further optional implementation, error feedback information is generated through a robot skill language debugger, specifically including:

[0099] S1031. When there are characters in the robot skill language that have not been successfully decomposed, the robot skill language compiler generates lexical analysis error information based on the undecomposed character segments and sends the lexical analysis error information to the robot skill language debugger.

[0100] S1032. When there is an incorrect structural relationship between basic lexical units, the robot skill language compiler generates syntax analysis error information based on the basic lexical units with incorrect structural relationships and sends the syntax analysis error information to the robot skill language debugger.

[0101] S1033. When the abstract syntax tree fails semantic verification, the robot skill language compiler generates semantic verification error information based on the subtree that failed semantic verification and sends the semantic verification error information to the robot skill language debugger.

[0102] S1034. Generate corresponding error feedback information based on lexical analysis error information, syntax analysis error information, or semantic verification error information through the robot skill language compiler.

[0103] As a further optional implementation, the target prompt information is updated based on the error feedback information, specifically including:

[0104] S1035. Generate feedback description information based on error feedback information;

[0105] S1036. Insert the feedback description information into the target prompt information according to the preset prompt template to obtain the updated target prompt information.

[0106] like Figure 3 The diagram shown is a logic block diagram of robot skill language feedback fine-tuning provided in an embodiment of the present invention. The specific process of RSL program feedback fine-tuning is as follows:

[0107] 1) Collecting initial prompts: The user-inputted task (e.g., "grab the bottle on the table") and the initial information provided by the system (including LLM role identity and RSL syntax) together constitute the initial prompts. This prompt clarifies the task the robot needs to complete, serves as the input for the LLM to generate the initial RSL program, and is also the basis for subsequent feedback iterations.

[0108] 2) Error Analysis and Feedback Generation: The RSL compiler performs syntax and semantic checks on the generated initial RSL program. During the check, if problems are found, such as "keywords need to be lowercase" or "statements need to end with a semicolon," the debugger generates explicit error feedback.

[0109] 3) Combining error feedback with original prompts: The system integrates the user's initial prompts with the error feedback generated by the debugger to generate new prompts. These new prompts clearly indicate the specific areas needing improvement, providing clear guidance for LLM to rebuild the program.

[0110] 4) Generate the revised RSL program: LLM regenerates the RSL program based on the new prompts. Combining the original task description and error feedback, LLM improves the code to meet the compiler's requirements, such as correcting keyword capitalization or adding necessary semicolons.

[0111] 5) Iteratively verify and execute the generated program: The compiler checks the corrected code again. If the code fails syntax and semantic verification, steps 2) through 4) are repeated, iterating until verification is passed or the maximum number of repetitions is reached; if the code passes syntax and semantic verification, it is compiled into executable code for the robot. The executable code is passed to the robot module, and the robot performs tasks according to the code instructions, such as approaching a target object and performing a grasping operation.

[0112] As a further optional implementation, the robot skill language is compiled using a robot skill language compiler to generate a robot control language, which specifically includes:

[0113] S1041. Perform a depth-first traversal of the abstract syntax tree and generate corresponding robot code based on the type of each subtree.

[0114] S1042. Integrate the robot code according to the syntactic relationship of each subtree to obtain the robot control language.

[0115] Specifically, the compiler performs a depth-first traversal of the abstract syntax tree, generating corresponding robot code based on the type of each subtree. For example, the conditional subtree generates branch instructions, the loop subtree generates repetitive instructions, and the action subtree generates specific robot control commands, such as "approach object" or "grasp item." The generated code is directly mapped to the robot's interface commands, ensuring it can be loaded into the robot system for execution. Through this stage, the RSL program is ultimately transformed into an executable robot control language.

[0116] As an optional implementation, the robot control language generation method further includes the following steps:

[0117] S105. Send the robot control language to the target robot's programming interface, so that the target robot converts the robot control language into robot action instructions and executes the corresponding user task according to the robot action instructions.

[0118] The method steps of the embodiments of the present invention have been described above. It can be understood that the embodiments of the present invention achieve the conversion between the natural language describing user tasks and the executable robot control language by introducing Robot Skill Language (RSL) and RSL middleware (RSL compiler, RSL debugger); by abstracting complex robot program details based on the RSL language and using the RSL compiler to verify whether the generated RSL program conforms to syntactic and semantic requirements, the accuracy and executability of the robot control language are improved; by using the RSL debugger to generate semantically intuitive error feedback information, the large language model can iteratively fine-tune the RSL program based on the error feedback information, improving the semantic consistency of the generated RSL program and further improving the accuracy and executability of the robot control language; by using the large language model to automatically generate and debug robot programs, the requirements for user programming ability are reduced, enabling non-professional users to describe tasks in natural language and generate compliant robot code, improving the accessibility and usability of robot programming.

[0119] Compared with the prior art, the present invention also has the following advantages:

[0120] 1) This invention achieves an efficient conversion between natural language task descriptions and robot execution code through Robot Skill Language (RSL). RSL provides a concise set of instructions and syntax rules by abstracting low-level operational details, enabling complex robot tasks to be easily mapped to robot execution code through natural language descriptions and the RSL programming language.

[0121] 2) This invention verifies the syntax and semantics of RSL programs through an RSL compiler to ensure that the generated code meets the requirements of the robot task and can be executed successfully; the RSL debugger provides semantic-based error feedback, which allows LLM to adjust the program based on this feedback, thereby promoting code optimization and ensuring the correctness and executability of the generated program.

[0122] 3) This invention proposes a feedback-based optimization mechanism that assists the LLM in optimizing code by generating detailed error feedback during the program generation process. This mechanism guides the LLM to fine-tune the generated program based on the error feedback, effectively improving the correctness and execution performance of the robot program. This mechanism ensures that robot tasks can be iteratively optimized before execution, avoiding task failures caused by erroneous code.

[0123] 4) This invention can handle complex, multi-step or fuzzy robot tasks, improving the feasibility and accuracy of task generation; it can avoid the need for a large amount of computing resources and dedicated corpora for model training, thus improving its applicability in different robots and application scenarios.

[0124] Reference Figure 4 This invention provides a robot control language generation system, comprising:

[0125] The robot skill language generation module is used to obtain user task descriptions and system information, construct target prompt information based on user task descriptions and system information, input the target prompt information into the large language model, and obtain robot skill language;

[0126] The robot skill language analysis module is used to perform lexical analysis, syntax analysis and semantic checks on the robot skill language through the robot skill language compiler to determine whether the robot skill language conforms to the preset robot skill language rules.

[0127] The robot skill language fine-tuning module is used to generate error feedback information through the robot skill language debugger when the robot skill language does not conform to the robot skill language rules, update the target prompt information according to the error feedback information, and return to the step of inputting the target prompt information into the large language model;

[0128] The robot control language generation module is used to compile the robot skill language into a robot control language by means of a robot skill language compiler when the robot skill language conforms to the robot skill language rules.

[0129] The robot control language is the underlying Python code for robot control, and the robot skill language is a higher-level abstract description of the underlying Python code for robot control.

[0130] The content of the above method embodiments is applicable to this system embodiment. The specific functions implemented in this system embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0131] Reference Figure 5 This invention provides a robot control language generation device, comprising:

[0132] At least one processor;

[0133] At least one memory for storing at least one program;

[0134] When the above-mentioned at least one program is executed by the above-mentioned at least one processor, the above-mentioned at least one processor implements the above-mentioned robot control language generation method.

[0135] The content of the above method embodiments is applicable to the device embodiments. The specific functions implemented by the device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0136] This invention also provides a computer-readable storage medium storing a processor-executable program that, when executed by a processor, performs the aforementioned robot control language generation method.

[0137] This invention provides a computer-readable storage medium that can execute a robot control language generation method provided in the method embodiments of this invention. It can execute any combination of the implementation steps of the method embodiments and has the corresponding functions and beneficial effects of the method.

[0138] This invention also discloses a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, causing the computer device to perform... Figure 1 The method shown.

[0139] In some alternative embodiments, the functions / operations mentioned in the block diagrams may not occur in the order shown in the operation diagrams. For example, depending on the functions / operations involved, two consecutively shown blocks may actually be executed substantially simultaneously, or the aforementioned blocks may sometimes be executed in reverse order. Furthermore, the embodiments presented and described in the flowcharts of this invention are provided by way of example to provide a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of various operations is changed and sub-operations described as part of a larger operation are executed independently.

[0140] Furthermore, although the invention has been described in the context of functional modules, it should be understood that, unless otherwise stated, one or more of the aforementioned functions and / or features may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the invention. Rather, given the properties, functions, and internal relationships of the various functional modules in the apparatus disclosed herein, the actual implementation of the module will be understood within the scope of conventional skill of an engineer. Therefore, those skilled in the art can implement the invention as set forth in the claims using ordinary techniques without excessive experimentation. It is also understood that the specific concepts disclosed are merely illustrative and not intended to limit the scope of the invention, which is determined by the full scope of the appended claims and their equivalents.

[0141] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0142] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0143] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the aforementioned program can be printed, because the aforementioned program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or, if necessary, processing in other suitable ways, and then stored in computer memory.

[0144] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0145] In the foregoing description of this specification, references to terms such as "one embodiment," "another embodiment," or "some embodiments" indicate that a specific feature, structure, material, or characteristic described in connection with an embodiment or example is included in at least one embodiment or example of the present invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0146] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

[0147] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.

Claims

1. A method for generating robot control language, characterized in that, Includes the following steps: Obtain user task description and system information, construct target prompt information based on the user task description and system information, input the target prompt information into a large language model to obtain robot skill language; The robot skill language is subjected to lexical analysis, syntax analysis and semantic checking by a robot skill language compiler to determine whether the robot skill language conforms to preset robot skill language rules; When the robot skill language does not conform to the robot skill language rules, an error feedback message is generated by the robot skill language debugger, the target prompt message is updated according to the error feedback message, and the process returns to the step of inputting the target prompt message into the large language model. When the robot skill language conforms to the robot skill language rules, the robot skill language is compiled by the robot skill language compiler to generate the robot control language; The robot control language is the low-level control Python code for the robot, and the robot skill language is a higher-level abstract description of the low-level control Python code for the robot. The step of performing lexical analysis, syntactic analysis, and semantic checking on the robot skill language to determine whether the robot skill language conforms to preset robot skill language rules specifically includes: Iterate through each character of the robot skill language, decompose the robot skill language into multiple basic lexical units, and determine the lexical unit type of each basic lexical unit; Once all characters of the robot skill language have been successfully decomposed, a corresponding word sequence is generated according to the character arrangement order. An abstract syntax tree is constructed based on the word sequence, and the structural relationship between each basic word is verified based on the abstract syntax tree. When the structural relationships of each of the basic lexical units are correct, the abstract syntax tree is semantically verified. When the abstract syntax tree passes semantic verification, it is determined that the robot skill language conforms to the robot skill language rules; If a character in the robot skill language is not successfully decomposed, or if the structural relationship of the basic lexical units is incorrect, or if the abstract syntax tree fails semantic verification, it is determined that the robot skill language does not conform to the robot skill language rules.

2. The robot control language generation method according to claim 1, characterized in that, The step of constructing target prompt information based on the user task description and the system information specifically includes: The system information is used to determine the basic context required to generate the robot's skill language; The target prompt information is obtained by concatenating the user task description and the basic context according to the preset prompt template.

3. The robot control language generation method according to claim 1, characterized in that, The generation of error feedback information through the robot skill language debugger specifically includes: When a character in the robot skill language fails to be successfully decomposed, the robot skill language compiler generates lexical analysis error information based on the undecomposed character segment and sends the lexical analysis error information to the robot skill language debugger. When the structural relationship of the basic lexical unit is incorrect, the robot skill language compiler generates a syntax analysis error message based on the basic lexical unit with the incorrect structural relationship, and sends the syntax analysis error message to the robot skill language debugger. When the abstract syntax tree fails semantic verification, the robot skill language compiler generates semantic verification error information based on the subtree that failed semantic verification, and sends the semantic verification error information to the robot skill language debugger. The robot skill language compiler generates corresponding error feedback information based on the lexical analysis error information, the syntax analysis error information, or the semantic verification error information.

4. The robot control language generation method according to claim 1, characterized in that, The step of updating the target prompt information based on the error feedback information specifically includes: Generate feedback description information based on the error feedback information; The feedback description information is inserted into the target prompt information according to the preset prompt template to obtain the updated target prompt information.

5. The robot control language generation method according to claim 1, characterized in that, The step of compiling the robot skill language using the robot skill language compiler to generate the robot control language specifically includes: A depth-first traversal is performed on the abstract syntax tree, and corresponding robot code is generated according to the type of each subtree. The robot code is integrated according to the syntactic relationships of each subtree to obtain the robot control language.

6. A method for generating robot control language according to any one of claims 1 to 5, characterized in that, The robot control language generation method further includes the following steps: The robot control language is sent to the target robot's programming interface, so that the target robot converts the robot control language into robot action instructions and executes the corresponding user tasks according to the robot action instructions.

7. A robot control language generation system, characterized in that, include: The robot skill language generation module is used to obtain user task description and system information, construct target prompt information based on the user task description and system information, and input the target prompt information into a large language model to obtain robot skill language; The robot skill language analysis module is used to perform lexical analysis, syntactic analysis and semantic checks on the robot skill language through a robot skill language compiler, and to determine whether the robot skill language conforms to preset robot skill language rules. The robot skill language fine-tuning module is used to generate error feedback information through the robot skill language debugger when the robot skill language does not conform to the robot skill language rules, update the target prompt information according to the error feedback information, and return to the step of inputting the target prompt information into the large language model; A robot control language generation module is used to compile the robot skill language using the robot skill language compiler to generate a robot control language when the robot skill language conforms to the robot skill language rules. The robot control language is the low-level control Python code for the robot, and the robot skill language is a higher-level abstract description of the low-level control Python code for the robot. The step of performing lexical analysis, syntactic analysis, and semantic checking on the robot skill language to determine whether the robot skill language conforms to preset robot skill language rules specifically includes: Iterate through each character of the robot skill language, decompose the robot skill language into multiple basic lexical units, and determine the lexical unit type of each basic lexical unit; Once all characters of the robot skill language have been successfully decomposed, a corresponding word sequence is generated according to the character arrangement order. An abstract syntax tree is constructed based on the word sequence, and the structural relationship between each basic word is verified based on the abstract syntax tree. When the structural relationships of each of the basic lexical units are correct, the abstract syntax tree is semantically verified. When the abstract syntax tree passes semantic verification, it is determined that the robot skill language conforms to the robot skill language rules; If a character in the robot skill language is not successfully decomposed, or if the structural relationship of the basic lexical units is incorrect, or if the abstract syntax tree fails semantic verification, it is determined that the robot skill language does not conform to the robot skill language rules.

8. A robot control language generation device, characterized in that, include: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements a robot control language generation method as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a processor-executable program, characterized in that, The processor-executable program, when executed by the processor, is used to perform a robot control language generation method as described in any one of claims 1 to 6.

Images