Robot control method based on natural language dialogue and electronic device

Abstract

The application discloses a robot control method based on natural language dialogue and electronic equipment. The method comprises the following steps: performing character segmentation on a target operation instruction of a robot to obtain a character segmentation result; determining an initial operation intention for the robot according to the character segmentation result; if a text complexity score of the target operation instruction is greater than a preset complexity threshold, obtaining a target operation intention based on the initial operation intention and using a predetermined language model; if the text complexity score is less than or equal to the preset complexity threshold, determining the initial operation intention as the target operation intention; generating a motion trajectory instruction sequence of the robot based on the target operation intention; and sending the motion trajectory instruction sequence to the robot. The application solves the technical problem that, in related technologies, the accuracy of natural language instruction intention recognition and the response speed are difficult to be considered at the same time, resulting in low robot control efficiency and poor reliability.

Description

Technical Field

[0001] This invention relates to the field of robot control, and more specifically, to a robot control method and electronic device based on natural language dialogue. Background Technology

[0002] In the field of industrial robot control, natural language interaction has attracted attention due to its intuitiveness. However, the intention recognition of natural language commands in related technologies faces a technical challenge of balancing accuracy and response speed. On the one hand, if complex deep learning models are used for intention parsing to pursue high accuracy, the massive computational load leads to high inference latency, making it difficult to meet the real-time requirements of command response in industrial settings, resulting in low control efficiency. On the other hand, if simple rule matching or keyword recognition is used to ensure response speed, it is difficult to handle the ambiguity and complex context of natural language, leading to a significant decrease in intention recognition accuracy and a high risk of misjudgment. This dilemma of "high latency" or "low accuracy" means that robot control systems either suffer from operational smoothness due to response lag or execute incorrect paths due to misinterpretation of intention. Especially when it comes to generating robot motion trajectories, incorrect intention recognition directly leads to deviations in the generated motion trajectory command sequence, which in turn causes abnormal robot movements or collision risks, severely reducing the reliability of robot control and overall operational efficiency.

[0003] There is currently no effective solution to the above problems. Summary of the Invention

[0004] This invention provides a robot control method and electronic device based on natural language dialogue, which at least solves the technical problem that related technologies cannot simultaneously achieve both high accuracy and fast response speed in recognizing natural language command intent during robot control, resulting in low robot control efficiency and poor reliability.

[0005] According to one aspect of the present invention, a robot control method based on natural language dialogue is provided, comprising: performing character segmentation on a target operation instruction of the robot to obtain a character segmentation result, wherein the target operation instruction is an instruction input through natural language; determining an initial operation intention for the robot and a text complexity score of the target operation instruction based on the character segmentation result; if the text complexity score is greater than a preset complexity threshold, obtaining a target operation intention based on the initial operation intention and using a predetermined language model; if the text complexity score is less than or equal to the preset complexity threshold, determining the initial operation intention as the target operation intention; generating a motion trajectory instruction sequence for the robot based on the target operation intention; and sending the motion trajectory instruction sequence to the robot.

[0006] According to another aspect of the present invention, a non-volatile storage medium is also provided, the non-volatile storage medium storing a plurality of instructions adapted for loading by a processor and executing any one of the robot control methods based on natural language dialogue.

[0007] According to another aspect of the present invention, an electronic device is also provided, including one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement any of the natural language dialogue-based robot control methods described above.

[0008] According to another aspect of the present invention, a computer program product is also provided, including a computer program that, when executed by a processor, implements the steps of any of the natural language dialogue-based robot control methods described herein.

[0009] In this embodiment of the invention, character segmentation is performed on the target operation instruction of the robot to obtain a character segmentation result, wherein the target operation instruction is an instruction input through natural language; based on the character segmentation result, the initial operation intention for the robot and the text complexity score of the target operation instruction are determined; if the text complexity score is greater than a preset complexity threshold, the target operation intention is obtained based on the initial operation intention using a predetermined language model; if the text complexity score is less than or equal to the preset complexity threshold, the initial operation intention is determined as the target operation intention; based on the target operation intention, a motion trajectory instruction sequence for the robot is generated; and the motion trajectory instruction sequence is sent to the robot to achieve... This method involves character segmentation of the target operation command, matching with preset rules to determine the initial operation intent, and calculating a text complexity score. Based on whether the score exceeds a threshold, it dynamically selects to call a predefined language model or directly uses the initial intent as the target operation intent. Based on this operation intent, it accurately and efficiently generates the robot's motion trajectory command sequence. This achieves the technical effect of ensuring both rapid response to simple commands and accurate recognition of complex command intents, thereby improving the overall reliability and efficiency of the robot control scheme. It also solves the technical problem that related technologies struggle to balance the accuracy and response speed of natural language command intent recognition in robot control, leading to low robot control efficiency and poor reliability. Attached Figure Description

[0010] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0011] Figure 1This is a flowchart of a robot control method based on natural language dialogue according to an embodiment of the present invention;

[0012] Figure 2 This is a first schematic diagram of an optional user interface according to an embodiment of the present invention;

[0013] Figure 3 This is a second schematic diagram of an optional user interface according to an embodiment of the present invention;

[0014] Figure 4 This is a third schematic diagram of an optional user interface according to an embodiment of the present invention;

[0015] Figure 5 This is a schematic diagram of a robot control device based on natural language dialogue according to an embodiment of the present invention. Detailed Implementation

[0016] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0017] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0018] According to an embodiment of the present invention, a method embodiment for robot control based on natural language dialogue is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0019] Figure 1This is a flowchart of a robot control method based on natural language dialogue according to an embodiment of the present invention, such as... Figure 1 As shown, the method includes the following steps:

[0020] Step S102: Perform character segmentation on the target operation command of the robot to obtain the character segmentation result, wherein the target operation command is the command input through natural language.

[0021] In this embodiment, the target operation instruction input in natural language form is first subjected to character segmentation processing. This process aims to break down a continuous text sequence into independent character units, thereby forming a structured character segmentation result. This processing method transforms unstructured natural language instructions into discrete data units, providing basic data support for subsequent intent recognition, keyword matching, and logical parsing based on character granularity or substring patterns, ensuring that the system can accurately capture key information fragments in the instruction.

[0022] Optionally, the text can be split into ordered clauses based on Chinese conjunctions (and, also, and, then, next, again, etc.) and punctuation, such as "move to point A, bind the tool, then move to point B" being split into 3 clauses as the character segmentation result.

[0023] Step S104: Based on the character segmentation results, determine the initial operation intention for the robot and the text complexity score of the target operation instruction.

[0024] In this embodiment, the results obtained through character segmentation are first parsed to clarify the user's initial operational intent towards the robot. This step involves semantic or rule-level recognition of the segmented text units, transforming unstructured language fragments into specific operational instructions that the system can understand, thereby establishing the basic objective for subsequent action execution. Simultaneously, based on the segmentation results, the complexity of the target operational instruction is analyzed, and a text complexity score is calculated. This score quantifies the number of verbs or the structural complexity of the instruction, providing a quantitative basis for determining whether the instruction is a simple single-step operation or a complex multi-step task, thus deciding whether to proceed directly or enter a more complex planning process.

[0025] In one optional embodiment, determining the initial operation intention for the robot based on the character segmentation result includes: obtaining the original operation intention based on the character segmentation result; if there are multiple original operation intentions, optimizing the multiple original operation intentions based on a priority topology chain to obtain the initial operation intention, wherein the priority topology chain predefines the priority dependency relationship between different categories of operation intentions, and the optimization process includes suppressing or eliminating low-priority original operation intentions according to the priority topology chain; if there is only one original operation intention, using the original operation intention as the initial operation intention.

[0026] In this embodiment, obtaining the original operation intent based on the character segmentation result means mapping the discrete character sequence after character segmentation into operational semantics that the robot can understand. For example, identifying independent character fragments such as "grasp" and "move" as specific action instructions. When multiple potential original operation intents are identified (i.e., there is ambiguity, such as matching "grasp" and "place" simultaneously), conflict resolution is performed by introducing a predefined priority topology chain. This topology chain explicitly stores the dependencies and priority order between different types of intents in a hierarchical structure. For example, it stipulates that "positioning" or "activating the gripper" or other pre- or high-priority actions must be completed before performing the "grasp" action. At this time, the optimized processing... Based on the priority relationship in the topology chain, the process automatically suppresses or eliminates original operation intentions with lower priority, retaining only intentions that conform to high-priority logic as initial operation intentions. If the character segmentation result corresponds to only a single original operation intention, it is directly confirmed as the initial operation intention without additional filtering. By adopting the above mechanism, multiple intention conflicts are automatically sorted and eliminated through preset priority rules, which effectively avoids ambiguity in intention recognition caused by the fuzziness of natural language instructions. This ensures that regardless of the complexity of the instruction, the initial operation intention received by the robot is unique and deterministic, thereby significantly improving the accuracy of intention recognition and the reliability of subsequent motion trajectory generation.

[0027] Optionally, the robot's original operational intent can be obtained by identifying the operation intent of each clause in the character segmentation result through preset keywords or regular expression matching. Pattern matching for over 20 operation categories is performed on each clause, covering all operation types in offline programming. Some examples are shown in Table 1 below. A conflict suppression mechanism is further implemented, constructing a priority topology chain based on the offline programming business logic. When the same text clause triggers multi-feature intent conflicts, the lower-priority intent features are masked or filtered out according to the topology chain. Specifically, the topology relationships are: "suction cup blowing" suppresses "binding tool" (to avoid "blowing away" triggering suction binding simultaneously); "setting robot / loading tool / loading model" suppresses "modifying point position". Furthermore, expressions such as "add point position", "insert", and "join" are routed to "add motion" (trajectory operation) instead of "modify point position" to avoid confusion between point list management and trajectory editing.

[0028] Table 1

[0029]

[0030] Step S106: If the text complexity score is greater than the preset complexity threshold, the target operation intent is obtained by using a predetermined language model based on the initial operation intent.

[0031] In this embodiment, the predetermined language model can be, but is not limited to, a large language model. When the complexity score of the currently received user instruction text is determined to be higher than a preset complexity threshold, it indicates that the instruction contains multiple actions, logical branches, or high semantic complexity, exceeding the scope of simple rule matching. At this time, the predetermined language model is activated as a processing engine. The initial operation intent (i.e., the basic intent obtained after preliminary parsing or rule matching) is input into the predetermined language model, and its semantic understanding and reasoning capabilities are used to conduct in-depth analysis and transformation of the initial intent, ultimately generating a target operation intent that can accurately cover the user's complex needs, thereby ensuring the accurate identification and processing of highly complex natural language instructions.

[0032] Step S108: If the text complexity score is less than or equal to the preset complexity threshold, the initial operation intention is determined as the target operation intention.

[0033] In this embodiment, the complexity of natural language instructions is first assessed by statistically analyzing the frequency of conjunctions and specific action keywords to determine the processing difficulty of the instructions. When the calculated text complexity score is less than or equal to a preset complexity threshold, it indicates that the instruction is a simple task. The instruction is then directly parsed and confirmed as the target operation intent, thereby avoiding the need to call time-consuming AI models for fallback classification and ensuring rapid response and execution of routine simple instructions.

[0034] Step S110: Based on the target operation intention, generate a sequence of motion trajectory instructions for the robot.

[0035] In this embodiment, a sequence of motion trajectory instructions for the robot is generated based on the target operation intention. This step first parses the specific operation target given by the user in natural language, converting it into an executable logical intention (i.e., the target operation intention). Then, based on this intention, an underlying algorithm or planner is invoked to calculate the motion path of the robot's end effector in three-dimensional space. This path is then decomposed into a discrete set of instructions containing parameters such as position, attitude, velocity, and I / O control, ultimately forming a sequence of motion trajectory instructions arranged in chronological order for subsequent simulation verification or direct execution.

[0036] In one optional embodiment, generating a sequence of motion trajectory instructions for the robot based on the target operation intention includes: determining a motion trajectory generation strategy for the robot according to the target operation intention; and obtaining a sequence of motion trajectory instructions based on the motion trajectory generation strategy.

[0037] In this embodiment, determining the robot's motion trajectory generation strategy refers to selecting or planning a specific execution path or action logic based on the parsed target operation intention, combined with the robot's current joint state, the distribution of environmental obstacles, and task priority. For example, the abstract intention of "grabbing an object" is transformed into a specific sequence of steps such as "first move to the top of the object, then descend vertically and close the end effector." Based on the motion trajectory generation strategy, the motion trajectory instruction sequence is obtained, which means further refining the above motion trajectory generation strategy into parameters such as the specific position, velocity, and acceleration of each joint of the robot on the time axis, forming a low-level instruction stream that can be directly parsed and executed by the controller, such as specific motor pulse sequences or coordinate point data. By introducing this intermediate layer, the risk of abrupt actions, path conflicts, or collisions caused by the lack of specific execution path planning when jumping directly from natural language intention to low-level motion instructions is avoided. This achieves a smooth transition from high-level semantic understanding to low-level physical execution, significantly improving the path rationality, motion smoothness, and overall control efficiency when the robot performs complex operations.

[0038] In one optional embodiment, determining the robot's motion trajectory generation strategy based on the target operation intention includes: determining whether the target operation intention belongs to a preset operation type, wherein the preset operation type refers to a query operation or a parameter modification operation; if the target operation intention belongs to a preset operation type, determining the motion trajectory generation strategy as a first strategy, wherein the first strategy is used to instruct the direct generation of a motion trajectory instruction sequence; if the target operation intention does not belong to a preset operation type, determining the motion trajectory generation strategy as a second strategy, wherein the second strategy is used to instruct the invocation of a predetermined language model to generate a motion trajectory instruction sequence.

[0039] In this embodiment, the robot's motion trajectory generation strategy is determined based on the target operation intention. Specifically, this includes determining whether the intention belongs to a preset operation type, where the preset operation type is a simple operation type, which may include, but is not limited to, query operations or parameter modification operations. These operations typically do not involve complex physical space pose planning but focus on information acquisition or system configuration. When the target operation intention is determined to belong to the aforementioned preset operation type, the first strategy is directly adopted. The first strategy instructs the direct generation of a motion trajectory instruction sequence, meaning that the time-consuming language model inference process is bypassed, and instructions are quickly output through a preset rule base or mapping table, thereby achieving immediate response to simple instructions. Conversely, if the intention does not belong to the preset operation type, the robot's motion trajectory generation strategy is determined based on the target operation intention. When the target operation intent does not belong to the preset operation type, such as when it involves complex assembly, handling, or obstacle avoidance tasks, a second strategy is adopted. The second strategy instructs the invocation of a pre-defined language model, which utilizes its semantic understanding and logical reasoning capabilities to generate a sequence of motion trajectory instructions adapted to the complex scenario. Through this differentiated routing mechanism based on intent business type, the optimal processing path can be automatically matched according to the complexity of the operation instruction. This avoids the invalid computational delay caused by calling large language models in simple query or parameter adjustment operations, and ensures that complex operations can obtain accurate trajectory planning. As a result, the robot's overall processing efficiency and control reliability for various natural language instructions are improved.

[0040] In one optional embodiment, a motion trajectory instruction sequence is obtained based on a motion trajectory generation strategy, including: when the motion trajectory generation strategy is a first strategy, obtaining the motion trajectory instruction sequence in the following manner: acquiring the robot's real-time data status; performing corresponding query operations or parameter modification operations on the robot's original control data according to the real-time data status and the target operation intention to obtain target control data; and converting the target control data into a motion trajectory instruction sequence.

[0041] In this embodiment, the motion trajectory generation strategy refers to the specific execution path determined based on the semantic features of the target operation intention and the context. The first strategy refers to the processing mode adopted for operations involving data query or parameter correction intentions. When the first strategy is adopted, the robot's real-time data status is first acquired, i.e., the latest environmental information such as joint angles, end-effector pose, and load status is read through sensors or internal registers to ensure that subsequent processing is based on the current actual working conditions. Subsequently, based on the real-time status and the target operation intention, query operations or parameter modification operations are performed on the pre-stored or basic configured raw control data. For example, when the user instruction is "adjust the gripping force to 50N", the query or parameter modification operation is performed directly on the raw data. Instead of recalculating complex inverse kinematics, the algorithm focuses on locating and updating the force parameter, thus obtaining updated target control data. Finally, the precisely corrected or confirmed target control data is directly mapped and converted into specific joint motion angle or velocity sequences, i.e., motion trajectory command sequences. Through these steps, this solution establishes a dedicated, high-efficiency processing channel for simple query or parameter modification commands, avoiding redundant calculations in general trajectory planning algorithms. This not only significantly shortens the response time from intent recognition to command generation but also ensures strict consistency between control data and the robot's real-time state. As a result, while improving control efficiency, it enhances the reliability and accuracy of the robot in executing such fine-tuning commands.

[0042] As an optional embodiment, based on the real-time data status and the target operation intention, the robot's original control data is subjected to corresponding query operations or parameter modification operations to obtain target control data, including: when the target operation intention indicates that a new motion point is to be inserted into the trajectory, obtaining the anchor point position and insertion direction specified by the user; finding the target motion point corresponding to the anchor point in the real-time data status; identifying the boundary of non-motion elements adjacent to the target motion point, wherein the non-motion elements include at least one of IO control instructions, wait instructions, or tool binding instructions; inserting the new motion point at the boundary of the non-motion element, and automatically inheriting the motion parameters of the motion points adjacent to the target motion point.

[0043] In this embodiment, when the target operation intention indicates the insertion of a new motion point into the trajectory, the anchor point position and insertion direction specified by the user are first obtained to determine the reference point and extension trend of the insertion operation. Then, the target motion point corresponding to the anchor point is found in the real-time data status, mapping the abstract user instruction to a specific internal data node of the robot. Next, the boundaries of non-motion elements adjacent to the target motion point are identified, where non-motion elements include at least one of IO control instructions, waiting instructions, or tool binding instructions. This means that it checks whether there are non-path control logic instructions before and after the target motion point, thereby accurately defining the "safe zone" suitable for inserting new actions. Finally, the new motion point is inserted at the identified boundaries of non-motion elements, and the motion parameters of the adjacent motion points of the target motion point, such as velocity and acceleration, are automatically inherited to ensure that the newly inserted motion point maintains a smooth transition in motion characteristics with the original trajectory, avoiding mechanical impact or trajectory breakage caused by parameter abrupt changes. Through the above steps, this solution can intelligently locate and insert new motion commands in complex control processes, effectively avoiding conflicts with non-motion elements such as IO and waiting, while ensuring the smoothness and rationality of the motion at the newly inserted point, thereby improving the flexibility of robot trajectory editing and the reliability of control data.

[0044] In one optional embodiment, a motion trajectory instruction sequence is obtained based on a motion trajectory generation strategy, including: when the motion trajectory generation strategy is a second strategy, the motion trajectory instruction sequence is obtained in the following manner: generating structured input data based on preset rules, the robot's real-time data status, and historical context data, wherein the preset rules are used to instruct the encoding of proprietary business knowledge in the offline programming domain into safety and logical constraints, the safety and logical constraints include at least one of prohibiting the fabrication of point names and IO port numbers, symmetrical loading and unloading patterns, automatic insertion rules for upper points, and automatic insertion rules for transition points, and the historical context data includes historical operation instructions before the target operation instruction and the corresponding system response summary; and obtaining the motion trajectory instruction sequence based on the structured input data using a predetermined language model.

[0045] In this embodiment, when the motion trajectory generation strategy is the second strategy, it refers to determining whether to adopt a high-level generation method based on a language model, rather than simple rule mapping, based on the complexity of the target operation intention or specific needs. At this time, structured input data is generated based on preset rules, the robot's real-time data status, and historical context data. This integrates static business logic, dynamic environmental information, and past dialogue memories into an input format recognizable by the model. The preset rules are used to transform proprietary business knowledge from the offline programming domain into specific "safety and logic constraints," such as forcibly stipulating "prohibition of fabricating point names and IO port numbers" to prevent invalid instructions, "symmetrical loading and unloading patterns" to ensure process balance, and "automatic insertion rules for upper points" and "automatic insertion rules for transition points" to optimize motion smoothness. The constraints directly limit the output space of the language model, requiring the generated trajectory to conform to industrial safety standards and process logic. Simultaneously, the historical context data, composed of previous operation instructions and corresponding system response summaries, enables the model to perceive the context of multi-turn interactions, avoiding repetitive or contradictory instructions. By processing this structured input data using a predefined language model to obtain the motion trajectory instruction sequence, not only is the semantic understanding capability of the language model utilized, but the hard constraints and contextual dependencies in the structured input ensure that the output trajectory instructions possess high reliability, safety, and logical consistency in industrial robot control scenarios. This effectively avoids the problem of general models fabricating incorrect points or violating process logic due to a lack of domain knowledge constraints, thereby achieving high-precision intention execution and stable robot control efficiency.

[0046] In one optional embodiment, the structured input data includes a role definition field, a skill catalog constraint field, a real-time status field, a dialogue history context field, a safety and logic constraint field, and an output format constraint and user instruction field. The role definition field defines the role of the predefined language model as an offline programming trajectory planning assistant. The skill catalog constraint field lists the name, function description, parameter list, and return value structure of the robot's operational skills corresponding to the steps in the motion trajectory instruction sequence. The real-time status field is used to populate the real-time data status, which includes the network interface address of the offline programming software, as well as a list of known point names, loaded tool names, and trajectory instructions at the current moment. The system includes the number of entries and the suction cup I / O port cache status. The known point name list includes multiple points and their corresponding point labels. Point labels include role labels, associated object labels, applicable tool labels, and parent association labels. Parent association labels indicate the relationship between the corresponding point and its parent point, which is either the approach point or the return point of the corresponding point. The dialogue history context field stores historical context data. The security and logical constraint field stores security and logical constraints. The output format constraint and user instruction field limits the output of the predefined language model to a predefined array format containing step numbers, operation skill names, parameter lists, and step purpose descriptions, along with the target operation instruction.

[0047] In this embodiment, the construction of structured input data involves six specific fields working together to transform proprietary business knowledge in the offline programming domain into explicit constraints on the language model. The role definition field limits the model's identity to an offline programming trajectory planning assistant, focusing it on professional domain tasks. The skill catalog constraint field explicitly lists the names, functions, parameters, and return values ​​of available operational skills, defining legal operational boundaries for the model and preventing it from calling non-existent instructions. The real-time status field includes the network interface address, a list of known point names (including roles, associated objects, applicable tools, and parent association tags, where the parent association tags indicate the hierarchical relationship between the point and the approach or return point), and loaded tools. Real-time data, including the number of trajectory commands and the status of suction cup I / O port caches, ensures that the point names and I / O port numbers generated by the model originate from the real device status, rather than fictitious data generated by the model's illusion. The dialogue history context field retains historical operation commands and system response summaries, enabling the model to perform coherent planning based on the context. The safety and logical constraint field forcibly incorporates safety rules such as prohibiting the fabrication of points / I / O, symmetrical loading and unloading patterns, automatic insertion of upper points, and automatic insertion of transition points, logically avoiding illegal actions. The output format constraint and user command field standardizes the output to an array format containing step numbers, skill names, parameters, and purpose descriptions, ensuring the structured executability of the commands. Through this refined input data structure design, the problem of generating uncontrollable, unsafe, or non-industrial logic trajectory commands due to the lack of domain knowledge constraints in the language model is effectively avoided, significantly improving the determinism, safety, and logical correctness of robot motion trajectory command generation.

[0048] In one optional embodiment, the operational skill includes modifying an existing I / O skill; obtaining a sequence of motion trajectory instructions based on structured input data and a predetermined language model; and further includes: when the structured input data indicates that the existing I / O skill should be modified, determining a specific operational mode for modifying the existing I / O skill according to the instruction type in the target operational intent, wherein the specific operational mode includes a replacement mode, an expansion mode, or a contraction mode; the replacement mode is used to modify the port number or value of an existing I / O instruction in the trajectory; the expansion mode is used to insert additional port instructions after existing I / O instructions; and the contraction mode is used to delete I / O instructions of a specified port from the trajectory.

[0049] In this embodiment, the operational skills include modifying existing IO skills. This means that in the process of generating motion trajectory instruction sequences based on structured input data and a predetermined language model, not only are conventional point-to-point movements or action executions handled, but the ability to dynamically adjust input and output signals in the industrial field is also specifically incorporated. Specifically, determining the specific operational mode (replacement, expansion, or contraction) based on the instruction type in the target operational intent means that the system parses the semantic features in the user's natural language instructions. For example, when the instruction contains words like "change to" or "set as," it is determined to be a replacement mode; when it contains words like "add" or "insert," it is determined to be an expansion mode; and when it contains words like "remove" or "delete," it is determined to be a contraction mode. This transforms ambiguous modification intentions into definite atomic operation instructions. Specifically, the replacement mode directly targets existing IO instructions in the trajectory, modifying them... The corresponding port number or value can be changed, such as changing the port Y1 that originally controlled the clamping cylinder to Y2, or changing its output signal from normally open to normally closed. The expansion mode inserts new port instructions at the later position of the existing IO instruction sequence. For example, after the material picking action is completed, an additional signal instruction to reset the sensor is inserted to meet the new process requirements. The contraction mode removes the IO instructions of the specified port from the trajectory. For example, the heating signal that was originally used for preheating but is not actually needed is deleted to optimize the cycle time. Through this pattern matching mechanism based on instruction type, the system can accurately distinguish and execute the modification, addition or removal operation of the existing IO signal, avoiding IO control logic errors caused by fuzzy intent recognition when the AI ​​generates instructions. This significantly improves the accuracy, flexibility and reliability of IO control in complex material picking and placing tasks.

[0050] In one optional embodiment, when the real-time data status includes the currently loaded tool name and the suction cup I / O port cache status, a motion trajectory instruction sequence is obtained based on structured input data and using a predetermined language model. The method further includes: when the target operation intent involves suction cup binding or unbinding, first querying the suction cup I / O port cache status; if a specified port number exists in the suction cup I / O port cache status, the specified port number is directly reused to generate the motion trajectory instruction sequence; if no specified port number exists in the suction cup I / O port cache status, query information is injected into the structured input data to ask the user about the port number used by the suction cup.

[0051] In this embodiment, the real-time data status includes the currently loaded tool name and the suction cup I / O port cache status, which refers to a dynamic status table maintained internally by the system. The suction cup I / O port cache status specifically records the physical I / O port number bound to the suction cup in the current or most recent operation. If there is no valid record in the cache, it is considered that no specified port exists. When the target operation intent involves suction cup binding or unbinding, the suction cup I / O port cache status is queried first. This means that after the language model parses the user intent as "bind suction cup" or "unbind suction cup," the control logic prioritizes searching this status table to obtain port information, rather than directly entering the instruction generation stage. If a specified port number exists in the suction cup I / O port cache status, the specified port number is directly reused to generate the motion trajectory instruction sequence. This means that when a historical port number (e.g., port 8) exists in the cache, that port number is directly filled into the instruction template without requiring the user to input it again. By skipping the query step, a complete motion trajectory instruction sequence containing correct IO control parameters is quickly generated. If the specified port number is not present in the suction cup IO port cache, query information is injected into the structured input data to ask the user about the port number used by the suction cup. This means that when the cache is empty or invalid, text such as "Please specify the IO port number bound to the suction cup" is inserted into the structured data stream, initiating a multi-round interaction process to wait for the user's response. Once a clear port number is obtained, the instruction generation continues. Through the above logic, this solution achieves intelligent management of IO port numbers in suction cup operation scenarios. Compared with related technologies where unclear port status leads to frequent manual input or instruction generation failures, this solution improves the response speed and operational efficiency of instruction generation by reusing cached ports, and ensures the accuracy and completeness of IO control parameters in the generated instructions through precise anomaly queries, effectively improving the reliability of robot control.

[0052] In an optional embodiment, the method further includes: after generating a planned sequence containing suction cup binding operations using a predetermined language model, scanning a list of known point names; identifying points whose names contain preset keywords as candidate transition points, wherein the preset keywords include "transition point" or "passing through"; grouping the candidate transition points by name prefix and sorting them by number suffix to generate a recommended transition point list; and returning the recommended transition point list as a prompt to the user terminal so that the user can choose whether to pass through the candidate transition points before confirming execution.

[0053] In this embodiment, after the predetermined language model generates a planned sequence including suction cup binding operations, the known point name list is scanned to obtain the information of registered physical points in the current system from the real-time status. The known point name list refers to a set of multiple points and their tags stored in the real-time data status field and provided by the offline programming software interface. Points containing preset keywords in their names are identified as candidate transition points. Specifically, by matching specific words such as "transition point" or "passing through," points with transit or buffering properties are accurately selected from a massive number of points, thereby excluding irrelevant starting or ending points. Candidate transition points are then categorized by name prefix. Grouping and sorting by numerical suffix to generate a recommended list of transition points involves utilizing the regularity of point naming conventions. Points with the same function or regional attributes (such as those with the same prefix) are first grouped together, and then sorted in ascending or descending order within each group based on numerical value. This eliminates the randomness of point order and forms a logically clear recommended list. Returning the recommended list of transition points to the user terminal as a prompt allows the user to select whether to pass through candidate transition points before confirming execution. This means displaying the organized point list on the operation interface, enabling the user to intuitively review and select the required intermediate points before issuing instructions, achieving a combination of manual intervention and intelligent generation. Through these steps, potential transition points that were originally hidden in the model generation results are made explicit and structured, avoiding the difficulty for users in quickly locating and confirming points when faced with an unordered list. This not only improves the accuracy of material handling trajectory arrangement and user operation efficiency but also enhances the convenience and reliability of human-computer interaction.

[0054] In one optional embodiment, a motion trajectory instruction sequence is obtained based on structured input data and using a predetermined language model, including: obtaining a plan sequence containing step numbers, operation skill names, parameter lists, and step purpose descriptions based on structured input data and using a predetermined language model; verifying whether the parameters of each operation step in the plan sequence are within a preset allowable range, whether the point names and IO port numbers referenced by each operation step exist in the real-time data state, and whether the plan sequence meets safety and logical constraints; and converting the plan sequence into a motion trajectory instruction sequence if the plan sequence passes the verification.

[0055] In this embodiment, a predetermined language model is used to generate a plan sequence containing step numbers, operation skill names, parameter lists, and step purpose descriptions. This aims to transform natural language instructions into a structured, executable intermediate logic layer. The plan sequence is not the final machine code, but rather a draft instruction containing detailed semantic information. Subsequently, a verification step is performed, specifically, the system checks each operation step in the plan sequence to ensure that the parameter values ​​fall within the preset safety limits, that the referenced point names and I / O port numbers actually exist in the robot's current real-time data status (such as the registered point table and available I / O list), and that the overall process is... By satisfying safety and logical constraints such as prohibiting the fabrication of points and ensuring symmetrical loading and unloading patterns, this process is equivalent to conducting a rigorous "compliance review" before execution. Only when all the above checks pass will the planned sequence be converted into the final motion trajectory instruction sequence. If the checks fail, the conversion will be blocked, thereby avoiding risks such as parameter out-of-bounds errors, reference to non-existent points, or violations of industrial safety logic (such as missing safety access points) caused by language model illusions. This ensures that the generated motion trajectory instructions comply with industrial robot control standards in terms of parameter accuracy, resource availability, and logical security, significantly improving the reliability of robot control.

[0056] Step S112: Send the motion trajectory command sequence to the robot.

[0057] In this embodiment, a sequence of motion trajectory instructions is sent to the robot. This step involves transmitting planned or generated instruction data, including parameters such as the robot's motion path, speed, acceleration, and I / O control, from the offline programming software or intermediate control layer to the controller of the physical robot. This data transmission action delivers the motion trajectory instruction sequence from the offline programming environment to the physical execution end, enabling the physical robot to perform corresponding automated operations based on the received instruction sequence, thus completing a closed loop from virtual simulation to physical execution.

[0058] In an optional embodiment, the method further includes: identifying the error type that caused the failure when the planned sequence verification fails, wherein the error type includes at least one of the following: parameters exceeding a preset allowable range, the referenced point name or IO port number not existing in the real-time data state, or violation of security and logic constraints; generating a prompt message containing the error cause and correction suggestions based on the error type; returning the prompt message to the user terminal to guide modification of the target operation instruction or confirmation of the corrected operation parameters; and updating the historical context data to obtain new structured input data upon receiving a correction instruction based on the prompt message, and re-executing the step of generating a new motion trajectory instruction sequence based on the new structured input data using a predetermined language model.

[0059] In this embodiment, when the planned sequence generated by the predetermined language model fails the security or logic check, the system first identifies the specific error type. This error type specifically refers to one or more of the following: parameter values ​​exceeding the robot's safe operation threshold; the called point coordinates or IO port number not being registered or unavailable in the current real-time data state; and the operation steps violating pre-set safety prohibitions (such as prohibiting the fabrication of false points or violating symmetrical loading logic). Subsequently, based on the identified specific error type, a prompt message containing a clear explanation of the error cause and targeted correction suggestions is generated and fed back to the user terminal, aiming to guide the user in making corrections. Natural language commands or confirmation of adjusted operating parameters; upon receiving a correction command input by the user based on prompts, the correction content is integrated into historical context data to construct new structured input data containing the latest correction information, and the predetermined language model is invoked again to generate an updated sequence of motion trajectory commands based on the new data; through this closed-loop feedback mechanism, the root cause of the failure can be accurately located and human intervention can be guided to correct it when planning fails, avoiding direct interruption of the process or dangerous actions caused by AI planning deviations, and significantly improving the reliability of intent recognition, operational safety, and the consistency and efficiency of human-machine interaction in the industrial robot control process.

[0060] In one optional embodiment, generating a sequence of motion trajectory instructions for the robot based on the target operation intention further includes: stopping the generation of the motion trajectory instruction sequence when the predetermined language model detects the absence of parameter information necessary for generating the motion trajectory instruction sequence, and sending a missing information prompt to the user terminal based on historical context data; after receiving a correction instruction from the user regarding the missing information prompt, appending the correction instruction as new historical context data to the historical context data; and restarting the predetermined language model to generate the motion trajectory instruction sequence based on the updated structured input data.

[0061] In this embodiment, when the predetermined language model detects missing necessary parameter information during the generation of the motion trajectory instruction sequence, it immediately stops the current generation process and sends a missing information prompt to the user terminal based on historical context data. Historical context data refers to the dialogue records accumulated during the interaction, containing user instructions and model response states, aiming to maintain the continuity of human-computer interaction. For example, when the model cannot determine the specific grasping force of the robotic arm, it will immediately pause and prompt the user to supplement the parameter. Subsequently, after receiving the user's correction instruction for the prompt, this correction instruction is added to the original historical context data as new historical context data, thereby achieving dynamic completion of missing parameters and updating of the context. Finally, based on the updated structured input data containing the new parameters, the predetermined language model is restarted to regenerate the complete motion trajectory instruction sequence. This mechanism, through a closed-loop process of "interruption-feedback-completion-restart," avoids generation interruptions or erroneous execution caused by missing parameters, significantly improving the continuity and reliability of robot control under complex natural language instructions.

[0062] Through steps S102 to S112, the target operation instruction input via natural language is first received and segmented into characters to obtain the character segmentation result. Next, based on the character segmentation result, the initial operation intent for the robot is determined, and the text complexity score of the target operation instruction is calculated. Then, differential processing is performed based on whether the text complexity score is greater than a preset complexity threshold: if the score is greater than the threshold, it indicates that the instruction is relatively complex; in this case, a predetermined language model is used for deep analysis based on the initial operation intent to obtain the target operation intent, ensuring the accuracy of recognition. If the score is less than or equal to the threshold, it indicates that the instruction is relatively simple and direct; in this case, the initial operation intent is directly determined as the target operation intent, thereby skipping the complex model reasoning process and improving response speed. Finally, based on the determined target operation intent, a sequence of motion trajectory instructions for the robot is generated and sent to the robot for execution. By introducing text complexity scoring as a conditional judgment criterion, this solution realizes a hierarchical processing mechanism for instructions of different complexities, enabling low-complexity instructions to obtain responses quickly and high-complexity instructions to achieve high-precision recognition. This effectively solves the problem of difficulty in balancing the accuracy of intent recognition and response speed in natural language instructions in industrial robot control scenarios, achieving a technical effect that balances control efficiency and reliability.

[0063] Offline programming for industrial robots refers to planning, simulating, and verifying the robot's motion trajectory using dedicated offline programming software in a computer environment separate from the actual robot, before finally sending the program to the physical robot for execution. The traditional process includes: loading robot and tool models, importing the workpiece's 3D model, manually teaching or inputting coordinates to create points, combining these points into a motion trajectory and adding I / O controls (such as suction cup suction / desorption), tool binding and other auxiliary instructions, setting motion parameters (speed, acceleration, motion type), and exporting the program after simulation verification. This process is cumbersome, especially for complex material handling tasks involving multiple points, I / O timing, and tool switching, requiring the manual arrangement of dozens of instructions, resulting in low efficiency and a high risk of errors.

[0064] In recent years, AI dialogue technology, represented by large language models, has developed rapidly. Some solutions have attempted to introduce natural language interaction into the field of industrial robot control, but currently there is no mature dialogue programming solution specifically designed for offline programming software. Robot control solutions in related technologies mainly fall into two categories: one is a robot motion command generation system based on natural language. The typical architecture of this type of system is: user input command → natural language processing module parses the intent → motion planning module generates trajectory → sent to robot controller for execution. Some collaborative robot manufacturers already support simple voice command control. The other type is robot control based on the network application programming interface (API) of offline programming software. Mainstream offline programming software provides an API that allows external programs to remotely manipulate points and trajectory data through standardized commands, but it only supports structured data input and lacks natural language understanding capabilities.

[0065] As can be seen from the above, in the first approach of the related technologies, motion commands are directly generated and sent to the controller for execution, bypassing the offline programming software. This results in the inability to utilize simulation verification capabilities, the inability to visually edit AI-generated paths, and the inability to link with existing point / tool / workpiece data in the software. More seriously, in actual industrial settings, if motion paths directly generated by large models are sent to physical robots for execution without simulation verification, there is an extremely high risk of hardware collisions. Furthermore, due to the lack of singularity point verification and joint limit checks, motor overspeeding or overload may occur, causing equipment damage or even safety accidents. In addition, general natural language processing solutions struggle to distinguish fine-grained operational intentions in offline programming (e.g., "change DO0 to DO1" could mean modifying the IO port number or modifying the point coordinates), and lack the ability to decompose deterministic tasks for material handling scenarios. In the second approach of the related technologies, the interface requires the caller to have programming skills and to construct a data structure containing more than ten fields such as motion type, point name, and speed; it only provides a data channel and lacks intelligent capabilities such as intent understanding and conflict detection. Furthermore, related technologies lack fine-grained intent conflict suppression mechanisms when processing natural language commands in offline programming, which can easily lead to command misinterpretation. Simultaneously, directly generating robot motion paths from large models lacks deterministic business knowledge constraints specific to the industrial domain, resulting in trajectories that lack controllability and safety. Therefore, the primary technical problem addressed in this embodiment is: how to deeply integrate natural language dialogue capabilities with the proprietary data system of industrial robot offline programming software, enabling users to complete the entire offline programming process from workpiece import to trajectory arrangement through everyday dialogue, while ensuring that AI-generated trajectory data must undergo simulation verification and motion calibration by the offline programming software before execution, thus eliminating the physical safety hazards caused by the uncertainty of large model paths at the architectural level.

[0066] To address the aforementioned problems, based on the above embodiments and optional embodiments, this invention proposes an optional implementation method for robot control based on natural language dialogue. This embodiment mainly includes three stages, wherein:

[0067] Phase 1: Environment Setup and Workpiece Import Figure 2 This is a first schematic diagram of an optional operation interface according to an embodiment of the present invention. The user starts the AI ​​dialogue assistant panel embedded in the operation interface of the offline programming software, loads the robot entity and end effector (such as suction cup, gripper, etc.) of a specified model through dialogue, and triggers a 3D model generation algorithm or a third-party 3D modeling service to automatically generate a 3D digital model of the workpiece through dialogue. After generation, it is automatically imported into the offline programming software scene, and the size of the workpiece can be adjusted through dialogue.

[0068] The second stage – manual teaching points: The user adjusts the relative position of the workpiece and the robot in the offline programming software, creates key points (picking point, unloading point, transition point, origin, etc.) through teaching, and sets label information for the points.

[0069] The third stage – conversational trajectory programming and execution: Users issue operation commands through dialogue (such as "get current position", "change speed to 100", "move workpiece from pick-up point to drop-off point", "generate trajectory", "run trajectory"). The system automatically recognizes the intent, executes the operation, and synchronizes the results to the offline programming software. Figure 3 This is a second schematic diagram of an optional operation interface according to an embodiment of the present invention. After the user previews and verifies, the generated motion trajectory command sequence is sent to the physical robot.

[0070] This embodiment establishes an AI dialogue intermediary layer between the user and the offline programming software, which consists of four core technology layers:

[0071] The hybrid intent detection layer involves the following steps: conjunction segmentation → keyword matching → conflict suppression → AI model fallback → secondary reclassification;

[0072] The action routing and execution layer involves the following operation process: simple instructions are executed directly / compound instructions are generated by the AI ​​planner into skill sequences → user confirmation → sequential execution;

[0073] The label arrangement and location management system involves the following operation process: multi-dimensional label matching → automatic calculation of proximity points → automatic generation of material pick-up and drop-off sequence → three-level degradation;

[0074] The offline programming software interface layer involves the following operation process: proxy forwarding → bidirectional state synchronization → two-step push protocol → automatic import pipeline of 3D models. The corresponding code is shown below.

[0075] graph TB

[0076] A [User's natural language input] --> B

[0077] subgraph Layer1["① Hybrid Intent Detection Layer"]

[0078] B [Conjunction segmentation] --> C [Keyword / Regular expression matching]

[0079] C --> D [Conflict Suppression]

[0080] D --> E {Rule match successful}

[0081] E -->|Yes| F [Output Intent]

[0082] E -->|No| G[AI model fallback classification]

[0083] G --> H [Complexity Scoring Secondary Reclassification]

[0084] end

[0085] F --> I

[0086] H --> I

[0087] subgraph Layer2["② Action Routing and Execution Layer"]

[0088] I{Instruction Type Route} -->|Simple Instruction| J[Direct Execution Channel]

[0089] I -->|Compound / Trajectory Construction| K[AI Planning Channel]

[0090] K --> L [Dynamically assemble structured input data]

[0091] L --> M [AI planner generates skill sequences]

[0092] M --> N [User Confirmation / Automatic Execution]

[0093] end

[0094] J --> O

[0095] N --> O

[0096] Subgraph Layer 3 ["③ Label Arrangement and Location Management Layer"]

[0097] O [Multidimensional Label Matching] --> P [Automatic Proximity Point Calculation]

[0098] P --> Q [Automatic generation of material pick-up and drop-off sequence]

[0099] end

[0100] Q --> R

[0101] Subgraph Layer4["④ Offline Programming Software Interface Layer"]

[0102] R [Proxy Forwarding] --> S [Two-way State Synchronization]

[0103] S --> T [Two-step push protocol]

[0104] T --> U [Offline Programming Software]

[0105] end

[0106] U --> V [Simulation verification and point data feedback].

[0107] For the technical layer ①, namely the hybrid intent detection layer: it is used to implement the hybrid intent detection mechanism, namely the intent recognition method for offline programming, which specifically includes: a two-stage detection of user natural language instructions (i.e., target operation instructions input by the user in natural language form) using "rule priority and AI model as a backup".

[0108] The first stage executes rule matching (response time < 10 milliseconds), specifically including: Conjunction segmentation: Chinese conjunctions (and, also, and, then, next, again, etc.) and punctuation are used to split the text into ordered clauses, such as "Move to point A, bind the tool, then move to point B" being split into 3 clauses for separate identification. Keyword / regular expression matching: Pattern matching for over 20 operation categories is performed on each clause, covering all operation types in offline programming. Some examples are shown in Table 1 above. Conflict suppression mechanism: A priority topology chain based on offline programming business logic is constructed. When the same text clause triggers multi-feature intent conflicts, the lower-priority intent features are masked or filtered out according to the topology chain. Specifically, the topology relationship is: "Suction cup blowing" suppresses "Binding the tool" (to avoid "blowing away" triggering suction binding simultaneously); "Set robot / Load tool / Load model" suppresses "Modify point". Furthermore, expressions such as "Add point", "Insert", and "Add" are routed to "Add motion" (trajectory operation) instead of "Modify point" to avoid confusion between point list management and trajectory editing.

[0109] In the second stage, a fallback strategy based on a large language model (i.e., an AI model, or a predefined language model) is adopted (only invoked when rule matching fails): the large language model is called for intent classification, using low-randomness parameters and structured input constraints, limiting the output to a predefined intent enumeration (16 intents + unrecognized intents) and key parameter extraction results, with a 15-second timeout to avoid blocking. A second fallback: if the rule has detected an action but the text complexity score is ≥2 (based on the frequency of keywords such as "then," "again," "next," "take," "get," "put," "clamp," "suck," etc., a text complexity score ≥2 is considered a complex task), the AI ​​model is invoked to reclassify the action, even if the rule already has a result. This may upgrade a simple action to a multi-step planning task. For example, "go to the pick-up point to get the workpiece, pass through the transition point, and put the workpiece at the drop-off point," the rule may only recognize "adding movement," but with a complexity score of 3, it will be upgraded to a complete pick-up and drop-off planning task upon triggering. These three levels of detection ensure the accuracy and response speed of intent recognition in professional offline programming scenarios.

[0110] For the technical layer ②, namely the action routing and execution layer: this layer is used to implement dual-channel action routing and skill planning, namely a skill planning method based on the knowledge constraints of a large language model. Specifically, it includes: a direct execution channel (simple instructions): query operations (getting points, getting trajectories, getting tool libraries, etc.) and parameter modification operations (modifying point coordinates, adjusting model dimensions, etc.) use the direct execution channel. For example, for a point modification operation, the following process is executed: pre-refresh (automatically getting the latest point data) → AI analyzes the modification intent → triple verification (prohibiting the creation / deletion / renaming of points) → field-level merging (only updating the modified fields) → automatic push back to the offline programming software. The system does not support modifying the point list itself through dialogue (adding / deleting / renaming points); such operations will be intercepted and the user will be guided to complete them in the offline programming software. For point insertion within the trajectory: Supports inserting new moving points at specified locations within the trajectory. Users can specify anchor points (e.g., "insert p3 after p1") and insertion directions (forward / backward). The system precisely inserts the point at the boundaries of non-moving elements (IO / wait / bind) near the anchor point and automatically inherits the velocity and acceleration parameters of adjacent moving points. For loading / switching tools: Calls the loading tool interface of the offline programming software to synchronously update the tool configuration.

[0111] For other operation types (compound operations), an AI planning channel is used, specifically including: Planner routing extension: not only are compound task modes (such as "get XX to XX" or "move XX to XX") handled by the AI ​​planner, but all operations involving trajectory construction (adding motion, binding tools, adding IO control, suction cup blowing, etc.) are also uniformly routed to the AI ​​planning channel, enabling the AI ​​planner (i.e., the large language model) to perform more intelligent planning using dialogue history context data. Only query-type operations (getting points, getting trajectories, etc.) still use the direct execution channel.

[0112] Structured Input Data Dynamic Assembly Mechanism: This embodiment does not simply pass user instructions directly to the large language model for processing. Instead, the system dynamically assembles and generates structured text input data containing multi-dimensional safety and logical constraints based on preset rules, the robot's real-time data status, and context data. This injects proprietary knowledge and constraints from the offline programming domain into the model's input, ensuring that the model's output is confined within a compliant and safe operating space. The specific construction process is as follows:

[0113] The following six pieces of information are combined and merged into a complete piece of structured input data according to a fixed template. This structured input data includes:

[0114] Role definition field (fixed template): Defines the role of the model as "offline programming trajectory planning assistant", whose task is to generate an ordered execution plan (i.e. the robot's task execution plan) based on the user's compound instructions.

[0115] The skill catalog constraint field (fixed template) lists all atomic operation skills (11 in total: Get Point, Get Tool Configuration, Add Motion, Trigger IO Control, Wait for Specified Duration, Wait for IO to Reach Target Value, Suction Cup Binding, Suction Cup Inflation, Push Trajectory, Adjust Part Size, Modify Existing IO) defined in the system in declarative format, one by one in structured text. Each skill includes: name, function description, parameter definition table (parameter name, data type, required, default value, allowed value range, upper limit of value), and return value structure. The Modify Existing IO skill supports three modes: Replace (modify port number or value), Expand (insert additional port instructions after existing IO instructions), and Shrink (delete IO instructions for a specified port from the trajectory).

[0116] Real-time status field (dynamically generated): The system extracts real-time data from the session context and populates it into the template, including: the network interface address of the offline programming software, the list of currently known point names (obtained in real-time from the offline software), the name of the currently loaded tool, the number of current trajectory instructions, and the current suction cup I / O port cache status. This field ensures that the point names and tool names referenced during model planning come from real data, rather than being fabricated out of thin air.

[0117] Dialogue history context field (dynamically generated): Injects summaries of user instructions and system responses from previous dialogue rounds into the input data, enabling the planner to have multi-round context awareness capabilities, supporting step-by-step instruction delivery (such as saying "move to the pick-up point", then "inhale", and finally "move to the put-down point") and question-and-answer interaction.

[0118] Safety and logical constraint fields (fixed template): Safety and logical constraints are encoded in a numbered format. Core constraints include: prohibiting the fabrication of point names and IO port numbers; cross-step data reference format; plans must end with the push trajectory; symmetrical loading and unloading mode (loading = suction at pick-up point → degassing at discharge point, unloading = suction at discharge point → degassing via reverse transition point → degassing at pick-up point); automatic insertion of above points (automatically searching and inserting "XX above point" as a safe approach / retreat point before and after each pick-up / discharge target point); automatic insertion of transition points (automatically detecting and inserting matching transition points in the movement path after the suction cup is bound, loading in forward order, unloading in reverse order); single-step instruction processing (when the user gives simple instructions in steps, only the corresponding steps are generated, and it is not automatically completed into a complete pick-up / discharge process); optional suction cup blowing (only added when explicitly requested by the user, not as a necessary step for pick-up / discharge).

[0119] Output format constraints and user instruction fields: The model output is limited to a JSON array format, with each element containing a step number, skill name, parameter list, and step purpose description; the user's original instruction text is also included. The specific code representation of the structured input data implementation is as follows.

[0120] graph TB

[0121] A [User sends compound instructions] --> B [Dynamically assembles structured input data]

[0122] B --> C

[0123] B --> D

[0124] B --> E

[0125] B --> F

[0126] B --> G

[0127] B --> H

[0128] subgraph FixedFields["Fixed template fields (remain unchanged each time)"]

[0129] C["Field 1: Role Definition"]

[0130] D["Field 2: Skill Catalog Constraints (11 Skills × Parameter Table)"]

[0131] G["Field 5: Security and Logical Constraints"]

[0132] end

[0133] subgraph DynamicFields["Dynamic data fields (updated in real time)"]

[0134] E["Field 3: Real-time Status (Location / Tool / IO Port)"]

[0135] F["Field 4: Dialogue History Context"]

[0136] end

[0137] subgraph InputField["current input field"]

[0138] H["Field 6: Output format constraints + User-defined instructions"]

[0139] end

[0140] C --> I [Concatenate and merge into complete structured input data]

[0141] D --> I

[0142] E --> I

[0143] F --> I

[0144] G --> I

[0145] H --> I

[0146] I --> J [Input to AI Planner].

[0147] The above concatenation is dynamically executed each time a user sends a compound command: the role definition field, skill catalog constraint field, and safety and logic constraint field are relatively fixed templates; the real-time status field and dialogue history context field are updated in real time with the dialogue status; and the safety and logic constraint field is filled with the current user command. The key to this mechanism is that it systematically constructs input data to constrain the output space of the large model, making it a controlled planning engine rather than relying on the model's general reasoning capabilities. In particular, the rules in the safety and logic constraint fields, such as the loading / unloading symmetry pattern, automatic insertion of top points, and automatic insertion of transition points, are proprietary business knowledge in the field of industrial robot material handling, and are encoded into structured constraints that the model must follow.

[0148] Two-stage execution and automatic execution: After the model generates the plan, there are two execution paths: one is for plans containing safety risks, which are first displayed to the user in natural language for confirmation (e.g., "Step 1: Move to the picking point → Step 2: Suction cup suction →..."), and then executed sequentially after confirmation; the other is for regular trajectory construction operations, where the plan is executed automatically without user confirmation, improving interaction efficiency. Before each step is executed, parameter verification (e.g., speed does not exceed the upper limit) and safety checks (intercepting dangerous keyword instructions) are performed.

[0149] Multi-round planning interaction: When the planner detects that necessary information is missing (such as the IO port number is not specified), it does not generate a plan but instead asks the user a question (such as "Please specify the DO port number used by the suction cup"). After the user answers in the next round, the system automatically restores the previous planning context and continues to generate a complete plan, realizing progressive planning driven by multi-round dialogue.

[0150] Automatic transition point suggestion: After the suction cup binding operation is performed, the system automatically scans the current point list, identifies points whose names contain keywords such as "transition point" or "pass through", and recommends them to the user after grouping by name prefix and sorting by number suffix (e.g., "Available transition points detected: transition point 1, transition point 2, transition point 3. If you need to pass through, please say 'pass through all transition points'"), reducing the threshold for users to discover and use transition points.

[0151] IO port context reuse: After the user specifies the port number, the binding and unbinding operations of the same task are automatically reused and inherited across dialogue rounds; if no port number is specified and there is no cache, the user is actively asked and the default value is not used.

[0152] In the technical layer ②, a declarative skill catalog and a multi-feature field splicing and fusion mechanism of structured input data are used to constrain the output space of the AI ​​model, making the planning results controllable and auditable; a two-stage confirmation mechanism ensures the safety of industrial operations.

[0153] For technology layer ③, namely the label arrangement and point management layer, it is used to realize the automatic arrangement of material handling trajectories based on the label system, that is, a deterministic trajectory arrangement scheme based on multi-dimensional labels. The core problem solved by this technology layer is: when users give natural language commands (such as "bring the workpiece to the placement point"), how can the system accurately and deterministically select key points (picking points, placement points, approach points, transition points, etc.) from a large number of points and generate a complete trajectory sequence? Specifically, by setting up a multi-dimensional point label system and adopting a three-layer degradation matching strategy, the robustness and accuracy of trajectory generation are ensured.

[0154] First, each key point is assigned a four-dimensional structured label to represent its semantic attributes and spatial relationships. These four-dimensional labels include: Role Label: Identifying the functional role of the point in the process flow, such as: pick-up point, unload point, transition point, return point, approach point, etc.; Associated Object Label: Identifying the specific workpiece or equipment object associated with the point, such as: workpiece A, pallet B, etc., aiming to resolve ambiguity in multi-workpiece scenarios; Applicable Tool Label: Identifying the type of end effector required to perform the operation at this point, such as: suction cup, gripper, welding gun, etc.; Parent Tag: Identifying the hierarchy or dependency relationship between points. For example, the parent label of "pick-up approach point" points to "pick-up point," and the parent label of "unload approach point" points to "unload point," thus establishing a logical binding between the approach point and the main point. Furthermore, for "transition points," the label also includes a sequence field to determine the order in which multiple transition points are traversed in the path. A corresponding code example is shown below.

[0155] graph TB

[0156] A[Point Label Structure (4D Label)] --> B["Material Pickup Point"]

[0157] A --> C ["Discharge Point"]

[0158] A --> D ["Transition point"]

[0159] A --> E["origin"]

[0160] B --- B1 ["Role: Material Collector"]

[0161] B --- B2["Object: Workpiece A"]

[0162] B --- B3 ["Tool: Suction Cup"]

[0163] B --- B4["Parent: Material proximity point"]

[0164] C --- C1["Role: Leaking Information"]

[0165] C --- C2["Object: Workpiece A"]

[0166] C --- C3["Tool: Suction Cup"]

[0167] C --- C4["Parent: Approach point for material release"]

[0168] D --- D1["Role: Transition"]

[0169] D --- D2["Sequence: 1"]

[0170] E --- E1["Character: Return to Origin"].

[0171] The deterministic trajectory arrangement scheme based on multi-dimensional labels involves the following steps: Deterministic point resolution: The pick-up point, release point, origin, approach point, and transition point are determined by multi-dimensional label matching. The entire process is deterministic and does not rely on AI models. Automatic generation of standard pick-up and release sequences: A complete trajectory is generated according to a fixed pattern: Pick-up approach point → Pick-up point → Wait → IO setting → Mechanical binding → Wait → [Transition point] → Release approach point → Release point → Wait → IO reset → Mechanical unbinding → Wait → [Origin] → Push trajectory. Three-level degradation: Complete label → Exact matching; Partially missing → Label + semantic mixture; No label → Semantic matching + AI inference and prompt for supplementation. Specifically, when a user issues a natural language command containing the intention to select points, it does not directly rely on an AI model for fuzzy matching, but uses deterministic logic to resolve points in three levels. If a higher-level matching fails or the confidence level is insufficient, it automatically degrades to the next level for processing:

[0172] The first layer is: complete label precise matching (high-precision deterministic matching). The corresponding trigger condition is: the system detects that the target point has complete four-dimensional labels (role, associated object, applicable tool, and parent association all exist and are valid). The execution logic is: the system parses entity keywords in the user's natural language command (such as "workpiece A", "suction cup", "material picking"); in the point database, it matches all the above label dimensions simultaneously. For example, if the user command is "use the suction cup to pick up workpiece A", the system will search for all points that meet the corresponding conditions; if a unique point is matched, it is directly confirmed as the target point; if multiple points are matched, they are sorted and selected according to the priority weight in the labels or the user's historical habits. This layer achieves 100% predictable and reproducible point selection, is entirely based on rule logic, eliminates the uncertainty of the AI ​​model, and ensures the absolute accuracy of the selection of core process points.

[0173] The second layer employs a hybrid tag and semantic matching approach (robust matching even with partial missing tags). The triggering conditions are: partial missing tags for the target location (e.g., only character and associated object tags are available, but applicable tool tags or parent-related tags are missing), or the first layer's exact matching fails to find a unique result. The execution logic is as follows: Tag filtering: Initial screening is performed using existing tag dimensions. For example, if the "applicable tool" tag is missing, only the corresponding condition is used to filter the candidate location set. Semantic enhancement: Natural language processing technology is introduced into the candidate location set to calculate the semantic similarity between the user command text and the location name, notes, or tag description. Combining the tag matching score and semantic similarity score, a weighted algorithm determines the location with the highest confidence. This layer improves the system's adaptability to incomplete tag data, using semantic understanding to compensate for the deficiencies of structured data while retaining the constraints of tag matching, avoiding widespread misjudgments that might result from pure semantic matching.

[0174] The third layer consists of semantic matching + AI inference and supplementary prompts (a fallback for unlabeled or low-confidence points). The triggering conditions are: the target point is completely unlabeled, or the confidence level after the second-layer matching is lower than a preset safety threshold (e.g., too many candidate points or extremely low semantic similarity). The execution logic is as follows: AI Inference: The system calls a Large Language Model (LLM) to infer the user's command, combining the current dialogue context, workpiece model information, and process common sense. For example, if the user says "Go to the first location to get something," and there is only one unlabeled point in the scene, the LLM can infer that the point is a material pick-up point. Risk Warning and Interaction: The system does not directly execute the inference result but generates a prompt message for the user. The prompt content is, for example: "Point 'Point_001' has been detected as a possible material pick-up point, but it lacks the 'material pick-up' role label and the 'workpiece' association label. Do you confirm using this point? If optimization is needed, please add a label to this point." User Confirmation: The system waits for user confirmation or correction. If the user confirms, a temporary mapping relationship is established for the point; if the user corrects, the user is guided to enter the point editing mode to supplement the label. As the last line of defense, it ensures the system's availability in extreme data loss situations. At the same time, through the "prompt-confirmation" mechanism, it returns the final decision-making power to the user, ensuring the safety and controllability of industrial field operations and avoiding the issuance of erroneous instructions caused by AI "illusions".

[0175] After deterministically identifying each key point using the aforementioned three-layer degradation strategy, a complete material handling trajectory sequence is automatically generated based on a preset standard process template. Taking a typical "suction cup handling" task as an example, the generated trajectory sequence follows a fixed pattern: For the material handling stage, the following operations are performed: move to the material handling proximity point (safe height); move to the material handling point; execute the waiting instruction (to ensure stable positioning); execute the IO set instruction (e.g., suction cup vacuum opening); execute the mechanical binding instruction (if any); execute the waiting instruction (to ensure firm adhesion); transition point detection: if there are associated transition points, automatically insert them into the transition point sequence (sorted by the order field). For the material handling stage, the following operations are performed: move to the material handling proximity point (safe height); move to the material handling point; execute the waiting instruction; execute the IO reset instruction (e.g., suction cup vacuum closing / blowing); execute the mechanical unbinding instruction (if any); execute the waiting instruction (to ensure complete release). For the end stage, the following operations are performed: move back to the origin point (or the starting point of the next task); push the trajectory: push the generated complete trajectory data to offline programming software for simulation verification. Through the above-mentioned "three-layer degradation" strategy, the solution in this embodiment not only ensures industrial-grade determinism when the data is complete, but also takes into account flexibility and security when the data is missing, thus achieving efficient and reliable conversion of natural language commands into robot trajectories.

[0176] For technology layer ④, namely the offline programming software interface layer, it is used to realize embedded integration and bidirectional state synchronization, that is, the integration method of embedding the conversational AI assistant into the offline programming software. The proxy forwarding architecture is as follows: the network interface of the offline programming software does not support the browser's cross-domain resource sharing strategy. In this embodiment, a proxy layer is set up in the backend to forward requests and is responsible for automatic retries (exponential backoff, up to 3 times) when there is a network error. The corresponding code is shown below.

[0177] graph TB

[0178] subgraph Frontend["browser frontend"]

[0179] A [Dialogue Assistant Interface]

[0180] end

[0181] subgraph Backend["Backend Gateway Service"]

[0182] B [Intent Detection and Routing]

[0183] C[Agent service]

[0184] end

[0185] subgraph ProxyLayer["proxy forwarding layer"]

[0186] D [Request forwarding (resolves cross-origin issues)]

[0187] E[Index retreat and retry]

[0188] end

[0189] F [Offline Programming Software]

[0190] A -->|Carries 7-dimensional session state| B

[0191] B --> C

[0192] C --> D

[0193] D --> F

[0194] F -->|Location / Trajectory Data| D

[0195] D --> C

[0196] C -->|Updated Status| A

[0197] A -->|Incremental Merge| A

[0198] F -.->|Simulation Verification Feedback| A.

[0199] The integration method of embedding the conversational AI assistant into offline programming software specifically involves the following steps: Two-way state synchronization: The front-end maintains the session state (7 dimensions including point list, trajectory sequence, trajectory name, plan to be confirmed, construction confirmation status, part name, and operation options), uploading the entire state with each request. The back-end returns the updated state, and the front-end incrementally merges the updates. Construction confirmation mechanism: When a new trajectory already exists, it is not directly overwritten when generating a new trajectory. Instead, "overwrite / append / cancel" operation buttons are embedded in the dialogue for the user to choose from. Figure 4 This is a third schematic diagram of an optional operation interface according to an embodiment of the present invention. Selecting "Overwrite" clears the trajectory but retains the point list (for reuse in subsequent planning); selecting "Append" retains the existing trajectory and continues adding. Automatic import pipeline for 3D models: After the dialog triggers the generation of the 3D model, the front end periodically polls the generation status and displays the progress. Upon successful completion, the 3D digital model file is automatically loaded into the offline programming software scene.

[0200] This embodiment of the method can achieve at least one of the following effects: 1) It realizes a complete data closed loop from natural language to offline programming software, eliminating the safety hazards of AI path uncertainty from the architecture level: Unlike related technical solutions that directly send AI-generated motion commands to the robot controller for execution (which poses serious safety risks such as hardware collisions and motor overspeed), the output target of this solution is the trajectory data structure inside the offline programming software. All AI-generated trajectories must be simulated, collided, and calibrated by the offline software before they can be sent to the physical robot for execution, thereby completely solving the physical safety hazards caused by the uncertainty of the large model generation path at the system architecture level. Users can complete the entire offline programming process from loading the robot, importing the workpiece, arranging the trajectory to running verification in the same dialog interface. 2) Three-level intent detection balances speed and coverage: rule matching (less than 10 milliseconds) handles most common commands, AI model fallback (less than 15 seconds) handles long-tail expressions, and the second fallback prevents misjudgment of complex tasks. Conflict suppression rules specifically address keyword ambiguity issues unique to offline programming (e.g., does "changing DO0 to DO1" mean modifying the IO port or the point coordinates?), a domain-specific problem that general natural language processing solutions cannot solve. 3) The tagging system enables deterministic arrangement of material pick-up and drop-off trajectories: Pick-up points, drop-off points, approach points, and transition points are selected through deterministic matching of multi-dimensional tags. With complete tags, the trajectory sequence is 100% predictable and reproducible. The automatically generated trajectory can be safely executed in actual workstations. 4) Structured input data constrains the AI ​​model output space: A six-field concatenation and fusion mechanism dynamically assembles input data (role definition + skill catalog + real-time status + dialogue history + security logic constraints + output format), systematically encoding offline programming domain knowledge into the model's input constraints. The constraints cover business knowledge specific to areas such as symmetrical loading / unloading modes, automatic insertion of upper / transition points, single-step instruction processing, and three modes of IO modification. Combined with parameter upper limits / enumeration constraints in the declarative skill catalog, two-stage user confirmation, step-by-step parameter verification, and security keyword interception, a multi-layered security protection is formed to ensure that AI planning results are controllable and auditable. 5) Embedded integration reduces deployment complexity: proxy forwarding solves cross-domain limitations, bidirectional state synchronization ensures data consistency across multiple rounds of dialogue, a confirmation mechanism prevents accidental overwriting (retaining the point list during overwriting), trajectory integrity checks help users discover missing steps, and automatic 3D model import achieves end-to-end automation.

[0201] It should be noted that some operations in this embodiment can also be replaced by the following alternatives: 1) Intent detection: can be replaced by a pure AI model solution (sacrificing speed) or a pure rule solution (sacrificing coverage), and keyword matching can be replaced by a machine learning text classifier. 2) Tag system: can be replaced by coordinate-based geometric matching or pure semantic matching, which has lower reliability. Tag dimensions can be expanded (e.g., adding grab direction, maximum load, etc.). 3) Proximity point calculation: the offset calculation interface can be replaced by the dialogue assistant implementing forward kinematics solution on its own (increasing coupling), or requiring the user to only use Cartesian coordinate points (limiting flexibility). 4) Proxy forwarding: can be replaced by configuring cross-domain response headers on the offline software side or inter-process communication of desktop applications, each with its own limitations. 5) State synchronization: can be replaced by real-time bidirectional communication using WebSockets, which has lower latency but increased complexity. 6) AI planning: can be replaced by hard-coded rule templates (high determinism but low flexibility) or reinforcement learning solutions.

[0202] This embodiment also provides a robot control device based on natural language dialogue, which is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the terms "module" and "device" can refer to a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0203] According to embodiments of the present invention, an apparatus embodiment for implementing the above-described robot control method based on natural language dialogue is also provided. Figure 5 This is a schematic diagram of the structure of a robot control device based on natural language dialogue according to an embodiment of the present invention, as shown below. Figure 5 As shown, the above-mentioned robot control device based on natural language dialogue includes: a character segmentation module 500, a complexity scoring module 502, a first operation intent determination module 504, a second operation intent determination module 506, an instruction sequence generation module 508, and an instruction sending module 510, wherein:

[0204] The character segmentation module 500 is used to segment the target operation instructions of the robot into characters and obtain the character segmentation results. The target operation instructions are instructions input through natural language.

[0205] The complexity scoring module 502 is connected to the character segmentation module 500 and is used to determine the text complexity score of the robot's initial operation intention and the target operation instruction based on the character segmentation results.

[0206] The first operation intent determination module 504 is connected to the complexity scoring module 502. It is used to obtain the first target operation intent based on the initial operation intent and using a predetermined language model when the text complexity score is greater than the preset complexity threshold.

[0207] The second operation intent determination module 506 is connected to the first operation intent determination module 504 and is used to determine the initial operation intent as the second target operation intent when the text complexity score is less than or equal to a preset complexity threshold.

[0208] The instruction sequence generation module 508 is connected to the second operation intention determination module 506 and is used to generate a sequence of motion trajectory instructions for the robot based on the first target operation intention or the second target operation intention.

[0209] The instruction sending module 510 is connected to the instruction sequence generation module 508 and is used to send the motion trajectory instruction sequence to the robot.

[0210] It should be noted that the above modules can be implemented by software or hardware. For example, for the latter, it can be implemented in the following ways: the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0211] It should be noted that the character segmentation module 500, complexity scoring module 502, first operation intent determination module 504, second operation intent determination module 506, instruction sequence generation module 508, and instruction sending module 510 correspond to steps S102 to S112 in the embodiments. The instances and application scenarios implemented by the above modules and corresponding steps are the same, but are not limited to the content disclosed in the above embodiments. It should be noted that the above modules, as part of the device, can run in a computer terminal.

[0212] It should be noted that the optional or preferred implementation methods of this embodiment can be found in the relevant descriptions in the embodiments, and will not be repeated here.

[0213] The aforementioned robot control device based on natural language dialogue may also include a processor and a memory. The character segmentation module 500, complexity scoring module 502, first operation intention determination module 504, second operation intention determination module 506, instruction sequence generation module 508, instruction sending module 510, etc., are all stored in the memory as program modules, and the processor executes the aforementioned program modules stored in the memory to realize the corresponding functions.

[0214] The processor contains a core that retrieves the corresponding program modules from memory. One or more cores may be configured. Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory includes at least one memory chip.

[0215] According to an embodiment of this application, an embodiment of a non-volatile storage medium is also provided. Optionally, in this embodiment, the non-volatile storage medium includes a stored program, wherein, when the program runs, it controls the device where the non-volatile storage medium is located to execute any of the aforementioned robot control methods based on natural language dialogue.

[0216] Optionally, in this embodiment, the non-volatile storage medium may be located in any computer terminal in a group of computer terminals in a computer network, or in any mobile terminal in a group of mobile terminals, and the non-volatile storage medium includes stored programs.

[0217] Optionally, a program that controls the device containing the non-volatile storage medium to execute any of the above-mentioned robot control method steps based on natural language dialogue during program execution.

[0218] According to an embodiment of this application, an embodiment of a processor is also provided. Optionally, in this embodiment, the processor is used to run a program, wherein the program executes any of the above-described robot control methods based on natural language dialogue.

[0219] According to an embodiment of this application, an embodiment of a computer program product is also provided, which, when executed on a data processing device, is adapted to execute a program that initializes the robot control method steps based on natural language dialogue, which include any of the above-described steps.

[0220] This invention provides an electronic device, which includes a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps of any of the above-described robot control methods based on natural language dialogue.

[0221] The order of the above embodiments of the present invention is merely for description and does not represent the superiority or inferiority of the embodiments.

[0222] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0223] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of modules described above can be a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between modules, and may be electrical or other forms.

[0224] The modules described above as separate components may or may not be physically separate. Similarly, the components shown as modules may or may not be physical modules; they may be located in one place or distributed across multiple modules. Some or all of the modules can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0225] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0226] If the aforementioned integrated modules are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable non-volatile storage medium. Based on this understanding, the technical solution of this invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a non-volatile storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned non-volatile storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0227] The above are merely preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A robot control method based on natural language dialogue, characterized in that, include: The target operation instruction of the robot is segmented into characters to obtain the character segmentation result, wherein the target operation instruction is an instruction input through natural language; Based on the character segmentation results, the initial operational intent for the robot and the text complexity score of the target operational instruction are determined. If the text complexity score is greater than a preset complexity threshold, the target operation intent is obtained based on the initial operation intent and using a predetermined language model. If the text complexity score is less than or equal to the preset complexity threshold, the initial operation intention is determined as the target operation intention; Based on the target operation intention, a sequence of motion trajectory instructions for the robot is generated; The sequence of motion trajectory instructions is sent to the robot.

2. The method according to claim 1, characterized in that, Determining the initial operational intent for the robot based on the character segmentation result includes: Based on the character segmentation results, the original operation intent is obtained; When there are multiple original operation intentions, the multiple original operation intentions are optimized based on the priority topology chain to obtain the initial operation intention. The priority topology chain predefines the priority dependency relationship between different categories of operation intentions. The optimization process includes suppressing or eliminating low-priority original operation intentions according to the priority topology chain. If there is only one original operation intention, then the original operation intention shall be used as the initial operation intention.

3. The method according to claim 1, characterized in that, The step of generating the robot's motion trajectory instruction sequence based on the target operation intention includes: Based on the target operation intention, determine the robot's motion trajectory generation strategy; Based on the motion trajectory generation strategy, the motion trajectory instruction sequence is obtained.

4. The method according to claim 3, characterized in that, The step of determining the robot's motion trajectory generation strategy based on the target operation intention includes: Determine whether the target operation intent belongs to a preset operation type, wherein the preset operation type refers to a query operation or a parameter modification operation; If the target operation intention belongs to the preset operation type, the motion trajectory generation strategy is determined to be the first strategy, wherein the first strategy is used to indicate the direct generation of the motion trajectory instruction sequence; If the target operation intention does not belong to the preset operation type, the motion trajectory generation strategy is determined to be the second strategy, wherein the second strategy is used to instruct the invocation of the predetermined language model to generate the motion trajectory instruction sequence.

5. The method according to claim 3, characterized in that, The process of obtaining the motion trajectory instruction sequence based on the motion trajectory generation strategy includes: When the motion trajectory generation strategy is the first strategy, the motion trajectory instruction sequence is obtained in the following manner: Obtain the real-time data status of the robot; Based on the real-time data status and the target operation intention, perform corresponding query operations or parameter modification operations on the robot's original control data to obtain the target control data; The target control data is converted into the motion trajectory instruction sequence.

6. The method according to claim 3, characterized in that, The process of obtaining the motion trajectory instruction sequence based on the motion trajectory generation strategy includes: When the motion trajectory generation strategy is the second strategy, the motion trajectory instruction sequence is obtained in the following manner: Based on preset rules, the robot's real-time data status, and historical context data, structured input data is generated. The preset rules are used to instruct the encoding of proprietary business knowledge in the offline programming field into safety and logical constraints. The safety and logical constraints include at least one of the following: prohibiting the fabrication of point names and IO port numbers, symmetrical loading and unloading patterns, automatic insertion rules for upper points, and automatic insertion rules for transition points. The historical context data includes historical operation instructions prior to the target operation instruction and corresponding system response summaries. Based on the structured input data, the predetermined language model is used to obtain the motion trajectory instruction sequence.

7. The method according to claim 6, characterized in that, The structured input data includes role definition fields, skill catalog constraint fields, real-time status fields, dialogue history context fields, security and logic constraint fields, and output format constraint and user instruction fields. The role definition field is used to define the role of the predefined language model as an offline programming trajectory planning assistant; The skill catalog constraint field is used to list the name, function description, parameter list and return value structure of the operation skill of the robot in the corresponding step of the motion trajectory instruction sequence; The real-time status field is used to populate the real-time data status, wherein the real-time data status includes the network interface address of the offline programming software, as well as the list of known point names corresponding to the current moment, the name of the loaded tool, the number of trajectory instructions, and the suction cup IO port cache status. The list of known point names includes multiple points and point labels corresponding to each of the multiple points. The point labels include role labels, associated object labels, applicable tool labels, and parent association labels. The parent association labels are used to indicate the relationship between the corresponding point and its corresponding parent point. The parent point is the approach point or return point of the corresponding point. The dialogue history context field is used to store the history context data; The security and logical constraint field is used to store the security and logical constraints. The output format constraints and user instruction fields are used to limit the output of the predetermined language model to a predetermined array format containing step number, operation skill name, parameter list and step purpose description, and attached with the target operation instruction.

8. The method according to claim 6, characterized in that, The process of obtaining the motion trajectory instruction sequence based on the structured input data and using the predetermined language model includes: Based on the structured input data, the predetermined language model is used to obtain a plan sequence containing step numbers, operation skill names, parameter lists, and step purpose descriptions; Verify whether the parameters of each operation step in the planned sequence are within the preset allowable range, whether the point name and IO port number referenced by each operation step exist in the real-time data status, and whether the planned sequence meets the security and logical constraints. If the planned sequence passes the verification, the planned sequence is converted into the motion trajectory instruction sequence.

9. The method according to claim 8, characterized in that, The method further includes: If the planned sequence verification fails, the error type that caused the verification to fail is identified, wherein the error type includes parameters exceeding the preset allowable range, the referenced point name or IO port number not existing in the real-time data state, or violation of at least one of the security and logic constraints. Based on the error type, generate a prompt message containing the error cause and correction suggestions; The prompt message is returned to the user terminal to guide the modification of the target operation command or to confirm the corrected operation parameters; Upon receiving a correction instruction based on the prompt information, the historical context data is updated to obtain new structured input data, and the step of generating a new motion trajectory instruction sequence based on the new structured input data using the predetermined language model is re-executed.

10. An electronic device, characterized in that, It includes one or more processors and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the robot control method based on natural language dialogue as described in any one of claims 1 to 9.

Images