Processing methods for embodied agents based on skill graph-enhanced industrial large-scale models

Abstract

This application provides a processing method for embodied intelligent agents based on skill graph-enhanced industrial large-scale models. The method includes: receiving a task processing request, which requests the embodied intelligent agent to process a target task; parsing the target task to obtain at least one subtask, and determining the execution order and task requirements of each subtask; determining the skill chain of the target task based on the execution order, task requirements, and skill graph; wherein the skill graph includes multiple skill nodes and the relationships between them; the skill chain includes at least one skill node from the multiple skill nodes; and processing the at least one subtask sequentially based on the at least one skill node to obtain the processing result of the target task. This method improves the reliability of the embodied intelligent agent in processing the target task.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method for processing embodied intelligent agents based on skill graph-enhanced industrial large models. Background Technology

[0002] An industrial intelligent agent is an integrated hardware and software intelligent system that uses sensors to perceive the environment, processes information with the help of artificial intelligence models, and drives industrial equipment to complete target tasks. In some scenarios, industrial intelligent agents need to perform specific tasks in industrial environments.

[0003] In related technologies, industrial intelligent agents can decompose the task requirements of a target task using a large language model or a visual language model, resulting in at least one sub-task. For each sub-task, the corresponding skill or tool is invoked to process it, obtaining the processing result of that sub-task. Based on the processing results of at least one sub-task, the target processing result of the target task is determined. However, the reliability of industrial intelligent agents processing target tasks in this way is relatively low. Summary of the Invention

[0004] This application provides a processing method for embodied intelligent agents based on skill graph-enhanced industrial large models, in order to improve the reliability of embodied intelligent agents in processing target tasks.

[0005] In a first aspect, embodiments of this application provide a method for processing embodied intelligent agents based on skill graph-enhanced industrial large models, including:

[0006] Receive task processing requests, which are used to request the embodied intelligent agent to process the target task;

[0007] The target task is analyzed to obtain at least one subtask, and the execution order and task requirements of each subtask are determined.

[0008] Based on the execution order of at least one subtask, the task requirements of at least one subtask, and the skill graph, the skill chain of the target task is determined; wherein, the skill graph includes multiple skill nodes and the relationships between the multiple skill nodes; the skill chain includes at least one skill node among the multiple skill nodes; a skill node is an executable functional unit that encapsulates functional semantics, input and output parameters, execution constraints, applicable conditions, invocation methods, and historical execution data;

[0009] At least one subtask is processed sequentially based on at least one skill node to obtain the processing result of the target task.

[0010] In some embodiments, determining the skill chain of the target task based on the execution order of at least one subtask, the task requirements of at least one subtask, and the skill map of each subtask includes:

[0011] Based on the execution order of at least one subtask, the task requirements of at least one subtask, and the relationship between multiple skill nodes, determine the skill node of at least one subtask from among the multiple skill nodes.

[0012] Determine the skill chain based on the execution order of at least one subtask and the skill nodes of at least one subtask.

[0013] In some embodiments, determining the skill node for each of at least one subtask from among multiple skill nodes based on the execution order of each of the at least one subtask, the task requirements of each of the at least one subtask, and the association between multiple skill nodes includes:

[0014] For the first subtask that is executed first, the skill node of the first subtask is determined from multiple skill nodes according to the task requirements of the first subtask;

[0015] For the i-th subtask whose execution order is i-th, the skill node of the i-th subtask is determined from among the multiple skill nodes based on the association between multiple skill nodes, the task requirements of the i-th subtask, and the skill node of the (i-1)-th subtask, where i is an integer greater than 1.

[0016] In some embodiments, determining the skill node of the first subtask from among multiple skill nodes based on the task requirements of the first subtask includes:

[0017] For each skill node among multiple skill nodes, determine the degree of functional matching between the skill node and the first subtask based on the task requirements of the first subtask;

[0018] The skill node with the highest functional match with the first subtask among multiple skill nodes is identified as the skill node of the first subtask.

[0019] In some embodiments, determining the skill node of the i-th subtask from among the multiple skill nodes based on the association relationships between multiple skill nodes, the task requirements of the i-th subtask, and the skill nodes of the (i-1)-th subtask includes:

[0020] Based on the task requirements of the i-th subtask, determine at least one first candidate node among multiple skill nodes that matches the function of the i-th subtask;

[0021] Based on the skill node of the (i-1)th subtask and the association between multiple skill nodes, determine the skill node of the i-th subtask from at least one first candidate node.

[0022] In some embodiments, the relationships between multiple skill nodes include: the weight of input-output compatibility between pairs of skill nodes, the weight of execution order, the weight of functional collaboration, the weight of fault correction, and the weight of historical execution experience.

[0023] In some embodiments, determining the skill node of the i-th subtask among at least one first candidate node includes:

[0024] Based on the execution order and functional collaboration relationship between the skill nodes of the (i-1)th subtask and each first candidate node, at least one second candidate node is determined from at least one first candidate node;

[0025] Based on the input-output compatibility between the skill node of the (i-1)th subtask and each second candidate node, determine at least one third candidate node from at least one second candidate node;

[0026] Based on the historical execution experience and fault correction relationships between the skill node of the (i-1)th subtask and each third candidate node, the skill node of the i-th subtask is determined from at least one third candidate node.

[0027] In some embodiments, after determining the processing result of the target task, the processing method for enhancing the embodied agent of the industrial large model based on the skill graph further includes:

[0028] Determine the input data, output data, execution status, and runtime log of at least one skill node;

[0029] Based on the processing results, the input data of at least one skill node, the output data of at least one skill node, the execution status of at least one skill node, and the running log of at least one skill node, the relationships between multiple skill nodes are updated.

[0030] In some embodiments, updating the association relationships between multiple skill nodes includes:

[0031] Based on the processing results, determine the processing status of the target task, which is either a successful processing status or a failed processing status.

[0032] When the processing status is successful, increase the weight of the association relationship between adjacent skill nodes in at least one skill node;

[0033] When the processing status is a processing failure status, based on the input data of at least one skill node, the output data of at least one skill node, the execution status of at least one skill node, and the running log of at least one skill node, the faulty node in at least one skill node is identified, and the association weight between the faulty node and its adjacent nodes is reduced.

[0034] Secondly, embodiments of this application provide a processing device for enhancing embodied intelligent agents in large industrial models based on skill graphs, the device comprising:

[0035] The receiving module is used to receive task processing requests, which are used to request the embodied intelligent agent to process the target task.

[0036] The parsing module is used to parse the target task, obtain at least one subtask, and determine the execution order and task requirements of each subtask.

[0037] The determination module is used to determine the skill chain of the target task based on the execution order of at least one subtask, the task requirements of at least one subtask, and the skill graph. The skill graph includes multiple skill nodes and the relationships between the skill nodes. The skill chain includes at least one skill node among the multiple skill nodes. The skill node is an executable functional unit that encapsulates functional semantics, input and output parameters, execution constraints, applicable conditions, invocation methods, and historical execution data.

[0038] The processing module is used to process at least one subtask sequentially based on at least one skill node to obtain the processing result of the target task.

[0039] Thirdly, embodiments of this application provide an electronic device, including: a memory and a processor;

[0040] The memory stores the instructions that the computer executes;

[0041] The processor executes computer execution instructions stored in memory, causing the processor to perform the first aspect and / or various possible implementations of the first aspect as described above.

[0042] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the first aspect and / or various possible implementations of the first aspect.

[0043] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the first aspect and / or various possible implementations of the first aspect.

[0044] The processing method of embodied intelligent agents based on skill graph-enhanced industrial large-scale models provided in this application involves the embodied intelligent agent generating a skill chain for a target task based on a skill graph containing standardized skill nodes and multiple relationships. The agent then processes at least one sub-task sequentially based on at least one skill node in the skill chain to obtain the processing result of the target task. Compared to the mode of simply temporarily breaking down the target task and randomly calling skills based on the industrial large-scale model, the method provided in this application standardizes the combination logic and execution flow between skill nodes, thereby improving the stability of the determined skill chain and the coherence of processing at least one sub-task through the skill chain, reducing problems such as input / output parameter mismatch and execution logic disorder. Simultaneously, the embodied intelligent agent ensures the compliance of sub-task processing based on the execution constraints and historical data built into the skill nodes. Therefore, the processing method of embodied intelligent agents based on skill graph-enhanced industrial large-scale models provided in this application improves the reliability of embodied intelligent agents in processing target tasks. Attached Figure Description

[0045] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0046] Figure 1 This is a schematic diagram of an application scenario provided by an embodiment of this application;

[0047] Figure 2 A flowchart illustrating a method for processing embodied intelligent agents based on a skill graph-enhanced industrial large model, provided in an embodiment of this application;

[0048] Figure 3 A flowchart illustrating the process of determining a skill chain is provided for an embodiment of this application;

[0049] Figure 4 A schematic diagram of an industrial knowledge generation device that integrates a large model and a knowledge graph, provided for an embodiment of this application;

[0050] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0051] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0052] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0053] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0054] First, for ease of understanding, the technical terms involved in the embodiments of this application will be explained.

[0055] Embodied intelligent agents are a type of integrated hardware and software intelligent system that uses sensors to perceive the environment, processes information with the help of large industrial models, and drives industrial equipment to complete target tasks.

[0056] Industrial large-scale models are large-scale artificial intelligence models customized for all business scenarios in industry. For example, an industrial large-scale model can be a multimodal industrial model that integrates language and vision processing capabilities, is responsible for analyzing industrial tasks, recognizing the on-site environment, and providing basic decision-making capabilities for embodied intelligent agents.

[0057] In related technologies, industrial intelligent agents can decompose the task requirements of a target task using a large language model or a visual language model to obtain at least one sub-task. For each sub-task, based on the functional descriptions of multiple skills or tools, the agent calls the corresponding skill or tool from a list of skills or tools to process the sub-task, obtaining the processing result of the sub-task. Based on the processing results of at least one sub-task, the target processing result of the target task is determined.

[0058] However, in industrial settings, the target tasks are standardized assembly line processes, with clearly defined functions for each subtask, a fixed execution sequence, and interconnected processes. Furthermore, embodied agents need to repeatedly perform similar production tasks over long periods and at high frequencies, placing high demands on the consistency of skill combinations, operational stability, and fault tolerance for continuous operation.

[0059] The above method only stores various work tools and skills in a flat, list-style, independent manner, and only labels the basic functions of each skill. When an embodied agent handles multi-process tasks in a production line, it can only filter and match skills in isolation for a single subtask, completely ignoring the need for linkage and adaptation between skills in preceding and subsequent processes. As a result, when the embodied agent repeatedly executes the same set of production line tasks, the skill combinations selected for each subtask are highly random each time, making it difficult to form a stable and reliable standardized skill execution link. This may lead to problems such as skill timing mismatch, incompatible input and output data, and failure of process collaboration, resulting in low reliability of the embodied agent in handling the target task using the above method.

[0060] Furthermore, the on-site environment and equipment conditions in industrial settings are dynamically changing. For example, parameters such as equipment wear and tear, mechanical load, and signal transmission status fluctuate in real time in industrial settings. At the same time, occasional equipment failures, data transmission anomalies, and process parameter deviations occur randomly on the production line. The skills determined in the above way lack fault substitution and dynamic adaptation scheduling logic. Once a skill malfunctions, there is no backup collaborative skill to take over the process, which can easily cause the entire production line to stop, further exacerbating the problem of low reliability in handling the target task.

[0061] Secondly, because the skill combination for handling at least one subtask is unstable, when faced with similar long-term tasks, the embodied agent needs to re-plan the skill combination for at least one subtask. This not only further reduces the reliability of processing the target task but also increases the processing overhead and reduces the processing efficiency of the target task.

[0062] Third, the above methods only use natural language to simply describe the function of each skill or tool, without standardizing the input and output parameters, usage rules, and operational feedback of each skill or tool. This makes the connection logic between two skills unclear, and the skill combination of at least one subtask is prone to problems such as abnormal data transmission, incompatible formats, and information transmission distortion, which further reduces the reliability of processing the target task.

[0063] Fourth, the above methods can only provide feedback on the processing results obtained from processing the target task. In the event of processing failure, the embodied intelligent agent cannot investigate the cause of the failure and lacks the ability to trace and locate the source of the skill combination for at least one subtask.

[0064] Finally, in the above-mentioned approach, the industrial intelligent agent, in the process of processing the target task, has difficulty in conforming to the task constraints, skill association rules and execution requirements of the target task, which causes the processing of the target task to deviate from the execution plan of practical logic, thereby further reducing the reliability of the processing of the target task.

[0065] Based on this, this application provides a processing method for embodied intelligent agents using a skill graph-enhanced industrial large-scale model. The embodied intelligent agent generates a skill chain for the target task based on a skill graph containing standardized skill nodes and multiple relationships, and processes at least one sub-task sequentially according to at least one skill node in the skill chain to obtain the processing result of the target task. Compared to the mode of simply temporarily breaking down the target task and randomly calling skills based on the industrial large-scale model, the method provided in this application standardizes the combination logic and execution flow between skill nodes, thereby improving the stability of the determined skill chain and the coherence of processing at least one sub-task through the skill chain, reducing problems such as input / output parameter mismatch and execution logic disorder. Simultaneously, the embodied intelligent agent ensures the compliance of sub-task processing based on the execution constraints and historical data built into the skill nodes. Therefore, the processing method for embodied intelligent agents using a skill graph-enhanced industrial large-scale model provided in this application improves the reliability of embodied intelligent agents in processing target tasks.

[0066] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0067] Figure 1 Please refer to the schematic diagram of an application scenario provided in this application embodiment. Figure 1 It includes a client 11 and an embodied intelligent agent 12. The client 11 can be a device such as a mobile phone or computer, and the embodied intelligent agent 12 stores a skill map.

[0068] In practical applications, data can be transmitted between the client 11 and the embodied intelligent agent 12. For example, when it is necessary to process a target task, the client 11 can send a task processing request to the embodied intelligent agent 12. The task processing request is used to request the embodied intelligent agent to process the target task.

[0069] After receiving a task processing request, the embodied intelligent agent 12 can process the target task according to the skill map and obtain the processing result of the target task.

[0070] It should be noted that, Figure 1 This is merely an example to illustrate one application scenario, and is not intended to limit the application scenario.

[0071] The technical solutions of this application and how they solve the aforementioned technical problems are described in detail below with specific embodiments. These specific embodiments may exist independently or in combination with each other. Identical or similar concepts or processes may not be repeated in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0072] Figure 2 This is a flowchart illustrating a method for processing embodied intelligent agents based on skill graph-enhanced industrial large-scale models, as provided in an embodiment of this application. Please refer to... Figure 2 The method may include the following steps:

[0073] S21. Receive a task processing request. The task processing request is used to request the embodied intelligent agent to process the target task.

[0074] A task processing request is an instruction sent by a client to an embodied intelligent agent. In some embodiments, the task processing request carries relevant content, execution requirements, and scenario information for processing a target task.

[0075] An embodied intelligent agent is an integrated hardware and software system that combines artificial intelligence algorithms, data processing modules, and industrial physical execution equipment. The embodied intelligent agent can perceive the industrial environment based on sensors, and use artificial intelligence models to analyze task processing requests, plan tasks, and output decisions. It then issues control commands to drive industrial equipment to complete the processing of the target task.

[0076] The target task is the industrial operation that the embodied intelligent agent needs to process. The target task can be a complete and complex industrial operation, or it can be broken down into one or more ordered sub-tasks to be executed step by step. For example, the target task could be "sorting cylinders on the table into blue boxes".

[0077] In some embodiments, the client may send a task processing request to the embodied agent. Correspondingly, the embodied agent receives the task processing request sent by the client.

[0078] S22. Analyze the target task to obtain at least one subtask, and determine the execution order of each subtask and the task requirements of each subtask.

[0079] A subtask is a subdivided work unit obtained by logically breaking down a target task. Each subtask corresponds to a specific, independently executable operation. In some embodiments, an embodied agent can complete the processing of the target task by sequentially processing at least one subtask.

[0080] For example, taking the target task as "sorting the cylinders on the table into the blue box", the embodied intelligent agent processes the target task and obtains at least one sub-task including: identifying the current scene, locating candidate objects, determining whether the target object is a cylinder, obtaining the target's three-dimensional position, generating a grasping pose, identifying the position of the blue box, planning the placement pose, and performing the grasping and placement action.

[0081] The execution order of subtasks is determined by the embodied intelligent agent's operational logic for the target task, industrial site constraints, and operating specifications, and is the order in which at least one subtask is processed.

[0082] The task requirements for a subtask are the clearly defined execution standards, scope of work, constraints, and expected results for a single subtask. For example, the task requirements for a subtask may include its functional requirements, parameter specifications, and usage scenarios.

[0083] In some embodiments, an industrial big data model is deployed in the embodied agent. The embodied agent can input a target task into the industrial big data model, process the target task through the industrial big data model, and obtain at least one subtask, the execution order of each of the at least one subtask, and the task requirements of each of the at least one subtask.

[0084] S23. Determine the skill chain of the target task based on the execution order of at least one subtask, the task requirements of at least one subtask, and the skill graph; wherein, the skill graph includes multiple skill nodes and the relationships between the multiple skill nodes; the skill chain includes at least one skill node among the multiple skill nodes; the skill node is an executable functional unit that encapsulates functional semantics, input and output parameters, execution constraints, applicable conditions, invocation methods, and historical execution data.

[0085] A skill graph is a structured graph network built for industrial tasks, used to uniformly combine, manage, and invoke industrial executable functional units. In some embodiments, a skill graph includes multiple skill nodes and the relationships between them. For example, a skill graph G can be represented as G = (V, E), where V represents a set of skill nodes, which includes multiple skill nodes; and E represents the relationships between the multiple skill nodes.

[0086] Skill nodes are standardized, packaged industrial executable functional units. For example, a skill node is an industrial executable functional unit obtained by uniformly packaging functional modules such as environmental perception, identification, localization, 3D mapping, grasping planning, placement planning, execution, and feedback diagnosis.

[0087] For example, multiple skill nodes may include image segmentation skills, target number localization skills, semantic recognition skills, two-dimensional coordinate to three-dimensional coordinate mapping skills, grasping pose generation skills, placement area planning skills, and robot execution control skills, etc.

[0088] In some embodiments, skill nodes integrate functional semantics, input / output parameters, execution constraints, applicable conditions, invocation methods, and historical execution data. Embodied agents can independently complete specific tasks through skill nodes.

[0089] Functional semantics defines the specific functions, operational roles, and application scenarios that a single skill node can achieve. Input and output parameters are the prerequisite data that the skill node needs to receive during operation and the result data that is output after execution. Input data parameters may include, for example, the data type, data format, and parameter range of the skill node's input and output data.

[0090] Execution constraints are various rules and boundary conditions that limit the normal operation of skill nodes. Execution constraints may include, for example, the skill node's operating thresholds, operating procedures, environmental limitations, and process standards. Applicable conditions are used to define the prerequisites for the normal startup and stable operation of a skill node. Applicable conditions may include, for example, the industrial environment status of the skill node, the completion status of prerequisite tasks, and operating condition requirements.

[0091] The invocation method indicates the triggering rules, interface protocols, and operation procedures followed to start and run the skill node. Historical execution data records complete information about the skill node's historical task execution. Historical execution data may include, for example, the success rate of the skill node's historical task execution, runtime, any exceptions encountered, and the type of failure.

[0092] The relationships between multiple skill nodes are logical connections established between any two skill nodes in the skill graph, used to standardize the combination and connection rules between skill nodes. In some embodiments, the relationships between multiple skill nodes include: the weight of input-output compatibility between any two skill nodes, the weight of execution order, the weight of functional collaboration, the weight of fault correction, and the weight of historical execution experience.

[0093] The input-output compatibility weight is used to characterize the degree of matching in data interaction between two skill nodes. Specifically, it indicates whether the data format, parameter type, and data content output by the preceding skill node can be used as input for the subsequent skill node and correctly received, parsed, and invoked. Therefore, input-output compatibility is a fundamental condition for ensuring the coherent operation of multiple skill nodes and reducing parameter transmission errors. A higher input-output compatibility weight indicates a higher degree of matching in data interaction between the two skill nodes; conversely, a lower weight indicates a lower degree of matching.

[0094] The weight of the execution order is a quantified value set by the temporal correlation between two skill nodes in the skill graph. It is used to represent the compliance of executing two skill nodes in a predetermined order. The larger the weight of the execution order, the stronger the temporal correlation between the two skill nodes; the smaller the weight of the execution order, the weaker the temporal correlation between the two skill nodes.

[0095] The weight of functional collaboration relationships is a quantitative indicator in the skill map that sets the functional cooperation effect between two skill nodes. It is used to measure the adaptability, collaboration efficiency, and stability of two skill nodes when working together. The larger the weight of the functional collaboration relationship, the stronger the functional complementarity between the two skill nodes and the better the collaborative operation effect; the smaller the weight of the functional collaboration relationship, the weaker the functional complementarity between the two skill nodes and the worse the collaborative operation effect.

[0096] The weight of a fault correction relationship is a quantitative value in the skill graph representing the fault adaptation and remediation effect between two skill nodes. It is used to measure the reliability of another skill node in completing fault fallback, problem repair, or function replacement when one skill node experiences an anomaly. The larger the weight of the fault correction relationship, the higher the reliability of the other skill node in completing fault fallback, problem repair, or function replacement when one skill node experiences an anomaly; the smaller the weight of the fault correction relationship, the lower the reliability of the other skill node in completing fault fallback, problem repair, or function replacement when one skill node experiences an anomaly.

[0097] The weight of historical execution experience is a numerical value in the skill graph that quantifies the performance of a combination of two skill nodes in historical tasks. It indicates the success rate, operational stability, and task completion quality of the combination of the two skill nodes. The larger the weight of historical execution experience, the higher the success rate, operational stability, and task completion quality of the combination of the two skill nodes; the smaller the weight of historical execution experience, the lower the success rate, operational stability, and task completion quality of the combination of the two skill nodes.

[0098] It should be noted that multiple skill nodes can have the same or different skill types. The skill type indicates the storage location, execution mode, and operation method of the skill node.

[0099] In some embodiments, skill types include local skill types and online skill types. Local skill types refer to skill nodes stored in the embodied agent or skill nodes generated by the embodied agent. For example, a skill node of a local skill type could be a robotic arm control skill node stored in the embodied agent.

[0100] Online skill types refer to skill nodes that an embodied agent invokes online through a network or application programming interface (API). For example, skill nodes for online skill types may include skill nodes such as repositories and data packages downloaded by the embodied agent from an open-source platform, or skill nodes such as large open-source models invoked through APIs.

[0101] A skill chain is an execution sequence formed by combining at least one skill node from multiple skill nodes in an orderly manner, based on the execution order of at least one subtask, the task requirements of at least one subtask, and the skill graph.

[0102] In some embodiments, an embodied intelligent agent can input the execution order of at least one subtask, the task requirements of at least one subtask, and the skill map of at least one subtask into an industrial big model. The industrial big model processes the execution order of at least one subtask, the task requirements of at least one subtask, and the skill map of at least one subtask to obtain the skill chain of the target task.

[0103] S24. Process at least one subtask sequentially based on at least one skill node to obtain the processing result of the target task.

[0104] The processing result is the final output of the complete work outcome after the embodied intelligent agent calls upon each skill node according to the skill chain to complete all sub-tasks. In some embodiments, the processing result includes various information such as industrial sensing data, equipment execution status, task completion status, and work output.

[0105] In some embodiments, for the first subtask, the embodied agent can process the first subtask through at least one skill node of the first subtask to obtain a sub-processing result of the first subtask. The embodied agent inputs the sub-processing result of the (i-1)th subtask to the skill node of the ith subtask, and processes the ith subtask through the skill node of the ith subtask to obtain a sub-processing result of the ith subtask, until i is greater than or equal to the number of at least one subtask, and then determines the sub-processing result of the ith subtask as the processing result of the target task. Here, i is an integer greater than or equal to 2.

[0106] For example, taking i=2, the embodied agent can process the first subtask through the skill node of the first subtask to obtain the sub-processing result of the first subtask. Then, the embodied agent inputs the sub-processing result of the first subtask into the skill node of the first subtask, and processes the second subtask through the skill node of the second subtask to obtain the sub-processing result of the second subtask. The embodied agent can determine the sub-processing result of the second subtask as the processing result of the target task.

[0107] exist Figure 2In the illustrated embodiment, the embodied agent generates a skill chain for the target task based on a skill graph containing standardized skill nodes and multiple relationships. It then processes at least one sub-task sequentially based on at least one skill node in the skill chain to obtain the result of the target task. Compared to the approach of simply temporarily breaking down the target task and randomly calling skills based on an industrial big data model, the method provided in this application standardizes the combination logic and execution flow between skill nodes, thereby improving the stability of the determined skill chain and the coherence of processing at least one sub-task through the skill chain, reducing problems such as input / output parameter mismatch and execution logic errors. Simultaneously, the embodied agent ensures the compliance of sub-task processing based on the execution constraints and historical data built into the skill nodes. Therefore, the processing method of the embodied agent based on the skill graph-enhanced industrial big data model provided in this application improves the reliability of the embodied agent in processing the target task.

[0108] exist Figure 2 Based on the illustrated embodiment, the following, in conjunction with Figure 3 The process of determining the target task in the embodiments of this application will be described in detail.

[0109] Figure 3 This is a schematic diagram illustrating a process for determining a skill chain, provided as an embodiment of this application. Figure 3 As shown, the process may include the following steps:

[0110] S31. Based on the execution order of at least one subtask, the task requirements of at least one subtask, and the relationship between multiple skill nodes, determine the skill node of at least one subtask from among the multiple skill nodes.

[0111] For each subtask in at least one subtask, the skill node of the subtask is an executable functional unit selected from the skill graph that matches the function and execution rules of the subtask. In other words, the skill node of the subtask is the execution carrier for completing the subtask.

[0112] In some embodiments, the embodied agent may determine the skill node of at least one subtask from among multiple skill nodes as follows: for the first subtask which is executed first, the skill node of the first subtask is determined from among multiple skill nodes according to the task requirements of the first subtask; for the i-th subtask which is executed i-th, the skill node of the i-th subtask is determined from among multiple skill nodes according to the association between multiple skill nodes, the task requirements of the i-th subtask, and the skill node of the (i-1)-th subtask, where i is an integer greater than 1.

[0113] In some embodiments, the embodied agent may determine the skill node of the first subtask in the following way: for each skill node among multiple skill nodes, determine the degree of functional matching between the skill node and the first subtask according to the task requirements of the first subtask; and determine the skill node with the highest degree of functional matching with the first subtask among the multiple skill nodes as the skill node of the first subtask.

[0114] In some embodiments, the embodied agent determines the skill node of the i-th subtask in the following manner: based on the task requirements of the i-th subtask, at least one first candidate node that matches the function of the i-th subtask is determined from among multiple skill nodes; based on the skill node of the (i-1)-th subtask and the association relationship between multiple skill nodes, the skill node of the i-th subtask is determined from at least one first candidate node.

[0115] The first subtask is the subtask that executes first in the execution order of at least one subtask, and it is the starting point for processing the target task. The i-th subtask is the subtask that executes i-th in the execution order of at least one subtask.

[0116] The i-th subtask has a preceding execution step, and the preceding execution step of the i-th subtask is the (i-1)-th subtask. Therefore, when determining the skill node of the i-th subtask, it is necessary to combine the skill node of the (i-1)-th subtask and the relationship between the skill nodes to determine the skill node of the i-th subtask.

[0117] The functional matching degree between a skill node and the first subtask quantifies the degree of fit between the skills node's functions, operating rules, and the task requirements of the first subtask. In some embodiments, a higher functional matching degree indicates a higher degree of fit between the skills node's functions, operating rules, and the task requirements of the first subtask, meaning the skills node is more closely aligned with the execution requirements of the first subtask. Conversely, a lower functional matching degree indicates a lower degree of fit between the skills node's functions, operating rules, and the task requirements of the first subtask, meaning the skills node is less closely aligned with the execution requirements of the first subtask.

[0118] The first candidate node is the skill node among multiple skill nodes that can satisfy the task requirements of the i-th subtask.

[0119] In some embodiments, for each of the multiple skill nodes, the skill node encapsulates its functional semantics, input / output parameters, execution constraints, applicable conditions, invocation method, and historical execution data. The embodied agent can determine the degree of matching between the functional semantics, input / output parameters, execution constraints, applicable conditions, invocation method, and historical execution data and the task requirements of the first subtask. Then, it performs a weighted calculation on the degree of matching between each of these elements and the task requirements of the first subtask to obtain the functional matching degree between the skill node and the first subtask.

[0120] The embodied agent compares the degree of functional matching between multiple skill nodes and the first subtask, and determines the skill node with the highest degree of functional matching with the first subtask as the skill node of the first subtask.

[0121] For example, suppose the multiple skill nodes include skill node A, skill node B, and skill node C. The embodied agent determines that the functional matching degree between skill node A and the first subtask is 20%, the functional matching degree between skill node B and the first subtask is 40%, and the functional matching degree between skill node C and the first subtask is 80%. Therefore, the embodied agent determines that the skill node of the first subtask is skill node C (20% < 40% < 80%).

[0122] In some embodiments, for each of the multiple skill nodes, the embodied agent can determine the degree of functional matching between the skill node and the i-th subtask, and determine the skill node whose degree of functional matching with the i-th subtask is greater than or equal to the degree threshold as the first candidate node.

[0123] In some embodiments, the embodied agent determines the skill node of the i-th subtask among at least one first candidate node in the following manner: determining at least one second candidate node among at least one first candidate node based on the weight of the execution order and the weight of the functional cooperation relationship between the skill node of the (i-1)-th subtask and each first candidate node; determining at least one third candidate node among at least one second candidate node based on the weight of the input-output compatibility between the skill node of the (i-1)-th subtask and each second candidate node; and determining the skill node of the i-th subtask among at least one third candidate node based on the weight of the historical execution experience and the weight of the fault correction relationship between the skill node of the (i-1)-th subtask and each third candidate node.

[0124] The second candidate node is at least one of the first candidate nodes that conforms to the job timing rules with the skill node of the (i-1)th subtask and can cooperate with the skill node of the (i-1)th subtask to complete the i-th subtask.

[0125] The third candidate node is a skill node among at least one of the second candidate nodes that can normally complete data interaction with the skill node of the (i-1)th subtask and meets the task requirements of the i-th subtask.

[0126] In some embodiments, since the relationships between multiple skill nodes include weights for the execution order between pairs of skill nodes and weights for functional collaboration relationships, for each first candidate node, the embodied agent determines the weight of the execution order between the skill node of the (i-1)th subtask and the first candidate node from the weights of the execution order between pairs of skill nodes. The embodied agent also determines the weight of the functional collaboration relationship between the skill node of the (i-1)th subtask and the first candidate node from the weights of the functional collaboration relationships between pairs of skill nodes.

[0127] If the weight of the execution order between the skill node of the (i-1)th subtask and the first candidate node is greater than or equal to the first weight threshold, and the weight of the functional collaboration relationship between the skill node of the (i-1)th subtask and the first candidate node is greater than or equal to the second weight threshold, it indicates that the reliability of the timing combination between the first candidate node and the skill node of the (i-1)th subtask meets the job requirements, and the functional collaboration effect between the two also meets the preset standard. That is, the adaptability of the first candidate node and the (i-1)th subtask meets the job requirements, so the first candidate node is determined as the second candidate node.

[0128] In some embodiments, since the association between multiple skill nodes includes the weight of input-output compatibility between pairs of skill nodes, for each second candidate node, the embodied agent can determine the weight of input-output compatibility between the skill node of the (i-1)th subtask and the second candidate node from the weight of input-output compatibility between pairs of skill nodes.

[0129] If the weight of the input-output compatibility between the skill node of the (i-1)th subtask and the second candidate node is greater than or equal to the third weight threshold, it indicates that the matching degree and transmission reliability of the data interaction between the second candidate node and the skill node of the (i-1)th subtask meet the job requirements. The second candidate node and the skill node of the (i-1)th subtask can normally complete data transmission and parsing. Therefore, the second candidate node is determined as the third candidate node.

[0130] In some embodiments, since the association between multiple skill nodes includes the weights of historical execution experience between each pair of skill nodes and the weights of fault correction relationships, for each third candidate node, the embodied agent can determine the weight of the historical execution experience between the (i-1)th skill node and the third candidate node from the weights of the historical execution experience between each pair of skill nodes. The embodied agent can also determine the weight of the fault correction relationship between the skill node of the (i-1)th subtask and the third candidate node from the weights of the fault correction relationships between each pair of skill nodes.

[0131] If the weight of the fault correction relationship between the skill node of the (i-1)th subtask and the third candidate node is greater than or equal to the fourth weight threshold, it indicates that the fault response and alternative remedy capabilities between the third candidate node and the (i-1)th skill node meet the operational requirements. Therefore, the third candidate node is determined as the fourth candidate node. Then, the embodied agent determines the skill node with the highest weight of historical execution experience with the skill node of the (i-1)th subtask from at least one fourth candidate node as the skill node of the i-th subtask.

[0132] S32. Determine the skill chain based on the execution order of at least one subtask and the skill nodes of at least one subtask.

[0133] An embodied intelligent agent can sequentially arrange the skill nodes of each subtask in the execution order of each subtask, so that the arrangement order of the skill nodes of each subtask is consistent with the execution order of each subtask. In this way, the skill nodes of each subtask are formed into a complete and ordered skill node sequence, and this skill node sequence is determined as a skill chain.

[0134] In some embodiments, the skill chain C can be represented as C = (v_1, v_2, ..., v_n), where v_1 is the skill node of the first subtask, v_2 is the skill node of the second subtask, and v_n is the skill node of the nth subtask.

[0135] exist Figure 3 In the illustrated embodiment, the embodied agent employs differentiated skill node selection logic for subtasks with different execution orders. The first subtask selects its skill node based on the degree of functional matching. Subsequent subtasks select their skill nodes by combining the preceding skill nodes with the multi-dimensional correlation weights between skill nodes. This ensures that at least one skill node is obtained that can accurately match the job requirements of each subtask, avoiding the problem of mismatch between skill nodes and subtask functions.

[0136] In addition, the embodied intelligent agent sequentially verifies multiple weighted indicators such as the execution order, functional collaboration, and input-output compatibility between skill nodes, and filters out suitable candidate nodes layer by layer to ensure that the timing of skill nodes is compliant, the functional collaboration is smooth, and the data interaction is normal. From the process connection level, it avoids faults such as abnormal data transmission, disordered work order, and disconnection of functional cooperation, making the obtained skill chain more coherent and stable, thereby improving the reliability of the embodied intelligent agent in processing the target task.

[0137] In determining the skill chain, the embodied agent combines historical execution experience weights and fault correction relationship weights, prioritizing skill node combinations with excellent historical performance and fault emergency recovery capabilities. This reduces the probability of failures during task execution and enhances the overall operational resilience and reliability. Furthermore, the embodied agent transforms various relationships between skill nodes into quantifiable weight indicators, while setting corresponding screening thresholds and hierarchical screening processes. This forms standardized and quantifiable node selection rules, facilitating application in industrial scenarios. The weights can also be dynamically adjusted based on actual execution conditions, ensuring the node matching rules continuously adapt to on-site operational needs.

[0138] Based on the above embodiments, after determining the processing result of the target task, the processing method of the embodied agent based on the skill graph-enhanced industrial large model provided in this application can also update the association relationship between multiple skill nodes. Specifically, the embodied agent can update the association relationship between multiple skill nodes as follows: determine the input data, output data, execution status, and operation log of at least one skill node; update the association relationship between multiple skill nodes according to the processing result, the input data, output data, execution status, and operation log of at least one skill node.

[0139] The input data for a skill node consists of the information and parameters it needs to receive to start running. In some embodiments, the input data for the i-th skill node can be the output data of the (i-1)-th skill node.

[0140] The output data of a skill node consists of the results, operating parameters, and execution feedback provided after the skill node completes its task. For example, the output data of an image segmentation skill node includes a target mask and candidate regions; the output data of a semantic recognition skill node includes the target category and target number; the output data of a grasping planning skill node is the grasping pose; and the output data of a placement planning skill node includes the placement position.

[0141] The execution status of a skill node indicates its working status during operation. The execution status can be any of the following: idle, running, paused, abnormal, or completed. The skill node's execution log is a complete record of the entire execution process. The skill node's execution log may include, for example, the skill node's runtime, operation steps, data flow records, exception alarms, and error messages.

[0142] In some embodiments, for each skill node in at least one skill node, the embodied agent can collect the operational data of the skill node in real time during the operation of the skill node, thereby determining the input data, output data, execution status, and operation log of the skill node. Specifically, the embodied agent can capture the content transmitted by the preceding node during the skill node startup phase and determine it as the input data of that skill node. The embodied agent can summarize the various result information of the skill node at the end of the skill node's operation and use it as the output data of that skill node.

[0143] During the operation of a skill node, the embodied intelligent agent can monitor the working status of the skill node in real time and determine the execution status of the skill node based on the working status. At the same time, it can fully record the execution sequence, data flow, operation behavior, abnormal alarms and error information of the skill node, and integrate the above content into a standardized operation log.

[0144] In some embodiments, the embodied agent updates the association between multiple skill nodes in the following manner: Based on the processing result, determine the processing status of the target task, which is either a successful processing status or a failed processing status; if the processing status is a successful processing status, increase the association weight between adjacent skill nodes in at least one skill node; if the processing status is a failed processing status, based on the input data, output data, execution status, and running log of each of the at least one skill node, determine the faulty node in at least one skill node and decrease the association weight between the faulty node and its adjacent nodes.

[0145] The processing status is the overall operational status of the target task, determined based on the processing result. The processing status can be either successful or failed. A successful processing status indicates that the target task has been executed completely and compliantly according to preset requirements, with all skill nodes in the skill chain cooperating smoothly and without any abnormal issues. A failed processing status indicates that the target task failed to meet preset requirements, the task process endpoint or job result did not meet standards, or there was a processing abnormality or failure during operation.

[0146] A fault node is a skill node in the skill chain that triggers the task executor table when the processing state is "processing failure".

[0147] For example, suppose the target task is "move the water cup from region a to region b", the embodied agent processes the target task, and the processing result is "the water cup has been moved from region a to region c", then the processing state is determined to be a processing failure state.

[0148] When the processing status is successful, it indicates that the adjacent skill nodes in the skill chain perform well in terms of timing coordination, data interaction, functional collaboration, and risk response. The adaptability and reliability of various relationships between nodes have been verified in practice. Therefore, the embodied agent can increase the weight of the relationship between adjacent skill nodes in at least one skill node to strengthen the priority of the skill chain.

[0149] If the processing status is "processing failed", it means that there are problems such as abnormal operation of skill nodes, data interaction errors, or failure of functional coordination in the skill chain, which leads to the inability to complete the overall task normally.

[0150] In this scenario, for each skill node, the embodied agent can determine whether there are any missing data transmissions or parameter interactions based on the node's input and output data; determine whether the node experiences any abnormal operating conditions such as lag, crashes, or errors during operation based on its execution status; and retrieve abnormal records, alarm information, and fault trajectories from the node's operation logs. If the skill node exhibits at least one of the above-mentioned fault issues, the embodied agent determines that the skill node is a faulty node.

[0151] For faulty nodes, the timing coordination, data interaction, functional collaboration, or fault response capabilities between the faulty node and its neighboring nodes do not meet the operational requirements, and the reliability and adaptability of the combined operation are poor. Therefore, embodied intelligent agents can reduce the weight of the association relationship between the faulty node and its neighboring nodes.

[0152] Optionally, an embodied agent can identify alternative skill nodes for a failed skill node among multiple skill nodes and update the skill chain based on the alternative skill node.

[0153] In the above embodiments, after completing the target task, the embodied agent increases the association weights of adjacent nodes in the skill chain if the task is successfully executed, thus strengthening the priority of selecting high-quality combinations. If the task fails, the agent locates the faulty node based on various operational data and decreases the association weights between the faulty node and its adjacent nodes. This enables autonomous iterative optimization of the skill graph based on actual operational feedback, continuously selecting and retaining highly adaptable and reliable skill node combinations, constantly improving the skill graph, and enhancing the success rate, operational stability, and environmental adaptability of the embodied agent in subsequent task executions.

[0154] Based on the above embodiments, the following describes the specific application of the processing method for embodied intelligent agents based on skill graph enhancement of industrial large models provided in this application in industrial scenarios, with reference to specific embodiments.

[0155] Take the scenario of an embodied intelligent agent screening abnormal industrial parts as an example. The objective is to "identify abnormal industrial parts on the product line and transfer them to a defective product storage box".

[0156] An embodied intelligent agent can input the target task into an industrial big data model, process the target task through the industrial big data model, and obtain at least one sub-task including: collecting visual images of parts on the production line, determining the dataset of the appearance dimensions of the parts, determining whether the parts are abnormal parts, driving a robotic arm to grasp the abnormal parts if the parts are abnormal parts, and controlling the robotic arm to move the abnormal parts to the defective parts storage box.

[0157] The execution order of at least one subtask includes: 1) collecting visual images of parts on the product line; 2) determining the dataset of part appearance dimensions; 3) determining whether a part is an abnormal part; 4) driving a robotic arm to grab the abnormal part if it is an abnormal part; and 5) controlling the robotic arm to move the abnormal part to the defective product storage box.

[0158] The requirements for having at least one subtask include:

[0159] The task of collecting visual images of parts on the production line is to acquire high-definition visual images of each industrial part in real time during the conveyor belt process, fully preserve the image information such as the overall appearance, edges, and surface texture of the industrial parts, and output image data with uniform resolution and uniform transmission format, without problems such as blurry images, occlusion, or missed images.

[0160] The task requirements for determining the size dataset of a part are as follows: extract size features based on the collected visual images of the part, perform pixel conversion and numerical calculation by benchmarking against the size threshold of a standard part, and generate a size dataset containing information such as the length, width, height, hole diameter, and contour tolerance of the part.

[0161] The task requirement for determining whether a part is abnormal is to compare the part size dataset with the standard dataset of qualified parts, identify various defects such as size differences, surface scratches, deformation, and missing materials, and output the judgment result on whether the industrial part is abnormal.

[0162] When a part is defective, the task of driving the robotic arm to grasp the defective part is as follows: after receiving the judgment result, quickly locate the real-time coordinates of the defective part on the production line, output precise motion control commands to drive the robotic arm gripper to smoothly close and grasp the part, with moderate grasping force to avoid the part slipping or being damaged, and only perform grasping on defective parts, while qualified parts skip the grasping process.

[0163] The task of controlling the robotic arm to move the abnormal part to the defective product storage box is as follows: after the robotic arm grabs the abnormal part, it moves smoothly along the preset trajectory to the designated placement point above the defective product storage box, releases the gripper to complete the placement of the part, and automatically resets to the assembly line picking and waiting position after placement.

[0164] Assume that multiple skill nodes include image acquisition skill nodes, environmental acquisition skill nodes, data acquisition skill nodes, part grasping skill nodes, defect detection skill nodes, part transfer skill nodes, and dynamic operation skill nodes.

[0165] For the subtask of acquiring visual images of parts on the product line, the embodied agent determines the degree of functional matching between the above-mentioned multiple skill nodes and the subtask, and determines the skill node with the highest degree of functional matching as the image acquisition skill node.

[0166] For a dataset subset task that determines the appearance dimensions of a part, the embodied agent, based on the task requirements of the dataset, identifies skill nodes among multiple skill nodes whose functional matching degree with the dataset subset task is greater than a certain threshold as data acquisition skill nodes and environment acquisition skill nodes.

[0167] The embodied agent determines that the execution order between the data acquisition skill node and the image acquisition skill node has a weight greater than a first weight threshold, and the functional collaboration relationship between the data acquisition skill node and the image acquisition skill node has a weight greater than a second weight threshold. Similarly, the embodied agent determines that the execution order between the environment acquisition skill node and the image acquisition skill node has a weight greater than the first weight threshold, and the functional collaboration relationship between the environment acquisition skill node and the image acquisition skill node has a weight greater than the second weight threshold. Therefore, the embodied agent determines the environment acquisition skill node and the data acquisition skill node as the second candidate nodes.

[0168] The embodied agent determines that the weight for the input-output compatibility between the data acquisition skill node and the image acquisition skill node is greater than the third weight threshold. The weight for the input-output compatibility between the environment acquisition skill node and the image acquisition skill node is less than the third weight threshold. Therefore, the embodied agent determines the image acquisition skill node as the skill node for "determining the dataset of part appearance dimensions".

[0169] Similar to how the embodied agent determines the skill node for "dataset of part appearance dimensions," the skill node for "determining whether a part is an abnormal part" is a defect detection skill node. The skill node for "driving a robotic arm to grasp the abnormal part if it is an abnormal part" is a part grasping skill node. The skill node for "controlling the robotic arm to move the abnormal part to the defective parts storage box" is a part transfer skill node.

[0170] The embodied intelligent agent acquires visual images of parts on the production line through image acquisition skill nodes, obtaining image information of the industrial parts. This image information is then input into a data acquisition skill node, which generates a dimensional dataset of the industrial parts. The embodied intelligent agent inputs this dimensional dataset into a defect detection skill node, which determines the assessment result of the industrial parts. The assessment result indicates whether any anomalies exist in the industrial parts.

[0171] The embodied agent inputs the judgment result to the part grasping skill node. When the judgment result indicates an anomaly in the industrial part, the part grasping skill node determines the coordinates of the industrial part on the production line and controls the robotic arm to grasp the part at those coordinates. After the robotic arm grasps the industrial part, the embodied agent uses the part transfer skill node to transfer the industrial part to a designated drop-off point above the defective product collection bin and controls the robotic arm to release the gripper, thus completing one target task.

[0172] In some embodiments, after completing a target task, the embodied agent can determine the processing status of the target task. If the processing status is successful, the embodied agent can increase the association weights between adjacent skill nodes in the image acquisition skill node, data acquisition skill node, defect detection skill node, part grasping skill node, and part transfer skill node. This increases the probability that the skill chain determined by the embodied agent during the execution of the next target task is the same as the skill chain determined during the execution of the previous target task, thereby increasing the success rate of the embodied agent in executing the next target task.

[0173] Over time, embodied agents can continuously accumulate high-quality skill chains adapted to current production line conditions, constantly strengthen the scheduling priority of standardized and highly adaptable skill chains, reduce random fluctuations in skill node combinations, and form stable and reusable skill chains. Furthermore, when facing dynamically changing conditions in industrial scenarios such as lighting, materials, and equipment wear and tear, embodied agents do not need to randomly match skill nodes each time. They can quickly reuse reliable skill chains validated by actual tasks, continuously improving the execution stability and operational efficiency of repetitive sorting tasks on the production line, reducing the frequency of failures such as missed parts, gripping failures, and misplacement, thereby improving the reliability of embodied agents in handling target tasks.

[0174] Figure 4 This is a schematic diagram of a processing device for enhancing an embodied intelligent agent in a large industrial model based on a skill graph, as provided in an embodiment of this application. Figure 4 As shown, the device includes a receiving module 41, a parsing module 42, a determining module 43, and a processing module 44, wherein:

[0175] The receiving module 41 is used to receive a task processing request, which is used to request the embodied intelligent agent to process the target task.

[0176] The parsing module 42 is used to parse the target task, obtain at least one subtask, and determine the execution order and task requirements of each of the at least one subtask.

[0177] The determination module 43 is used to determine the skill chain of the target task based on the execution order of at least one subtask, the task requirements of at least one subtask, and the skill graph; wherein, the skill graph includes multiple skill nodes and the relationships between the multiple skill nodes; the skill chain includes at least one skill node among the multiple skill nodes; the skill node is an executable functional unit that encapsulates functional semantics, input and output parameters, execution constraints, applicable conditions, invocation methods, and historical execution data;

[0178] The processing module 44 is used to process at least one subtask sequentially based on at least one skill node to obtain the processing result of the target task.

[0179] In some embodiments, the determining module 43 is specifically used for:

[0180] Based on the execution order of at least one subtask, the task requirements of at least one subtask, and the relationship between multiple skill nodes, determine the skill node of at least one subtask from among the multiple skill nodes.

[0181] Determine the skill chain based on the execution order of at least one subtask and the skill nodes of at least one subtask.

[0182] In some embodiments, the determining module 43 is specifically used for:

[0183] For the first subtask that is executed first, the skill node of the first subtask is determined from multiple skill nodes according to the task requirements of the first subtask;

[0184] For the i-th subtask whose execution order is i-th, the skill node of the i-th subtask is determined from among the multiple skill nodes based on the association between multiple skill nodes, the task requirements of the i-th subtask, and the skill node of the (i-1)-th subtask, where i is an integer greater than 1.

[0185] In some embodiments, the determining module 43 is specifically used for:

[0186] For each skill node among multiple skill nodes, determine the degree of functional matching between the skill node and the first subtask based on the task requirements of the first subtask;

[0187] The skill node with the highest functional match with the first subtask among multiple skill nodes is identified as the skill node of the first subtask.

[0188] In some embodiments, the determining module 43 is specifically used for:

[0189] Based on the task requirements of the i-th subtask, determine at least one first candidate node among multiple skill nodes that matches the function of the i-th subtask;

[0190] Based on the skill node of the (i-1)th subtask and the association between multiple skill nodes, determine the skill node of the i-th subtask from at least one first candidate node.

[0191] In some embodiments, the relationships between multiple skill nodes include: the weight of input-output compatibility between pairs of skill nodes, the weight of execution order, the weight of functional collaboration, the weight of fault correction, and the weight of historical execution experience.

[0192] In some embodiments, the determining module 43 is specifically used for:

[0193] Based on the execution order and functional collaboration relationship between the skill nodes of the (i-1)th subtask and each first candidate node, at least one second candidate node is determined from at least one first candidate node;

[0194] Based on the input-output compatibility between the skill node of the (i-1)th subtask and each second candidate node, determine at least one third candidate node from at least one second candidate node;

[0195] Based on the historical execution experience and fault correction relationships between the skill node of the (i-1)th subtask and each third candidate node, the skill node of the i-th subtask is determined from at least one third candidate node.

[0196] In some embodiments, after determining the processing result of the target task, the processing apparatus for enhancing the embodied intelligent agent of the industrial large model based on the skill graph further includes an update module, which is specifically used for:

[0197] Determine the input data, output data, execution status, and runtime log of at least one skill node;

[0198] Based on the processing results, the input data of at least one skill node, the output data of at least one skill node, the execution status of at least one skill node, and the running log of at least one skill node, the relationships between multiple skill nodes are updated.

[0199] In some embodiments, the update module is specifically used for:

[0200] Based on the processing results, determine the processing status of the target task, which is either a successful processing status or a failed processing status.

[0201] When the processing status is successful, increase the weight of the association relationship between adjacent skill nodes in at least one skill node;

[0202] When the processing status is a processing failure status, based on the input data of at least one skill node, the output data of at least one skill node, the execution status of at least one skill node, and the running log of at least one skill node, the faulty node in at least one skill node is identified, and the association weight between the faulty node and its adjacent nodes is reduced.

[0203] The processing device 40 for embodied intelligent agents based on skill graph-enhanced industrial large models provided in this application embodiment can execute the processing method for embodied intelligent agents based on skill graph-enhanced industrial large models provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described again here.

[0204] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 5 As shown, the electronic device 50 provided in this application embodiment includes: a memory 51 and a processor 52;

[0205] Memory 51 stores instructions executed by the computer;

[0206] The processor 52 executes the computer execution instructions stored in the memory 51, causing the processor 52 to execute the processing method of the embodied intelligent agent based on the skill graph-enhanced industrial big model provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0207] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0208] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0209] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0210] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described processing method for embodied intelligent agents based on skill graph-enhanced industrial big model.

[0211] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described processing method for embodied intelligent agents based on skill graph-enhanced industrial large-scale models.

[0212] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0213] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0214] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0215] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0216] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0217] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0218] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0219] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A method for processing embodied intelligent agents based on skill graph-enhanced industrial large-scale models, characterized in that, Applied to embodied intelligent agents, including: Receive a task processing request, the task processing request being used to request the embodied intelligent agent to process a target task; The target task is parsed to obtain at least one subtask, and the execution order and task requirements of each of the at least one subtask are determined. The skill chain of the target task is determined based on the execution order of each of the at least one subtask, the task requirements of each of the at least one subtask, and the skill graph; wherein, the skill graph includes multiple skill nodes and the relationships between the multiple skill nodes; the skill chain includes at least one skill node among the multiple skill nodes; the skill node is an executable functional unit encapsulated with functional semantics, input and output parameters, execution constraints, applicable conditions, invocation methods, and historical execution data; Based on the at least one skill node, the at least one subtask is processed sequentially to obtain the processing result of the target task.

2. The method according to claim 1, characterized in that, Determining the skill chain of the target task based on the execution order of the at least one subtask, the task requirements of the at least one subtask, and the skill map includes: Based on the execution order of each of the at least one subtask, the task requirements of each of the at least one subtask, and the association between the multiple skill nodes, the skill node of each of the at least one subtask is determined from the multiple skill nodes; The skill chain is determined based on the execution order of each of the at least one subtask and the skill nodes of each of the at least one subtask.

3. The method according to claim 2, characterized in that, The step of determining the skill node for each of the at least one subtask from among the multiple skill nodes based on the execution order of each of the at least one subtask, the task requirements of each of the at least one subtask, and the association between the multiple skill nodes includes: For the first subtask that is executed first, the skill node of the first subtask is determined from the plurality of skill nodes according to the task requirements of the first subtask; For the i-th subtask whose execution order is i-th, the skill node of the i-th subtask is determined from the multiple skill nodes according to the association between the multiple skill nodes, the task requirements of the i-th subtask, and the skill node of the (i-1)-th subtask, where i is an integer greater than 1.

4. The method according to claim 3, characterized in that, The step of determining the skill node of the first subtask from among the plurality of skill nodes according to the task requirements of the first subtask includes: For each of the multiple skill nodes, the degree of functional matching between the skill node and the first subtask is determined according to the task requirements of the first subtask; The skill node with the highest functional matching degree with the first subtask among the multiple skill nodes is determined as the skill node of the first subtask.

5. The method according to claim 3 or 4, characterized in that, The step of determining the skill node of the i-th sub-task from among the multiple skill nodes based on the association relationships between the multiple skill nodes, the task requirements of the i-th sub-task, and the skill nodes of the (i-1)-th sub-task includes: Based on the task requirements of the i-th sub-task, at least one first candidate node that matches the function of the i-th sub-task is determined from the plurality of skill nodes; Based on the skill node of the (i-1)th subtask and the association relationship between the multiple skill nodes, the skill node of the i-th subtask is determined from the at least one first candidate node.

6. The method according to claim 5, characterized in that, The relationships between the multiple skill nodes include: the weight of input-output compatibility between pairs of skill nodes, the weight of execution order, the weight of functional collaboration, the weight of fault correction, and the weight of historical execution experience.

7. The method according to claim 6, characterized in that, Determining the skill node of the i-th subtask from the at least one first candidate node includes: Based on the weights of the execution order and functional collaboration relationships between the skill nodes of the (i-1)th subtask and each first candidate node, at least one second candidate node is determined from the at least one first candidate node. Based on the weights of the input-output compatibility between the skill nodes of the (i-1)th subtask and each of the second candidate nodes, at least one third candidate node is determined from the at least one second candidate node; The skill node of the i-th subtask is determined from the at least one third candidate node based on the weights of the historical execution experience between the skill node of the i-1-th subtask and each third candidate node, and the weights of the fault correction relationship.

8. The method according to any one of claims 1-4, characterized in that, After determining the processing result of the target task, the method further includes: Determine the input data, output data, execution status, and running log of each of the at least one skill node; Based on the processing results, the input data of each of the at least one skill node, the output data of each of the at least one skill node, the execution status of each of the at least one skill node, and the running log of each of the at least one skill node, the association between the multiple skill nodes is updated.

9. The method according to claim 8, characterized in that, The updating of the association relationships between the multiple skill nodes includes: Based on the processing result, the processing status of the target task is determined, which is either a successful processing status or a failed processing status. When the processing status is the processing success status, the association weight between adjacent skill nodes in the at least one skill node is increased; When the processing state is the processing failure state, based on the input data of the at least one skill node, the output data of the at least one skill node, the execution state of the at least one skill node, and the running log of the at least one skill node, the faulty node among the at least one skill node is determined, and the association weight between the faulty node and its adjacent nodes is reduced.

10. A processing device for enhancing embodied intelligent agents in large industrial models based on skill graphs, characterized in that, The device includes: A receiving module is used to receive a task processing request, wherein the task processing request is used to request the embodied intelligent agent to process a target task; The parsing module is used to parse the target task, obtain at least one subtask, and determine the execution order and task requirements of the at least one subtask. A determination module is used to determine the skill chain of the target task based on the execution order of the at least one subtask, the task requirements of the at least one subtask, and the skill graph; wherein the skill graph includes multiple skill nodes and the relationships between the multiple skill nodes; the skill chain includes at least one skill node among the multiple skill nodes; the skill node is an executable functional unit encapsulated with functional semantics, input / output parameter execution constraints, and historical execution data; The processing module is used to process the at least one subtask sequentially based on the at least one skill node to obtain the processing result of the target task.

Images