A multi-agent driven industrial software component assembly method and system

Abstract

The application discloses a kind of multi-agent driven industrial software component assembly method and system, wherein the assembly method is by obtaining the subtask sequence corresponding to target task instruction and the first component sequence corresponding to first subtask sequence;According to target task instruction, the first subtask sequence and the first component sequence are regenerated and optimized, and the second subtask sequence and the second component sequence are obtained;Inquire the historical task log corresponding to second subtask sequence and second component sequence;According to historical task log, the second subtask sequence and the second component sequence are regenerated and optimized, and the optimized response representation is obtained;According to historical task log, parameter inference is carried out to optimized response representation, and target task workflow is obtained.The assembly method can reduce the development time of task workflow, reduce the work intensity of staff, improve the generation quality of task workflow.The application relates to the technical field of deep learning.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of deep learning, and in particular to a multi-agent driven industrial software component assembly method and system. BACKGROUND

[0002] In the industrial software scenario, the traditional development method is usually to manually write and configure a large number of repetitive code structures and interaction logic between various components, thereby realizing the generation of task workflow. This method requires a lot of development time and high personnel work intensity.

[0003] In addition, there is also a development method based on a large language model (LLM). Specifically, the large language model is fine-tuned based on pre-collected fine-tuning data, and then the fine-tuned large language model is used to realize the development and generation of task workflow. However, in practice, the data quality and size of the fine-tuning data are often limited, and the fine-tuned large language model often cannot well understand specific tasks and lacks generalization ability when facing novel or complex task scenarios, resulting in unsatisfactory quality of task workflow generation.

[0004] Therefore, the problems of the prior art still need to be solved and optimized. SUMMARY

[0005] The present application aims to at least partially solve one of the technical problems in the related art.

[0006] To this end, an object of an embodiment of the present application is to provide a multi-agent driven industrial software component assembly method and system, wherein the assembly method can reduce the development time of task workflow, reduce the work intensity of workers, and improve the generation quality of task workflow.

[0007] In order to achieve the above technical purpose, the technical solutions adopted by the embodiments of the present application include:

[0008] In a first aspect, an embodiment of the present application provides a multi-agent driven industrial software component assembly method, comprising:

[0009] A first language agent obtains a target task instruction and performs instruction decomposition on the target task instruction to obtain a first sub-task sequence;

[0010] A component agent performs component retrieval on the first sub-task sequence according to the target task instruction to obtain a first component sequence;

[0011] The first language agent performs first regenerative optimization on the first subtask sequence and the first component sequence according to the target task instruction, to obtain a second subtask sequence and a second component sequence, each subtask in the second subtask sequence corresponding to a task component in the second component sequence;

[0012] The retrieval agent queries task logs corresponding to the second subtask sequence and the second component sequence, to obtain a first historical task log and a second historical task log, the first historical task log being used to record historical task instructions, and historical subtask sequences and historical component sequences corresponding to the historical task instructions, and the second historical task log being used to record component information of each task component in the second component sequence, and component execution logs corresponding to the component information;

[0013] The first language agent performs second regenerative optimization on the second subtask sequence and the second component sequence according to the first historical task log, to obtain an optimized response representation, the optimized response representation being used to record a third subtask sequence, a third component sequence and scene graph information corresponding to the target task instruction;

[0014] A second language agent performs parameter inference on the optimized response representation according to the second historical task log, to obtain a target task workflow, the target task workflow being used to represent the third component sequence and component parameters corresponding to each task component in the third component sequence.

[0015] In addition, the assembly method according to the above-mentioned embodiments of the present application can have the following additional technical features:

[0016] Further, in an embodiment of the present application, the instruction decomposition of the target task instruction to obtain the first subtask sequence comprises:

[0017] obtaining a first prompt and an initial scene graph corresponding to the target task instruction, the first prompt being used to instruct to generate a plurality of subtasks corresponding to the target task instruction;

[0018] prompt decomposition of the target task instruction according to the first prompt and the initial scene graph to obtain the first subtask sequence.

[0019] Further, in an embodiment of the present application, the component retrieval of the first subtask sequence according to the target task instruction to obtain the first component sequence comprises:

[0020] obtaining a preset component library;

[0021] According to the component library and the target task instruction, component similarity calculation is performed on the first sub-task sequence to obtain the first component sequence.

[0022] Further, in an embodiment of the present application, the query corresponds to the second sub-task sequence and the second component sequence The task log obtains the first historical task log and the second historical task log, including:

[0023] Obtain a preset knowledge graph database and a knowledge vector database;

[0024] According to the second sub-task sequence and the second component sequence, at least one of the knowledge graph database and the knowledge vector database is first log inquiry, and the target historical log is obtained The first historical task log and the second historical task log.

[0025] Further, in an embodiment of the present application, the parameter inference of the optimization response representation according to the second historical task log obtains the target task workflow, including:

[0026] Obtain a preset second prompt, the second prompt is used to indicate the acquisition method of the component parameter corresponding to each task component of the third component sequence;

[0027] According to the second prompt, the second historical task log, the third sub-task sequence and the scene graph information, the parameter of the third component sequence is predicted, and the target task workflow is obtained.

[0028] Further, in an embodiment of the present application, the assembly method further comprises:

[0029] The reflection agent obtains the execution data corresponding to the target task workflow, the execution data includes execution result and execution information, the execution result is used to indicate whether the target task component in the target task workflow is successfully executed;

[0030] If the execution result is execution failure, the reflection agent obtains the third historical task log corresponding to the target task workflow, and according to the third historical task log, the information extraction is performed on the execution information to obtain the execution summary;

[0031] The first language intelligent agent performs third regeneration optimization on the optimization response representation according to the execution summary and the target task instruction, and obtains the fourth sub-task sequence;

[0032] The second language intelligent agent performs repair reasoning on the target task workflow according to the execution summary and the fourth sub-task sequence, and obtains the repaired target task workflow.

[0033] Further, in the embodiment of the present application, the repairing inference of the target task workflow according to the execution summary and the fourth subtask sequence comprises:

[0034] obtaining a fourth component sequence corresponding to the fourth subtask sequence, and determining a fourth historical task log corresponding to the fourth component sequence according to the fourth component sequence;

[0035] performing parameter repairing on the target task workflow according to the execution summary, the fourth historical task log, the fourth subtask sequence and the fourth component sequence, to obtain the repaired target task workflow.

[0036] Further, in the embodiment of the present application, the information extraction of the execution information according to the third historical task log comprises:

[0037] obtaining an expected scene graph, a first scene graph and a second scene graph corresponding to the execution result, the first scene graph being an actual scene graph corresponding to the target task component, and the second scene graph being a previous frame scene graph of the first scene graph;

[0038] performing execution deviation reflection on the first scene graph according to the expected scene graph, to obtain a scene reflection result;

[0039] performing event reflection on the first scene graph according to the second scene graph, to obtain an event reflection result;

[0040] performing summary and extraction on the scene reflection result and the event reflection result according to the third historical task log, to obtain the execution summary.

[0041] Further, in the embodiment of the present application, the fourth historical task log is obtained by the following steps:

[0042] performing component screening on the fourth component sequence according to the third component sequence, to obtain an updated task component, so that the second language intelligent agent calls the retrieval intelligent agent to perform second log inquiry on the updated task component, to obtain the fourth historical task log.

[0043] In a second aspect, the embodiment of the present application provides an industrial software component assembly system driven by multiple intelligent agents, comprising:

[0044] The first language intelligent agent is configured to obtain a target task instruction, perform instruction decomposition on the target task instruction to obtain a first subtask sequence, perform first regeneration optimization on the first subtask sequence and a first component sequence according to the target task instruction to obtain a second subtask sequence and a second component sequence, each subtask in the second subtask sequence corresponding to a task component in the second component sequence, and perform second regeneration optimization on the second subtask sequence and the second component sequence according to a first historical task log to obtain an optimized response representation, the optimized response representation being used to record a third subtask sequence, a third component sequence and scene graph information corresponding to the target task instruction.

[0045] The component intelligent agent is configured to perform component retrieval on the first subtask sequence according to the target task instruction to obtain the first component sequence.

[0046] The retrieval intelligent agent is configured to query a task log corresponding to the second subtask sequence and the second component sequence to obtain the first historical task log and a second historical task log, the first historical task log being used to record a historical task instruction and a historical subtask sequence and a historical component sequence corresponding to the historical task instruction, and the second historical task log being used to record component information of each task component in the second component sequence and a component execution log corresponding to the component information.

[0047] The second language intelligent agent is configured to perform parameter inference on the optimized response representation according to the second historical task log to obtain a target task workflow, the target task workflow being used to represent the third component sequence and a component parameter corresponding to each task component in the third component sequence.

[0048] In a third aspect, an electronic device is provided, including:

[0049] at least one processor;

[0050] at least one memory configured to store at least one program;

[0051] When the at least one program is executed by the at least one processor, the at least one processor implements the method described above.

[0052] In a fourth aspect, a computer readable storage medium is provided, which stores a program executable by a processor, and the program executable by the processor is used to implement the method described above when executed by the processor.

[0053] The advantages and beneficial effects of the present application will be partially given in the following description, partially will become obvious from the following description, or will be learned through the practice of the present application:

[0054] The embodiment of the present application discloses a multi-agent driven industrial software component assembly method and system, wherein the assembly method obtains a target task instruction through a first language agent, and performs instruction decomposition on the target task instruction to obtain a first sub-task sequence; a component agent performs component retrieval on the first sub-task sequence according to the target task instruction to obtain a first component sequence; the first language agent performs first regeneration optimization on the first sub-task sequence and the first component sequence according to the target task instruction to obtain a second sub-task sequence and a second component sequence, each sub-task in the second sub-task sequence corresponds to a task component in the second component sequence; a retrieval agent queries a task log corresponding to the second sub-task sequence and the second component sequence to obtain a first historical task log and a second historical task log, the first historical task log is used for recording a historical task instruction, and a historical sub-task sequence and a historical component sequence corresponding to the historical task instruction, the second historical task log is used for recording component information of each task component in the second component sequence, and a component execution log corresponding to the component information; the first language agent performs second regeneration optimization on the second sub-task sequence and the second component sequence according to the first historical task log to obtain an optimized response representation, the optimized response representation is used for recording a third sub-task sequence, a third component sequence and scene graph information corresponding to the target task instruction; a second language agent performs parameter inference on the optimized response representation according to the second historical task log to obtain a target task workflow, the target task workflow is used for representing the third component sequence and a component parameter corresponding to each task component of the third component sequence. The assembly method based on the mutual cooperation between the first language agent, the component agent, the retrieval agent and the second language agent, specifically through the component agent performing component retrieval on the sub-task sequence, can reduce the influence of irrelevant components on the generation quality of the task workflow, effectively avoiding the phenomenon of language agent illusion; in addition, the historical task log obtained by the retrieval agent querying the task log corresponding to the sub-task sequence and the component sequence, based on the language agent inferring and predicting the component sequence and the parameters of the target task workflow, can effectively reduce the development time of the task workflow, reduce the work intensity of the staff, and improve the generation quality of the task workflow. BRIEF DESCRIPTION OF DRAWINGS

[0055] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following introduces the drawings of the related technical solutions in the embodiments of the present application or the prior art. It should be understood that the drawings in the following introduction are only for the convenience of expressing part of the embodiments of the technical solutions of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the premise that there is no creative labor.

[0056] Figure 1 A flowchart of a multi-agent driven industrial software component assembly method provided by an embodiment of the present application is shown in the figure.

[0057] Figure 2 A structural framework diagram of a multi-agent driven industrial software component assembly system provided by an embodiment of the present application is shown in the figure.

[0058] Figure 3 A structural diagram of an electronic device provided by an embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0059] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the drawings, wherein the same or similar reference numerals represent the same or similar elements or elements with the same or similar functions throughout. The embodiments described below by referring to the drawings are exemplary and are only used to explain the present application, and cannot be understood as a limitation of the present application. For the step numbers in the following embodiments, they are only set for the convenience of explanation and description, and the order between the steps is not limited in any way, and the execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0060] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0061] In the industrial software scenario, the traditional development method is usually to manually write and configure a large number of repetitive code structures and interaction logic between various components, so as to realize the generation of task workflow. This kind of method requires a lot of development time and high personnel work intensity.

[0062] In addition, there is a development mode based on a large language model (LLM), which is specifically based on pre-collected fine-tuning data to fine-tune the large language model, and then generates a task workflow through the fine-tuned large language model. However, in practice, the data quality and size of the fine-tuning data are often limited, and the fine-tuned large language model often cannot well understand specific tasks when facing novel or complex task scenarios, and the generalization ability is insufficient, and the generation quality of the task workflow is not satisfactory. In addition, when the generated task workflow does not meet the expectations, manual intervention is often required to redo or further iterate and optimize, and it lacks an effective feedback adjustment mechanism.

[0063] Therefore, the embodiment of the present application provides a multi-agent driven industrial software component assembly method, which is based on the mutual cooperation between a first language agent, a component agent, a retrieval agent and a second language agent. Specifically, the component agent performs component retrieval on the sub-task sequence, which can reduce the influence of irrelevant components on the generation quality of the task workflow and effectively avoid the illusion phenomenon of the language agent. In addition, the assembly method queries the historical task log of the task log corresponding to the sub-task sequence and the component sequence through the retrieval agent, and performs inference and prediction on the component sequence and the parameters of the target task workflow based on the language agent, which can effectively reduce the development time of the task workflow, reduce the work intensity of the workers, and improve the generation quality of the task workflow. In addition, the assembly method further refines the execution information of the failed target task workflow based on the reflection agent, and optimizes the optimization response representation and the target task workflow based on the refined execution summary, which can improve the adaptability of the target task workflow to complex or novel tasks and improve the generation quality of the target task workflow.

[0064] Reference Figure 1 In the embodiment of the present application, a multi-agent driven industrial software component assembly method comprises:

[0065] Step 110, the first language agent obtains the target task instruction, and performs instruction decomposition on the target task instruction to obtain a first sub-task sequence;

[0066] In some embodiments, the step 110 of decomposing the target task instruction to obtain a first sub-task sequence comprises:

[0067] A1, obtain a first prompt and an initial scene graph corresponding to the target task instruction, the first prompt being used to indicate the generation of a plurality of sub-tasks corresponding to the target task instruction;

[0068] A2, decompose the target task instruction according to the first prompt and the initial scene graph to obtain the first sub-task sequence.

[0069] In the embodiments of the present application, the first language agent, the component agent, the retrieval agent, the second language agent and the reflection agent can be single agents for implementing specific functions in a multi-agent system (Multi-Agent Systems) based on a large language model (LLM).

[0070] It can be understood that step 110 can specifically be that the user provides a complex target task instruction to the first language agent in the multi-agent system, and then decomposes the first prompt into a plurality of first sub-tasks according to the first prompt, and determines the first sub-task sequence according to all the first sub-tasks obtained by the decomposition.

[0071] It should be noted that for a robot task in an industrial software application scenario, the first language agent can first obtain a first prompt constructed based on a prompt engineering, and then mobilize an environment observation module, which can typically be a device with image shooting function, to obtain an RGB-D image (i.e., an initial scene graph) corresponding to the target task instruction; then, a decomposition prompt text of the first language agent is constructed according to the first prompt and the initial scene graph, and the target task instruction is decomposed by the first language agent to obtain a plurality of executable first sub-tasks, which can be execution operations planned by the robot to achieve the target task instruction.

[0072] Exemplarily, one of the decomposition prompt texts of the first language agent in the embodiments of the present application can be: # First language agent task description. You are the high-level planner of the industrial software workflow. Your task is to decompose complex user instructions into a series of executable sub-tasks, each of which can be completed independently by a specific component. User instructions may involve multiple steps and conditions, which need to be decomposed into simple and understandable sub-tasks, ensuring that each sub-task is clearly expressed, independently executable, and functionally self-consistent for execution by a robot or software component. For a robot task, you will also accept the environment state, please combine the task instruction and the environment state to generate a sub-task sequence, and the related objects and their expected states (such as positions) corresponding to the sub-task sequence. The sub-tasks should have clear goals, actions and descriptions of expected movements, so that the robot or system can understand and execute them.

[0073] Step 120, the component agent retrieves the first sub-task sequence according to the target task instruction to obtain a first component sequence;

[0074] In some embodiments, the step 120 of performing component retrieval on the first subtask sequence according to the target task instruction to obtain a first component sequence comprises:

[0075] B1, obtaining a preset component library;

[0076] B2, performing component similarity calculation on the first subtask sequence according to the component library and the target task instruction to obtain the first component sequence.

[0077] In the embodiments of the present application, the component library records a plurality of task components for executing subtasks; after the first language intelligent agent generates the first subtask sequence, it can send the first subtask sequence and the obtained target task instruction to the component intelligent agent. For a certain first subtask in the first subtask sequence, step B2 can be to obtain the task description corresponding to the first subtask in the target task instruction; then input the first subtask and the task description as a retrieval prompt text into the component intelligent agent, so that the component intelligent agent performs similarity calculation on each task component in the component library based on the prompt text. The specific similarity calculation method can be any one of cosine similarity, Euclidean distance, etc. By comparing the similarity between the first subtask and the task component, and the similarity between the task description and the task component description, the task component matched with the first subtask is determined.

[0078] It can be understood that the remaining first subtasks in the first subtask sequence can be simply analogized in the same way. There are many ways to construct the retrieval prompt text, for example, it can be analogized simply by referring to the aforementioned decomposition prompt text, which will not be repeated here. After obtaining the task component corresponding to each first subtask, the first component sequence can be determined based on all the obtained task components. It is worth mentioning that the embodiments of the present application retrieve the task component related to the first subtask from the component library based on the component intelligent agent, which can reduce the phenomenon of too long prompt text caused by too many irrelevant components, avoid the situation that the prompt text exceeds the input upper limit of the large language model, and at the same time can reduce the influence of irrelevant components on the quality of the target task workflow, effectively reducing the possibility of hallucination of the large language model.

[0079] Step 130, the first language intelligent agent performs first regeneration optimization on the first subtask sequence and the first component sequence according to the target task instruction to obtain a second subtask sequence and a second component sequence, each subtask in the second subtask sequence corresponds to a task component in the second component sequence;

[0080] In the embodiment of the present application, after the component intelligent agent retrieves the first component sequence matching the first subtask sequence, the component intelligent agent can send the first component sequence to the first language intelligent agent. The first regeneration optimization in step 130 can be to construct a first optimization prompt text from the target task instruction, the first subtask sequence and the first component sequence, and input the first optimization prompt text into the first language intelligent agent to generate the component execution order and other relationships of each task in the first component sequence through the first language intelligent agent, so as to obtain the second subtask sequence and the second component sequence with the component execution order and other relationships.

[0081] For example, the first optimization prompt text in the embodiment of the present application can be obtained by adding "clearly indicating the mapping relationship between each subtask and the corresponding component in the result" on the basis of the aforementioned decomposition prompt text. The specific construction method of the first optimization prompt text can be various, and the examples of the present application do not limit the first optimization prompt text.

[0082] Step 140, the retrieval intelligent agent queries the task log corresponding to the second subtask sequence and the second component sequence to obtain the first historical task log and the second historical task log. The first historical task log is used to record the historical task instruction, the historical subtask sequence and the historical component sequence corresponding to the historical task instruction. The second historical task log is used to record the component information of each task component of the second component sequence and the component execution log corresponding to the component information.

[0083] In some embodiments, the step 140 of querying the task log corresponding to the second subtask sequence and the second component sequence to obtain the first historical task log and the second historical task log comprises:

[0084] C1, obtaining a preset knowledge graph database and a knowledge vector database;

[0085] C2, according to the second subtask sequence and the second component sequence, at least one of the knowledge graph database and the knowledge vector database is first queried to obtain a target historical log. The target historical log is the first historical task log and the second historical task log.

[0086] In the embodiment of the present application, after the first language intelligent agent generates the second subtask sequence and the second component sequence, the second subtask sequence and the second component sequence can be sent to the retrieval intelligent agent. Step 140 can be constructing a log retrieval prompt text according to the second subtask sequence and the second component sequence. The log retrieval prompt text can be constructed in various ways, which are not limited in the present application. Then, the log retrieval prompt text is input into the retrieval intelligent agent, so that the retrieval intelligent agent generates corresponding first historical task logs and second historical logs from the vector data and / or the knowledge graph database, which records a plurality of historical task logs.

[0087] It can be understood that for step C1, after the retrieval intelligent agent receives the log retrieval prompt text, the first search method can be a knowledge graph search method based on the retrieval intelligent agent. The knowledge graph information in the knowledge graph database is extracted, and then the obtained knowledge graph information is converted into a knowledge graph prompt, so that the retrieval intelligent agent can generate corresponding historical task logs based on the knowledge graph prompt. Specifically, the data structure of the knowledge graph database is based on the form of a graph, which can be represented by nodes and edge types. The nodes include target user instruction nodes, subtask nodes, component nodes, etc. The target user instruction nodes are used to represent natural language instructions input by the user. The subtask nodes are used to represent specific subtasks corresponding to the target user instructions. The component nodes are used to represent task components and task component parameters corresponding to the subtasks.

[0088] It should be noted that the edge types include target user instruction to subtask edges, subtask to task component edges, component to component edges, different target user instruction edges, subtask to subtask edges, etc. The target user instruction to subtask edges are used to represent a plurality of subtasks corresponding to a target user instruction. Each edge under this edge type has a serial number, which is used to represent the execution order of the corresponding subtask under the target user instruction. The subtask to task component edges are used to represent the components and component parameters required to complete a subtask, including parameters, inputs, outputs, etc. The component to component edges are used to represent the possible relationships between component parameters, such as data flow, dependency or call relationship. The different target user instruction edges are used to represent the relationships between different target user instructions, such as sequential relationship, context dependency relationship, correction relationship, etc. The subtask to subtask edges are used to represent the relationships between subtasks, such as sequential execution relationship, parallel execution relationship, data dependency relationship, etc.

[0089] It should be noted that for step C1, after receiving the log retrieval prompt text, the retrieval agent can extract knowledge vector information in the knowledge vector database in a second search mode based on vector similarity search, which can be an approximate nearest neighbor (ANN) algorithm, etc., and then convert the obtained knowledge vector information into a knowledge vector prompt, so that the retrieval agent can generate a corresponding historical task log based on the knowledge vector prompt. The third search mode can extract knowledge vector information in the knowledge vector database and knowledge graph information in the knowledge graph database by using the GraphRAG technology based on the retrieval agent, and then integrate and convert the obtained knowledge vector information and knowledge graph information into a knowledge prompt, so that the retrieval agent generates a corresponding historical task log based on the knowledge prompt, thereby obtaining the first historical task log and the second historical task log. In addition, in actual application, the search mode used by the retrieval agent each time can be determined by the retrieval agent based on the information level requirement corresponding to the log retrieval prompt text, which is not described herein.

[0090] It is worth mentioning that step C2 can be first clustering and dividing the knowledge graph database into several entity communities of different granularity levels by using a community detection algorithm such as the Leiden algorithm, and then using the GraphRAG technology based on graph retrieval enhancement generation to perform global log query on the entity communities, thereby finding the first historical task log and the second historical task log corresponding to the second sub-task sequence and the second component sequence. The first historical task log is used to represent the same or similar log data as the second sub-task sequence and the second component sequence, that is, the first historical task log records historical task instructions, a historical sub-task sequence and a historical component sequence; the second historical task log is used to record component information and component execution logs corresponding to each task component of the second component sequence.

[0091] It should be noted that the embodiments of the present application can better assist the planning and execution of sub-tasks based on the retrieval agent obtaining the first historical task log and the second historical task log corresponding to the target task instruction, the second sub-task sequence, the second component sequence, etc., so as to obtain a target task workflow with better generation quality; in addition, the embodiments of the present application can obtain the knowledge graph database through the retrieval agent, can realize semantic association across work logs through the knowledge graph, discover implicit relationships, form a more comprehensive log set, and can improve the accuracy and practicality of the first historical task log matching.

[0092] In step 150, the first language agent performs second regeneration optimization on the second subtask sequence and the second component sequence according to the first historical task log, to obtain an optimized response representation, which is used to record a third subtask sequence, a third component sequence, and scene graph information corresponding to the target task instruction.

[0093] In the embodiments of the present application, after the retrieval agent generates the first historical task log, the retrieval agent can send the first historical task log to the first language agent. Step 150 can be to construct a log prompt text according to the first historical log, so that the first language agent reprocesses and optimizes the execution order of the second subtask sequence and the execution order of the second component sequence based on the log prompt text, so as to improve the quality of the response output by the first language agent, and obtain an optimized response representation, which includes a third subtask sequence, a third component sequence, and scene graph information used to indicate the environment and system state when each subtask of the third subtask sequence is executed.

[0094] In step 160, the second language agent performs parameter inference on the optimized response representation according to the second historical task log, to obtain a target task workflow, which is used to represent the third component sequence and the component parameters corresponding to each task component of the third component sequence.

[0095] In some embodiments, step 160 includes performing parameter inference on the optimized response representation according to the second historical task log, to obtain a target task workflow, which includes:

[0096] D1, obtaining a preset second prompt, which is used to indicate the acquisition method of the component parameters corresponding to each task component of the third component sequence;

[0097] D2, performing parameter prediction on the third component sequence according to the second prompt, the second historical task log, the third subtask sequence, and the scene graph information, to obtain the target task workflow.

[0098] In the embodiments of the present application, the first prompt is used to indicate the method of acquiring the component parameters of each task component, which can be “There are multiple sources of component parameters: ①inference from user instruction context; ②dependent on the output of the previous component; ③ask the user”, which is only for illustration.

[0099] It can be understood that step D2 can be constructing a parameter inference prompt text based on the second prompt, the second historical task log, the third sub-task sequence and the scene graph information, and then inferring, by the second language intelligent agent, the parameter type and description of each task component in the third component sequence based on the parameter inference prompt text to obtain the target task workflow.

[0100] Exemplarily, the parameter inference prompt text in the embodiments of the present application can be specifically "# second language intelligent agent task description. You are a mechanical hand with a mobile base. Given user instructions, environment and system information and logs, you need to optimize and refine the matching of each sub-task and component, and generate the corresponding parameters. You need to give reasons for the selected parameters. There are various sources of parameters: ①inference from user instruction context; ②dependent on the output of the previous component; ③ask the user. # Robot constraints. You can only grab one object at a time".

[0101] In some embodiments, the assembly method further comprises:

[0102] E1, the reflection intelligent agent acquires execution data corresponding to the target task workflow, the execution data comprising execution results and execution information, the execution results being used to indicate whether a target task component in the target task workflow is successfully executed;

[0103] In the embodiments of the present application, after the second language intelligent agent generates the target task workflow, the platform can gradually execute the component actions of the target task workflow and record the execution situation, thereby generating execution data corresponding to the target task workflow and sending it to the reflection intelligent agent. The target task component can be any task component in the target task workflow.

[0104] E2, if the execution result is execution failure, the reflection intelligent agent acquires a third historical task log corresponding to the target task workflow, and performs information extraction on the execution information according to the third historical task log to obtain an execution summary;

[0105] Further, step E2, performing information extraction on the execution information according to the third historical task log to obtain an execution summary, comprises:

[0106] E21, acquiring an expected scene graph, a first scene graph and a second scene graph corresponding to the execution result, the first scene graph being an actual scene graph corresponding to the target task component, and the second scene graph being a previous frame scene graph of the first scene graph;

[0107] E22, performing execution deviation reflection on the first scene graph according to the expected scene graph to obtain a scene reflection result;

[0108] E23, performing event reflection on the first scenario graph according to the second scenario graph, to obtain an event reflection result;

[0109] E24, performing summary and extraction on the scenario reflection result and the event reflection result according to the third historical task log, to obtain the execution summary.

[0110] In the embodiments of the present application, if the execution result is execution failure, the reflection agent can first obtain a third historical task log corresponding to the target task workflow, the third historical task log being used for log data identical or similar to the third component sequence and the component parameters, the log data recording a historical component sequence identical or similar to the third component sequence, and the component parameters and the log of execution success or failure corresponding to the historical component sequence; then the reflection agent can perform information extraction on the execution information based on the obtained third historical task log. Specifically, the expected scenario graph in step E1 can be an expected environment state corresponding to the target task component with execution failure; when the execution result of the target task component is execution failure, the first scenario graph can be an actual environment state corresponding to the execution failure of the target task component; and the second scenario graph can be a previous frame scenario graph of the first scenario graph.

[0111] It can be understood that the execution deviation reflection of step E22 can be to calculate the deviation between the actual environment state represented by the first scenario graph and the expected environment state represented by the expected scenario graph, so as to obtain a scenario reflection result, the scenario reflection result being used to represent the deviation between the execution result of the subtask corresponding to the target task component and the task planning; and the event reflection of step E23 is the same, and the event reflection result is used to provide the task event information when the fault is sent, which will not be described herein.

[0112] It should be noted that step E24 can be based on the third historical task log, constructing reflection prompts from the event reflection result and the scene reflection result and inputting to the reflection intelligent agent, summarizing the above two reflection results by the reflection intelligent agent, so as to obtain an execution summary in the form of a triple (error type, reason, suggestion), so that the reflection intelligent agent thinks and obtains whether the failure or success of the execution of the subtask corresponding to the component is caused by the failure to meet the component condition based on the component sequence in the third historical task log and the log of the execution success or failure, combined with the scene reflection result and the event reflection result; in addition, the reflection intelligent agent can also think and obtain the reason for the failure or success of the execution of the subtask corresponding to the component caused by the existence of specific use restrictions of the component (such as the frequency, power consumption, temperature, etc. of the component corresponding to the component in the actual execution environment) based on the component parameters in the third historical task log and the log of the execution success or failure, combined with the scene reflection result and the event reflection result. After the reflection intelligent agent thinks and obtains the reason for the failure or success of the execution of the subtask corresponding to the component, the execution summary can be determined based on the obtained component condition reason and component limitation or constraint reason.

[0113] Exemplarily, the reflection prompt can be "On the premise that you already know the component and the execution result, please explain the task failure. The task name is [TASK_NAME]. In the [k]th round, an error occurred, causing the task execution to fail. Please judge the error type, explain the error reason, and give improvement suggestions based on the provided historical task log, event reflection result, and scene reflection result. Please output the following information in the form of a triple (error type, reason, suggestion): Error type: identify the specific error type in task execution, such as semantic error, syntax error, logical error, physical error (such as motion obstruction), program running error (such as communication error), planning error (such as LLM hallucination, task component selection error, component parameter error), etc. Reason: explain the root cause of the error in detail, including component configuration problem, incorrect input parameter, execution environment not meeting expectations, etc. Suggestion: propose targeted improvement suggestions to solve the current error and optimize subsequent task execution, including adjusting component parameters, modifying task planning, improving execution strategy, etc.".

[0114] E3、the first language intelligent agent performs third regenerative optimization on the optimization response representation based on the execution summary and the target task instruction, to obtain a fourth subtask sequence;

[0115] In the embodiments of the present application, the planning repair prompt instruction can be constructed according to the execution summary and the target task instruction, and the first language intelligent agent can perform re-reasoning on the subtask planning in the third subtask sequence in the optimized response representation based on the planning repair prompt instruction, so as to obtain a fourth subtask sequence representing the adjusted subtask planning, and the planning adjustment includes subtask adjustment and subtask execution order adjustment, etc. Specifically, the subtasks in the fourth subtask sequence may be the same as or different from the subtasks in the third subtask sequence. For example, the subtasks in the third subtask sequence are task A, task B, task C and task D, and the subtasks in the fourth subtask sequence are task A, task B, task D and task E. The present examples are for illustration only.

[0116] E4, the second language intelligent agent performs repair reasoning on the target task workflow according to the execution summary and the fourth subtask sequence, to obtain a repaired target task workflow.

[0117] Further, the step E4, performing repair reasoning on the target task workflow according to the execution summary and the fourth subtask sequence, to obtain a repaired target task workflow, comprises:

[0118] E41, obtaining a fourth component sequence corresponding to the fourth subtask sequence, and determining a corresponding fourth historical task log according to the fourth component sequence;

[0119] Further, the fourth historical task log in the step E41 is obtained by the following steps:

[0120] E411, performing component screening on the fourth component sequence according to the third component sequence, to obtain an updated task component, so that the second language intelligent agent calls the retrieval intelligent agent to perform second log inquiry on the updated task component, to obtain the fourth historical task log.

[0121] In the embodiments of the present application, after obtaining the fourth component sequence corresponding to the fourth subtask sequence, the component screening in the step E411 can first obtain the common task component between the fourth component sequence and the third component sequence, and then screen the fourth component sequence based on the common task component, to determine the remaining task components of the fourth component sequence except the common task component as the updated task component.

[0122] It can be understood that the second log inquiry of the retrieval intelligent agent on the updated task component is similar to the content of the aforementioned first log inquiry, and can be simply analogized to obtain the fourth historical task log corresponding to the fourth component sequence, which is not described herein.

[0123] E42, according to the execution summary, the fourth historical task log, the fourth subtask sequence and the fourth component sequence, parameter repairing is performed on the target task workflow, and a repaired target task workflow is obtained.

[0124] In the embodiment of the application, step E42 can specifically be that a parameter repairing prompt text is constructed according to the execution summary, the fourth historical task log, the fourth subtask sequence and the fourth component sequence. The parameter repairing prompt text can be constructed in various ways, so that the second language intelligent agent optimizes and regenerates the task components and component parameters of the target task workflow, thereby completing parameter repairing of the target task workflow and obtaining a repaired target task workflow.

[0125] It can be understood that if the execution data corresponding to the repaired target task workflow still has a failure execution result, steps E1 to E4 can be performed again, which can improve the adaptability of the multi-agent to novel or complex task scenarios and effectively guarantee the generation quality of the target task workflow.

[0126] A multi-agent driven industrial software component assembly system is described in detail below with reference to the accompanying drawings according to an embodiment of the application.

[0127] Referring to Figure 2 The multi-agent driven industrial software component assembly system according to the embodiment of the application includes a first language intelligent agent 101, a component intelligent agent 102, a retrieval intelligent agent 103 and a second language intelligent agent 104.

[0128] The first language intelligent agent is configured to obtain a target task instruction, perform instruction decomposition on the target task instruction, obtain a first subtask sequence, perform first regeneration optimization on the first subtask sequence and a first component sequence according to the target task instruction, obtain a second subtask sequence and a second component sequence, each subtask in the second subtask sequence corresponding to a task component in the second component sequence, and perform second regeneration optimization on the second subtask sequence and the second component sequence according to a first historical task log, to obtain an optimized response representation, the optimized response representation being used to record a third subtask sequence, a third component sequence and scene graph information corresponding to the target task instruction.

[0129] The component intelligent agent is configured to perform component retrieval on the first subtask sequence according to the target task instruction, and obtain the first component sequence.

[0130] A retrieval agent is used to query the task log corresponding to the second subtask sequence and the second component sequence to obtain the first historical task log and the second historical task log, wherein the first historical task log is used to record historical task instructions and the historical subtask sequence and historical component sequence corresponding to the historical task instructions; the second historical task log is used to record component information of each task component of the second component sequence, and a component execution log corresponding to the component information;

[0131] The second language agent is used to perform parameter reasoning on the optimized response representation based on the second historical task log to obtain a target task workflow, where the target task workflow is used to characterize the third component sequence and the component parameters corresponding to each task component of the third component sequence.

[0132] It can be understood that the contents of the above method embodiments are all applicable to the present system embodiments, the functions specifically implemented by the present system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0133] Reference Figure 3 , an embodiment of the present application further provides an electronic device, including:

[0134] at least one processor 201;

[0135] At least one memory 202, configured to store at least one program;

[0136] When the at least one program is executed by the at least one processor 201 , the at least one processor 201 implements the above method embodiment.

[0137] Similarly, it can be understood that the contents of the above method embodiments are applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0138] An embodiment of the present application further provides a computer-readable storage medium, in which a program executable by the processor 201 is stored. The program executable by the processor 201 is used to implement the above-mentioned method embodiment when executed by the processor 201.

[0139] Similarly, the contents of the above method embodiments are applicable to the computer-readable storage medium embodiments. The functions specifically implemented by the computer-readable storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0140] In alternative embodiments, the functions / operations in the flow diagrams can occur in sequences other than those depicted. For example, two operations shown in succession can in fact be executed substantially concurrently or the operations sometimes can be executed in the reverse order depending upon the functionality / operations involved. Also, embodiments presented and described in this application are provided by way of example only. The

[0141] Moreover, while this application has been described in the context of functional modules, it is to be understood that one or more of the functions and / or features can be integrated in a single physical device and / or software module or one or more functions and / or features can be implemented in separate physical devices or software modules. It will also be appreciated that detailed discussion of the actual implementation of each module is not necessary for an understanding of the application. Rather, the actual implementation is a matter of choice apart from the conception and the disclosure provided that the advantages of the embodiments are maintained. The skilled artisan will appreciate that the functionality of the various modules disclosed herein can be carried out in different ways and using different hardware and / or software elements. Further, it is expected that the disclosure is implemented only after a successful determination has been made as to the feasibility of the implementation and mode of operation thereof. Accordingly, details concerning the implementation of the modules are not provided unless relevant to an understanding of the present application.

[0142] If the functions are implemented in software, the functions can be stored in or implemented as one or more computer program products. The computer program product can be stored in a computer readable storage medium, which is not a signal. The computer readable storage medium can include, but is not limited to, RAM, ROM, electrically erasable programmable read only memory (EEPROM), flash memory, compact disk read-only memory (CD-ROM), digital versatile disks (DVDs), and other types of memory. The computer program product can be distributed in whole or in part over one or more computer readable storage media.

[0143] The logic and / or steps represented in the flow diagrams or otherwise described herein, for example, can be embodied in non-transitory computer-readable media in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor- containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions, or in conjunction with which the instructions can be executed. For purposes of this specification, a "computer-readable medium" can be any apparatus that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. The computer-readable medium can be, for example but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium include the following: an electrical connection having one or more wires (electrical connections), a portable computer diskette (magnetic), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium can also be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, for example via optical scanning of the paper or other medium, then compiled, interpreted, or otherwise processed in a suitable manner, if necessary, and stored in a computer memory.

[0144] More specific examples (a non-exhaustive list) of the computer-readable medium include the following: an electrical connection having one or more wires (electrical connections), a portable computer diskette (magnetic), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium can also be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, for example via optical scanning of the paper or other medium, then compiled, interpreted, or otherwise processed in a suitable manner, if necessary, and stored in a computer memory.

[0145] It should be understood that aspects of the present application can be implemented in hardware, software, firmware or combinations thereof. In the above described embodiments, the various steps or methods can be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, and in another embodiment, any of the following technologies, known in the art, or combinations thereof, can be used: discrete logic circuitry having logic gates for implementing logic functions upon an application of data signals, application specific integrated circuits having appropriate combinational logic gates, programmable gate arrays (PGA), field programmable gate arrays (FPGA), and the like.

[0146] In the above description of the present specification, the description using the terms "one embodiment / one example", "another embodiment / another example" or "some embodiments / some examples" and the like means that the relevant described specific features, structures, materials or characteristics are included in at least one embodiment or example of the present application. In the present specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Also, the described specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in an appropriate manner.

[0147] While the embodiments of the application have been shown and described, it is to be understood that the embodiments can be varied, modified, substituted and changed by those skilled in the art without departing from the principles and spirit of the application, the scope of which is defined by the claims and their equivalents.

[0148] The above is a specific description of the preferred embodiments of the application, but the application is not limited to the embodiments, and those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the application, and these equivalent modifications or substitutions are all included in the scope defined by the claims of the application.

Claims

1. A multi-agent driven industrial software component assembly method, characterized in that: include: The first language agent obtains a target task instruction and decomposes the target task instruction to obtain a first subtask sequence; The component agent performs component retrieval on the first subtask sequence according to the target task instruction to obtain a first component sequence; The first language agent performs a first regenerative optimization on the first subtask sequence and the first component sequence according to the target task instruction to obtain a second subtask sequence and a second component sequence, wherein each subtask in the second subtask sequence corresponds to a task component in the second component sequence; The retrieval agent queries the task log corresponding to the second subtask sequence and the second component sequence to obtain a first historical task log and a second historical task log, wherein the first historical task log is used to record historical task instructions and historical subtask sequences and historical component sequences corresponding to the historical task instructions, and the second historical task log is used to record component information of each task component of the second component sequence and a component execution log corresponding to the component information; The first language agent performs a second regenerative optimization on the second subtask sequence and the second component sequence based on the first historical task log to obtain an optimized response representation, wherein the optimized response representation is used to record a third subtask sequence, a third component sequence, and scene graph information corresponding to the target task instruction, wherein the scene graph information is used to indicate an environment and system state when each subtask of the third subtask sequence is executed; The second language agent performs parameter reasoning on the optimized response representation based on the second historical task log to obtain a target task workflow, where the target task workflow is used to represent the third component sequence and component parameters corresponding to each task component of the third component sequence; The first language agent performs a first regeneration optimization on the first subtask sequence and the first component sequence according to the target task instruction to obtain a second subtask sequence and a second component sequence, including: Constructing an optimization prompt text according to the target task instruction, the first subtask sequence, and the first component sequence; Inputting the optimization prompt text into the first language agent to obtain the second subtask sequence and the second component sequence; The first language agent performs a second regeneration optimization on the second subtask sequence and the second component sequence according to the first historical task log to obtain an optimized response representation, including: Constructing a log prompt text according to the first historical task log; According to the log prompt text, the execution order of the second subtask sequence and the second component sequence is reprocessed and optimized by the first language agent to obtain the optimized response representation.

2. The assembly method according to claim 1, characterized in that: Decomposing the target task instruction to obtain a first subtask sequence includes: Obtaining a first prompt and an initial scene graph corresponding to the target task instruction, wherein the first prompt is used to instruct the generation of a plurality of subtasks corresponding to the target task instruction; The target task instruction is decomposed according to the first prompt and the initial scene graph to obtain the first subtask sequence.

3. The assembly method according to claim 1, wherein: The step of performing component retrieval on the first subtask sequence according to the target task instruction to obtain a first component sequence includes: Get the preset component library; Component similarity calculation is performed on the first subtask sequence according to the component library and the target task instruction to obtain the first component sequence.

4. The assembly method according to claim 1, characterized in that: The querying of the task log corresponding to the second subtask sequence and the second component sequence to obtain the first historical task log and the second historical task log includes: Obtain the preset knowledge graph database and knowledge vector database; According to the second subtask sequence and the second component sequence, a first log query is performed on at least one of the knowledge graph database and the knowledge vector database to obtain a target historical log, which is the first historical task log and the second historical task log.

5. The assembly method according to claim 1, wherein: The performing parameter reasoning on the optimized response representation according to the second historical task log to obtain a target task workflow includes: Obtaining a preset second prompt, where the second prompt is used to indicate a method for obtaining component parameters corresponding to each task component of the third component sequence; Parameters of the third component sequence are predicted based on the second prompt, the second historical task log, the third subtask sequence, and the scene graph information to obtain the target task workflow.

6. The assembly method according to claim 1, characterized in that: The assembly method further comprises: The reflective agent obtains execution data corresponding to the target task workflow, wherein the execution data includes an execution result and execution information, and the execution result is used to indicate whether the target task component in the target task workflow is successfully executed; If the execution result is execution failure, the reflective agent obtains a third historical task log corresponding to the target task workflow, and extracts information from the execution information based on the third historical task log to obtain an execution summary; The first language agent performs a third regenerative optimization on the optimized response representation according to the execution summary and the target task instruction to obtain a fourth subtask sequence; The second language agent performs repair reasoning on the target task workflow according to the execution summary and the fourth subtask sequence to obtain a repaired target task workflow.

7. The assembly method according to claim 6, characterized in that: The step of performing repair reasoning on the target task workflow according to the execution summary and the fourth subtask sequence to obtain a repaired target task workflow includes: Acquire a fourth component sequence corresponding to the fourth subtask sequence, and determine a corresponding fourth historical task log according to the fourth component sequence; According to the execution summary, the fourth historical task log, the fourth subtask sequence, and the fourth component sequence, parameters of the target task workflow are repaired to obtain the repaired target task workflow.

8. The assembly method according to claim 6, characterized in that: The step of extracting the execution information based on the third historical task log to obtain an execution summary includes: Obtaining an expected scene graph, a first scene graph, and a second scene graph corresponding to the execution result, wherein the first scene graph is an actual scene graph corresponding to the target task component, and the second scene graph is a scene graph of a previous frame of the first scene graph; Performing execution deviation reflection on the first scene graph according to the expected scene graph to obtain a scene reflection result; Performing event reflection on the first scene graph according to the second scene graph to obtain an event reflection result; The scenario reflection results and the event reflection results are summarized and refined according to the third historical task log to obtain the execution summary.

9. The assembly method according to claim 7, characterized in that: The fourth historical task log is obtained by the following steps: According to the third component sequence, the fourth component sequence is subjected to component screening to obtain an update task component, so that the second language agent calls the retrieval agent to perform a second log query on the update task component to obtain the fourth historical task log.

10. A multi-agent driven industrial software component assembly system, characterized in that: It includes first language agent, component agent, retrieval agent and second language agent; The first language agent is used to obtain a target task instruction and perform instruction decomposition on the target task instruction to obtain a first subtask sequence; according to the target task instruction, the first subtask sequence and the first component sequence are subjected to a first regeneration optimization to obtain a second subtask sequence and a second component sequence, wherein each subtask in the second subtask sequence corresponds to a task component in the second component sequence; according to the first historical task log, the second subtask sequence and the second component sequence are subjected to a second regeneration optimization to obtain an optimized response representation, wherein the optimized response representation is used to record a third subtask sequence, a third component sequence and scene graph information corresponding to the target task instruction, wherein the scene graph information is used to indicate the environment and system status when each subtask of the third subtask sequence is executed; A component agent, configured to perform component retrieval on the first subtask sequence according to the target task instruction to obtain the first component sequence; A retrieval agent is used to query the task log corresponding to the second subtask sequence and the second component sequence to obtain the first historical task log and the second historical task log, wherein the first historical task log is used to record historical task instructions and the historical subtask sequence and historical component sequence corresponding to the historical task instructions; the second historical task log is used to record component information of each task component of the second component sequence, and a component execution log corresponding to the component information; a second language agent, configured to perform parameter reasoning on the optimized response representation based on the second historical task log to obtain a target task workflow, wherein the target task workflow is used to represent the third component sequence and component parameters corresponding to each task component of the third component sequence; The step of performing a first regeneration optimization on the first subtask sequence and the first component sequence according to the target task instruction to obtain a second subtask sequence and a second component sequence includes: Constructing an optimization prompt text according to the target task instruction, the first subtask sequence, and the first component sequence; Inputting the optimization prompt text into the first language agent to obtain the second subtask sequence and the second component sequence; The performing a second regeneration optimization on the second subtask sequence and the second component sequence according to the first historical task log to obtain an optimized response representation includes: Constructing a log prompt text according to the first historical task log; According to the log prompt text, the execution order of the second subtask sequence and the second component sequence is reprocessed and optimized by the first language agent to obtain the optimized response representation.

Images