Large language model agent based on verification cycle industrial software task execution method, system, electronic equipment and storage medium

Abstract

The application discloses a kind of big language model agent industrial software task execution method, system, electronic equipment and storage medium based on verification cycle, including receiving task target proposed in natural language, analysis is carried out by big language model agent, and task plan containing multiple execution steps is generated by searching enterprise business knowledge base;Each execution step is executed in sequence and is checked;If the check is not passed, the context information generated this time is returned to the task plan step to be re-planned, until all steps are checked through;The running state of task and each step is stored in structured record form persistently;Each step execution result is checked, errors can be found and repaired within the same task round, the running state is persistent, task planning integrates enterprise private business knowledge, significantly improves the reliability, recoverability, traceability and business fit of big language model agent in industrial software scene Complex task execution.

Description

Technical Field

[0001] This invention relates to the field of intelligent enterprise business development and management technology, and in particular to a method for executing industrial software tasks based on a large language model intelligent agent with a verification loop. It also relates to a system, electronic device and storage medium for implementing the method. Background Technology

[0002] With the rapid development of Large Language Model (LLM) technology, LLM-based agents have been widely used in the field of automated task execution, especially demonstrating great potential in the computer-aided design, data generation, and business execution of industrial software. However, existing LLM-based application solutions and agent architectures typically suffer from the following significant drawbacks when applied to industrial enterprise applications, limiting their reliability and practicality: The execution chain is unidirectional and lacks closed-loop feedback verification: Most existing technologies adopt a one-time end-to-end generation chain of "Prompt—Model—Response". In this mode, once the code, configuration or task plan generated by the model is produced, it is directly delivered to the downstream system for execution, lacking automatic feedback verification of the execution results. When an execution step fails due to environmental differences, data boundary conditions or logical errors, the system cannot automatically capture the exception and initiate repair within the current task round. This results in erroneous execution results directly contaminating downstream processes, or requiring manual intervention for secondary correction, indicating a low level of automation.

[0003] The operational state is tightly coupled with the dialogue, lacking controllability: Current agent operations typically exist only within the user's session context. Task execution state is lost when the browser is refreshed or the session is interrupted, making it impossible to resume progress for long-running tasks after interruption. Furthermore, this design pattern does not support task pauses, audit traceability, or manual approval intervention. For scenarios involving high-risk operations (such as modifying industrial parameters), existing solutions cannot suspend tasks before execution to await manual confirmation, posing significant operational risks.

[0004] General-purpose models lack support from enterprise-specific business knowledge: In industrial software application scenarios, enterprise process routes, data dictionaries, material coding rules, and business processes are typically highly proprietary and complex. General-purpose basic models lack an understanding of these enterprise-specific business rules, and the task plans, API call parameters, or data structures they generate are often disconnected from the enterprise's existing business logic. This results in extremely low usability of the generated plans, sometimes requiring extensive manual editing before they can be implemented.

[0005] Lack of state management leads to high operation and maintenance costs: Existing solutions lack a structured and persistent storage mechanism for the task execution process. Once a task fails, it is difficult for operation and maintenance personnel to quickly locate which step's input parameters were incorrect or which verification condition was not met through structured records. This "black box" execution process not only increases the difficulty of debugging but also makes it difficult for the retry mechanism to accurately recover at the breakpoint after failure.

[0006] To address the aforementioned issues, it is necessary to further improve the methods for executing industrial software tasks using large language models. Summary of the Invention

[0007] The technical problem to be solved by the present invention is to provide a method for executing industrial software tasks based on a large language model intelligent agent with verification loop, which can improve the reliability of complex task execution, and has recoverability, traceability and business fit.

[0008] To solve the above-mentioned technical problems, the technical solution of the present invention is: a method for executing industrial software tasks based on a large language model intelligent agent with a verification loop, comprising, S1. Receive the task target input by the user in natural language, call the large language model agent to perform intent parsing on the task target, and generate a task plan based on the intent parsing result. The task plan includes multiple execution steps arranged in sequence, and each execution step is associated with a capability interface to be called and the expected verification conditions corresponding to that step. S2. Execute multiple execution steps in sequence. For the current execution step, call its associated capability interface to perform the corresponding operation, call the expected verification condition corresponding to the current execution step, and collect the actual execution result of the current execution step. S3. Compare and verify the actual execution result with the expected verification conditions corresponding to the current execution step; S4. In response to the failure of the verification, the current context information, including the actual execution result, is carried back to the task plan generation stage or plan adjustment stage in step S1. The plan adjustment stage is where the large language model agent re-plans the subsequent execution steps based on the current context information and continues to execute step S2 to form a closed loop iteration. In response to successful verification, proceed to the next execution step until all execution steps have passed verification, and then output the task completion result. S5. During the execution of the task objective, the running status of the task objective, as well as the step status, input parameters, actual execution results, and verification records of each execution step, are persistently stored in the running status library in the form of structured records. The running status of the task objective includes at least five states: running, paused, pending approval, failed, and completed. The structured records are stored independently of user session messages and support the resumption of paused tasks, retrying failed execution steps, and suspending execution when the task is in the pending approval state and resuming execution after receiving an approval instruction.

[0009] As a preferred technical solution, the expected verification conditions in step S3 include at least one of the following. The structured output returned by the capability interface is matched and validated against the preset JSON Schema; The program code generated by the execution steps passes the verification of automated test cases running in an isolated execution environment; The correctness of the interface is verified by visual model analysis of the screenshot of the page generated by the execution steps after being rendered by the browser. Consistency verification between the output data of the execution steps and the business rule constraints.

[0010] As a preferred technical solution, the replanning in step S4 includes injecting the identifiers of the failed execution steps, the difference information between the actual execution results and the expected verification conditions, and the list of completed execution steps into the context of the large language model, and generating a corrected sequence of subsequent execution steps by the large language model agent; and when the number of replanning times for a single task reaches a preset threshold, the running status of the task is set to failure and the reason for failure and the retained intermediate products are output.

[0011] As a preferred technical solution, in the execution steps where the risk level meets the preset high-risk condition, the running status of the task is set to pending approval before execution, and the operation preview information of the step is displayed to the user; the corresponding operation is only executed by calling the capability interface associated with the step after receiving the user's confirmation and approval instruction. During the asynchronous execution of the steps, task progress events are sent to the user terminal in real time via server push event stream. The task progress events include at least plan creation, step start, approval request, step completion, and task completion events. After the user terminal refreshes or reconnects, the task progress is restored and displayed based on the structured records in the runtime status database.

[0012] As a preferred technical solution, the structured record in step S5 includes two layers: task record and step record. The task record includes task identifier, task target, current running status and creation information. The step record includes step identifier, belonging task identifier, step type, input parameters, actual execution result, verification result and timestamp. The resumption of execution includes reading all step records of the target task, determining the last execution step that passed verification, and continuing execution from the next execution step of that step.

[0013] As a preferred technical solution, the capability interface includes basic capability interfaces that are independent of business operations. The basic capability interfaces include one or more combinations of file system operations, command line execution, code execution, browser operations, database operations, network requests, and deployment operations. Each of the basic capability interfaces has a typed input / output mode definition, permission scope declaration, and timeout retry strategy.

[0014] As a preferred technical solution, during the asynchronous execution of the execution steps, the task progress events are sent to the user terminal in real time via a server push event stream. The task progress events include at least the plan creation, step start, approval request, step completion, and task completion events. After the user terminal refreshes or reconnects, the task progress is restored and displayed based on the structured records in the running status database.

[0015] As an improvement to the above technical solution, the intent parsing in step S1 further includes retrieving the business knowledge base of the enterprise to which the caller belongs. The business knowledge base stores the enterprise's business rules, data dictionary, process documents, historical project records, and knowledge entries precipitated from the historical execution products of the large language model agent. The retrieved business knowledge related to the task objective is injected into the planning context of the large language model, so that the data structure in the generated task plan and execution steps conforms to the enterprise's business rules. The business knowledge base is stored in isolation by enterprise tenant, and the business knowledge bases of different enterprises are not visible to each other. Moreover, the scope of knowledge retrieval inherits the access control permissions of the knowledge entries at the time of entry.

[0016] This invention also provides a large language model intelligent agent industrial software task execution system based on a verification loop for implementing the above-described method, comprising, The planning module is used to receive the task objective input by the user in natural language, call the large language model agent to perform intent parsing and generate a task plan. The task plan includes multiple execution steps arranged in sequence, and each execution step is associated with the capability interface to be called and the corresponding expected verification conditions. The planning module is also used to retrieve the tenant-isolated business knowledge base of the enterprise to which the caller belongs, and inject the retrieved business rules and data dictionary into the planning context of the large language model. The execution module is used to sequentially call the capability interfaces associated with each execution step to perform the corresponding operations and collect the actual execution results of each step; The verification module is used to compare and verify the actual execution results of each step with the corresponding expected verification conditions. The replanning module is used to trigger the large language model agent to replan subsequent execution steps when the verification fails, carrying the current context information, forming a closed-loop iteration of "planning-execution-observation-verification" until all execution steps pass the verification. The runtime status library is used to persistently store the runtime status of tasks and the step status of each execution step, input parameters, actual execution results and verification records in a structured record format independent of user session messages. It also supports task suspension, resumption, failure retry and manual approval suspension.

[0017] The present invention also provides an electronic device, including a processor and a memory, wherein the memory stores a computer program, and the computer program, when executed by the processor, implements the above-described method for executing industrial software tasks based on a large language model intelligent agent using a verification loop.

[0018] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the above-described method for executing industrial software tasks based on a large language model intelligent agent using a verification loop.

[0019] By adopting the above technical solution, the present invention has the following beneficial effects: Error self-correction within the same round, requiring no manual intervention: This invention constructs a closed-loop iterative mechanism of "planning-execution-verification-replanning" to automatically detect and correct execution deviations within the same task execution round. When the actual output of any execution step fails to meet the preset expected verification conditions, the difference information, error identifier, and context of the completed steps are automatically sent back to the large language model, triggering precise local replanning. This completely changes the one-off link mode in existing technologies, improving the execution success rate from passively depending on the initial generation quality of the model to approaching 100% through iterative convergence, significantly reducing the burden of manual operation and maintenance in industrial scenarios.

[0020] Tasks can be resumed across sessions, and long-running tasks are no longer lost due to refreshes: This invention persistently stores the task's running status, the step status of each execution step, input parameters, actual execution results, and verification records in a structured record format in a running status database. This structured record is completely independent of the user's session message storage, so task progress is not lost when the user closes the browser or refreshes the page; the execution state can be completely restored based on the records in the status database. If a task is suspended due to external dependency delays or manual approval, it can be accurately resumed at any time and on any terminal. Resumption continues from the next execution step after the last verified step, without starting from the beginning. Failed steps support targeted retries, eliminating the need to re-execute the entire task. This enables the automation of long-process tasks in industrial software that take hours or even days.

[0021] High-risk steps can be suspended for approval to meet industrial safety and compliance requirements: For high-risk operations in industrial software scenarios, such as parameter modification, data deletion, and deployment, a built-in manual approval suspension mechanism is provided in the execution chain. For execution steps that meet preset high-risk conditions, the system automatically sets the task running status to pending approval before execution and displays a preview of the operation information to the user (including the interfaces to be called, input parameters, and expected impact). Execution only occurs after receiving confirmation and approval from the user. This mechanism maintains a high degree of automation while retaining manual fallback capabilities at key decision-making nodes, enabling the automated execution solution to meet the rigid safety and compliance requirements of industrial enterprises.

[0022] Progress is visible in real time and refreshed without loss, ensuring a consistent user experience: Task progress events (including plan creation, step start, approval request, step completion, and task completion) are pushed to the user terminal in real time via server-side event streams, ensuring that users can perceive the task's progress status throughout the entire process. Since all status data is persistently stored in the runtime status database, even if the user terminal loses connection due to network fluctuations or active refresh, the system can still fully restore and display the task progress based on structured records after reconnection. This decouples real-time push and persistent storage, guaranteeing both the immediacy of monitoring and the durability of data.

[0023] Task planning integrates enterprise-specific business knowledge and ensures data security through tenant isolation: Addressing the core pain point of general-purpose large models in industrial scenarios—"understanding natural language but not business rules"—an enhanced enterprise business knowledge base retrieval mechanism is introduced. During the task planning phase, the system automatically retrieves the business knowledge base of the calling enterprise. This knowledge base stores the enterprise's business rules, data dictionary, process documents, historical project records, and knowledge entries accumulated from the historical execution output of the large language model agent. The retrieved content is then injected into the planning context of the large language model, ensuring that the data structure, parameter values, and process sequence in the generated task plan are strictly aligned with the enterprise's actual business specifications. The generated code or configuration can be implemented and executed in the enterprise's existing business systems without modification. More importantly, the business knowledge base is stored in an isolated manner by enterprise tenant, making the knowledge bases of different enterprises invisible to each other. Furthermore, the scope of knowledge retrieval strictly inherits the access control permissions of the knowledge entries at the time of entry, fundamentally guaranteeing the security and privacy of the enterprise's core business data while improving planning accuracy.

[0024] In summary, this invention systematically addresses the implementation bottlenecks faced by existing LLM intelligent agent solutions in industrial software scenarios from five dimensions: execution reliability, task recoverability, operational controllability, user experience continuity, and business knowledge fusion security. It has significant practical value and commercial application prospects. Attached Figure Description

[0025] The following figures are intended only to illustrate and explain the present invention and do not limit the scope of the invention. Wherein: Figure 1 This is a diagram illustrating step S1 using an energy management project as an example in an embodiment of the present invention; Figure 2 This is a partial illustration of the capability interface calls during the execution of the "AI Agent code modification" step in an energy management project, as an example, according to an embodiment of the present invention. Figure 3 This is a partial illustration of code modification during the execution of the "AI Agent code modification" step in an energy management project, as an example, according to an embodiment of the present invention. Figure 4 This is a partial illustration of the verification process using the "AI Agent code modification" step in an energy management project as an example in an embodiment of the present invention. Figure 5 This embodiment of the invention takes the execution step "AI Agent code modification" of an energy management project as an example, and each execution step has a corresponding verification diagram. Figure 6 This is a schematic diagram illustrating the operational status of an energy management project task objective according to an embodiment of the present invention; Figure 7This is a diagram illustrating how energy management projects are persistently stored in a runtime database in the form of structured records, according to an embodiment of the present invention. Figure 8 yes Figure 7 A diagram showing the specific content stored in a file. Figure 9 This is a partial step-by-step illustration of an energy management project, as an example, from an embodiment of the present invention; Figure 10 This is a diagram illustrating the failure of the execution steps in an embodiment of the present invention, taking an industrial production reporting platform project as an example. Figure 11 This is a diagram illustrating the continuation of execution after a failed execution step in an embodiment of the present invention, using an industrial production work reporting platform project as an example. Figure 12 This is a structural block diagram of the industrial software task execution system based on a large language model intelligent agent with a verification loop, according to an embodiment of the present invention. Figure 13 This is a structural block diagram of an electronic device that installs the system and executes the application method according to an embodiment of the present invention. Detailed Implementation

[0026] The present invention will be further described below with reference to the accompanying drawings and embodiments. In the following detailed description, only certain exemplary embodiments of the invention are described by way of illustration. Undoubtedly, those skilled in the art will recognize that various modifications can be made to the described embodiments without departing from the spirit and scope of the invention. Therefore, the drawings and description are illustrative in nature and not intended to limit the scope of the claims.

[0027] A method for industrial software task execution based on a large language model intelligent agent with a validation loop is applied to software generation and business execution in industrial enterprise information management, specifically including: S1. Receive the task objective input by the user in natural language, call the large language model agent to perform intent parsing on the task objective, and generate a task plan based on the intent parsing result. The task plan includes multiple execution steps arranged in sequence, and each execution step is associated with a capability interface to be called and the expected verification conditions corresponding to that step.

[0028] like Figure 1As shown in the diagram, specifically the content displayed within the red box, in the dialog box at the bottom left where "Continue to describe the content to be modified" is located, input "Make a simple energy system" in natural language and send and upload the information. Afterwards, the task objective "Make a simple energy system" will be formed and displayed above this column. After receiving the above task objective, the large language model agent generates five execution steps through intent parsing, including "Prepare the isolation workspace", "AI Agent modifies the code", "Independent inspection and production build", "Save Git Revision" and "Generate Preview".

[0029] The intent parsing in this step also includes retrieving the business knowledge base of the caller's company. This business knowledge base stores the company's business rules, data dictionary, process documents, historical project records, and knowledge entries accumulated from the historical execution output of the large language model agent. The retrieved business knowledge related to the task objective is injected into the planning context of the large language model, ensuring that the data structure in the generated task plan and execution steps conforms to the company's business rules. The business knowledge base is stored in isolation by company tenant; different companies' business knowledge bases are not visible to each other, and the knowledge retrieval scope inherits the access control permissions of the knowledge entries at the time of input. For example, if the natural language task objective "build a simple energy system" is entered in the dialog box, the large language model agent will automatically generate the execution steps based on the business knowledge base of the user's company.

[0030] S2. Execute multiple execution steps sequentially. For the current execution step, call its associated capability interface to perform the corresponding operation, call the expected verification conditions corresponding to the current execution step, and collect the actual execution result of the current execution step. See details for specific steps. Figures 2 to 4 As shown, the capability interfaces include basic capability interfaces that are independent of business logic. These basic capability interfaces include one or more combinations of file system operations, command line execution, code execution, browser operations, database operations, network requests, and deployment operations. Each basic capability interface has a typed input / output mode definition, permission scope declaration, and timeout retry policy.

[0031] Before execution, steps that meet the preset high-risk conditions will have their task status set to pending approval and will display a preview of the operation to the user. The corresponding operation will only be performed by calling the associated capability interface after receiving the user's confirmation and approval instruction.

[0032] Furthermore, during the asynchronous execution of the steps, the task progress events are sent to the user terminal in real time via a server-pushed event stream. These task progress events include at least the events of plan creation, step start, approval request, step completion, and task completion. After the user terminal refreshes or reconnects, the task progress is restored and displayed based on the structured records in the runtime status database.

[0033] S3. Compare and verify the actual execution result with the expected verification conditions corresponding to the current execution step, specifically as follows: Figure 4 and Figure 5 As shown.

[0034] The expected verification conditions described in this step include at least one of the following: The structured output returned by the capability interface is matched and validated against the preset JSON Schema; The program code generated by the execution steps passes the verification of automated test cases running in an isolated execution environment; The correctness of the interface is verified by visual model analysis of the screenshot of the page generated by the execution steps after being rendered by the browser. Consistency verification between the output data of the execution steps and the business rule constraints.

[0035] S4. In response to the failure of the verification, the current context information, including the actual execution result, is carried back to the task plan generation stage or plan adjustment stage in step S1. The plan adjustment stage is where the large language model agent re-plans the subsequent execution steps based on the current context information and continues to execute step S2 to form a closed loop iteration. In response to successful verification, proceed to the next execution step until all execution steps have passed verification, at which point the task completion result will be output.

[0036] The replanning in this step includes injecting the identifiers of execution steps that fail the verification, the difference information between the actual execution result and the expected verification conditions, and the list of completed execution steps into the context of the large language model. The large language model agent then generates a corrected sequence of subsequent execution steps. When the number of replanning attempts for a single task reaches a preset threshold, the running status of the task is set to failure, and the reason for failure and the retained intermediate products are output.

[0037] S5. During the execution of the task objective, the running status of the task objective, as well as the step status, input parameters, actual execution results, and verification records of each execution step, are persistently stored in the running status database in the form of structured records. For example... Figure 1 , Figures 3 to 5 As shown. Furthermore, the operational status of the task objective includes at least five states: running, paused, pending approval, failed, and completed, as shown. Figure 6As shown, based on the actual operation of the "energy system" of the constructed project, only two states are displayed: failure and completion.

[0038] The structured record is stored independently of user session messages and supports resuming paused tasks, retrying failed execution steps, and suspending tasks while they are in an approval pending state, resuming execution upon receiving an approval instruction. Specifically, as follows... Figure 7 and Figure 8 As shown.

[0039] The structured record mentioned in this step includes two layers: task record and step record. The task record contains the task identifier, task objective, current running status, and creation information, specifically as follows: Figure 1 and Figure 6 As shown, "You" is the task identifier, "Make a simple energy system" and "Continue to complete the unfinished task" are the task objectives, "Failed" and "Completed" are the current running status, and the creation information includes the time when the task objective was uploaded.

[0040] The step record includes a step identifier, a task identifier, a step type, input parameters, an actual execution result, a verification result, and a timestamp, as detailed below. Figure 9 As shown. The resumption of execution includes reading all step records of the target task, determining the last execution step that passed verification, and continuing execution from the next execution step after that step, such as... Figure 10 and Figure 11 As shown.

[0041] Example 1: This embodiment uses the scenario of "generating work reporting dashboards based on natural language descriptions" commonly found in manufacturing workshops as an example, and fully demonstrates the entire process from user task target input to task completion, from step S1 to step S5.

[0042] S1. Intent Analysis and Plan Generation – Knowledge Injection Phase Use Case: A production manager at a machining company inputs the following natural language into the system: "Generate a workshop work report dashboard page to display the planned output and actual output of each process this week. The page should be grouped by work order number and presented using a bar chart. The data should be read from the work order table in the MES system." This forms the task objective.

[0043] Simultaneously, the large language model AI will obtain the enterprise identifier (e.g., enterprise ID "ZJ_2024") of the current production supervisor's company and retrieve the tenant-isolated business knowledge base based on this identifier. The enterprise's business knowledge base stores the following task-related private knowledge entries: Data dictionary: Work order table (work_order) field definitions: work order number (order_no, VARCHAR(32)), product code (product_code, VARCHAR(20)), planned output (plan_qty, DECIMAL(10,2)), actual output (actual_qty, DECIMAL(10,2)), process number (process_code, VARCHAR(10)), and statistical date (stat_date, DATE).

[0044] Business rules: Only records with a status of "Completed" (status='COMPLETED') are counted in the work report data; when the difference between planned output and actual output exceeds ±20%, it should be highlighted in red in the chart as a warning; Naming conventions: The title of the Kanban page should uniformly use the format "[Workshop Work Reporting Kanban] - [Current Week]"; the color scheme of the charts must conform to the company's VI specifications (main color: #1A6DF5, warning color: #FF4D4F).

[0045] The large language model agent injects the retrieved business knowledge into the planning context of the large language model in the form of structured prompts. The large language model generates the task plan shown in Table 1 based on the enhanced context.

[0046] Table 1 Step number Step description Related capability interfaces Expected verification conditions 1 Query the MES system work order table to extract the work order number, product code, planned output, actual output, and process number of the work orders completed this week. Database operation interface (DatabaseQuery) The returned result set fields must include order_no, product_code, plan_qty, actual_qty, process_code, and stat_date, with plan_qty and actual_qty being numeric types; the result set must contain more than 0 records. 2 Group and summarize planned and actual outputs by work order number, calculate the difference rate, and mark records with an absolute difference rate exceeding 20% ​​as 'requiring attention'. Code execution interface (PythonExecutor) The output summary dataset includes work order number, summary planned output, summary actual output, variance rate, and risk marker; the variance rate calculation result is accurate to two decimal places; JSON Schema validation passes. 3 HTML code for generating bar charts based on aggregated data. The charts compare planned and actual output for each work order, with bars showing deviations highlighted in red. Code execution interface (PythonExecutor + ECharts) The generated HTML file can be rendered correctly in a browser; screenshot analysis using a visual model confirms that the chart includes a legend, axis labels, and the main body of the bar chart, and that the number of red highlighted bars matches the number of records marked as 'requiring attention' in step 2. 4 Assemble the final Kanban page, embed charts, add titles, and update timestamps. File system operation interface (FileWrite) The output file path exists and the file size is greater than 1KB; the page title contains the keyword '

Workshop Work Reporting Board

[0047] Step 1: Call the database operation interface to connect to the MES system's read-only slave database and execute an SQL query. The interface returns a result set in JSON format.

[0048] The result was validated, including: (a) whether the returned JSON structure conformed to the preset schema definition and whether all required fields were present; and (b) whether the `plan_qty` and `actual_qty` fields were numeric types and not empty. This step passed the validation.

[0049] Step 2: Call the code execution interface, pass the data returned in Step 1 into the Python execution environment, and run the data aggregation script. The script outputs the summary results.

[0050] When the output was validated, it was found that the difference rate calculation result was only retained to the integer part, instead of the expected two decimal places, and the validation failed.

[0051] S4, Replanning and Iteration The detailed information of the verification failure (including the step identifier "Step2", the difference description between the expected requirement "difference rate accurate to two decimal places" and the actual output "difference rate is an integer" and the original data summary of Step 1 output) is packaged into the current context information.

[0052] The aforementioned contextual information is injected into the large language model, triggering a local replanning. After analysis, the large language model determines that there is no need to regenerate the full plan; only a data formatting step needs to be inserted between steps S2 and S3. The corrected step sequence is updated as shown in Table 2.

[0053] Table 2 Step number Step description Related capability interfaces Expected verification conditions 2 Group planned output and actual output by work order number and calculate the difference rate. Code execution interface (PythonExecutor) The aggregated dataset structure has passed validation. 2.5 (New) Format the difference rate field as a floating-point number with two decimal places, and map the risk marker field to a boolean value. Code execution interface (PythonExecutor) The difference rate field is a numeric type with two decimal places, and the risk marker field is a boolean type. 3 Generate a bar chart based on formatted data. Code execution interface (PythonExecutor + ECharts) The rendered screenshots have passed visual model verification. Continue from step 2.5. After successful verification, proceed to step 3. During step 3, the visual model's analysis and verification of the rendered screenshots passed, confirming that the chart meets the requirements.

[0054] Then, step 4 in Table 1 was executed, and the file system operation interface successfully generated the Kanban HTML file, and the verification passed.

[0055] After all execution steps pass verification, the system will output the final generated Kanban page file path and access link as the task completion result, and set the task's running status to "Completed".

[0056] S5, persistent state storage Throughout the entire task execution process, the runtime state database persistently stores the following information in the form of structured records: Task record: Task identifier (Task_20260820_001), original text of task objective, current running status (finally "Completed"), creator, creation time.

[0057] Step record: Each of the 5 steps (including the newly added step 2.5 for replanning) includes its own step identifier, input parameter summary, actual execution result storage path, verification result (pass / fail), and timestamp.

[0058] When the task reached step 3, the user terminal refreshed the page due to network fluctuations. Since the completion records of steps 1, 2, and 2.5, as well as the current execution progress of step 3, have been persisted in the runtime state database, after the user terminal reconnects, the system reads the structured record in the runtime state database through server push event stream (SSE) and restores the task progress to "Step 3 in progress" instead of resetting it to an empty state, thus achieving the technical effect of not losing progress across refreshes.

[0059] Example 2 This embodiment is intended to demonstrate the system's approval suspension and resumption mechanism when high-risk operations are involved in the execution of a task.

[0060] The use case is that an enterprise's operations and maintenance personnel input the natural language task objective: "Deploy the production environment's work reporting dashboard application to the Kubernetes cluster and update the database connection configuration."

[0061] The task plan generated by the large language model agent includes a step called "update database connection configuration." This step is associated with a deployment operation interface, the operation type is "configuration write," and the target environment is a "production environment." A risk level assessment is performed on this step before execution, based on the following logic: Operation type is "configuration write" → basic risk score is 30 points; The target environment is a "production environment" → risk weighting factor is 3.0; The data source involved is the "core business database" → additional risk score of 20 points; Overall risk score: 30 × 3.0 + 20 = 110 points, exceeding the preset high-risk threshold (80 points).

[0062] The step is determined to meet preset high-risk conditions. Before executing this step, the large language model agent automatically changes the task's running status from "running" to "pending approval" and pushes an approval request event to the user's terminal. The user sees the following operation preview information on the terminal: Preview of operations pending approval Operation type: Kubernetes Secret configuration update Target cluster: Production cluster (prod-k8s-master-01) Change details: The database connection address has been changed from 192.168.1.100:3306 to 192.168.1.101:3306. Affected areas: The deployment / workshop-dashboard application will restart and read the new configuration. Estimated duration of disruption: Approximately 30 seconds. Please confirm whether you wish to continue: [Confirm] [Reject] During this period, the large language model agent suspends task execution, does not call the capability interface associated with this step, and does not advance to the next execution step. At the same time, the task record in the runtime status database is updated to "pending approval," and the step record records the approval request time and preview information hash value of this step for audit traceability.

[0063] After the user confirms and clicks the [Confirm] button, the system receives the confirmation and approval instruction, restores the task status to "Running," and calls the deployment operation interface associated with this step to perform a configuration update. Subsequent steps continue to execute until the task is completed.

[0064] If the user clicks [Reject], the system will set the status of the step to "Cancelled", the task running status to "Failed", output the reason for failure "User refused to perform high-risk operation", and retain the completed intermediate products (such as the generated Kanban HTML file) for the user to manually process or adjust later and re-initiate the task.

[0065] During the aforementioned approval suspension period, if the user terminal unexpectedly disconnects, the system will automatically restore the preview of the pending approval operation after the user logs back in, since the runtime state library has persistently stored the "pending approval" status and approval request context. The user can still complete the approval operation normally.

[0066] Example 3: This embodiment aims to demonstrate the complete process by which the system achieves self-repair through a replanning closed loop when the program code generated by the execution steps fails the automated test.

[0067] The use case is in the task execution process of Example 1. It is assumed that the actual execution method of step 3 (generating HTML code for a bar chart based on summary data) is not to directly call the chart library to generate HTML, but to generate a Python + ECharts drawing script by the large language model, and then execute the script through the code execution interface to produce the chart.

[0068] Execution and initial verification failed: The execution module calls the code execution interface to run the plotting script generated by the large language model in an isolated Python execution environment. The expected verification condition for this step is set as "the generated program code passing the automated test cases run in the isolated execution environment." Specifically, the system automatically executes a set of test cases for this script, including: Test case TC-01: Input a standard dataset (containing normal data from 3 work orders) and verify whether the output chart contains 3 sets of bar charts; Test case TC-02: Input a dataset containing out-of-tolerance records and verify whether the bars corresponding to the out-of-tolerance records are marked in red (#FF4D4F). Test case TC-03: Input an empty dataset and verify that the script returns a friendly empty status message instead of throwing an exception.

[0069] Test case TC-02 failed to execute – the script correctly generated the bar chart, but when judging the out-of-tolerance threshold, it incorrectly used "absolute difference rate exceeds 10%" instead of "exceeds 20%" as specified in the enterprise business rules, causing a large number of bars that should not have been marked in red to be marked in red.

[0070] Replanning Trigger: The large language model agent uses test failure information (test case TC-02 failed, the reason for failure is "the judgment standard for exceeding the tolerance threshold does not meet the business rules", and the difference between the actual difference rate list and the expected red-marked list) as the current context information.

[0071] Iterative fixes: The large language model agent injects the aforementioned context information into the large language model, along with the original text of the rule retrieved from the business knowledge base: "When the difference rate exceeds ±20%, it should be highlighted in red in the chart as a warning." The large language model then regenerates the revised plotting script, changing the threshold judgment condition from "abs(diff_rate)>0.1" to "abs(diff_rate)>0.2".

[0072] The revised script was executed again, and the automated test case set was run again. This time, all three test cases passed. After successful verification, the task continued with subsequent steps.

[0073] Retry threshold protection: In the task execution process of this embodiment, if the script generated by the first replanning still fails the test, the second and third replanning will be triggered. When the number of replanning attempts reaches the preset threshold of 3, no further attempts will be made, the task's running status will be set to "failed", the reason for failure will be output as "the chart generation script still failed the automated test after 3 replanning attempts", and the generated failed scripts and test case logs will be retained for developers to analyze offline.

[0074] The above mechanism ensures that even if the initial generation of the large language model has defects, the system still has the ability to automatically repair it within the same round, while the setting of the repair threshold prevents the system from falling into an infinite loop and ensures the controllable consumption of system resources.

[0075] In all the above embodiments, the progress synchronization mechanism during task execution remains consistent. That is, during the asynchronous execution of the aforementioned steps, the system sends task progress events to the user terminal in real time via Server-SentEvents (SSE). The task progress events include at least the plan creation, step start, approval request, step completion, and task completion events.

[0076] When the user terminal refreshes or reconnects, the system does not rely on the temporary cache in the session messages, but instead restores and displays the task progress based on the structured records in the runtime state library. The specific logic for resuming execution includes: reading all step records of the target task, determining the last execution step that passed verification, and continuing execution from the next execution step after that step.

[0077] This design ensures that users who refresh the page due to network fluctuations in Example 1, approvers who switch terminals to log in midway in Example 2, and maintenance personnel who refresh to view the results after waiting for a long time for test execution in Example 3 can all receive continuous and consistent task status feedback, and the user experience is not affected by the interruption of the underlying execution process.

[0078] The above embodiments are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

[0079] like Figure 12 As shown, this embodiment also provides a large language model intelligent agent industrial software task execution system based on a verification loop for implementing the above method, including, The planning module is used to receive the task objective input by the user in natural language, call the large language model agent to perform intent parsing and generate a task plan. The task plan includes multiple execution steps arranged in sequence, and each execution step is associated with the capability interface to be called and the corresponding expected verification conditions. The planning module is also used to retrieve the tenant-isolated business knowledge base of the enterprise to which the caller belongs, and inject the retrieved business rules and data dictionary into the planning context of the large language model.

[0080] The execution module is used to sequentially call the capability interfaces associated with each execution step to perform the corresponding operations and collect the actual execution results of each step.

[0081] The verification module is used to compare and verify the actual execution results of each step with the corresponding expected verification conditions.

[0082] The replanning module is used to trigger the large language model agent to replan subsequent execution steps when the verification fails, carrying the current context information, forming a closed-loop iteration of "planning-execution-observation-verification" until all execution steps pass the verification.

[0083] The runtime status library is used to persistently store the runtime status of tasks and the step status of each execution step, input parameters, actual execution results and verification records in a structured record format independent of user session messages. It also supports task suspension, resumption, failure retry and manual approval suspension.

[0084] The above system can execute the industrial software task execution method based on the large language model intelligent agent with verification loop provided in any embodiment of the present invention. The specific functional descriptions of each module are consistent with the aforementioned steps and will not be repeated here.

[0085] like Figure 13As shown, the present invention also provides an electronic device for installing the above-described system and capable of running the above-described method, comprising a processor and a memory, wherein the memory stores a computer program. The memory includes at least a read-only memory (ROM) and a random access memory (RAM), and various appropriate actions and processes can be executed through the computer program in the memory. The RAM also stores various programs and data required for the operation of the electronic device. A communication interface is also connected to the processor, including I / O output interfaces for input / output interaction such as a touchscreen, touchpad, keyboard, mouse, camera, and microphone, for human-computer interaction. The communication interface allows the electronic device to communicate wirelessly or wiredly with other devices to exchange data. The computer program contains program code for implementing the functions of the above-described modules, and can be downloaded and installed from a network via a communication device, or installed from the memory. When the computer program is executed by the processor, it implements the corresponding functions defined in the large language model intelligent agent industrial software task execution method based on verification loop of the embodiments of the present invention.

[0086] According to the method provided in the embodiments of the present invention, the present invention can also provide a computer program product comprising: a computer program or a set of instructions, wherein when the computer program or set of instructions is run on a computer, a method for executing industrial software tasks based on a large language model intelligent agent using a verification loop can be successfully implemented.

[0087] According to the method provided in the embodiments of the present invention, the present invention also provides a computer-readable storage medium, which may be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk or solid-state drive, etc., and stores a computer program thereon. When the computer program is executed by a processor, it implements the above-described method for executing industrial software tasks based on a large language model intelligent agent with a verification loop.

[0088] The description of this invention is given for illustrative and descriptive purposes only and is not intended to be exhaustive or to limit the invention to the forms disclosed. Many modifications and variations will be apparent to those skilled in the art. The embodiments were chosen and described in order to better illustrate the principles and practical application of the invention and to enable those skilled in the art to understand the invention and to design various embodiments with various modifications suitable for a particular purpose.

Claims

1. A method for executing industrial software tasks using a large language model intelligent agent based on a validation loop, characterized by: include, S1. Receive the task target input by the user in natural language, call the large language model agent to perform intent parsing on the task target, and generate a task plan based on the intent parsing result. The task plan includes multiple execution steps arranged in sequence, and each execution step is associated with a capability interface to be called and the expected verification conditions corresponding to that step. S2. Execute multiple execution steps in sequence. For the current execution step, call its associated capability interface to perform the corresponding operation, call the expected verification condition corresponding to the current execution step, and collect the actual execution result of the current execution step. S3. Compare and verify the actual execution result with the expected verification conditions corresponding to the current execution step; S4. In response to the failure of the verification, the current context information, including the actual execution result, is carried back to the task plan generation stage or plan adjustment stage in step S1. The plan adjustment stage is where the large language model agent re-plans the subsequent execution steps based on the current context information and continues to execute step S2 to form a closed loop iteration. In response to successful verification, proceed to the next execution step until all execution steps have passed verification, and then output the task completion result. S5. During the execution of the task objective, the running status of the task objective, as well as the step status, input parameters, actual execution results, and verification records of each execution step, are persistently stored in the running status library in the form of structured records. The running status of the task objective includes at least five states: running, paused, pending approval, failed, and completed. The structured records are stored independently of user session messages and support the resumption of paused tasks, retrying failed execution steps, and suspending execution when the task is in the pending approval state and resuming execution after receiving an approval instruction.

2. The method for executing industrial software tasks based on a large language model intelligent agent using a verification loop as described in claim 1, characterized in that: The expected verification conditions mentioned in step S3 include at least one of the following: The structured output returned by the capability interface is matched and validated against the preset JSON Schema; The program code generated by the execution steps passes the verification of automated test cases running in an isolated execution environment; The correctness of the interface is verified by visual model analysis of the screenshot of the page generated by the execution steps after being rendered by the browser. Consistency verification between the output data of the execution steps and the business rule constraints.

3. The method for executing industrial software tasks based on a large language model intelligent agent using a verification loop as described in claim 1, characterized in that: The replanning in step S4 includes injecting the identifiers of execution steps that fail verification, the difference information between the actual execution result and the expected verification conditions, and the list of completed execution steps into the context of the large language model. The large language model agent then generates a corrected sequence of subsequent execution steps. When the number of replanning attempts for a single task reaches a preset threshold, the running status of the task is set to failure, and the reason for failure and the retained intermediate products are output.

4. The method for executing industrial software tasks based on a large language model intelligent agent using a verification loop as described in claim 1, characterized in that: Before execution, for each step in the execution process that meets the preset high-risk condition, the running status of the task is set to pending approval, and the operation preview information of the step is displayed to the user; the corresponding operation is only executed by calling the capability interface associated with the step after receiving the user's confirmation and approval instruction. During the asynchronous execution of the execution steps, task progress events are sent to the user terminal in real time via server push event stream. The task progress events include at least the plan creation, step start, approval request, step completion, and task completion events. After the user terminal refreshes or reconnects, the task progress is restored and displayed based on the structured records in the running status database.

5. The method for executing industrial software tasks based on a large language model intelligent agent using a verification loop as described in claim 1, characterized in that: The structured record in step S5 includes two layers: task record and step record. The task record includes task identifier, task target, current running status and creation information. The step record includes step identifier, belonging task identifier, step type, input parameters, actual execution result, verification result and timestamp. Resuming execution includes reading all step records of the target task, determining the last verified execution step, and continuing execution from the next execution step of that step.

6. The method for executing industrial software tasks based on a large language model intelligent agent using a verification loop as described in claim 1, characterized in that: The capability interfaces include basic capability interfaces unrelated to business logic. These basic capability interfaces include one or more combinations of file system operations, command line execution, code execution, browser operations, database operations, network requests, and deployment operations. Each basic capability interface has a typed input / output mode definition, permission scope declaration, and timeout retry strategy.

7. The method for executing industrial software tasks based on a large language model intelligent agent using a verification loop as described in claim 1, applied to software generation and business execution in industrial enterprises, is characterized in that: The intent parsing in step S1 also includes retrieving the business knowledge base of the enterprise to which the caller belongs. The business knowledge base stores the enterprise's business rules, data dictionary, process documents, historical project records, and knowledge entries precipitated from the historical execution products of the large language model intelligent agent. The retrieved business knowledge related to the task objective is injected into the planning context of the large language model, so that the data structure in the generated task plan and execution steps conforms to the business rules of the enterprise; wherein, the business knowledge base is stored in isolation by enterprise tenant, the business knowledge bases of different enterprises are not visible to each other, and the knowledge retrieval scope inherits the access control permissions of the knowledge entries at the time of entry.

8. An industrial software task execution system based on a large language model intelligent agent with a validation loop, characterized in that: include, The planning module is used to receive the task objective input by the user in natural language, call the large language model agent to perform intent parsing and generate a task plan. The task plan includes multiple execution steps arranged in sequence, and each execution step is associated with the capability interface to be called and the corresponding expected verification conditions. The planning module is also used to retrieve the tenant-isolated business knowledge base of the enterprise to which the caller belongs, and inject the retrieved business rules and data dictionary into the planning context of the large language model. The execution module is used to sequentially call the capability interfaces associated with each execution step to perform the corresponding operations and collect the actual execution results of each step; The verification module is used to compare and verify the actual execution results of each step with the corresponding expected verification conditions. The replanning module is used to trigger the large language model agent to replan subsequent execution steps when the verification fails, carrying the current context information, forming a closed-loop iteration of "planning-execution-observation-verification" until all execution steps pass the verification. The runtime status library is used to persistently store the runtime status of tasks and the step status of each execution step, input parameters, actual execution results and verification records in a structured record format independent of user session messages. It also supports task suspension, resumption, failure retry and manual approval suspension.

9. An electronic device, characterized in that: It includes a processor and a memory, the memory storing a computer program that, when executed by the processor, implements the industrial software task execution method for large language model intelligent agents based on verification loops as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the industrial software task execution method for large language model intelligent agents based on verification loops as described in any one of claims 1 to 7.

Images