An agent continuous training and effect evaluation closed loop method and system fusing work order feedback and a medium

Abstract

The application relates to an intelligent agent continuous training and effect evaluation closed loop method, system and medium fusing work order feedback, and relates to the technical field of deep learning. The intelligent agent continuous training and effect evaluation closed loop method comprises the following steps: receiving and analyzing work order feedback information, recording the number of error information and the information storage time length in combination with a preset structure sample label; dynamically determining the number of error information and the information storage time length, triggering a retraining rule of an original intelligent agent model, distributing candidate information weights, and screening a target sample set; according to the retraining rule in combination with the target sample set, the original intelligent agent model is directionally trained and optimized to generate a new intelligent agent model; a historical work order problem set is constructed, and the new intelligent agent model is compared and verified in combination with the original intelligent agent model to generate an intelligent agent evaluation table, and the candidate information weights are corrected; through accurate focusing on weak links, the continuous and efficient enhancement of the model capacity is realized, so that the intelligent agent answer accuracy is significantly improved, and AI illusion is effectively inhibited.

Description

Technical Field

[0001] This application relates to the field of deep learning technology, and in particular to a closed-loop method, system, and medium for continuous training and performance evaluation of intelligent agents that integrates work order feedback. Background Technology

[0002] The knowledge base updates of intelligent agents typically rely on periodic, batch data imports, making it difficult to promptly incorporate the latest and most accurate knowledge generated through work order verification processes. Even if the correct answer is verified manually, safely and compliantly integrating it into the existing knowledge base presents a challenge. Direct addition may lead to conflicts between old and new knowledge, information duplication, or logical inconsistencies, introducing new confusion. After obtaining structured training samples transformed from work orders and an updated knowledge base, the challenge lies in efficiently utilizing this data to retrain the agent and scientifically evaluating the training effectiveness.

[0003] Existing patents disclose an agent decision-making model based on memory-learning collaboration and its application, wherein: Step S1, constructing an agent feature attribute module, the feature attribute module including static attribute features and dynamic attribute features of each agent; Step S2, constructing an agent behavior set module, the agent behavior set module is used to select individual agent behaviors and organizational behaviors; Step S3, constructing an agent behavior pattern module, the agent behavior pattern module combines its own state with perceived information, and refers to its accumulated memory set, and generates the next action plan under the guidance of behavior rules according to the following formula, thereby updating its own state; Step S4. Constructing an interaction mechanism between intelligent agents, including: a memory mechanism consisting of an individual memory bank, a group memory bank, and a memory buffer pool; establishing an interaction mechanism between the intelligent agent and the environment and other intelligent agents, with the intelligent agent continuously updating its decision-making strategy through continuous learning; establishing a collaborative decision-making mechanism of memory and learning that introduces the dual role of "history + experience" in the decision-making process of the intelligent agent; establishing a cyclical feedback mechanism between the intelligent agent's decision-making memory and learning. The above invention proposes a memory credibility assessment mechanism, which realizes adaptive updating of memory by dynamically assessing memory credibility and provides effective support in the decision-making process, thereby improving the modeling quality of individual agents.

[0004] The existing technical solutions mentioned above have the following drawbacks: 1. The traditional approach is to periodically retrain the entire model, which is costly, time-consuming, and difficult to accurately evaluate the effect of correcting historical errors, resulting in slow model iteration, insignificant improvement on specific weaknesses reflected in work orders, and a lag and blind spot in the process of improving the accuracy of the agent. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this application aims to provide a closed-loop method, system, and medium for continuous training and performance evaluation of intelligent agents that integrates work order feedback. This method forms a self-iteratory intelligent optimization closed loop through work order parsing, target selection, model reconstruction, and targeted optimization. It utilizes a dynamic threshold algorithm to trigger training rules, combines an attention mechanism to allocate sample weights, and employs efficient parameter fine-tuning techniques for targeted repair or global optimization. Finally, it generates an evaluation table through A / B testing and regression analysis to iteratively correct weights, achieving adaptive continuous optimization of the intelligent agent, significantly improving response accuracy, and effectively suppressing AI illusions.

[0006] This was achieved using the following technical solutions: Firstly, this application provides a closed-loop method for continuous training and performance evaluation of an agent that integrates work order feedback, including: Receive and parse work order feedback information, combine it with preset structured sample tags, and record the number of error messages and the information storage time; The system dynamically determines the number of error messages and the storage time of the messages, triggers the retraining rules of the original intelligent agent model, assigns weights to candidate messages, and filters the target sample set. Based on the retraining rules and the target sample set, the original agent model is optimized through targeted training to generate a new agent model. Construct a set of historical work orders and compare and verify the new intelligent agent model with the original intelligent agent model, generate an intelligent agent evaluation table, and correct the weights of candidate information.

[0007] By adopting the above technical solution, the system receives and analyzes work order feedback, combines the number of errors recorded by structured tags with storage duration, and uses dynamic threshold algorithms (such as weighted calculation based on error accumulation and timeliness) to determine the triggering of differentiated retraining rules (local fixed-point repair, global fine-tuning, or inference deepening). An attention weight allocation mechanism is then used to select a high-value target sample set. Subsequently, techniques such as efficient parameter fine-tuning (such as LoRA) are used to optimize the model in a targeted manner to generate new intelligent agents. Finally, a historical work order test set is constructed for A / B comparison evaluation, and the sample weights are adjusted based on the evaluation results, forming a self-iteratory intelligent optimization closed loop. By precisely focusing on weak links, the system achieves continuous and efficient enhancement of model capabilities, thereby significantly improving the accuracy of the intelligent agent's responses and effectively suppressing AI illusions.

[0008] This application is further configured to: receive and parse work order feedback information, combine it with preset structured sample tags, and record the number of error messages and the information storage duration, including: Receive work order feedback information from each client, identify the client in advance, and record the frequency of work order feedback; The work order feedback information is structured and parsed to extract feedback alarm level, behavior execution time sequence and resource status consumption data; Information is extracted from the execution sequence chain to obtain the error context content and tool call sequence; Based on the information constraint boundaries, the error context content is logically deconstructed, the logical structure of the content is extracted, and the logical coherence and content exceedance are calculated. If the content exceeds the upper limit of the preset content restriction range and the logical coherence is less than the lower limit of the preset logical continuity range, then the current content logic structure is determined to be guided logic. If the content exceeds the limit but is not within the content restriction range within the number of constraint attempts, and the logical coherence is within the logical continuity range, then the current content logical structure is determined to be accidental logic. If the content exceeds the limit within the content restriction range and the logical coherence is greater than the upper limit of the logical continuity range, then the current content logical structure is determined to be spontaneous logic. The resource status consumption data is time-series processed, the timestamps of abnormal states are extracted, and the feedback alarm level is corrected based on the logical type to obtain the corrected alarm level. Based on the preset structural sample labels and the corrected alarm level, semantic matching is performed on the error context content to obtain the core knowledge entities and error expression fragments; Based on the knowledge point graph, the core knowledge entities are associated and matched to determine the knowledge point tags; The knowledge point tags are verified based on historical similar samples. If the tags are the same, the incorrect statement fragments are analyzed to determine the reasoning deviation nodes. No, then the timing of the tool calls corresponding to the erroneous description fragments is compared to determine if the call deviates from the timing sequence; Based on the client identifier and the error type, the work order feedback information is classified to obtain a single-category error information set; Based on the frequency of work order feedback, the single-type error information set is deduplicated, and the number of error information is counted in conjunction with the correction alarm level, and the information storage time is recorded.

[0009] By adopting the above technical solution, the system receives and parses client work order feedback, extracts alarm level, behavior time sequence, and resource consumption data, uses semantic analysis algorithms to calculate the logical coherence and exceedance of content to determine the logic type (guided / accidental / spontaneous), and corrects the alarm level by combining time sequence analysis. Based on knowledge graph association and matching, core entities are identified to determine knowledge point tags, and inference deviation nodes are located through historical sample verification or tool-based time sequence comparison. Finally, based on client identifiers and error type classification, the number of deduplicated error messages and storage duration are statistically analyzed, achieving in-depth structured analysis of work order feedback and accurate attribution of error root causes. Through multi-dimensional intelligent discrimination and knowledge association, the quality of error sample annotation and the relevance of training data are significantly improved.

[0010] This application is further configured to: dynamically determine the number of error messages and the storage duration of the messages, trigger the retraining rules of the original intelligent agent model, assign weights to candidate messages, and filter the target sample set, including: Based on the corrected alarm level, the information error types are prioritized and quantitative thresholds are determined to obtain the information storage capacity and information temporary storage period, which are then compared with the number of error messages and the information storage time. If the number of error messages reaches the information storage limit, and / or the information storage duration reaches the information temporary storage period, the current agent is determined to be in a state of waiting for training, and a retraining instruction is generated. Based on the retraining instructions and the number of error types, the training mechanism for the original agent model is selected, and the retraining rules are determined. If the number of error types does not reach the preset global trigger threshold, and the correction warning level is high risk, then the retraining rule is determined to be a local fixed-point repair rule. If the number of error types reaches the global trigger threshold, and all core knowledge entities are of known categories, then the retraining rule is determined to be a global parameter fine-tuning rule. If the number of error types reaches the global trigger threshold, but the core knowledge entity contains unknown categories, then the retraining rule is determined to be the inference node deepening rule. Based on the retraining rules and the joint collaborative training algorithm, the corresponding node data in the original agent model is extracted and trained to construct the original data items. The candidate information weights are obtained by assigning weights to the original data items based on the correction warning level. If the weight of the candidate information is greater than the preset target information threshold, the original data items corresponding to the current candidate information weight are extracted and aggregated into the target sample set.

[0011] By adopting the above technical solution, a dynamic threshold algorithm is used to monitor the number of error messages and storage time. When the preset information storage volume or temporary storage period is reached, a training instruction is triggered. A multi-level rule-based judgment logic (based on whether the number of error types reaches a global threshold, whether core knowledge entities contain unknown categories, and the correction warning level, etc.) intelligently selects differentiated retraining rules such as local fixed-point repair, global parameter fine-tuning, or inference node deepening. Subsequently, candidate information weights are assigned to model node data according to the correction warning level using an attention weight allocation mechanism. By comparing these weights with the target information threshold, a high-value target sample set is accurately selected and aggregated. This achieves adaptive and intelligent training triggering and strategy selection, enabling the adoption of the optimal training path for different error types and knowledge states, and ensuring that the sample set used for optimization is highly accurate and effective, thereby significantly improving model iteration efficiency and correction effects.

[0012] This application further specifies that the joint collaborative training algorithm includes: Sample the original data items and the single-type error message set separately to construct a mixed data sample; The mixed data samples are trained in parallel based on the original intelligent agent model, and the language modeling loss function and instruction following loss function are calculated. The language modeling loss function L is based on the number of training steps. CPT (θ) and instruction follower loss function L SFT (θ) is dynamically weighted to obtain the total loss function L for the first stage. Stage1 And thereby determine the new benchmark model π Stage1 : L Stage1 =λ CPT (t)⋅L CPT (θ)+λ SFT (t)⋅L SFT (θ); Where: λ CPT (t) = 1 − t / T is the language modeling weight coefficient, which decays as the number of training steps increases; λ SFT (t) = t / T is the instruction following weight coefficient, which increases with the number of training steps; t is the current number of training steps; T is the total number of training steps; θ is the model parameter; Based on the target sample set and the KL divergence regularization term, the new benchmark model π is applied. Stage1 Perform constrained optimization and calculate the total loss function L in the second stage. Stage2 (θ); L Stage2 (θ)=L DPO (θ)+γ⋅E x∼D [KL(π θ (⋅∣x)∣∣π Stage1 (⋅∣x))]; Where: L DPO (θ) represents the optimal selection loss function; γ is the regularization intensity hyperparameter; E x∼D [] represents the total expectation function; π θ () represents the new intelligent agent model; x represents the question-and-answer test instruction.

[0013] By adopting the above technical solution and merging the CPT and SFT stages, the total training time is reduced; knowledge learning and instruction learning occur simultaneously and mutually reinforce each other (for example, the learned domain knowledge can directly help generate more professional answers). The regularization design in the second stage ensures the robustness of preference learning, resulting in a model that is both useful and reliable. It ensures that when the model learns to select more preferred answer styles, it does not lose the domain knowledge and basic instruction execution capabilities acquired in the first stage, thereby effectively mitigating the catastrophic forgetting problem.

[0014] This application further specifies: based on retraining rules and the target sample set, targeted training is performed to optimize the original agent model and generate a new agent model, including: Based on the retraining rules and information error types, the target sample set is associated and aligned, specific associated samples are selected, and the retraining tolerance value is calculated. If the retraining rule is a local fixed-point repair rule, then feature extraction is performed on the erroneous description fragment and specific associated samples to generate a joint feature vector; If the retraining tolerance value is greater than the tolerance trigger value, then the weights of the erroneous associated nodes of the original agent model are calculated based on the joint feature vector to construct the reoperation weight matrix; The recomputation weight matrix is ​​dynamically iterated and trained until the constraint convergence parameters are reached, thus generating a new intelligent agent model. If the retraining rule is a global parameter fine-tuning rule, then historical connection rules and resource channel capacity are combined to construct a global optimization objective function; Constrain the direction of the global associated nodes of the original intelligent agent model according to the temporal logic to obtain the real-time status of the resource channel capacity; If the real-time state fluctuates, the learning rate is adjusted according to the preset dynamic correction coefficient, and the weight allocation matrix of each topology level is updated to obtain a new intelligent agent model. If the retraining rule is an inference node deepening rule, then the unknown category entities are weighted according to the error node expansion matrix to create an unknown entity feature matrix; Based on historical node association rules and node hierarchy constraints, deep reasoning is performed on the feature matrix of unknown entities, and the comprehensive weight matrix of the original intelligent agent model is fused and calculated to generate a new intelligent agent model.

[0015] By adopting the above technical solution, intelligent training optimization is driven based on differentiated retraining rules (local fixed-point repair, global parameter fine-tuning, and inference node deepening): fixed-point repair is achieved through joint feature extraction and iterative recomputation of the weight matrix; parameter fine-tuning is completed by constructing a global optimization objective function and introducing dynamic learning rate correction; and inference nodes are deepened by fusing unknown entity feature matrices with deep inference technology. Through adaptive selection of the optimal training path and a refined parameter adjustment mechanism, the targeting and efficiency of model iteration are significantly improved. This effectively corrects known errors, enhances generalization ability, significantly reduces AI illusions, and drives the agent's accuracy to continuously approach its limit.

[0016] This application further includes: constructing a historical work order problem set, comparing and validating the new intelligent agent model with the original intelligent agent model, generating an intelligent agent evaluation table, and correcting the candidate information weights, including: Based on the feedback alarm level, historical feedback work orders are clustered for error parsing to construct a set of historical work order issues and obtain the corresponding standard answers; The historical work order problem set is deduplicated and its identifiers are cleaned to obtain an unidentified work order problem set. The unlabeled work order problem set is classified according to the type of information error to obtain single-category problem work order groups; The work order groups for each type of problem are divided according to the preset data division ratio to obtain single-type training groups and single-type validation groups; Mix and cross-reference all single-class training groups to obtain the work order problem training set; The work order problem training set is input into the original agent model and the new agent model for several iterations of training, and the original verification weight matrix and the new verification weight matrix are extracted. The original validation weight matrix is ​​compared with the original question-answering weight matrix of the original agent model to extract the difference weight matrix; Based on the single-class validation set, the original validation weight matrix and the new validation weight matrix are validated separately to obtain the original output answer and the new output answer: If the original output answer is the same as the standard answer, the difference weight matrix is ​​used to correct the current type of problem work order, and self-correcting problems are counted. If the original output answer differs from the standard answer, then the original validation weight matrix is ​​considered equivalent to the question-answer weight matrix. If the new output answer is the same as the standard answer, then the current type of problem ticket is determined to have been completely corrected and marked as a corrected problem; If the newly output answer is partially the same as the standard answer, then the error pattern of the current type of problem ticket is determined to be partially corrected. If the new output answer differs from the standard answer, the current problem work order is deemed to have failed to be corrected. The number of error types is recorded, the correction alarm level and core knowledge body are marked, and the corresponding retraining rules are triggered. The targeted remediation rate is calculated based on the total number of issues and the number of issues that have been corrected. The regression rate is calculated based on the total number of problems and the number of error types, and the corresponding weight correction coefficient is generated based on the error pattern. Attribution analysis is performed on the new agent model based on the targeted repair rate, regression rate, and error pattern to generate an agent evaluation table; The weights of candidate information are adjusted based on the weight adjustment coefficient, and a new sample set is selected.

[0017] By employing the above technical solution, a set of unlabeled historical work orders is constructed using a semantic clustering algorithm. Stratified sampling is used to divide the model into training and validation groups, and an A / B testing framework is used to compare the differences in weight matrices and output results between the old and new models. Based on the standard answer, the problem correction status (complete / partial / failure) is determined. Weight correction coefficients are generated by calculating the targeted repair rate, regression rate, and error pattern analysis, thereby driving the dynamic optimization of candidate information weights and iterative selection of the sample set. By establishing a closed-loop evaluation and feedback mechanism, the model iteration effect can be accurately quantified, residual error patterns can be located, and adaptive optimization of training data can be achieved, thus continuously improving the accuracy and generalization ability of the agent.

[0018] Secondly, this application also provides a closed-loop system for continuous training and performance evaluation of intelligent agents that integrates work order feedback, employing the following technical solution: A closed-loop system for continuous agent training and performance evaluation integrating work order feedback, comprising the following methods for implementing continuous agent training and performance evaluation: The work order parsing module is used to receive and parse work order feedback information, and record the number of error messages and the information storage time by combining the preset structure sample tags. The target selection module is used to dynamically determine the number of error messages and the storage time of the messages, trigger the retraining rules of the original intelligent agent model, assign weights to candidate messages, and select the target sample set. The model reconstruction module is used to optimize the original agent model by combining the retraining rules with the target sample set and then perform targeted training to generate a new agent model. The targeted optimization module is used to construct a set of historical work orders, compare and verify the new intelligent agent model with the original intelligent agent model, generate an intelligent agent evaluation table, and correct the weights of candidate information.

[0019] By adopting the above technical solution, through the closed-loop linkage of four modules—work order parsing, target selection, model reconstruction, and targeted optimization—the training rules are triggered by a dynamic threshold algorithm, sample weights are allocated by an attention mechanism, and parameter fine-tuning technology is used for point-to-point repair or global optimization. Finally, an evaluation table is generated through A / B testing and regression analysis to iteratively correct the weights, thereby achieving adaptive continuous optimization of the agent, significantly improving the accuracy of responses and effectively suppressing AI illusions.

[0020] Thirdly, this application also provides an electronic device, comprising: One or more processors; Memory, used to store one or more programs; When one or more programs are executed by one or more processors, the one or more processors implement any of the methods in the above scheme.

[0021] Fourthly, this application also provides a storage medium storing at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, at least one program, code set, or instruction set is loaded and executed by a processor to implement the closed-loop method for continuous training and effect evaluation of intelligent agents with integrated work order feedback as described above.

[0022] In summary, the beneficial technical effects of this application are as follows: By calculating the logical coherence and excess of content to determine the logical type (guided / accidental / spontaneous), and combining it with time series analysis to correct the alarm level; by associating and matching core entities based on knowledge graph to determine knowledge point tags, and by verifying through historical samples or calling tools to compare time series to locate inference deviation nodes; finally, based on client identification and error type classification, the number of deduplicated error messages and storage time are statistically analyzed, realizing in-depth structured analysis of work order feedback and accurate attribution of error root causes; By adaptively selecting the optimal training path and employing a refined parameter adjustment mechanism, the targeting and efficiency of model iteration are significantly improved. This effectively corrects known errors, enhances generalization ability, greatly reduces AI illusions, and drives the agent's accuracy to continuously approach its limit. Attached Figure Description

[0023] Figure 1 This is a schematic diagram of the overall process of the closed-loop method for continuous training and effect evaluation of the agent in this application; Figure 2 This is a flowchart illustrating step S2 in the closed-loop method for continuous training and effect evaluation of the agent in this application. Figure 3 This is a flowchart illustrating step S4 in the closed-loop method for continuous training and effect evaluation of the agent in this application. Figure 4 This is a schematic diagram of the closed-loop system for continuous training and performance evaluation of the agent in this application. Detailed Implementation

[0024] The present application will be further described in detail below with reference to the accompanying drawings.

[0025] Reference Figure 1 This application discloses a closed-loop method for continuous training and performance evaluation of an agent that integrates work order feedback, comprising: S1: Receive and parse work order feedback information, combine it with preset structured sample tags, and record the number of error messages and the information storage time; S2: Dynamically determine the number of error messages and the storage time of the messages, trigger the retraining rules of the original intelligent agent model, assign weights to candidate messages, and filter the target sample set; S3: Based on the retraining rules and the target sample set, perform targeted training to optimize the original agent model and generate a new agent model; S4: Construct a set of historical work orders and compare and verify the new agent model with the original agent model, generate an agent evaluation table, and correct the weights of candidate information.

[0026] The implementation principle of this embodiment is as follows: By structuring the feedback information and monitoring error accumulation, the adaptive training rule engine is triggered when the dynamic threshold condition is met. High-value sample sets are selected based on attention weight allocation, and then the model is optimized in a targeted manner using a differentiated fine-tuning algorithm. Finally, A / B testing is conducted by constructing a historical problem set and weight correction coefficients are generated, forming a complete closed-loop iterative system from problem discovery, accurate training to effect verification, thereby continuously improving the performance of the intelligent agent.

[0027] Preferably, step S1 includes: Receive work order feedback information from each client, identify the client in advance, and record the frequency of work order feedback; The work order feedback information is structured and parsed to extract feedback alarm level, behavior execution time sequence and resource status consumption data; In this embodiment, the work order feedback information records key operations of the intelligent agent itself, such as startup, shutdown, configuration changes, and service status switching, as well as fluctuations in system resource usage (such as CPU, memory, and disk); it records user-input instructions, query content, operation requests, and the intelligent agent's response output, facilitating the tracing of dialogue history and user intent; it records data, signals, or events received by the intelligent agent from the external environment (such as sensors and APIs), and the processing results of this information; it records in detail errors, warnings, and abnormal events that occur during system operation, including error codes, stack information, and occurrence times, for fault diagnosis and problem investigation; it records events related to security auditing, such as login attempts, permission changes, data access, and security alarms, to support security analysis and threat detection; and it records performance data such as request processing time, response latency, and throughput to evaluate intelligent agent efficiency and optimize resource allocation.

[0028] Information is extracted from the execution sequence chain to obtain the error context content and tool call sequence; Based on the information constraint boundaries, the error context content is logically deconstructed, the logical structure of the content is extracted, and the logical coherence and content exceedance are calculated. If the content exceeds the upper limit of the preset content restriction range and the logical coherence is less than the lower limit of the preset logical continuity range, then the current content logic structure is determined to be guided logic. If the content exceeds the limit but is not within the content restriction range within the number of constraint attempts, and the logical coherence is within the logical continuity range, then the current content logical structure is determined to be accidental logic. If the content exceeds the limit within the content restriction range and the logical coherence is greater than the upper limit of the logical continuity range, then the current content logical structure is determined to be spontaneous logic. In this embodiment, the guiding logic includes: triggering when the content exceeds the preset threshold (e.g., >70%) and the logical coherence is lower than the threshold (e.g., <30%).

[0029] External factors (such as malicious injection or algorithm misdirection) can actively disrupt the logic chain, for example, by injecting irrelevant parameters to cause the system to execute unexpected branches.

[0030] Malicious request interception in API security filtering (such as forcibly interrupting the process when UrlHandleFilter detects SQL injection).

[0031] Accidental logic includes: logical coherence in the middle range (30%~60%), and content exceeding the limit briefly exceeding the threshold before falling back.

[0032] Local logical breaks can be caused by momentary interference (such as network jitter or concurrent conflicts), but the system's self-healing mechanism can restore the main logic.

[0033] During data synchronization, network latency may cause fields to be missing, but the retry mechanism will automatically repair them (the retry logic of VectorDBClient.sync()).

[0034] Spontaneous logic includes: high logical coherence (>80%) but the content contains fundamental contradictions, and the content exceeds the limit but does not exceed the standard (<20%).

[0035] Due to inherent cognitive biases (such as incorrect assumptions) or data defects (such as training set bias), outputs that appear reasonable on the surface but are fundamentally wrong are generated.

[0036] Vector queries return logically consistent but actually incorrect results due to incorrect configuration of the associated table (TableRelationForm definition error).

[0037] The resource status consumption data is time-series processed, the timestamps of abnormal states are extracted, and the feedback alarm level is corrected based on the logical type to obtain the corrected alarm level. Based on the preset structural sample labels and the corrected alarm level, semantic matching is performed on the error context content to obtain the core knowledge entities and error expression fragments; Based on the knowledge point graph, the core knowledge entities are associated and matched to determine the knowledge point tags; The knowledge point tags are verified based on historical similar samples. If the tags are the same, the incorrect statement fragments are analyzed to determine the reasoning deviation nodes. No, then the timing of the tool calls corresponding to the erroneous description fragments is compared to determine if the call deviates from the timing sequence; Based on the client identifier and the error type, the work order feedback information is classified to obtain a single-category error information set; Based on the frequency of work order feedback, the single-type error information set is deduplicated, and the number of error information is counted in conjunction with the correction alarm level, and the information storage time is recorded.

[0038] In this embodiment, the "standard answer package" attached to the work order is parsed, and the core knowledge entities, erroneous expression fragments, and marked error types (such as "factual error", "logical contradiction", "time delay") are automatically extracted from it.

[0039] The extracted error statements are semantically matched with a pre-defined, continuously improving error classification tree to determine their primary and secondary error types. Simultaneously, the relevant core knowledge entities are associated with nodes in the existing knowledge graph and tagged with knowledge points. It is also necessary to determine whether the current classification is logically consistent with the classification of similar historical samples. If new error patterns or knowledge point associations are discovered, an extended review process for the classification system is triggered to ensure that the archive structure remains both stable and adaptable.

[0040] Preferably, refer to Figure 2 Step S2 includes: Based on the corrected alarm level, the information error types are prioritized and quantitative thresholds are determined to obtain the information storage capacity and information temporary storage period, which are then compared with the number of error messages and the information storage time. If the number of error messages reaches the information storage limit, and / or the information storage duration reaches the information temporary storage period, the current agent is determined to be in a state of waiting for training, and a retraining instruction is generated. Based on the retraining instructions and the number of error types, the training mechanism for the original agent model is selected, and the retraining rules are determined. If the number of error types does not reach the preset global trigger threshold, and the correction warning level is high risk, then the retraining rule is determined to be a local fixed-point repair rule. If the number of error types reaches the global trigger threshold, and all core knowledge entities are of known categories, then the retraining rule is determined to be a global parameter fine-tuning rule. If the number of error types reaches the global trigger threshold, but the core knowledge entity contains unknown categories, then the retraining rule is determined to be the inference node deepening rule. Based on the retraining rules and the joint collaborative training algorithm, the corresponding node data in the original agent model is extracted to construct the original data items; The candidate information weights are obtained by assigning weights to the original data items based on the correction warning level. If the weight of the candidate information is greater than the preset target information threshold, the original data items corresponding to the current candidate information weight are extracted and aggregated into the target sample set.

[0041] In this embodiment, the local fixed-point repair rule is as follows: when the error type does not reach the global threshold and the correction warning is high risk, data that is strongly correlated with a specific error type in the target sample set (such as operation records and samples with excessive frequency of abnormal responses) are selected.

[0042] Global parameter fine-tuning rules: If the error type reaches the global threshold and all entities are of known categories, then extract the joint feature vector of all abnormal responses in the sample set and align the event tags based on the time window.

[0043] Inference node deepening rules: When there are unknown category entities, it is necessary to separate the known entities and unknown entities in the sample and construct an independent new entity feature space (such as the extended dimension of action trajectory features).

[0044] Two main conditions are continuously monitored: first, the stock condition, which checks whether the number of samples under each error type / knowledge point category has reached the preset "cumulative threshold," which may be dynamically adjusted based on the severity of the error or the model's historical performance in that category; second, the period condition, which determines whether the preset regular training time point (such as weekly / monthly) has been reached. When multiple conditions are met simultaneously or partially, arbitration logic is activated, for example, prioritizing the training task of the category with the fastest sample accumulation (indicating a prominent problem) or the highest error level, while tasks from adjacent periods may be merged.

[0045] Preferably, the joint collaborative training algorithm includes: Sample the original data items and the single-type error message set separately to construct a mixed data sample; The mixed data samples are trained in parallel based on the original intelligent agent model, and the language modeling loss function and instruction following loss function are calculated. The language modeling loss function L is based on the number of training steps. CPT (θ) and instruction follower loss function L SFT (θ) is dynamically weighted to obtain the total loss function L for the first stage. Stage1 And thereby determine the new benchmark model π Stage1 : L Stage1 =λ CPT (t)⋅L CPT (θ)+λ SFT (t)⋅L SFT (θ); Where: λ CPT(t) = 1 − t / T is the language modeling weight coefficient, which decays as the number of training steps increases; λ SFT (t) = t / T is the instruction following weight coefficient, which increases with the number of training steps; t is the current number of training steps; T is the total number of training steps; θ is the model parameter; Based on the target sample set and the KL divergence regularization term, the new benchmark model π is applied. Stage1 Perform constrained optimization and calculate the total loss function L in the second stage. Stage2 (θ); L Stage2 (θ)=L DPO (θ)+γ⋅E x∼D [KL(π θ (⋅∣x)∣∣π Stage1 (⋅∣x))]; Where: L DPO (θ) represents the optimal selection loss function; γ is the regularization intensity hyperparameter; E x∼D [] represents the total expectation function; π θ () represents the new intelligent agent model; x represents the question-and-answer test instruction.

[0046] In this embodiment, the training process is divided into two core stages: Phase 1: Knowledge-Skills Joint Training Phase - CPT and SFT are conducted simultaneously.

[0047] Phase 2: Robust optimization phase based on preference awareness - Introducing DPO on the basis of joint training and imposing constraints to prevent forgetting.

[0048] Phase 1 Operating Mechanism and Formula: Within a specific domain, one can simultaneously master "knowledge" (language modeling ability) and "skills" (instruction following ability).

[0049] Each training batch consists of a mixture of two data parts: Domain text data (for CPT): Randomly sampled from large-scale unlabeled text in the target domain.

[0050] Instruction-Response Pair Data (for SFT): Sample from a high-quality SFT dataset.

[0051] L CPT (θ)=−∑logP(w t |w <t ;θ): Standard language modeling loss function, applied to domain text data, w t These are the weighting coefficients.

[0052] L SFT (θ)=−∑logP(y t ∣y<t ,x;θ): Standard supervised fine-tuning loss function applied to instruction-response pairs; y t To predict the output value.

[0053] λ CPT (t) and λ SFT (t) is a dynamic weighting coefficient that changes over time (training steps t).

[0054] In the initial stage of training, λ is set. CPT Higher (e.g., close to 1), λ SFT Low (e.g., close to 0). This forces the model to prioritize absorbing domain knowledge.

[0055] As training progresses, λ is smoothly reduced. CPT The weights, while increasing λ SFT The weights are adjusted accordingly. This means that the model gradually shifts its learning focus to understanding and executing instructions. This avoids the rapid forgetting of CPT knowledge that may occur during the SFT stage in the traditional sequential process due to "disruptive" training, achieving a smooth transition and mutual promotion of knowledge and skill learning.

[0056] L dpo =DPOLoss(batch dpo ;θ,π stage1 ,β), calculate the standard DPO loss; L kl =KLDivergence(batch dpo ;θ,πstage1), calculate the KL regularization term (based on the question-answering test instruction x); The KL divergence term acts like a rubber band, "tying" the model being optimized to the baseline model.

[0057] If a model starts to deviate from its original correctness and completeness in order to cater excessively to biased data (e.g., generating very short but potentially missing key information answers), the KL divergence value will increase sharply, thus creating a strong pull in the loss function to prevent this harmful deviation.

[0058] Preferably, step S3 includes: Based on the retraining rules and information error types, the target sample set is associated and aligned, specific associated samples are selected, and the retraining tolerance value is calculated. If the retraining rule is a local fixed-point repair rule, then feature extraction is performed on the erroneous description fragment and specific associated samples to generate a joint feature vector; If the retraining tolerance value is greater than the tolerance trigger value, then the weights of the erroneous associated nodes of the original agent model are calculated based on the joint feature vector to construct the reoperation weight matrix; The recomputation weight matrix is ​​dynamically iterated and trained until the constraint convergence parameters are reached, thus generating a new intelligent agent model. If the retraining rule is a global parameter fine-tuning rule, then historical connection rules and resource channel capacity are combined to construct a global optimization objective function; Constrain the direction of the global associated nodes of the original intelligent agent model according to the temporal logic to obtain the real-time status of the resource channel capacity; If the real-time state fluctuates, the learning rate is adjusted according to the preset dynamic correction coefficient, and the weight allocation matrix of each topology level is updated to obtain a new intelligent agent model. If the retraining rule is an inference node deepening rule, then the unknown category entities are weighted according to the error node expansion matrix to create an unknown entity feature matrix; Based on historical node association rules and node hierarchy constraints, deep reasoning is performed on the feature matrix of unknown entities, and the comprehensive weight matrix of the original intelligent agent model is fused and calculated to generate a new intelligent agent model.

[0059] In this embodiment, the constraint convergence parameter can be either the error of three consecutive iterations being <0.5% or the learning rate × 0.7 when the capacity load is >80%.

[0060] A comprehensive assessment of the training task's multiple attributes is necessary: ​​the training objective (is it "point-to-point repair" for a specific error type or broad "capability enhancement"?), the scale and characteristics of the training data, and the available computational resources and time window. For example, if the objective is to repair a single error type and the sample size is moderate, a parameter-efficient fine-tuning strategy (such as LoRA) is more likely to be chosen to minimize resource consumption and avoid catastrophic forgetting; if the objective is to incorporate a large amount of new knowledge across categories, a more comprehensive incremental learning strategy may be selected. Before training begins, the selected samples undergo a secondary evaluation. Based on factors such as the verification authority of their source work orders, the typicality of the errors, and the complexity of the problems, different training weights are dynamically assigned to ensure the model focuses more on key errors. During training, key indicators such as the loss function's decline curve and performance on the validation set are monitored in real time to determine if the learning process is normal and effective. If abnormal indicators occur (e.g., the loss increases instead of decreasing), it may be judged as overfitting or a mismatched training strategy. The system will automatically intervene, taking corrective measures such as adjusting the learning rate, stopping early, or rolling back to a checkpoint to ensure that training proceeds in the correct direction to improve the target capability.

[0061] Preferably, refer to Figure 3 Step S4 includes: Based on the feedback alarm level, historical feedback work orders are clustered for error parsing to construct a set of historical work order issues and obtain the corresponding standard answers; The historical work order problem set is deduplicated and its identifiers are cleaned to obtain an unidentified work order problem set. The unlabeled work order problem set is classified according to the type of information error to obtain single-category problem work order groups; The work order groups for each type of problem are divided according to the preset data division ratio to obtain single-type training groups and single-type validation groups; Mix and cross-reference all single-class training groups to obtain the work order problem training set; The work order problem training set is input into the original agent model and the new agent model for several iterations of training, and the original verification weight matrix and the new verification weight matrix are extracted. The original validation weight matrix is ​​compared with the original question-answering weight matrix of the original agent model to extract the difference weight matrix; Based on the single-class validation set, the original validation weight matrix and the new validation weight matrix are validated separately to obtain the original output answer and the new output answer: If the original output answer is the same as the standard answer, the difference weight matrix is ​​used to correct the current type of problem work order, and self-correcting problems are counted. If the original output answer differs from the standard answer, then the original validation weight matrix is ​​considered equivalent to the question-answer weight matrix. If the new output answer is the same as the standard answer, then the current type of problem ticket is determined to have been completely corrected and marked as a corrected problem; If the newly output answer is partially the same as the standard answer, then the error pattern of the current type of problem ticket is determined to be partially corrected. If the new output answer differs from the standard answer, the current problem work order is deemed to have failed to be corrected. The number of error types is recorded, the correction alarm level and core knowledge body are marked, and the corresponding retraining rules are triggered. The targeted remediation rate is calculated based on the total number of issues and the number of issues that have been corrected. The regression rate is calculated based on the total number of problems and the number of error types, and the corresponding weight correction coefficient is generated based on the error pattern. Attribution analysis is performed on the new agent model based on the targeted repair rate, regression rate, and error pattern to generate an agent evaluation table; The weights of candidate information are adjusted based on the weight adjustment coefficient, and a new sample set is selected.

[0062] In this embodiment, all relevant work orders are extracted from historical records, including a detailed description of each work order, classification tags (such as problem type and severity), solution steps, and standard answer. This ensures the problem set covers a variety of scenarios and anomalies to comprehensively evaluate model performance.

[0063] Remove duplicate work orders and clean incomplete or erroneous data to ensure the accuracy and consistency of the issue set; label and categorize work orders according to issue type and severity to facilitate subsequent analysis. For example, categorize issues into "troubleshooting," "configuration adjustment," and "software update."

[0064] The problem set is divided into a training set, a validation set, and a test set to ensure that the problems in each set are evenly distributed and representative. The training set is used for model training, the validation set is used for adjusting model parameters, and the test set is used for final evaluation.

[0065] Build an evaluation and analysis model, and run both the original agent model and the new agent model simultaneously. Ensure that both models are evaluated on the exact same problem set to guarantee the fairness and accuracy of the comparison results.

[0066] For each work order question, record the output results of the original model and the new model, compare them with the standard answer, and calculate: Correction success rate: Whether the new model obtains the correct answer to the question on which it previously made a mistake.

[0067] Performance retention rate: Whether the new model maintains the same performance as the original model on the originally correct questions, i.e., whether it triggers regression.

[0068] Error mode change: For problems that still have errors, analyze whether the error mode has changed (e.g., from "completely wrong" to "partially correct").

[0069] Calculate the percentage of times the new model successfully corrects previously made mistakes. The formula is as follows: Targeted repair rate = (Number of successfully corrected issues / Total number of issues) × 100% This metric reflects the effectiveness of the new model in improving specific errors.

[0070] Calculate the proportion of questions where the new model makes mistakes, even if the original model was correct. The formula is as follows: Regression rate = (Number of errors in the problem / Total number of problems) × 100%; This metric is used to assess whether a new model has introduced new errors.

[0071] For problems where errors still exist, analyze changes in error patterns. For example, record cases where an error changes from "completely wrong" to "partially correct," and evaluate the performance of the new model during the incremental improvement process.

[0072] The results of targeted repair rate, regression rate and error pattern analysis are summarized to form a detailed evaluation table, including the specific value of each indicator, the trend of change and the possible causes. By analyzing the differences between the output results and the standard answers, the differences can be attributed to specific model parameters or structural adjustments. For example, if a significant improvement in the success rate of correcting a certain type of question is found, it can be traced back to the optimization of the corresponding parameters of the model.

[0073] Based on the evaluation results, further improvement suggestions are proposed. For example, if the new model is found to perform poorly on specific problem types, it may be suggested to increase the training on relevant samples, adjust the model structure to improve performance, and formulate a strategy for adjusting the weights of candidate information. For example, if it is found that a certain type of feature has a significant impact on the model output, the weight of that feature may be appropriately increased.

[0074] A dynamic optimization algorithm is employed to automatically adjust the weights of candidate information based on real-time evaluation results. This ensures that the model gradually approaches the optimal weight allocation during continuous iteration.

[0075] After each weight adjustment, the evaluation system is rerun to verify the effect of the adjustment. If the improvement is significant, the adjustment results are retained; otherwise, the system is rolled back to the previous version and iterative optimization continues.

[0076] The implementation principle of this embodiment is as follows: By structuring the client feedback, semantic analysis and logical coherence calculation (combining preset thresholds to distinguish guided, accidental, and spontaneous logic) are applied, along with resource status time-series analysis, to correct the alarm level and extract core knowledge entities and error fragments; then, based on the dynamic threshold algorithm (monitoring the number of errors and storage duration), the training rule engine is triggered, and differentiated training strategies are intelligently selected according to the number of error types and entity categories, such as local fixed-point repair (using joint feature vector construction and weight matrix iteration), global parameter fine-tuning (integrating resource channel capacity constraints and dynamic learning rate adjustment), or inference node deepening (constructing an unknown entity feature matrix for deep inference). Subsequently, a high-value sample set is selected using an attention weight allocation mechanism, and a parameter-efficient fine-tuning technique is employed to optimize the model in a targeted manner. Finally, an A / B test is conducted by constructing a historical work order problem set, and weight correction coefficients are generated based on the targeted repair rate, regression rate, and error pattern analysis. This forms a complete closed-loop optimization system from problem discovery, adaptive strategy selection, precise training to effect verification and data weight iteration, thereby achieving continuous improvement in the accuracy of the intelligent agent and effective suppression of AI illusion.

[0077] Reference Figure 4 A closed-loop system for continuous training and performance evaluation of an intelligent agent, integrating work order feedback, is applied to a fault detection method, including: The work order parsing module is used to receive and parse work order feedback information, and record the number of error messages and the information storage time by combining the preset structure sample tags. The target selection module is used to dynamically determine the number of error messages and the storage time of the messages, trigger the retraining rules of the original intelligent agent model, assign weights to candidate messages, and select the target sample set. The model reconstruction module is used to optimize the original agent model by combining the retraining rules with the target sample set and then perform targeted training to generate a new agent model. The targeted optimization module is used to construct a set of historical work orders, compare and verify the new intelligent agent model with the original intelligent agent model, generate an intelligent agent evaluation table, and correct the weights of candidate information.

[0078] The implementation principle of this embodiment is as follows: the work order parsing module performs structured analysis and logical type discrimination on the feedback information, combined with the dynamic threshold algorithm to monitor error accumulation and trigger differentiated training rules, and uses the attention weight allocation mechanism to screen high-value samples. Then, the efficient parameter fine-tuning technology is used to perform local fixed-point repair, global optimization or inference deepening on the model. Finally, the evaluation table is generated by comparing and regressing the historical work order set with A / B tests and regression analysis, and the data weights are iteratively corrected to form a complete intelligent enhancement closed loop from problem perception to adaptive optimization.

[0079] An electronic device, comprising: One or more processors; Memory, used to store one or more programs; When one or more programs are executed by one or more processors, the one or more processors implement any of the methods in the above scheme.

[0080] A storage medium storing at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, at least one program, code set, or instruction set is loaded and executed by a processor to implement a closed-loop method for continuous training and effect evaluation of an agent with integrated work order feedback as described above.

[0081] The embodiments described in this specific implementation are preferred embodiments of this application and are not intended to limit the scope of protection of this application. Therefore, all equivalent changes made in accordance with the structure, shape and principle of this application should be covered within the scope of protection of this application.

Claims

1. A closed-loop method for continuous training and performance evaluation of an agent that integrates work order feedback, characterized in that, include: The process involves receiving and parsing work order feedback information, combining it with preset structured sample tags, and recording the number of error messages and the information storage duration. Specific steps include: Based on the client identifier and the error type, the work order feedback information is classified to obtain a single-category error information set; The single-type error information set is deduplicated based on the frequency of work order feedback, and the number of error information is counted in conjunction with the correction alarm level, and the information storage time is recorded. The steps include dynamically determining the number of error messages and the storage duration of the messages, triggering the retraining rule of the original agent model, assigning weights to candidate messages, and filtering the target sample set. Based on the corrected alarm level, the information error types are prioritized and quantitative thresholds are determined to obtain the information storage capacity and information temporary storage period, which are then compared with the number of error messages and the information storage time. If the number of error messages reaches the information storage capacity, and / or the information storage duration reaches the information temporary storage period, then the current agent is determined to be in a state of waiting for training, and a retraining instruction is generated. Based on the retraining instructions and the number of error types, the training mechanism for the original agent model is selected, and the retraining rules are determined. If the number of error types does not reach the preset global trigger threshold, and the correction warning level is high risk, then the retraining rule is determined to be a local fixed-point repair rule. If the number of error types reaches the global trigger threshold, and all core knowledge entities are known categories, then the retraining rule is determined to be a global parameter fine-tuning rule. If the number of error types reaches the global trigger threshold, but the core knowledge entity contains an unknown category, then the retraining rule is determined to be an inference node deepening rule. Based on the retraining rules and the joint collaborative training algorithm, the corresponding node data in the original agent model is extracted and trained to construct the original data items. The original data items are weighted according to the corrected warning level to obtain the candidate information weights; If the weight of the candidate information is greater than the preset target information threshold, then the original data item corresponding to the current candidate information weight is extracted and aggregated into the target sample set; Based on the retraining rules and the target sample set, the original agent model is optimized through targeted training to generate a new agent model. Construct a historical work order problem set, and compare and verify the new intelligent agent model with the original intelligent agent model to generate an intelligent agent evaluation table and correct the candidate information weights. The specific steps include: The targeted remediation rate is calculated based on the total number of issues and the number of issues that have been corrected. Based on the total number of problems and the number of error types, the regression rate is calculated, and the corresponding weight correction coefficient is generated based on the error pattern. Attribution analysis is performed on the new agent model based on the targeted repair rate, regression rate, and error pattern to generate an agent evaluation table; The weights of the candidate information are adjusted according to the weight adjustment coefficients to select a new sample set.

2. The closed-loop method for continuous training and effect evaluation of an agent based on integrated work order feedback as described in claim 1, characterized in that, The process of receiving and parsing work order feedback information, combined with preset structured sample tags, records the number of error messages and the information storage duration, including: Receive work order feedback information from each client, identify the client in advance, and record the frequency of work order feedback; The work order feedback information is structured and parsed to extract feedback alarm level, behavior execution timing chain and resource status consumption data; Information is extracted from the execution timing chain to obtain the error context content and tool call timing; Based on the information constraint boundaries, the error context content is logically deconstructed to extract the content logical structure and calculate the logical coherence and content exceedance. If the content exceeds the upper limit of the preset content restriction range and the logical coherence is less than the lower limit of the preset logical continuity range, then the current content logic structure is determined to be guided logic. If the content exceedance is not within the content restriction range within the constraint number of times, and the logical coherence is within the logical continuity value range, then the current content logical structure is determined to be accidental logic. If the content exceedance is within the content restriction range, and the logical coherence is greater than the upper limit of the logical continuity range, then the current content logic structure is determined to be spontaneous logic.

3. The closed-loop method for continuous training and effect evaluation of an agent based on integrated work order feedback as described in claim 2, characterized in that, The process of receiving and parsing work order feedback information, combined with preset structured sample tags, and recording the number of error messages and the information storage duration, also includes: The resource status consumption data is time-series processed, the timestamps of abnormal states are extracted, and the feedback alarm level is corrected based on the logical type to obtain the corrected alarm level. Based on the preset structural sample labels and the corrected alarm level, semantic matching is performed on the error context content to obtain core knowledge entities and error expression fragments; Based on the knowledge point graph, the core knowledge entities are associated and matched to determine the knowledge point tags; The knowledge point tags are verified based on historical similar samples. If the tags are the same, the erroneous statement fragments are analyzed to determine the reasoning deviation nodes. If not, the timing of the tool calls corresponding to the erroneous statement fragment is compared to determine if the call deviates from the timing.

4. The closed-loop method for continuous training and effect evaluation of an agent based on integrated work order feedback as described in claim 1, characterized in that, The joint collaborative training algorithm includes: Sample the original data items and the single-type error message set separately to construct a mixed data sample; The mixed data samples are trained in parallel based on the original intelligent agent model, and the language modeling loss function and instruction following loss function are calculated. The language modeling loss function L is based on the number of training steps. CPT (θ) and instruction follower loss function L SFT (θ) is dynamically weighted to obtain the total loss function L for the first stage. Stage1 And thereby determine the new benchmark model π Stage1 : L Stage1 =λ CPT (t)⋅L CPT (θ)+λ SFT (t)⋅L SFT (i); Where: λ CPT (t) = 1 − t / T is the language modeling weight coefficient, which decays as the number of training steps increases; λ SFT (t) = t / T is the instruction following weight coefficient, which increases with the number of training steps; t is the current number of training steps; T is the total number of training steps; θ is the model parameter; Based on the target sample set and the KL divergence regularization term, the new benchmark model π is applied. Stage1 Perform constrained optimization and calculate the total loss function L in the second stage. Stage2 (θ); L Stage2 (θ)=L DPO (θ)+γ⋅E x∼D [KL(π θ (⋅∣x)∣∣π Stage1 (⋅∣x))]; Where: L DPO (θ) represents the optimal selection loss function; γ is the regularization intensity hyperparameter; E x∼D [] represents the total expectation function; π θ () represents the new intelligent agent model; x represents the question-and-answer test instruction.

5. The closed-loop method for continuous training and effect evaluation of an agent based on integrated work order feedback as described in claim 1, characterized in that, The step of optimizing the original agent model by retraining according to the retraining rules and the target sample set, and generating a new agent model, includes: Based on the retraining rules and information error types, the target sample set is associated and aligned, specific associated samples are selected, and the retraining tolerance value is calculated. If the retraining rule is a local fixed-point repair rule, then feature extraction is performed on the erroneous statement fragment and the specific associated sample to generate a joint feature vector; If the retraining tolerance value is greater than the tolerance trigger value, then the weights of the erroneous associated nodes of the original agent model are calculated based on the joint feature vector to construct a reoperation weight matrix; The recomputation weight matrix is ​​dynamically iteratively trained until the constraint convergence parameter is reached, thereby generating a new intelligent agent model. If the retraining rule is a global parameter fine-tuning rule, then historical connection rules and resource channel capacity are integrated to construct a global optimization objective function; Constrain the global associated node direction of the original intelligent agent model according to the temporal logic, and obtain the real-time status of the resource channel capacity; If the real-time state is fluctuating, the learning rate is corrected according to the preset dynamic correction coefficient, and the weight allocation matrix of each topological level is updated to obtain the new intelligent agent model. If the retraining rule is an inference node deepening rule, then the unknown category entities are weighted according to the error node expansion matrix to create an unknown entity feature matrix; Based on historical node association rules and node hierarchy constraints, deep reasoning is performed on the unknown entity feature matrix, and the comprehensive weight matrix of the original intelligent agent model is fused and calculated to generate the new intelligent agent model.

6. The closed-loop method for continuous training and effect evaluation of an agent based on integrated work order feedback as described in claim 1, characterized in that, The process of constructing a historical work order problem set, comparing and validating the new intelligent agent model with the original intelligent agent model, generating an intelligent agent evaluation table, and correcting the candidate information weights includes: Based on the feedback alarm level, historical feedback work orders are clustered for error parsing to construct a set of historical work order issues and obtain the corresponding standard answers; The historical work order problem set is deduplicated and its identifiers are cleaned to obtain an unidentified work order problem set. The unidentified work order problem set is classified according to the type of information error to obtain single-category problem work order groups; The work order groups for each type of problem are divided according to the preset data division ratio to obtain single-type training groups and single-type validation groups; Perform a crossover operation on all the single-class training groups to obtain the work order problem training set; The work order problem training set is input into the original agent model and the new agent model respectively for several iterations of training, and the original verification weight matrix and the new verification weight matrix are extracted. The original verification weight matrix is ​​compared with the original question-answering weight matrix of the original agent model to extract the difference weight matrix; Based on the single-class validation set, the original validation weight matrix and the new validation weight matrix are validated to obtain the original output answer and the new output answer: If the original output answer is the same as the standard answer, then the difference weight matrix is ​​used to correct the current type of problem work order, and self-correcting problems are counted. If the original output answer differs from the standard answer, then the original verification weight matrix is ​​deemed equivalent to the question-answer weight matrix.

7. The closed-loop method for continuous training and effect evaluation of an agent based on integrated work order feedback as described in claim 6, characterized in that, The process of constructing a historical work order problem set, comparing and validating the new intelligent agent model with the original intelligent agent model, generating an intelligent agent evaluation table, and correcting the candidate information weights also includes: If the new output answer is the same as the standard answer, then the current type of problem ticket is determined to have been completely corrected and marked as a corrected problem; If the new output answer is partially the same as the standard answer, then the error pattern of the current type of problem ticket is determined to be partially corrected successfully. If the new output answer differs from the standard answer, the current problem work order is deemed to have failed to be corrected. The number of error types is recorded, the correction alarm level and core knowledge body are marked, and the corresponding retraining rule is triggered.

8. A closed-loop system for continuous agent training and effect evaluation integrating work order feedback, used to implement the closed-loop method for continuous agent training and effect evaluation as described in any one of claims 1-7, characterized in that, include: The work order parsing module is used to receive and parse work order feedback information, and record the number of error messages and the information storage time by combining the preset structure sample tags. The target filtering module is used to dynamically determine the number of error messages and the storage duration of the messages, trigger the retraining rules of the original intelligent agent model, assign weights to candidate messages, and filter the target sample set. The model reconstruction module is used to perform targeted training and optimization of the original agent model based on the retraining rules and the target sample set, and generate a new agent model. The targeted optimization module is used to construct a set of historical work orders, compare and verify the new intelligent agent model with the original intelligent agent model, generate an intelligent agent evaluation table, and correct the weights of the candidate information.

9. A storage medium storing at least one instruction, at least one program, a code set, or an instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the closed-loop method for continuous training and effect evaluation of an agent as described in any one of claims 1 to 7.

Images