Intelligent agent continuous training and effect evaluation closed-loop method and system fusing work order feedback, and medium

Abstract

The invention relates to an intelligent agent continuous training and effect evaluation closed-loop method and system fusing work order feedback, and a medium, and relates to the technical field of deep learning. The agent continuous training and effect evaluation closed-loop method comprises the following steps: receiving and analyzing work order feedback information, and recording the number of error information and information storage duration in combination with a preset structure sample label; dynamically judging the number of error information and the information storage duration, triggering a retraining rule of an original agent model, distributing candidate information weights, and screening a target sample set; directionally training and optimizing the original agent model according to a retraining rule in combination with the target sample set, and generating a new agent model; constructing a historical work order problem set, comparing and verifying a new agent model in combination with an original agent model, generating an agent evaluation table, and correcting a candidate information weight; by accurately focusing the weak link, the continuous and efficient enhancement of the model capability is realized, so that the answer accuracy of the intelligent agent is remarkably improved, and the AI illusion is effectively inhibited.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of deep learning, in particular to an intelligent agent continuous training and effect evaluation closed-loop method and system fusing work order feedback and a medium BACKGROUND

[0002] The knowledge base of the intelligent agent is usually updated by periodic and batch data import, and cannot timely absorb the latest and most accurate knowledge generated through the work order verification process. Even if the correct answer is verified by manual verification, it is a challenge to safely and compliantly integrate it into the existing knowledge base. Direct addition may lead to conflicts between new and old knowledge, repeated information or inconsistent logic, and even introduce new confusion. After obtaining the structured training samples converted from the work order and the updated knowledge base, how to efficiently utilize the data to retrain the intelligent agent and scientifically evaluate the training effect.

[0003] The existing patent discloses an intelligent agent decision model based on memory-learning cooperation and application thereof, wherein: step S1, an intelligent agent feature attribute module is constructed, the feature attribute module includes static attribute features and dynamic attributes of each intelligent agent; step S2, an intelligent agent behavior set module is constructed, the intelligent agent behavior set module is used for selecting intelligent agent individual behavior and organizational behavior; step S3, an intelligent agent behavior mode module is constructed, the intelligent agent behavior mode module combines its own state with perceived information, and refers to its accumulated memory set, generates a next action plan under the guidance of a behavior rule according to the following formula, and thereby updates its own state; step S4, an interaction mechanism between intelligent agents is constructed, including: an individual memory bank, a group memory bank and a memory buffer pool constitute a memory mechanism; an interaction between an intelligent agent and an environment and other intelligent agents is established, and the intelligent agent continuously updates a decision strategy learning mechanism in continuous learning; a memory and learning cooperative decision mechanism is introduced in a double role of history + experience in the decision process of the intelligent agent; a cyclic feedback mechanism between decision memory and learning of the intelligent agent is established, the above application proposes a memory credibility evaluation mechanism, the memory is adaptively updated through dynamic evaluation of the memory credibility, and effective support is provided in the decision process, so that the individual agent modeling quality is improved.

[0004] The existing technical solution in the above has the following defects: 1. The traditional method is to periodically retrain the full model, which is costly and time-consuming, and it is difficult to accurately evaluate the repair effect on historical errors, resulting in slow model iteration and no obvious improvement on specific weaknesses reflected in the work order. The intelligent agent accuracy improvement process has a lag and a blind area. SUMMARY

[0005] To overcome the deficiencies in the prior art, the present application aims to provide an intelligent agent continuous training and effect evaluation closed-loop method, system and medium that fuses work order feedback, forms a self-iterative intelligent optimization closed loop through work order analysis, target selection, model reconstruction and directional optimization, triggers training rules using a dynamic threshold algorithm, assigns sample weights using an attention mechanism, and performs pinpoint repair or global optimization using parameter-efficient fine-tuning techniques, and finally generates an evaluation table through A / B testing and regression analysis to iteratively correct weights, achieving adaptive and continuous optimization of the intelligent agent, significantly improving answer accuracy and effectively suppressing AI hallucinations.

[0006] The following technical solutions are adopted to achieve the above-mentioned purposes: In a first aspect, the present application provides an intelligent agent continuous training and effect evaluation closed-loop method that fuses work order feedback, comprising: Receiving and analyzing work order feedback information, combining pre-set structured sample labels, recording the number of error information and information storage duration; Dynamically determining the number of error information and information storage duration, triggering the retraining rules of the original intelligent agent model, assigning candidate information weights, and selecting target sample sets; According to the retraining rules combined with the target sample sets, the original intelligent agent model is directionally trained and optimized to generate a new intelligent agent model; Constructing a historical work order problem set and comparing and verifying the new intelligent agent model with the original intelligent agent model to generate an intelligent agent evaluation table and correct the candidate information weights.

[0007] By adopting the above technical solutions, the work order feedback is received and analyzed, the number of errors and storage duration are recorded in combination with structured labels, the differential retraining rules (local pinpoint repair, global fine-tuning or reasoning deepening) are triggered using a dynamic threshold algorithm (such as weighted calculation based on error accumulation and time effectiveness), and the high-value target sample sets are selected using an attention weight distribution mechanism; then the new intelligent agent is generated by directionally optimizing the model using parameter-efficient fine-tuning (such as LoRA) and other techniques; finally, the sample weights are corrected through the evaluation results feedback by constructing a historical work order test set for A / B comparison and evaluation, forming a self-iterative intelligent optimization closed loop. By precisely focusing on weak links, the model capability is continuously and efficiently enhanced, thereby significantly improving the intelligent agent answer accuracy and effectively suppressing AI hallucinations.

[0008] The present application further provides that the receiving and analyzing work order feedback information, combining pre-set structured sample labels, recording the number of error information and information storage duration, comprises: Receiving work order feedback information from each client, identifying the client in advance, and recording the work order feedback frequency; Structurally analyzing the work order feedback information, extracting feedback alarm levels, behavior execution time sequence links and resource state consumption data; extracting information from the execution time link to obtain error context content and tool call timing; logically deconstructing the error context content according to the information constraint boundary, extracting content logical structure, calculating logical coherence and content overrun; if the content overrun is greater than the upper limit value of the preset content limit interval, and the logical coherence is less than the lower limit value of the preset logical through value interval, it is determined that the current content logical structure is a guiding type logic; if the content overrun is not located in the content limit interval within the constraint times, and the logical coherence is located in the logical through value interval, it is determined that the current content logical structure is an accidental type logic; if the content overrun is located in the content limit interval, and the logical coherence is greater than the upper limit value of the logical through value interval, it is determined that the current content logical structure is a spontaneous type logic; time series of resource state consumption data, extract abnormal state timestamp, and combine logical type to correct feedback alarm level to obtain corrected alarm level; According to the preset structure sample label combined with the corrected alarm level, the semantic matching of the error context content is carried out, and the core knowledge entity and the error expression segment are obtained; According to the knowledge point graph, the core knowledge entity is associated and matched to determine the knowledge point label; According to the historical similar sample, the knowledge point label is verified, if the two labels are the same, the error expression segment is analyzed, and the reasoning deviation node is determined; Otherwise, compare the tool call timing corresponding to the error expression segment to determine the call deviation timing; According to the customer identification combined with the information error type, the single error information set is obtained by classifying the work order feedback information; According to the work order feedback frequency, the single error information set is de-duplicated, combined with the corrected alarm level, the number of error information is counted, and the information storage time is recorded.

[0009] By adopting the above technical scheme, by receiving and analyzing the client work order feedback, the alarm level, behavior time link and resource consumption data are extracted, the content logical coherence and overrun are calculated by using semantic analysis algorithm to determine the logical type (guiding type / accidental type / spontaneous type), combined with the timing analysis to correct the alarm level; Based on the knowledge graph association matching core entity to determine the knowledge point label, through historical sample verification or tool call timing comparison positioning reasoning deviation node; Finally, according to the customer identification and error type classification, the number of error information after de-duplication and storage time are counted, the deep structured analysis and error root cause accurate attribution of work order feedback are realized; Through multi-dimensional intelligent discrimination and knowledge association, the quality and pertinence of error sample labeling and training data are significantly improved.

[0010] The application is further configured to dynamically determine the number of error information and the information storage time length, trigger the retraining rule of the original intelligent agent model, assign the candidate information weight, and screen the target sample set, including: According to the priority ranking and quantitative threshold determination of the information error type according to the modified warning level, the information storage amount and the information temporary storage period are obtained, and are compared with the number of error information and the information storage time length; If the number of error information reaches the information storage amount, and / or the information storage time length reaches the information temporary storage period, it is determined that the current intelligent agent is in a training state, and a retraining instruction is generated; According to the retraining instruction combined with the number of error types, the training mechanism of the original intelligent agent model is selected, and the retraining rule is determined; If the number of error types does not reach the preset global trigger threshold, and the modified warning level is high risk, it is determined that the retraining rule is a local point repair rule; If the number of error types reaches the global trigger threshold, and the core knowledge entity is all known categories, it is determined that the retraining rule is a global parameter fine-tuning rule; If the number of error types reaches the global trigger threshold, but the core knowledge entity contains unknown categories, it is determined that the retraining rule is a reasoning node deepening rule; According to the retraining rule combined with the joint collaborative training algorithm, the corresponding node data in the original intelligent agent model is extracted and trained to construct the original data item; According to the modified warning level, the original data item is given a weight to obtain a candidate information weight; If the candidate information weight is greater than the preset target information threshold, the original data item corresponding to the current candidate information weight is extracted, and is aggregated as a target sample set.

[0011] By using the above technical solution, based on the dynamic threshold algorithm, the number of error information and the storage time length are monitored, when reaching the preset information storage amount or temporary storage period, the training instruction is triggered, and the multi-level rule determination logic (according to whether the number of error types reaches the global threshold, whether the core knowledge entity contains unknown categories, the modified warning level, etc.) is used to intelligently select the differential retraining rules such as local point repair, global parameter fine-tuning or reasoning node deepening. Subsequently, according to the modified warning level, the model node data is given the attention weight distribution mechanism of the candidate information weight, and through comparison with the target information threshold, the high-value target sample set is accurately screened and aggregated. The adaptability and intelligence of training triggering and strategy selection are realized, the optimal training path can be taken for different error properties and knowledge states, and the sample set used for optimization is highly accurate and effective, thereby significantly improving the model iteration efficiency and correction effect.

[0012] The application is further configured that the joint collaborative training algorithm includes: Sample the original data items and the single-type error message set separately to construct a mixed data sample; The mixed data samples are trained in parallel based on the original intelligent agent model, and the language modeling loss function and instruction following loss function are calculated. The language modeling loss function L is based on the number of training steps. CPT (θ) and instruction follower loss function L SFT (θ) is dynamically weighted to obtain the total loss function L for the first stage. Stage1 And thereby determine the new benchmark model π Stage1 : L Stage1 =λ CPT (t)⋅L CPT (θ)+λ SFT (t)⋅L SFT (θ); Where: λ CPT (t) = 1 − t / T is the language modeling weight coefficient, which decays as the number of training steps increases; λ SFT (t) = t / T is the instruction following weight coefficient, which increases with the number of training steps; t is the current number of training steps; T is the total number of training steps; θ is the model parameter; Based on the target sample set and the KL divergence regularization term, the new benchmark model π is applied. Stage1 Perform constrained optimization and calculate the total loss function L in the second stage. Stage2 (θ); L Stage2 (θ)=L DPO (θ)+γ⋅E x∼D [KL(π θ (⋅∣x)∣∣π Stage1 (⋅∣x))]; Where: L DPO (θ) represents the optimal selection loss function; γ is the regularization intensity hyperparameter; E x∼D [] represents the total expectation function; π θ () represents the new intelligent agent model; x represents the question-and-answer test instruction.

[0013] By adopting the above technical solution and merging the CPT and SFT stages, the total training time is reduced; knowledge learning and instruction learning occur simultaneously and mutually reinforce each other (for example, the learned domain knowledge can directly help generate more professional answers). The regularization design in the second stage ensures the robustness of preference learning, resulting in a model that is both useful and reliable. It ensures that when the model learns to select more preferred answer styles, it does not lose the domain knowledge and basic instruction execution capabilities acquired in the first stage, thereby effectively mitigating the catastrophic forgetting problem.

[0014] The application is further configured to: according to the retraining rule in combination with the target sample set, training and optimizing the original intelligent agent model, generating a new intelligent agent model, including: According to the retraining rule in combination with the information error type, the target sample set is associated and aligned, the specific associated sample is screened, and the retraining tolerance value is calculated; If the retraining rule is a local fixed-point repair rule, the feature extraction is performed on the error expression segment and the specific associated sample to generate a joint feature vector; If the retraining tolerance value is greater than the tolerance trigger value, the weight calculation is performed on the error associated node of the original intelligent agent model according to the joint feature vector, and a re-operation weight matrix is constructed; The re-operation weight matrix is dynamically iteratively trained until the constraint convergence parameter is reached, and a new intelligent agent model is generated; If the retraining rule is a global parameter fine-tuning rule, the historical connection rule and the resource channel capacity are fused to construct a global optimization objective function; According to the timing logic, the direction of the global associated node of the original intelligent agent model is constrained, and the real-time state of the resource channel capacity is obtained; If the real-time state is fluctuation, the learning rate is modified according to the preset dynamic correction coefficient, and the weight distribution matrix of each topology level is updated to obtain a new intelligent agent model; If the retraining rule is a reasoning node deepening rule, the weight distribution is performed on the unknown class entity according to the error node expansion matrix, and an unknown entity feature matrix is created; According to the historical node association rule in combination with the node level constraint, the depth reasoning is performed on the unknown entity feature matrix, and the comprehensive weight matrix of the operation original intelligent agent model is fused to generate a new intelligent agent model.

[0015] By adopting the above technical solutions, the intelligent driving training optimization is realized based on the differentiated retraining rules (local fixed-point repair, global parameter fine-tuning, reasoning node deepening): the joint feature extraction and the re-operation weight matrix iteration are adopted to realize the fixed-point repair; the global optimization objective function is constructed and the dynamic learning rate correction is introduced to complete the parameter fine-tuning; the unknown entity feature matrix and the depth reasoning fusion technology are used to deepen the reasoning node. By adaptively selecting the optimal training path and the fine-tuning parameter adjustment mechanism, the pertinence and efficiency of model iteration are significantly improved, so as to effectively repair the known errors, enhance the generalization ability, and significantly reduce the AI illusion and promote the intelligent agent accuracy to continuously approach the limit.

[0016] The application is further configured to: construct a historical work order problem set, and compare and verify the new intelligent agent model in combination with the original intelligent agent model to generate an intelligent agent evaluation table, and correct the candidate information weight, including: According to the feedback alarm level, the historical feedback work order is error-parsed and clustered, a historical work order problem set is constructed, and the corresponding standard answer is obtained; The historical work order problem set is information-de-duplicated and identification-cleaned to obtain a non-identification work order problem set; According to the information error type, the non-identification work order problem set is classified to obtain a single-class problem work order group; According to a preset data division ratio, each single-class problem work order group is divided to obtain a single-class training group and a single-class verification group; The single-class training groups are mixed and crossed to obtain a work order problem training set; The work order problem training set is input into the original intelligent agent model and the new intelligent agent model for several times of iteration training, and the original verification weight matrix and the new verification weight matrix are extracted; According to the question and answer weight matrix of the original intelligent agent model, the original verification weight matrix is compared, and the difference weight matrix is extracted; According to the single-class verification group, the original verification weight matrix and the new verification weight matrix are respectively output verified to obtain the original output answer and the new output answer: If the original output answer is the same as the standard answer, it is determined that the difference weight matrix corrects the current class problem work order, and the self-corrected problem is counted; If the original output answer is different from the standard answer, it is determined that the original verification weight matrix is equivalent to the question and answer weight matrix; If the new output answer is the same as the standard answer, it is determined that the current class problem work order has been completely corrected successfully, and the corrected problem is marked; If the new output answer is partially the same as the standard answer, it is determined that the error mode of the current class problem work order is partially corrected successfully; If the new output answer is different from the standard answer, it is determined that the current class problem work order is corrected unsuccessfully, the number of error types is recorded, the correction alarm level and the core knowledge body are marked, and the corresponding retraining rule is triggered; According to the total number of problems combined with the number of corrected problems, the directional repair rate is calculated; According to the total number of problems combined with the number of error types, the regression rate is calculated, and the corresponding weight correction coefficient is generated combined with the error mode; According to the directional repair rate, the regression rate and the error mode, the new intelligent agent model is attributed to analysis to generate an intelligent agent evaluation table; According to the weight correction coefficient, the candidate information weight is corrected, and a new sample set is screened.

[0017] By adopting the technical scheme, the semantic clustering algorithm is adopted to construct the no-identity historical work order problem set, the hierarchical sampling is adopted to divide the training / verification group, and the A / B test framework is used to compare the weight matrix difference and output result of the new and old models; based on the standard answer, the problem correction state (complete / partial / failure) is determined, the weight correction coefficient is generated by calculating the directional repair rate, regression rate and error mode analysis, and then the dynamic optimization of the candidate information weight and the iterative screening of the sample set are driven. Through the establishment of the closed-loop evaluation and feedback mechanism, the model iteration effect can be accurately quantified, the residual error mode can be positioned, and the adaptive optimization of the training data can be realized, so that the accuracy and generalization ability of the intelligent agent are continuously improved.

[0018] In a second aspect, the present application also provides an intelligent agent continuous training and effect evaluation closed-loop system fusing work order feedback, which adopts the following technical scheme: An intelligent agent continuous training and effect evaluation closed-loop system fusing work order feedback, for realizing the intelligent agent continuous training and effect evaluation closed-loop method, comprising: A work order analysis module, configured to receive and analyze work order feedback information, record the number of error information and information storage time length in combination with a preset structure sample label; A target screening module, configured to dynamically determine the number of error information and information storage time length, trigger the retraining rule of the original intelligent agent model, assign the candidate information weight, and screen the target sample set; A model reconstruction module, configured to combine the target sample set according to the retraining rule to directionally train and optimize the original intelligent agent model, and generate a new intelligent agent model; A directional optimization module, configured to construct a historical work order problem set, and compare and verify the new intelligent agent model in combination with the original intelligent agent model to generate an intelligent agent evaluation table and correct the candidate information weight.

[0019] By adopting the above technical scheme, through the four-module closed-loop linkage of work order analysis, target screening, model reconstruction and directional optimization, the dynamic threshold algorithm is used to trigger the training rule, the sample weight is assigned in combination with the attention mechanism, and the parameter efficient fine-tuning technology is used for point repair or global optimization, and finally the evaluation table is generated through A / B test and regression analysis to iteratively correct the weight, so that the adaptive continuous optimization of the intelligent agent is realized, the answer accuracy is significantly improved, and the AI illusion is effectively suppressed.

[0020] In a third aspect, the present application also provides an electronic device, comprising: One or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method of any of the above schemes.

[0021] In a fourth aspect, the present application also provides a storage medium, wherein the storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement the intelligent agent continuous training and effect evaluation closed loop method for fusion work order feedback as in the above solutions.

[0022] In summary, the present application has the following beneficial technical effects: By calculating the content logical coherence and over-limit to determine the logical type (guiding type / accidental type / spontaneous type), combined with timing analysis to correct the alarm level; based on the knowledge graph association to match the core entity to determine the knowledge point label, through historical sample verification or tool calling timing comparison to locate the reasoning deviation node; finally, according to the client identification and error type classification, the number of error information after deduplication and storage time length are counted, which realizes the deep structured analysis and accurate attribution of error root cause of work order feedback; By adaptively selecting the optimal training path and fine-tuning parameter adjustment mechanism, the pertinence and efficiency of model iteration are significantly improved, so as to effectively repair known errors, enhance generalization ability, and significantly reduce AI hallucination and promote the accuracy of intelligent agents to approach the limit. BRIEF DESCRIPTION OF DRAWINGS

[0023] Figure 1 is the overall flowchart of the intelligent agent continuous training and effect evaluation closed loop method in the present application; Figure 2 is the flowchart of step S2 in the intelligent agent continuous training and effect evaluation closed loop method in the present application; Figure 3 is the flowchart of step S4 in the intelligent agent continuous training and effect evaluation closed loop method in the present application; Figure 4 is the structure diagram of the intelligent agent continuous training and effect evaluation closed loop system in the present application. DETAILED DESCRIPTION

[0024] The present application will be further described in detail below with reference to the accompanying drawings.

[0025] Reference Figure 1 A kind of intelligent agent continuous training and effect evaluation closed loop method for fusion work order feedback disclosed in the present application, comprising: S1: receiving and analyzing work order feedback information, combining with preset structure sample label, record error information number and information storage time length; S2: dynamically determine error information number and information storage time length, trigger the retraining rule of original intelligent agent model, distribute candidate information weight, and screen target sample set; S3: According to the retraining rule combined with the target sample set, the original intelligent agent model is trained and optimized to generate a new intelligent agent model; S4: Construct a historical work order problem set, and compare and verify the new intelligent agent model combined with the original intelligent agent model to generate an intelligent agent evaluation table and correct the candidate information weight.

[0026] The implementation principle of the embodiment is: through structured processing of feedback information and monitoring of error accumulation, when the dynamic threshold condition is met, the adaptive training rule engine is triggered, and based on attention weight distribution, a high-value sample set is screened, and then a differentiated fine-tuning algorithm is used to optimize the model, and finally through A / B testing evaluation by constructing a historical problem set and generating a weight correction coefficient, a complete closed-loop iterative system from problem discovery, accurate training to effect verification is formed, thereby continuously improving the performance of the intelligent agent.

[0027] Preferably, step S1 comprises: Receiving the work order feedback information of each client, identifying the client in advance, and recording the work order feedback frequency; Structurally analyzing the work order feedback information, extracting feedback alarm levels, behavior execution time sequence links, and resource state consumption data; In the embodiment, the work order feedback information records the key operations of the intelligent agent itself, such as starting, closing, configuration changing, service state switching, etc., and the fluctuations of system resource usage (such as CPU, memory, disk); records the instructions, query contents, operation requests input by the user, and the response output of the intelligent agent, which is convenient for backtracking the dialogue history and user intent; records the data, signals or events received by the intelligent agent from the external environment (such as sensors, APIs), and the processing results of these information; detailed records of errors, warnings, abnormal events occurring during system operation, including error codes, stack information, occurrence time, etc., for fault diagnosis and problem troubleshooting; records login attempts, permission changes, data access, security alerts and other security audit related events to support security analysis and threat detection; records performance data such as request processing time, response delay, throughput to evaluate intelligent agent efficiency and optimize resource allocation.

[0028] Extracting information from the execution time sequence link to obtain error context content and tool call time sequence; According to the information constraint boundary, the error context content is logically deconstructed to extract the content logical structure, calculate the logical coherence and content overrun; If the content overrun is greater than the upper limit value of the preset content limit interval, and the logical coherence is less than the lower limit value of the preset logical penetration value interval, it is determined that the current content logical structure is a guiding type of logic; If the content overrun is not in the content limit interval within the constraint number, and the logical coherence is in the logical coherence interval, it is determined that the current content logical structure is accidental logic; If the content overrun is in the content limit interval, and the logical coherence is greater than the upper limit value of the logical coherence interval, it is determined that the current content logical structure is spontaneous logic. In this embodiment, the guiding type logic includes triggering when the content overrun exceeds a preset threshold (such as > 70%) and the logical coherence is lower than a threshold (such as < 30%).

[0029] External factors (such as malicious injection, algorithm misdirection) actively destroy the logical chain, for example, by injecting irrelevant parameters to make the system execute an unexpected branch.

[0030] Malicious request interception in API security filtering (such as UrlHandleFilter detecting SQL injection and forcibly interrupting the process).

[0031] Accidental logic includes: the logical coherence is in the middle interval (30%~60%), and the content overrun temporarily breaks through the threshold and then falls back.

[0032] Local logic breakage caused by transient interference (such as network jitter, concurrent conflict), but the system self-healing mechanism can restore the main line.

[0033] Data synchronization process caused by network delay resulting in field missing, but the retry mechanism automatically repairs (retry logic of VectorDBClient.sync()).

[0034] Spontaneous logic includes: high logical coherence (> 80%) but content implies fundamental contradictions, and content overrun is not over-standard (< 20%).

[0035] Due to internal cognitive bias (such as false assumption) or data defect (such as training set bias), generate surface reasonable but essentially wrong output.

[0036] When querying vectors, due to configuration errors of associated tables (TableRelationForm definition errors), return logically self-consistent but actually incorrect results.

[0037] Time sequence the resource state consumption data, extract the abnormal state timestamp, and correct the feedback alarm level combined with the logical type to obtain the corrected alarm level. According to the preset structure sample label combined with the corrected alarm level, the semantic matching of the error context content is performed to obtain the core knowledge entity and the error expression segment. According to the knowledge point graph, the core knowledge entity is associated and matched to determine the knowledge point label. According to the historical same sample verification knowledge point label, if the labels are the same, the error expression segment is analyzed to determine the reasoning deviation node; If not, the tool call timing of the error expression segment is compared to determine the call deviation timing; According to the customer identification combined information error type, the work order feedback information is classified to obtain a single error information set; According to the work order feedback frequency, the single error information set is de-duplicated, combined with the correction warning level, the number of error information is counted, and the information storage time is recorded.

[0038] In this embodiment, the "standard answer package" attached to the work order is parsed to automatically extract core knowledge entities, error expression segments, and labeled error types (such as "factual error", "logical contradiction", and "time lag").

[0039] The extracted error expression is semantically matched with the preset and constantly improved error classification tree to determine the primary and secondary error types. At the same time, the core knowledge entities involved are associated with the nodes in the existing knowledge graph, and are labeled with knowledge points. It is also necessary to determine whether the classification of this time is logically consistent with the classification of the historical same sample. If a new error mode or knowledge point association is found, the classification system expansion audit process will be triggered to ensure that the archiving structure can remain stable and evolve.

[0040] Preferably, referring to Figure 2 , step S2 includes: According to the correction warning level, the information error type is prioritized and the quantitative threshold is determined to obtain the information storage amount and the information temporary storage period, and compared with the number of error information and the information storage time; If the number of error information reaches the information storage amount, and / or the information storage time reaches the information temporary storage period, it is determined that the current intelligent agent is in a training state, and a retraining instruction is generated; According to the retraining instruction combined with the number of error types, the training mechanism of the original intelligent agent model is selected to determine the retraining rule; If the number of error types does not reach the preset global trigger threshold, and the correction warning level is high risk, it is determined that the retraining rule is a local point repair rule; If the number of error types reaches the global trigger threshold, and the core knowledge entities are all known categories, it is determined that the retraining rule is a global parameter fine-tuning rule; If the number of error types reaches the global trigger threshold, but the core knowledge entities contain unknown categories, it is determined that the retraining rule is a reasoning node deepening rule; According to the retraining rule combined with the joint collaborative training algorithm, the corresponding node data in the original intelligent agent model is extracted to construct the original data item; According to the modified warning level, weight assignment is performed on the original data item to obtain a candidate information weight; If the candidate information weight is greater than a preset target information threshold, the original data item corresponding to the current candidate information weight is extracted, and is aggregated into a target sample set.

[0041] In this embodiment, the local point repair rule is: when the error type does not reach the global threshold and the correction warning is high risk, the data in the target sample set that is strongly associated with a specific error type (such as operation records and samples with abnormal reply frequency exceeding the standard) is screened.

[0042] The global parameter fine-tuning rule is: if the error type reaches the global threshold and the entities are all known categories, the joint feature vector of all abnormal replies in the sample set is extracted, and the event label is aligned based on the time sequence window.

[0043] The reasoning node deepening rule is: when there is an unknown category entity, the known entities and unknown entities in the sample are separated, and an independent new entity feature space (such as the expansion dimension of the action trajectory feature) is constructed.

[0044] Two main conditions are continuously monitored: one is the inventory condition, that is, checking whether the number of samples under each error type / knowledge point category has reached a preset "cumulative threshold", which can be dynamically adjusted according to the error severity or the historical performance of the model in this category; the second is the period condition, that is, judging whether it is a preset periodic training time point (such as every week / month). When multiple conditions are met or partially met, the arbitration logic is started, for example, the training task of the sample that accumulates the fastest (which indicates that the problem is prominent) or the error level of the highest category is triggered first, and at the same time, the tasks of adjacent periods can be combined.

[0045] Preferably, the joint collaborative training algorithm comprises: The original data items and the single-class error information set are respectively sampled to construct mixed data samples; The mixed data samples are trained in parallel according to the original intelligent agent model, and a language modeling loss function and an instruction following loss function are calculated; The language modeling loss function L CPT (θ) and the instruction following loss function L SFT (θ) are dynamically weighted according to the number of training steps to obtain a first-stage total loss function L Stage1 , and a new baseline model π Stage1 is determined therefrom. L Stage1 =λ CPT (t)⋅L CPT (θ)+λ SFT (t)⋅L SFT (θ); Where: λ CPT(t) = 1 - t / T is the language modeling weight coefficient, which decays as the training step increases; t is the current training step; T is the total training steps; and SFT (t) = t / T is the instruction following weight coefficient, which grows as the training step increases; t is the current training step; T is the total training steps; and is constrained and optimized according to the target sample set and the KL divergence regularization term, to calculate the total loss function L Stage1 of the second stage Stage2 (θ). L Stage2 (θ) = L DPO (θ) + γ · E x∼D [KL(π θ (· | x) | | π Stage1 (· | x))]. Wherein: L DPO (θ) is the optimization selection loss function; γ is the regularization strength hyperparameter; E x∼D [] is the full expectation function; π θ () is the new intelligent agent model; and x is the question and answer test instruction.

[0046] In this embodiment, the training process is divided into two core stages: The first stage: knowledge-skill joint training stage - CPT and SFT are carried out simultaneously.

[0047] The second stage: preference-aware robust optimization stage - DPO is introduced on the basis of joint training, and is constrained to prevent forgetting.

[0048] The first stage running mechanism and formula: In a specific field, “knowledge” (language model capability) and “skill” (instruction following capability) are simultaneously mastered.

[0049] Each training batch (Batch) is composed of two parts of data: Domain text data (for CPT): randomly sampled from large-scale unlabeled text in the target domain.

[0050] Instruction-answer pair data (for SFT): sampled from a high-quality SFT data set.

[0051] L CPT (θ) = -∑logP(w t | w <t ; θ) : standard language modeling loss function, acting on domain text data, w t is the weight coefficient.

[0052] L SFT (θ) = -∑logP(y t | y<t L(θ) = Lsupervised(θ), where Lsupervised(x; θ): standard supervised fine-tuning loss function operating on instruction-answer pair data; y t is the predicted output value.

[0053] λ CPT (t) and λ SFT (t) are dynamic weight coefficients that vary over time (training step t).

[0054] At the beginning of training, λ CPT is set to a high value (e.g., close to 1) and λ SFT is set to a low value (e.g., close to 0). This forces the model to focus primarily on absorbing domain knowledge.

[0055] As training progresses, the weight of λ CPT is smoothly reduced while the weight of λ SFT is increased. This means that the model gradually shifts its focus from learning to understanding and executing instructions. This avoids the rapid forgetting of CPT knowledge that can occur in traditional serial processes due to "subversive" training, achieving a smooth transition and mutual promotion of knowledge and skill learning.

[0056] L dpo = DPOLoss(batch dpo ; θ, π stage1 , β), calculates the standard DPO loss; L kl = KLDivergence(batch dpo ; θ, πstage1), calculates the KL divergence term (based on the question-answer test instructions x); The KL divergence term acts like a "rubber band" that "ties" the model being optimized to the baseline model.

[0057] If the model starts to deviate from its original correctness and completeness in order to excessively cater to the preference data (e.g., generating very short but possibly missing key information answers), the KL divergence value will increase sharply, resulting in a strong pulling force in the loss function, preventing such harmful deviation.

[0058] Preferably, step S3 comprises: According to the retraining rule combined with the information error type, the target sample set is associated and aligned, specific associated samples are selected, and a retraining tolerance value is calculated; If the retraining rule is a local point repair rule, feature extraction is performed on the error expression segment and the specific associated sample to generate a joint feature vector; If the retraining tolerance value is greater than the tolerance trigger value, weight calculation is performed on the error association node of the original intelligent agent model according to the joint feature vector, and a reoperation weight matrix is constructed; The weight matrix is dynamically iteratively trained until the constraint convergence parameter is reached, and a new agent model is generated; If the retraining rule is a global parameter fine-tuning rule, the historical connection rule and the resource channel capacity are fused to construct a global optimization objective function; According to the timing logic, the global associated node direction of the original agent model is constrained, and the real-time state of the resource channel capacity is obtained; If the real-time state is fluctuation, the learning rate is corrected according to the preset dynamic correction coefficient, and the weight distribution matrix of each topology level is updated to obtain a new agent model; If the retraining rule is a reasoning node deepening rule, the unknown class entity is weighted and distributed according to the error node expansion matrix, and an unknown entity feature matrix is created; According to the historical node association rule combined with the node level constraint, the unknown entity feature matrix is deeply reasoned, and the comprehensive weight matrix of the original agent model is fused to generate a new agent model.

[0059] In this embodiment, the constraint convergence parameter can be that the iteration error is less than 0.5% for three times in succession, or the capacity load is greater than 80% and the learning rate is 0.7.

[0060] The multiple attributes of this training task are comprehensively judged: the training target (is it a "point repair" for a specific error type or a "ability enhancement" in general?), the training data size and characteristics, the available computing resources and time window. For example, if the goal is to repair a single error type and the sample size is moderate, it is more inclined to choose parameter efficient fine-tuning (such as LoRA) to minimize resource consumption and avoid catastrophic forgetting; if the goal is to integrate a large amount of new knowledge across categories, a more comprehensive incremental learning strategy may be chosen; Before training begins, the selected samples are judged again according to their source work order verification authority, error typicality, problem complexity and other factors, and different training weights are dynamically allocated to make the model focus more on key errors. During the training process, key indicators such as loss function descent curve and performance on the reserved validation set are monitored in real time to determine whether the learning process is normal and effective. If there are abnormal indicators (such as loss not decreasing but increasing), it may be determined that overfitting or training strategy mismatch has occurred, and the system will automatically intervene to take corrective measures such as adjusting the learning rate, stopping early or rolling back to the checkpoint, to ensure that the training is proceeding in the right direction to improve the target ability.

[0061] Preferably, with reference to Figure 3 , step S4 comprises: According to the feedback alarm level, the historical feedback work order is error analyzed and clustered, a historical work order problem set is constructed, and the corresponding standard answer is obtained; Information deduplication and identification cleaning are performed on the historical work order problem set to obtain a work order problem set without identification; The work order problem set without identification is classified according to information error types to obtain single-class problem work order groups; Each single-class problem work order group is divided according to a preset data division ratio to obtain a single-class training group and a single-class verification group; All single-class training groups are mixed and crossed to obtain a work order problem training set; The work order problem training set is input into the original intelligent agent model and the new intelligent agent model for several times of iterative training, and an original verification weight matrix and a new verification weight matrix are extracted; The original verification weight matrix is compared with the question and answer weight matrix of the original intelligent agent model to extract a difference weight matrix; The original verification weight matrix and the new verification weight matrix are respectively output verified according to the single-class verification group to obtain an original output answer and a new output answer: If the original output answer is the same as the standard answer, it is determined that the difference weight matrix corrects the current class problem work order, and the self-corrected problem is counted; If the original output answer is different from the standard answer, it is determined that the original verification weight matrix is equivalent to the question and answer weight matrix; If the new output answer is the same as the standard answer, it is determined that the current class problem work order has been completely corrected successfully, and the corrected problem is marked; If the new output answer is partially the same as the standard answer, it is determined that the error mode of the current class problem work order is partially corrected successfully; If the new output answer is different from the standard answer, it is determined that the current class problem work order fails to correct, the number of error types is recorded, the correction warning level and the core knowledge body are marked, and the corresponding retraining rule is triggered; According to the total number of problems combined with the number of corrected problems, a directional repair rate is calculated; According to the total number of problems combined with the number of error types, a regression rate is calculated, and a corresponding weight correction coefficient is generated combined with the error mode; According to the directional repair rate, the regression rate and the error mode, an agent evaluation table is generated by performing attribution analysis on the new intelligent agent model; The weight correction coefficient is used to correct the candidate information weight, and a new sample set is selected.

[0062] In this embodiment, all relevant work orders are extracted from historical records, including detailed descriptions of each work order, classification labels (such as problem type, severity), solution steps and standard answers. Ensure that the problem set covers various scenarios and abnormal situations to comprehensively evaluate the model performance.

[0063] Remove duplicate tickets, clean incomplete or incorrect data, ensure the accuracy and consistency of the problem set; label and classify tickets according to problem type and severity, facilitate subsequent analysis. For example, classify problems into "troubleshooting", "configuration adjustment", "software update" and other categories.

[0064] Divide the problem set into training set, validation set and test set, ensure the uniform distribution and representativeness of problems in each set. Training set is used for model training, validation set is used for adjusting model parameters, and test set is used for final evaluation.

[0065] Build an evaluation analysis model, run the original agent model and the new agent model at the same time. Ensure that the two models are evaluated on the same problem set to ensure the fairness and accuracy of the comparison results.

[0066] For each ticket problem, record the output results of the original model and the new model, and compare them with the standard answer, and calculate: Correction success rate: whether the new model gets the correct answer on the problem it previously made mistakes.

[0067] Performance retention rate: whether the new model maintains the same performance as the original model on the originally correct problems, i.e. whether it causes regression.

[0068] Error pattern change: for problems that still have errors, analyze whether the error pattern has changed (e.g. from "complete error" to "partial correct").

[0069] Calculate the proportion of problems that the new model successfully corrects on the problems it previously made mistakes. The formula is as follows: Directional repair rate = number of corrected problems / total number of problems × 100%; This indicator reflects the effectiveness of the new model in improving specific errors.

[0070] Calculate the proportion of problems that the new model makes mistakes on the originally correct problems. The formula is as follows: Regression rate = number of problems with errors / total number of problems × 100%; This indicator is used to evaluate whether the new model introduces new errors.

[0071] For problems that still have errors, analyze the change in error patterns. For example, record cases where "complete error" changes to "partial correct", and evaluate the performance of the new model in the process of gradual improvement.

[0072] Summarize the results of directional repair rate, regression rate and error pattern analysis to form a detailed evaluation table, including specific values of each indicator, change trend and possible cause analysis; By analyzing the differences between the output results and the standard answers, the specific model parameters or structural adjustments are attributed. For example, if the correction success rate of a certain type of problem is found to be significantly improved, it can be traced back to the optimization of the model on the corresponding parameters.

[0073] Based on the evaluation results, further improvement suggestions are proposed. For example, if the new model performs poorly on a specific problem type, it may be suggested to increase the training of related samples or adjust the model structure to improve performance, and develop an adjustment strategy for candidate information weights. For example, if a certain feature has a greater impact on the model output, the weight of that feature can be appropriately increased.

[0074] A dynamic optimization algorithm is used to automatically adjust the candidate information weights based on real-time evaluation results. This ensures that the model gradually approaches the optimal weight distribution during continuous iteration.

[0075] After each weight adjustment, the evaluation system is re-run to verify the adjustment effect. If the improvement is significant, the adjustment result is retained; otherwise, it is rolled back to the previous version for continued iterative optimization.

[0076] The implementation principle of the embodiment is: through structured processing of client feedback, semantic analysis and logical coherence calculation (combined with a pre-set threshold to distinguish between guided, accidental and spontaneous logic) and combined with resource state time series analysis, the alarm level is corrected and core knowledge entities and error fragments are extracted; then, based on a dynamic threshold algorithm (monitoring error quantity and storage duration), a training rule engine is triggered, and according to the number of error types and entity categories, a differentiated training strategy is intelligently selected, such as local fixed-point repair (using joint feature vector construction and weight matrix iteration), global parameter fine-tuning (combining resource channel capacity constraints and dynamic learning rate adjustment) or reasoning node deepening (constructing unknown entity feature matrix for deep reasoning); Subsequently, a high-value sample set is selected using an attention weight distribution mechanism, and a parameter-efficient fine-tuning technique is used to optimize the model; finally, an A / B test evaluation is performed by constructing a historical work order problem set, a weight correction coefficient is generated based on the targeted repair rate, regression rate and error mode analysis, and a complete closed-loop optimization system is formed from problem discovery, strategy adaptive selection, precise training to effect verification and data weight iteration, thereby continuously improving the accuracy of the intelligent agent and effectively suppressing AI hallucinations.

[0077] Reference Figure 4 A closed-loop system for intelligent agent continuous training and effect evaluation combined with work order feedback is applied to a fault detection method, comprising: A work order analysis module is used to receive and analyze work order feedback information, combine pre-set structure sample labels, record error information quantity and information storage duration. The target screening module is configured to dynamically determine the number of error information and the information storage duration, trigger the retraining rule of the original intelligent agent model, assign the candidate information weight, and screen the target sample set. The model reconstruction module is configured to combine the target sample set according to the retraining rule, directionally train and optimize the original intelligent agent model, and generate a new intelligent agent model. The directional optimization module is configured to construct a historical work order problem set, combine the original intelligent agent model to verify the new intelligent agent model, generate an intelligent agent evaluation table, and correct the candidate information weight.

[0078] The implementation principle of the embodiment is as follows: the feedback information is analyzed and logically classified by the work order analysis module, the error accumulation is monitored by the dynamic threshold algorithm, and the differentiated training rule is triggered, the high-value samples are screened by using the attention weight distribution mechanism, then the model is locally fixed, globally optimized or deepened by using the parameter efficient fine-tuning technology, finally, the evaluation table is generated by A / B testing and regression analysis of the historical work order set, and the data weight is iteratively corrected, thereby forming a complete intelligent enhancement closed loop from problem perception to adaptive optimization.

[0079] An electronic device includes: one or more processors; a memory configured to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method of any of the above solutions.

[0080] A storage medium, the storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, at least one program, code set or instruction set is loaded and executed by a processor to implement the intelligent agent continuous training and effect evaluation closed loop method of fused work order feedback as in the above solutions.

[0081] The embodiments of the specific implementation are the preferred embodiments of the present application, and do not limit the protection scope of the present application, so that: any equivalent changes made according to the structure, shape, principle of the present application should be covered within the protection scope of the present application.

Claims

1. A closed-loop method for continuous training and performance evaluation of an agent that integrates work order feedback, characterized in that, include: Receive and parse work order feedback information, combine it with preset structured sample tags, and record the number of error messages and the information storage time; The number of error messages and the storage duration of the messages are dynamically determined, triggering the retraining rules of the original intelligent agent model, allocating candidate message weights, and filtering the target sample set. Based on the retraining rules and the target sample set, the original agent model is optimized through targeted training to generate a new agent model. Construct a set of historical work orders and compare and verify the new intelligent agent model with the original intelligent agent model to generate an intelligent agent evaluation table and correct the weights of the candidate information.

2. The closed-loop method for continuous training and effect evaluation of an agent based on integrated work order feedback as described in claim 1, characterized in that, The process of receiving and parsing work order feedback information, combined with preset structured sample tags, records the number of error messages and the information storage duration, including: Receive work order feedback information from each client, identify the client in advance, and record the frequency of work order feedback; The work order feedback information is structured and parsed to extract feedback alarm level, behavior execution timing chain and resource status consumption data; Information is extracted from the execution timing chain to obtain the error context content and tool call timing; Based on the information constraint boundaries, the error context content is logically deconstructed to extract the content logical structure and calculate the logical coherence and content exceedance. If the content exceeds the upper limit of the preset content restriction range and the logical coherence is less than the lower limit of the preset logical continuity range, then the current content logic structure is determined to be guided logic. If the content exceedance is not within the content restriction range within the constraint number of times, and the logical coherence is within the logical continuity value range, then the current content logical structure is determined to be accidental logic. If the content exceedance is within the content restriction range, and the logical coherence is greater than the upper limit of the logical continuity range, then the current content logic structure is determined to be spontaneous logic.

3. The closed-loop method for continuous training and effect evaluation of an agent based on integrated work order feedback as described in claim 2, characterized in that, The process of receiving and parsing work order feedback information, combined with preset structured sample tags, and recording the number of error messages and the information storage duration, also includes: The resource status consumption data is time-series processed, the timestamps of abnormal states are extracted, and the feedback alarm level is corrected based on the logical type to obtain the corrected alarm level. Based on the preset structural sample labels and the corrected alarm level, semantic matching is performed on the error context content to obtain core knowledge entities and error expression fragments; Based on the knowledge point graph, the core knowledge entities are associated and matched to determine the knowledge point tags; The knowledge point tags are verified based on historical similar samples. If the tags are the same, the erroneous statement fragments are analyzed to determine the reasoning deviation nodes. No, then the timing of the tool calls corresponding to the erroneous statement fragments is compared to determine if the call deviates from the timing sequence; Based on the client identifier and the information error type, the work order feedback information is classified to obtain a single-category error information set; The single-type error information set is deduplicated based on the frequency of work order feedback, and the number of error information is counted in conjunction with the corrected alarm level, and the information storage time is recorded.

4. The closed-loop method for continuous training and effect evaluation of an agent based on integrated work order feedback as described in claim 1, characterized in that, The dynamic determination of the number of error messages and the storage duration of the messages triggers the retraining rules of the original agent model, assigns weights to candidate messages, and filters the target sample set, including: Based on the corrected alarm level, the information error types are prioritized and quantitative thresholds are determined to obtain the information storage capacity and information temporary storage period, which are then compared with the number of error messages and the information storage time. If the number of error messages reaches the information storage capacity, and / or the information storage duration reaches the information temporary storage period, then the current agent is determined to be in a state of waiting for training, and a retraining instruction is generated. Based on the retraining instructions and the number of error types, the training mechanism for the original agent model is selected, and the retraining rules are determined. If the number of error types does not reach the preset global trigger threshold, and the correction warning level is high risk, then the retraining rule is determined to be a local fixed-point repair rule. If the number of error types reaches the global trigger threshold, and all core knowledge entities are known categories, then the retraining rule is determined to be a global parameter fine-tuning rule. If the number of error types reaches the global trigger threshold, but the core knowledge entity contains an unknown category, then the retraining rule is determined to be an inference node deepening rule. Based on the retraining rules and the joint collaborative training algorithm, the corresponding node data in the original agent model is extracted and trained to construct the original data items. The original data items are weighted according to the corrected warning level to obtain the candidate information weights; If the weight of the candidate information is greater than the preset target information threshold, the original data items corresponding to the current candidate information weight are extracted and aggregated into a target sample set.

5. The closed-loop method for continuous training and effect evaluation of an agent based on integrated work order feedback as described in claim 4, characterized in that, The joint collaborative training algorithm includes: Sample the original data items and the single-type error message set separately to construct a mixed data sample; The mixed data samples are trained in parallel based on the original intelligent agent model, and the language modeling loss function and instruction following loss function are calculated. The language modeling loss function L is based on the number of training steps. CPT (θ) and instruction follower loss function L SFT (θ) is dynamically weighted to obtain the total loss function L for the first stage. Stage1 And thereby determine the new benchmark model π Stage1 : L Stage1 =λ CPT (t)⋅L CPT (θ)+λ SFT (t)⋅L SFT (i); Where: λ CPT (t) = 1 − t / T is the language modeling weight coefficient, which decays as the number of training steps increases; λ SFT (t) = t / T is the instruction following weight coefficient, which increases with the number of training steps; t is the current number of training steps; T is the total number of training steps; θ is the model parameter; Based on the target sample set and the KL divergence regularization term, the new benchmark model π is applied. Stage1 Perform constrained optimization and calculate the total loss function L in the second stage. Stage2 (θ); L Stage2 (θ)=L DPO (θ)+γ⋅E x∼D [KL(π θ (⋅∣x)∣∣π Stage1 (⋅∣x))]; Where: L DPO (θ) represents the optimal selection loss function; γ is the regularization intensity hyperparameter; E x∼D [] represents the total expectation function; π θ () represents the new intelligent agent model; x represents the question-and-answer test instruction.

6. The closed-loop method for continuous training and effect evaluation of an agent based on integrated work order feedback as described in claim 1, characterized in that, The step of optimizing the original agent model by retraining according to the retraining rules and the target sample set, and generating a new agent model, includes: Based on the retraining rules and information error types, the target sample set is associated and aligned, specific associated samples are selected, and the retraining tolerance value is calculated. If the retraining rule is a local fixed-point repair rule, then feature extraction is performed on the erroneous statement fragment and the specific associated sample to generate a joint feature vector; If the retraining tolerance value is greater than the tolerance trigger value, then the weights of the erroneous associated nodes of the original agent model are calculated based on the joint feature vector to construct a reoperation weight matrix; The recomputation weight matrix is ​​dynamically iteratively trained until the constraint convergence parameter is reached, thereby generating a new intelligent agent model. If the retraining rule is a global parameter fine-tuning rule, then historical connection rules and resource channel capacity are integrated to construct a global optimization objective function; Constrain the global associated node direction of the original intelligent agent model according to the temporal logic, and obtain the real-time status of the resource channel capacity; If the real-time state is fluctuating, the learning rate is corrected according to the preset dynamic correction coefficient, and the weight allocation matrix of each topological level is updated to obtain the new intelligent agent model. If the retraining rule is an inference node deepening rule, then the unknown category entities are weighted according to the error node expansion matrix to create an unknown entity feature matrix; Based on historical node association rules and node hierarchy constraints, deep reasoning is performed on the feature matrix of the unknown entity, and the comprehensive weight matrix of the original intelligent agent model is fused and calculated to generate the new intelligent agent model.

7. The closed-loop method for continuous training and effect evaluation of an agent based on integrated work order feedback as described in claim 1, characterized in that, The process of constructing a historical work order problem set, comparing and validating the new intelligent agent model with the original intelligent agent model, generating an intelligent agent evaluation table, and correcting the candidate information weights includes: Based on the feedback alarm level, historical feedback work orders are clustered for error parsing to construct a set of historical work order issues and obtain the corresponding standard answers; The historical work order problem set is deduplicated and its identifiers are cleaned to obtain an unidentified work order problem set. The unidentified work order problem set is classified according to the type of information error to obtain single-category problem work order groups; The work order groups for each type of problem are divided according to the preset data division ratio to obtain single-type training groups and single-type validation groups; Perform a crossover operation on all the single-class training groups to obtain the work order problem training set; The work order problem training set is input into the original agent model and the new agent model respectively for several iterations of training, and the original verification weight matrix and the new verification weight matrix are extracted. The original verification weight matrix is ​​compared with the original question-answering weight matrix of the original agent model to extract the difference weight matrix; Based on the single-class validation set, the original validation weight matrix and the new validation weight matrix are validated to obtain the original output answer and the new output answer: If the original output answer is the same as the standard answer, then the difference weight matrix is ​​used to correct the current type of problem work order, and self-correcting problems are counted. If the original output answer differs from the standard answer, then the original verification weight matrix is ​​deemed equivalent to the question-answer weight matrix.

8. The closed-loop method for continuous training and effect evaluation of an agent based on integrated work order feedback as described in claim 7, characterized in that, The process of constructing a historical work order problem set, comparing and validating the new intelligent agent model with the original intelligent agent model, generating an intelligent agent evaluation table, and correcting the candidate information weights also includes: If the new output answer is the same as the standard answer, then the current type of problem ticket is determined to have been completely corrected and marked as a corrected problem; If the new output answer is partially the same as the standard answer, then the error pattern of the current type of problem ticket is determined to be partially corrected successfully. If the new output answer differs from the standard answer, the current problem work order is deemed to have failed to be corrected. The number of error types is recorded, the correction alarm level and core knowledge body are marked, and the corresponding retraining rule is triggered. The targeted repair rate is calculated based on the total number of issues and the number of issues that have been corrected. Based on the total number of problems and the number of error types, the regression rate is calculated, and based on the error patterns, the corresponding weight correction coefficients are generated. Based on the directional repair rate, the regression rate, and the error pattern, attribution analysis is performed on the new agent model to generate an agent evaluation table; The weights of the candidate information are adjusted according to the weight adjustment coefficients to select a new sample set.

9. A closed-loop system for continuous agent training and effect evaluation integrating work order feedback, used to implement the closed-loop method for continuous agent training and effect evaluation as described in any one of claims 1-8, characterized in that, include: The work order parsing module is used to receive and parse work order feedback information, and record the number of error messages and the information storage time by combining the preset structure sample tags. The target filtering module is used to dynamically determine the number of error messages and the storage duration of the messages, trigger the retraining rules of the original intelligent agent model, assign weights to candidate messages, and filter the target sample set. The model reconstruction module is used to perform targeted training and optimization of the original agent model based on the retraining rules and the target sample set, and generate a new agent model. The targeted optimization module is used to construct a set of historical work orders, compare and verify the new intelligent agent model with the original intelligent agent model, generate an intelligent agent evaluation table, and correct the weights of the candidate information.

10. A storage medium storing at least one instruction, at least one program, a code set, or an instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the closed-loop method for continuous training and effect evaluation of an agent as described in any one of claims 1 to 8.

Images