A question and answer intelligent agent model training method and system based on data labeling collaboration

Abstract

The application provides a kind of based on data annotation cooperation's question and answer intelligent agent model training method and system, belong to the technical field of model training, method includes to historical training set, external circulation optimization data and hierarchical annotation data are carried out data set arrangement;Incremental data set is carried out model parameter optimization to the question and answer intelligent agent model to be trained;Based on optimization model parameter carries out model deployment and scene adaptation;Based on model API endpoint, the model is tested and evaluated;From optimization suggestion, the optimization parameter is extracted, and based on evaluation score, the model is put online and operated;Optimization parameter, hierarchical data packet replace external circulation optimization data, hierarchical annotation data are carried out data set arrangement process, until the output target model, the target model is applied to question and answer intelligent agent, the application can avoid resource waste and improve model optimization efficiency, to improve the real-time performance, effectiveness and accuracy of government affairs intelligent question and answer.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of intelligent question and answer body model training, and particularly relates to a question and answer intelligent agent model training method and system based on data annotation collaboration. BACKGROUND

[0002] In the field of artificial intelligence technology, the quality of data annotation directly affects the model training effect, and the traditional annotation method has the following defects:

[0003] 1. Limitations of traditional data annotation

[0004] Current data annotation mainly relies on manual annotation teams or crowdsourcing platforms, which has problems such as high annotation cost, low efficiency, and unstable annotation quality. Especially in dynamic operation scenarios (such as real-time question and answer systems), the traditional annotation process cannot respond to feedback in real time, resulting in model update lag and difficulty in adapting to rapidly changing user needs.

[0005] 2. One-way problem of automatic learning

[0006] Existing automatic learning schemes (such as online learning and reinforcement learning) rely mainly on one-way data input and lack active use of user feedback. For example, user behaviors such as "like / dislike" of model output are not systematically collected or are only used for local parameter fine-tuning, without forming a closed-loop optimization mechanism, resulting in low model iteration efficiency.

[0007] 3. Optimization bottleneck of internal and external cycles

[0008] In mainstream solutions, user feedback and expert / high-level model optimization usually operate independently:

[0009] Internal cycle defects: self-optimization relying only on user annotation is prone to local optimization and difficult to break through the model capability boundary;

[0010] External cycle defects: optimization suggestions of external advanced models (such as GPT-4) lack real scenario feedback verification and have insufficient landing adaptability.

[0011] 4. Lack of collaboration mechanism

[0012] Existing technologies have not yet achieved multi-level collaboration of "crowd annotation - model self-learning - external intelligent agent optimization", and annotation data does not form a two-way flow between internal and external cycles, resulting in resource waste and optimization efficiency decay.

[0013] The defects of the above-mentioned traditional annotation methods will lead to the following defects of the actual question and answer intelligent agent:

[0014] 1. In the traditional project process, after changes in policies, regulations, required materials, and service locations, the intelligent agent is not updated in a timely manner, resulting in outdated answers returned by the system when users consult;

[0015] 2. The traditional single-cycle training method can lead to a single evaluation standard, and the high-frequency label is taken as the training target, which can lead to the generation of wrong answers. SUMMARY

[0016] To solve the above technical problems, the present application provides a question and answer intelligent agent model training method and system based on data labeling collaboration to solve the problems in the above background art.

[0017] In a first aspect, the present application provides the following technical solution, a question and answer intelligent agent model training method based on data labeling collaboration, comprising:

[0018] Obtaining target question and answer field knowledge, data organizing the target question and answer field knowledge and converging output according to the format of the question and answer intelligent agent to obtain the historical training set, obtaining the interaction information of the user to the output content of the question and answer intelligent agent to obtain the hierarchical labeling data, obtaining the outer loop optimization data output after three-dimensional evaluation of the question and answer intelligent agent, data set organizing the historical training set, the outer loop optimization data and the hierarchical labeling data to obtain the incremental data set;

[0019] Obtaining the question and answer intelligent agent model to be trained, model parameter optimization of the incremental data set to the question and answer intelligent agent model to be trained to obtain the optimized model parameter;

[0020] Model deployment and scene adaptation based on the optimized model parameter to obtain the model API endpoint;

[0021] Testing and evaluating the model based on the model API endpoint to obtain the evaluation score and optimization suggestion;

[0022] Extracting the optimization parameter from the optimization suggestion and performing model online and operation based on the evaluation score to obtain the hierarchical data package;

[0023] Replacing the outer loop optimization data and the hierarchical labeling data with the optimization parameter and the hierarchical data package to perform the data set organizing process to obtain a new data set, inputting the new data set into the question and answer intelligent agent model to be trained to perform optimization training to obtain a target model, and applying the target model to the question and answer intelligent agent.

[0024] Compared with the existing technology, the beneficial effects of the present application are as follows: the method of using data annotation collaboration and automation to assist in improving model training is embodied in the "annotation-learning dual-loop collaborative architecture", which includes an inner loop collaborative mechanism and an outer loop collaborative mechanism. The inner loop collaborative mechanism is: designing a lightweight annotation interaction component to achieve two-way communication between users and models, implementing selective parameter updates based on dynamic confidence thresholds, generating scene feature vectors by clustering annotation data features, and guiding the scenario-based deployment of model parameters; the outer loop reinforcement mechanism is: constructing a target multi-agent evaluation matrix: integrating deep reinforcement learning and generative adversarial networks to construct a three-dimensional evaluation system, adopting a gradient direction weighted fusion algorithm to achieve compatible training of crowd-annotated data and agent optimization parameters, establishing a version update strategy based on the Markov decision process, balancing model stability and evolution requirements, establishing a hierarchical flow channel for annotation data, and using high-frequency crowd annotations for real-time fine-tuning, thereby avoiding resource waste and improving model optimization efficiency. This can improve the accuracy of the question-answering agent's answers while avoiding the output of incorrect answers, and also achieve the output of real-time accurate answers.

[0025] Preferably, the step of arranging the historical training set, the outer loop optimization data, and the hierarchical labeled data to obtain an incremental data set includes:

[0026] Dynamically fuse the historical training set, the outer loop optimization data, and the hierarchical annotation data to obtain a fused data set :

[0027] ;

[0028] ;

[0029] Where, is the scaling factor, is the decay rate parameter, To mark the delay time, is the scaling factor, For the The semantic density of the data, For historical data sets, Respectively represent the outer loop optimization data and the hierarchical labeling data, is the data weight;

[0030] Performing scene clustering enhancement on the fused dataset using a K-means clustering method to obtain an enhanced dataset;

[0031] Conflicting samples and low-quality supplementary samples are removed from the enhanced dataset to obtain an incremental dataset.

[0032] Preferably, the step of performing model parameter optimization on the to-be-trained question and answer agent model by using the incremental data set to obtain optimized model parameters comprises:

[0033] inputting the incremental data set into the to-be-trained question and answer agent model and determining an inner loop gradient and an outer loop gradient ;

[0034] ;

[0035] ;

[0036] wherein, is a gradient, is a human-annotated loss function, respectively represent a model predicted value and a human expected value, is a KL divergence, 、 respectively represent output probability distributions of the to-be-trained question and answer agent model and a reference model;

[0037] traversing all to-be-updated parameters in the to-be-trained question and answer agent model, and calculating parameter gradients of the to-be-updated parameters based on the inner loop gradient and the outer loop gradient :

[0038] ;

[0039] wherein, is a gradient noise standard deviation, is a user gradient weight, is a gradient scaling factor, is a basic learning rate, is a conflict decay factor, is a function of extracting directional characteristics of gradient conflicts, is a Hadamard product;

[0040] calculating a gradient confidence of the parameter gradient , and if the gradient confidence is greater than a confidence threshold, performing model parameter optimization based on the inner loop gradient and the outer loop gradient to obtain optimized model parameters :

[0041] ;

[0042] wherein, is a to-be-updated model parameter, respectively learning rate coefficient of inner loop user gradient, learning rate coefficient of outer loop AI gradient, is an indicator function.

[0043] Preferably, the step of performing model deployment and scene adaptation based on the optimized model parameters to obtain a model API endpoint comprises:

[0044] obtaining a scene feature library, extracting a user environment feature vector matched with the user from the scene feature library, calculating a similarity matrix based on the user environment feature vector and the environment feature vector of the trained user, triggering knowledge transfer of the optimized model parameters and the trained model to obtain a model API endpoint if an element in the similarity matrix is greater than a preset similarity, wherein the similarity matrix is:

[0045] ;

[0046] wherein, respectively Query matrix and Key matrix in the trainable weight matrix, respectively user environment feature and environment feature vector of the trained user, is a feature projection dimension.

[0047] Preferably, the step of testing and evaluating the model based on the model API endpoint to obtain an evaluation score and an optimization suggestion comprises:

[0048] performing conflict test on the model based on the model API endpoint to obtain a conflict rate , outputting a first sub-result and triggering a rollback condition if the conflict rate is greater than a preset conflict rate, outputting a second sub-result if the conflict rate is not greater than the preset conflict rate, and determining a first test result based on the first sub-result and the second sub-result:

[0049] ;

[0050] wherein, is a core index of conflict determination;

[0051] performing response delay test on the model by a 99th percentile time statistical method and performing accuracy test on the model by a reserved validation set method to obtain a second test result and a third test result respectively;

[0052] integrating the first test result, the second test result and the third test result to obtain a test report;

[0053] obtaining test data and model production logs, and evaluating the model based on the target multi-agent, the test data, the model production logs and the test report to obtain an integrity score , an accuracy score , and an explainability score ;

[0054] based on the integrity score , an accuracy score , and an explainability score calculate an evaluation score :

[0055] ;

[0056] based on the evaluation score, perform defect positioning analysis on the model to obtain optimization suggestions.

[0057] Preferably, the step of based on the evaluation score, performing model online and operation to obtain a hierarchical data package comprises:

[0058] construct a version evolution decision tree, and determine whether the evaluation score of the current version of the model is greater than the sum of the evaluation score of the last version of the model and a first preset score, if the evaluation score of the current version of the model is greater than the sum of the evaluation score of the last version of the model and the first preset score, immediately online, if the evaluation score of the current version of the model is not greater than the sum of the evaluation score of the last version of the model and the first preset score, determine whether the evaluation score of the current version of the model is greater than the sum of the evaluation score of the last version of the model and a second preset score, if the evaluation score of the current version of the model is greater than the sum of the evaluation score of the last version of the model and the second preset score, perform AB test, if the evaluation score of the current version of the model is not greater than the sum of the evaluation score of the last version of the model and the second preset score, automatically roll back;

[0059] After the model is online, real-time labeling stream and scene feature library are obtained, if the same scene density of the real-time labeling stream is greater than a density threshold, L1 real-time stream is output, if the same scene density of the real-time labeling stream is not greater than the density threshold, determine whether the similarity of the real-time labeling stream and the scene feature in the scene feature library is less than a similarity threshold, if the similarity of the real-time labeling stream and the scene feature in the scene feature library is less than a preset similarity, L2 batch stream is output, if the similarity of the real-time labeling stream and the scene feature in the scene feature library is not less than the preset similarity, L3 sedimentation tank is output, to obtain a hierarchical data package.

[0060] In a second aspect, the present application provides the following technical solution, a question and answer intelligent agent model training system based on data labeling collaboration, the system comprises:

[0061] The collation module is configured to obtain target question and answer field knowledge, perform data collation on the target question and answer field knowledge, and output the data collation in a format of a question and answer intelligent agent to obtain a historical training set, obtain interaction information of a user on output content of the question and answer intelligent agent to obtain hierarchical labeling data, obtain external loop optimization data output after three-dimensional evaluation of the question and answer intelligent agent, and perform data set collation on the historical training set, the external loop optimization data, and the hierarchical labeling data to obtain an incremental data set.

[0062] The optimization module is configured to obtain a to-be-trained question and answer intelligent agent model, perform model parameter optimization on the to-be-trained question and answer intelligent agent model by using the incremental data set to obtain optimized model parameters.

[0063] The deployment module is configured to perform model deployment and scene adaptation based on the optimized model parameters to obtain a model API endpoint.

[0064] The evaluation module is configured to test and evaluate the model based on the model API endpoint to obtain an evaluation score and optimization suggestions.

[0065] The operation module is configured to extract optimized parameters from the optimization suggestions, perform model online and operation based on the evaluation score to obtain hierarchical data packets.

[0066] The output module is configured to replace the external loop optimization data and the hierarchical labeling data with the optimized parameters and the hierarchical data packets to perform a data set collation process to obtain a new data set, input the new data set into a to-be-trained question and answer intelligent agent model to perform optimization training to obtain a target model, and apply the target model to the question and answer intelligent agent.

[0067] Preferably, the collation module comprises:

[0068] The fusion sub-module is configured to dynamically fuse the historical training set, the external loop optimization data, and the hierarchical labeling data to obtain a fusion data set. :

[0069] ;

[0070] ;

[0071] In the formula, is a scaling factor, is a decay rate parameter, is a labeling delay time, is a scaling coefficient, is a semantic density, is a historical data set, respectively represent the external loop optimization data and the hierarchical labeling data, data weights;

[0072] an enhancer module, configured to perform scene clustering enhancement on the fusion dataset by using a K-means clustering method to obtain an enhanced dataset;

[0073] a rejection sub-module, configured to reject conflict samples and low-quality supplementary samples in the enhanced dataset to obtain an incremental dataset.

[0074] In a third aspect, the present application provides a computer, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the data annotation collaboration-based question and answer agent model training method as described above when executing the computer program.

[0075] In a fourth aspect, the present application provides a storage medium, and the storage medium stores a computer program, and the computer program is executable on a processor to implement the data annotation collaboration-based question and answer agent model training method as described above. BRIEF DESCRIPTION OF DRAWINGS

[0076] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0077] Figure 1 a flowchart of the data annotation collaboration-based question and answer agent model training method provided by the first embodiment of the present application;

[0078] Figure 2 a structural block diagram of the data annotation collaboration-based question and answer agent model training system provided by the second embodiment of the present application;

[0079] Figure 3 a hardware structure schematic diagram of the computer provided by another embodiment of the present application.

[0080] The present application will be further described below in combination with the drawings and the description of the drawings. DETAILED DESCRIPTION

[0081] The embodiments of the present application will be described in detail below, and the examples of the embodiments are shown in the drawings, wherein the same or similar reference signs represent the same or similar elements or elements with the same or similar functions throughout. The embodiments described below by referring to the drawings are exemplary and are intended to explain the embodiments of the present application, and cannot be understood as a limitation of the present application.

[0082] In the description of the embodiments of the application, it should be understood that the terms "length", "width", "upper", "lower", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer" and the like indicate the orientation or positional relationship shown in the drawings based on the orientation or positional relationship shown in the drawings, and are only for the purpose of facilitating the description of the embodiments of the application and simplifying the description, and do not indicate or imply that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation of the application.

[0083] In addition, the terms "first", "second" are only for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined with "first", "second" can explicitly or implicitly include one or more of the features. In the description of the embodiments of the application, the meaning of "multiple" is two or more, unless otherwise explicitly specified and limited.

[0084] In the embodiments of the application, unless otherwise explicitly specified and limited, the terms "mounting", "connection", "connection", "fixing" and the like should be understood in a broad sense, for example, it can be fixedly connected, or it can be detachably connected, or it can be integrated; it can be mechanically connected, or it can be electrically connected; it can be directly connected, or it can be indirectly connected through an intermediate medium; it can be the internal communication of two elements or the interaction relationship between two elements. For those skilled in the art, the specific meaning of the above terms in the embodiments of the application can be understood according to the specific circumstances.

[0085] Embodiment one

[0086] In one embodiment of the application, as shown in Figure 1 A question and answer intelligent agent model training method based on data annotation collaboration includes:

[0087] S1, obtaining target question and answer field knowledge, data arrangement is carried out on the target question and answer field knowledge, and convergent output is carried out according to the format of the question and answer intelligent agent, so as to obtain a historical training set, obtaining the interactive information of the user to the output content of the question and answer intelligent agent, so as to obtain the hierarchical labeling data, obtaining the outer loop optimization data output after the three-dimensional evaluation of the question and answer intelligent agent, data set arrangement is carried out on the historical training set, the outer loop optimization data and the hierarchical labeling data, so as to obtain an incremental data set;

[0088] Specifically, for the data set arrangement here, the hierarchical annotation data issued by the user is obtained through the inner loop, the optimization sample generated by the high-order model is obtained through the outer loop, that is, the outer loop optimization data, and the value of the new and old data is balanced through the weight function, and for the historical training set, first, rules, systems, policies, knowledge bases, etc. are obtained from various government departments, and are aggregated into a basic training set in the format required by the government affair matter question and answer agent through artificial arrangement and system data arrangement, forming a historical training set. For the outer loop optimization data and the hierarchical annotation data, 1) integrate deep reinforcement learning (such as DeepSeek-R1) and generative adversarial network (such as GPT-4o) to build a three-dimensional evaluation system as an outer loop optimization means to optimize and refine data to obtain outer loop optimization data; 2) through the hierarchical annotation data issued by the user, an instant feedback channel is formed to realize the bidirectional communication between the user and the model through a lightweight annotation interaction component (three-state annotation of like / dislike / supplement) to obtain hierarchical annotation data. Finally, through an incremental automatic learning engine: based on a dynamic confidence threshold (dynamic threshold = basic threshold x annotation density correction coefficient) to implement selective parameter update, the initial basic training set is optimized and arranged;

[0089] At the same time, the hierarchical here specifically refers to the fine classification of user feedback. For example: simply classifying feedback into "satisfied" and "dissatisfied" is a two-level classification; adding "partially satisfied" and "needs improvement" dimensions forms a multi-level classification. This hierarchical classification can guide the model optimization direction better than binary feedback.

[0090] The step S1 includes:

[0091] S11, dynamically fuse the historical training set, the outer loop optimization data and the hierarchical annotation data to obtain a fused data set :

[0092] ;

[0093] ;

[0094] In the formula, is a scaling factor, is a decay rate parameter, is a labeling delay time, is a scaling coefficient, is the semantic density of the i-th data, is a historical data set, respectively represent the outer loop optimization data and the hierarchical annotation data, is a data weight;

[0095] ​Specifically, in the dynamic fusion process here, the data in the sedimentation pool needs to be skipped, and at the same time Item improves timeliness, that is, new labeled data is learned preferentially, and the semantic density is weighted Item, the user supplements the content to strengthen learning, and the dynamic fusion process here is specifically: the external loop optimization data and the hierarchical labeled data are taken as new data points, that is, the new data points are merged with the historical training set, the merging process is the fusion process, and each new data point is assigned a data weight, which quantifies the relative importance or reliability of the new data point at the current fusion time, so that the fusion data set includes historical data and weighted new data, and through the weighted fusion data set in the subsequent model training process, weighted fusion is realized;

[0096] The semantic density here specifically refers to the proportion of meaningful data in a certain amount of data, that is, in a data segment with an amount of A, B data are meaningful, that is, have semantic information, and the semantic density of the data is B / A.

[0097] S12, the K-means clustering method is used for scene clustering enhancement on the fusion data set to obtain an enhanced data set.

[0098] Specifically, the scene clustering enhancement process here is specifically: first, the BERT model pre-trained by the question and answer data is used to extract the semantic features of the questions, the semantic features of the answers and the context of the question and answer text in the fusion data set, then the above three features are spliced to obtain a comprehensive feature data, and then the K-means clustering method is used for clustering on the spliced comprehensive feature data. In the clustering process, there will be some outlier data points that cannot be divided into any cluster, so in the actual determination of the enhanced data, the outlier data will be removed, and the data within the cluster will be retained, that is, the enhanced data set can be obtained. At the same time, the data can be enhanced through the semantic features of the questions, the semantic features of the answers and the context, so that the enhanced data set can provide more explicit scene information. The obtained enhanced data set can realize dimension reduction and abstraction, reveal the potential scene categories or patterns in the data that are difficult for humans to define, similar scenes are classified into the same category, and the model can learn to process "one category of scenes" rather than countless independent scenes with subtle differences, which is helpful for generalization. The enhanced data set provides clear scene category information for the downstream task, so that the generated scene representation can better reflect the real environment.

[0099] S13, removing conflict samples and low-quality supplementary samples in the enhanced data set to obtain an incremental data set;

[0100] Specifically, in actual cases, there are multiple labeling conflicts, that is, data with a deletion conflict rate greater than a preset conflict rate is deleted, the preset conflict rate is 0.4, and the conflict rate is calculated as follows: for each sample, collect all the labels of the annotators, and determine the majority vote result. For example, if there are K annotators, the majority opinion is the label with the most support, the conflict rate = (the number of annotators who disagree with the majority opinion) / (the total number of annotators), and low-quality supplementary data is removed, that is .

[0101] S2, obtain a to-be-trained question and answer intelligent agent model, and perform model parameter optimization on the to-be-trained question and answer intelligent agent model based on the incremental data set to obtain optimized model parameters.

[0102] Specifically, in the process of model training, the inner loop is based on user labeling to calculate a light gradient, and the outer loop is to inject a high-order model optimization gradient, and a conflict attenuation term eliminates optimization direction conflicts and dynamically weighted balance double-loop training intensity.

[0103] The step S2 includes:

[0104] S21, input the incremental data set into the to-be-trained question and answer intelligent agent model and determine an inner loop gradient and an outer loop gradient .

[0105] .

[0106] .

[0107] In the formula, is a gradient, is a manual labeling loss function, respectively represent a model predicted value, a manual expected value, is a KL divergence, , respectively represent an output probability distribution of the to-be-trained question and answer intelligent agent model and a reference model;

[0108] Specifically, in this step, the inner loop gradient is the gradient of the manually labeled data, which maintains the sensitivity to user feedback, and the outer loop gradient is the gradient of AI optimization, which introduces professional knowledge.

[0109] S22, traverse all to-be-updated parameters in the to-be-trained question and answer intelligent agent model, and calculate parameter gradients of the to-be-updated parameters based on the inner loop gradient and the outer loop gradient :

[0110] ;

[0111] wherein, is a gradient noise standard deviation, is a user gradient weight, is a gradient scaling factor, is a base learning rate, is a conflict decay factor, is a function of extracting directional characteristics of gradient conflicts, is a Hadamard product;

[0112] Specifically, the user gradient weight is usually determined by the labeling density, and the gradient scaling factor is used to control the gradient amplitude, to provide a directional guide for the penalty term.

[0113] S23, calculating the gradient confidence of the parameter gradient If the gradient confidence is greater than a confidence threshold, then based on the inner loop gradient and the outer loop gradient perform model parameter optimization to obtain optimized model parameters :

[0114] ;

[0115] wherein, is a model parameter to be updated, are learning rate coefficients of the inner loop user gradient and the outer loop AI gradient, respectively, is an indicator function;

[0116] Specifically, for parameters with a gradient confidence not greater than a confidence threshold, they are stored in an outer loop optimization pool for the next round of optimization process. The gradient confidence here is used to quantify the reliability of the gradient. The gradient confidence can be obtained by calculating the variance of the parameter gradient and dividing the variance by the square of the norm of the parameter gradient. The indicator function is an Indicator Function.

[0117] S3, based on the optimized model parameters, performing model deployment and scene adaptation to obtain a model API endpoint;

[0118] The step S3 includes:

[0119] obtaining a scene feature library, extracting a user environment feature vector matching the user from the scene feature library, calculating a similarity matrix based on the user environment feature vector and the environment feature vector of the trained user, if an element in the similarity matrix is greater than a preset similarity, triggering knowledge transfer of the optimized model parameter and the trained model to obtain a model API endpoint, wherein the similarity matrix is:

[0120]

[0121] wherein, are respectively a Query matrix and a Key matrix in a trainable weight matrix, respectively represent a user environment feature and an environment feature vector of a trained user, is a feature projection dimension;

[0122] wherein, is a specific activation function, which normalizes the input so that the output probability value represents the possibility of the input belonging to each category. Since the denominator of the Softmax function contains the sum of the exponential terms of all categories, the output is affected by each element in the input vector, i.e., the relative size of each category.

[0123] Specifically, in the actual migration process, the environment feature vector of the trained user has trained the model, and the trained model is denoted as a trained model. The purpose of knowledge transfer is to trigger the current user's corresponding model to learn knowledge from the trained model through the similarity matrix, and to transfer the model parameters in the trained model to the current user's corresponding model through knowledge transfer. The preset similarity is 0.8, and when it is greater than 0.8, knowledge transfer is triggered, and the knowledge retention rate is The actual processing process includes scene adaptation migration, server capacity conversation, traffic management, blue-green deployment for 10% traffic diversion of the new version, automatic scaling for dynamically adjusting the number of replicas according to QPS, after obtaining the model API endpoint, the downstream transmits the model API endpoint to the test platform and the online user. In actual situations, in the process of obtaining the optimized model parameter, the user feature vector of one of the users has been used for training. The user feature vector that has been trained here is the environment feature vector of the trained user. Then, whether to trigger the migration operation is determined by the similarity matrix between the user and any other user. If it is greater than the preset similarity, the migration operation is triggered. At the same time, in the process of determining the user environment feature vector, the user's environment feature is first obtained according to the user-provided question and answer context and the user portrait analysis method in the prior art, and then the cosine similarity between the environment feature and the vector stored in the scene feature library is calculated. The vector with the highest similarity is taken as the user environment feature vector.

[0124] ​S4, testing and evaluating the model based on the model API endpoint to obtain an evaluation score and optimization suggestions;

[0125] The step S4 comprises:

[0126] S41, performing conflict testing on the model based on the model API endpoint to obtain a conflict rate If the conflict rate is greater than a preset conflict rate, a first sub-result is output and a rollback condition is triggered, and if the conflict rate is not greater than the preset conflict rate, a second sub-result is output, and a first test result is determined based on the first sub-result and the second sub-result:

[0127] ;

[0128] In the formula, is a core index of conflict determination;

[0129] Specifically, the preset conflict rate is 0.4, the conflict rate is a gradient direction conflict, and the core index of conflict determination is used for global quantification of a model parameter space. If the conflict rate is not greater than the preset conflict rate, the corresponding conflict rate is taken as the first test result. The core index of conflict determination includes multi-person labeling errors, gradient direction conflicts, and the like. After the first sub-result is output, a rollback condition is triggered. If the conflict rate is not greater than the preset conflict rate, the corresponding data is high-quality data, and the conflict rate and the corresponding data are directly taken as the second sub-result. The first test result is obtained by comprehensively considering the first sub-result and the second sub-result of all data. The conflict rate of multi-person labeling errors has been described in step S13. The conflict rate of the gradient direction conflict is calculated as follows: for each data, the inner loop gradient and the outer loop gradient are calculated, the dot product of the inner loop gradient and the outer loop gradient is determined, and it is determined whether the dot product result is less than 0. If it is less than 0, it indicates that the gradient directions are opposite, and the data is marked as having gradient conflict. Then, the number of data having gradient conflict is divided by the total number of data to obtain the conflict rate of the gradient direction conflict. Therefore, in the present application, the number of core indexes is 2, which are the gradient direction conflict and the multi-person labeling conflict.

[0130] S42, performing response delay testing on the model by a 99th percentile time statistical method and performing accuracy testing on the model by a reserved validation set method to obtain a second test result and a third test result, respectively;

[0131] Specifically, if the response delay is greater than the preset delay or the accuracy test result is less than the preset accuracy, the corresponding response delay and accuracy test result are output as the second test result and the third test result, and then a rollback condition is triggered; if the response delay is not greater than the preset delay or the accuracy test result is not less than the preset accuracy, the response delay, the accuracy test result and the corresponding data are stored in the second test result and the third test result respectively.

[0132] S43, the first test result, the second test result and the third test result are integrated to obtain a test report.

[0133] S44, test data and model production logs are obtained, and the model is evaluated based on the target multi-agent, the test data, the model production logs and the test report to obtain an integrity score , an accuracy score and an explainability score ;

[0134] Specifically, the target multi-agent here is DeepSeek agent, GPT-4 agent and LIME interpreter, the integrity score is to check whether the model output covers all necessary information points, the accuracy score is to verify the authenticity of the output content, and the explainability score is to analyze the transparency of the model decision.

[0135] S45, based on the integrity score , the accuracy score and the explainability score , an evaluation score is calculated:

[0136] ;

[0137] S46, based on the evaluation score, defect positioning analysis is performed on the model to obtain optimization suggestions;

[0138] Specifically, in the actual defect positioning process, the completeness, accuracy, and explainability are checked in turn, and it is judged whether the score of each dimension is lower than the preset threshold. If it is found that the score of a certain dimension is lower than the threshold, the optimization_target function is immediately called, and the dimension is passed in as a parameter, and the result is returned. If all dimensions meet the threshold requirement (score ≥ threshold), the function will implicitly return after traversing all dimensions, and this place depends on external variables, including the optimization_target function, which generates an optimization target according to the dimension parameter and generates an optimization suggestion, such as the AI optimization direction. For example, if the evaluation score is low, optimization suggestions can be made from the completeness score , accuracy score , and explainability score from three angles. For completeness, knowledge graph expansion, prompt engineering, post-processing completion, and training data enhancement can be performed. For accuracy, real-time data access, multi-source verification reinforcement, confidence threshold warning, and adversarial training can be performed. For explainability, forced reasoning chain, attention visualization, explanation template, and explainability fine-tuning can be performed. Comprehensive optimization suggestions can also be made, such as scene priority: adjusting the dimension weight according to the application scenario (such as prioritizing accuracy in medical scenarios and prioritizing explainability in education scenarios); iterative optimization cycle: periodically collecting user feedback and re-evaluating the three scores for continuous iteration; and multi-agent collaboration: calling special repair agents when a certain dimension score is low.

[0139] Specifically, the completeness score calculation process is as follows: first, define a set of key points, then check how many key points are covered in the model output, and take the ratio between the covered key points and the number of defined key points as the completeness score. The accuracy score calculation process is as follows: compare the factual statements (such as dates, data, events, etc.) in the model output with the trusted knowledge sources (such as databases, authoritative websites), use the analysis ability of GPT-4 agent to judge whether the statements conform to known facts or logical consistency, and take the ratio between the number of correct statements and the total number of statements as the accuracy score. The explainability score calculation process is as follows: use LIME interpreter to perform word segmentation or sentence segmentation on the model output and extract features, observe the output changes by perturbing the input, determine the key features, and score according to the explicitness of the key features in the output (such as whether they are explicitly mentioned or have logical connection words) to obtain the explainability score.

[0140] S5, extract optimization parameters from the optimization suggestions, and perform model online and operation based on the evaluation score to obtain hierarchical data packets;

[0141] wherein the step S5 comprises:

[0142] S51, a version evolution decision tree is constructed, and it is judged whether the evaluation score of the model of the current version is greater than the sum of the evaluation score of the model of the last version and a first preset score, if the evaluation score of the model of the current version is greater than the sum of the evaluation score of the model of the last version and the first preset score, it is immediately put online, if the evaluation score of the model of the current version is not greater than the sum of the evaluation score of the model of the last version and the first preset score, it is judged whether the evaluation score of the model of the current version is greater than the sum of the evaluation score of the model of the last version and a second preset score, if the evaluation score of the model of the current version is greater than the sum of the evaluation score of the model of the last version and the second preset score, AB testing is performed, if the evaluation score of the model of the current version is not greater than the sum of the evaluation score of the model of the last version and the second preset score, it is automatically rolled back;

[0143] Here, the first preset score is 0.15. The second preset score is 0.05.

[0144] S52, after the model is put online, real-time labeling flow and scene feature library are obtained, if the same scene density of the real-time labeling flow is greater than a density threshold, L1 real-time flow is output, if the same scene density of the real-time labeling flow is not greater than the density threshold, it is judged whether the similarity of the real-time labeling flow and the scene feature in the scene feature library is less than a similarity threshold, if the similarity of the real-time labeling flow and the scene feature in the scene feature library is less than a preset similarity, L2 batch flow is output, if the similarity of the real-time labeling flow and the scene feature in the scene feature library is not less than the preset similarity, L3 sediment pool is output, to obtain a hierarchical data package;

[0145] Specifically, the real-time labeling flow specifically refers to the labeling evaluation data of the model output by the user, such as scoring, “like / dislike” behavior, etc. The same scene density here can be expressed as the ratio between the scene feature points of the real-time labeling flow and the feature length of the real-time labeling flow;

[0146] At the same time, L1 real-time flow is high-value real-time data, which is directly used for model iteration, L2 batch flow needs to be reviewed, which belongs to medium-value data, and can be used for model iteration after review, L3 sediment pool is equivalent to an observation area, which is redundant data, therefore, the hierarchical data package includes data in one of L1 real-time flow, L2 batch flow and L3 sediment pool, but when the condition of “high density” is met, L1 real-time flow is output, when the condition of “low density and low similarity” is met, L2 batch flow is output, when the condition of “low density and high similarity” is met, L3 sediment pool is output, at the same time, the purpose of the hierarchy is to allocate the real-time labeling flow to different processing paths according to the density and the similarity with historical scene features, so as to realize the hierarchical process.

[0147] S6, replacing the optimization parameters, the hierarchical data packet, the outer loop optimization data, the hierarchical labeled data with a data set collation process to obtain a new data set, inputting the new data set into a to-be-trained question and answer intelligent agent model for optimization training to obtain a target model, and applying the target model to a question and answer intelligent agent;

[0148] Specifically, after obtaining the optimization parameters and the hierarchical data packet, the above steps S1-S5 are repeated by replacing the optimization parameters and the hierarchical data packet with the outer loop optimization data and the hierarchical labeled data, that is, the optimization parameters and the hierarchical data packet are actually the answers generated by the model in real time, and the user operation behavior log. The user triggers instant feedback (like / dislike / supplement) by clicking, activates the supplement editing interface by long pressing, automatically collects environmental features (device type / position / network status, etc.), simultaneously performs multi-modal labeling and coding, that is, emotion vector, semantic supplement embedding and 6-dimensional scene features, generates labeled annotation triples according to dynamic weights, and then repeats the above steps until the optimal target model is output. After that, the optimal target model is used as the main part of the intelligent question and answer to execute the process of intelligent question and answer output.

[0149] It should be noted that the question and answer intelligent agent model training method based on data labeling collaboration provided by the application is applied to model training of a government affair intelligent question and answer system, and is aimed at improving model performance and improving the real-time performance, effectiveness and accuracy of the government affair intelligent question and answer.

[0150] In summary, the question and answer intelligent agent model training method based on data labeling collaboration in the above embodiments of the application uses data labeling collaboration and automatic assistance to improve the model training method, which is embodied in a "labeling-learning double-loop collaborative architecture" including an inner loop collaborative mechanism and an outer loop collaborative mechanism. The inner loop collaborative mechanism is: a lightweight labeling interaction component is designed to realize bidirectional communication between the user and the model, selective parameter updating is implemented based on a dynamic confidence threshold, scene feature vectors are generated through labeling data feature clustering to guide the scene deployment of model parameters; and the outer loop reinforcement mechanism is: a target multi-agent evaluation matrix is constructed: a three-dimensional evaluation system is constructed by integrating deep reinforcement learning and a generative adversarial network, a gradient direction weighted fusion algorithm is used to realize compatible training of crowd labeling data and intelligent agent optimization parameters, a version update strategy is established based on a Markov decision process to balance model stability and evolution needs, a labeling data grading circulation channel is established, and high-frequency crowd labeling is used for real-time fine-tuning to avoid resource waste and improve model optimization efficiency. In this way, the accuracy of the question and answer intelligent agent can be improved while avoiding the output of incorrect answers, and real-time and accurate answers can also be output.

[0151] Embodiment two

[0152] As Figure 2As shown, the embodiment two of the present application provides a question and answer intelligent agent model training system based on data annotation collaboration, which comprises:

[0153] The arrangement module 1 is used for obtaining target question and answer field knowledge, performing data arrangement on the target question and answer field knowledge, and converging output according to the format of the question and answer intelligent agent to obtain a historical training set, obtaining interaction information of a user on the output content of the question and answer intelligent agent to obtain hierarchical labeling data, obtaining external loop optimization data output after three-dimensional evaluation of the question and answer intelligent agent, and performing data set arrangement on the historical training set, the external loop optimization data and the hierarchical labeling data to obtain an incremental data set.

[0154] The optimization module 2 is used for obtaining a to-be-trained question and answer intelligent agent model, performing model parameter optimization on the to-be-trained question and answer intelligent agent model by using the incremental data set to obtain optimized model parameters.

[0155] The deployment module 3 is used for model deployment and scene adaptation based on the optimized model parameters to obtain a model API endpoint.

[0156] The evaluation module 4 is used for testing and evaluating the model based on the model API endpoint to obtain an evaluation score and optimization suggestions.

[0157] The operation module 5 is used for extracting optimized parameters from the optimization suggestions and performing model online and operation based on the evaluation score to obtain hierarchical data packets.

[0158] The output module 6 is used for replacing the external loop optimization data and the hierarchical labeling data with the optimized parameters and the hierarchical data packets to perform a data set arrangement process to obtain a new data set, inputting the new data set into a to-be-trained question and answer intelligent agent model to perform optimization training to obtain a target model, and applying the target model to the question and answer intelligent agent.

[0159] The arrangement module 1 comprises:

[0160] The fusion sub-module is used for dynamically fusing the historical training set, the external loop optimization data and the hierarchical labeling data to obtain a fusion data set. :

[0161] ;

[0162] ;

[0163] In the formula, is a scaling factor, is a decay rate parameter, is a labeling delay time, is a scaling coefficient, the semantic density of the i-th data, the historical data set, respectively represent the outer loop optimization data and the hierarchical annotation data, the data weight;

[0164] an enhancer module, configured to perform scene clustering enhancement on the fusion data set by using a K-means clustering method to obtain an enhanced data set;

[0165] a pruning submodule, configured to remove conflict samples and low-quality supplementary samples in the enhanced data set to obtain an incremental data set.

[0166] The optimization module 2 comprises:

[0167] a gradient submodule, configured to input the incremental data set into the to-be-trained question and answer intelligent agent model and determine an inner loop gradient and an outer loop gradient ;

[0168] ;

[0169] ;

[0170] wherein, is a gradient, is a manual annotation loss function, respectively represent a model predicted value and a manual expected value, is a KL divergence, , respectively represent an output probability distribution of the to-be-trained question and answer intelligent agent model and a reference model;

[0171] a traversal submodule, configured to traverse all to-be-updated parameters in the to-be-trained question and answer intelligent agent model, and calculate a parameter gradient of the to-be-updated parameters based on the inner loop gradient and the outer loop gradient :

[0172] ;

[0173] wherein, is a gradient noise standard deviation, is a user gradient weight, is a gradient scaling factor, is a basic learning rate, is a conflict decay factor, is a function of extracting the directional characteristics of gradient conflicts, is a Hadamard product;​

[0174] an updating parameter submodule, configured to calculate a gradient confidence of the parameter gradient, and if the gradient confidence is greater than a confidence threshold, perform model parameter optimization based on the inner loop gradient and the outer loop gradient to obtain optimized model parameters

[0175]

[0176] wherein, w is a model parameter to be updated, and are learning rate coefficients of an inner loop user gradient and an outer loop AI gradient respectively, and is an indicator function. The deployment module 3 is configured to:

[0177] obtain a scene feature library, extract a user environment feature vector matching the user from the scene feature library, calculate a similarity matrix based on the user environment feature vector and an environment feature vector of a trained user, and if an element in the similarity matrix is greater than a preset similarity, trigger knowledge transfer of the optimized model parameters and a trained model to obtain a model API endpoint, wherein the similarity matrix is:

[0178]

[0179]

[0180] wherein, are a Query matrix and a Key matrix in a trainable weight matrix respectively, represent a user environment feature and an environment feature vector of a trained user respectively, and is a feature projection dimension. The evaluation module 4 includes:

[0181] a first test submodule configured to perform conflict testing on a model based on the model API endpoint to obtain a conflict rate, if the conflict rate is greater than a preset conflict rate, output a first sub-result and trigger a rollback condition, if the conflict rate is not greater than the preset conflict rate, output a second sub-result, and determine a first test result based on the first sub-result and the second sub-result:

[0182]

[0183] wherein, is a core index of conflict determination.

[0184]

[0185] ​​​​​​​​​​​a second test submodule configured to perform a response delay test on the model by using a 99th percentile time statistical method and perform an accuracy test on the model by using a reserved validation set method, to obtain a second test result and a third test result, respectively;

[0186] a comprehensive submodule configured to comprehensively integrate the first test result, the second test result, and the third test result to obtain a test report;

[0187] a score submodule configured to obtain test data and a model production log, and evaluate the model based on a target multi-agent, the test data, the model production log, and the test report to obtain an integrity score, an accuracy score, and an explainability score; ;

[0188] an evaluation submodule configured to calculate an evaluation score based on the integrity score, the accuracy score, and the explainability score:

[0189] ;

[0190] a positioning submodule configured to perform a defect positioning analysis on the model based on the evaluation score to obtain an optimization suggestion.

[0191] The operation module 5 includes:

[0192] a construction submodule configured to construct a version evolution decision tree, and determine whether an evaluation score of a model of a current version is greater than a sum of an evaluation score of a model of a previous version and a first preset score; if the evaluation score of the model of the current version is greater than the sum of the evaluation score of the model of the previous version and the first preset score, the model of the current version is immediately put into use; if the evaluation score of the model of the current version is not greater than the sum of the evaluation score of the model of the previous version and the first preset score, the submodule determines whether the evaluation score of the model of the current version is greater than a sum of the evaluation score of the model of the previous version and a second preset score; if the evaluation score of the model of the current version is greater than the sum of the evaluation score of the model of the previous version and the second preset score, the model of the current version is subjected to an AB test; if the evaluation score of the model of the current version is not greater than the sum of the evaluation score of the model of the previous version and the second preset score, the model of the current version is automatically rolled back;

[0193] ​​​​​​The hierarchical sub-module is used to obtain a real-time labeling stream and a scene feature library after the model is put online, and if the same scene density of the real-time labeling stream is greater than a density threshold, an L1 real-time stream is output, if the same scene density of the real-time labeling stream is not greater than the density threshold, whether the similarity of the real-time labeling stream and a scene feature in the scene feature library is less than a similarity threshold is judged, if the similarity of the real-time labeling stream and the scene feature in the scene feature library is less than a preset similarity, an L2 batch stream is output, if the similarity of the real-time labeling stream and the scene feature in the scene feature library is not less than the preset similarity, an L3 sediment pool is output, so as to obtain a hierarchical data packet.

[0194] In some embodiments of the present application, the embodiments of the present application provide the following technical solutions, a computer comprising a memory 102, a processor 101 and a computer program stored on the memory 102 and executable on the processor 101, wherein the processor 101 implements the data labeling collaborative based question and answer agent model training method as described above when executing the computer program.

[0195] Specifically, the processor 101 can include a central processing unit (CPU), or a specific integrated circuit (Application Specific Integrated Circuit, referred to as ASIC), or can be configured to implement one or more integrated circuits of the embodiments of the present application.

[0196] The memory 102 can include mass storage for data or instructions. By way of example, and not limitation, the memory 102 can include a Hard Disk Drive (HDD), a floppy disk drive, a Solid State Drive (SSD), a flash drive, a Compact Disc Read Only Memory (CD-ROM), a magneto-optical disk, magnetic tape, or a Universal Serial Bus (USB) drive or a combination of two or more of these. Storage memory 102, where appropriate, can be removable or non-removable (or fixed) media. Storage memory 102, where appropriate, can be internal or external to data processing apparatus. In particular embodiments, storage memory 102 is nonvolatile memory. In particular embodiments, storage memory 102 includes Read-Only Memory (ROM) and Random Access Memory (RAM). Where appropriate, this ROM can be mask- programmed ROM, Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Electrically Alterable Read-Only Memory (EAROM), or flash memory (FLASH), or a combination of two or more of these. Where appropriate, this RAM can be Static Random-Access Memory (SRAM) or Dynamic Random Access Memory (DRAM), which can be Fast Page Mode Dynamic Random Access Memory (FPMDRAM), Extended Data Output Dynamic Random Access Memory (EDO DRAM), Extended Data Output

[0197] The memory 102 can be used to store or buffer various data files required for processing and / or communication, and possible computer program instructions executed by the processor 101.

[0198] The processor 101 realizes the above-mentioned data annotation collaborative-based question and answer agent model training method by reading and executing the computer program instructions stored in the memory 102.

[0199] In some embodiments, the computer can also include a communication interface 103 and a bus 100. As shown, the processor 101, the memory 102, and the communication interface 103 are connected through the bus 100 and complete communication with each other. Figure 3

[0200] The communication interface 103 is used to realize the communication between the modules, devices, units and / or equipment in the embodiments of the present application. The communication interface 103 can also realize data communication with other components, such as external devices, image / data acquisition devices, databases, external storage, and image / data processing workstations, etc.

[0201] ​Bus 100 includes hardware, software, or both, to couple components of the computer device to each other and to couple components to other components in the environment. Bus 100 includes, but is not limited to, at least one of the following: a data bus, an address bus, a control bus, an expansion bus, a local bus, and the like. By way of example and not limitation, bus 100 can include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or another suitable bus or combination of two or more of these. Where appropriate, bus 100 can include one or more buses. Although the present embodiments describe and show a particular bus, the present embodiments contemplate any suitable bus or interconnect.

[0202] The computer can execute the data annotation collaboration-based question and answer agent model training method according to the data annotation collaboration-based question and answer agent model training system, so as to realize the data annotation collaboration-based question and answer agent model training.

[0203] In some embodiments of the present application, in combination with the above-mentioned data annotation collaboration-based question and answer agent model training method, the present application provides the following technical solutions: a storage medium, the storage medium has a computer program stored thereon, and the computer program is executed by a processor to realize the above-mentioned data annotation collaboration-based question and answer agent model training method.

[0204] Those skilled in the art will appreciate that the logic and / or steps represented in the flow diagrams, or otherwise described herein, for example, can be thought of as a list of executable instructions for implementing logic functions, and can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions, or a combination of both. In the context of this specification, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.

[0205] More specific examples (a non-exhaustive list) of the computer-readable medium include the following: an electrical connection (electronic) having one or more wires, a portable computer diskette (magnetic), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium can also be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, for example, via an optical scanner, then compiled, interpreted, or otherwise processed, and stored in a computer memory in a form that is then employable by a computer.

[0206] It should be understood that aspects of the application can be implemented in hardware, software, firmware or combinations thereof. In the above embodiments, various steps or methods can be implemented in software or firmware that is stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any of the following technologies, known in the art, or combinations thereof, can be used: a discrete logic circuit having logic gates for implementing logic functions upon data signals, an application specific integrated circuit having appropriate combinational logic gates, a programmable gate array (PGA), a field programmable gate array (FPGA), or the like.

[0207] The technical features of the above-described embodiments can be combined in any manner, and for the sake of brevity, not all possible combinations are described, however, any combination of the technical features is considered to be within the scope of the present specification.

[0208] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent. It should be noted that for ordinary skilled persons in the art, without departing from the concept of the present application, several modifications and improvements can be made, which are within the protection scope of the present application. Therefore, the protection scope of the present application patent should be subject to the appended claims.

Claims

1. A method for training a question and answer agent model based on data annotation collaboration, characterized in that, The method comprises the following steps: Acquiring target question and answer domain knowledge, data collation is performed on the target question and answer domain knowledge, and converged output is performed according to the format of a question and answer intelligent agent to obtain a historical training set, interaction information of a user to output content of the question and answer intelligent agent is acquired to obtain hierarchical annotation data, and after three-dimensional evaluation of the question and answer intelligent agent, output of outer loop optimization data is acquired, and the historical training set, the outer loop optimization data and the hierarchical annotation data are dynamically fused to obtain a fusion data set : ； ; In the formula, is a scaling factor, is a decay rate parameter, is a labeling delay time, is a scaling coefficient, is the semantic density of the first data, is a historical data set, respectively represent the outer loop optimization data and the hierarchical labeling data, is a data weight; scene clustering enhancement is performed on the fusion dataset by using a K-means clustering method to obtain an enhanced dataset; conflicting samples and low-quality supplementary samples are removed from the enhanced dataset to obtain an incremental dataset; an intelligent agent model to be trained is obtained, and the incremental dataset is used to optimize the model parameters of the intelligent agent model to be trained to obtain optimized model parameters; model deployment and scene adaptation are performed based on the optimized model parameters to obtain a model API endpoint; the model is tested and evaluated based on the model API endpoint to obtain an evaluation score and optimization suggestions; optimized parameters are extracted from the optimization suggestions, and the model is put online and operated based on the evaluation score to obtain a hierarchical data package; the optimized parameters and the hierarchical data package are used to replace the outer loop optimization data and the hierarchical labeled data to perform a dataset collation process to obtain a new dataset, the new dataset is input into the intelligent agent model to be trained to perform optimization training to obtain a target model, and the target model is applied to the intelligent agent.

2. The question and answer agent model training method based on data labeling collaboration according to claim 1, characterized in that, The step of optimizing the model parameters of the intelligent agent model to be trained by using the incremental dataset to obtain optimized model parameters comprises the following steps: inputting the incremental dataset into the question-answer intelligent agent model to be trained and determining inner-loop gradient and outer-loop gradient ; ； ； In the formula, is a gradient, is a manual annotation loss function, respectively represent model prediction value, manual expected value, is a KL divergence, , respectively represent the output probability distribution of the question and answer intelligent agent model to be trained and the reference model. traversing all the to-be-updated parameters in the to-be-trained question answering intelligent agent model, and calculating parameter gradients of the to-be-updated parameters based on the inner loop gradient and the outer loop gradient of the inner loop gradient : ； wherein is a gradient noise standard deviation, is a user gradient weight, is a gradient scaling factor, is a base learning rate, is a conflict decay factor, is a function that extracts directional characteristics of gradient conflicts, is a Hadamard product; computing a gradient confidence of the parameter gradient if the gradient confidence is greater than a confidence threshold, performing model parameter optimization based on the inner loop gradient and the outer loop gradient to obtain optimized model parameters : ； In the formula, is the model parameter to be updated, respectively, learning rate coefficient of inner loop user gradient, learning rate coefficient of outer loop AI gradient, is an indicator function.

3. The method of claim 1, wherein the method further comprises: The step of performing model deployment and scene adaptation based on the optimized model parameters to obtain a model API endpoint comprises the following steps: obtaining a scene feature library, extracting a user environment feature vector matched with a user from the scene feature library, calculating a similarity matrix based on the user environment feature vector and an environment feature vector of a trained user, and triggering knowledge transfer of the optimized model parameters and a trained model if an element in the similarity matrix is greater than a preset similarity to obtain a model API endpoint, wherein the similarity matrix is: ； In the formula, Query matrix, Key matrix in the trainable weight matrix, respectively, User environment feature vector, trained user environment feature vector, respectively, Feature projection dimension.

4. The question and answer agent model training method based on data labeling collaboration according to claim 2, characterized in that, The step of testing and evaluating the model based on the model API endpoint to obtain an evaluation score and optimization suggestions comprises the following steps: perform conflict testing on the model based on the model API endpoint to obtain a conflict rate , and output a first sub-result and trigger a rollback condition if the conflict rate is greater than a preset conflict rate, and output a second sub-result if the conflict rate is not greater than the preset conflict rate, and determine a first test result based on the first sub-result and the second sub-result: ； In the formula, is the core index for conflict determination; response delay testing of the model is performed by using a 99th percentile time statistical method, and accuracy testing of the model is performed by using a reserved validation set method to obtain a second test result and a third test result, respectively; the first test result, the second test result, and the third test result are integrated to obtain a test report; acquiring test data and model production logs, evaluating the model based on the target multi-agent, the test data, the model production logs, and a test report to obtain a completeness score , an accuracy score , and an explainability score ; Based on the integrity score , the accuracy score , and the explainability score Compute the evaluation score : ； defect positioning analysis of the model is performed based on the evaluation score to obtain optimization suggestions.

5. The method of claim 1, wherein the method further comprises: The step of putting the model online and operating based on the evaluation score to obtain a hierarchical data package comprises the following steps: A version evolution decision tree is constructed, and it is determined whether the evaluation score of the current version model is greater than the sum of the evaluation score of the last version model and a first preset score. If the evaluation score of the current version model is greater than the sum of the evaluation score of the last version model and the first preset score, the current version model is immediately put online. If the evaluation score of the current version model is not greater than the sum of the evaluation score of the last version model and the first preset score, it is determined whether the evaluation score of the current version model is greater than the sum of the evaluation score of the last version model and a second preset score. If the evaluation score of the current version model is greater than the sum of the evaluation score of the last version model and the second preset score, AB testing is performed. If the evaluation score of the current version model is not greater than the sum of the evaluation score of the last version model and the second preset score, automatic rollback is performed. After the model is put online, real-time annotation streams and a scene feature library are obtained. If the same scene density of the real-time annotation streams is greater than a density threshold, an L1 real-time stream is output. If the same scene density of the real-time annotation streams is not greater than the density threshold, it is determined whether the similarity between the real-time annotation streams and the scene features in the scene feature library is less than a similarity threshold. If the similarity between the real-time annotation streams and the scene features in the scene feature library is less than a preset similarity, an L2 batch stream is output. If the similarity between the real-time annotation streams and the scene features in the scene feature library is not less than the preset similarity, an L3 sedimentation tank is output, so as to obtain a hierarchical data package.

6. A question and answer intelligent agent model training system based on data annotation collaboration, characterized in that, The system comprises: The arrangement module is used for obtaining target question and answer field knowledge, performing data arrangement on the target question and answer field knowledge, and converging output in the format of a question and answer intelligent agent to obtain a historical training set, obtaining interaction information of a user on output content of the question and answer intelligent agent to obtain hierarchical annotation data, obtaining outer loop optimization data output after three-dimensional evaluation on the question and answer intelligent agent, and dynamically fusing the historical training set, the outer loop optimization data, and the hierarchical annotation data to obtain a fused data set : ； ; In the formula, is a scaling factor, is a decay rate parameter, is a marked delay time, is a scaling coefficient, is the semantic density of the first data, is a historical data set, respectively represent the outer loop optimization data and the hierarchical annotation data, is a data weight; a K-means clustering method is used for scene clustering enhancement on the fusion data set to obtain an enhanced data set; conflict samples and low-quality supplementary samples are removed from the enhanced data set to obtain an incremental data set; An optimization module configured to obtain a to-be-trained question and answer intelligent agent model, and perform model parameter optimization on the to-be-trained question and answer intelligent agent model based on the incremental data set, so as to obtain optimized model parameters; A deployment module configured to perform model deployment and scene adaptation based on the optimized model parameters, so as to obtain a model API endpoint; An evaluation module configured to test and evaluate the model based on the model API endpoint, so as to obtain an evaluation score and an optimization suggestion; An operation module configured to extract an optimization parameter from the optimization suggestion, and perform model online and operation based on the evaluation score, so as to obtain a hierarchical data package; An output module configured to replace the outer loop optimization data and the hierarchical annotation data with the optimization parameter and the hierarchical data package, so as to perform a data set collation process, so as to obtain a new data set, input the new data set into the to-be-trained question and answer intelligent agent model for optimization training, so as to obtain a target model, and apply the target model to the question and answer intelligent agent.

7. A computer comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the question and answer intelligent agent model training method based on data annotation collaboration according to any one of claims 1 to 5.

8. A storage medium, characterized by The storage medium has a computer program stored thereon, and the computer program is executed by the processor to implement the question and answer intelligent agent model training method based on data annotation collaboration according to any one of claims 1 to 5.

Images