Model training method and device, intelligent agent, equipment and storage medium

Abstract

The invention provides a model training method and device, an intelligent agent, equipment and a storage medium, and relates to the technical field of artificial intelligence, in particular to the technical field of large models, intelligent agents and the like. According to the specific implementation scheme, label interaction information corresponding to a first label is obtained; obtaining a label labeling result according to the label interaction information; generating a training set according to the label labeling result; and training according to the training set initial model to obtain a target model.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and particularly to the fields of large models, intelligent agents, and other technologies. Background Art

[0002] An intelligent agent is an intelligent application or entity that can perceive the environment, make decisions, and interact with the environment, and can affect the environment by performing actions. An intelligent agent can be physical or virtual. An intelligent agent usually has one or more goals or tasks, and these goals can be achieved through an intelligent decision-making process. An intelligent agent can participate in model training through supervised learning, reinforcement learning, and other methods. Summary of the Invention

[0003] The present disclosure provides a model training method, apparatus, intelligent agent, device, and storage medium.

[0004] According to one aspect of the present disclosure, there is provided a model training method, including:

[0005] Obtaining label interaction information corresponding to a first label;

[0006] Obtaining a label annotation result according to the label interaction information;

[0007] Generating a training set according to the label annotation result;

[0008] Training an initial model according to the training set to obtain a target model.

[0009] According to another aspect of the present disclosure, there is provided a model training apparatus, including:

[0010] A first obtaining module, configured to obtain label interaction information corresponding to a first label;

[0011] A second obtaining module, configured to obtain a label annotation result according to the label interaction information;

[0012] A first generating module, configured to generate a training set according to the label annotation result;

[0013] A training module, configured to train an initial model according to the training set to obtain a target model.

[0014] According to another aspect of the present disclosure, there is provided an intelligent agent, including:

[0015] An input module, configured to input label interaction information corresponding to a first label;

[0016] A processing module, configured to, based on the label interaction information received by the input module, call a large model to execute the model training method according to any embodiment of the present disclosure to train and obtain a target model;

[0017] An output module, configured to output the target model obtained by the processing module.

[0018] According to another aspect of the present disclosure, there is provided an electronic device, including:

[0019] At least one processor; and

[0020] A memory communicatively connected to the at least one processor; wherein,

[0021] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute any method in the embodiments of the present disclosure.

[0022] According to another aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute any method in the embodiments of the present disclosure.

[0023] According to another aspect of the present disclosure, there is provided a computer program product, including a computer program, which implements any method in the embodiments of the present disclosure when executed by a processor.

[0024] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The drawings are used to better understand the solution and do not constitute a limitation to the present disclosure. Among them:

[0026] Figure 1 is a schematic flowchart of an agent-based model training method according to an embodiment of the present disclosure;

[0027] Figure 2 is a schematic flowchart of an agent-based model training method according to another embodiment of the present disclosure;

[0028] Figure 3 is a schematic structural diagram of a questionnaire according to an embodiment of the present disclosure;

[0029] Figure 4 is a schematic flowchart of task execution judgment according to an embodiment of the present disclosure;

[0030] Figure 5 is a schematic flowchart of an agent-based model training method according to another embodiment of the present disclosure;

[0031] Figure 6It is a schematic flowchart of a model training method according to an embodiment of the present disclosure;

[0032] Figure 7 It is a schematic diagram of the model training result;

[0033] Figure 8 It is a schematic flowchart of task execution judgment according to an embodiment of the present disclosure;

[0034] Figure 9 It is a schematic diagram of the clustering relationship according to an embodiment of the present disclosure;

[0035] Figure 10 It is a schematic flowchart of tracing to the training set according to an embodiment of the present disclosure;

[0036] Figure 11 It is a schematic flowchart of resampling according to an embodiment of the present disclosure;

[0037] Figure 12 It is a schematic flowchart of model training according to an embodiment of the present disclosure;

[0038] Figure 13 It is a schematic flowchart of label acquisition according to an embodiment of the present disclosure;

[0039] Figure 14 It is a schematic flowchart of training set acquisition according to an embodiment of the present disclosure;

[0040] Figure 15 It is a schematic structural diagram of a model training device according to an embodiment of the present disclosure;

[0041] Figure 16 It is a schematic structural diagram of a model training device according to another embodiment of the present disclosure;

[0042] Figure 17 It is a schematic structural diagram of an agent according to an embodiment of the present disclosure;

[0043] Figure 18 It is a block diagram of an electronic device for implementing the embodiments of the present disclosure. Detailed implementation manners

[0044] The following describes exemplary embodiments of the present disclosure with reference to the accompanying drawings. Various details of the embodiments of the present disclosure are included to facilitate understanding, and they should be considered merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of the present disclosure. Similarly, for clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0045] Figure 1It is a schematic flowchart of a model training method 100 according to an embodiment of the present disclosure. In one implementation, the method may include:

[0046] S101. Obtain label interaction information corresponding to a first label;

[0047] S102. Obtain a label annotation result according to the label interaction information;

[0048] S103. Generate a training set according to the label annotation result;

[0049] S104. Train an initial model according to the training set to obtain a target model.

[0050] In the embodiments of the present disclosure, an agent can perceive the environment and take actions to achieve specific goals. The agent can have autonomy, adaptability, and interaction capabilities. By perceiving changes in the environment, the agent makes judgments and decisions based on the knowledge and algorithms learned by itself, and then executes actions to affect the environment or achieve a predetermined goal.

[0051] In the embodiments of the present disclosure, a variety of agents can be included in the agent system. For example, a label design agent, a data analysis agent, etc. Different agents can use different interaction interfaces. For example, the data analysis agent can analyze the data uploaded by the client through the data analysis agent interface. The label design agent can add appropriate labels to the data through the label design agent interface. The data between some agents can be transmitted or shared with each other. The agent interaction interface can display various information of the interaction between the user and the agent through the client. The agent interaction interface can be deployed on the client.

[0052] In the embodiments of the present disclosure, the label interaction information corresponding to the first label can be displayed on the agent interaction interface. The first label can include labels that the user is interested in or that are supported by the client used by the user. The labels that the user is interested in can include labels actively selected or input by the user. The labels supported by the client can include labels that have been generated by the label design agent of the client, or labels that the client can obtain, etc. For example, the label design agent can first understand and analyze the data uploaded by the user and automatically add labels. Then, the data analysis agent can select the labels that the user is interested in or that are supported by the client used by the user from the labels generated by the label design agent.

[0053] In the embodiments of the present disclosure, the agent can trace back to the original data or original fragment corresponding to the first label according to the first label. Then, generate label interaction information according to the first label and its corresponding original data or original fragment, and display the label interaction information on the display interface of the agent.

[0054] In the embodiments of the present disclosure, the display forms of the label interaction information on the intelligent agent interaction interface may include questionnaires, check boxes, inquiry information, pop-up windows, annotation examples, etc., and the present disclosure does not limit this. The intelligent agent can obtain the label annotation result of the client according to the label interaction information corresponding to the first label. In one way, the user can, according to the label interaction information displayed on the display interface of the intelligent agent, perform operations such as selection, confirmation, and submission on the client to obtain the label annotation result. For example, on the client, the display interface of the intelligent agent displays label interaction information in the form of a questionnaire composed of the first label and its corresponding original text data. The questionnaire may include items to be annotated for one or more labels, and one item to be annotated can be used to annotate whether the label conforms to the original text under this option. After the user fills out the questionnaire and submits it, the intelligent agent generates a label annotation result based on the submitted questionnaire. A batch of items to be annotated can be displayed on the intelligent agent interaction interface each time. After the user completes the annotation through the client, another batch of items to be annotated can be displayed to continue the annotation. This continues until all items to be annotated are annotated or the user decides to stop the annotation.

[0055] In the embodiments of the present disclosure, the label annotation result may include the labels confirmed by the user through the client and their corresponding original text data. For example, if the label interaction information on the interaction interface includes a batch of N items to be annotated for label A and label B, and the user confirms that the original text of M items (M is less than or equal to N) conforms to label A and label B. These M items can be added to the label annotation result. After multiple operations, multiple batches of items that conform to label A and label B are added to the label annotation result. The label annotation result may include the original text fragments or original text data of these items, or may also include one or more labels corresponding to these items. The intelligent agent can organize the original text data annotated with labels in the label annotation result to obtain the training set of the model.

[0056] In the embodiments of the present disclosure, the initial model can be an untrained or basic version of a machine learning model (such as a randomly initialized neural network), and the target model can be the final model trained by the training set. The types of models can include various types, such as composite label models, classification label models, data analysis models, text segmentation models, etc., and the present disclosure does not limit this.

[0057] According to the embodiments of the present disclosure, the label annotation result of the client can be obtained through interaction on the intelligent agent interaction interface, and a training set that meets the requirements can be obtained. Then, based on the training set, a target model that better meets the requirements can be trained, which can simplify the model training process and improve the efficiency of model training.

[0058] Figure 2It is a schematic flowchart of a model training method 200 according to another embodiment of the present disclosure. This method 200 can be used to implement step S101 in the model training method 100. In one implementation, the method 200 includes: obtaining label interaction information corresponding to a first label, which further includes:

[0059] S201. Trace back to the first original text data according to the first label;

[0060] S202. Generate the label interaction information according to the first label and the first original text data, and obtain the label interaction information.

[0061] In the embodiments of the present disclosure, through the data processing ability of the intelligent agent, the original text data corresponding to the first label is traced back in reverse. For example, through the label "Technology Configuration|Digital Cockpit|Dash Cam", the original text data 1 "Can the dash cam be viewed on the mobile phone? What software should be used?", original text data 2 "I would like to consult how to connect the purchased dash cam to the electronic display screen?", and original text data 3 "Why isn't there a video in the local album?" etc. can be traced back.

[0062] In the embodiments of the present disclosure, the server (or analysis end) of the intelligent agent can generate label interaction information according to the first label and its corresponding original text data. Further, the label interaction information can be displayed on the intelligent agent interaction interface. For example, according to label A and its corresponding original text data 1, original text data 2, and original text data 3, a questionnaire including 3 options to be labeled can be generated. The first item includes whether the original text data 1 conforms to label A, the second item includes whether the original text data 2 conforms to label A, and the third item includes whether the original text data 3 conforms to label A. The label interaction information is displayed on the interaction page of the intelligent agent. There can be various display methods. As Figure 3 shown, one example may include: three items 302 are displayed below label A of the questionnaire 301; the original text data or original text fragments are displayed in the display area of each item, and the corresponding options 303 of the item are displayed: a conform button and a non-conform button. If the user selects the conform button, it means that the original text data of the item conforms to the label; if the user selects the non-conform button, it means that the original text data of the item does not conform to the label.

[0063] According to the embodiments of the present disclosure, on the interaction interface of the intelligent agent, through the label interaction information, interaction can be carried out between the client and the user to obtain a label annotation result that better conforms to the user's needs, and further obtain a model training result that better conforms to the user's needs.

[0064] In one implementation, tracing back to the first original text data according to the first label includes:

[0065] Search for the identifier of the first cluster corresponding to the first label;

[0066] Find the set of original text segments corresponding to the identifier of the first cluster;

[0067] Find the first original data corresponding to the text identifier in the set of original text segments.

[0068] In the embodiments of the present disclosure, the first label can be obtained by clustering the first original data through a clustering algorithm such as K-means, DBSCAN, etc.; alternatively, a label design agent or other agents with the ability to automatically add labels to data can be used to add the first label to the first original data. The first original data may include multiple original text segments, and one original text segment can be added with one or more labels, that is to say, one original data can be added with one or more labels. The original text segments with the same or related labels can be clustered into different clusters, and one cluster can correspond to one or more labels.

[0069] In the embodiments of the present disclosure, the corresponding relationship between the first label and the cluster can be stored in a temporary label. The temporary label can be stored in the form of a table, a data set, etc., that is, the corresponding relationship can be stored in a temporary label table or a temporary label set. The identifier of the first cluster corresponding to the first label can be obtained by indexing in the temporary label (temporary label table or temporary label set).

[0070] In the embodiments of the present disclosure, the corresponding relationship between the identifier of the cluster and the original text segment can be stored in the label clustering result. The label clustering result can be stored in the form of a table, a data set, etc., that is, the corresponding relationship can be stored in a label clustering table or a label clustering set. For example, the original text segments S1, S2, and S3 can be obtained by indexing with the identifier C1 of the first cluster. Among them, S1 and S2 correspond to the original data T1, and S3 corresponds to the original data T2. In this example, T1 and T2 can be understood as the text identifiers of the original data. One identifier of a cluster can correspond to one or more original text segments. In the case where the identifier of a cluster corresponds to multiple original text segments, these multiple original text segments can be referred to as a set of original text segments. Each original text segment can have a corresponding text identifier, and each text identifier can correspond to one original data. One original data can include one or more original text segments.

[0071] In the embodiments of the present disclosure, the correspondence between the original text segment and the first original data can be stored in the server (or called the analysis end) of the intelligent agent, and the corresponding original data can be indexed in the server through the text identifier of the original text segment. After finding the set of original text segments corresponding to the identifier of the first cluster, the first original data corresponding to the text identifiers in the set of original text segments can be found in the server data. For example, through the text identifier "T1" corresponding to the original segment in the set of original text segments, the corresponding original data T1 can be traced, and the text identifier "T2" corresponding to the original segment can be traced to the corresponding original data T2. To sum up, the relationship between the cluster and the original text segment can be one-to-one or one-to-many, and the relationship between the original data and the original text segment can also be one-to-one or one-to-many.

[0072] According to the embodiments of the present disclosure, by clustering data with high label correlation, the efficiency of tracing the original text according to the label can be improved, the speed of constructing the training set can be increased, and the model training process can be accelerated.

[0073] In one implementation, as Figure 2 shown, the method 200 can be used to implement step S102 in the model training method 100. The label interaction information includes annotation options for annotating the first label and its associated first original data. In one implementation, the method 200 includes: obtaining a label annotation result according to the label interaction information, and further including:

[0074] S203. In response to a selection command for the annotation option, determine whether there is an annotation relationship between the first label and its corresponding first original data;

[0075] S204. According to the annotation relationship between the first label and its corresponding first original data, add the second original data that conforms to the first label to the label annotation result.

[0076] In the embodiments of the present disclosure, after the label interaction information is displayed on the agent display interface, the user can interact with the agent through the client to perform a selection operation on the annotation options in the label interaction information, and confirm whether there is an annotation relationship between the first label and the corresponding first original data. For example, after the agent displays the label interaction information in the form of a questionnaire on the display interface, the user can fill out the questionnaire and select the conforming option or non-conforming option in the annotation options under each question (item) of the questionnaire. After the user selects the conforming option under a certain question, a selection command for the conforming option can be generated on the client. In response to this selection command of the client, it can be confirmed that the original data corresponding to this question has an annotation relationship with the first label, that is, the first label should be added to the original data. After the user selects the non-conforming option under a certain question, a selection command for the non-conforming option can be generated on the client. In response to this selection command of the client, it can be confirmed that the original data corresponding to this question has no annotation relationship with the first label, that is, the first label should not be added to the original data.

[0077] In the embodiments of the present disclosure, after the user performs a selection operation on the annotation options in the label interaction information, each first original data that is confirmed to conform to the first label can be referred to as the second original data. The second original data and the first original data are only distinguished in different processing stages, and the specific content of some original data may be the same. For example, the first original data corresponding to the first label used for interface display includes T1, T2, T3, T4, and T5. After being confirmed by the user, the second original data that conforms to the first label includes T1, T3, and T5. In this case, the content of the original data of T1, T3, and T5 is the same. The label annotation result may include the second original data, or may include the first label corresponding to the second original data.

[0078] According to the embodiments of the present disclosure, through the label interaction information on the agent interaction interface, the user can be requested to confirm whether the annotation relationship between the label and the original text is consistent, so as to obtain the label that meets the user's needs and its corresponding original data, and then quickly construct a training set to train a target model that meets the user's needs.

[0079] In one implementation manner, the method may further include: in response to a replacement command for the label interaction information, obtaining the replaced label interaction information, where the replaced label interaction information includes the replaced first original data corresponding to the first label.

[0080] In the embodiments of the present disclosure, a first tag can trace back to obtain a plurality of first original data. In the case where the number of the first original data is too large, some of the first original data and the first tag can be selected first to generate tag interaction information. In the case where the length of a certain original data is too long, a partial segment of the original data, that is, an original segment, can be displayed in the tag interaction information. The intelligent agent display interface can display the tag interaction information composed of this part of the first original data and the first tag in batches. If the original text of a batch of items displayed in the current interface does not match the first tag, the user can perform a replacement operation to replace a batch of original data. This replacement operation can generate a replacement command for the tag interaction information on the client side. In response to the replacement command for the tag interaction information, the replaced tag interaction information can be displayed on the intelligent agent interaction interface. The replaced tag interaction information can include the replaced first original data corresponding to the first tag. For example, click the replacement button, click the refresh button, trigger the replacement shortcut, etc. In this case, the client can generate a replacement command, and in response to the replacement command of the client, another batch of items' original text (original data or original segments) can be displayed on the current interface. The user can perform a selection operation on the annotation options for this newly replaced batch of items.

[0081] According to the embodiments of the present disclosure, the tag interaction information displayed in the intelligent agent interaction interface, such as the original data therein, can be replaced to quickly annotate the tags and their corresponding original data that meet the user's needs, thereby improving the annotation efficiency.

[0082] In one implementation, as Figure 2 shown, the method 200 can be used to implement step S103 in the model training method 100. In one implementation, the method 200 includes: generating a training set according to the tag annotation result, which further includes:

[0083] S205. Generate a training set according to the second original data and its corresponding first tag in the tag annotation result; the training set is used to train an initial model.

[0084] In the embodiments of the present disclosure, the tag annotation result may include several second original data that conform to the first tag. According to the samples required by the training set, a part of the second original data and their corresponding first tags can be selected from the tag annotation result to construct a training set. Training the initial model based on the training set can obtain a target model. The initial model can be automatically matched by the intelligent agent according to the user's needs, or can be obtained according to the user's active selection.

[0085] According to the embodiments of the present disclosure, a training set that better meets the user's needs can be quickly obtained according to the tag annotation result, and a target model that better meets the user's needs can be trained.

[0086] In one embodiment, as Figure 2 shown, the method may further include:

[0087] S206. Generate a validation set according to the second original data and its corresponding first label in the label annotation result; the validation set is used to validate the target model.

[0088] In the embodiments of the present disclosure, similar to the training set, according to the samples required by the training set, a part of the second original data and its corresponding first label can be selected from the label annotation result to construct a validation set. The samples in the validation set and the training set can be partially repeated, not repeated, or a part of the samples can be directly selected from the validation set as the validation set.

[0089] According to the embodiments of the present disclosure, a validation set that better meets the user's needs can be quickly obtained according to the label annotation result, improving the validation efficiency of the target model and the accuracy of the output result of the target model.

[0090] In one embodiment, the method may further include one of the following:

[0091] When there is a target agent record in the session of the agent and the task type in the sub - session of the target agent is model training, perform model training;

[0092] When there is a target agent record in the session of the agent and the task type in the sub - session of the target agent is model prediction, perform model prediction;

[0093] When there is no target agent record in the session of the agent and there is a target agent record in the agent analysis report, perform model training;

[0094] When there is no target agent record in the session of the agent and there is no target agent record in the agent analysis report, perform model prediction.

[0095] In the embodiments of the present disclosure, the session of the agent may include the interaction content between the user and the agent through the client. According to the session of the agent, the task type, the agent analysis report, etc., it can be determined whether model prediction (or model selection) needs to be performed or model training needs to be performed.

[0096] See Figure 4 , an exemplary judgment process includes:

[0097] S401. Determine whether there is a target agent record in the session of the agent. If so, execute S402; otherwise, execute S403.

[0098] S402. Obtain the task type in the sub-session of the target agent. If the task type is model training, execute S404; if the task type is model prediction, execute S405.

[0099] S403. Determine whether there is a record of the target agent in the analysis report of the agent. If so, execute S404; otherwise, execute S405.

[0100] S404. Execute model training.

[0101] S405. Execute model prediction.

[0102] In the embodiments of the present disclosure, taking the target model as a data analysis agent as an example, if the session of the agent, such as agent_session, includes a record of the data analysis agent, the task type can be further obtained in the sub-session of the target agent, such as agent_analysis_sub_session. If the task type is model training, the data analysis agent can start the model training process. If the task type is model prediction, the data analysis agent can start the model prediction process.

[0103] In the embodiments of the present disclosure, if the session of the agent, such as agent_session, does not include a record of the data analysis agent, it can be further checked whether the analysis report of the agent, such as agent_analysis_report, includes a record of the data analysis agent. The analysis report of the agent can include reports on the prediction or analysis of input data by the agent using various models. If there is no record of the data analysis agent in both the session and the analysis report of the agent, it means that the agent has not been used for analysis or prediction yet, and no model training task has been created based on the agent, and the model prediction process can be entered. If there is no record of the data analysis agent in the session, but there is a record of the data analysis agent in the analysis report of the agent, it means that the agent has been used, and the agent can be used to train the model, and the model training process can be entered.

[0104] In the embodiments of the present disclosure, the model training process can refer to the relevant descriptions in the above embodiments. An example of a model prediction process may include: model recommendation information obtained through agent planning based on the demand information of the client; displaying the model recommendation information on the agent interaction interface; in response to a selection command for the model recommendation information, obtaining the target model; inputting the data that the client needs to process into the target model for processing to obtain an output result. For example, inputting content such as the demand description provided by the user through the client into a large model for operations such as demand understanding and / or intent recognition to obtain the demand information of the client. The content provided by the user through the client may include, but is not limited to: model scope, basis for model selection, number of models, model application, etc. Then, the demand information of the client can be sent to the planner of the agent. The planner of the agent has functions such as a chain of thought and sub-goal decomposition. By decomposing and thinking about the demand information through the planner of the agent, the recommended model information that meets the demand information can be obtained. The recommended model information may include specific information of one or more recommended models, such as one or more of the capabilities, names, descriptions, etc. of the recommended models. The recommended model information may also include some optional functions such as a model removal function, a model addition function, a model confirmation function, etc.

[0105] According to the embodiments of the present disclosure, the demand can be automatically distinguished based on the information in the session, and the corresponding task execution process can be entered, improving the efficiency of model training.

[0106] Figure 5 It is a schematic flowchart of a model training method 500 according to another embodiment of the present disclosure. In one implementation, the method may further include:

[0107] S501. Create a prompt according to the prediction task corresponding to the target model to obtain prediction demand information;

[0108] S502. Input the prediction demand information into the target model to execute the prediction task.

[0109] In the embodiments of the present disclosure, after the target model is trained, the intelligent agent can display a prediction task creation prompt on the client side through the interaction interface, asking the user whether to perform data analysis or prediction based on the target model. For example, "The model training is completed. Do you want to execute the prediction task?" When the user selects to perform data analysis or prediction based on the target model, the prediction requirement information can be input according to the prediction task creation prompt on the interaction interface. For example, "Please input the data to be predicted and the requirements for analysis..." The prediction task creation prompt can be in the form of a questionnaire or a question-and-answer, and can include one round or multiple rounds. By asking the user through the prediction task creation prompt and collecting the user's answers, the prediction requirement information can be obtained by analyzing the question-and-answer information through a large model. According to the prediction requirement information and the input data, a prompt word can be constructed, and the target model can execute the prediction task according to the prompt word to obtain the prediction result.

[0110] According to the embodiments of the present disclosure, prediction can be performed based on the trained target model to obtain a prediction result that better meets the user's needs, improving the accuracy of the prediction result.

[0111] In the embodiments of the present disclosure, after the user has selected and submitted the label interaction information, the first label and the first original data for generating the label interaction information can be cleared in the cache. When the model training starts and the training set has been input, the first label and the second original data included in the training set and the label annotation result can be cleared in the cache. Clearing after using the label and the original data can reduce the consumption of computing power and improve the calculation speed.

[0112] Figure 6 is a schematic flowchart of a model training method according to an embodiment of the present disclosure. As Figure 6 shown, after the label design intelligent agent (Agent) helps the user design the label and the user selects the label, the data analysis (Agent) can continue the insight process to help or guide the user to train the model corresponding to the label. The method may include:

[0113] S601. The user interacts with the label design intelligent agent (Agent) to obtain a designed label system. The user can select the label and add the labels they need to their own label library. The user behavior in the embodiments of the present disclosure can be understood as the behavior performed by the user through the client.

[0114] S602. After the user selects the label, the background traces back to prepare the original data corresponding to the label, and a certain number of data, such as 10-20 pieces of data, are prepared for each label.

[0115] After the data preparation is completed, the data analysis agent actively sends a message to the user, such as a message to confirm the training model, indicating that it can help the user train the model (for the labels selected by the user).

[0116] After the user confirms to train the model, the data analysis agent sends a questionnaire to the user. For example, the content of a labeled questionnaire can include: label + original data, and the user is required to judge right or wrong (i.e., label the data).

[0117] The user submits the labeled data (or called the labeling result).

[0118] The data analysis agent creates a corresponding model on the management side, uses the labeled data as the training data, and automatically trains the model. After the model training is completed, the data analysis agent sends the model information to the user and asks the user whether to create a data prediction task.

[0119] After the user confirms to create a task, the data analysis agent automatically creates a data prediction task and informs the user that they can upload data to view the results (guide the user to use various functions on the analysis side).

[0120] The user uploads the data to be predicted. This step can also be executed in advance. For example, the user can upload data at any time according to their own needs.

[0121] Use the trained model to execute the prediction task and output the prediction result.

[0122] I. Functional description of the data analysis agent

[0123] The data analysis agent and the label design agent can be separate agent dialogue windows on the dialogue side (client side), and the dialogue between the agent and the users in each tenant can be separate. The data analysis agent can perform the following settings:

[0124] (1) Permissions: At the role permissions, permission points for the data analysis agent can be added to the dialogue side part.

[0125] (2) Message time and unread mark: Add the display of the message sending time between the agent and the user and the mark effect of unread messages.

[0126] Message time: The specific message sending time between the user and the agent is displayed in the left agent list. For example, if it is a message sent 24 hours ago, the date of the last sent message is displayed.

[0127] Unread Mark: When the Agent sends a message to the user and the user has not read it, a specific color is displayed. For example, a red mark and the number of unread messages. Another example is that when the number of unread messages exceeds 99, at most 99+ is displayed.

[0128] (3) User uploads data and synchronizes it to the analysis end: The user submits data by uploading files in the Label Design Agent, Data Analysis Agent, and subsequent other Agents, and synchronizes the data to the data management of the analysis end.

[0129] After the user selects a label, the Data Analysis Agent can initiate an interaction actively. The following describes the interaction process through specific examples.

[0130] (1) The Data Analysis Agent prepares the corresponding original data of the selected label in the background.

[0131] After the original data preparation is completed, the Data Analysis Agent sends a message actively. For example, the message content example includes "Hello! I am your data analysis consultant and am glad to provide services for you. I can help you perform model training for the label you just selected, but you need to first judge the label results corresponding to the data. Do you think it's okay?"

[0132] If the original data preparation fails or is not completed, the message asking whether the user needs model training will not be sent to the user. When the user enters the Data Analysis Agent, the direct use function is used by default. For example, the reply is: "Hello! I am your data analysis consultant and am glad to provide services for you. You can first provide the data to be analyzed and then describe the demand scenario you want to analyze. I will select a suitable model to perform data analysis for you." After the Agent completes a complete data analysis process for the user, if the data preparation is successful, a message guiding model training will be sent to the user.

[0133] Give subsequent replies according to the user's selection operation. For example, if the user clicks and selects "No", the Agent replies to the user: "Okay, if you need it later, you can come to me again." Another example is that if the user clicks and selects "Okay", the Agent sends a labeling questionnaire to the user for data labeling. The Agent first replies: "Okay, please first give a judgment on the following data and the corresponding label results. If the label results do not match the data, you can click 'Does not match' to modify the labeling results. After all judgments are completed, submit the results." Then, the Agent sends a labeling questionnaire to the user, and all labeling options in the questionnaire can be initially default selected as "Match". When the user judges that the label does not match the data, they can click "Does not match" to modify the labeling results (when the user clicks "Does not match", the selection effect switches from "Match" to "Does not match").

[0134] An example of the data volume of a labeling questionnaire is as follows (training set data volume + validation set data volume):

[0135] When the label < 50, the data corresponding to a single label does not exceed 10 + 5 items.

[0136] When 50 ≤ label ≤ 80, the data corresponding to a single label does not exceed 20 + 5 items.

[0137] When 80 < label ≤ 100, the data corresponding to a single label does not exceed 30 + 10 items.

[0138] Dialogue data: When the original text data is long text data such as dialogue and work orders, the original text data of the questionnaire can only show the corresponding part of the original text fragment of the drawn label to the user (without showing the complete data), which is convenient for the user to mark, and the corresponding complete original text data is used for training when training the model.

[0139] "Change to another batch": If the user feels that the randomly drawn original text data has poor effect, they support clicking "Change to another batch" on the right side of the label to change the original text data of the label.

[0140] "Submit": After the user has marked all, click "Submit" to submit the marking result.

[0141] The marked data is displayed all at once. When the amount of data is large, slide down the content to display.

[0142] (2) User submits the marking result

[0143] The data analysis Agent shows that the submission is successful and replies, for example: "Next, I will train the model for you, which will take some time. I will send you a message to inform you when it is completed."

[0144] (3) Agent creates a model and conducts training

[0145] The Agent automatically creates a model in the "management end". After the model creation is completed, the user's marking result will be used as the training set data for model training. At this time, a model record created by the Agent can be synchronously added in the "model management" of the management end. The "label management", "training set management", and "training task" in the model management can also synchronously update data such as "user-selected label", "user-marked training set", and "training task".

[0146] An example of a model record is as follows:

[0147] Model name: Automatically generated (the length requirement is within 10 characters and cannot be named repeatedly).

[0148] Model ability: Default composite label model.

[0149] Label: The label selected by the user in the label design Agent.

[0150] Training task name: Automatically generated.

[0151] Training set name: Automatically generated.

[0152] Training set type: Only labeled tags.

[0153] Training set data: User-annotated data results.

[0154] Validation set: Default configuration.

[0155] Proportion of validation set data: 20%.

[0156] Learning rate: 0.1‰.

[0157] Number of training epochs: 20.

[0158] Basic model selection: Options such as a 12-layer model and a 20-layer model are provided. The 12-layer model has a fast training speed and is suitable for some simple tasks, such as tasks with less than 100 labels.

[0159] Furthermore, the management side can add a newly created model. The model management also adds a new training task. For example, if the user sends other messages to the Agent while the Agent is training the model, the Agent can reply to the user: "I am helping you train the model. I will come to help you with other tasks after the training is completed."

[0160] (4) After the model training is completed, send the model information and messages to the user.

[0161] For example, if the training fails, the Agent replies to the user: "Sorry, the model training task was not successful. You can go to 'Management - Model Management' to try manual training again."

[0162] The user can directly click on "Management - Model Management" in the message to jump to the model management page for manual attempt. For example, if the training is successful, as Figure 7 shown, send the model information, including the model name and model effect. The user can click on the "model name" to jump to the training task under "Management - Model Management" to view the corresponding model training details.

[0163] The Agent can send an inquiry message in the conversation to guide the user to publish the trained model to the template market. The user can click on "One-click Publish" in the message text and then fill in the corresponding information in the pop-up window to perform template publishing.

[0164] (5) When the user clicks "End Task", execute the task end logic.

[0165] After the user predicts and clicks "Data Prediction", the Agent sends a message asking the user to provide prediction data. For example, the data analysis Agent replies: "Now I'm creating a data analysis task for you to perform data prediction. Please provide the data you need to predict to me."

[0166] A "Send Data" button can be displayed in the session, enabling the user to submit data to the Agent.

[0167] A data selection prompt can also be displayed in the session, such as: "You can select existing data within the system or send me a data file."

[0168] There are multiple ways for the user to upload data, as shown in the following examples:

[0169] 1. The user can choose "Data within the System" or "Table Import".

[0170] Data within the System: The user is supported to filter the data in the data management of the analysis terminal within the system.

[0171] Table Import: The user is supported to upload data through an Excel file.

[0172] (1) File upload is supported (refer to the table import data in the management terminal insight Agent).

[0173] (2) The data submitted by the user through file upload needs to be synchronized to the "Analysis Terminal" - Data Management so that the user can view the prediction results in the data management after the prediction is completed.

[0174] 2. If the user sends other messages to the Agent without selecting data, the Agent can reply: "You haven't given me any data yet. Please give me some data first so that I can help you with data analysis."

[0175] 3. After the user provides the data, the upload data button can change color, for example, be grayed out, to prevent the user from providing data repeatedly.

[0176] (1) During the data import process, the Agent replies to the user: "Receiving your data, please wait a moment."

[0177] (2) If the data is not successfully imported, the Agent replies to the user: "Sorry, I couldn't understand the data you provided. Please provide me with another copy of the data."

[0178] (3) If the data is successfully imported, the Agent replies to the user: "I have successfully received your data. Now I'm creating a prediction task for you, which will take some time. I will send you a message to inform you when it's completed."

[0179] (6) After the prediction task is created, data prediction is automatically started.

[0180] For example, if the data prediction fails, the Agent replies to the user: "Sorry, the data prediction task was not successful. You can go to 'Analysis Terminal - Data Prediction' and try creating it manually again." The user can directly click on 'Analysis Terminal - Data Prediction' in the message to jump to the data prediction page for manual attempts.

[0181] Another example, if the data prediction is completed, the Agent replies to the user: "The data analysis prediction is completed. Please 'View Prediction Results'. If you need to view the BI report of the data prediction results, you can generate the report and view it in 'Analysis Terminal - Intelligent Analysis'."

[0182] If the user clicks on 'View Prediction Results', they will jump to 'Analysis Terminal - Data Management' to view the data prediction results.

[0183] If the user clicks on 'Analysis Terminal - Intelligent Analysis', they will jump to 'Analysis Terminal - Intelligent Analysis'. The drop-down box for selecting data is default filled with the data results of the prediction task created by the Agent. The user can directly send requests and then generate the report.

[0184] (7) An example of the task end logic is as follows:

[0185] When the following status occurs, it is considered that the current task has ended. The Agent resumes its initial state and replies to the user: "This data analysis task has ended. If you have any further data analysis needs in the future, you can come to me again."

[0186] (1) When the Agent has no output results.

[0187] (2) When the user's conversation input indicates an intention to end the task.

[0188] (3) When the user has no operations or replies within 24 hours.

[0189] Note: The user is supported to manually end the current task. During the task process, a "Reset" button is displayed above the dialog box.

[0190] When hovering the mouse (hover), a tooltip text is displayed: "Click the button to end the current data analysis task."

[0191] When the user clicks the button, the current task ends and the Agent resumes its initial state.

[0192] II. Session Management

[0193] In the disclosed embodiments, session management can solve the problem of which session the current conversation belongs to. For session management in the data analysis Agent, there are the following problems:

[0194] 1: When to add a record in the agent_session of the agent

[0195] In the interface for querying the mapping relationship between the agent_type and sessionId. If the current sessionId does not exist, a new record is added to agent_session.

[0196] 2: How to determine whether to enter the training process or the prediction process

[0197] It is judged through the agent_analysis_sub_session and agent_analysis_report tables. An example of the judgment logic is as Figure 8 shown.

[0198] After the user enters the data analysis agent, which task (training task and prediction task) should be entered? The example of the judgment process is as follows:

[0199] S801. Query whether there is a record of the data analysis agent in agent_session. If there is no record, execute S802. If there is a record, execute S803.

[0200] S802. Query whether there is such a record in the agent_analysis_report table (query according to the latest label sub_session_id).

[0201] If there is no record, execute the prediction process. For example, insert data into agent_analysis_sub_session (prediction task), and then return a reply to the front end to enter the data prediction task.

[0202] If there is a record, execute the training process. For example, insert data into the agent_analysis_sub_session table (training task), and then push replies related to model training.

[0203] S803. Query the agent_analysis_sub_session table of the data analysis agent.

[0204] Judge the current task_type field in this table. If it is "train", it means that the current is in the training task and enter the training process; if it is "predict", it means that the current is a prediction task and enter the prediction process.

[0205] III. Obtain the dataset

[0206] (1) Main Phases

[0207] 1.1. Label Data Preparation:

[0208] a. Mark the data in the questionnaire as the original label text for the convenience of users to mark. After the marked data is stored in the training set, the data in the training set is the original text data.

[0209] b. The data preparation is for the coverage logic: For example, during the data preparation process, if the user redesigned a set of label systems and selected labels in the label design Agent, triggering the data preparation logic, the data prepared in the previous batch will be overwritten.

[0210] c. During the data preparation process, when the user enters the data analysis Agent to start executing the model planning task, after the data preparation is completed and the planning task is executed, a message needs to be sent to notify the customer.

[0211] 1.2. Model Training Phase:

[0212] a. If the number of labels is less than 100, use a 12 - layer model for training. If it is greater than or equal to 100, use a 20 - layer model for training.

[0213] b. Analysis task names: Data analysis model_{uuid}, Data analysis training set_{uuid}, Data analysis task_{uuid}.

[0214] (2) Marking Questionnaire

[0215] Sample data from the original text fragments corresponding to the labels output by the label design Agent to form a marking questionnaire for users to mark. After the users submit, the questionnaire data will enter the training set for model training.

[0216] To ensure the model effect, examples of the amount of marked data under each label are as follows:

[0217] a. When the label < 50, the data corresponding to a single label is not less than 10 pieces.

[0218] b. When 50 ≤ label ≤ 80, the data corresponding to a single label is not less than 20 pieces.

[0219] c. When 80 < label ≤ 100, the data corresponding to a single label is not less than 30 pieces.

[0220] Since 25% of the data can be extracted from the training set as the validation set when training the model, the data volume can be appropriately expanded. Examples are as follows:

[0221] a. When the label < 50, the data corresponding to a single label is not more than 15 pieces.

[0222] b. When 50 ≤ label ≤ 80, the number of data corresponding to a single label is no more than 25.

[0223] c. When 80 < label ≤ 100, the number of data corresponding to a single label is no more than 40.

[0224] 2.1 Data Preparation

[0225] After the label design Agent designs and outputs labels, after the user performs the operation of adding labels, the label data preparation logic is triggered to output the user-annotated questionnaire. It is necessary to associate the corresponding original text data through the label.

[0226] 2.1.1 Prerequisites

[0227] (1) The data access analysis end data management of the data imported by the label design Agent file.

[0228] (2) The work order / conversation ID is unique in the analysis end data index and can be used to retrieve the complete document.

[0229] 2.1.2 Examples of data relationships, such as Figure 9 as shown.

[0230] 2.1.3 Examples of task processes, such as Figure 10 as shown.

[0231] (1) Label to clustering result: See Figure 9 , the relationship between the label and the clustering result can be one-to-one (1:1).

[0232] See Figure 10 , in S1001, the cluster_id (cluster ID) corresponding to this label is stored in the current intelligent agent temporary label (agent_tmp_label) table, and the cluster to which this label belongs can be located.

[0233] (2) Clustering result to original text fragment: See Figure 9 , the relationship between the clustering result and the original text fragment can be one-to-many (1:N).

[0234] In S1002, through the sub-session identifier (label_agent_sub_session_id) of the label design intelligent agent and the cluster_id, the set of original text fragments corresponding to the label of this cluster can be retrieved from the intelligent agent label clustering result (agent_label_cluster_result) table of the clustering result. The data in this table is generated after parsing the clustering result file in the data preparation stage of the label generation task.

[0235] Table 1 Clustering Result Table

[0236] Name Type Description id varchar(32) PK agent_id varchar(32) Tenant ID agent_label_session_id varchar(32) Session ID cluster_id varchar(32) Cluster ID text_id varchar(1024) Original Text ID txt text Original Data created_at datetime Creation Time updated_at datetime Update Time

[0237] (3)S1003. Original text segment to original text: See Figure 9 The original text (or original data) and the original text segment can have a one-to-many (1:N) relationship.

[0238] Prerequisite: In the analysis end data management, text identifiers such as work order / dialog ID (text_id) can be used as the unique key to retrieve documents.

[0239] The set of original text segments retrieved from agent_label_cluster_result has a text identifier (text_id) field (such as work order / dialog ID). For example, in S1003, sample and label the text identifier (text_id) required in the questionnaire from the set of original text segments. In S1004, retrieve the complete original text from the analysis end data index through this text ID field. In S1005, a training set can be established using the complete original text and stored in the database.

[0240] In this example, the voc_agent.agent_tmp_label can have a new field such as in_use, which can be used to record whether the label is selected by the user.

[0241] When the user adds a label to a label group, the intelligent agent microservice (voc_agent) can create a new data table agent_analysis_report to record the annotation questionnaire information. The example is as follows:

[0242] Table 2 Agent Analysis Report Table

[0243]

[0244] Create a new data table agent_analysis_report_data to record the data questionnaire data. The example is as follows:

[0245] Table 3 Agent Analysis Report Data Table

[0246]

[0247] 2.1.4 Data Cleaning

[0248] (1) agent_label_cluster_result: After the user submits the questionnaire, execute the cleaning logic.

[0249] (2) agent_analysis_report_data: After the data is stored in the training set, execute the cleaning logic.

[0250] 2.2 Replace Data.

[0251] Perform a new batch operation on all the data to be labeled under a certain tag. As Figure 11 shown, the example is as follows:

[0252] S1101. According to agent_label_cluster_result, resample to obtain new questionnaire data.

[0253] S1102. Clean the original questionnaire data according to agent_analysis_report_data.

[0254] S1103. Store the new questionnaire data in the database.

[0255] (III) Model Training

[0256] After the user submits the labeled questionnaire, the model training logic is triggered. After the training is completed, a message indicating the successful model training is pushed to the user and the model training details are displayed.

[0257] After the user submits the questionnaire, change the status in the questionnaire data record to the status of the user passing or being rejected. Start an asynchronous task to execute the model training logic, and its timing is as Figure 12 shown:

[0258] After the questionnaire is submitted, clean the labeled result data. voc-agent initiates the following operations to the microservice dataset (voc-java-dataset):

[0259] S1201. Create an analysis task: Create an analysis task (which can generate an analysis task ID) under the Agent management terminal to which the current session belongs, and the type is a composite label model.

[0260] S1202. Execute the batch label addition operation (label import, which can generate a label version): Add the labels selected by the user to the analysis task.

[0261] S1203. Effect the labels.

[0262] S1204. Create a training set (which can generate a training set ID): Create a training set corresponding to the currently effective labels, and the type is only labeled labels.

[0263] S1205. voc-agent performs the training set data preparation:

[0264] a. Retrieve the data passed by the user in the labeled questionnaire.

[0265] b. Retrieve the original text data from the analysis end data management according to the original text / dialog ID.

[0266] c. Store the data in the training set file in the database.

[0267] d. Update the amount of training set data.

[0268] S1206. After the training set data is prepared, questionnaire data cleaning can be performed. voc-agent sends a model training request to the model engine (voc-modelengine). voc-modelengine can return a response corresponding to the training set ID to voc-agent.

[0269] S1206.1 voc-agent sends a timed status synchronization to voc-modelengine.

[0270] S1206.2. Task callback: After the model training is completed, voc-modelengine calls back to voc-agent, and the training process ends.

[0271] S1207. voc-modelengine submits a training container (Pod) to the container service (k8s).

[0272] S1207.1. k8s returns to voc-modelengine that the monitoring (Watch) of the Pod has been completed.

[0273] The asynchronous tasks in the process can synchronize the task status through webhooks and timed status synchronization. Among them, S1206.2 can be executed after S1207.1.

[0274] 3.1 Label import, as Figure 13 shown. The example is as follows:

[0275] S1301. voc-agent filters and selects labels.

[0276] S1302. voc-agent sends the selected labels to voc-agent-dataset to store the selected labels in the database.

[0277] S1303. voc-agent-dataset returns the label version to voc-agent.

[0278] S1304. voc-agent sends the effective labels to voc-agent-dataset.

[0279] 3.2 Training set management

[0280] (1) Create a training set.

[0281] (2) Prepare the training set data file.

[0282] (3) The logic of randomly sampling the validation set is currently completed during the training phase.

[0283] An example of a process for generating a training set file is as Figure 14 shown:

[0284] S1401. Collect questionnaire data through a questionnaire.

[0285] S1402. Screen the passed annotation items from the questionnaire data to obtain the annotation data.

[0286] S1403. Search for the original text set of labels (or called the original text fragment set) through the annotation data.

[0287] S1404. Obtain the original text ID set according to the original text set of labels.

[0288] The original text ID set retrieves the original text (or called the original text data) from the analysis end data index.

[0289] What is saved in the annotated questionnaire is the original text of the label rather than the complete original text. Therefore, it is necessary to retrieve the complete document from the analysis end data index according to the original text ID (text_id).

[0290] S1406. Associate the original text with its corresponding label to obtain the training set file.

[0291] Since the complete original text is split into multiple original text fragments of labels, after retrieving the original text, it is necessary to perform an association operation with the belonging label. Store it in the Ernie training set file.

[0292] 3.3 Model Training

[0293] The following is an introduction to the parameters related to model training:

[0294] Table 4 Model Training Parameters

[0295]

[0296] To support the training task webhook, it is necessary to expand the voc_modelengine.train_task field. The example is as follows:

[0297] Table 5 Microservice Model Engine Training Task Fields

[0298]

[0299] 4. Data Prediction

[0300] Analysis end data prediction status:

[0301] (1) For tasks originating from analysis tasks: The online prediction method relies on the deployed model service.

[0302] (2) For tasks originating from the template market: The offline prediction method does not rely on the online model service. Just submit the model prediction Pod to k8s.

[0303] For the model planning task of the data analysis Agent, the models output by the Planner originate from the template market. Therefore, the offline prediction method can be adopted.

[0304] For the model training task of the data analysis Agent, since the model has not been released, the online prediction method can be adopted. This method requires deploying the model in the management - end analysis task first and then starting the prediction task.

[0305] The prediction task on the analysis side can be the prediction task of the data analysis Agent training task and can adopt the offline prediction logic.

[0306] Figure 15 FIG. 16 is a schematic structural diagram of a model training apparatus 1500 according to an embodiment of the present disclosure, including:

[0307] A first acquisition module 1501, configured to acquire label interaction information corresponding to a first label;

[0308] A second acquisition module 1502, configured to acquire a label annotation result according to the label interaction information;

[0309] A first generation module 1503, configured to generate a training set according to the label annotation result;

[0310] A training module 1504, configured to train an initial model according to the training set to obtain a target model.

[0311] Figure 16 FIG. 17 is a schematic structural diagram of a model training apparatus 1600 according to another embodiment of the present disclosure. The apparatus 1600 may include: a first acquisition module 1601, a second acquisition module 1602, a first generation module 1603, and a training module 1604. The functions of the above - mentioned modules can refer to the functions of the respective modules of the model training apparatus 1500 in the above - mentioned embodiment. In one implementation manner, the first acquisition module 1601 is configured to trace back to first original data according to the first label; generate the label interaction information according to the first label and the first original data, and acquire the label interaction information.

[0312] In one embodiment, the first acquisition module 1601 obtains first original text data by tracing the first tag, including: finding the identifier of the first cluster corresponding to the first tag; finding the set of original text segments corresponding to the identifier of the first cluster; finding the first original text data corresponding to the text identifier in the set of original text segments. For example, find the identifier of the first cluster corresponding to the first tag in the temporary tag; find the set of original text segments corresponding to the identifier of the first cluster in the tag clustering result; find the first original text data corresponding to the text identifier in the set of original text segments in the server data.

[0313] In one embodiment, the tag interaction information includes annotation options for annotating the first tag and its associated first original text data. For example, Figure 16 as shown, the second acquisition module 1602 is configured to, in response to a selection command for the annotation option, determine whether there is an annotation relationship between the first tag and its corresponding first original text data; and add second original text data that conforms to the first tag to the tag annotation result according to the annotation relationship between the first tag and its corresponding first original text data.

[0314] In one embodiment, as Figure 16 shown, the apparatus further includes:

[0315] A replacement module 1605, configured to, in response to a replacement command for the tag interaction information, obtain the replaced tag interaction information, where the replaced tag interaction information includes replaced first original text data corresponding to the first tag.

[0316] In one embodiment, as Figure 16 shown, the first generation module 1603 is configured to generate a training set according to the second original text data and its corresponding first tag in the tag annotation result; the training set is used to train an initial model.

[0317] In one embodiment, as Figure 16 shown, the apparatus further includes:

[0318] A second generation module 1606, configured to generate a validation set according to the second original text data and its corresponding first tag in the tag annotation result; the validation set is used to validate the target model.

[0319] In one embodiment, the apparatus further includes an execution module, and the execution module is configured to perform one of the following:

[0320] When there is a target agent record in the session of the agent and the task type in the sub - session of the target agent is model training, perform model training;

[0321] There is a target agent record in the session of the agent, and when the task type in the sub - session of the target agent is model prediction, model prediction is executed;

[0322] When there is no target agent record in the session of the agent and there is a target agent record in the analysis report of the agent, model training is executed;

[0323] When there is no target agent record in the session of the agent and there is no target agent record in the analysis report of the agent, model prediction is executed.

[0324] In one implementation, as Figure 16 shown, the device further includes:

[0325] A creation module 1607, configured to create prompt to obtain prediction requirement information according to the prediction task corresponding to the target model;

[0326] An input module 1608, configured to input the prediction requirement information into the target model to execute the prediction task.

[0327] For the specific functions and examples of each module and sub - module of the device in the embodiments of the present disclosure, reference may be made to the relevant descriptions of the corresponding steps in the above - mentioned method embodiments, which will not be elaborated herein.

[0328] Figure 17 is a schematic structural diagram of an agent 1700 according to an embodiment of the present disclosure. The agent may include:

[0329] An input module 1701, configured to input label interaction information corresponding to the first label;

[0330] A processing module 1702, configured to call a large - model to execute a target model trained by the model training method in any embodiment of the present disclosure based on the label interaction information received by the input module;

[0331] An output module 1703, configured to output the target model obtained by the processing module.

[0332] For the specific functions and examples of each module of the agent in the embodiments of the present disclosure, reference may be made to the relevant descriptions of the corresponding steps in the above - mentioned method embodiments, which will not be elaborated herein.

[0333] In the technical solution of the present disclosure, the acquisition, storage, and application of the user's personal information involved all comply with the provisions of relevant laws and regulations and do not violate public order and good customs.

[0334] According to the embodiments of the present disclosure, the present disclosure further provides an electronic device, a readable storage medium, and a computer program product.

[0335] Figure 18 FIG. 1800 is a schematic block diagram of an exemplary electronic device that may be used to implement embodiments of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as, for example, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as, for example, personal digital assistants, cellular telephones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely exemplary and are not intended to limit the implementations of the present disclosure described and / or claimed herein.

[0336] As Figure 18 shown, the device 1800 includes a computing unit 1801 that can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1802 or a computer program loaded from a storage unit 1808 into a random access memory (RAM) 1803. In the RAM 1803, various programs and data required for the operation of the device 1800 can also be stored. The computing unit 1801, the ROM 1802, and the RAM 1803 are connected to each other via a bus 1804. An input / output (I / O) interface 1805 is also connected to the bus 1804.

[0337] A plurality of components in the device 1800 are connected to the I / O interface 1805, including: an input unit 1806, such as a keyboard, a mouse, etc.; an output unit 1807, such as various types of displays, speakers, etc.; a storage unit 1808, such as a magnetic disk, an optical disk, etc.; and a communication unit 1809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1809 allows the device 1800 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0338] The computing unit 1801 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1801 executes the various methods and processes described above, such as the model training method. For example, in some embodiments, the model training method can be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as the storage unit 1808. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 1800 via the ROM 1802 and / or the communication unit 1809. When the computer program is loaded into the RAM 1803 and executed by the computing unit 1801, one or more steps of the model training method described above can be executed. Alternatively, in other embodiments, the computing unit 1801 can be configured to execute the model training method in any other suitable manner (e.g., by means of firmware).

[0339] Various embodiments of the systems and techniques described above in this document can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special-purpose or general-purpose programmable processor, and can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0340] The program code for implementing the methods of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to the processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing devices, such that when the program codes are executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program codes can be executed entirely on the machine, partially on the machine, as an independent software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0341] In the context of this disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0342] In order to provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, speech input, or tactile input).

[0343] The systems and techniques described herein can be implemented in a computing system including backend components (e.g., as a data server), or a computing system including middleware components (e.g., an application server), or a computing system including frontend components (e.g., a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system including any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0344] A computer system can include a client and a server. The client and the server are generally far apart from each other and typically interact through a communication network. The relationship between the client and the server is generated by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, can also be a server of a distributed system, or a server incorporating a blockchain.

[0345] It should be understood that the various forms of processes shown above can be used, with steps reordered, added or deleted. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and no limitations are imposed herein.

[0346] The above specific embodiments do not constitute a limitation on the protection scope of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub - combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the principles of this disclosure shall be included within the protection scope of this disclosure.

Claims

1. A model training method, comprising: Obtaining tag interaction information corresponding to the first tag; Acquire a label marking result according to the label interaction information; Generate a training set according to the label annotation results; Training is performed according to the initial model of the training set to obtain a target model.

2. The method according to claim 1, wherein: The acquiring the tag interaction information corresponding to the first tag includes: Obtaining first original data by tracing the first tag; The tag interaction information is generated according to the first tag and the first original data, and the tag interaction information is acquired.

3. The method according to claim 2, wherein: The first original data is obtained by tracing the source of the first tag, including: Finding an identifier of a first cluster corresponding to the first label; Searching for a set of original text segments corresponding to the identifier of the first cluster; The first original text data corresponding to the text identifier in the original text segment set is searched.

4. The method according to claim 2 or 3, wherein: The tag interaction information includes a tagging option, and the tagging option is used to tag the first tag and the first original data associated with the first tag. The tag tagging result is obtained according to the tag interaction information, including: In response to a selection command for the annotation option, determining whether there is an annotation relationship between the first tag and the first original text data corresponding to the first tag; According to the annotation relationship between the first tag and the first original data corresponding to the first tag, the second original data matching the first tag is added to the tag annotation result.

5. The method according to any one of claims 2 to 4, further comprising: In response to a command to replace the tag interaction information, replaced tag interaction information is acquired, where the replaced tag interaction information includes replaced first original text data corresponding to the first tag.

6. The method according to claim 4 or 5, wherein: Generating a training set according to the label annotation result includes: A training set is generated according to the second original text data in the labeling result and its corresponding first label; the training set is used to train the initial model.

7. The method according to claim 4 or 5, further comprising: Generate a verification set according to the second original text data in the label annotation result and the first label corresponding to it; The validation set is used to validate the target model.

8. The method according to any one of claims 1 to 7, further comprising one of the following: If there is a target agent record in the agent's session and the task type in the target agent's subsession is model training, perform model training; If there is a target agent record in the agent's session and the task type in the target agent's subsession is model prediction, perform model prediction; Perform model training when there is no target agent record in the agent's session and there is a target agent record in the agent analysis report; Perform model prediction when there is no target agent record in the agent's session and no target agent record in the agent's analysis report.

9. The method according to any one of claims 1 to 8, further comprising: Create prompts based on the forecast task corresponding to the target model to obtain forecast demand information; The forecast demand information is input into the target model to perform the forecast task.

10. A model training device, comprising: A first acquisition module, used to acquire tag interaction information corresponding to a first tag; A second acquisition module, used to acquire a label marking result according to the label interaction information; A first generating module, used to generate a training set according to the label marking result; The training module is used to perform training according to the initial model of the training set to obtain a target model.

11. An intelligent agent, comprising: An input module, used for inputting tag interaction information corresponding to the first tag; A processing module, configured to call a large model to execute training according to any one of the model training methods in claims 1 to 9 to obtain a target model based on the label interaction information received by the input module; An output module is used to output the target model obtained by the processing module.

12. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 9.

13. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-9.

14. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 9.

Images