Model determination method and device, electronic equipment and computer readable storage medium
By acquiring intent corpus datasets and determining label information, the model is trained and its performance is compared. A target model that is superior to the initial model is selected, which solves the problem of the decline in prediction performance of intent recognition models after the addition of new user statements and business changes. This achieves efficient model updates and reduces deployment costs.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- 北京中关村科金技术有限公司
- Filing Date
- 2022-06-23
- Publication Date
- 2026-06-02
AI Technical Summary
In existing technologies, intent recognition models are prone to introducing incorrect knowledge after the addition of new user statements and changes in customer business, resulting in poor prediction performance, and offline training cannot meet the timeliness requirements.
By acquiring the intent corpus dataset, determining the corresponding label information of the data, training the model based on this information, comparing the model's prediction performance, and selecting the target model that is superior to the initial model to avoid a decline in prediction performance.
This effectively avoids the deterioration of the prediction performance of the intent recognition model, meets the timeliness requirements of model retraining, and reduces deployment costs.
Smart Images
Figure CN115905851B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of intent recognition technology, and in particular to a method, apparatus, electronic device, and computer-readable storage medium for determining a model. Background Technology
[0002] Intent recognition is a crucial function for building dialogue systems and intelligent customer service systems. After the system is officially deployed, a large number of new user statements accumulate over time, or customer business changes lead many clients to demand the ability to retrain the intent recognition model. Related technologies involve directly adding new corpora to the training data for retraining; however, this may introduce erroneous knowledge into the existing model, resulting in poor prediction performance. Furthermore, directly adding new corpora to the training data for retraining is usually done offline, which cannot meet the timeliness requirements. Summary of the Invention
[0003] To address the problems in the aforementioned related technologies, this application provides a method, apparatus, electronic device, and computer-readable storage medium for determining a model, which can prevent the prediction performance of the obtained first target model from deteriorating.
[0004] This application provides a method for determining a model, the method comprising:
[0005] Obtain the first intent corpus dataset;
[0006] The first intent corpus dataset is input into a pre-established first model to determine the label information corresponding to each intent corpus data in the first intent corpus dataset;
[0007] The first model is trained based on the first intent corpus dataset and the label information corresponding to each intent corpus data in the first intent corpus dataset to obtain the second model; and the prediction effect of the second model is determined.
[0008] The first target model is determined based on the prediction performance of the second model and the prediction performance of the first model.
[0009] In some embodiments, determining the first target model based on the prediction performance of the second model and the prediction performance of the first model includes:
[0010] If the prediction performance of the second model is better than that of the first model, the second model is determined as the first target model.
[0011] If the prediction performance of the first model is better than that of the second model, the first model is determined as the first target model.
[0012] In some embodiments, the method further includes:
[0013] Obtain the second intent corpus dataset;
[0014] The initial model is trained based on the second intent corpus dataset to obtain the first model;
[0015] Save the first model and its prediction results.
[0016] In some embodiments, the method further includes:
[0017] The label information corresponding to each intent corpus data in the first intent corpus dataset is optimized.
[0018] The step of training the first model based on the first intent corpus dataset and the label information corresponding to each intent corpus data in the first intent corpus dataset includes: training the first model based on the optimized first intent corpus dataset and the label information corresponding to each intent corpus data in the first intent corpus dataset.
[0019] In some embodiments, determining the first target model based on the prediction performance of the second model and the prediction performance of the first model includes:
[0020] If the prediction performance of the first model is better than that of the second model, output a prompt message to encourage the user to optimize again;
[0021] Having obtained the first intent corpus dataset after further optimization and the label information corresponding to each intent corpus data in the first intent corpus dataset, the first model is trained using the first intent corpus dataset after further optimization and the label information corresponding to each intent corpus data in the first intent corpus dataset to obtain the third model.
[0022] The first target model is determined based on the prediction performance of the third model and the prediction performance of the first model.
[0023] In some embodiments, the method further includes:
[0024] Save the first target model and its prediction results;
[0025] Once the intent corpus data is obtained, it is input into the first target model to determine the predicted label corresponding to the intent corpus data.
[0026] In some embodiments, the method further includes:
[0027] Obtain the third-intention corpus dataset;
[0028] The third intent corpus dataset is input into the first target model to determine the label information corresponding to each intent corpus data in the third intent corpus dataset;
[0029] The first target model is trained based on the third intent corpus dataset and the label information corresponding to each intent corpus data in the third intent corpus dataset to obtain the second target model; and the prediction effect of the second target model is determined.
[0030] The third target model is determined based on the prediction performance of the second target model and the prediction performance of the first target model.
[0031] This application provides a model determining apparatus, comprising:
[0032] The first acquisition module is used to acquire the first intent corpus dataset;
[0033] The first determining module is used to input the first intent corpus dataset into a pre-established first model and determine the label information corresponding to each intent corpus data in the first intent corpus dataset;
[0034] The first training model is used to train the first model based on the first intent corpus dataset and the label information corresponding to each intent corpus data in the first intent corpus dataset to obtain the second model; and to determine the prediction effect of the second model.
[0035] The second determining module determines the first target model based on the prediction performance of the second model and the prediction performance of the first model.
[0036] This application provides an electronic device, including a memory and a processor. The memory stores a computer program, which, when executed by the processor, performs a method for determining a model as described in any of the preceding claims.
[0037] This application provides a computer-readable storage medium storing a computer program that can be executed by one or more processors and can be used to implement the model determination method described in any of the above claims.
[0038] This application provides a method, apparatus, electronic device, and computer-readable storage medium for determining a model. By inputting a first intent corpus dataset into a pre-established first model, the label information corresponding to each intent corpus data in the first intent corpus dataset is determined. Then, the first model is trained based on the first intent corpus dataset and the label information corresponding to each intent corpus data in the first intent corpus dataset to obtain a second model. A first target model is determined based on the prediction effect of the second model and the prediction effect of the first model, which can avoid the prediction effect of the obtained first target model from deteriorating. Attached Figure Description
[0039] The present application will be described in more detail below based on embodiments and with reference to the accompanying drawings.
[0040] Figure 1 A schematic diagram illustrating the implementation flow of a model determination method provided in an embodiment of this application;
[0041] Figure 2 A schematic diagram illustrating the implementation flow of another model determination method provided in this application embodiment;
[0042] Figure 3 A schematic diagram of the structure of a model determining device provided in an embodiment of this application;
[0043] Figure 4 This is a schematic diagram of the composition structure of the electronic device provided in the embodiments of this application.
[0044] In the accompanying drawings, the same parts are referred to by the same reference numerals, and the drawings are not drawn to scale. Detailed Implementation
[0045] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0046] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.
[0047] If the application documents contain similar descriptions such as "first, second, third", the following explanation shall be added: In the following description, the terms "first, second, third" are used only to distinguish similar objects and do not represent a specific order of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.
[0048] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.
[0049] Before introducing the embodiments of this application, a brief overview of the problems in the related technologies will be provided.
[0050] Intent recognition is a crucial function for building dialogue systems and intelligent customer service systems. After the system is officially deployed, a large number of new user queries accumulate over time, or customer business changes lead many clients to demand the ability to retrain the intent recognition model. Related technologies involve directly adding new corpora to the training data for retraining; however, this may introduce incorrect knowledge into the existing model, resulting in poor prediction performance. Furthermore, directly adding new corpora to the training data for retraining is usually done offline, which cannot meet the timeliness requirements.
[0051] A related technology provides a model retraining method based on local offline learning, including:
[0052] Step S1: Prepare the intent dataset by dividing it into a training set and a test set;
[0053] Step S2: Construct an intent recognition model;
[0054] Step S3: Perform model training and model performance evaluation, and save the best model locally;
[0055] Step S4: Deploy the optimal model to the user server;
[0056] Step S5: Based on business changes, collect new intent corpus data, and after processing by annotation and algorithm personnel, add it to the original intent dataset and divide it into new training and test sets.
[0057] Step S6: Combine the intent recognition model and train, evaluate, and deploy it according to steps S3 and S4.
[0058] Step S7: Continue following steps S5 and S6.
[0059] However, one model retraining method based on local offline learning requires that after the new data is labeled, the model needs to be trained locally using the new data before it can be deployed to the customer's production environment. If the customer's business changes, the model needs to be redeployed after retraining, which is costly. Local training is offline training and cannot meet the timeliness requirements of online training.
[0060] Related technologies also provide a method for retraining online learning models, the method comprising:
[0061] Step S21: Prepare the intent dataset by dividing it into a training set and a test set;
[0062] Step S22: Construct an intent recognition model;
[0063] Step S23: Deploy the model to the user server;
[0064] Step S24: Perform model training, evaluate model performance, and save the best model to the server;
[0065] Step S25: Based on actual production data, business personnel screen the intent data, annotate it, add it to the original intent dataset, and divide it into new training and test sets.
[0066] Step S26: Combine the intent recognition model and train, evaluate, and save the model according to step S24.
[0067] Step S27: Continue with steps S25 and S26.
[0068] However, in one of the related technologies, when an online learning model retraining method is used in a production environment, the newly added business corpus is labeled by business personnel, and the labeling quality cannot be guaranteed, which may introduce a large amount of noisy data; the model trained with the newly added data may have a risk of deteriorating performance.
[0069] Based on the problems existing in related technologies, this application provides a method for determining a model. The method is applied to electronic devices, which may specifically include mobile phones, tablets, laptops, ultra-mobile personal computers (UMPCs), handheld computers, netbooks, personal digital assistants (PDAs), wearable electronic devices, in-vehicle devices (also known as vehicle systems), virtual reality devices, servers, etc. This application does not impose any limitations on these. The function implemented by the model determination method provided in this application can be achieved by the processor of the electronic device calling program code, wherein the program code can be stored in a computer storage medium. This application provides a method for determining a model. Figure 1 This application provides a schematic diagram of the implementation process of a method for determining a model, as illustrated in the embodiments below. Figure 1 As shown, it includes:
[0070] Step S1: Obtain the first intent corpus dataset.
[0071] In this embodiment, the first intent corpus dataset includes multiple intent corpus data sets, which may be generated during the application of the first model. In this embodiment, the first intent corpus dataset can be obtained from a business system. In some embodiments, the first intent corpus dataset can be obtained via the Internet.
[0072] Step S2: Input the first intent corpus dataset into the pre-established first model to determine the label information corresponding to each intent corpus data in the first intent corpus dataset.
[0073] In this embodiment, a second intent corpus dataset can be obtained; an initial model can be trained based on the second intent corpus dataset to obtain the first model; and the first model and its prediction results can be saved. In this embodiment, the input of the first model is intent corpus data, and the output of the first model is the label information corresponding to each intent corpus data, wherein the label information may be the result of intent recognition.
[0074] In this embodiment, the step of training an initial model based on the second intent corpus dataset to obtain the first model can be achieved as follows: dividing the second intent corpus dataset into a training set and a test set, constructing an initial model according to the intent recognition task, training the initial model using the training set, and evaluating and optimizing the trained initial model using the test set. In this embodiment, the prediction performance is used to characterize the model's accuracy in intent recognition. In this embodiment, during the optimization process, the optimal model can be determined based on the model's prediction performance, and the obtained optimal model is the first model.
[0075] In this embodiment of the application, when the first intent corpus dataset is input into the first model, the label information corresponding to each intent corpus data in the first intent corpus dataset can be predicted.
[0076] In the embodiments of this application, the individual corpus data in the first intent corpus dataset are usually different from the individual intent corpus data in the second intent corpus dataset.
[0077] Step S3: Train the first model based on the first intent corpus dataset and the label information corresponding to each intent corpus data in the first intent corpus dataset to obtain the second model; and determine the prediction effect of the second model.
[0078] In this embodiment of the application, the first intent corpus dataset and the label information corresponding to each intent corpus data in the first intent corpus dataset can be determined as a sample set. The sample set is divided into a training set and a test set to train the first model. The test set is used to evaluate and optimize the first model. During the optimization process, the optimal model can be determined based on the prediction effect of the model. The obtained optimal model is the second model.
[0079] Step S4: Determine the first target model based on the prediction performance of the second model and the prediction performance of the first model.
[0080] In this embodiment of the application, the prediction effects of the second model and the first model can be compared. If the prediction effect of the second model is better than that of the first model, it means that the introduction of the first intent corpus dataset and the label information corresponding to each intent corpus data in the first intent corpus dataset has not made the prediction effect of the trained second model worse than that of the first model. Therefore, the second model is determined as the first target model.
[0081] If the prediction performance of the first model is better than that of the second model, it means that the introduction of the first intent corpus dataset and the label information corresponding to each intent corpus data in the first intent corpus dataset makes the prediction performance of the trained second model worse than that of the first model. Therefore, the first model is determined as the first target model.
[0082] In this embodiment of the application, after the first target model is determined, the first target model and the prediction effect corresponding to the first target model can be saved, so as to continue to optimize the first target model when new training data is obtained.
[0083] This application provides a method for determining a model. By inputting the first intent corpus dataset into a pre-established first model, the label information corresponding to each intent corpus data in the first intent corpus dataset is determined. Then, the first model is trained based on the first intent corpus dataset and the label information corresponding to each intent corpus data in the first intent corpus dataset to obtain a second model. The first target model is determined based on the prediction effect of the second model and the prediction effect of the first model, which can avoid the prediction effect of the obtained first target model from deteriorating.
[0084] In some embodiments, prior to step S3, the method further includes:
[0085] Step S31: Optimize the first intent corpus dataset and the label information corresponding to each intent corpus data in the first intent corpus dataset.
[0086] In this embodiment, the optimization can be a review and correction, and can receive user operations to optimize the first intent corpus dataset and the label information corresponding to each intent corpus data in the first intent corpus dataset. In this embodiment, the user can be a professional.
[0087] In some embodiments, step S3, "training the first model based on the first intent corpus dataset and the label information corresponding to each intent corpus data in the first intent corpus dataset," can be achieved through the following steps:
[0088] The first model is trained based on the optimized first intent corpus dataset and the label information corresponding to each intent corpus data in the first intent corpus dataset.
[0089] In some embodiments, step S4, "determining the first target model based on the prediction results of the second model and the prediction results of the first model," can also be achieved through the following steps:
[0090] Step S41: If the prediction effect of the first model is better than that of the second model, output a prompt message to encourage the user to optimize again.
[0091] In this embodiment of the application, by inputting a prompt message for further optimization, the user can further optimize the first intent corpus dataset and the label information corresponding to each intent corpus data in the first intent corpus dataset, thereby improving the quality of the newly added intent corpus.
[0092] Step S42: After obtaining the first intent corpus dataset that has been optimized again and the label information corresponding to each intent corpus data in the first intent corpus dataset, the first model is trained on the first intent corpus dataset that has been optimized again and the label information corresponding to each intent corpus data in the first intent corpus dataset to obtain the third model.
[0093] Step S43: Determine the first target model based on the prediction results of the third model and the prediction results of the first model.
[0094] In this embodiment of the application, the prediction effects of the third model and the first model can be compared. If the prediction effect of the third model is better than that of the first model, it means that the introduced optimized first intent corpus dataset and the label information corresponding to each intent corpus data in the first intent corpus dataset have not made the prediction effect of the trained third model worse. Therefore, the third model is determined as the first target model.
[0095] If the prediction performance of the first model is better than that of the third model, it indicates that the introduction of the further optimized first intent corpus dataset and the label information corresponding to each intent corpus data in the first intent corpus dataset worsens the performance of the trained first model. Therefore, the first model is determined as the first target model. In this embodiment, the user can be further prompted to optimize the further optimized first intent corpus dataset and the label information corresponding to each intent corpus data in the first intent corpus dataset.
[0096] After step S4, the method further includes:
[0097] Step S5: Save the first target model and its prediction effect.
[0098] Step S6: If the intent corpus data is obtained, input the intent corpus data into the first target model to determine the prediction label corresponding to the intent corpus data.
[0099] In some embodiments, after step S5, the method further includes:
[0100] Step S7: Obtain the third intent corpus dataset;
[0101] Step S8: Input the third intent corpus dataset into the first target model and determine the label information corresponding to each intent corpus data in the third intent corpus dataset;
[0102] Step S9: Train the first target model based on the third intent corpus dataset and the label information corresponding to each intent corpus data in the third intent corpus dataset to obtain the second target model; and determine the prediction effect of the second target model.
[0103] Step S10: Determine the third target model based on the prediction performance of the second target model and the prediction performance of the first target model.
[0104] Based on the foregoing embodiments, this application further provides a method for determining the model. Figure 2 This application provides a schematic diagram of the implementation process of a method for determining a model, as illustrated in the embodiments below. Figure 2 As shown, the method includes:
[0105] Step S21, Data preparation.
[0106] In this embodiment, a certain amount of intent corpus can be prepared to form an intent dataset (similar to the second intent corpus dataset in the above embodiment), and it can be divided into a training set and a test set.
[0107] Step S22, Model Construction.
[0108] In this embodiment of the application, a preliminary model (initial model) is constructed based on the intent recognition task.
[0109] Step S23, Model Deployment.
[0110] In this embodiment of the application, the initially constructed model and dataset are deployed together to the client server.
[0111] Step S24, Model Evaluation.
[0112] In this embodiment, the intent dataset is used for training, prediction, and evaluation and optimization of the model performance.
[0113] Step S25: Save the model.
[0114] In this embodiment of the application, the optimal model (first model) and its parameters are obtained after the model effect meets the requirements, based on the model effect, for subsequent use.
[0115] Step S26, pre-annotation.
[0116] In this embodiment of the application, for newly added intent corpus accumulated in practical applications, the intent category is initially predicted using the optimal model, where the intent category is the tag information.
[0117] Step S27, manual review.
[0118] In this embodiment of the application, the pre-labeled data is reviewed and corrected by professionals from the client.
[0119] Step S28, Model Evaluation.
[0120] In this embodiment, the model is retrained and predicted based on the original optimal model by combining the data after manual review, and the model performance is evaluated and optimized.
[0121] Step S29, effect comparison.
[0122] In this embodiment, the next processing method is selected based on the change in the model's effect. If the effect worsens, step S30 is executed; if the effect improves, step S25 is executed.
[0123] Step S30: Model rollback.
[0124] In this embodiment of the application, if the model performance deteriorates, the model will be rolled back to the model maintained in step S25.
[0125] Step S31, data optimization.
[0126] In this embodiment of the application, after the model rollback, the verification data needs to be optimized.
[0127] Step S32, model prediction.
[0128] If the model performs better, the model will be updated to obtain the optimal model and its parameters, which will then be saved for later use.
[0129] In this embodiment, the currently saved optimal model parameters are used for model prediction tasks, etc.
[0130] This paper presents a method for determining models, which can effectively control the quality of data and models, respond promptly to changes in customer business, meet the timeliness requirements of model retraining and model deployment, and reduce costs.
[0131] Based on the foregoing embodiments, this application provides a model determination device. The modules and units included in the device can be implemented by a processor in a computer device; of course, they can also be implemented by specific logic circuits. In the implementation process, the processor can be a central processing unit (CPU), a microprocessor unit (MPU), a digital signal processor (DSP), or a field programmable gate array (FPGA), etc.
[0132] This application provides a model determining device 300. Figure 3 This is a schematic diagram of the structure of a model determining device provided in an embodiment of this application, as shown below. Figure 3 As shown, the model determining device 300 includes:
[0133] The first acquisition module 301 is used to acquire the first intent corpus dataset;
[0134] The first determining module 302 is used to input the first intent corpus dataset into a pre-established first model and determine the label information corresponding to each intent corpus data in the first intent corpus dataset;
[0135] The first training model 303 is used to train the first model based on the first intent corpus dataset and the label information corresponding to each intent corpus data in the first intent corpus dataset to obtain the second model; and to determine the prediction effect of the second model.
[0136] The second determining module 304 determines the first target model based on the prediction effect of the second model and the prediction effect of the first model.
[0137] In some embodiments, the second determining module includes:
[0138] The first determining unit is used to determine the second model as the first target model when the prediction effect of the second model is better than that of the first model.
[0139] The second determining unit is used to determine the first model as the first target model when the prediction effect of the first model is better than that of the second model.
[0140] In some embodiments, the model determining device 300 further includes:
[0141] The second acquisition model is used to acquire the second intent corpus dataset;
[0142] The second training module is used to train an initial model based on the second intent corpus dataset to obtain the first model;
[0143] The first storage module is used to store the first model and its prediction results.
[0144] In some embodiments, the model determining device 300 further includes:
[0145] The first optimization module is used to optimize the first intent corpus dataset and the label information corresponding to each intent corpus data in the first intent corpus dataset;
[0146] The first training model is specifically used to train the first model based on the optimized first intent corpus dataset and the label information corresponding to each intent corpus data in the first intent corpus dataset.
[0147] In some embodiments, the second determining module includes:
[0148] The prompting unit is used to output a prompting message to prompt the user to perform further optimization when the prediction effect of the first model is better than that of the second model.
[0149] The training unit is used to train the first model using the optimized first intent corpus dataset and the label information corresponding to each intent corpus data in the first intent corpus dataset, to obtain the third model.
[0150] The determining unit is used to determine the first target model based on the prediction performance of the third model and the prediction performance of the first model.
[0151] In some embodiments, the model determining device 300 further includes:
[0152] The second storage module is used to store the first target model and the prediction effect of the first target model.
[0153] The prediction module is used to input the intent corpus data into the first target model to determine the prediction label corresponding to the intent corpus data when the intent corpus data is obtained.
[0154] In some embodiments, the model determining device 300 further includes:
[0155] The third acquisition module is used to acquire the third intent corpus dataset;
[0156] The third determining module is used to input the third intent corpus dataset into the first target model and determine the label information corresponding to each intent corpus data in the third intent corpus dataset;
[0157] The third training module is used to train the first target model based on the third intent corpus dataset and the label information corresponding to each intent corpus data in the third intent corpus dataset to obtain the second target model; and to determine the prediction effect of the second target model.
[0158] The fourth determining module is used to determine the third target model based on the prediction performance of the second target model and the prediction performance of the first target model.
[0159] It should be noted that, in the embodiments of this application, if the above-described method for determining the model is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of this application, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this application are not limited to any specific hardware and software combination.
[0160] Accordingly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the model determination method provided in the above embodiments.
[0161] This application provides an electronic device; Figure 4 This is a schematic diagram of the composition structure of the electronic device provided in the embodiments of this application, such as... Figure 4 As shown, the electronic device 500 includes: a processor 501, at least one communication bus 502, a user interface 503, at least one external communication interface 504, and a memory 505. The communication bus 502 is configured to enable communication between these components. The user interface 503 may include a display screen, and the external communication interface 504 may include standard wired and wireless interfaces. The processor 501 is configured to execute a model determination method program stored in the memory to implement the steps in the model determination method provided in the above embodiment.
[0162] The descriptions of the above embodiments of the electronic devices and storage media are similar to those of the above method embodiments, and have similar beneficial effects. For technical details not disclosed in the embodiments of the computer devices and storage media of this application, please refer to the descriptions of the method embodiments of this application for understanding.
[0163] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence numbers of the above-described processes do not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the above-described embodiments are merely descriptive and do not represent the superiority or inferiority of the embodiments.
[0164] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.
[0165] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.
[0166] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.
[0167] In addition, each functional unit in the various embodiments of this application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.
[0168] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.
[0169] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a controller to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROMs, magnetic disks, or optical disks.
[0170] The above description is merely an embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. A method for determining a model, characterized in that, include: Obtain the first intent corpus dataset; The first intent corpus dataset is input into a pre-established first model to determine the label information corresponding to each intent corpus data in the first intent corpus dataset; The first model was obtained by training an initial model based on the second intent corpus dataset; The first model is trained based on the first intent corpus dataset and the label information corresponding to each intent corpus data in the first intent corpus dataset to obtain the second model; and the prediction effect of the second model is determined. If the prediction performance of the second model is better than that of the first model, the second model is determined as the first target model. If the prediction performance of the first model is better than that of the second model, the first model is determined as the first target model.
2. The method according to claim 1, characterized in that, The method further includes: Obtain the second intent corpus dataset; The initial model is trained based on the second intent corpus dataset to obtain the first model; Save the first model and its prediction results.
3. The method according to claim 1, characterized in that, The method further includes: The label information corresponding to each intent corpus data in the first intent corpus dataset is optimized. The step of training the first model based on the first intent corpus dataset and the label information corresponding to each intent corpus data in the first intent corpus dataset includes: training the first model based on the optimized first intent corpus dataset and the label information corresponding to each intent corpus data in the first intent corpus dataset.
4. The method according to claim 3, further comprising: If the prediction performance of the first model is better than that of the second model, output a prompt message to encourage the user to optimize again; Having obtained the first intent corpus dataset after further optimization and the label information corresponding to each intent corpus data in the first intent corpus dataset, the first model is trained using the first intent corpus dataset after further optimization and the label information corresponding to each intent corpus data in the first intent corpus dataset to obtain the third model. The first target model is determined based on the prediction performance of the third model and the prediction performance of the first model.
5. The method according to claim 4, characterized in that, The method further includes: Save the first target model and its prediction results; When new intent corpus data is acquired, the intent corpus data is input into the first target model to determine the prediction label corresponding to the intent corpus data.
6. The method according to claim 5, characterized in that, The method further includes: Obtain the third-intention corpus dataset; The third intent corpus dataset is input into the first target model to determine the label information corresponding to each intent corpus data in the third intent corpus dataset; The first target model is trained based on the third intent corpus dataset and the label information corresponding to each intent corpus data in the third intent corpus dataset to obtain the second target model; and the prediction effect of the second target model is determined. The third target model is determined based on the prediction performance of the second target model and the prediction performance of the first target model.
7. A model determining device, characterized in that, include: The first acquisition module is used to acquire the first intent corpus dataset; The first determining module is used to input the first intent corpus dataset into a pre-established first model and determine the label information corresponding to each intent corpus data in the first intent corpus dataset; The first model was obtained by training an initial model based on the second intent corpus dataset; The first training model is used to train the first model based on the first intent corpus dataset and the label information corresponding to each intent corpus data in the first intent corpus dataset to obtain the second model; and to determine the prediction effect of the second model. The second determining module is used to determine the second model as the first target model when the prediction effect of the second model is better than that of the first model; and to determine the first model as the first target model when the prediction effect of the first model is better than that of the second model.
8. An electronic device, characterized in that, include: It includes a memory and a processor, wherein the memory stores a computer program that, when executed by the processor, performs the method for determining the model as described in any one of claims 1 to 6.
9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which is executed by a computer to perform the method for determining the model as described in any one of claims 1-6.