Optimization method and device of vertical domain question and answer model, electronic equipment and readable medium

Abstract

The embodiment of the invention provides a vertical domain question and answer model optimization method and device, electronic equipment and a readable medium, and the method comprises the steps: responding to first information input by a user, and calling an intention classification model for carrying out intention classification on the first information to classify the first information; classifying the determined vertical field according to the intention classification model, and calling a Lora model corresponding to the vertical field to answer the first information; the intention classification model and at least two vertical domain Lora models form a vertical domain question and answer model; the intention classification model and the Lora model of each vertical field are pre-trained models. According to the method, the problems that in the prior art, data updating iteration efficiency is low, and each field cannot achieve an expected effect are solved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an optimization method for a vertical field question-answering model, a training method for a vertical field question-answering model, a construction device for a vertical field question-answering model, an electronic device, and a computer-readable medium. Background Art

[0002] The generative language model provides a new idea for the question-answering model, allowing the language model to be fine-tuned on the data in the vertical field, making its output in the vertical field more credible and controllable. However, the catastrophic forgetting characteristics of the language model make it difficult for it to perform well in multiple fields at the same time. Data mixing and matching training is one way to solve this problem. When the question-answering model needs to answer questions in multiple fields, it is necessary to mix the data from different fields to fine-tune the semantic model. However, the data matching is a critical point at this time, and multiple attempts are required to achieve good results. However, if the data in one or several fields is updated and iterated, it is necessary to re-run the data matching experiment. The iteration efficiency of this method is very low, and it is difficult to ensure that each field achieves good results. Summary of the invention

[0003] An embodiment of the present invention provides an optimization method, device, electronic device and computer-readable storage medium for a vertical field question-answering model to solve the problem in the prior art that data update iteration efficiency is low and each vertical field cannot achieve the expected effect.

[0004] The embodiment of the present invention discloses a method for optimizing a vertical field question-answering model, including:

[0005] In response to first information input by a user, calling an intent classification model for classifying intent of the first information to classify the first information;

[0006] According to the vertical field determined by the intent classification model, the Lora model corresponding to the vertical field is called to answer the first information;

[0007] The intent classification model and at least two vertical field Lora models constitute a vertical field question-answering model;

[0008] The intent classification model and the Lora model in each vertical field are pre-trained models.

[0009] Optionally, the method further comprises:

[0010] Get all the pre-annotated data;

[0011] The full amount of data is used to train an intent classification model, so that the intent classification model can perform domain classification on the first information input into the question-answering model;

[0012] Obtain pre-labeled in-domain training data for each vertical field;

[0013] Use the training data in the field to train the Lora model of the vertical field to obtain the trained Lora model of the vertical field;

[0014] The LoRa models for all verticals are identical when not trained.

[0015] Optionally, the method further comprises:

[0016] If the intent classification model cannot classify the first information, a classification result without a vertical field is output;

[0017] Then the base model without domain limitation is called to respond to the first information;

[0018] The base model, intent classification model and Lora models in at least two vertical fields constitute a vertical field question-answering model.

[0019] Optionally, the method further comprises:

[0020] If the data in the specified vertical field is updated, the Lora model in the specified vertical field is trained using the updated training data in the specified vertical field to obtain the Lora model trained after the data is updated, and

[0021] The intent classification model is trained using the updated training data in the specified vertical field to obtain the intent classification model trained after the data is updated.

[0022] Optionally, the method further comprises:

[0023] Store conversation context information in real time within a specified time period;

[0024] Accordingly, calling an intent classification model for classifying the intent of the first information to classify the first information includes:

[0025] Invoking an intent classification model to classify a combination of the first information and context information associated with the first information within a specified time period;

[0026] The Lora model corresponding to the vertical field is called to respond to the first information, including:

[0027] The Lora model corresponding to the vertical field is called to answer the combination of the first information and the context information associated with the first information within a specified time period.

[0028] The embodiment of the present invention also discloses a training method for a vertical field question answering model, including:

[0029] Get all the pre-annotated data;

[0030] The full amount of data is used to train an intent classification model, so that the intent classification model can perform domain classification on the first information input into the question-answering model;

[0031] Obtain pre-labeled in-domain training data for each vertical field;

[0032] Use the training data in the field to train the Lora model of the vertical field to obtain the trained Lora model of the field;

[0033] The trained intent classification model and the trained Lora models in at least two vertical fields constitute the trained question-answering model.

[0034] Optionally, it also includes:

[0035] Use pre-labeled full data to train the base model;

[0036] The trained base model, the trained intent classification model, and the trained Lora models of at least two vertical fields constitute the trained question-answering model.

[0037] The embodiment of the present invention also discloses an optimization device for a vertical field question-answering model, including:

[0038] A domain classification unit, configured to respond to first information input by a user and call an intent classification model for classifying the intent of the first information to classify the first information;

[0039] An answering unit, configured to answer the first information by calling a Lora model corresponding to the vertical field determined by the intent classification model according to the vertical field classified by the intent classification model;

[0040] The intent classification model and at least two vertical field Lora models constitute a vertical field question-answering model;

[0041] The intent classification model and the Lora model in each vertical field are pre-trained models.

[0042] The embodiment of the present invention further discloses an electronic device, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0043] The memory is used to store computer programs;

[0044] The processor is used to implement the method described in the embodiment of the present invention when executing the program stored in the memory.

[0045] The embodiment of the present invention further discloses one or more computer-readable media on which instructions are stored. When executed by one or more processors, the processors are enabled to execute the method described in the embodiment of the present invention.

[0046] The embodiments of the present invention include the following advantages:

[0047] The question-answering model of the embodiment of the present invention can achieve good results in multiple vertical fields, and the model is updated quickly after data iteration, which improves the question-answering capabilities across multiple vertical fields and can be quickly iterated and upgraded.

[0048] In the embodiment of the present invention, a Lora model of each vertical field is pre-trained separately for the vertical field, so that the Lora model is more professional. During use, after the trained intent classification model is used for field classification, the Lora model trained in the vertical field can be selected for question answering, which generates highly professional and customized questions and answers. At the same time, they are independent of each other, and are convenient for maintenance and collaborative development.

[0049] Furthermore, the iteration of the question-answering model in the embodiment of the present invention is more convenient, and the data and Lora model of a single vertical field can be iterated independently without affecting other vertical fields, that is, fast updating is achieved. The Lora model in the embodiment of the present invention can be continuously optimized, and the overall performance can be greatly improved.

[0050] The question-answering model of the embodiment of the present invention can quickly add question-answering capabilities in new vertical fields, is easy to expand horizontally, and has strong vertical specialization. Each vertical field only needs to store the Lora model, which has low demand for video memory, saves hardware resources, and reduces duplicate development. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 It is a flowchart of the steps of an optimization method of a vertical field question-answering model provided in an embodiment of the present invention;

[0052] Figure 2 is a flowchart of the steps of a method for using a vertical field question-answering model provided in an embodiment of the present invention;

[0053] Figure 3 is a flowchart of the steps of a method for using a vertical field question-answering model provided in an embodiment of the present invention;

[0054] Figure 4 is a flowchart of the steps of a method for using a vertical field question-answering model provided in an embodiment of the present invention;

[0055] Figure 5It is a flowchart of the steps of a training method for a vertical field question answering model provided in an embodiment of the present invention;

[0056] Figure 6 It is a structural block diagram of an optimization device for a vertical field question-answering model provided in an embodiment of the present invention;

[0057] Figure 7 is a block diagram of an electronic device provided in an embodiment of the present invention;

[0058] Figure 8 is a schematic diagram of a computer-readable medium provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0059] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0060] Vertical field question answering models are widely used in multiple industries and can provide professional and accurate information and services. By combining knowledge graphs, natural language processing and machine learning technologies, vertical field question answering models can effectively understand and answer questions in specific fields.

[0061] The current vertical field question-and-answer model can be a system specially trained for knowledge and data in a specific field. It is professional and accurate, and can better understand and answer questions in the field. It is applied to the medical and health field. Patients can ask about symptoms, disease prevention and treatment methods through question-and-answer methods. For financial management, investors can ask about information and suggestions on investment products such as stocks, funds, and bonds. It can also be used for legal consultation to facilitate the search for answers to professional cases such as contract law and intellectual property law. It can also be used in the field of education and counseling.

[0062] Reference Figure 1 , shows a flowchart of the steps of an optimization method for a vertical field question-answering model provided in an embodiment of the present invention, which may specifically include the following steps:

[0063] 101. In response to first information input by a user, calling an intent classification model for classifying intent of the first information to classify the first information;

[0064] 102. According to the vertical field determined by the intent classification model, call the Lora model corresponding to the vertical field to answer the first information;

[0065] The intent classification model and at least two vertical field Lora models constitute a vertical field question-answering model;

[0066] The intent classification model and the Lora model in each vertical field are pre-trained models.

[0067] In this embodiment, a Lora model for each vertical field is pre-trained separately, making the Lora model more professional. During use, after using the trained intent classification model for field classification, the Lora model trained in the vertical field can be selected for question answering, which generates highly professional and customized questions and answers. At the same time, the Lora models in different vertical fields are independent of each other, which is convenient for maintenance and collaborative development.

[0068] It should be noted that the Lora (Low-Rank Adaptation) model used in this embodiment is actually a model for fine-tuning the question-answering model, which is particularly useful when resources are limited. The core idea of ​​LoRA is to fine-tune the model parameters by introducing a low-rank matrix on the basis of the question-answering model, rather than directly updating all the parameters of the entire question-answering model. This method not only reduces the computing resources required for training, but also improves the generalization ability of the model while maintaining model performance. Since the number of parameters of the question-answering model is large and the training cost is too high, the LoRA model adopts a method of only training low-rank matrices, and injecting the parameters of the LoRA model into the question-answering model when using it, thereby changing the generation style of the question-answering model so that it has a better output effect in a specific vertical field.

[0069] Before the question-answering model is used, it is necessary to train the question-answering model. Therefore, the above method also includes the following steps not shown in the figure:

[0070] 100a. Obtain a full amount of pre-labeled data; use the full amount of data to train an intent classification model so that the intent classification model can perform domain classification on the first information input into the question-answering model.

[0071] 100b. Obtain pre-labeled in-domain training data for each vertical field; use the in-domain training data to train a Lora model for the vertical field, and obtain a trained Lora model for the vertical field.

[0072] In this embodiment, multiple vertical field question and answer data sets can be pre-collected and annotated to form field training data in each vertical field. The annotation of each collected question and answer data set includes: annotation of questions and answers in each vertical field, and also includes the intent classification of each question, i.e., field classification.

[0073] In this embodiment, the data distribution characteristics of different fields are analyzed to better train the model parameters of the Lora models in each subsequent vertical field. That is, the Lora models in all vertical fields are the same before training. The model parameters of the Lora models in each vertical field are trained through the information of the data distribution characteristics of the training data in the corresponding vertical field, so that the question-answering model as a whole can perform style transfer in the corresponding vertical field, thereby improving the output effect of the question-answering model when dealing with question-answering tasks in the corresponding vertical field, and at the same time solving the problem that the existing question-answering model cannot achieve the expected effect in every vertical field.

[0074] The trained intent classification model and the trained Lora models in at least two vertical fields constitute the trained question-answering model.

[0075] When a new vertical field is added to the training, you only need to add a new Lora model for the vertical field, add the data of the new vertical field to the intent classification model, and iterate and update the intent classification model.

[0076] The question-answering model of this embodiment can quickly add question-answering capabilities in new vertical fields, is easy to expand horizontally, and has strong vertical specialization. Each field only needs to store the Lora model, which has a small demand for video memory, saves hardware resources, and reduces duplicate development.

[0077] Of course, for data updates in one or more vertical fields, the Lora model of the updated vertical field can be trained separately at this time; for example, if the data in a specified vertical field is updated, the Lora model of the vertical field is trained using the updated training data in the specified vertical field to obtain the Lora model trained after the vertical field data is updated, and the intent classification model is trained using the updated training data in the specified vertical field to obtain the intent classification model trained after the data in one or more vertical fields is updated.

[0078] That is, the iteration of the question-answering model is more convenient, and the data and Lora model of a single vertical field can be iterated independently without affecting other fields, that is, fast updating is achieved. The Lora model in the embodiment of the present invention can be continuously optimized, and the overall performance can be greatly improved.

[0079] like Figures 2 to 4 As shown, this embodiment provides an optimization method for a vertical field question-answering model, which is divided into two process steps according to the specific component modules of the question model.

[0080] First, if the question-answering model consists of an intent classification model and at least two vertical domain Lora models, the question-answering model responds to the user's question and answer process specifically includes the following steps:

[0081] 101. In response to first information input by a user, call an intent classification model for classifying the intent of the first information to classify the first information; the intent classification model and the Lora model of each vertical field are pre-trained models.

[0082] 102. According to the vertical field determined by the intent classification model, call the Lora model corresponding to the vertical field to respond to the first information.

[0083] The intent classification model and the Lora model in each vertical field are pre-trained models.

[0084] Second, if the question-answering model consists of a base model, an intent classification model, and at least two vertical field Lora models, the question-answering model responds to the user's question and answer process specifically includes the following steps:

[0085] 101. In response to first information input by a user, call an intent classification model for classifying the intent of the first information to classify the first information.

[0086] If the intent classification model is unable to classify the first information, a classification result without a vertical field is output.

[0087] The intent classification model and the Lora model in each vertical field are pre-trained models.

[0088] 102. According to the vertical field determined by the intent classification model, call the Lora model corresponding to the vertical field to respond to the first information.

[0089] If the intent classification model is unable to classify the first information and outputs a classification result without a vertical field, then the base model without field limitation is called to respond to the first information.

[0090] The above-mentioned base model can be a Lora model with all model parameters set to 0, or a model trained on a large amount of text data. Since the base models have been trained with a large amount of data, they often have strong generalization capabilities, that is, they can handle unseen data or tasks well without additional training. Therefore, the first information for different vertical fields will have a relatively balanced output effect.

[0091] Retaining a general domain base model outside of all vertical fields, or a Lora model with all zeros, can maintain the general question-answering capability of the question-answering model. When the intent classification model determines that the user's question is not a question in each vertical field, the general model can be used to answer the user's question, solving the catastrophic forgetting problem of the model.

[0092] It should be noted that the intent classification model in step 101 is used to classify the vertical field to which the first information input by the user belongs. Specifically, the intent classification model is used to determine the vertical field to which the question raised by the user or the content input by the user belongs based on the question raised by the user or the content input (i.e., the first information), so that step 102 calls the corresponding Lora model to answer according to the vertical field.

[0093] The intent classification model can specifically be a convolutional neural network model (CNN), a recurrent neural network model (RNN), a long short-term memory network model (LSTM), a Transformer, a T5 (Text-to-Text Transfer Transformer) model, etc. These models can automatically extract features from the original text and perform well in processing sequence data. Preferably, in order to adapt to application scenarios with limited resources, the intent classification model can be a lightweight model such as a FastText model or an optimized convolutional neural network model (CNN) or a long short-term memory network model (LSTM).

[0094] Preferably, the intent classification model is a FastText model. The FastText model is a shallow neural network model based on the Bag of Words (BoW) model, but the effect of the model is improved by introducing word embeddings and subword units. Word embedding means that the FastText model uses pre-trained word embeddings to initialize the model, which helps the model better capture the semantic information of the vocabulary. If there is no pre-trained word embedding, the word embedding can also be trained from scratch. Subword units mean that each word can be decomposed into smaller units (such as character n-grams). This method helps to deal with out-of-vocabulary words, that is, words that have not appeared in the training data. Through subword units, the model can use partial information of the word to predict the meaning of the entire word. By introducing word embeddings and subword units, the FastText model can quickly build and train text classification models, which is particularly suitable for processing large-scale data sets and resource-constrained environments. By introducing subword units, it can effectively deal with out-of-vocabulary problems and improve its robustness and generalization ability as an intent classification model.

[0095] In addition, in practical applications, in order to provide a more accurate answer, the method further includes: storing the conversation context information within a specified time period in real time.

[0096] The actual conversation will switch between different LoRa models, and additional design is required to maintain the conversation history. In this embodiment, context information can be stored to enable the model to better respond to questions.

[0097] Specifically, calling the intent classification model for classifying the intent of the first information to classify the first information in step 101 includes:

[0098] The intent classification model is called to classify the combination of the first information and the context information associated with the first information within a specified time period.

[0099] Accordingly, calling the Lora model corresponding to the vertical field in the aforementioned step 102 to respond to the first information includes:

[0100] The Lora model corresponding to the vertical field is called to answer the combination of the first information and the context information associated with the first information within a specified time period.

[0101] The intent classification model in this embodiment can be used in conjunction with the conversation history to divide the domain, that is, the context can be associated to achieve smooth switching of domains, achieve conversation coherence, and provide efficient and more professional answers. At the same time, resources can be reused to reduce repeated development.

[0102] like Figures 2 to 4 As shown, in this embodiment, provident fund data, medical insurance data, and social security data can be annotated in advance, and the annotations include question and answer annotations and field annotations.

[0103] The labeled provident fund data is used to train the Lora model of the provident fund, the labeled medical insurance data is used to train the Lora model of the medical insurance, and the labeled social security data is used to train the Lora model of the social security.

[0104] Use all labeled provident fund data, medical insurance data, and social security data to train the intent classification model.

[0105] During the usage stage, the system can receive questions input by users, use the trained intent classification model to determine the field of the question, i.e., medical insurance, social security or provident fund, and then select the Lora model of the determined field to answer the above questions.

[0106] Furthermore, assuming that the social security data needs to be supplemented and iterated, and more data is added to the question-answering system, at this time, the updated social security data is labeled, the social security Lora model is retrained, and the intent classification model is retrained using all the data, and nothing else needs to be changed.

[0107] That is, independent iteration of data and models in a single domain is achieved without affecting other domains, thus enabling rapid updates.

[0108] Reference Figure 5 , shows a flowchart of the steps of a training method for a vertical field question answering model provided in an embodiment of the present invention, which may specifically include the following steps:

[0109] 501. Obtain a full amount of pre-labeled data; use the full amount of data to train an intent classification model so that the intent classification model can perform domain classification on the first information input into the question-answering model.

[0110] 502. Obtain pre-labeled in-domain training data for each vertical field; use the in-domain training data to train the Lora model of the field to obtain the trained Lora model of the field; the trained intent classification model and the trained Lora models of at least two vertical fields constitute the trained question-answering model.

[0111] In this embodiment, the Lora model of each vertical field is pre-trained separately in the field, so that the Lora model is more professional, and the intent classification model is trained at the same time. Therefore, it can be easily expanded, and the training is independent and professional.

[0112] During use, after using the trained intent classification model for field classification, the Lora model trained in that field can be selected for question answering. It produces highly professional and customized questions and answers, which are independent of each other, making maintenance and collaborative development convenient.

[0113] Furthermore, the above method also includes the following steps not shown in the figure:

[0114] The pre-labeled full data is used to train the base model; the trained base model, the trained intent classification model and the trained Lora models of at least two vertical fields constitute the trained question-answering model.

[0115] In order to maintain the general question-answering capability of the question-answering model, when the intent classification model determines that the user's question is not a question in a vertical field, a general base model is used to answer the user's question to solve the catastrophic forgetting problem of the model. At the same time, the user experience is improved.

[0116] After the trained question-and-answer model is obtained based on the above training steps, it can be used for user questions and answers. When the domain data of one or more vertical fields is updated, the corresponding Lora model in the question-and-answer model can be retrained according to the updated domain data to update the Lora model according to the updated domain data. It should be emphasized that no matter which vertical field the Lora model corresponding to is retrained, the updated domain data needs to be added to the full data at the same time, and the intent classification field needs to be retrained so that the intent classification model can be kept updated synchronously.

[0117] Specifically, the above-mentioned step of retraining according to the updated domain data includes:

[0118] If the data in the specified vertical field is updated, the updated training data in the specified vertical field is used to train the Lora model in the vertical field to obtain the Lora model trained after the data is updated, and the updated training data in the specified vertical field is used to train the intent classification model to obtain the intent classification model trained after the data is updated.

[0119] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0120] Reference Figure 6 , shows a structural block diagram of an optimization device for a vertical field question-answering model provided in an embodiment of the present invention, which may specifically include the following units:

[0121] The domain classification unit is used to respond to the first information input by the user and call the intent classification model for classifying the intent of the first information to classify the first information.

[0122] Specifically, the domain classification unit stores the conversation context information within the specified time period in real time; calls the intention classification model to classify the first information and the combination of context information associated with the first information within the specified time period;

[0123] The Lora model corresponding to the vertical field is called to respond to the first information, including:

[0124] The Lora model corresponding to the vertical field is called to answer the combination of the first information and the context information associated with the first information within a specified time period.

[0125] The answering unit is used to answer the first information by calling the Lora model corresponding to the vertical field determined by the intention classification model.

[0126] Furthermore, the answering unit calls the Lora model corresponding to the vertical field to answer the combination of the first information and the context information associated with the first information within the specified time period. The context information is the conversation context information within the specified time period stored in real time by the field classification unit.

[0127] The intent classification model and at least two vertical field Lora models constitute a vertical field question-answering model.

[0128] The intent classification model and the Lora model in each vertical field are pre-trained models.

[0129] The intent classification model training unit is used to obtain the full amount of pre-labeled data; the full amount of data is used to train the intent classification model so that the intent classification model can perform domain classification on the first information input into the question-answering model.

[0130] The Lora model training unit is used to obtain pre-labeled in-domain training data for each vertical field; the in-domain training data is used to train the Lora model of the vertical field to obtain the trained Lora model of the vertical field; the Lora models of all vertical fields are the same before training.

[0131] The general unit is used to call a base model without domain limitation to answer the first information when the intent classification model cannot classify the first information and outputs a classification result without vertical domain.

[0132] An updating unit is used to train the Lora model of a specified vertical field using the updated training data in the specified vertical field when the data in the specified vertical field is updated, so as to obtain the Lora model trained after the data is updated, and to train the intent classification model using the updated training data in the specified vertical field, so as to obtain the intent classification model trained after the data is updated.

[0133] The optimization device for the vertical field question and answer model provided in this embodiment pre-trains the Lora model of each vertical field separately, so that the Lora model is more professional. In use, after using the trained intent classification model to perform field classification, the Lora model trained in the vertical field can be selected for question answering, which generates highly professional and customized questions and answers. At the same time, the Lora models in different vertical fields are independent of each other, which is convenient for maintenance and collaborative development.

[0134] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0135] In addition, an embodiment of the present invention further provides an electronic device, such as Figure 7 As shown, it includes a processor 1301, a communication interface 1302, a memory 1303 and a communication bus 1304, wherein the processor 1301, the communication interface 1302, and the memory 1303 communicate with each other through the communication bus 1304.

[0136] Memory 1303, used for storing computer programs;

[0137] The processor 1301 is used to implement the following steps when executing the program stored in the memory 1303:

[0138] In response to first information input by a user, calling an intent classification model for classifying intent of the first information to classify the first information;

[0139] According to the vertical field determined by the intent classification model, the Lora model corresponding to the vertical field is called to answer the first information;

[0140] The intent classification model and at least two vertical field Lora models constitute a vertical field question-answering model;

[0141] The intent classification model and the Lora model in each vertical field are pre-trained models.

[0142] And, obtain the full amount of pre-labeled data;

[0143] The full amount of data is used to train an intent classification model, so that the intent classification model can perform domain classification on the first information input into the question-answering model;

[0144] Obtain pre-labeled in-domain training data for each vertical field;

[0145] The training data in the field is used to train the Lora model in the field to obtain the trained Lora model in the field.

[0146] The LoRa models for all verticals are identical when not trained.

[0147] If the intent classification model cannot classify the first information, a classification result without a vertical field is output;

[0148] Then the base model without domain limitation is called to respond to the first information;

[0149] The base model, intent classification model and Lora models in at least two vertical fields constitute a vertical field question-answering model.

[0150] If the data in the specified vertical field is updated, the Lora model in the specified vertical field is trained using the updated training data in the specified vertical field to obtain the Lora model trained after the data is updated, and

[0151] The intent classification model is trained using the updated training data in the specified vertical field to obtain the intent classification model trained after the data is updated.

[0152] Store conversation context information in real time within a specified time period;

[0153] Accordingly, calling an intent classification model for classifying the intent of the first information to classify the first information includes:

[0154] Invoking an intent classification model to classify a combination of the first information and context information associated with the first information within a specified time period;

[0155] The Lora model corresponding to the vertical field is called to respond to the first information, including:

[0156] The Lora model corresponding to the vertical field is called to answer the combination of the first information and the context information associated with the first information within a specified time period.

[0157] The electronic device provided in this embodiment, when executing the program stored in the memory 1303, pre-trains the Lora model of each vertical field separately, so that the Lora model is more professional. During use, after using the trained intent classification model to perform field classification, the Lora model trained in the vertical field can be selected for question answering, which generates highly professional and customized questions and answers. At the same time, the Lora models in different vertical fields are independent of each other, which is convenient for maintenance and collaborative development.

[0158] The communication bus mentioned in the above terminal can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0159] The communication interface is used for communication between the above terminal and other devices.

[0160] The memory may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0161] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0162] like Figure 8 As shown, in another embodiment provided by the present invention, a computer-readable storage medium 1401 is also provided, in which instructions are stored. When the computer-readable storage medium 1401 is run on a computer, the computer executes the optimization method or training method of the vertical field question and answer model described in the above embodiment.

[0163] In another embodiment provided by the present invention, a computer program product comprising instructions is also provided, which, when executed on a computer, enables the computer to execute the optimization method or training method of the vertical field question-answering model described in the above embodiment.

[0164] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk (SSD)), etc.

[0165] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0166] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0167] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A method for optimizing a vertical field question answering model, characterized in that: include: In response to first information input by a user, calling an intent classification model for classifying intent of the first information to classify the first information; According to the vertical field determined by the intent classification model, the Lora model corresponding to the vertical field is called to answer the first information; The intent classification model and at least two vertical field Lora models constitute a vertical field question-answering model; The intent classification model and the Lora model in each vertical field are pre-trained models.

2. The method according to claim 1, characterized in that The method further comprises: Get all the pre-annotated data; The full amount of data is used to train an intent classification model, so that the intent classification model can perform domain classification on the first information input into the question-answering model; Obtain pre-labeled in-domain training data for each vertical field; Use the training data in the field to train the Lora model of the vertical field to obtain the trained Lora model of the vertical field; The LoRa models for all verticals are identical when not trained.

3. The method according to claim 1, characterized in that The method further comprises: If the intent classification model cannot classify the first information, a classification result without a vertical field is output; Then the base model without domain limitation is called to respond to the first information; The base model, intent classification model and Lora models in at least two vertical fields constitute a vertical field question-answering model.

4. The method according to claim 1, characterized in that: The method further comprises: If the data in the specified vertical field is updated, the Lora model in the specified vertical field is trained using the updated training data in the specified vertical field to obtain the Lora model trained after the data is updated, and The intent classification model is trained using the updated training data in the specified vertical field to obtain the intent classification model trained after the data is updated.

5. The method according to claim 1, characterized in that The method further comprises: Store conversation context information in real time within a specified time period; Accordingly, calling an intent classification model for classifying the intent of the first information to classify the first information includes: Invoking an intent classification model to classify a combination of the first information and context information associated with the first information within a specified time period; The Lora model corresponding to the vertical field is called to respond to the first information, including: The Lora model corresponding to the vertical field is called to answer the combination of the first information and the context information associated with the first information within a specified time period.

6. A training method for a vertical field question answering model, characterized in that: include: Get all the pre-annotated data; The full amount of data is used to train an intent classification model, so that the intent classification model can perform domain classification on the first information input into the question-answering model; Obtain pre-labeled in-domain training data for each vertical field; Use the training data in the field to train the Lora model of the vertical field to obtain the trained Lora model of the vertical field; The trained intent classification model and the trained Lora models in at least two vertical fields constitute the trained question-answering model.

7. The method according to claim 6, characterized in that Also includes: Use pre-labeled full data to train the base model; The trained base model, the trained intent classification model, and the trained Lora models of at least two vertical fields constitute the trained question-answering model.

8. An optimization device for a vertical field question-answering model, characterized in that: include: A domain classification unit, configured to respond to first information input by a user and call an intent classification model for classifying the intent of the first information to classify the first information; An answering unit, configured to answer the first information by calling a Lora model corresponding to the vertical field determined by the intent classification model according to the vertical field classified by the intent classification model; The intent classification model and at least two vertical field Lora models constitute a vertical field question-answering model; The intent classification model and the Lora model in each vertical field are pre-trained models.

9. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; The memory is used to store computer programs; The processor is used to implement the method according to any one of claims 1 to 5 when executing the program stored in the memory.

10. A computer-readable medium having instructions stored thereon, which, when executed by one or more processors, cause the processors to perform the method according to any one of claims 1 to 5.

Images