Method and device for implementing intelligent question and answer service based on large model

Abstract

The application discloses a method and device for realizing intelligent question and answer service based on a large model, the method comprises the following steps: acquiring real-time text data; inputting the real-time text data into a trained intelligent question and answer network model to perform corresponding answer prediction, and outputting multiple candidate answers according to the answer prediction result; acquiring scores corresponding to the multiple candidate answers, and outputting an optimal answer in the multiple candidate answers according to the sorting result of the scores. The application adds a personalized capability to the large model, solves the problem that the large model is seriously illusory and difficult to achieve ideal effects of problems by relying on knowledge base retrieval, and can effectively and accurately predict and output answers to problems in multiple fields.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent question answering, and in particular to a method and device for realizing intelligent question answering service based on a large model. BACKGROUND

[0002] The existing individualization of various large models mainly provides a specified role for a character when inputting a dialogue, and mainly relies on knowledge base retrieval to limit the range of questions answered by the large model and analyze the questions from the specified knowledge base according to the strong understanding ability of the large model.

[0003] The individualization of the large model is not intelligent enough, and a specified role can only be realized by manually inputting role characteristics. The hallucination of the large model is still relatively serious, and it is difficult to achieve the ideal effect by relying on knowledge base retrieval. Even in the specified knowledge base, the large model may still produce a large number of hallucinations and cannot well match the answers to the user's questions. SUMMARY

[0004] The present application aims to at least solve one of the technical problems in the related art to some extent.

[0005] To this end, the present application proposes a method for realizing intelligent question answering service based on a large model, which aims to increase the individualization ability of the large model, improve the accuracy of the large model in the field of question answering, and suppress the hallucination of the large model, so that the answers to the user's questions can be effectively matched.

[0006] Another object of the present application is to propose a device for realizing intelligent question answering service based on a large model.

[0007] To achieve the above-mentioned purpose, the present application proposes, in one aspect, a method for realizing intelligent question answering service based on a large model, comprising:

[0008] obtaining real-time text data;

[0009] inputting the real-time text data into a trained intelligent question answering network model for corresponding answer prediction, and outputting a plurality of candidate answers according to the answer prediction result;

[0010] obtaining scores corresponding to the plurality of candidate answers, and outputting an optimal answer from the plurality of candidate answers according to the sorting result of the scores.

[0011] The method for realizing intelligent question answering service based on a large model of the present application embodiment can further have the following additional technical features:

[0012] In one embodiment of the present application, before inputting the text data into the trained intelligent question answering network model for corresponding answer prediction, the method further comprises training the intelligent question answering network model, comprising:

[0013] Obtaining sample data of the real-time text data; the sample data at least includes specific service field data and personalized data;

[0014] Inputting the sample data into an intelligent question and answer network model to perform model parameter training, and obtaining the trained intelligent question and answer network model based on the trained model parameters.

[0015] In an embodiment of the present application, before the sample data is input into the intelligent question and answer network model to perform model parameter training, the method further comprises: preprocessing the sample data, including:

[0016] Cleaning the sample data to obtain cleaned data; wherein cleaning the sample data includes removing text containing illegal characters, duplicates, containing sensitive information and causing user privacy leakage;

[0017] Segmenting the cleaned specific service field data, and storing the segmented data in a block manner to obtain a first data storage result; and,

[0018] Segmenting the discrete and categorical data in the cleaned personalized data, and normalizing the numerical data, to store the segmented personalized data and the normalized data in a block manner to obtain a second data storage result.

[0019] In an embodiment of the present application, the intelligent question and answer network model includes a first personalized model, a second personalized model, a joint model, an adversarial network model, a character vector jittering adversarial learning model, a layer-by-layer parameter jittering adversarial model and a reinforcement learning model.

[0020] In an embodiment of the present application, inputting the sample data into the intelligent question and answer network model to perform model parameter training, and obtaining the trained intelligent question and answer network model based on the trained model parameters, includes:

[0021] Inputting the data in the first data storage result and the second data storage result into the first personalized model to perform model training to obtain first model parameters.

[0022] In an embodiment of the present application, the second personalized model includes a first generator and a first discriminator; inputting the sample data into the intelligent question and answer network model to perform model parameter training, and obtaining the trained intelligent question and answer network model based on the trained model parameters, further includes:

[0023] Inputting real user data and corresponding text data in the sample data into the first generator to perform training to output pseudo text;

[0024] inputting the pseudo text and the real user data into the first discriminator for training to obtain a first discrimination result of the pseudo text and the real user data;

[0025] obtaining second model parameters of the second personalized model based on the first discrimination result and the first model parameters.

[0026] In an embodiment of the present application, the sample data is input into an intelligent question and answer network model for model parameter training, and the trained intelligent question and answer network model is obtained based on the trained model parameters, and further comprising:

[0027] obtaining a question and answer data pair of the specific service field data;

[0028] inputting the question and answer data pair into the joint model for training to obtain a model output result; wherein the model output result comprises named entity recognition data, intent recognition data and knowledge graph data of a relationship triple;

[0029] training third model parameters based on the second model parameters and the model output result.

[0030] In an embodiment of the present application, the adversarial network model comprises a second generator and a second discriminator, the sample data is input into an intelligent question and answer network model for model parameter training, and the trained intelligent question and answer network model is obtained based on the trained model parameters, and further comprising:

[0031] obtaining a character vector of the question and answer data pair of the specific service field data; wherein the question and answer data pair comprises at least a first query text and a first answer text;

[0032] concatenating the character vector with personalized data to obtain personalized features, and sending the personalized features and the first query text to the second generator to generate a pseudo answer text;

[0033] inputting the pseudo answer text and the first answer text into the second discriminator for discrimination to output a second discrimination result;

[0034] training the adversarial network model based on the second discrimination result and the third model parameters to output fourth model parameters.

[0035] In an embodiment of the present application, the sample data is input into an intelligent question and answer network model for model parameter training, and the trained intelligent question and answer network model is obtained based on the trained model parameters, and further comprising:

[0036] obtaining a second query text and a second answer text of the question and answer data pair;

[0037] inputting the second query text and the second answer text into the character vector jittered adversarial learning model to perform a slight jitter on the forward propagation of embedding of each token, and not performing jitter on the embedding when back propagating to calculate a gradient, so as to train the character vector jittered adversarial learning model;

[0038] based on the fourth model parameter and the trained character vector jittered adversarial learning model, outputting a fifth model parameter.

[0039] In an embodiment of the present application, the sample data is input into an intelligent question and answer network model for model parameter training, and the trained intelligent question and answer network model is obtained based on the trained model parameter, and further comprises:

[0040] training a semantic consistency detection model based on the labeled data of the question and answer data pair and a preset public data set; wherein the question and answer data pair further comprises a third query text and a second answer text;

[0041] obtaining continuous input texts in a user conversation process based on a user log, and detecting a sentence pair with semantic consistency of the continuous input texts by using the semantic consistency detection model;

[0042] filtering out data with similar literal meanings in the sentence pair with semantic consistency to obtain filtered data, and inputting the filtered data, the third query text and the third answer text into the semantic consistency detection model for training to obtain a layer-by-layer parameter jittered adversarial model, so as to output a sixth model parameter based on the fifth model parameter and the layer-by-layer parameter jittered adversarial model.

[0043] In an embodiment of the present application, the sample data is input into an intelligent question and answer network model for model parameter training, and the trained intelligent question and answer network model is obtained based on the trained model parameter, and further comprises:

[0044] inputting the question and answer data pair into a reinforcement learning model for reinforcement learning training, and outputting a plurality of predicted answers;

[0045] judging and scoring the plurality of predicted answers to obtain an optimal question and answer according to a scoring result;

[0046] optimizing parameters of the reinforcement learning model based on the optimal question and answer and the sixth model parameter to obtain a seventh model parameter, and obtaining a trained intelligent question and answer network model based on the seventh model parameter.

[0047] To achieve the above purpose, another aspect of the present application provides a device for implementing intelligent question and answer services based on a large model, comprising:

[0048] a data acquisition module configured to acquire real-time text data;

[0049] a prediction output module configured to input the real-time text data into a trained intelligent question and answer network model to perform corresponding answer prediction, and output a plurality of candidate answers according to an answer prediction result;

[0050] an optimal selection module configured to acquire scores corresponding to the plurality of candidate answers, and output an optimal answer from the plurality of candidate answers according to a ranking result of the scores.

[0051] The method and device for implementing intelligent question and answer services based on a large model according to the embodiments of the present application add personalized capabilities to the large model, improve the accuracy of the large model in the question and answer field, and inhibit hallucinations of the large model, so that the problem and answer of the user can be effectively matched.

[0052] Additional aspects and advantages of the present application will be in part apparent and in part pointed out hereinafter. BRIEF DESCRIPTION OF DRAWINGS

[0053] The above and / or additional aspects and advantages of the present application will become apparent and be readily understood from the following description, taken in conjunction with the accompanying drawings, in which:

[0054] Figure 1 is a flowchart of a method for implementing intelligent question and answer services based on a large model according to an embodiment of the present application;

[0055] Figure 2 is a flowchart of training of an intelligent question and answer network model according to an embodiment of the present application;

[0056] Figure 3 is a schematic diagram of reinforcing the capabilities of a large model according to an embodiment of the present application;

[0057] Figure 4 is a schematic diagram of giving a large model a primary personalized capability according to an embodiment of the present application;

[0058] Figure 5 is a schematic diagram of introducing independent named entity recognition, knowledge graph, and intent recognition according to an embodiment of the present application;

[0059] Figure 6 is a schematic diagram of introducing hierarchical adversarial learning according to an embodiment of the present application;

[0060] Figure 7 is a schematic diagram of adversarial learning of input character vector jittering according to an embodiment of the present application;

[0061] Figure 8 is a device structure diagram for implementing intelligent question and answer services based on a large model according to an embodiment of the present application. DETAILED DESCRIPTION

[0062] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0063] In order for those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below in combination with the accompanying drawings of the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should belong to the scope of protection of the present application.

[0064] The method and device for realizing intelligent question and answer service based on large model according to the embodiments of the present application will be described below with reference to the accompanying drawings.

[0065] Figure 1 is a flowchart of the method for realizing intelligent question and answer service based on large model according to the embodiments of the present application.

[0066] As shown in Figure 1 , the method includes but is not limited to the following steps:

[0067] S1, acquiring real-time text data;

[0068] S2, inputting the real-time text data into the trained intelligent question and answer network model for corresponding answer prediction, and outputting a plurality of candidate answers according to the answer prediction result;

[0069] S3, acquiring scores corresponding to the plurality of candidate answers, and outputting the optimal answer in the plurality of candidate answers according to the sorting result of the scores.

[0070] Further, before step S2, sample data of the real-time text data is acquired; the sample data at least includes specific service field data and personalized data; the sample data is input into the intelligent question and answer network model for model parameter training, and the trained intelligent question and answer network model is obtained based on the trained model parameters.

[0071] Further, before step S2, the sample data is preprocessed, including: cleaning the sample data to obtain cleaned data; wherein the sample data is cleaned, including removing illegal characters, repetitions, sensitive information, and text that may lead to user privacy leakage; performing word segmentation on the cleaned data in the specific service field, and storing the segmented data in a block manner to obtain a first data storage result; and performing word segmentation on discrete and categorical data in the cleaned personalized data, and performing normalization on numerical data, to store the word- segmented personalized data and the normalized data in a block manner to obtain a second data storage result.

[0072] Further, the intelligent question and answer network model includes a first personalized model, a second personalized model, a joint model, an adversarial network model, a character vector jittering adversarial learning model, a layer-by-layer parameter jittering adversarial model, and a reinforcement learning model.

[0073] Further, the data in the first data storage result and the second data storage result is input to the first personalized model for model training to obtain first model parameters.

[0074] Further, the second personalized model includes a first generator and a first discriminator; real user data and corresponding text data in the sample data are input to the first generator for training to output pseudo-text; the pseudo-text and the real user data are input to the first discriminator for training to obtain a first discrimination result of the pseudo-text and the real user data; and second model parameters of the second personalized model are obtained based on the first discrimination result and the first model parameters.

[0075] Further, a question and answer data pair of the specific service field data is obtained; the question and answer data pair is input to the joint model for training to obtain a model output result; wherein the model output result includes named entity recognition data, intent recognition data, and knowledge graph data of a relationship triple; and third model parameters are obtained based on the second model parameters and the model output result.

[0076] Further, the adversarial network model includes a second generator and a second discriminator, and a character vector of a question and answer data pair of the specific service field data is obtained; wherein the question and answer data pair includes at least a first query text and a first answer text; the character vector and the personalized data are spliced to obtain personalized features, and the personalized features and the first query text are sent to the second generator to generate pseudo-answer text; the pseudo-answer text and the first answer text are input to the second discriminator for discrimination to output a second discrimination result; and the adversarial network model is trained based on the second discrimination result and the third model parameters to output fourth model parameters.

[0077] Further, the second query text and the second answer text of the question and answer data pair are acquired, the second query text and the second answer text are input into the character vector jittering adversarial learning model, forward propagation after embedding of each token is slightly jittered, and the character vector jittering adversarial learning model is trained by not jittering embedding when back propagation is performed to calculate a gradient; and the fifth model parameter is output based on the fourth model parameter and the trained character vector jittering adversarial learning model.

[0078] Further, the semantic consistency detection model is trained based on the labeled data of the question and answer data pair and a preset public data set; the continuous input text in a user conversation process is obtained based on a user log, and a sentence pair with semantic consistency of the continuous input text is detected by using the semantic consistency detection model; data with similar literal meanings in the sentence pair with semantic consistency is screened to obtain screened data, and the screened data, the third query text and the third answer text are input into the semantic consistency detection model for training to obtain a layer-by-layer parameter jittering adversarial model, and the sixth model parameter is output based on the fifth model parameter and the layer-by-layer parameter jittering adversarial model.

[0079] Further, the question and answer data pair is input into the reinforcement learning model for reinforcement learning training, and a plurality of predicted answers are output; the plurality of predicted answers are judged and scored to obtain an optimal question and answer according to a scoring result; the seventh model parameter is obtained by optimizing parameters of the reinforcement learning model based on the optimal question and answer and the sixth model parameter, and the trained intelligent question and answer network model is obtained based on the seventh model parameter.

[0080] Therefore, the method for implementing intelligent question and answer services based on a large model provided by the application acquires real-time text data, inputs the real-time text data into the trained intelligent question and answer network model for corresponding answer prediction, and outputs a plurality of candidate answers according to an answer prediction result; scores corresponding to the plurality of candidate answers are acquired, and an optimal answer in the plurality of candidate answers is output according to a sorting result of the scores.

[0081] Figure 2 is a flowchart of training of the intelligent question and answer network model of the embodiment of the application. As shown in Figure 2 It can be understood that the intelligent question and answer network model of the application can be divided into seven modules:

[0082] It can be understood that the present application realizes the personalization of the model on the basis of the general large model framework, and achieves the maximum inhibition of model hallucination. Model training is a secondary optimization based on a public large model. First, a personalization processing module is added at the model input layer. The input of the general large model only has user input text information and text position information. User id information, user access ip information, and user input time information are added. Weather information, traffic information, and nearby business and office information of the user's location are obtained using the user ip and time information. User historical data is also retained according to the user id and ip information. Before training, the user historical data is analyzed to analyze and summarize the user's age, gender, question and answer field preference, activity level, work location, home address, and behavior characteristics at different times, and a detailed user portrait is made.

[0083] As shown in Figure 3 The first personalized model strengthens the understanding ability of the large model for specific service field data and the fusion ability of the personalized module. The general model is additionally trained to adapt to specific service field data, and the output result is a large model that preliminarily adapts to the personalized module and the specific service field in a large amount of data.

[0084] The embodiment of the present application adds a large amount of specific service field data on the basis of an open source model, including national, Beijing, Liaoning government website question and answer data such as provident fund withdrawal, driving license handling, marriage certificate, and ID card handling, and also adds some data in other fields, maintaining a 0.5:0.5 ratio of specific service field data and other field data. The purpose is to strengthen the understanding ability of the model for the specific service field, and not to lose the understanding ability of the general field. Here, because the general model is modified and a large number of personalized features are added, the original model does not have these features during training. These personalized features are randomly initialized in this step to preliminarily adapt the model to these newly added features.

[0085] Preferably, all input text data needs to be cleaned before being input into the model, removing text containing illegal characters, removing duplicate text, using technical means to remove pornographic or other sensitive information, and removing data that may cause user privacy leakage.

[0086] Preferably, the data is stored in a paragraph manner. In order to accelerate the utilization rate of resources such as GPU during model training, the data is pre-processed by word segmentation, and the segmented data is stored in a block manner. In this way, during training, one data block can be loaded at a time, and the speed of gpu / cpu can be greatly improved.

[0087] Preferably, for the additional personalized data, some discrete data and numerical data and category data, the processing method and the post-tokenization data are consistent for the discrete data and the category data, for the numerical data, each group of data is normalized, that is, the data variation range is compressed to 0-1, and then the data is stored in blocks. In training the first personalized model, these data are read into memory in the form of a whole block to improve data throughput. After the training data of the first personalized model enter the model, they are converted into model parameters preliminarily adapted to a specific business field, and saved as a model file for further optimization.

[0088] As shown in FIG. 1, it is a second personalized model, which gives the primary personalized ability of the large model. It can be understood that the general large model has no personalized ability. The first personalized model output model preliminarily integrates the personalized module, but still has no personalized ability. Figure 4

[0089] In the embodiment of the application, on the basis of the first personalized model output model parameters, the model's ability to capture personalization is strengthened, and adversarial learning is introduced. The personalized features of different users, including time, place, gender, preferences and all other personalized data and text data that can be collected, are sent to the generator for training, and a piece of pseudo-text is output. The pseudo-text is then sent to the discriminator together with the real user data to train the discriminator to distinguish between pseudo-text generated by the generator and real user data. In this step of training, other parameters of the model are fixed, and only the embedding layer is released. By mixing general data and personalized specific domain data, a model with personalized ability is obtained.

[0090] As shown in FIG. 1, it is a second personalized model, which gives the primary personalized ability of the large model. It can be understood that the general large model has no personalized ability. The first personalized model output model preliminarily integrates the personalized module, but still has no personalized ability. Figure 5 As shown in FIG. 1, it is a second personalized model, which gives the primary personalized ability of the large model. It can be understood that the general large model has no personalized ability. The first personalized model output model preliminarily integrates the personalized module, but still has no personalized ability.

[0091] In the embodiment of the application, on the basis of the first personalized model output model parameters, the model's ability to capture personalization is strengthened, and adversarial learning is introduced. The personalized features of different users, including time, place, gender, preferences and all other personalized data and text data that can be collected, are sent to the generator for training, and a piece of pseudo-text is output. The pseudo-text is then sent to the discriminator together with the real user data to train the discriminator to distinguish between pseudo-text generated by the generator and real user data. In this step of training, other parameters of the model are fixed, and only the embedding layer is released. By mixing general data and personalized specific domain data, a model with personalized ability is obtained.

[0092] ​In the embodiment of the application, at the input end of the large model, the named entity recognition (NER) information of each word is added again, and the intention information is added after each attention output, so that the model can more accurately understand the information contained in the input characters and inhibit the chance of hallucination of the model. When training the joint large model based on the named entity recognition, intention recognition and knowledge graph of the large model, a large amount of open source data is collected, and the named entity recognition data, intention recognition data and knowledge graph data of relationship triples in a specific application field are labeled at the same time. By mixing these data together, the accuracy and recall rate of the named entity recognition, intention recognition and knowledge graph extraction based on the large model are improved.

[0093] As shown in Figure 6 , for the adversarial network model, hierarchical adversarial learning is introduced to inhibit hallucination of the large model from the generation perspective.

[0094] It can be understood that, since the training of the large model is slow in the first place, the adversarial learning of the large model is even slower, and it is impossible to perform full-network adversarial learning on a large data set. Therefore, the layer-by-layer network adversarial learning mode is selected, that is, all parameters of the discriminator model and the generator model are fixed, and the trainable parameters are released layer by layer for adversarial learning.

[0095] In the embodiment of the application, the module data mainly adopts specific domain question and answer data pairs. The data is organized in a query / answer mode, for example, query: “After walking around the supermarket for half a day, it was found that the wallet was stolen by a thief when paying the bill, and the identity card was in it”, answer: “loss and replacement of the identity card”, and at the same time, personalized data is added. After obtaining the vector of each character in the input sequence, the personalized data is spliced. The large model has multiple layers. First, all parameters of the generator and the discriminator are fixed, and then the parameters of one layer are released layer by layer. The personalized features and the query part are transmitted to the generator, and the generator generates a pseudo answer. Then the pseudo answer and the real answer are input to the discriminator for discrimination. Since only one layer of network is trained each time, the model convergence speed is greatly improved.

[0096] As shown in Figure 7 , the character vector jitter adversarial learning model, the input character vector jitter adversarial learning, improves the model's ability to resist interference in literal differences.

[0097] It can be understood that when using the large model, it is found that the model is very sensitive to the user input prompt. Sometimes the input semantics is consistent, but the model output result is quite different just because of a few different words. Obviously, the model's anti-interference ability for input is insufficient. Although adversarial learning is added in the fourth step, it is intended to further and more directly improve the model's anti-interference and generalization ability.

[0098] In the embodiment of the application, the module data mainly adopts specific domain question and answer data pairs, and two processing methods are adopted. One is that several characters in the query text are randomly extracted and replaced with synonyms and near synonyms, and the embedding is obtained after inputting the model, and no jitter is performed on the embedding. The other method is that the query text is not transformed, and the embedding is directly inputted and then jittered. In the input layer of the model, the embedding of each token is taken after forward propagation, and no jitter is performed on the embedding when the gradient is calculated by back propagation. The experimental results show that this optimization can greatly improve the anti-interference ability of the model.

[0099] Further, the layer-by-layer parameter jittering adversarial model improves the anti-interference ability of the model in deep semantics. The model parameters output by the character vector jittering adversarial learning model have initially the ability to resist character interference, and the semantic anti-interference ability is further deepened, that is, the same output is generated for the case where the characters are quite different but the literal meanings are consistent, such as the user inputs "how old is someone" and "the age of someone". The literal meanings are completely irrelevant, but the semantics are consistent. The module accesses external information in each layer of the network, and the forward propagation jitter is used to improve the model's sensitivity to individualization and anti-interference ability. This module mainly uses specific domain question and answer data pairs. Since it is hoped that the model can deeply understand semantic information instead of staying at the literal meaning, a large number of data sets with strong semantic similarity and far apart literal meanings are needed. It is difficult to collect a sufficient amount of these data at one time.

[0100] In the embodiment of the application, a part of corpus is manually labeled, such as labeling the sentence pairs "how old is someone" and "the age of someone", and a semantic consistency detection model is trained in combination with some public data sets. On the other hand, the user's click logs on the Internet are mined, and the text input by the user in a continuous conversation process is mined. The text consistency detection model is used to detect the semantic consistency of the sentence pairs, and the data with close literal meanings is deleted to obtain a large number of sentence pairs with semantic consistency and far apart literal meanings. With these sentence pairs with semantic consistency and far apart literal meanings, the training data of this module is provided in combination with the query / answer sentence pairs.

[0101] Further, the reinforcement learning model intelligently selects the optimal output. A combination of general reinforcement learning and artificial reinforcement learning is adopted. The data of this module is mainly specific domain question and answer data. That is, the general reinforcement learning is trained for a certain number of steps, and then the model is used to predict multiple sentences for the question. The multiple answers output by the model for a question are judged and scored by artificial means, and the optimal output answer is selected. These optimal answers are fed back to the reinforcement learning process to assist the model to converge to the optimal parameters. Then the general reinforcement learning is trained again, and the artificial reinforcement learning is used for feedback, and the cycle is repeated until the model converges. Through this means, experiments prove that the reinforcement learning process can quickly converge.

[0102] To achieve the above-mentioned embodiments, as shown in the Figure 8 The device 10 for implementing intelligent question and answer services based on a large model is also provided in the embodiment, and the device 10 comprises a data acquisition module 100, a prediction output module 200, and an optimal selection module 300.

[0103] The data acquisition module 100 is configured to acquire real-time text data.

[0104] The prediction output module 200 is configured to input the real-time text data into the trained intelligent question and answer network model to perform corresponding answer prediction, and output multiple candidate answers according to the answer prediction result.

[0105] The optimal selection module 300 is configured to acquire scores corresponding to the multiple candidate answers, and output an optimal answer from the multiple candidate answers according to a sorting result of the scores.

[0106] The device for implementing intelligent question and answer services based on a large model in the embodiment of the application adds personalized capability to the large model, improves the accuracy of the large model in the question and answer field, suppresses the illusion of the large model, and enables the problem and answer of the user to be effectively matched.

[0107] In the description of the present specification, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any appropriate manner in any one or more embodiments or examples. In addition, the skilled person in the art can combine and combine the different embodiments or examples described in the present specification and the features of the different embodiments or examples without contradiction.

[0108] Furthermore, the terms "first", "second", "third", "fourth", "fifth" and "sixth" are used herein for descriptive purposes only and are not to be construed as indicating or implying relative importance or a significant nature of so described technical features. It is to be understood that a technical feature described with the "first", "second", "third", "fourth", "fifth" or "sixth" can implicitly or explicitly include at least one of the technical features described with the "first", "second", "third", "fourth", "fifth" or "sixth". In the description of the present application, the meaning of "a plurality" is at least two, for example, two, three, etc., unless otherwise specifically defined.

Claims

1. A method for implementing an intelligent question and answer service based on a large model, characterized in that, The method comprises: obtaining real-time text data and sample data of the real-time text data, the sample data comprising at least specific service field data and personalized data; inputting the sample data into an intelligent question and answer network model for model parameter training, wherein the intelligent question and answer network model comprises a first personalized model, a second personalized model, a joint model, an adversarial network model, a character vector jittering adversarial learning model, a layer-by-layer parameter jittering adversarial model and a reinforcement learning model; obtaining a trained intelligent question and answer network model based on the trained model parameters; inputting the real-time text data into the trained intelligent question and answer network model for corresponding answer prediction, and outputting a plurality of candidate answers according to the answer prediction result; obtaining scores corresponding to the plurality of candidate answers, and outputting an optimal answer from the plurality of candidate answers according to the score ranking result; wherein the adversarial network model comprises a second generator and a second discriminator, the sample data is inputted into the intelligent question and answer network model for model parameter training, the trained intelligent question and answer network model is obtained based on the trained model parameters, and further comprising: obtaining a character vector of a question and answer data pair of the specific service field data, wherein the question and answer data pair comprises at least a first query text and a first answer text; concatenating the character vector with personalized data to obtain personalized features, and sending the personalized features and the first query text to the second generator to generate a pseudo answer text; inputting the pseudo answer text and the first answer text into the second discriminator for discrimination to output a second discrimination result; training the adversarial network model based on the second discrimination result and third model parameters to output fourth model parameters; obtaining a second query text and a second answer text of the question and answer data pair; inputting the second query text and the second answer text into the character vector jittering adversarial learning model to perform micro-jittering on the embedded forward propagation of each token, and not to jitter the embedding when calculating the gradient in the backward propagation to train the character vector jittering adversarial learning model; outputting fifth model parameters based on the fourth model parameters and the trained character vector jittering adversarial learning model.

2. The method of claim 1, wherein, Before inputting the sample data into the intelligent question and answer network model for model parameter training, the method further comprises preprocessing the sample data, comprising: cleaning the sample data to obtain cleaned data, wherein cleaning the sample data comprises removing texts containing illegal characters, duplicates, sensitive information and user privacy leakage; segmenting the cleaned specific service field data, and storing the segmented data in a block manner to obtain a first data storage result; and segmenting discrete and categorical data in the cleaned personalized data, and normalizing numerical data to store the segmented personalized data and the normalized data in a block manner to obtain a second data storage result.

3. The method of claim 2, wherein, The sample data is input into an intelligent question and answer network model for model parameter training, and the trained intelligent question and answer network model is obtained based on the trained model parameters, including: The first data storage result and the second data storage result are input into the first personalized model for model training to obtain first model parameters.

4. The method of claim 3, wherein, The second personalized model includes a first generator and a first discriminator. The sample data is input into an intelligent question and answer network model for model parameter training, and the trained intelligent question and answer network model is obtained based on the trained model parameters, further including: The real user data and the corresponding text data in the sample data are input into the first generator for training to output pseudo-text; The pseudo-text and the real user data are input into the first discriminator for training to obtain a first discrimination result of the pseudo-text and the real user data; The second model parameters of the second personalized model are obtained based on the first discrimination result and the first model parameters.

5. The method of claim 4, wherein, The sample data is input into an intelligent question and answer network model for model parameter training, and the trained intelligent question and answer network model is obtained based on the trained model parameters, further including: Obtain the question and answer data pair of the specific service field data; The question and answer data pair is input into the joint model for training to obtain a model output result; wherein the model output result includes named entity recognition data, intent recognition data, and knowledge graph data of relationship triplets; The third model parameters are trained based on the second model parameters and the model output result.

6. The method of claim 1, wherein, The sample data is input into an intelligent question and answer network model for model parameter training, and the trained intelligent question and answer network model is obtained based on the trained model parameters, further including: Based on the labeled data of the question and answer data pair and the pre-set public data set, a semantic consistency detection model is trained; wherein the question and answer data pair further includes a third query text and a second answer text; Based on the user log, continuous input text in the user conversation process is obtained, and the semantic consistency detection model is used to detect the semantic consistency of the sentence pair of the continuous input text; The data with similar literal meanings in the semantic consistency sentence pair is filtered to obtain filtered data, and the filtered data, the third query text, and the third answer text are input into the semantic consistency detection model for training to obtain a layer-by-layer parameter jittering adversarial model, and the sixth model parameters are output based on the fifth model parameters and the layer-by-layer parameter jittering adversarial model.

7. The method of claim 6, wherein, The sample data is input into an intelligent question and answer network model for model parameter training, and the trained intelligent question and answer network model is obtained based on the trained model parameters, further including: The question and answer data pair is input into a reinforcement learning model for reinforcement learning training, and a plurality of predicted answers are output; The plurality of predicted answers are judged and scored to obtain an optimal question and answer based on the scoring result; Based on the optimal problem answer and the sixth model parameter, parameters of the reinforcement learning model are optimized to obtain seventh model parameters, and a trained intelligent question and answer network model is obtained based on the seventh model parameters.

8. An apparatus for implementing an intelligent question and answer service based on a large model, the apparatus comprising: The device is used to implement the method for implementing an intelligent question and answer service based on a large model, and the device comprises: a data acquisition module configured to acquire real-time text data; a prediction output module configured to input the real-time text data into the trained intelligent question and answer network model to perform corresponding answer prediction, and output a plurality of candidate answers according to an answer prediction result; an optimal selection module configured to acquire scores corresponding to the plurality of candidate answers, and output an optimal answer from the plurality of candidate answers according to a ranking result of the scores.

Images