Reading model optimization method, device, equipment and medium based on big data

Abstract

The present invention discloses a method for optimizing a reading model based on big data, comprising: pre-training a first reading comprehension model, a question generation model, and a second reading comprehension model based on a labeled data set; predicting an unlabeled data set using the pre-trained second reading comprehension model to obtain binary data pairs related to an article and an answer; predicting the binary data pairs using the pre-trained question generation model to obtain ternary data pairs related to an article, a question, and an answer; filtering the ternary data pairs using the pre-trained first reading comprehension model; screening the filtered ternary data pairs based on the topics of the articles in the labeled data set to generate a pseudo-labeled data set; and optimizing the pre-trained first reading comprehension model based on the pseudo-labeled data set and the labeled data set. The present invention solves the problems of small training samples and low model accuracy in existing reading comprehension technologies due to the high cost of obtaining labeled data.

Description

Technical Field

[0001] The present invention relates to the field of information technology, and in particular to a reading model optimization method, device, equipment and medium based on big data. Background Art

[0002] Reading comprehension technology is a highly complex and widely used information processing technique in the field of natural language processing. It aims to find the answer to a question from a given article or document, and can even determine whether the question is answerable. Excellent reading comprehension models require human-like language comprehension and knowledge reasoning capabilities to deeply explore and analyze the article, focusing on different parts of the article or viewpoints to find the correct answer based on the specific question. Therefore, this is a highly challenging task. Currently, excellent reading comprehension models are based on complex deep learning architectures, requiring vast amounts of training data for the model to learn. As the definition of reading comprehension technology suggests, training data for reading comprehension requires pre-annotation to locate information about the article, the question, and the answer. However, annotating training data is particularly difficult because the annotator must first read the entire article and then generate the answer based on the question, making it difficult to ensure both efficiency and accuracy. Due to the high cost of obtaining annotated data, in practice, reading comprehension models are often trained on relatively small amounts of training data, failing to find optimal solutions in the parameter space, limiting the model's accuracy.

[0003] Therefore, finding a method to solve the problems of small training samples and low model accuracy caused by the high cost of obtaining labeled data in existing reading comprehension technology has become a technical problem that technical personnel in this field urgently need to solve. Summary of the Invention

[0004] Embodiments of the present invention provide a reading model optimization method, apparatus, device, and medium based on big data to solve the problems of small training samples and low model accuracy caused by the high cost of obtaining labeled data in existing reading comprehension technologies.

[0005] A reading model optimization method based on big data, comprising:

[0006] Obtaining a labeled data set, and pre-training a preset first reading comprehension model, a question generation model, and a second reading comprehension model based on the labeled data set;

[0007] Obtaining an unlabeled dataset, and performing prediction on the unlabeled dataset using a pre-trained second reading comprehension model to obtain binary data pairs of articles and answers in the unlabeled dataset;

[0008] Predicting the binary data pairs using the pre-trained question generation model to obtain ternary data pairs of the article, question, and answer in the unlabeled dataset;

[0009] Filtering the triple data pairs through a pre-trained first reading comprehension model;

[0010] According to the topics of the articles in the annotated dataset, the filtered triple data pairs are screened to generate a pseudo-annotated dataset;

[0011] The pre-trained first reading comprehension model is optimized and trained according to the pseudo-annotated dataset and the annotated dataset.

[0012] Furthermore, the obtaining of the labeled data set and pre-training of the preset first reading comprehension model, question generation model, and second reading comprehension model according to the labeled data set includes:

[0013] Obtain a labeled data set, wherein the labeled data set includes a plurality of labeled data pairs, each labeled data pair including article information, question information, and corresponding answer information;

[0014] Pre-training the first reading comprehension model using article information and question information in the labeled dataset;

[0015] Pre-training the question generation model using article information and answer information in the labeled dataset;

[0016] The second reading comprehension model is pre-trained using article information in the labeled dataset.

[0017] Furthermore, the unlabeled dataset is predicted by the pre-trained second reading comprehension model to obtain binary data pairs of articles and answers in the unlabeled dataset, including:

[0018] Inputting the unlabeled data set into a pre-trained second reading comprehension model, and obtaining an output of the pre-trained second reading comprehension model as a first predicted answer;

[0019] Performing named entity recognition on the unlabeled data set to obtain a second predicted answer;

[0020] Obtaining a third prediction answer from the unlabeled dataset using a bidirectional long short-term memory network and conditional random field technology;

[0021] The first predicted answer, the second predicted answer, and the third predicted answer are combined to obtain binary data pairs of the article and the answer in the unlabeled dataset.

[0022] Furthermore, filtering the ternary data pairs by using the pre-trained first reading comprehension model includes:

[0023] Traversing the ternary data pairs, predicting the article information and question information in the ternary data pairs using the pre-trained first reading comprehension model, and obtaining a predicted answer corresponding to the ternary data pairs;

[0024] Comparing the predicted answer corresponding to the three-dimensional data pair with the answer information in the three-dimensional data pair;

[0025] If the predicted answer corresponding to the ternary data pair is different from the answer information in the ternary data pair, deleting the ternary data pair;

[0026] If the predicted answer corresponding to the ternary data pair is the same as the answer information in the ternary data pair, the ternary data pair is retained.

[0027] Furthermore, the filtering of the filtered triple data pairs according to the topics of the articles in the annotated data set to generate the pseudo-annotated data set includes:

[0028] Using the Dirichlet distribution topic model to perform similarity analysis on the article information of the filtered triple data pairs and the article information of the labeled data set, to obtain the topic similarity between the triple data pairs and the labeled data set;

[0029] The three-dimensional data pairs with topic similarity higher than the preset threshold are obtained to construct a pseudo-annotated dataset.

[0030] A reading model optimization device based on big data, comprising:

[0031] A pre-training module is used to obtain a labeled data set and pre-train a preset first reading comprehension model, a question generation model, and a second reading comprehension model based on the labeled data set;

[0032] A binary data pair generation module is used to obtain an unlabeled data set, and predict the unlabeled data set using a pre-trained second reading comprehension model to obtain binary data pairs of articles and answers in the unlabeled data set;

[0033] A ternary data pair generation module is used to predict the binary data pairs using a pre-trained question generation model to obtain ternary data pairs of articles, questions, and answers in the unlabeled dataset;

[0034] A filtering module, configured to filter the ternary data pairs using a pre-trained first reading comprehension model;

[0035] A screening module is used to screen the filtered triple data pairs according to the topics of the articles in the annotated dataset to generate a pseudo-annotated dataset;

[0036] The optimization training module is used to optimize the training of the pre-trained first reading comprehension model based on the pseudo-annotated dataset and the annotated dataset.

[0037] Furthermore, the binary data pair generation module includes:

[0038] A first answer prediction unit, configured to input the unlabeled data set into a pre-trained second reading comprehension model, and obtain an output of the pre-trained second reading comprehension model as a first predicted answer;

[0039] A second answer prediction unit, configured to perform named entity recognition on the unlabeled data set to obtain a second predicted answer;

[0040] A third answer prediction unit, configured to obtain a third predicted answer from the unlabeled dataset using a bidirectional long short-term memory network and a conditional random field technique;

[0041] A binary data pair generating unit is used to merge the first predicted answer, the second predicted answer and the third predicted answer to obtain a binary data pair of the article and the answer in the unlabeled dataset.

[0042] Furthermore, the filtering module includes:

[0043] A prediction unit, configured to traverse the ternary data pairs, predict the article information and question information in the ternary data pairs using the pre-trained first reading comprehension model, and obtain a predicted answer corresponding to the ternary data pairs;

[0044] a comparison unit, configured to compare the predicted answer corresponding to the ternary data pair with the answer information in the ternary data pair;

[0045] A filtering unit is used to delete the ternary data pair if the predicted answer corresponding to the ternary data pair is different from the answer information in the ternary data pair; if the predicted answer corresponding to the ternary data pair is the same as the answer information in the ternary data pair, retain the ternary data pair.

[0046] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the above-mentioned reading model optimization method based on big data is implemented.

[0047] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the above-mentioned reading model optimization method based on big data.

[0048] The embodiment of the present invention pre-trains a reading comprehension model by using a small amount of annotated data sets; then, the pre-trained reading comprehension model is used to predict a large number of unannotated data sets in related fields to generate relatively rough three-dimensional data pairs, namely (article, question, answer); high-quality three-dimensional data pairs are then selected from the relatively rough three-dimensional data pairs to construct a pseudo-annotated data set, and added to the original annotated data set for retraining the reading comprehension model; thereby greatly enriching the annotated data set used to train the reading comprehension model, effectively solving the problem of small training samples caused by the high cost of obtaining annotated data in existing reading comprehension technologies, and facilitating the finding of better parameters during the training process, thereby obtaining a better model than the previous reading comprehension model and improving the accuracy of the reading comprehension model. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0050] Figure 1 This is a flow chart of a reading model optimization method based on big data in one embodiment of the present invention;

[0051] Figure 2 is a flow chart of step S101 in a reading model optimization method based on big data in another embodiment of the present invention;

[0052] Figure 3 is a flow chart of step S102 in the reading model optimization method based on big data in another embodiment of the present invention;

[0053] Figure 4 is a flow chart of step S104 in the reading model optimization method based on big data in another embodiment of the present invention;

[0054] Figure 5 is a flow chart of step S105 in the reading model optimization method based on big data in another embodiment of the present invention;

[0055] Figure 6 This is a principle block diagram of a reading model optimization device based on big data in one embodiment of the present invention;

[0056] Figure 7FIG. 1 is a schematic diagram of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0057] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0058] The method for optimizing a reading model based on big data provided in an embodiment of the present invention is a method for improving a reading comprehension model based on self-training, and its purpose is to overcome the problem of being unable to find a better solution in the parameter space due to insufficient reading comprehension annotation data. The method for optimizing a reading model based on big data is applied to a server. The server can be implemented as an independent server or a server cluster composed of multiple servers. In one embodiment, Figure 1 As shown, a reading model optimization method based on big data is provided, which includes the following steps:

[0059] In step S101 , a labeled data set is obtained, and a preset first reading comprehension model, a question generation model, and a second reading comprehension model are pre-trained based on the labeled data set.

[0060] Here, the embodiment of the present invention first uses a small amount of existing labeled data sets to pre-train the reading comprehension model to pave the way for obtaining pseudo-labeled data from unlabeled data sets. Figure 2 The step S101 of obtaining a labeled data set and pre-training the preset first reading comprehension model, question generation model, and second reading comprehension model according to the labeled data set includes:

[0061] In step S201 , a labeled data set is obtained, where the labeled data set includes a number of labeled data pairs, and each labeled data pair includes article information, question information, and corresponding answer information.

[0062] Here, each annotated data pair includes article information, question information, and corresponding answer information, and the article information, question information, and answer information in each annotated data pair have been manually labeled using a specified method. For example, the triple data pairs can be implemented using a preset composition format, such as designing each annotated data pair as a triple: (article information Passage, question information Query, answer information); of course, it can also be implemented using identification information.

[0063] Taking customer service robots as an example, a labeled dataset refers to a collection of data pairs obtained through manual annotation of historical customer service conversations. Article information represents historical customer service conversations, question information represents questions asked in historical customer service conversations, and answer information represents answers to those questions. Because historical customer service conversation data is limited, the number of labeled data pairs in a labeled dataset is also limited. Furthermore, due to the high cost and difficulty of manual annotation, even if the volume of historical customer service conversation data is large enough, it is difficult to obtain sufficient training samples in a short period of time.

[0064] In step S202, the first reading comprehension model is pre-trained using the article information and question information in the labeled data set.

[0065] Here, the first reading comprehension model is a big data model that predicts answer information based on article information and question information, with article information and question information as input and answer information as output. Optionally, the first reading comprehension model includes but is not limited to the R-net machine reading comprehension model and the BERT machine reading comprehension model. In this embodiment of the present invention, the article information and question information in each labeled data pair in the labeled dataset are used as input to the first reading comprehension model to pre-train the first reading comprehension model.

[0066] In step S203, the question generation model is pre-trained using the article information and answer information in the labeled data set.

[0067] Here, the question generation model is a big data model that predicts question information based on article information and answer information, with the input being article information and answer information, and the output being question information. Exemplarily, the question generation model can employ a sequence-to-sequence + copy-based question generation model. In this embodiment of the present invention, the article information and answer information for each annotated data pair in the annotated dataset are used as input to the question generation model, and the model is pre-trained.

[0068] In step S204, the second reading comprehension model is pre-trained using the article information in the labeled dataset.

[0069] Here, the second reading comprehension model is a big data model that predicts answer information based on article information, with article information as input and answer information as output. In this embodiment of the present invention, the article information of each labeled data pair in the labeled dataset is used as input for the second reading comprehension model, and the second reading comprehension model is pre-trained.

[0070] After pre-training, the first reading comprehension model, question generation model, and second reading comprehension model are used to subsequently generate complete article, question, and answer triplet data pairs based on the unlabeled data set, and then generate a pseudo-labeled data set. Among them, the question generation model is used to generate question information based on the unlabeled data set. As mentioned above, the question generation model is usually pre-trained with a large amount of labeled article information to achieve the effect that the input is a sentence in the article information and the output is the previous sentence of the input. The second reading comprehension model is used to generate answer information based on the unlabeled data set as the source of one of the answers to the question information. The first reading comprehension model is used to filter the question information and the answer information. The reading model optimization method based on big data also includes:

[0071] In step S102, an unlabeled data set is obtained, and the unlabeled data set is predicted by a pre-trained second reading comprehension model to obtain binary data pairs of articles and answers in the unlabeled data set.

[0072] Here, the unlabeled dataset refers to unlabeled article information obtained from the internet, such as Wikipedia or WeChat public accounts, through crawling or downloading. Compared to the scale of labeled datasets, it is very large. Taking a customer service robot as an example, the unlabeled dataset consists of unlabeled article data obtained from the internet, as well as historical customer service conversation data that has not yet been labeled. In this embodiment of the present invention, this unlabeled dataset is transmitted as input to a pre-trained second reading comprehension model for prediction, obtaining answer information from the unlabeled dataset. The article information and answer information are then combined to obtain binary data pairs for the article and answer.

[0073] Optionally, the second reading comprehension model can be combined with multiple methods to predict answer information. Figure 3 As shown, in step S102, the unlabeled dataset is predicted by the pre-trained second reading comprehension model to obtain binary data pairs about articles and answers in the unlabeled dataset, including:

[0074] In step S301 , the unlabeled data set is input into a pre-trained second reading comprehension model, and an output of the pre-trained second reading comprehension model is obtained as a first predicted answer.

[0075] The first predicted answer is answer information obtained by predicting the unlabeled dataset through a second reading comprehension model.

[0076] In step S302, named entity recognition is performed on the unlabeled data set to obtain a second predicted answer.

[0077] The second predicted answer is the answer information predicted by using named entity recognition technology on the unlabeled dataset. Named Entity Recognition (NER) is a rule-based method that extracts predicted answer information using regular expressions, a pre-built entity dictionary, and an open-source syntax tree.

[0078] In step S303, a bidirectional long short-term memory network and conditional random field technology are used to obtain a third prediction answer from the unlabeled data set.

[0079] The third predicted answer is the answer information obtained by predicting the unlabeled dataset using a bidirectional long short-term memory (LSTM) network and conditional random fields (CRF) technology. Using bidirectional LSTM and CRF technology is currently the mainstream method for entity recognition using models. Bidirectional LSTM is used for feature extraction, and then CRF technology is used to incorporate dependencies between labels into the extracted features and predict the answer location to obtain the answer information.

[0080] In step S304, the first predicted answer, the second predicted answer, and the third predicted answer are merged to obtain binary data pairs of the article and the answer in the unlabeled dataset.

[0081] The aforementioned processes of predicting the first answer information using the second reading comprehension model, extracting the second predicted answer using named entity recognition technology, and extracting the third predicted answer using bidirectional LSTM and conditional random field technology are performed synchronously and in parallel. The obtained first predicted answer, second predicted answer, and third predicted answer are all used as one of the answer information corresponding to the unlabeled dataset. By combining multiple methods to obtain the answer information corresponding to the unlabeled dataset, the embodiments of the present invention can effectively increase the number and diversity of predicted answers, thereby expanding the number of binary and ternary data pairs.

[0082] In step S103, the binary data pairs are predicted by the pre-trained question generation model to obtain the ternary data pairs of the unlabeled dataset regarding the article, question and answer.

[0083] Here, this embodiment of the present invention uses the binary data pairs of the article and answer as input and passes them into the pre-trained question generation model. The question generation model then predicts the binary data pairs to obtain question information. The binary data pairs and question information are then combined to obtain ternary data pairs of the article, question, and answer in the unlabeled dataset. Each ternary data pair includes article information, question information, and an answer signal, and can be represented in the format of (article, question, answer).

[0084] In step S104, the ternary data pairs are filtered using a pre-trained first reading comprehension model.

[0085] In the embodiment of the present invention, the ternary data pairs generated by S103 have high noise. If the whole data is retained directly, the introduction of noise will produce adverse effects. Therefore, the embodiment of the present invention further filters the ternary data pairs through the pre-trained first reading comprehension model to reduce noise and improve data quality. Figure 4 As shown, the filtering of the ternary data pairs by the pre-trained first reading comprehension model in step S104 includes:

[0086] In step S401, the ternary data pairs are traversed, and the article information and question information in the ternary data pairs are predicted by the pre-trained first reading comprehension model to obtain a predicted answer corresponding to the ternary data pairs.

[0087] Here, the embodiment of the present invention takes the article information and question information in the ternary data pair as input and passes them into the pre-trained first reading comprehension model, so that the first reading comprehension model predicts the answer information based on the article information and question information in the ternary data, i.e., the predicted answer corresponding to the ternary data pair. It should be noted that the answer information obtained in step S401 is obtained based on the prediction of the article information and question information in the ternary data by the pre-trained first reading comprehension model, and the first predicted answer obtained in step S301 is obtained based on the prediction of the article information in the unlabeled dataset by the pre-trained second reading comprehension model, and is used as the answer information in the ternary data pair. Therefore, the predicted answer corresponding to the ternary data pair may be the same as or different from the answer information in the ternary data pair.

[0088] In step S402, the predicted answer corresponding to the ternary data pair is compared with the answer information in the ternary data pair.

[0089] Comparison methods include, but are not limited to, checking whether the predicted answer corresponding to the ternary data pair is completely consistent with the answer information in the ternary data pair, whether there is overlap, or whether there is a containment relationship. Depending on the amount of data in the actual application and the training effect of the reading comprehension model, one or any combination of the above methods can be selected to perform the comparison and determine which ternary data pairs to retain.

[0090] In step S403, if the predicted answer corresponding to the ternary data pair is different from the answer information in the ternary data pair, the ternary data pair is deleted.

[0091] In step S404, if the predicted answer corresponding to the ternary data pair is the same as the answer information in the ternary data pair, the ternary data pair is retained.

[0092] Depending on the comparison method adopted, the deleted and retained ternary data pairs are not exactly the same. When the comparison method is to determine whether the predicted answer corresponding to the ternary data pair is completely consistent with the answer information in the ternary data pair, then if and only if the predicted answer corresponding to the ternary data pair is completely consistent with the answer information in the ternary data pair, it is considered to be the same, otherwise it is considered to be different; when the comparison method is to determine whether the predicted answer corresponding to the ternary data pair and the answer information in the ternary data pair overlap, then if the predicted answer corresponding to the ternary data pair and the answer information in the ternary data pair overlap, it is considered to be the same, otherwise it is considered to be different; when the comparison method is to determine whether the predicted answer corresponding to the ternary data pair and the answer information in the ternary data pair have an inclusion relationship, then if the predicted answer corresponding to the ternary data pair and the answer information in the ternary data pair have an inclusion relationship, it is considered to be the same, otherwise it is considered to be different. In this embodiment of the present invention, ternary data pairs whose predicted answers are different from the answer information in the ternary data pair are deleted, and ternary data pairs whose predicted answers are the same as the answer information in the ternary data pair are retained.

[0093] In step S105 , the filtered triple data pairs are screened according to the topics of the articles in the annotated data set to generate a pseudo-annotated data set.

[0094] After filtering the triple data pairs, the present embodiment further selects thematically relevant triple data pairs from the retained triple data pairs based on the topics of the articles in the annotated dataset. Since these thematically relevant triple data pairs are not manually annotated, they are referred to as pseudo-annotated datasets in this embodiment to distinguish them from the annotated datasets.

[0095] Alternatively, as Figure 5 As shown, the step S105 of screening the filtered triple data pairs according to the topics of the articles in the labeled data set to generate a pseudo-labeled data set includes:

[0096] In step S501, a Dirichlet distribution topic model is used to perform similarity analysis on the article information of the filtered triple data pairs and the article information of the labeled data set to obtain the topic similarity between the triple data pairs and the labeled data set.

[0097] Here, the Dirichlet distribution topic model is an unsupervised model that does not require labeled data. A distribution analysis can be performed on an input article to determine the probability that the article belongs to each topic. In this embodiment of the present invention, the article information in the filtered, retained ternary data pairs and the article information in the labeled dataset are input into the Dirichlet distribution topic model to determine the probability that the article information in the filtered, retained ternary data pairs and the article information in the labeled dataset belong to each topic. The greater the probability, the greater the similarity, thus determining the thematic similarity of the article information, i.e., the thematic similarity between the ternary data pairs and the labeled dataset.

[0098] In step S502, three-dimensional data pairs with a subject similarity higher than a preset threshold are obtained to construct a pseudo-annotated data set.

[0099] In the embodiment of the present invention, a threshold is pre-set to compare the subject similarity of each triple data pair with the labeled data set and the preset threshold, and the triple data pairs with subject similarity higher than the preset threshold are screened out as pseudo-labeled data to construct a pseudo-labeled data set.

[0100] Since unlabeled data comes from various fields, and many fields are very different from the fields of labeled data sets, the embodiment of the present invention filters by subject similarity, retaining the ternary data pairs that are the same and / or similar to the fields to which the labeled data sets belong, which is conducive to reducing noise.

[0101] Here, the pseudo-annotated dataset and the annotated dataset constitute new training samples for the reading comprehension model, thereby expanding the annotated data for training the reading comprehension model, avoiding manual annotation, and reducing the cost of acquiring the annotated data.

[0102] In step S106 , the pre-trained first reading comprehension model is optimized and trained based on the pseudo-annotated dataset and the annotated dataset.

[0103] In an embodiment of the present invention, the pseudo-annotated data and the annotated data are used together to train the first reading comprehension model, which greatly enriches the annotated data for training the reading comprehension model and is conducive to finding better parameters during the training process, thereby obtaining a better model than the previous reading comprehension model.

[0104] The embodiment of the present invention pre-trains a reading comprehension model by using a small amount of annotated data sets; then, the pre-trained reading comprehension model is used to predict a large number of unannotated data sets in related fields to generate relatively rough three-dimensional data pairs (articles, questions, answers); high-quality three-dimensional data pairs are then selected from the relatively rough three-dimensional data pairs to construct a pseudo-annotated data set, and added to the original annotated data set for retraining the reading comprehension model; thereby greatly enriching the annotated data used to train the reading comprehension model, effectively solving the problem of small training samples caused by the high cost of obtaining annotated data in existing reading comprehension technologies, and facilitating the finding of better parameters during the training process, thereby obtaining a better model than the previous reading comprehension model and improving the accuracy of the reading comprehension model.

[0105] The reading model optimization method based on big data provided by the embodiment of the present invention can alleviate the problem that the reading comprehension model cannot find a better solution in the parameter space due to insufficient labeled data, and effectively improve the accuracy of the reading comprehension model. The reading comprehension model trained by the embodiment of the present invention is mainly used in the task of information extraction, and the information extraction task is one of the very important modules such as customer service robots and chat robots. The reading comprehension model trained by the present invention can more accurately and quickly extract the answers to the questions that users want to ask from massive documents, achieve accurate answers to user questions, reduce the number of user inquiry rounds in the customer service robot and the overall load of the customer service robot. If the customer service robot answers the user's questions quickly and accurately, the number of polling rounds is small; if the answer is inaccurate, the user will often continue to ask questions, increasing the number of inquiry rounds.

[0106] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0107] In one embodiment, a reading model optimization device based on big data is provided, and the reading model optimization device based on big data corresponds to the reading model optimization method based on big data in the above embodiment. Figure 6 As shown, the reading model optimization device based on big data includes a pre-training module 61, a binary data pair generation module 62, a ternary data pair generation module 63, a filtering module 64, a screening module 65, and an optimization training module 66. The functional modules are described in detail as follows:

[0108] A pre-training module 61 is configured to obtain a labeled data set and pre-train a preset first reading comprehension model, a question generation model, and a second reading comprehension model based on the labeled data set;

[0109] A binary data pair generation module 62 is configured to obtain an unlabeled data set, and predict the unlabeled data set using a pre-trained second reading comprehension model to obtain binary data pairs of articles and answers in the unlabeled data set;

[0110] A ternary data pair generation module 63 is configured to predict the binary data pairs using a pre-trained question generation model to obtain ternary data pairs of the article, question, and answer in the unlabeled dataset;

[0111] A filtering module 64, configured to filter the ternary data pairs using a pre-trained first reading comprehension model;

[0112] A screening module 65 is used to screen the filtered triple data pairs according to the topics of the articles in the annotated data set to generate a pseudo-annotated data set;

[0113] The optimization training module 66 is configured to perform optimization training on the pre-trained first reading comprehension model based on the pseudo-annotated dataset and the annotated dataset.

[0114] Optionally, the pre-training module 61 includes:

[0115] An acquisition unit is configured to acquire a labeled data set, wherein the labeled data set includes a plurality of labeled data pairs, each labeled data pair including article information, question information, and corresponding answer information;

[0116] A first pre-training unit, configured to pre-train the first reading comprehension model using article information and question information in the labeled dataset;

[0117] A second pre-training unit, configured to pre-train the question generation model using the article information and answer information in the labeled dataset;

[0118] The third pre-training unit is used to pre-train the second reading comprehension model using the article information in the labeled data set.

[0119] Optionally, the binary data pair generating module 62 includes:

[0120] A first answer prediction unit, configured to input the unlabeled data set into a pre-trained second reading comprehension model, and obtain an output of the pre-trained second reading comprehension model as a first predicted answer;

[0121] A second answer prediction unit, configured to perform named entity recognition on the unlabeled data set to obtain a second predicted answer;

[0122] A third answer prediction unit, configured to obtain a third predicted answer from the unlabeled dataset using a bidirectional long short-term memory network and a conditional random field technique;

[0123] A binary data pair generating unit is used to merge the first predicted answer, the second predicted answer and the third predicted answer to obtain a binary data pair of the article and the answer in the unlabeled dataset.

[0124] Optionally, the filtering module 64 includes:

[0125] A prediction unit, configured to traverse the ternary data pairs, predict the article information and question information in the ternary data pairs using the pre-trained first reading comprehension model, and obtain a predicted answer corresponding to the ternary data pairs;

[0126] a comparison unit, configured to compare the predicted answer corresponding to the ternary data pair with the answer information in the ternary data pair;

[0127] A filtering unit is used to delete the ternary data pair if the predicted answer corresponding to the ternary data pair is different from the answer information in the ternary data pair; if the predicted answer corresponding to the ternary data pair is the same as the answer information in the ternary data pair, retain the ternary data pair.

[0128] Optionally, the screening module 65 includes:

[0129] Using the Dirichlet distribution topic model to perform similarity analysis on the article information of the filtered triple data pairs and the article information of the labeled data set, to obtain the topic similarity between the triple data pairs and the labeled data set;

[0130] The three-dimensional data pairs with topic similarity higher than the preset threshold are obtained to construct a pseudo-annotated dataset.

[0131] The specific definition of the big data-based reading model optimization device can be found in the definition of the big data-based reading model optimization method above, and will not be repeated here. The various modules in the above-mentioned big data-based reading model optimization device can be implemented in whole or in part through software, hardware, or a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory of the computer device in software form, so that the processor can call and execute the corresponding operations of the above-mentioned modules.

[0132] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 7As shown. The computer device includes a processor, a memory, a network interface, and a database connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a reading model optimization method based on big data is implemented.

[0133] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the following steps are performed:

[0134] Obtaining a labeled data set, and pre-training a preset first reading comprehension model, a question generation model, and a second reading comprehension model based on the labeled data set;

[0135] Obtaining an unlabeled dataset, and performing prediction on the unlabeled dataset using a pre-trained second reading comprehension model to obtain binary data pairs of articles and answers in the unlabeled dataset;

[0136] Predicting the binary data pairs using the pre-trained question generation model to obtain ternary data pairs of the article, question, and answer in the unlabeled dataset;

[0137] Filtering the triple data pairs through a pre-trained first reading comprehension model;

[0138] According to the topics of the articles in the annotated dataset, the filtered triple data pairs are screened to generate a pseudo-annotated dataset;

[0139] The pre-trained first reading comprehension model is optimized and trained according to the pseudo-annotated dataset and the annotated dataset.

[0140] Those skilled in the art will appreciate that all or part of the processes in the above-described embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the above-described embodiments. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct RAM bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM).

[0141] Those skilled in the art will clearly understand that for the sake of convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0142] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention, and should all be included in the scope of protection of the present invention.

Claims

1. A reading model optimization method based on big data, characterized in that: include: Obtaining a labeled data set, wherein the labeled data set includes a plurality of labeled data pairs, each labeled data pair including article information, question information, and corresponding answer information; pre-training a first reading comprehension model using the article information and question information in the labeled data set; pre-training a question generation model using the article information and answer information in the labeled data set; and pre-training a second reading comprehension model using the article information in the labeled data set; Obtaining an unlabeled dataset, and performing prediction on the unlabeled dataset using a pre-trained second reading comprehension model to obtain binary data pairs of articles and answers in the unlabeled dataset; Predicting the binary data pairs using the pre-trained question generation model to obtain ternary data pairs of the article, question, and answer in the unlabeled dataset; Filtering the triple data pairs through a pre-trained first reading comprehension model; According to the topics of the articles in the annotated dataset, the filtered triple data pairs are screened to generate a pseudo-annotated dataset; The pre-trained first reading comprehension model is optimized and trained according to the pseudo-annotated dataset and the annotated dataset.

2. The method for optimizing a reading model based on big data according to claim 1, wherein: The method of predicting the unlabeled dataset using the pre-trained second reading comprehension model to obtain binary data pairs of articles and answers in the unlabeled dataset includes: Inputting the unlabeled data set into a pre-trained second reading comprehension model, and obtaining an output of the pre-trained second reading comprehension model as a first predicted answer; Performing named entity recognition on the unlabeled data set to obtain a second predicted answer; Obtaining a third prediction answer from the unlabeled dataset using a bidirectional long short-term memory network and conditional random field technology; The first predicted answer, the second predicted answer, and the third predicted answer are combined to obtain binary data pairs of the article and the answer in the unlabeled dataset.

3. The method for optimizing a reading model based on big data according to claim 1, wherein: The filtering of the ternary data pairs by the pre-trained first reading comprehension model includes: Traversing the ternary data pairs, predicting the article information and question information in the ternary data pairs using the pre-trained first reading comprehension model, and obtaining a predicted answer corresponding to the ternary data pairs; Comparing the predicted answer corresponding to the three-dimensional data pair with the answer information in the three-dimensional data pair; If the predicted answer corresponding to the ternary data pair is different from the answer information in the ternary data pair, deleting the ternary data pair; If the predicted answer corresponding to the ternary data pair is the same as the answer information in the ternary data pair, the ternary data pair is retained.

4. The method for optimizing a reading model based on big data according to claim 1, wherein: The step of screening the filtered triple data pairs according to the topics of the articles in the labeled data set to generate a pseudo-labeled data set includes: Using the Dirichlet distribution topic model to perform similarity analysis on the article information of the filtered triple data pairs and the article information of the labeled data set, to obtain the topic similarity between the triple data pairs and the labeled data set; The three-dimensional data pairs with topic similarity higher than the preset threshold are obtained to construct a pseudo-annotated dataset.

5. A reading model optimization device based on big data, characterized in that: include: A pre-training module is configured to obtain a labeled data set, wherein the labeled data set includes a plurality of labeled data pairs, each labeled data pair including article information, question information, and corresponding answer information; pre-train a first reading comprehension model using the article information and question information in the labeled data set; pre-train a question generation model using the article information and answer information in the labeled data set; and pre-train a second reading comprehension model using the article information in the labeled data set; A binary data pair generation module is used to obtain an unlabeled data set, and predict the unlabeled data set using a pre-trained second reading comprehension model to obtain binary data pairs of articles and answers in the unlabeled data set; A ternary data pair generation module is used to predict the binary data pairs using a pre-trained question generation model to obtain ternary data pairs of articles, questions, and answers in the unlabeled dataset; A filtering module, configured to filter the ternary data pairs using a pre-trained first reading comprehension model; A screening module is used to screen the filtered triple data pairs according to the topics of the articles in the annotated dataset to generate a pseudo-annotated dataset; The optimization training module is used to optimize the training of the pre-trained first reading comprehension model based on the pseudo-annotated dataset and the annotated dataset.

6. The reading model optimization device based on big data according to claim 5, characterized in that: The binary data pair generation module includes: A first answer prediction unit, configured to input the unlabeled data set into a pre-trained second reading comprehension model, and obtain an output of the pre-trained second reading comprehension model as a first predicted answer; A second answer prediction unit, configured to perform named entity recognition on the unlabeled data set to obtain a second predicted answer; A third answer prediction unit, configured to obtain a third predicted answer from the unlabeled dataset using a bidirectional long short-term memory network and a conditional random field technique; A binary data pair generating unit is used to merge the first predicted answer, the second predicted answer and the third predicted answer to obtain a binary data pair of the article and the answer in the unlabeled dataset.

7. The reading model optimization device based on big data according to claim 5 or 6, characterized in that: The filtering module includes: A prediction unit, configured to traverse the ternary data pairs, predict the article information and question information in the ternary data pairs using the pre-trained first reading comprehension model, and obtain a predicted answer corresponding to the ternary data pairs; a comparison unit, configured to compare the predicted answer corresponding to the ternary data pair with the answer information in the ternary data pair; A filtering unit is used to delete the ternary data pair if the predicted answer corresponding to the ternary data pair is different from the answer information in the ternary data pair; if the predicted answer corresponding to the ternary data pair is the same as the answer information in the ternary data pair, retain the ternary data pair.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the reading model optimization method based on big data is implemented as described in any one of claims 1 to 4.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the reading model optimization method based on big data as described in any one of claims 1 to 4 is implemented.

Images