A dynamic enhanced multi-hop text reading recognition processing method for low-data

Abstract

The present invention discloses a method for dynamically enhanced multi-hop text reading recognition processing for low data. The document data set is corrected and preprocessed; a dynamically enhanced answer prediction model is constructed; a training set is used to train the dynamically enhanced answer prediction model as a teacher model; a portion of the unlabeled data set is randomly selected and input into the teacher model to predict the label result to establish a pseudo label, and the data set with the pseudo label is added to the training set to form a new training set; the teacher model is retrained with the new training set to obtain a student model; the steps are continuously repeated iteratively until the model accuracy result of the validation set meets the preset threshold requirement; the final student model is used to predict the reading document to be tested, and the prediction is output to obtain the answer to the reading document to be tested. The present invention uses a dynamic enhancement method to expand data, which can reduce the input length, solve the multi-hop reading comprehension problem in the case of little label data, and enhance the generalization ability of the model.

Description

Technical Field

[0001] This invention belongs to the field of natural language processing technology and relates to a text data processing method, specifically a dynamic enhanced multi-hop text reading and recognition processing method for low-data applications. Background Technology

[0002] Machine reading comprehension tasks require machines to answer questions based on given context, and can be used in fields such as search engines and intelligent assistants to provide users with high-quality information services. Early reading comprehension systems were small in scale and had domain-specific limitations, hindering their widespread application. With the superior performance of deep learning in acquiring contextual information and the emergence of many large benchmark datasets, some machine reading comprehension models have achieved significant performance improvements on certain single-hop machine reading comprehension datasets. However, these models still lack the ability to reason across multiple sentences. In recent years, the emergence of multi-hop reading comprehension datasets has required models to reason across multiple disjointed sentences or even documents. Currently, there is considerable research on this topic, all using large pre-trained models as feature extractors, followed by fine-tuning on specific reading comprehension tasks. This approach requires a large amount of data to train the model. However, data annotation in the real world is very time-consuming and labor-intensive, and in some domains, there are not enough labeled samples.

[0003] Most multi-hop reading comprehension tasks are supported by large amounts of data, with little research on low-data scenarios. However, given the high cost of annotation, directly using existing reading comprehension models often yields poor results. Data augmentation is a good option for low-data situations. Current data augmentation methods in the text domain focus on text classification tasks, but no papers have demonstrated significant effectiveness in reading comprehension. Sliding windows, as a data augmentation technique for reading comprehension tasks, are often used in single-hop tasks; however, they cannot guarantee that all supporting sentences are within the window, making them unsuitable for multi-hop scenarios. Summary of the Invention

[0004] Training neural networks requires large-scale datasets, but labeling these datasets is extremely time-consuming and labor-intensive. To address the challenges of multi-hop reading recognition and understanding tasks and background techniques in low-data scenarios with insufficient data, this invention focuses on low-data-data situations and proposes a dynamic context-enhanced multi-hop text reading recognition processing method. This method uses a self-training approach to introduce external knowledge and adds auxiliary datasets to improve model performance.

[0005] like Figure 1 As shown, the objective of this invention is achieved through the following technical solutions:

[0006] Step 1: Perform preprocessing on the document dataset to eliminate semantic ambiguity caused by anonymization;

[0007] Step 2: Build a dynamically enhanced answer prediction model;

[0008] Step 3: Using the dataset with known labels processed in Step 1 as the training set, train a dynamically enhanced answer prediction model using the training set, and use the trained answer prediction model as the teacher model.

[0009] Step 4: Randomly select a portion of the unlabeled dataset and input it into the teacher model obtained in Step 3 to predict the label results. Use the predicted labels as pseudo-labels and attach them to the unlabeled dataset to form a dataset with pseudo-labels. Then, add the dataset with pseudo-labels to the training set to form a new training set.

[0010] Step 5: Use the new training set obtained in Step 4 to retrain the teacher model obtained in Step 3. The trained teacher model will then be used as the student model.

[0011] Step 6: Use another dataset with known labels as the validation set. Input the validation set into the student model obtained in Step 5 to test and obtain the model accuracy results.

[0012] If the model accuracy meets the preset threshold requirement, proceed to the next step;

[0013] If the model accuracy result does not meet the preset threshold requirement, return to step 3 and use the student model as the teacher model for iterative processing. Repeat steps 3-5 for iteration until the model accuracy result obtained through the validation set meets the preset threshold requirement.

[0014] Step 7: Use the student model obtained in Step 6 to predict the reading document to be tested, and output the predicted labels of the reading document to be tested and the answers therein.

[0015] The dataset includes documents, questions, and tags. The tags include answers, supporting sentences, and answer categories. The answers are composed of the start position and the end position of the answer.

[0016] The question refers to a textual data reading comprehension exercise, and the answer refers to the response to the question, specifically a text element appearing in the document.

[0017] The supporting sentences mentioned are specifically sentences that support the reasoning for the answer in reading comprehension, and they appear in one place in the document.

[0018] The answer categories mentioned are generally divided into, for example, two categories, three categories, four categories, but are not limited to these.

[0019] In step 2, the dynamically enhanced answer prediction model specifically refers to:

[0020] 2.1. Segment the documents in the dataset to obtain individual sentences, extract partial or complete parts from each sentence, and then combine them with the corresponding questions in the document to form context text;

[0021] 2.2 Input the context text into the pre-trained answer prediction model. Specifically, the answer prediction model can be the BERT model. Obtain the overall feature vector CLS_ALL of the context text, as well as the decoded feature vectors of the question and each word of each sentence in the context text. The question vector Q is composed of the decoded feature vectors of each word of the question, and the context vector C is composed of the decoded feature vectors of each word of all sentences. 2.3

[0023] A. After processing the overall feature vector CLS_ALL through a linear layer, the predicted answer category type is obtained, expressed by the following formula:

[0024] type=softmax(Linear(CLS_ALL) / τ)∈R 1×4

[0025] Where softmax() represents the softmax activation function, and τ represents the hyperparameter;

[0026] B. Based on the question vector Q and the context vector C, obtain the context feature vector C′ as follows. Then, based on the context feature vector C′, extract two results using a multilayer perceptron, which are respectively used as the predicted answer start position (start) and answer end position (end):

[0027] Q′=Attention_pooling(Q)∈R 1×d

[0028] C′=Norm(w1·CLS_ALL+w2·Q′+w3·C)∈R l×d

[0029]

[0030] In the formula, w1, w2, and w3 represent the first, second, and third weights, respectively; d represents the dimension of the latent vector; l represents the sentence length; Q′ represents the dimension reduction problem vector; Norm() represents the normalization function; MLP() represents the multilayer perceptron; and R 1×4 The four types of answers correspond to four dimensions, where d represents the dimension of the hidden layer and l represents the length of the context. This represents the probability that each position in the context is the start or end position of the answer, corresponding to the start position and end position of the answer, respectively.

[0031] C. Based on the overall feature vector CLS_ALL, the question vector Q, and the context vector C, the supporting sentences for the predicted output are obtained by processing them together in the following manner:

[0032] SFeature=W·AttentionPooling(C,CLS_ALL,Q)

[0033]

[0034] In the formula, W represents the weight matrix; sigmoid() represents the sigmoid activation function; and AttentionPooling() represents the attention pooling operation. Supporting sentences indicating the predicted output.

[0035] Step 7 specifically includes:

[0036] Step 7.1: Segment the reading document to be tested to obtain individual sentences. Then, filter each sentence and the question together using a filtering model to obtain the relevance between each sentence and the question. Select the K sentences with the highest relevance and arrange them in the order in which they appear in the reading document to be tested to form a new document.

[0037] Step 7.2: Input the new document into the student model obtained in Step 6, output the predicted label of the new document, and extract the answer from the label as the answer to the document to be read.

[0038] The screening model specifically uses a Chinese BERT pre-trained model and a Roberta model to encode the question sentence pairs.

[0039] The method of this invention first trains a two-stage reading model as a teacher network. This reading model is a two-stage model. The first stage is a sentence selection model. Considering that sentences other than true supporting sentences do not participate in the reasoning process, the selection model selects the Top K sentences strongly related to the question, reducing the interference of irrelevant sentences and addressing the drawback of long text inputs that cannot be directly input into the pre-trained model. The second stage is an answer prediction model that infers the answer and supporting sentences, adding an answer category prediction task. The answer is extracted only when it is present in the text. Due to the limited training dataset, a dynamically enhanced answer prediction model is proposed. During model training, each batch of training data consists of true supporting sentences and several sentences randomly extracted from the context, dynamically updating the input to increase the model's generalization ability. During reasoning, the sentences selected by the selection model are directly used as input to the answer prediction model. Furthermore, this invention uses a self-training method to add pseudo-labels to unlabeled data to expand the dataset.

[0040] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0041] This invention establishes a dynamic augmentation multi-hop reading recognition processing model for low-data scenarios. To address the problem of limited label data, a dynamic augmentation method is used to expand the dataset during the training process of the answer prediction model.

[0042] To address the issue of not being able to input all documents into the model under low-resource conditions, non-supporting sentences are randomly input during the training phase, and a filtering model is used during the prediction phase to filter sentences related to the question answer to reduce the input length. The prediction part of the supporting sentences no longer uses mainstream graph networks for encoding, but instead uses an improved Transformer to re-encode and learn the relationships between sentences. Furthermore, an external dataset is introduced, and a self-trained pseudo-labeling method is used to enhance the model's generalization ability.

[0043] The invention has been tested and shows significant improvement on the CAIL 2020 reading comprehension dataset in the Chinese legal field, proving the effectiveness of the solution and demonstrating its application value and practical significance for text data on the internet. Attached Figure Description

[0044] Figure 1 This is a diagram illustrating the overall architecture of the self-training learning method used in this invention.

[0045] Figure 2 This is a diagram of the sentence filtering model architecture described in this invention;

[0046] Figure 3 This is a diagram of the answer prediction model architecture described in this invention;

[0047] Figure 4 This is a diagram of the context sentence feature extraction architecture described in this invention;

[0048] Figure 5 This is a data example used in this invention. Detailed Implementation

[0049] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to specific embodiments and accompanying drawings.

[0050] like Figure 1 As shown, the embodiments of the present invention and their implementation process are as follows:

[0051] The embodiments of the present invention were implemented and tested on the CAIL 2020 reading comprehension dataset in the Chinese legal field.

[0052] The CAIL 2020 Reading Comprehension Dataset is a Chinese legal reading comprehension dataset. The training set consists partly of the CJRC training set and partly of 5100 re-annotated question-answer pairs, covering civil, criminal, and administrative domains. The validation and test sets contain approximately 1900 and 2600 question-answer pairs respectively, making the dataset relatively small. The external dataset CJRC used in the experiments was proposed by the CAIL 2019 Reading Comprehension track. The data mainly comes from judgments published on the China Judgments Online website, covering criminal and civil domains. The training set contains 40,000 questions, while the validation and test sets each contain approximately 5000 questions.

[0053] Step 1: Perform preprocessing on the document dataset to eliminate semantic ambiguity caused by anonymization;

[0054] In step 1, for each document in the dataset, each sentence is traversed first, and regular expressions are used to match triples. The extracted triples are added to the name list. Then, the entire document is traversed again to split the name and number parts, matching from shortest to longest. This achieves disambiguation processing for documents, addressing the semantic ambiguity caused by data anonymization during word segmentation.

[0055] Step 2: Build a dynamically enhanced answer prediction model;

[0056] like Figure 3 As shown, the dynamically enhanced answer prediction model is as follows:

[0057] 2.1. Segment the documents in the dataset to obtain individual sentences. Extract partial or complete sentences from each sentence and combine them with the corresponding questions in the document to form context text. In practice, the question corresponding to the document is placed at the beginning of the context text, and the sentences are ordered according to their original order in the document. For example... Figure 3As shown in the figure, SEP represents the separator.

[0058] 2.2 Input the context text into the pre-trained BERT model to obtain the overall feature vector CLS_ALL of the context text and the decoded feature vector of each word of the question and each sentence in the context text. The question vector Q is composed of the decoded feature vector of each word of the question, and the context vector C is composed of the decoded feature vector of each word of all sentences. 2.3

[0060] A. After processing the overall feature vector CLS_ALL through a linear layer, the predicted answer category type is obtained, which is represented by the following formula:

[0061] type=softmax(Linear(CLS_ALL) / τ)∈R 1×4

[0062] Where softmax() represents the softmax activation function, and τ represents the weights;

[0063] B. Based on the question vector Q and the context vector C, obtain the context feature vector C′ as follows. Then, based on the context feature vector C′, extract two results using a multilayer perceptron, which are respectively used as the predicted answer start position (start) and answer end position (end):

[0064] Q′=Attention_pooling(Q)∈R 1×d

[0065] C′=Norm(w1·CLS_ALL+w2·Q′+w3·C)∈R l×d

[0066]

[0067] In the formula, w1, w2, and w3 represent the first, second, and third weights, respectively; d represents the dimension of the latent vector; l represents the sentence length; Q′ represents the dimension reduction problem vector; Norm() represents the normalization function; MLP() represents the multilayer perceptron; and R 1×4 The four types of answers correspond to four dimensions, where d represents the dimension of the hidden layer and l represents the length of the context. This represents the probability that each position in the context is the start or end position of the answer, corresponding to the start position and end position of the answer, respectively.

[0068] C, such as Figure 4 As shown, the supporting sentences for the predicted output are obtained by processing the overall feature vector CLS_ALL, the question vector Q, and the context vector C together in the following manner:

[0069] SFeature=W·AttentionPooling(C,CLS_ALL,Q)

[0070]

[0071] In the formula, W represents the weight matrix, initialized to a constant; sigmoid() represents the sigmoid activation function; and AttentionPooling() represents the attention pooling operation. Supporting sentences indicating the predicted output.

[0072] As can be seen from the above, the answer category prediction of the present invention is to perform a four-class classification task, apply a weight τ to each output value as a penalty, and take the one with the highest probability as the answer category.

[0073] If the answer is in the context, an answer extraction task is used to extract the answer from the document. The answer extraction task involves reducing the dimensionality of the re-encoded Q, then weighting and summing it with CLS_ALL along with each dimension of C, and finally using a multilayer perceptron to transform the output into 2-dimensionality to predict the start and end positions of the answer.

[0074] In practice, the sentence vectors are re-encoded by adjusting the dimensions of q and k during computation. A parameter matrix is ​​used to fuse the sentence features by superimposing the re-encoded information. Then, a multilayer perceptron is used to convert the vectors into an n×1 dimensional output. Entries exceeding a threshold in each dimension are identified as supporting sentences, such as... Figure 4 As shown.

[0075] Step 3: Using the dataset with known labels processed in Step 1 as the training set, train a dynamically enhanced answer prediction model using the training set, and use the trained answer prediction model as the teacher model.

[0076] Step 4: Randomly select a portion of the unlabeled dataset and input it into the teacher model obtained in Step 3 to predict the label results. Use the predicted labels as pseudo-labels and attach them to the unlabeled dataset to form a dataset with pseudo-labels. Then, add the dataset with pseudo-labels to the training set to form a new training set.

[0077] This self-training method adds pseudo-labels to the unlabeled dataset and incorporates them into the labeled data, increasing the training set size and improving training effectiveness to achieve a more accurate model and its predictions.

[0078] Step 5: Use the new training set obtained in Step 4 to retrain the teacher model obtained in Step 3. The trained teacher model will then be used as the student model.

[0079] Step 6: Use another dataset with known labels as the validation set. Input the validation set into the student model obtained in Step 5 to test and obtain the model accuracy results.

[0080] If the model accuracy meets the preset threshold requirement, proceed to the next step;

[0081] If the model accuracy result does not meet the preset threshold requirement, return to step 3 and use the student model as the teacher model for iterative processing. Repeat steps 3-5 for iteration until the model accuracy result obtained through the validation set meets the preset threshold requirement.

[0082] In the specific implementation of this invention, the teacher model and the student model use the same topology. Dropout noise is added in the student model stage to increase the learning difficulty, while no noise is added when generating pseudo-labels in the teacher model stage.

[0083] Step 7: Use the student model obtained in Step 6 to predict the reading document to be tested, and output the predicted labels of the reading document to be tested and the answers therein. The answers are included in the labels.

[0084] The reading text to be tested is also preprocessed in step 1 and then input into the student model.

[0085] Step 7.1: Segment the document to be tested into individual sentences. Then, process each sentence and the question using a filtering model to determine the relevance between each sentence and the question. Select the K sentences with the highest relevance and arrange them in the order they appear in the document to form a new document. For example... Figure 2 As shown;

[0086] Step 7.2: Input the new document into the student model obtained in Step 6, output the predicted label of the new document, and extract the answer from the label as the answer to the document to be read.

[0087] The specific screening model implemented uses a Chinese BERT pre-trained model, which encodes the question sentence pairs. The Chinese BERT pre-trained model loads pre-trained weights, and the entire training process uses fine-tuning. The loss value is calculated using the binary cross-entropy loss function and optimized using a gradient descent optimizer with Adam adaptive learning rate based on the Warmup mechanism.

[0088] Therefore, this invention, by using a screening model to process the document to be read, can better reduce the interference of irrelevant noise in the test situation, and achieve the effect and advantage of utilizing the effective information of long text information while reducing the interference of irrelevant information.

[0089] Based on the idea of ​​multi-task learning, the answer prediction model is trained using dynamic augmentation on all contextual sentences, including training of three sub-tasks: answer category prediction, answer extraction, and supporting sentence prediction.

[0090] During the training process, the answer prediction model randomly selects a portion of sentences from the documents in the dataset, excluding the supporting sentences, to form context text together with the supporting sentences, serving as dynamically enhanced context sentences.

[0091] During the test processing, the answer prediction model uses the new document obtained by the filtered model in step 7.1 as the context text.

[0092] During training, all random seeds are fixed, and hyperparameters such as batch size remain the same; the loss function L consists of three parts: loss function for answer category, span prediction, and supporting sentences.

[0093]

[0094]

[0095] Where L represents the overall loss function, α represents the weight of the answer category loss function, and β represents the weight of the supporting data prediction. y represents the probability that the predicted answer belongs to each category. type Indicates the category of the true answer. y represents the probability that each predicted sentence is a supporting sentence. sf Labels indicating true supporting sentences, This represents the probability that each position in the predicted context is the starting position of the answer. y represents the probability that each position in the predicted context is the end position of the answer. start y represents the starting position of the true answer within the context. end This indicates the position of the true answer within the context; CE() represents the multivariate cross-entropy loss function; BCE() represents the binary cross-entropy loss function; L ans The loss function represents the answer.

[0096] In the loss function described above, cross-entropy loss is calculated separately at the start and end positions, and the average is then added to the overall loss calculation. Since the learning difficulty varies for each task, two weights, α and β, are added to control for different differences; in the experiment, their values ​​were 0.5 and 0.8, respectively.

[0097] The example is as follows:

[0098] Figure 5This is a sample data in the CAIL 2020 reading comprehension dataset. Taking this sample as an example, the dynamic enhanced multi-hop reading comprehension method for low data corresponding to the present invention will be described below.

[0099] (1) Preprocess the document and perform semantic disambiguation.

[0100] (1) Use the labeled data in the training set to train a sentence screening model and an answer prediction model, and use them as teacher models; use the teacher models to randomly select a part of the unlabeled data, label them with pseudo-labels using the self-training method, and add them to the labeled data; use the obtained new training set to train a student model, and iterate the student model to the teacher model; repeat this process until the metrics on the validation set no longer improve; then test the test samples.

[0101] (1) Regarding Figure 5 For the shown example, after splitting the document into sentences according to punctuation, first obtain several desensitized names such as "Wu x0, Wu x2, Chen x3, Wu x6, Wu x9, Chang Yu 17", and then disambiguate the "Wu x2 2.1 mu, Chen x3 1.95 mu, Wu x6 1.98 mu, Wu x9 0.99 mu, Chang Yu 17 1.47 mu" in the 5th sentence according to the obtained names, and obtain the sentence "In 2001, the plaintiff Wu x0 subcontracted the land he contracted to Wu x2 for 2.1 mu, Chen x3 for 1.95 mu, Wu x6 for 1.98 mu, Wu x9 for 0.99 mu, and Chang Yu 17 for 1.47 mu".

[0102] (1) Pass each sentence context and the question through the screening model for screening respectively, obtain the relevance scores of each clause and the question, and save the K sentences with the highest relevance to the question answer in the order of sentence appearance. In this dataset, K is taken as 15, and the screened sentences obtained in this example are [2], [3], [4], [5], [9],

[10] ,

[11] ,

[12] ,

[13] ,

[14] ,

[15] ,

[17] ,

[18] ,

[19] ,

[21] .

[0103] (1) Concatenate the question and the sentences screened in (4) and input them into the answer prediction model to obtain the overall feature vector CLS_ALL, as well as the question vector Q and the context vector C; use the CLS_ALL vector to perform a four-classification task to predict the answer category, and the obtained answer category is extractive, so the answer needs to be extracted from the document. Reduce the dimension of the re-encoded question vector Q, then perform weighted summation with CLS_ALL in each dimension of C, and then use a multi-layer perceptron to change the output to 2 dimensions to predict the start and end positions of the answer respectively.

[0104] <00002​​Q′ = Attention_pooling(Q) ∈ R 1×d

[0106] C′ = Norm(w1·CLS_ALL + w2·Q′ + w3·C) ∈ R l×d

[0107]

[0108] In the formula, w represents the weight, d represents the dimension of the hidden vector, and l represents the length of the sentence.

[0109] (6) At the start and end positions, take 10 positions with the largest scores respectively, match them pairwise, remove the position pairs where the answer position is not in the context segment or the end position is at the start position, and take the pair with the largest sum of the scores of the start and end positions as the start and end positions of the final answer, obtaining the answer "Wu x6" to the question.

[0110] (7) Based on the Transformer structure, increase the dimension sizes of q and k during calculation to re-encode the sentence vectors. At the same time, use a parameter matrix to stack the sentence information obtained from each re-encoding for sentence feature fusion. Then use MLP to convert it into an n×1-dimensional output, and determine the sentences with each dimension higher than the threshold of 0.5 as the supporting sentences;

[0111] SFeature = W·AttentionPooling(C, CLS_ALL, Question)

[0112]

[0113] In the formula, W represents the weight matrix, which is initialized as a constant.

[0114] (8) Obtain the supporting sentences [3], [8], [9],

[11] from (7), and then map them back to [5],

[13] ,

[14] ,

[17] to obtain the final supporting sentences.

[0115] The above description of the embodiments is to facilitate those of ordinary skill in the art to understand and apply the present invention. Obviously, those skilled in the art can easily make various modifications to the above embodiments and apply the general principles described herein to other embodiments without creative efforts. Therefore, the present invention is not limited to the above embodiments, and the improvements and modifications made by those skilled in the art to the present invention should be within the protection scope of the present invention.

Claims

1. A low-data-oriented dynamic enhanced multi-hop text reading recognition processing method, characterized in that: The following steps are involved: Step 1: Preprocess the document dataset; Step 2: Build a dynamically enhanced answer prediction model; 2.

1. Separate the documents in the dataset into sentences, extract part or all of the sentences, and combine them with the questions corresponding to the documents to form the context text; 2.

2. Input the context text into the pre-trained answer prediction model to obtain the overall feature vector CLS_ALL of the context text and the decoded feature vectors of each word in the question and each sentence in the context text. The decoded feature vectors of each word in the question form the question vector Q, and the decoded feature vectors of each word in all sentences form the context vector C. 2.3、 A. After the overall feature vector CLS_ALL is processed by the linear layer, the predicted answer category type is obtained, which is expressed as the following formula: type=softmax(Linear(CLS_ALL) / τ)∈R 1×4 Among them, softmax() represents the softmax activation function, and τ represents the hyperparameter; B. Process the question vector Q and the context vector C in the following way to obtain the context feature vector C′, and then extract two results as the predicted answer start position start and answer end position end through the multi-layer perceptron: Q′=Attention_pooling(Q)∈R 1×d C′=Norm(w1·CLS_ALL+w2·Q′+w3·C)∈R l×d Where w1, w2, and w3 represent the first, second, and third weights, respectively; d represents the dimension of the latent vector; l represents the length of the sentence; Q′ represents the dimension reduction problem vector; Norm() represents the normalization function; MLP() represents the multi-layer perceptron; and R 1×4 The four dimensions corresponding to the four types of answers are represented, d represents the dimension of the hidden layer, and l represents the length of the context; Indicates the probability of whether each position in the context is the starting and ending position of the answer, corresponding to the answer starting position start and the answer ending position end respectively; C. The supporting sentence of the predicted output is obtained by processing the overall feature vector CLS_ALL, the question vector Q and the context vector C in the following manner: SFeature=W·AttentionPooling(C,CLS_ALL,Q) Where W represents the weight matrix; sigmoid() represents the sigmoid activation function, AttentionPooling() represents the attention pooling operation, Represents the supporting sentence of the predicted output; Step 3: Use the dataset with known labels processed in step 1 as the training set, use the training set to train the dynamically enhanced answer prediction model, and use the trained answer prediction model as the teacher model; Step 4: Randomly select a part of the unlabeled data set and input it into the teacher model obtained in step 3 to predict the label result. The predicted label is used as a pseudo label and attached to the unlabeled data set to form a data set with a pseudo label. Then, the data set with a pseudo label is added to the training set to form a new training set. Step 5: Use the new training set obtained in step 4 to retrain the teacher model obtained in step 3, and use the trained teacher model as the student model; Step 6: Use another dataset with known labels as a validation set, and input the validation set into the student model obtained in step 5 to test and obtain the model accuracy results. If the model accuracy result meets the preset threshold requirement, proceed to the next step; If the model accuracy result does not meet the preset threshold requirement, return to step 3 and use the student model as the teacher model for iterative processing, and repeat steps 3 to 5 for iteration until the model accuracy result obtained through the validation set meets the preset threshold requirement; Step 7: Use the student model obtained in step 6 to predict the test reading document, and output the predicted label of the test reading document and the answer therein.

2. The low-data-oriented dynamic enhanced multi-hop text reading recognition processing method according to claim 1, characterized in that: The dataset includes documents, questions and tags, and the tags include answers, supporting sentences and answer categories.

3. The low-data-oriented dynamic enhanced multi-hop text reading recognition processing method as claimed in claim 1, characterized in that: The step 7 specifically includes: Step 7.1, the test document is divided into sentences, and each sentence is filtered together with the question through the screening model to obtain the relevance between each sentence and the question, and the K sentences with the highest relevance are selected and arranged in the order in which the sentences appear in the test document to form a new document; Step 7.2: Input the new document into the student model obtained in step 6, output the predicted label of the new document, and extract the answer in the label as the answer to the reading document to be tested.

4. The low-data-oriented dynamic enhanced multi-hop text reading recognition processing method as claimed in claim 3, characterized in that: The screening model specifically uses the Chinese BERT pre-trained model, which encodes the question sentence pairs.

Images