Text analysis method and device

Abstract

The present specification provides a text analysis method and device, wherein the text analysis method comprises: splitting a text to be analyzed into multiple sentences, respectively calculating the similarity between each of the multiple sentences and the question to be answered; constructing an input sequence according to a target sentence whose similarity meets a preset condition and the question to be answered; obtaining the probability of each word unit in the input sequence being the starting position and the ending position of the answer corresponding to the question to be answered through a text analysis model, and determining the answer to the question to be answered according to the probabilities. The text analysis method according to the present specification shortens the length of the input sequence, reduces the amount of calculation of the text analysis model, and thus improves the prediction efficiency of the text analysis model.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a text analysis method and device. Background Art

[0002] Natural Language Processing (NLP) is an important direction in the fields of computer science and artificial intelligence. It studies various theories and methods that can enable effective communication between humans and computers using natural language.

[0003] For natural language processing tasks, the bidirectional attention neural network model (BidirectionalEncoder Representation from Transformers, BERT) is usually used for processing.

[0004] For machine question answering systems, in the prior art, text paragraphs and questions are input into the BERT model to predict the answers. The input sequence is long, so the training time and prediction time are long, resulting in low model efficiency. Summary of the invention

[0005] In view of this, the embodiment of this specification provides a text analysis method. This specification also relates to a text analysis device, a computing device, and a computer-readable storage medium to solve the technical defects existing in the prior art.

[0006] According to a first aspect of an embodiment of this specification, a text analysis method is provided, including:

[0007] Splitting the text to be analyzed into multiple sentences, and calculating the similarity between each of the multiple sentences and the question to be answered;

[0008] Constructing an input sequence according to the target sentence and the question to be answered whose similarity meets the preset condition;

[0009] The probability of each word unit in the input sequence being the starting position and the ending position of the answer corresponding to the question to be answered is obtained through the text analysis model, and the answer to the question to be answered is determined according to the probabilities.

[0010] Optionally, respectively calculating the similarity between each of the multiple statements and the question to be answered includes:

[0011] Perform character segmentation and vectorization on each sentence and question to be answered to obtain the word vector of each word unit;

[0012] Based on the word vectors of the word units in each sentence, calculate the word vector of each sentence;

[0013] Calculate the word vector of the question to be answered based on the word vector of the word unit in the question to be answered;

[0014] Based on the word vector of each sentence and the word vector of the question to be answered, calculate the similarity between each sentence and the question to be answered.

[0015] Optionally, respectively calculating the similarity between each of the multiple statements and the question to be answered includes:

[0016] Each sentence is concatenated with the question to be answered to form a concatenated text, wherein the concatenated text includes a first text segment and a second text segment;

[0017] Input the concatenated text into the similarity calculation model, perform character segmentation and vectorization processing on the concatenated text, and obtain the word vector of each word in the concatenated text;

[0018] Calculate the word vector of the first text segment according to the word vector of each word in the first text segment;

[0019] Calculate the word vector of the second text segment according to the word vector of each word in the second text segment;

[0020] Using the word vectors of the first text segment and the word vectors of the second text segment, calculate the similarity between each sentence and the question to be answered, and output a similarity identifier.

[0021] Optionally, constructing an input sequence according to the target sentence and the question to be answered whose similarity satisfies a preset condition comprises:

[0022] Determine a sentence with a similarity identifier as a first similarity identifier as a target sentence;

[0023] Performing character segmentation on the target sentence and the question to be answered to obtain each word unit in the target sentence and the question to be answered;

[0024] Build the input sequence based on each word unit.

[0025] Optionally, constructing an input sequence according to the target sentence and the question to be answered whose similarity satisfies a preset condition comprises:

[0026] Determine a sentence with a similarity identifier as a first similarity segment identifier as a target sentence;

[0027] Performing character segmentation on the target sentence and the question to be answered to obtain each word unit in the target sentence and the question to be answered;

[0028] Build the input sequence based on each word unit.

[0029] Optionally, constructing an input sequence according to each word unit includes:

[0030] An input sequence is constructed by placing the word units of the target sentence in front and the word units of the question to be answered in the back.

[0031] Optionally, obtaining, by means of a text analysis model, a probability of each word unit in the input sequence being a starting position and an ending position of an answer corresponding to the question to be answered, includes:

[0032] Calculate a word vector, a separation vector, and a position vector for each word unit in the input sequence;

[0033] Linear mapping and nonlinear transformation are performed on the sum of the word vector, the separation vector and the position vector to obtain the probability of each word unit being the starting position and the ending position of the answer corresponding to the question to be answered.

[0034] Optionally, determining the answer to the question to be answered according to the probability includes:

[0035] The first word unit with the highest probability of being the starting position, the second word unit with the highest probability of being the ending position, and the word units between the first word unit and the second word unit are determined as the answer to the question to be answered.

[0036] Optionally, the text analysis model is trained by the following method:

[0037] Obtaining training samples and sample labels, wherein the training samples include sample texts and sample questions, and the sample labels include correct answers corresponding to the sample texts and sample questions;

[0038] Splitting the sample text into multiple sentences, and calculating the similarity between each of the multiple sentences and the sample question;

[0039] Constructing an input sequence according to the target sentence and the sample question whose similarity meets the preset condition;

[0040] Inputting the input sequence into a text analysis model, obtaining the probability of each word unit in the input sequence being the starting position and the ending position of the answer corresponding to the sample question, and determining the predicted answer to the sample question according to the probabilities;

[0041] The predicted answer is compared with the correct answer, and based on the comparison result between the predicted answer and the correct answer, the text analysis model is updated.

[0042] According to a second aspect of the embodiments of this specification, a text analysis device is provided, including:

[0043] A calculation module is configured to split the text to be analyzed into multiple sentences, and respectively calculate the similarity between each of the multiple sentences and the question to be answered;

[0044] A construction module, configured to construct an input sequence according to a target sentence whose similarity satisfies a preset condition and the question to be answered;

[0045] The determination module is configured to obtain the probability of each word unit in the input sequence being the starting position and the ending position of the answer corresponding to the question to be answered through a text analysis model, and determine the answer to the question to be answered according to the probabilities.

[0046] Optionally, the calculation module is further configured to:

[0047] Perform character segmentation and vectorization on each sentence and question to be answered to obtain the word vector of each word unit;

[0048] Based on the word vectors of the word units in each sentence, calculate the word vector of each sentence;

[0049] Calculate the word vector of the question to be answered based on the word vector of the word unit in the question to be answered;

[0050] Based on the word vector of each sentence and the word vector of the question to be answered, calculate the similarity between each sentence and the question to be answered.

[0051] Optionally, the calculation module is further configured to:

[0052] Each sentence is concatenated with the question to be answered to form a concatenated text, wherein the concatenated text includes a first text segment and a second text segment;

[0053] Input the concatenated text into the similarity calculation model, perform character segmentation and vectorization processing on the concatenated text, and obtain the word vector of each word in the concatenated text;

[0054] Calculate the word vector of the first text segment according to the word vector of each word in the first text segment;

[0055] Calculate the word vector of the second text segment according to the word vector of each word in the second text segment;

[0056] Using the word vectors of the first text segment and the word vectors of the second text segment, calculate the similarity between each sentence and the question to be answered, and output a similarity identifier.

[0057] Optionally, the building block is further configured to:

[0058] Determine a sentence with a similarity identifier as a first similarity identifier as a target sentence;

[0059] Performing character segmentation on the target sentence and the question to be answered to obtain each word unit in the target sentence and the question to be answered;

[0060] Build the input sequence based on each word unit.

[0061] Optionally, the building block is further configured to:

[0062] Determine a sentence with a similarity identifier as a first similarity segment identifier as a target sentence;

[0063] Performing character segmentation on the target sentence and the question to be answered to obtain each word unit in the target sentence and the question to be answered;

[0064] Build the input sequence based on each word unit.

[0065] Optionally, the building block is further configured to:

[0066] An input sequence is constructed by placing the word units of the target sentence in front and the word units of the question to be answered in the back.

[0067] Optionally, the determining module is further configured to:

[0068] Calculate a word vector, a separation vector, and a position vector for each word unit in the input sequence;

[0069] Linear mapping and nonlinear transformation are performed on the sum of the word vector, the separation vector and the position vector to obtain the probability of each word unit being the starting position and the ending position of the answer corresponding to the question to be answered.

[0070] Optionally, the determining module is further configured to:

[0071] The first word unit with the highest probability of being the starting position, the second word unit with the highest probability of being the ending position, and the word units between the first word unit and the second word unit are determined as the answer to the question to be answered.

[0072] Optionally, the text analysis device further includes a training module, and the training module is configured to:

[0073] Obtaining training samples and sample labels, wherein the training samples include sample texts and sample questions, and the sample labels include correct answers corresponding to the sample texts and sample questions;

[0074] Splitting the sample text into multiple sentences, and calculating the similarity between each of the multiple sentences and the sample question;

[0075] Constructing an input sequence according to the target sentence and the sample question whose similarity meets the preset condition;

[0076] Inputting the input sequence into a text analysis model, obtaining the probability of each word unit in the input sequence being the starting position and the ending position of the answer corresponding to the sample question, and determining the predicted answer to the sample question according to the probabilities;

[0077] The predicted answer is compared with the correct answer, and based on the comparison result between the predicted answer and the correct answer, the text analysis model is updated.

[0078] According to a third aspect of an embodiment of this specification, a computing device is provided, including:

[0079] Memory and processor;

[0080] The memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions:

[0081] Splitting the text to be analyzed into multiple sentences, and calculating the similarity between each of the multiple sentences and the question to be answered;

[0082] Constructing an input sequence according to the target sentence and the question to be answered whose similarity meets the preset condition;

[0083] The probability of each word unit in the input sequence being the starting position and the ending position of the answer corresponding to the question to be answered is obtained through the text analysis model, and the answer to the question to be answered is determined according to the probabilities.

[0084] According to a fourth aspect of the embodiments of this specification, a computer-readable storage medium is provided, which stores computer-executable instructions, and when the instructions are executed by a processor, the steps of the text analysis method are implemented.

[0085] According to a fifth aspect of the embodiments of this specification, a chip is provided, which stores computer instructions, and when the instructions are executed by a processor, the steps of the text analysis method as described above are implemented.

[0086] The text analysis method provided in this specification is to split the text to be analyzed into multiple sentences, and respectively calculate the similarity between each sentence and the question to be answered; construct an input sequence based on the target sentence and the question to be answered whose similarity meets the preset conditions; obtain the probability of each word unit in the input sequence being the starting position and the ending position of the answer corresponding to the question to be answered through the text analysis model, and determine the answer to the question to be answered based on the probability. The text analysis method according to this specification does not directly construct an input sequence from paragraphs and questions, but instead constructs an input sequence from sentences and questions with high similarity after text splitting, which shortens the length of the input sequence, reduces the amount of calculation of the text analysis model, and thus improves the prediction efficiency of the text analysis model. BRIEF DESCRIPTION OF THE DRAWINGS

[0087] Figure 1 is a flow chart of a text analysis method provided in an embodiment of this specification;

[0088] Figure 2 is a schematic diagram of an input sequence of a text analysis method provided in an embodiment of this specification;

[0089] Figure 3 It is a schematic diagram of determining an answer according to a start position probability and an end position probability in a text analysis method provided in an embodiment of this specification;

[0090] Figure 4 It is a structural schematic diagram of a text analysis device provided in an embodiment of this specification;

[0091] Figure 5 It is a structural block diagram of a computing device provided in an embodiment of this specification. DETAILED DESCRIPTION

[0092] Many specific details are described in the following description to facilitate a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of this specification, so this specification is not limited to the specific implementation disclosed below.

[0093] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "a", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0094] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "in response to determining".

[0095] First, the terms involved in one or more embodiments of this specification are explained.

[0096] BERT model: A bidirectional attention neural network model proposed by Google in October 2018. The BERT model concatenates questions and articles, and uses the attention mechanism to obtain article information and the interdependence between questions and articles, thereby obtaining the interdependence representation vector of each word unit in the question and article, and finally obtains the probability of each word unit being the starting position and ending position of the answer through linear mapping and nonlinear transformation.

[0097] Token: Before any actual processing is done on the input text, it needs to be segmented into language units such as words, punctuation marks, numbers, or pure alphanumeric characters. These units are called tokens. For English text, a token can be a word, a punctuation mark, a number, etc. For Chinese text, the smallest token can be a character, a punctuation mark, a number, etc.

[0098] Word embedding: It means embedding a high-dimensional space with the dimension of all words into a continuous vector space with much lower dimension, where each word or phrase is mapped to a vector in the real number field.

[0099] Normalized exponential function Softmax: It is a generalization of the logical function. It can "compress" a K-dimensional vector containing any real number into another K-dimensional real vector so that the range of each element is between [0,1] and the sum of all elements is 1.

[0100] In this specification, a text analysis method is provided. This specification also relates to a text analysis device, a computing device, and a computer-readable storage medium, which are described in detail one by one in the following embodiments.

[0101] Figure 1 A processing flow chart of a text analysis method provided in an embodiment of the present specification is shown, which specifically includes steps 102 to 106.

[0102] Step 102: Split the text to be analyzed into multiple sentences, and calculate the similarity between each of the multiple sentences and the question to be answered.

[0103] The text to be analyzed can be a paragraph, multiple paragraphs, an article, etc. The text to be analyzed and the questions are given in the data set. For the extractive machine question-answering system, the questions are related to the text to be analyzed, and the original answer will exist in the text to be analyzed. For example, the text to be analyzed is the introduction of Company A, "Company A was founded by person B in 1988. The company is headquartered in City C. Company A has two subsidiaries, D and E, with branches in City F and City G respectively. The product line covers the H field, with well-known products such as I and G." The questions to be answered can be, for example, "Who founded Company A", "When was Company A founded", "How many subsidiaries does Company A have", etc.

[0104] The text to be analyzed can be split only according to the sentence terminators such as periods and exclamation marks, or according to the middle separators such as commas and semicolons and the sentence terminators. For example, the introduction of Company A mentioned above can be divided into 3 sentences according to the period, or it can be divided into 6 sentences according to the middle separators and the sentence terminators.

[0105] Take the example of the question "Who founded Company A?", which is segmented only by the sentence endings, and we get three sentences: "Company A was founded by person B in 1988.", "The company is headquartered in City C, and Company A has two subsidiaries, D and E, with branches in City F and City G respectively.", and "The product line covers the field H, with well-known products such as I and G." Then, the text similarity between each segmented sentence and the question is calculated using a text similarity algorithm or a pre-trained deep learning model.

[0106] The text similarity algorithm can adopt edit distance algorithms such as Jaro Winkler Distance, Levenshtein Distance, Jaccard coefficient, etc. The algorithms listed here are only examples and are not limited to this specification. Specifically, after being divided into multiple sentences, each sentence is segmented, and the segmentation can be performed by a word or a character to obtain multiple word units in each sentence, and then the word vector of each word unit is calculated. According to the word vector of each word unit in each sentence, the word embedding of each sentence can be obtained. Similarly, the question to be answered is segmented to obtain multiple word units in the question to be answered, and the word vector of each word unit is calculated. According to the word vector of each word unit in the question to be answered, the word vector of the question to be answered is obtained. After that, the similarity is calculated for the word vector of each sentence and the word vector of the question to be answered using edit distance algorithms such as Jaro Winkler Distance, Levenshtein Distance, and Jaccard coefficient.

[0107] The following is an explanation of how to calculate text similarity using the BERT deep learning model. First, the specific structure of the BERT model is described. The BERT model can include n encoding layers, which are connected in sequence. Each encoding layer also includes: a self-attention module, a normalization module, a fully connected module, and a normalization module. Each sentence and question that has been split is input into the BERT model respectively. The embedding layer encodes each word in the input sequence into a word vector. The separation vector, for example, consists of only 0 and 1, which is used to identify different sentences. The position vector is calculated by the formula, and then the sum of the three vectors is input into multiple encoding layers of the BERT model. Different encoding layers can obtain the similarity between the word embedding of each sentence and the word embedding of the question to be answered by learning different matrix parameters.

[0108] Specifically, when calculating the similarity between the first sentence in the text to be analyzed and the question to be answered, the first sentence and the question to be answered are concatenated to form a concatenated text, and the first sentence and the question to be answered respectively form the first text segment and the second text segment in the concatenated text; the concatenated text is input into the BERT deep learning model, and the concatenated text formed by the concatenation is subjected to character segmentation and vectorization processing to obtain the word vector of each word in the concatenated text; the word vector of the first sentence is calculated using the word vector of each word in the first sentence, and the word vector of the second sentence is calculated using the word vector of each word in the second sentence, and then the word vector of the first sentence is used to calculate the similarity between the word vector of the question to be answered, and a similarity mark is output. The similarity is calculated for each sentence in the paragraph and the question to be answered using the above method.

[0109] In one embodiment, the first sentence and the question to be answered are separated by [SEP], the first sentence corresponding position vector (including [CLS], [SEP]) is all 0, the question to be answered corresponding position vector (including the end [SEP]) is all 1, the first sentence and the question to be answered are input into the trained similarity calculation model, and the first sentence after concatenation and the question to be answered are obtained as similar and dissimilar. When training the similarity calculation model, the sample sentence, the sample question and the label are input into the model, and the label is whether the sample sentence and the sample question are similar, for example, the similar label is 1, and the dissimilar label is 0.

[0110] In another embodiment, the first sentence and the question to be answered are separated by [SEP], the corresponding position vector of the first sentence (including [CLS], [SEP]) is all 0, and the corresponding position vector of the question to be answered (including the ending [SEP]) is all 1, and the first sentence and the question to be answered are input into the trained similarity prediction model, and the similarity between the spliced ​​first sentence and the question to be answered is obtained in the similarity range of 50% to 70%, 70% to 90%, 90% to 100%. When training the similarity prediction model, sample sentences, sample questions and labels are input into the model, and the label is the similarity range of the two sentences, such as 0% to 50%, 50% to 70%, 70% to 90%, 90% to 100%.

[0111] Step 104: construct an input sequence according to the target sentence whose similarity meets the preset condition and the question to be answered.

[0112] In one embodiment, constructing an input sequence based on a target sentence and the question to be answered whose similarities satisfy preset conditions can be achieved as follows: determining a sentence with a similarity identifier of a first similarity identifier as a target sentence; performing character segmentation on the target sentence and the question to be answered to obtain each word unit in the target sentence and the question to be answered; and constructing an input sequence based on each word unit.

[0113] In another embodiment, constructing an input sequence based on a target sentence and the question to be answered whose similarities satisfy preset conditions can be achieved as follows: determining a sentence whose similarity is identified as a first similarity segment identifier as a target sentence; performing character segmentation on the target sentence and the question to be answered to obtain each word unit in the target sentence and the question to be answered; and constructing an input sequence based on each word unit.

[0114] In the above embodiment where the concatenated sentences and the questions to be answered are similar or dissimilar, multiple sentences with similar results and the questions to be answered are used to construct an input sequence. In the above embodiment where the similarity between the first concatenated sentence and the questions to be answered is in a similarity range of, for example, 0% to 50%, 50% to 70%, 70% to 90%, or 90% to 100%, for example, multiple sentences with a similarity range of 70% to 100% or 90% to 100% and the questions to be answered can be used to construct an input sequence.

[0115] One or more target sentences whose similarity meets the preset conditions are concatenated with the question, and sentences whose similarity does not meet the preset conditions do not participate in predicting the answer. In one embodiment, constructing an input sequence according to each word unit can be achieved by placing the word unit of the target sentence in front and the word unit of the question to be answered in the back to construct the input sequence. In actual operation, constructing an input sequence according to the target sentence and the question can be achieved by adding the [CLS] symbol at the beginning of the first sentence, and adding the [SEP] ending symbol at the end of each sentence and question, that is, the first word of the input is the starting symbol [CLS], which can be used for subsequent prediction tasks, and the last word is the ending symbol [SEP].

[0116] Step 106: Obtain, through a text analysis model, the probability of each word unit in the input sequence being the starting position and the ending position of the answer corresponding to the question to be answered, and determine the answer to the question to be answered based on the probabilities.

[0117] Step 106 can be implemented as follows: input the input sequence into a pre-trained BERT model, and calculate the word vector, separation vector, and position vector for each word unit in the input sequence; perform linear mapping and nonlinear transformation on the sum of the word vector, separation vector, and position vector, and obtain the probability of each word unit being the starting position and the probability of the ending position of the answer to the question to be answered, and determine the answer to the question to be answered based on the probability.

[0118] The input sequence is input into the BERT model, and the word vector, separation vector, and position vector of each word in the above one or more sentences and questions are calculated through the embedding layer of the BERT model.

[0119] In this embodiment, each sentence and question is segmented to obtain each character unit, and the character unit is embedded to convert each character unit into a fixed-dimensional word vector, i.e., word embedding. Embedding the character unit of each sentence and question can improve the extraction depth and richness of the text and question to be analyzed, and enhance the accuracy of analysis. The separation vector is a vector used to indicate which sentence the word belongs to. For example, the value of the separation vector of the words in all sentences is 0, and the separation vector of the words in the question is 1. The position vector is a vector representing the position, which allows the BERT model to learn a vector representation at each position to take into account the information of the sequence order, that is, to learn the sequential properties of the input.

[0120] The following example uses the sentence "Company A was founded by person B in 1988" and the question "Who founded Company A?" as an example. Figure 2As shown in the figure, after calculating the word embedding processing for each word in the sentence and the question respectively, the word vector, separation vector and position vector corresponding to each word are obtained, and the sum of these three vectors is input into the encoding layer of the BERT model, where the word vector represents the vector corresponding to each word unit, the letter subscript identifies the corresponding word unit, the letters in the separation vector represent different sentences, the first sentence is A and the question is B, and the Arabic numerals in the position vector represent the position of the word unit in the entire input sequence. When determining the position of the word unit in the input sequence, the text to be analyzed can be placed in front of the question to be answered, or the text to be analyzed can be placed behind the question to be answered.

[0121] After linear transformation and dimensional transformation of the word vector corresponding to each word unit, we get the K matrix (key matrix), Q matrix (query matrix) and V matrix (value matrix), multiply the Q matrix by the K matrix, multiply by a constant, perform a softmax operation, and finally multiply by the V matrix to calculate the attention score vector. In practical applications, the attention score vector corresponding to each word unit can be linearly mapped and nonlinearly transformed to obtain the probability of each word unit being the starting position and the ending position of the answer to the question.

[0122] Specifically, linear mapping is a mapping from one vector space V to another vector space W. Through linear mapping, the dimension of word vector is transformed to the dimension of sentence vector. Non-linear transformation is to transform the original feature vector nonlinearly to obtain a new feature vector. Using this new feature vector to perform linear classification corresponds to the original feature space, which is equivalent to performing nonlinear classification.

[0123] In practical applications, it can be implemented through various methods such as Softmax, and this application does not impose any restrictions on this.

[0124] The specific structure of the BERT model is described in detail below. The BERT model can include n encoding layers (attention layers), which are connected in sequence. Each encoding layer also includes: a self-attention module, a normalization module, a fully connected module, and a normalization module. The input sequence consisting of the target sentence and the question to be answered is input into the BERT model. The embedding layer encodes each word in the input sequence into a word vector. The separation vector consists of only 0 and 1, which is used to identify different sentences. The position vector is calculated by the formula, and then the sum of the three vectors is input into multiple encoding layers of the BERT model. Different encoding layers can obtain different information and features in the text by learning different matrix parameters. The last encoding layer outputs the starting probability and ending probability of the answer.

[0125] When n words are input, the BERT model outputs an n-dimensional vector representing the probability of each word being the starting position of the answer and an n-dimensional vector representing the probability of each word being the ending position of the answer, i.e., a start probability vector and an end probability vector. Each probability in the start probability vector indicates the probability that the n words input are the starting words of the answer, and each probability in the end probability vector indicates the probability that the n words input are the ending words of the answer. The word with the highest probability as the starting position and the word with the highest probability as the ending position and the words between them are taken as the answer to the question. Figure 2 Taking the sentence and question shown in the figure as an example, after inputting the word vector, separation vector, and position vector of each word in the sentence and question into the pre-trained BERT model, it is found that the 6th word unit has the highest probability as the starting position of the answer, and the 8th word unit has the highest probability as the ending position of the answer, that is, the word with the highest starting probability in the obtained starting probability vector is "B", and the word with the highest ending probability in the obtained ending probability vector is "member", so "B personnel" is taken as the answer to the question.

[0126] The present embodiment is further described below with reference to specific examples. For example, suppose the text to be analyzed includes "The Silk Road was a channel for transportation, trade and cultural exchanges between ancient China and foreign countries", and the question includes "Which dynasty did the Silk Road originate from?". The text to be analyzed and the question to be answered are segmented to generate an input sequence [CLS, silk, silk... dynasty, era, SEP]. The above input sequence is input into the text analysis model to obtain the attention score vector [E1, E2, E3, E4... E 37 、E 38 ], the above attention score vector is linearly mapped and nonlinearly transformed, and the probability of the 7th word unit being the starting position of the answer is the highest, and the probability of the 10th word unit being the ending position of the answer is the highest, generating the answer "Ancient China".

[0127] Combine the following Figure 3 To explain, Figure 3 As shown, the input is w1 to w7, and two sequences (vectors) are output. One sequence records the probability of each input character being the starting point of the answer, w3 is the largest, with a probability of 0.7, and the other sequence records the probability of each input character being the ending point of the answer, w5 is the largest, with a probability of 0.6. The answer is based on the character with the highest probability of being the starting point of the answer, the character with the highest probability of being the ending point of the answer, and the characters in between. The answer is w3w4w5.

[0128] The above describes the specific implementation method of determining the answer to the question to be answered through the text analysis model. The following describes the training method of the text analysis model. The text analysis model is trained by the following method:

[0129] Obtaining training samples and sample labels, wherein the training samples include sample texts and sample questions, and the sample labels include correct answers corresponding to the sample texts and sample questions;

[0130] Splitting the sample text into multiple sentences, and calculating the similarity between each of the multiple sentences and the sample question;

[0131] Constructing an input sequence according to the target sentence and the sample question whose similarity meets the preset condition;

[0132] Inputting the input sequence into a text analysis model, obtaining the probability of each word unit in the input sequence being the starting position and the ending position of the answer corresponding to the sample question, and determining the predicted answer to the sample question according to the probabilities;

[0133] The predicted answer is compared with the correct answer, and based on the comparison result between the predicted answer and the correct answer, the text analysis model is updated.

[0134] Specifically, the sample text is a written text containing certain information content. It can be a sentence, a paragraph, multiple paragraphs, an article or multiple articles, etc. It can also be a text in various languages ​​such as Chinese text, English text, Russian text, etc. This application does not impose any restrictions on this.

[0135] Sample questions are questions that require answers or explanations and are related to the information content in the sample text.

[0136] The sample label includes the correct answer corresponding to the sample text and the sample question. It should be noted that the correct answer is usually the content in the sample text. If the answer to the sample question does not exist in the sample text, the correct answer is blank. However, if the sample question can be answered correctly based on common knowledge, the correct answer can be the correct answer obtained based on common knowledge, and this application does not limit this.

[0137] The predicted answer is compared with the correct answer. If the predicted answer is inconsistent with the correct answer, the parameters of the text analysis model are adjusted, the text analysis model is updated, and the iterative training continues; if the predicted answer is consistent with the correct answer, the training ends.

[0138] Corresponding to the above method embodiment, this specification also provides a text analysis device embodiment, Figure 4 FIG. 1 shows a schematic diagram of the structure of a text analysis device provided in an embodiment of the present specification. Figure 4 As shown, the device comprises:

[0139] A calculation module 402 is configured to split the text to be analyzed into multiple sentences, and respectively calculate the similarity between each of the multiple sentences and the question to be answered;

[0140] A construction module 404 is configured to construct an input sequence according to the target sentence whose similarity meets a preset condition and the question to be answered;

[0141] The determination module 406 is configured to obtain the probability of each word unit in the input sequence being the starting position and the ending position of the answer corresponding to the question to be answered through a text analysis model, and determine the answer to the question to be answered according to the probabilities.

[0142] Optionally, the calculation module 402 is further configured to:

[0143] Perform character segmentation and vectorization on each sentence and question to be answered to obtain the word vector of each word unit;

[0144] Calculate the word vector of each sentence based on the word vector of the word unit in each sentence;

[0145] Calculate the word vector of the question to be answered based on the word vector of the word unit in the question to be answered;

[0146] Based on the word vector of each sentence and the word vector of the question to be answered, calculate the similarity between each sentence and the question to be answered.

[0147] Optionally, the calculation module 402 is further configured to:

[0148] Each sentence is concatenated with the question to be answered to form a concatenated text, wherein the concatenated text includes a first text segment and a second text segment;

[0149] Input the concatenated text into the similarity calculation model, perform character segmentation and vectorization processing on the concatenated text, and obtain the word vector of each word in the concatenated text;

[0150] Calculate the word vector of the first text segment according to the word vector of each word in the first text segment;

[0151] Calculate the word vector of the second text segment according to the word vector of each word in the second text segment;

[0152] Using the word vectors of the first text segment and the word vectors of the second text segment, calculate the similarity between each sentence and the question to be answered, and output a similarity identifier.

[0153] Optionally, the construction module 404 is further configured to:

[0154] Determine a sentence with a similarity identifier as a first similarity identifier as a target sentence;

[0155] Performing character segmentation on the target sentence and the question to be answered to obtain each word unit in the target sentence and the question to be answered;

[0156] Build the input sequence based on each word unit.

[0157] Optionally, the construction module 404 is further configured to:

[0158] Determine a sentence with a similarity identifier as a first similarity segment identifier as a target sentence;

[0159] Performing character segmentation on the target sentence and the question to be answered to obtain each word unit in the target sentence and the question to be answered;

[0160] Build the input sequence based on each word unit.

[0161] Optionally, the construction module 404 is further configured to:

[0162] An input sequence is constructed by placing the word units of the target sentence in front and the word units of the question to be answered in the back.

[0163] Optionally, the determining module 406 is further configured to:

[0164] Calculate a word vector, a separation vector, and a position vector for each word unit in the input sequence;

[0165] Linear mapping and nonlinear transformation are performed on the sum of the word vector, the separation vector and the position vector to obtain the probability of each word unit being the starting position and the ending position of the answer corresponding to the question to be answered.

[0166] Optionally, the determining module 406 is further configured to:

[0167] The first word unit with the highest probability of being the starting position, the second word unit with the highest probability of being the ending position, and the word units between the first word unit and the second word unit are determined as the answer to the question to be answered.

[0168] Optionally, the text analysis device further includes a training module, and the training module is configured to:

[0169] Obtaining training samples and sample labels, wherein the training samples include sample texts and sample questions, and the sample labels include correct answers corresponding to the sample texts and sample questions;

[0170] Splitting the sample text into multiple sentences, and calculating the similarity between each of the multiple sentences and the sample question;

[0171] Constructing an input sequence according to the target sentence and the sample question whose similarity meets the preset condition;

[0172] Inputting the input sequence into a text analysis model, obtaining the probability of each word unit in the input sequence being the starting position and the ending position of the answer corresponding to the sample question, and determining the predicted answer to the sample question according to the probabilities;

[0173] The predicted answer is compared with the correct answer, and based on the comparison result between the predicted answer and the correct answer, the text analysis model is updated.

[0174] The above is a schematic scheme of a text analysis device of this embodiment. It should be noted that the technical scheme of the text analysis device and the technical scheme of the above text analysis method belong to the same concept, and the details not described in detail in the technical scheme of the text analysis device can be referred to the description of the technical scheme of the above text analysis method.

[0175] It should be noted that the components in the device claim should be understood as the functional modules that must be established to implement the steps of the program flow or the steps of the method, and the functional modules are not actually divided or separated. The device claim defined by such a group of functional modules should be understood as the functional module architecture of the solution mainly implemented by the computer program recorded in the specification, and should not be understood as the physical device that mainly implements the solution by hardware.

[0176] Figure 5 The structure block diagram of a computing device 500 provided according to an embodiment of the present specification is shown. The components of the computing device 500 include but are not limited to a memory 510 and a processor 520. The processor 520 is connected to the memory 510 via a bus 530, and the database 550 is used to store data.

[0177] The computing device 500 also includes an access device 540 that enables the computing device 500 to communicate via one or more networks 560. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 540 may include one or more of any type of network interface (e.g., a network interface card (NIC)) whether wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a World Wide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, and the like.

[0178] In one embodiment of the present specification, the above components of the computing device 500 and Figure 5 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Figure 5 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0179] The computing device 500 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or PC. The computing device 500 may also be a mobile or stationary server.

[0180] The processor 520 is used to execute the following computer executable instructions:

[0181] Splitting the text to be analyzed into multiple sentences, and calculating the similarity between each of the multiple sentences and the question to be answered;

[0182] Constructing an input sequence according to the target sentence and the question to be answered whose similarity meets the preset condition;

[0183] The probability of each word unit in the input sequence being the starting position and the ending position of the answer corresponding to the question to be answered is obtained through the text analysis model, and the answer to the question to be answered is determined according to the probabilities.

[0184] The above is a schematic scheme of a computing device of this embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the above text analysis method belong to the same concept, and the details not described in detail in the technical scheme of the computing device can be referred to the description of the technical scheme of the above text analysis method.

[0185] An embodiment of the present specification further provides a computer-readable storage medium storing computer instructions, which are used when executed by a processor to:

[0186] Splitting the text to be analyzed into multiple sentences, and calculating the similarity between each of the multiple sentences and the question to be answered;

[0187] Constructing an input sequence according to the target sentence and the question to be answered whose similarity meets the preset condition;

[0188] The probability of each word unit in the input sequence being the starting position and the ending position of the answer corresponding to the question to be answered is obtained through the text analysis model, and the answer to the question to be answered is determined according to the probabilities.

[0189] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the above-mentioned text analysis method belong to the same concept, and the details not described in detail in the technical scheme of the storage medium can be referred to the description of the technical scheme of the above-mentioned text analysis method.

[0190] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0191] The computer instructions include computer program codes, which may be in source code form, object code form, executable files or some intermediate forms, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0192] It should be noted that, for the convenience of description, the aforementioned method embodiments are all described as a series of action combinations, but those skilled in the art should be aware that this specification is not limited by the order of the actions described, because according to this specification, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this specification.

[0193] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0194] The preferred embodiments of this specification disclosed above are only used to help explain this specification. The optional embodiments do not describe all the details in detail, nor do they limit the invention to only the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of this specification, so that technicians in the relevant technical field can well understand and use this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A text analysis method, characterized in that: include: Splitting the text to be analyzed into multiple sentences, and calculating the similarity between each of the multiple sentences and the question to be answered; Constructing an input sequence according to the target sentences whose similarities meet the preset conditions and the questions to be answered, wherein constructing the input sequence is to splice one or more target sentences whose similarities meet the preset conditions with the questions to be answered; The probability of each word unit in the input sequence being the starting position and the ending position of the answer corresponding to the question to be answered is obtained through the text analysis model, and the answer to the question to be answered is determined according to the probabilities.

2. The text analysis method according to claim 1, characterized in that: Calculating the similarity between each of the multiple statements and the question to be answered includes: Perform character segmentation and vectorization on each sentence and question to be answered to obtain the word vector of each word unit; Calculate the word vector of each sentence based on the word vector of the word unit in each sentence; Calculate the word vector of the question to be answered based on the word vector of the word unit in the question to be answered; Based on the word vector of each sentence and the word vector of the question to be answered, calculate the similarity between each sentence and the question to be answered.

3. The text analysis method according to claim 1, characterized in that: Calculating the similarity between each of the multiple statements and the question to be answered includes: Each sentence is concatenated with the question to be answered to form a concatenated text, wherein the concatenated text includes a first text segment and a second text segment; Input the concatenated text into the similarity calculation model, perform character segmentation and vectorization processing on the concatenated text, and obtain the word vector of each word in the concatenated text; Calculate the word vector of the first text segment according to the word vector of each word in the first text segment; Calculate the word vector of the second text segment according to the word vector of each word in the second text segment; Using the word vectors of the first text segment and the word vectors of the second text segment, calculate the similarity between each sentence and the question to be answered, and output a similarity identifier.

4. The text analysis method according to claim 3, characterized in that: The step of constructing an input sequence based on the target sentence and the question to be answered that meet the preset conditions in similarity includes: Determine a sentence with a similarity identifier as a first similarity identifier as a target sentence; Performing character segmentation on the target sentence and the question to be answered to obtain each word unit in the target sentence and the question to be answered; Build the input sequence based on each word unit.

5. The text analysis method according to claim 3, characterized in that: The step of constructing an input sequence based on the target sentence and the question to be answered that meet the preset conditions in similarity includes: Determine a sentence with a similarity identifier as a first similarity segment identifier as a target sentence; Performing character segmentation on the target sentence and the question to be answered to obtain each word unit in the target sentence and the question to be answered; Build the input sequence based on each word unit.

6. The text analysis method according to claim 4 or 5, characterized in that: The constructing of the input sequence according to each word unit comprises: An input sequence is constructed by placing the word units of the target sentence in front and the word units of the question to be answered in the back.

7. The text analysis method according to claim 1, characterized in that: Obtaining the probability of each word unit in the input sequence being the starting position and the ending position of the answer corresponding to the question to be answered through the text analysis model includes: Calculate a word vector, a separation vector, and a position vector for each word unit in the input sequence; Linear mapping and nonlinear transformation are performed on the sum of the word vector, the separation vector and the position vector to obtain the probability of each word unit being the starting position and the ending position of the answer corresponding to the question to be answered.

8. The text analysis method according to claim 7, characterized in that: Determining the answer to the question to be answered according to the probability includes: The first word unit with the highest probability of being the starting position, the second word unit with the highest probability of being the ending position, and the word units between the first word unit and the second word unit are determined as the answer to the question to be answered.

9. The text analysis method according to claim 1 or 2, characterized in that: The text analysis model is trained by the following method: Obtaining training samples and sample labels, wherein the training samples include sample texts and sample questions, and the sample labels include correct answers corresponding to the sample texts and sample questions; Splitting the sample text into multiple sentences, and calculating the similarity between each of the multiple sentences and the sample question; Constructing an input sequence according to the target sentence and the sample question whose similarity meets the preset condition; Inputting the input sequence into a text analysis model, obtaining the probability of each word unit in the input sequence being the starting position and the ending position of the answer corresponding to the sample question, and determining the predicted answer to the sample question according to the probabilities; The predicted answer is compared with the correct answer, and based on the comparison result between the predicted answer and the correct answer, the text analysis model is updated.

10. A text analysis device, characterized in that: include: A calculation module is configured to split the text to be analyzed into multiple sentences, and respectively calculate the similarity between each of the multiple sentences and the question to be answered; A construction module is configured to construct an input sequence according to the target sentences whose similarities meet the preset conditions and the questions to be answered, wherein the input sequence is constructed by concatenating one or more target sentences whose similarities meet the preset conditions with the questions to be answered; The determination module is configured to obtain the probability of each word unit in the input sequence being the starting position and the ending position of the answer corresponding to the question to be answered through a text analysis model, and determine the answer to the question to be answered according to the probabilities.

11. The text analysis device according to claim 10, characterized in that: The computing module is further configured to: Perform character segmentation and vectorization on each sentence and question to be answered to obtain the word vector of each word unit; Calculate the word vector of each sentence based on the word vector of the word unit in each sentence; Calculate the word vector of the question to be answered based on the word vector of the word unit in the question to be answered; Based on the word vector of each sentence and the word vector of the question to be answered, calculate the similarity between each sentence and the question to be answered.

12. The text analysis device according to claim 10, characterized in that: The computing module is further configured to: Each sentence is concatenated with the question to be answered to form a concatenated text, wherein the concatenated text includes a first text segment and a second text segment; Input the concatenated text into the similarity calculation model, perform character segmentation and vectorization processing on the concatenated text, and obtain the word vector of each word in the concatenated text; Calculate the word vector of the first text segment according to the word vector of each word in the first text segment; Calculate the word vector of the second text segment according to the word vector of each word in the second text segment; Using the word vectors of the first text segment and the word vectors of the second text segment, calculate the similarity between each sentence and the question to be answered, and output a similarity identifier.

13. The text analysis device according to claim 12, characterized in that: The building blocks are further configured to: Determine a sentence with a similarity identifier as a first similarity identifier as a target sentence; Performing character segmentation on the target sentence and the question to be answered to obtain each word unit in the target sentence and the question to be answered; Build the input sequence based on each word unit.

14. The text analysis device according to claim 12, characterized in that: The building blocks are further configured to: Determine a sentence with a similarity identifier as a first similarity segment identifier as a target sentence; Performing character segmentation on the target sentence and the question to be answered to obtain each word unit in the target sentence and the question to be answered; Build the input sequence based on each word unit.

15. The text analysis device according to claim 13 or 14, characterized in that: The building blocks are further configured to: An input sequence is constructed by placing the word units of the target sentence in front and the word units of the question to be answered in the back.

16. The text analysis device according to claim 11, characterized in that: The determination module is further configured to: Calculate a word vector, a separation vector, and a position vector for each word unit in the input sequence; Linear mapping and nonlinear transformation are performed on the sum of the word vector, the separation vector and the position vector to obtain the probability of each word unit being the starting position and the ending position of the answer corresponding to the question to be answered.

17. The text analysis device according to claim 16, characterized in that: The determination module is further configured to: The first word unit with the highest probability of being the starting position, the second word unit with the highest probability of being the ending position, and the word units between the first word unit and the second word unit are determined as the answer to the question to be answered.

18. The text analysis device according to claim 12 or 13, characterized in that: The text analysis device further includes a training module, which is configured to: Obtaining training samples and sample labels, wherein the training samples include sample texts and sample questions, and the sample labels include correct answers corresponding to the sample texts and sample questions; Splitting the sample text into multiple sentences, and calculating the similarity between each of the multiple sentences and the sample question; Constructing an input sequence according to the target sentence and the sample question whose similarity meets the preset condition; Inputting the input sequence into a text analysis model, obtaining the probability of each word unit in the input sequence being the starting position and the ending position of the answer corresponding to the sample question, and determining the predicted answer to the sample question according to the probabilities; The predicted answer is compared with the correct answer, and based on the comparison result between the predicted answer and the correct answer, the text analysis model is updated.

19. A computing device, characterized in that include: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the following method: Splitting the text to be analyzed into multiple sentences, and calculating the similarity between each of the multiple sentences and the question to be answered; Constructing an input sequence according to the target sentences whose similarities meet the preset conditions and the questions to be answered, wherein constructing the input sequence is to splice one or more target sentences whose similarities meet the preset conditions with the questions to be answered; The probability of each word unit in the input sequence being the starting position and the ending position of the answer corresponding to the question to be answered is obtained through the text analysis model, and the answer to the question to be answered is determined according to the probabilities.

20. A computer-readable storage medium, characterized in that: It stores computer instructions, which, when executed by a processor, implement the steps of the text analysis method described in any one of claims 1 to 9.

Images