A sentence reading prediction method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application provide a sentence reading prediction method and device, electronic equipment and storage medium. The method comprises: generating word vectors of each character in the to-be-processed text; inputting the word vectors into a sentence reading prediction model; the sentence reading prediction model comprises an input gate module, a forget gate module, a pause interval accumulation module and an output gate module, the cell state of the current time step is obtained through the input gate module and the forget gate module, the interval distance information of the current time step is obtained through the pause interval accumulation module, and the hidden state of the current time step is obtained through the output gate module based on the word vector input at the current time step, the hidden state output at the last time step, the cell state of the current time step and the interval distance information of the current time step; and the sentence reading prediction result of each character in the to-be-processed text is obtained according to the hidden state of each word vector. The problem of over-dense or over-dilute sentence reading prediction can be improved by considering the context semantics and the pause of every few words.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to a sentence reading prediction method and device, electronic equipment and storage medium. BACKGROUND

[0002] Sentence reading prediction is the use of machine learning algorithms to predict the pause position of a text. It is generally used to guide the pronunciation and pause of a speech synthesis system.

[0003] Compared with the usual punctuation, the sentence reading granularity is more fragmented, and there is not only a pause at the punctuation, but also a pause between the punctuation. Punctuation focuses more on the semantic of the text, while sentence reading focuses more on the pronunciation habit. Therefore, the punctuation marking method is more objective, and the sentence reading marking method is more subjective, and there may be different pause positions according to the pronunciation habits of different people, such as the "of" in the above example can also be paused, people who speak slowly like to pause more, and people who speak fast like to finish in one breath. In essence, as long as the bottom line of affecting the intelligibility of the listening is not broken, there is a larger subjective space for play, such as in the above example, if the pause is made in the middle of "see", it breaks the bottom line.

[0004] For machine learning algorithms, the more arbitrary the statistical distribution of a phenomenon is, the more difficult it is to train a good model. The subjective play of sentence reading is just such a phenomenon, which brings difficulties to machine learning algorithms, and it is difficult to learn stable statistical rules. The most obvious problem of the existing algorithm is instability, some local pauses are too dense, and some local pauses are too sparse; some places may pause every two or three words, and some places may not have a pause for ten or more words. SUMMARY

[0005] In view of the above problems, the present application embodiment is proposed in order to provide a sentence reading prediction method, device, electronic equipment and storage medium which overcomes the above problems or at least partially solves the above problems.

[0006] In order to solve the above problems, the present application embodiment discloses a sentence reading prediction method, comprising:

[0007] Obtaining a to-be-processed text;

[0008] Generating a word vector of each character in the to-be-processed text;

[0009] inputting the word vector into a pre-trained sentence reading prediction model for processing to obtain a hidden state of the word vector; wherein the sentence reading prediction model comprises an input gate module, a forget gate module, a pause interval accumulation module and an output gate module; the cell state of a current time step is updated based on the word vector input at the current time step, the hidden state output at a previous time step and the cell state output at the previous time step through the input gate module and the forget gate module; the interval distance information of the current time step is obtained based on the word vector input at the current time step, the hidden state output at the previous time step and the pause interval accumulation count of the current time step through the pause interval accumulation module; the hidden state of the current time step is obtained based on the word vector input at the current time step, the hidden state output at the previous time step, the cell state of the current time step and the interval distance information of the current time step through the output gate module.

[0010] obtaining the sentence reading prediction result of each character in the text to be processed according to the hidden state of each word vector output by the sentence reading prediction model.

[0011] Optionally, the interval distance information of the current time step is obtained based on the word vector input at the current time step, the hidden state output at the previous time step and the pause interval accumulation count of the current time step through the pause interval accumulation module, comprising:

[0012] obtaining the pause interval accumulation count of the previous time step, and obtaining the pause interval accumulation count of the current time step by adding a preset value to the pause interval accumulation count of the previous time step;

[0013] obtaining the statistical expectation mean and variance of the current time step based on the word vector input at the current time step and the hidden state output at the previous time step;

[0014] performing Gaussian likelihood calculation on the pause interval accumulation count of the current time step, the statistical expectation mean and the variance to obtain the interval distance information of the current time step.

[0015] Optionally, the hidden state of the current time step is obtained based on the word vector input at the current time step, the hidden state output at the previous time step, the cell state of the current time step and the interval distance information of the current time step through the output gate module, comprising:

[0016] calculating the cell state output weight value of the current time step based on the word vector input at the current time step and the hidden state output at the previous time step;

[0017] performing weighted summation on the cell state of the current time step and the interval distance information of the current time step based on the weight value to obtain the hidden state of the current time step.

[0018] Optionally, the hidden state of each of the word vectors is obtained according to the sentence reading prediction model, and a sentence reading prediction result of each character in the text to be processed is obtained.

[0019] The hidden state of each of the word vectors is input into a full connection layer to obtain a sentence reading label of each character in the text to be processed; and the sentence reading label of each character is input into a conditional random field module to perform semantic relationship judgment on adjacent labels to obtain a sentence reading prediction result of each character in the text to be processed.

[0020] Optionally, the sentence reading prediction model includes a forward sentence reading prediction model and a reverse sentence reading prediction model, and the inputting of the word vector into the pre-trained sentence reading prediction model to obtain the hidden state of the word vector includes:

[0021] The word vector is input into the pre-trained forward sentence reading prediction model to obtain a first hidden state of the word vector.

[0022] The word vector is input into the pre-trained reverse sentence reading prediction model to obtain a second hidden state of the word vector.

[0023] The first hidden state and the second hidden state of the word vector are added to obtain the hidden state of the word vector.

[0024] Optionally, the sentence reading prediction model is trained in the following manner:

[0025] A sample text is obtained.

[0026] A word vector of each character in the sample text is generated.

[0027] The word vector is input into a pre-trained sentence reading prediction model to obtain a hidden state of the word vector; wherein the sentence reading prediction model includes an input gate module, a forget gate module, a pause interval accumulation module, and an output gate module; a cell state of a current time step is updated based on a word vector input at the current time step, a hidden state output at a previous time step, and a cell state output at the previous time step through the input gate module and the forget gate module; an interval distance information of the current time step is obtained based on the word vector input at the current time step, the hidden state output at the previous time step, and a pause interval accumulation count of the current time step through the pause interval accumulation module; a hidden state of the current time step is obtained based on the word vector input at the current time step, the hidden state output at the previous time step, the cell state of the current time step, and the interval distance information of the current time step through the output gate module.

[0028] According to the hidden state of each word vector output by the sentence reading prediction model, a sentence reading prediction result of each character in the sample text data is obtained.

[0029] According to the sentence reading prediction result of each character in the sample text, output error information is determined.

[0030] According to the output error information, the parameters of the sentence reading prediction model are adjusted to train the sentence reading prediction model.

[0031] Optionally, the output error information is determined according to the sentence reading prediction result of each character in the sample text, comprising:

[0032] The standard sentence reading labels of each character in the sample text are labeled.

[0033] The sentence reading prediction result of each character in the sample text output by the sentence reading prediction model is compared with the standard sentence reading label of each character in the sample text, and output error information is obtained.

[0034] The embodiment of the application also discloses a title generation device, the device comprises:

[0035] A text acquisition module is configured to acquire a to-be-processed text.

[0036] A text conversion module is configured to generate a word vector of each character in the to-be-processed text.

[0037] A sentence reading prediction module is configured to input the word vector into a pre-trained sentence reading prediction model for processing to obtain a hidden state of the word vector; wherein the sentence reading prediction model comprises an input gate module, a forget gate module, a pause interval accumulation module and an output gate module; the cell state of the current time step is updated based on the word vector input at the current time step, the hidden state output at the last time step and the cell state output at the last time step through the input gate module and the forget gate module; the interval distance information of the current time step is obtained based on the word vector input at the current time step, the hidden state output at the last time step and the pause interval accumulation count of the current time step through the pause interval accumulation module; the hidden state of the current time step is obtained based on the word vector input at the current time step, the hidden state output at the last time step, the cell state of the current time step and the interval distance information of the current time step through the output gate module.

[0038] A sentence reading prediction result determination module is configured to obtain a sentence reading prediction result of each character in the to-be-processed text according to the hidden state of each word vector output by the sentence reading prediction model.

[0039] Optionally, the sentence reading prediction module is configured to obtain a pause interval accumulated count of a previous time step, increase the pause interval accumulated count of the previous time step by a preset value to obtain a pause interval accumulated count of a current time step, obtain a statistical expectation mean and variance of the current time step based on a word vector input of the current time step and a hidden state output of a previous time step, and perform Gaussian likelihood calculation on the pause interval accumulated count of the current time step, the statistical expectation mean and the variance to obtain interval distance information of the current time step.

[0040] Optionally, the sentence reading prediction module is configured to calculate a cell state output weight value of the current time step based on the word vector input of the current time step and the hidden state output of the previous time step, and perform weighted summation on the cell state of the current time step and the interval distance information of the current time step based on the weight value to obtain the hidden state of the current time step.

[0041] Optionally, the sentence reading prediction module comprises:

[0042] The forward sentence reading prediction submodule is configured to input the word vector into a pre-trained forward sentence reading prediction model to obtain a first hidden state of the word vector.

[0043] The backward sentence reading prediction submodule is configured to input the word vector into a pre-trained backward sentence reading prediction model to obtain a second hidden state of the word vector.

[0044] The first hidden state and the second hidden state of the word vector are summed to obtain the hidden state of the word vector.

[0045] Optionally, the sentence reading prediction result determination module is configured to input the hidden state of each word vector into a full connection layer to obtain a sentence reading label of each character in the text to be processed, and input the sentence reading label of each character into a conditional random field module to determine a semantic relationship between adjacent labels to obtain a sentence reading prediction result of each character in the text to be processed.

[0046] Optionally, the sentence reading prediction model is trained by the following modules:

[0047] The training data acquisition module is configured to acquire a sample text.

[0048] The word vector generation module is configured to generate a word vector of each character in the sample text.

[0049] The model training module is configured to input the word vector into a pre-trained sentence reading prediction model for processing to obtain a hidden state of the word vector; the sentence reading prediction model comprises an input gate module, a forget gate module, a pause interval accumulation module and an output gate module; the cell state of a current time step is updated based on the word vector input at the current time step, the hidden state output at a previous time step and the cell state output at the previous time step through the input gate module and the forget gate module; the interval distance information of the current time step is obtained based on the word vector input at the current time step, the hidden state output at the previous time step and the pause interval accumulation count of the current time step through the pause interval accumulation module; the hidden state of the current time step is obtained based on the word vector input at the current time step, the hidden state output at the previous time step, the cell state of the current time step and the interval distance information of the current time step through the output gate module; and the sentence reading prediction result of each character in the sample text is obtained according to the hidden state of each word vector output by the sentence reading prediction model.

[0050] The output error information determination module is configured to determine the output error information according to the sentence reading prediction result of each character in the sample text.

[0051] The model parameter adjustment module is configured to adjust the parameters of the sentence reading prediction model according to the output error information, so as to train the sentence reading prediction model.

[0052] The output error information determination module comprises:

[0053] The comparison submodule is configured to label the standard sentence reading label of each character in the sample text; and compare the sentence reading prediction result of each character in the sample text output by the sentence reading prediction model with the standard sentence reading label of each character in the sample text to obtain the output error information.

[0054] The embodiment of the present application also discloses an electronic device, comprising a processor, a memory and a computer program stored on the memory and capable of running on the processor, wherein the computer program is executed by the processor to implement the steps of the sentence reading prediction method.

[0055] The embodiment of the present application also discloses a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the sentence reading prediction method.

[0056] The embodiment of the present application has the following advantages:

[0057] In the present application, the characters in the text to be processed are converted into word vectors, and the word vectors are input into a pre-trained sentence reading prediction model. The sentence reading prediction model includes an input gate module, a forget gate module, a pause interval accumulation module, and an output gate module. The cell state of the current time step is obtained based on the word vector input at the current time step through the input gate module and the forget gate module. The interval distance information of the current time step is obtained based on the word vector input at the current time step through the pause interval accumulation module. The hidden state of the current time step is obtained based on the word vector input at the current time step, the hidden state output at the previous time step, the cell state of the current time step, and the interval distance information of the current time step through the output gate module. The sentence reading prediction result of each character in the text to be processed is obtained according to the hidden state of each word vector output by the sentence reading prediction model. The pause interval accumulation module in the sentence reading prediction model of the present application is used to control the average density of the interval. The input gate module and the forget gate module are used to combine the context features. Therefore, the sentence reading prediction result obtained based on the hidden state output by the sentence reading prediction model of the present application can take into account pausing every few words on the basis of combining the context semantics, thereby improving the problem of over-dense or over-sparse of the existing sentence reading prediction effect. BRIEF DESCRIPTION OF DRAWINGS

[0058] Figure 1 is a step flow chart of a sentence reading prediction method provided by an embodiment of the present application;

[0059] Figure 2 is an architecture diagram of a sentence reading prediction model provided by an embodiment of the present application;

[0060] Figure 3 is a step flow chart of another sentence reading prediction method provided by an embodiment of the present application;

[0061] Figure 4 is a structural block diagram of a sentence reading prediction device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0062] In order to make the above-mentioned objects, features and advantages of the present application more apparent and easy to understand, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0063] Since the existing technology is unstable when predicting the sentence reading effect, some local pauses are too dense, and some local pauses are too sparse. In view of this, the present application proposes a sentence reading prediction method, which takes into account the distance (number of words) between adjacent pauses; on the basis of combining the context semantics, it takes into account pausing every few words, so as to improve the problem of over-dense or over-sparse of the existing model effect.

[0064] Reference Figure 1 , a step flow chart of a sentence reading prediction method provided by an embodiment of the present application is shown, and the method can specifically include the following steps:

[0065] Step 101: Obtain the text to be processed.

[0066] The method described in this application can be applied to a server, such as a punctuation recognition server. The server can provide punctuation recognition services to clients (such as voice assistants, instant messaging apps, etc.).

[0067] The server is equipped with a punctuation recognition model, which can perform either a non-real-time punctuation prediction task or a real-time punctuation prediction task for the text to be processed, depending on the request.

[0068] Step 102: Generate the character vectors of each character in the text to be processed.

[0069] Character vector lookup is used to find a character vector for each character in the text to be processed. Each character can be a Chinese character or an English word; there are no restrictions. For example, given a Chinese text of length L, character vector lookup will find a character vector for each Chinese character, resulting in character vectors x1, x2, ..., x... t , ..., x L Where x1 is the character vector of the first character in the text, and so on, x t Let x be the character vector of the t-th character in the text. L Let L be the character vector of the Lth character in the text.

[0070] If the text contains punctuation marks, the punctuation marks can be removed first, and then the character vectors of the characters in the text can be searched to obtain the character vectors of each character in the text to be processed. Alternatively, punctuation marks can be treated as a single character, and the character vectors of all characters in the text, including those containing punctuation marks, can be searched to obtain the character vectors of each character in the text to be processed.

[0071] Step 103: Input the character vectors into the pre-trained punctuation prediction model for processing to obtain the hidden state of the character vectors; wherein, the punctuation prediction model includes an input gate module, a forget gate module, a pause interval accumulation module, and an output gate module; the input gate module and the forget gate module update the cell state of the current time step based on the character vector input at the current time step, the hidden state output at the previous time step, and the cell state output at the previous time step; the pause interval accumulation module accumulates the pause interval count based on the character vector input at the current time step, the hidden state output at the previous time step, and the pause interval of the current time step to obtain the interval distance information of the current time step; the output gate module obtains the hidden state of the current time step based on the character vector input at the current time step, the hidden state output at the previous time step, the cell state of the current time step, and the interval distance information of the current time step.

[0072] The sentence reading prediction model in the embodiments of the present application can be an EIRU (Explicit-Interval Recurrent Unit) model, which can be a model improved based on an LSTM (Long Short-Term Memory) model.

[0073] Referring to Figure 2 FIG. 1 is an architecture diagram of an EIRU model in the embodiments of the present application, where x t represents a word vector input at a current time step, x t-1 represents a word vector input at a previous time step, x t+1 represents a word vector input at a next time step; h t represents a hidden state output at the current time step, h t-1 represents a hidden state output at the previous time step, c t represents a cell state output at the current time step, c t-1 represents a cell state output at the previous time step, d t represents a pause interval accumulation count at the current time step, d t-1 represents a pause interval accumulation count output at the previous time step. The word vector x t input at the current time step is combined with the h t-1 , c t-1 and d t-1 output at the previous time step to output the hidden state h t at the current time step, the cell state c t at the current time step, and the pause interval accumulation count d t at the current time step.

[0074] Specifically, the sentence reading prediction model can include an input gate module, a forget gate module, a pause interval accumulation module, and an output gate module.

[0075] The cell state c t at the current time step is updated based on the word vector x t input at the current time step, the hidden state h t-1 output at the previous time step, and the cell state c t-1 output at the previous time step through the input gate module and the forget gate module.

[0076] In the forget gate module, it is determined which information in the cell state c t-1 is forgotten based on the word vector x t input at the current time step and the hidden state h t-1 output at the previous time step, and the specific calculation formula is:

[0077] (1)

[0078] wherein, is a neural network layer with sigmoid function as the activation function, W and b are the coefficient matrix and bias vector needed for matrix multiplication to generate the corresponding output, is the model parameter to be trained, the parameters W in the calculation formulas (1), (2), (3), (5), (6), (8), (9) are different, and the parameters b are also different. x t and h t-1 These two vectors are jointly input into the sigmoid neural network layer, and the output signal f t , f t is a number between 0 and 1, which is used to multiply c t-1 to determine which information in c t-1 will be discarded and which information will be retained.

[0079] In the input gate module, based on the word vector x t input at the current time step, the hidden state h t-1 output at the previous time step, it is determined which information in the cell state c t-1 will be updated, and the specific calculation formula is:

[0080] (2)

[0081] x t and h t-1 These two vectors are jointly input into the sigmoid neural network layer, and the output signal i t , i t is a number between 0 and 1, which is used to multiply the transformed c t-1 to select which information will be newly added to the cell state at the current step.

[0082] According to the output signals of the forget gate and the input gate, the cell state c t can be updated, and the specific calculation formula is:

[0083] (3)

[0084] wherein, tanh refers to a neural network layer with hyperbolic tangent function as the activation function, which receives the word vector x t input at the current time step and the hidden state h t-1 output at the previous time step, and these two vectors are jointly input into the tanh neural network layer to create a new state candidate vector tanh(W xc x t +W hc h t-1 +b c ), which is used to multiply it The multiplication selects which information will be newly added to the cell state of the current step.

[0085] The pause interval accumulation module obtains interval distance information of the current time step based on the word vector input at the current time step, the hidden state output at the last time step, and the pause interval accumulation count of the current time step, and the calculation formula is as follows:

[0086] Specifically, in the pause interval accumulation module, the calculation formula of the pause interval accumulation count of the current time step is as follows:

[0087] (4)

[0088] Wherein, sd is a global model parameter, and is an isolated parameter, which is learned and optimized together in the neural network training process, and reflects the speed of the step or the average density of the interval. In the calculation process of the pause interval accumulation count, (sd / 10000) is increased every new calculation step, and when predicting the sentence reading, it can also be multiplied by a multiple to externally control the pause density of the overall average.

[0089] The calculation formula of the pause interval accumulation module for obtaining the statistical expectation mean and variance of the current time step based on the word vector input at the current time step and the hidden state output at the last time step, and the Gaussian likelihood calculation of the pause interval accumulation count, the statistical expectation mean and the variance of the current time step to obtain the interval distance information of the current time step is as follows:

[0090] (5)

[0091] (6)

[0092] mu t and var t are the mean and log variance of the statistical expectation of cos(d t ) predicted based on the word vector x t-1 input at the current step, the hidden state h t output at the last time step. The trigonometric function frequency calculation is performed on the pause interval accumulation count d t of the current step to obtain cos(d t), the variable value is continuous and derivable, unlike the conventional direct word count as length, which is more conducive to the stability of neural network training and the out-of-sample scalability of prediction ability. The log is taken when calculating the variance to prevent the value range from changing too much, which is conducive to the stability of model training.

[0093] The Gaussian likelihood calculation is performed on cos(d t ), mu t and var t to obtain interval distance information I t , and the specific calculation formula is:

[0094] (7)

[0095] According to the word vector x t input at the current time step, the hidden state h t-1 output at the previous time step, the cell state output weight value o t at the current time step can be calculated, and the calculation formula is:

[0096] (8)

[0097] According to the weight value o t , the cell state c t at the current time step and the interval distance information I t at the current time step are weighted and summed to obtain the hidden state at the current time step, and the calculation formula is:

[0098] (9)

[0099] The calculation results of (4) to (7) are all one-dimensional scalars. I t obtained by (4) to (7) must be expanded in dimension to be added to the N-dimensional o t tanh(c t ) in (9), so I t is multiplied by the coefficient matrix W Ic to do a linear transformation and expand to N dimensions before weighted summation.

[0100] The word vector is input into the above EIRU model, and the hidden state of each word vector can be output through the processing of the word vector by the input gate module, the forget gate module, the pause interval accumulation module and the output gate module in the EIRU model. The hidden state contains the interval distance information of each word vector.

[0101] In an optional embodiment, refer to Figure 3The punctuation prediction model includes a forward punctuation prediction model and a reverse punctuation prediction model. The forward punctuation prediction model can be a forward EIRU model, and the reverse punctuation prediction model can be a reverse EIRU model. The character vectors are input into the pre-trained punctuation prediction model for processing to obtain the hidden states of the character vectors. This includes: inputting the character vectors into the pre-trained forward punctuation prediction model to obtain the first hidden state of the character vectors; inputting the character vectors into the pre-trained reverse punctuation prediction model to obtain the second hidden state of the character vectors; and summing the first and second hidden states of the character vectors to obtain the final hidden state of the character vectors.

[0102] In the positive sentence reading prediction model, the character vectors of the characters in the text are processed sequentially according to the order of the text from beginning to end, that is, the character vectors x1, x2~x3 are processed sequentially. L After processing, we obtain the first hidden states h1', h2' ~ h of the word vectors. L In the reverse sentence prediction model, the character vectors of the characters in the text are processed sequentially from back to front, that is, the character vectors x are processed sequentially. L Processing x2 and x1 yields the second hidden state h of the word vector. L ''~h2'', h1''. Summing h1' and h1'' gives the hidden state h1 of word vector x1. The hidden state of other word vectors is calculated in the same way.

[0103] Using a forward punctuation prediction model, the hidden state can be generated by combining the context information of the word vector. Using a reverse punctuation prediction model, the hidden state can be generated by combining the context information of the word vector. By combining the forward and reverse punctuation prediction models, the hidden state of the word vector can be obtained, capturing both past and future features at the current time step. In other words, the hidden state of the current word vector can be generated by combining the context information of the current word vector.

[0104] Step 104: Based on the hidden states of each character vector output by the sentence prediction model, obtain the sentence prediction results for each character in the text to be processed.

[0105] Specifically, the hidden states of each character vector can be input into a fully connected layer to obtain the punctuation labels for each character in the text to be processed. The punctuation label for each character can be 0 or 1, where 0 indicates a pause and 1 indicates no pause, or vice versa. Punctuation labels can also be represented in other ways; there are no restrictions on the representation of punctuation labels here.

[0106] After obtaining the sentence reading labels of each character in the text to be processed, the sentence reading labels of each character are input into a conditional random field module to judge the semantic relationship of adjacent labels, and the sentence reading prediction results of each character in the text to be processed are obtained.

[0107] The sentence reading labels of each character are input into a conditional random field (CRF) module to judge the semantic relationship of adjacent labels, and the final sentence reading prediction results are obtained by further optimizing the sentence reading labels according to the semantic relationship of adjacent labels.

[0108] In an optional embodiment, the training method of the sentence reading prediction model is as follows:

[0109] A sample text is obtained, and a word vector of each character in the sample text is generated. The word vector is input into a pre-trained sentence reading prediction model for processing to obtain a hidden state of the word vector. The sentence reading prediction model includes an input gate module, a forget gate module, a pause interval accumulation module, and an output gate module. The cell state of the current time step is updated based on the word vector input at the current time step, the hidden state output at the previous time step, and the cell state output at the previous time step through the input gate module and the forget gate module. The interval distance information of the current time step is obtained based on the word vector input at the current time step, the hidden state output at the previous time step, and the pause interval accumulation count at the previous time step through the pause interval accumulation module. The hidden state of the current time step is obtained based on the word vector input at the current time step, the hidden state output at the previous time step, the cell state of the current time step, and the interval distance information of the current time step through the output gate module. The sentence reading prediction results of each character in the sample text are obtained based on the hidden state of each word vector output by the sentence reading prediction model. The output error information is determined based on the sentence reading prediction results of each character in the sample text. The parameters of the sentence reading prediction model are adjusted based on the output error information to train the sentence reading prediction model.

[0110] The output error information is determined based on the sentence reading prediction results of each character in the sample text, which includes: labeling the standard sentence reading labels of each character in the sample text; comparing the sentence reading prediction results of each character in the sample text output by the sentence reading prediction model with the standard sentence reading labels of each character in the sample text to obtain the output error information.

[0111] In order for those skilled in the art to better understand the embodiments of the present application, the embodiments of the present application are described below through an example:

[0112] For a text containing L characters:

[0113] 1. Generate a word vector of each character in the text to obtain word vectors x1, x2, …, x L .

[0114] 2. Initialization, , is a N-dimensional all-zero vector.

[0115] 3. Input the word vector xi to the sentence reading prediction model, calculate according to the formulas (1) to (9), obtain the hidden state hi and the cell state ci, and the pause interval accumulated count di corresponding to the word vector xi. Output hi to the outside of the network, and pass hi, ci and di to the next moment.

[0116] 4. Repeat the operation of 3 for t = 2…L, and finally obtain hi, h2… h L outside the network. L Input to the next full connection network layer.

[0117] 5. The full connection network layer processes the hidden states hi, h2… h L of each word vector to obtain the sentence reading labels of each character. The sentence reading label is 0 or 1, wherein 0 represents no pause and 1 represents pause.

[0118] 6. Input the sentence reading labels of each character to the CRF module, judge the semantic relationship of adjacent labels, further optimize the sentence reading labels according to the semantic relationship of adjacent labels, and obtain the final sentence reading prediction result. The final sentence reading result is still represented by 0 or 1, wherein 0 represents no pause and 1 represents pause.

[0119] In the present application, the characters in the text to be processed are converted into word vectors, and the word vectors are input into a pre-trained sentence reading prediction model. The sentence reading prediction model includes an input gate module, a forget gate module, a pause interval accumulation module and an output gate module. The cell state of the current time step is obtained based on the word vector input at the current time step through the input gate module and the forget gate module. The interval distance information of the current time step is obtained based on the word vector input at the current time step through the pause interval accumulation module. The hidden state of the current time step is obtained based on the word vector input at the current time step, the hidden state output at the last time step, the cell state of the current time step and the interval distance information of the current time step through the output gate module. The sentence reading prediction result of each character in the text to be processed is obtained according to the hidden state of each word vector output by the sentence reading prediction model. The pause interval accumulation module in the sentence reading prediction model of the present application is used to control the average density of the interval, and the input gate module and the forget gate module are used to combine the context features. Therefore, the sentence reading prediction result obtained based on the hidden state output by the sentence reading prediction model of the present application can take into account pausing every few words on the basis of combining the context semantics, thereby improving the problem of over-dense or over-dense sentence reading prediction effect of the prior art.

[0120] It should be noted that, for the method embodiments, the series of acts combined is described for simplicity, but those skilled in the art should know that the embodiments of the present application are not limited to the order of the acts described, because according to the embodiments of the present application, certain steps can be performed in other orders or at the same time. Secondly, those skilled in the art should know that the embodiments described in the specification are all preferred embodiments, and the acts involved are not necessarily essential to the embodiments of the present application.

[0121] Referring to Figure 4 , a structural block diagram of a sentence reading prediction device provided by an embodiment of the present application is shown, which can specifically include the following modules:

[0122] The text acquisition module 401 is configured to acquire a text to be processed.

[0123] The text conversion module 402 is configured to generate a word vector of each character in the text to be processed.

[0124] The sentence reading prediction module 403 is configured to input the word vector into a pre-trained sentence reading prediction model for processing to obtain a hidden state of the word vector. The sentence reading prediction model includes an input gate module, a forget gate module, a pause interval accumulation module and an output gate module. The cell state of the current time step is obtained by updating the cell state of the previous time step based on the word vector input at the current time step, the hidden state output at the previous time step and the cell state output at the previous time step through the input gate module and the forget gate module. The interval distance information of the current time step is obtained by updating the pause interval accumulation count of the previous time step based on the word vector input at the current time step, the hidden state output at the previous time step and the pause interval accumulation count of the current time step through the pause interval accumulation module. The hidden state of the current time step is obtained based on the word vector input at the current time step, the hidden state output at the previous time step, the cell state of the current time step and the interval distance information of the current time step through the output gate module.

[0125] The sentence reading prediction result determination module 404 is configured to obtain a sentence reading prediction result of each character in the text to be processed according to the hidden state of each word vector output by the sentence reading prediction model.

[0126] Optionally, the sentence reading prediction module 403 is configured to acquire the pause interval accumulation count of the previous time step, increase the pause interval accumulation count of the previous time step by a preset value to obtain the pause interval accumulation count of the current time step, obtain the statistical expected mean and variance of the current time step based on the word vector input at the current time step and the hidden state output at the previous time step, and perform Gaussian likelihood calculation on the pause interval accumulation count of the current time step, the statistical expected mean and variance to obtain the interval distance information of the current time step.

[0127] Optionally, the sentence reading prediction module 403 is configured to calculate a cell state output weight value of a current time step according to a word vector input at the current time step and a hidden state output at a previous time step; and perform weighted summation on a cell state of the current time step and interval distance information of the current time step according to the weight value, to obtain a hidden state of the current time step.

[0128] Optionally, the sentence reading prediction module 403 comprises:

[0129] The forward sentence reading prediction submodule is configured to input the word vector into a pre-trained forward sentence reading prediction model, to obtain a first hidden state of the word vector.

[0130] The backward sentence reading prediction submodule is configured to input the word vector into a pre-trained backward sentence reading prediction model, to obtain a second hidden state of the word vector.

[0131] The first hidden state and the second hidden state of the word vector are summed to obtain a hidden state of the word vector.

[0132] Optionally, the sentence reading prediction result determination module 404 is configured to input the hidden state of each word vector into a full connection layer, to obtain a sentence reading label of each character in the text to be processed; and input the sentence reading label of each character into a conditional random field module to judge a semantic relationship between adjacent labels, to obtain a sentence reading prediction result of each character in the text to be processed.

[0133] Optionally, the sentence reading prediction model is trained by the following modules:

[0134] The training data acquisition module is configured to acquire a sample text.

[0135] The word vector generation module is configured to generate a word vector of each character in the sample text.

[0136] The model training module is configured to input the word vector into a pre-trained sentence reading prediction model for processing, to obtain a hidden state of the word vector; wherein the sentence reading prediction model comprises an input gate module, a forget gate module, a pause interval accumulation module and an output gate module; the cell state of the current time step is updated based on the word vector input at the current time step, the hidden state output at the previous time step and the cell state output at the previous time step through the input gate module and the forget gate module; the interval distance information of the current time step is obtained based on the word vector input at the current time step, the hidden state output at the previous time step and the pause interval accumulation count of the current time step through the pause interval accumulation module; the hidden state of the current time step is obtained based on the word vector input at the current time step, the hidden state output at the previous time step, the cell state of the current time step and the interval distance information of the current time step through the output gate module; and the sentence reading prediction result of each character in the sample text is obtained according to the hidden state of each word vector output by the sentence reading prediction model.

[0137] An output error information determination module is configured to determine the output error information according to the syllable reading prediction results of the characters in the sample text.

[0138] A model parameter adjustment module is configured to adjust parameters of the syllable reading prediction model according to the output error information, so as to train the syllable reading prediction model.

[0139] The output error information determination module comprises:

[0140] A comparison sub-module is configured to: annotate standard syllable reading labels of the characters in the sample text; and compare the syllable reading prediction results of the characters in the sample text output by the syllable reading prediction model with the standard syllable reading labels of the characters in the sample text, to obtain the output error information.

[0141] For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and relevant parts refer to the part of the method embodiment.

[0142] The embodiment of the present application further provides an electronic device, comprising:

[0143] The computer program is stored in the memory and can be run on the processor, and when the computer program is executed by the processor, each process of the above-mentioned syllable reading prediction method embodiment is realized, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0144] The embodiment of the present application further provides a computer readable storage medium, and the computer readable storage medium stores a computer program, and when the computer program is executed by the processor, each process of the above-mentioned syllable reading prediction method embodiment is realized, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0145] Each embodiment in the specification is described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same and similar parts of each embodiment can be referred to each other.

[0146] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, device or computer program product. Therefore, the embodiments of the present application can be in the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware aspects. Moreover, the embodiments of the present application can be in the form of a computer program product implemented on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program code.

[0147] The embodiments of the present application are described with reference to the flowchart illustrations and / or block diagrams of the methods, terminal devices (systems) and computer program products according to the embodiments of the present application. It is understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing terminal devices to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal devices, create means for implementing the functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams.

[0148] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal devices to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams.

[0149] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal devices, such that a series of operational steps are carried out on the computer or other programmable terminal devices to produce a computer implemented process so that the instructions executed on the computer or other programmable terminal devices provide steps for implementing the functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams.

[0150] Although preferred embodiments of the present application have been described, those skilled in the art will be able to make additional modifications and variations to these embodiments without departing from the scope of the present application. Accordingly, the appended claims are intended to encompass all such modifications and variations as falling within the scope of the present application.

[0151] Finally, it is to be understood that the phraseology or terminology such as "first" and "second" etc. used herein is merely intended to differentiate one entity or operation from another entity or operation, without necessarily requiring or implying any actual such relationship or order between such entities or operations. Moreover, the terms "comprising", "including", or any other closure, are intended to cover the non-exclusive inclusion such that a process, method, article, or apparatus that comprises a list of elements does not include those elements alone but can include other elements not expressly listed or even include elements inherent in such process, method, article, or apparatus. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0152] The above describes in detail the sentence reading prediction method and the sentence reading prediction device provided by the present application. The principles and implementation manners of the present application are described by using specific examples. The above description of the examples is only used to help understand the method of the present application and its core idea. Meanwhile, for those skilled in the art, the specific implementation manners and application ranges can be changed according to the idea of the present application. In summary, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A method for predicting sentence breaks, characterized in that, include: Get the text to be processed; Generate the character vectors for each character in the text to be processed; The character vectors are input into a pre-trained punctuation prediction model for processing to obtain the hidden state of the character vectors. The punctuation prediction model includes an input gate module, a forget gate module, a pause interval accumulation module, and an output gate module. The input gate module and the forget gate module update the cell state of the current time step based on the character vector input at the current time step, the hidden state output at the previous time step, and the cell state output at the previous time step. The pause interval accumulation module accumulates the pause interval count based on the character vector input at the current time step, the hidden state output at the previous time step, and the pause interval count of the current time step to obtain the interval distance information of the current time step. The output gate module obtains the hidden state of the current time step based on the character vector input at the current time step, the hidden state output at the previous time step, the cell state of the current time step, and the interval distance information of the current time step. Based on the hidden state of each character vector output by the sentence prediction model, the sentence prediction result of each character in the text to be processed is obtained. The step of obtaining the interval distance information of the current time step through the pause interval accumulation module based on the word vector input at the current time step, the hidden state output at the previous time step, and the cumulative count of pause intervals at the current time step includes: Get the cumulative count of pause intervals in the previous time step, and add a preset value to the cumulative count of pause intervals in the previous time step to get the cumulative count of pause intervals in the current time step. The expected mean and variance of the current time step are obtained based on the word vector input at the current time step and the hidden state output at the previous time step. Gaussian likelihood is calculated for the cumulative count of pause intervals at the current time step, the statistical expected mean, and the variance to obtain the interval distance information at the current time step. The step of obtaining the sentence prediction result for each character in the text to be processed based on the hidden state of each character vector output by the sentence prediction model includes: The hidden states of each character vector are input into a fully connected layer to obtain the punctuation labels of each character in the text to be processed; the punctuation labels of each character are input into a conditional random field module to determine the semantic relationship between adjacent labels, thereby obtaining the punctuation prediction results of each character in the text to be processed.

2. The method according to claim 1, characterized in that, The process of obtaining the hidden state of the current time step through the output gate module based on the word vector input at the current time step, the hidden state output at the previous time step, the cell state at the current time step, and the interval distance information at the current time step includes: The cell state output weight value for the current time step is calculated based on the word vector input at the current time step and the hidden state output at the previous time step. Based on the weight values, the cell state at the current time step and the interval distance information at the current time step are weighted and summed to obtain the hidden state at the current time step.

3. The method according to claim 1, characterized in that, The punctuation prediction model includes a forward punctuation prediction model and a reverse punctuation prediction model. The step of inputting the character vector into the pre-trained punctuation prediction model for processing to obtain the hidden state of the character vector includes: The word vector is input into the pre-trained positive sentence prediction model to obtain the first hidden state of the word vector; The word vector is input into the pre-trained reverse sentence prediction model to obtain the second hidden state of the word vector; The hidden state of the character vector is obtained by summing the first hidden state and the second hidden state.

4. The method according to claim 1, characterized in that, The sentence prediction model is trained as follows: Obtain sample text; Generate the character vectors for each character in the sample text; The character vectors are input into a pre-trained punctuation prediction model for processing to obtain the hidden state of the character vectors. The punctuation prediction model includes an input gate module, a forget gate module, a pause interval accumulation module, and an output gate module. The input gate module and the forget gate module update the cell state at the current time step based on the character vector input at the current time step, the hidden state output at the previous time step, and the cell state output at the previous time step. The pause interval accumulation module accumulates the pause interval count based on the character vector input at the current time step, the hidden state output at the previous time step, and the pause interval count at the current time step to obtain the interval distance information at the current time step. The output gate module obtains the hidden state at the current time step based on the character vector input at the current time step, the hidden state output at the previous time step, the cell state at the current time step, and the interval distance information at the current time step. Based on the hidden state of each character vector output by the sentence prediction model, the sentence prediction result of each character in the sample text is obtained; Based on the sentence prediction results of each character in the sample text, the output error information is determined; Based on the output error information, the parameters of the punctuation prediction model are adjusted to train the punctuation prediction model.

5. The method according to claim 4, characterized in that, The step of determining the output error information based on the sentence prediction results of each character in the sample text includes: Label each character in the sample text with standard sentence breaks and punctuation tags; The punctuation prediction results of each character in the sample text output by the punctuation prediction model are compared with the standard punctuation labels of each character in the sample text to obtain the output error information.

6. A punctuation prediction device, characterized in that, include: The text acquisition module is used to acquire the text to be processed. The text conversion module is used to generate character vectors for each character in the text to be processed; A punctuation prediction module is used to input the character vectors into a pre-trained punctuation prediction model for processing to obtain the hidden state of the character vectors. The punctuation prediction model includes an input gate module, a forget gate module, a pause interval accumulation module, and an output gate module. The input gate module and the forget gate module update the cell state of the current time step based on the character vector input at the current time step, the hidden state output at the previous time step, and the cell state output at the previous time step. The pause interval accumulation module accumulates the pause interval count based on the character vector input at the current time step, the hidden state output at the previous time step, and the pause interval count of the current time step to obtain the interval distance information of the current time step. The output gate module obtains the hidden state of the current time step based on the character vector input at the current time step, the hidden state output at the previous time step, the cell state of the current time step, and the interval distance information of the current time step. The punctuation prediction result determination module is used to obtain the punctuation prediction result of each character in the text to be processed based on the hidden state of each character vector output by the punctuation prediction model. The sentence prediction module is used to obtain the cumulative count of pause intervals in the previous time step, and add a preset value to the cumulative count of pause intervals in the previous time step to obtain the cumulative count of pause intervals in the current time step; based on the character vector input in the current time step and the hidden state output in the previous time step, the expected mean and variance of the current time step are obtained; Gaussian likelihood is calculated on the cumulative count of pause intervals, expected mean and variance of the current time step to obtain the interval distance information of the current time step; The sentence prediction result determination module is used to input the hidden state of each character vector into the fully connected layer to obtain the sentence label of each character in the text to be processed; the sentence label of each character is input into the conditional random field module to judge the semantic relationship between adjacent labels to obtain the sentence prediction result of each character in the text to be processed.

7. An electronic device, characterized in that, include: A processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program, when executed by the processor, implements the steps of the sentence prediction method as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, which, when executed by a processor, implements the steps of the sentence prediction method as described in any one of claims 1-5.

Images