Text error correction method and device based on artificial intelligence, equipment and storage medium

Abstract

The invention provides a text error correction method and device based on artificial intelligence, computer equipment and a computer readable storage medium, and belongs to the field of natural language processing. The artificial intelligence-based text error correction method comprises the following steps: performing feature coding processing on a to-be-processed text through a trained encoder to obtain a first character associated coding vector; feature decoding processing is carried out on the basis of the first character correlation coding vector through a trained decoder, a decoding state vector of the tth decoding time step of the trained decoder is obtained, the trained decoder is obtained through learning by using a target loss function, and the target loss function comprises a part-of-speech sequence loss item and a character sequence loss item; and based on the decoding state vector of the tth decoding time step of the trained decoder, predicting the tth character of the output character sequence to obtain an error-corrected text of the to-be-processed text. The text error can be detected and corrected by fusing part-of-speech information and context information, and the text error correction capability is improved.

Description

Technical Field

[0001] The present application relates to the technical field of natural language processing, and specifically to an artificial intelligence-based text error correction method, apparatus, computer equipment, and computer-readable storage medium. Background Art

[0002] Speech recognition technology is a technology that uses speech as a research object, and allows machines to automatically recognize and understand human spoken language through speech signal processing and pattern recognition. Speech recognition technology is a technology that allows machines to convert speech data into corresponding text through the recognition and understanding process.

[0003] However, due to the influence of subjective or objective factors such as accent and oral habits, speech recognition technology often has the problem of inaccurate recognition. Therefore, it is necessary to correct the speech recognition text, that is, to correct the errors in the speech recognition text to obtain a more accurate speech recognition text. Therefore, in actual business usage scenarios, text error correction is crucial for speech recognition technology; for example, in insurance services, text error correction technology can play an important role. It can automatically identify and correct wrong words or phrases for customers' voice input information, allowing customers to input information more smoothly, thereby improving customer experience and work efficiency. However, in related technologies, the text error correction accuracy of text error correction schemes is low. Summary of the invention

[0004] The present application provides a text error correction method, device, computer equipment and computer-readable storage medium based on artificial intelligence, which belongs to the field of natural language processing technology and can integrate part-of-speech information and context information to detect and correct text errors and improve text error correction capabilities.

[0005] In a first aspect, the present application provides a text error correction method based on artificial intelligence, the method comprising:

[0006] Get the text to be processed;

[0007] Performing feature encoding processing on the text to be processed by using a trained encoder to obtain a first character associated encoding vector of the text to be processed;

[0008] By means of a trained decoder, feature decoding processing is performed based on the first character association coding vector to obtain a decoding state vector of the trained decoder at the t-th decoding time step, wherein the trained decoder is learned using a target loss function, and the target loss function includes a part-of-speech sequence loss term and a character sequence loss term;

[0009] Based on the decoding state vector of the trained decoder at the t-th decoding time step, the t-th character of the output character sequence is predicted to obtain the error-corrected text of the text to be processed.

[0010] In a second aspect, the present application provides a text error correction device based on artificial intelligence, and the text error correction device based on artificial intelligence includes:

[0011] An acquisition unit, used for acquiring the text to be processed;

[0012] An encoding unit, used for performing feature encoding processing on the text to be processed by using a trained encoder to obtain a first character associated encoding vector of the text to be processed;

[0013] A decoding unit, configured to perform feature decoding processing based on the first character association encoding vector through a trained decoder to obtain a decoding state vector of the t-th decoding time step of the trained decoder, wherein the trained decoder is learned using a target loss function, and the target loss function includes a part-of-speech sequence loss term and a character sequence loss term;

[0014] The prediction unit is used to predict the t-th character of the output character sequence based on the decoding state vector of the t-th decoding time step of the trained decoder to obtain the error-corrected text of the text to be processed.

[0015] In a third aspect, the present application also provides a computer device, comprising a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computer program and implement the artificial intelligence-based text correction method when executing the computer program.

[0016] In a fourth aspect, the present application also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program is loaded by a processor to execute the artificial intelligence-based text correction method.

[0017] This application uses a trained decoder to perform feature decoding processing on the first character associated coding vector of the text to be processed, and obtains the decoding state vector of the t-th decoding time step of the trained decoder, which is used to predict the t-th character of the output character sequence, and obtains the corrected text of the text to be processed. Since the trained decoder is learned using the target loss function, and on the one hand, the target loss function includes a part-of-speech sequence loss term, the decoder can fully capture the part-of-speech features of the text to be processed, and the part-of-speech features are applied to the text error correction task. The part-of-speech features can be used to capture character errors in the text to be processed, improve the detection capability of text character errors, and thus more accurately correct typos in the text, and improve the text error correction capability. On the other hand, the target loss function also includes a character sequence loss term, so that the decoder can fully capture the contextual information of the text to be processed and apply it to the text error correction task, and improve the text error correction capability. Therefore, the embodiment of the present application can integrate part-of-speech information and contextual information to detect and correct text errors, and improve the text error correction capability. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0019] Figure 1 This is a flow chart of a text error correction method based on artificial intelligence provided in an embodiment of the present application;

[0020] Figure 2 This is a flowchart of an embodiment of the encoder and decoder training in the embodiment of the present application;

[0021] Figure 3 This is a flowchart of an embodiment of step 203 in the embodiment of the present application;

[0022] Figure 4 Schematic diagram illustrating the planning process of encoder and decoder training in an embodiment of the present application;

[0023] Figure 5 It is a schematic diagram of the structure of an embodiment of a text error correction device based on artificial intelligence provided in an embodiment of the present application;

[0024] Figure 6 It is a schematic block diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0026] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0027] In the description of the embodiments of the present application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0028] In order to enable any person skilled in the art to implement and use the present application, the following description is provided. In the following description, details are listed for the purpose of explanation. It should be understood that those of ordinary skill in the art can recognize that the present application can also be implemented without using these specific details. In other examples, the known process will not be elaborated in detail to avoid unnecessary details that make the description of the present application embodiment obscure. Therefore, the present application is not intended to be limited to the embodiments shown, but is consistent with the widest range of principles and features disclosed in accordance with the embodiments of the present application.

[0029] Embodiments of the present application provide a text error correction method, apparatus, computer device, and computer-readable storage medium based on artificial intelligence.

[0030] The executor of the artificial intelligence-based text correction method of the embodiment of the present application may be the artificial intelligence-based text correction device provided in the embodiment of the present application, or the computer device provided in the embodiment of the present application, wherein the artificial intelligence-based text correction device may be implemented in hardware or software.

[0031] Some embodiments of the present application are described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0032] See also Figure 1 , Figure 11 is a flowchart of a text error correction method based on artificial intelligence provided by an embodiment of the present application. The text error correction method based on artificial intelligence includes steps 101 to 104, wherein:

[0033] 101. Get the text to be processed.

[0034] The text to be processed may be text obtained through speech recognition technology. For example, in an insurance service scenario, the text automatically recognized from the customer's speech input information may be used as the text to be processed.

[0035] 102. Perform feature encoding processing on the text to be processed by using a trained encoder to obtain a first character associated encoding vector of the text to be processed.

[0036] The first character associated encoding vector is an encoding vector obtained by performing feature encoding processing on the text to be processed by a trained encoder.

[0037] Exemplarily, first, input the text to be processed into the trained encoder; then, through the trained encoder, embed each character of the text to be processed to obtain the embedding vector of the text to be processed; then, at the t-th encoding time step of the trained encoder, use the encoding state vector of the previous encoding time step (i.e., the t-1-th encoding time step) of the trained encoder and the embedding vector of the text to be processed to calculate, and obtain the encoding state vector of the t-th encoding time step of the trained encoder; the encoding state vector of the last encoding time step of the trained encoder is used as the encoding vector associated with the first character of the text to be processed. (Or, the encoding state vector sequence formed by the encoding state vectors of each encoding time step of the trained encoder is used as the encoding vector associated with the first character of the text to be processed.)

[0038] In order to better understand this embodiment, some names involved in this embodiment are introduced below:

[0039] 1. An encoder is used to perform feature encoding processing on the input text to obtain a character-associated encoding vector of the input text. The encoder takes a text of variable length as input, performs feature encoding processing, and outputs a vector expression of fixed length. The output vector expression of fixed length is the character-associated encoding vector of the input text. Exemplarily, the feature encoding processing process of the encoder can be specifically as follows:

[0040] 1.1、Receive text input;

[0041] 1.2. Perform character embedding on each character of the input text to obtain the embedding vector of the input text;

[0042] 1.3. At the t-th encoding time step, the encoding state vector of the previous encoding time step (i.e., the t-1-th encoding time step) and the embedding vector of the input text are used for calculation to obtain the encoding state vector of the t-th encoding time step; wherein, the encoding state vector of the previous encoding time step at the t=1-th time is the initialized encoding state vector;

[0043] 1.4. Output the encoding state vector of the last encoding time step as the character-associated encoding vector of the input text. (Or, output the encoding state vector sequence formed by the encoding state vectors of each encoding time step as the character-associated encoding vector of the input text.)

[0044] 2. A decoder is used to perform error correction processing based on the character-associated coding vector of the input text to obtain the corrected text of the input text. The decoder takes the character-associated coding vector from the encoder as input, performs feature decoding processing, and outputs a variable-length character sequence. The output variable-length character sequence is the corrected text of the input text. Exemplarily, the character sequence prediction (i.e., the error correction process) output by the decoder, the prediction of the part-of-speech sequence after error correction, the generation probability of each character, and the generation probability calculation of each part of speech can be specifically as follows:

[0045] 2.1. Receive character association encoding vector input of input text;

[0046] 2.2. As shown in Formula 1, at the t-th decoding time step, the decoding state vectors of each decoding time step before the t-th decoding time step (i.e., the decoding state vectors of the 1st to t-1th decoding time steps, including: the decoding state vector of the 1st decoding time step The decoding state vector of the second decoding time step ..., the decoding state vector at the t-1th decoding time step ) and the character associated encoding vector of the input text are calculated to obtain the decoding state vector of the t-th decoding time step; wherein the decoding state vector of each decoding time step before the t=1th time step is the initialized decoding state vector;

[0047]

[0048] In formula 1, represents the decoding state vector at the t-th decoding time step, H src A character-associative encoding vector representing the input text.

[0049] 2.3. According to the decoding state vector of the t-th decoding time step, predict the t-th character of the output character sequence, and use the final output character sequence as the error-corrected text of the input text;

[0050] 2.4. Perform part-of-speech tagging on the t-th character of the output character sequence (i.e., the corrected text of the input text) to obtain the corrected part-of-speech sequence of the input text;

[0051] For example, the input text is a character sequence X consisting of an incorrect sentence. w Taking "I want to go to Shenzhen in the next year" as an example, through the correction process in step 2.3, we can first obtain the character sequence Y that constitutes the correct sentence. w 'I want to go to Shenzhen this year' is used as the corrected text of the input text; then, through step 2.4, a part-of-speech tagger (such as Jieba2) is used to tag the character sequence Y of the correct sentence. w Perform part-of-speech tagging on each character in "I want to go to Shenzhen this year" (i.e. the text after error correction) to obtain the part-of-speech sequence Y p ' / r / v / t / t / v / ns / ns' is the corrected part-of-speech sequence of the input text.

[0052] Among them, after the t-th character in the corrected text of the input text is tagged with part of speech, the t-th part of speech Y in the corrected part of speech sequence is used. t p , and finally get the corrected part-of-speech sequence Y of the input text p .

[0053] 2.5. During the model training phase, on the one hand, linear functions and softmax functions are used for fitting to calculate the generation probability P of the tth character in the output character sequence. t v (w), as shown in Formula 2 below; on the other hand, linear function and softmax function are used for fitting processing to output the generation probability P of the tth part of speech sequence after error correction t pos (w), as shown in Formula 3 below.

[0054]

[0055] For example, please refer to Figure 2 In this embodiment, the decoder and the encoder can be trained by the following steps 201 to 205:

[0056] 201. Encode each sample text in the target data set by using a preset encoder to obtain a second character associated encoding vector of the sample text.

[0057] The second character-associated coding vector is a coding vector obtained by the encoder performing feature coding processing on the sample text.

[0058] Exemplarily, first, the sample text is input into a preset encoder; then, the preset encoder is used to embed each character of the sample text to obtain an embedding vector of the sample text; then, at the t-th encoding time step of the preset encoder, the encoding state vector of the previous encoding time step (i.e., the t-1-th encoding time step) of the preset encoder and the embedding vector of the sample text are used for calculation to obtain the encoding state vector of the t-th encoding time step of the preset encoder; the encoding state vector of the last encoding time step of the preset encoder is used as the encoding vector associated with the second character of the sample text. (Or, the encoding state vector sequence formed by the encoding state vectors of each encoding time step of the preset encoder is used as the encoding vector associated with the second character of the sample text.)

[0059] 202. Perform feature decoding processing based on the second character-associated coding vector through a preset decoder to obtain a decoding state vector of each decoding time step of the preset decoder.

[0060] Specifically, through the preset decoder, based on the second character associated coding vector and the decoding state vectors of the preset decoder from the 1st to the t-1th decoding time steps, the decoding state vector of the preset decoder at the tth decoding time step is determined.

[0061] For example, first, the second character associated coding vector is input into the preset decoder; then, at the first decoding time step of the preset decoder, the second character associated coding vector and the initialized decoding state vector are used to calculate to obtain the decoding state vector of the first decoding time step of the preset decoder; at the tth decoding time step of the preset decoder, the second character associated coding vector and the decoding state vectors of each decoding time step before the tth decoding time step of the preset decoder are used (for example, the decoding state vectors of each decoding time step before the t=4th decoding time step include: the decoding state vector of the first decoding time step The decoding state vector of the second decoding time step Decoding state vector at the 3rd decoding time step ) is calculated to obtain the decoding state vector of the preset decoder at the tth decoding time step

[0062] 203. Predicting is performed based on the decoding state vector of each decoding time step of the preset decoder to obtain a corrected text and a corrected part-of-speech sequence of the sample text.

[0063] Specifically, Figure 3 As shown, step 203 includes the following steps 2031-2032:

[0064] 2031. Predict the t-th character of the output character sequence according to the decoding state vector of the preset decoder at the t-th decoding time step to obtain the error-corrected text of the sample text.

[0065] 2032. Perform part-of-speech tagging on the t-th character of the corrected text of the sample text to obtain a corrected part-of-speech sequence of the sample text.

[0066] For example, the sample text is a character sequence X consisting of an incorrect sentence. w Taking "I want to go to Shenzhen in the next year" as an example, through steps 201 to 203, the character sequence Y constituting the correct sentence can be obtained. w 'I want to go to Shenzhen this year' (i.e. the text after error correction); then, use a part-of-speech tagger (such as Jieba2) to tag the character sequence Y of the correct sentence w Perform part-of-speech tagging on each character in "I want to go to Shenzhen this year" (i.e. the text after error correction) to obtain the part-of-speech sequence Y p ' / r / v / t / t / v / ns / ns' is the corrected part-of-speech sequence of the sample text.

[0067] 204. Obtain a part-of-speech sequence generation probability function and a character sequence generation probability function.

[0068] The part-of-speech sequence generation probability function is used to predict the generation probability of the part-of-speech sequence of the sample text after error correction, and the character sequence generation probability function is used to predict the generation probability of the sample text after error correction.

[0069] Exemplarily, as shown in Formula 4, the part-of-speech sequence generation probability function includes a normalization parameter and a part-of-speech transition parameter, wherein the part-of-speech transition parameter is used to indicate the tth part-of-speech Y of the part-of-speech sequence after error correction of the sample text. t p The t-1th part of speech of the corrected part of speech sequence of the sample text The part-of-speech sequence generation probability function is used to predict the generation probability of the part-of-speech sequence of the sample text after error correction according to the generation probability of each part-of-speech in the part-of-speech sequence of the sample text after error correction, the normalized transition parameter and the part-of-speech transition parameter. By introducing the part-of-speech transition parameter, the dependency relationship between adjacent parts of speech in the text can be modeled to improve the ability to capture part-of-speech information.

[0070]

[0071] In formula 4, represents the tth part of speech in the part-of-speech sequence after error correction, express The generation probability of (i.e., the generation probability of the tth part of speech in the part-of-speech sequence after error correction), the part-of-speech transition parameter express arrive The transition score, It is used as a parameter learning in the end-to-end training process. The normalization parameter Z(X) is calculated by the Vibit algorithm, and m represents the number of parts of speech in the part-of-speech sequence after error correction.

[0072] As shown in Formula 3, the generation probability P of the tth part of speech in the part-of-speech sequence of the sample text after error correction is t pos (w) is obtained by fitting the decoding state vector of the preset decoder at the t-th decoding time step. Specifically, the decoding state vector of the preset decoder at the t-th decoding time step is calculated using a linear function and a softmax function. Perform fitting processing to obtain the generation probability P of the tth part of speech in the part-of-speech sequence after error correction of the sample text t pos (w).

[0073] Exemplarily, as shown in Formula 5, the character sequence generation probability function is used to generate the probability P of each character in the error-corrected text of the sample text according to t v (w), predict the probability of generating the corrected text of the sample text Among them, m represents the number of characters in the text after error correction, P v (y j )=P t v (w) represents the generation probability of the jth character of the text after error correction.

[0074]

[0075] As shown in Formula 2, the generation probability P of the tth character in the error-corrected text of the sample text is t v (w) is obtained by fitting the decoding state vector of the preset decoder at the t-th decoding time step. Specifically, the decoding state vector of the preset decoder at the t-th decoding time step is calculated using a linear function and a softmax function. Perform fitting processing to obtain the generation probability P of the tth character in the sample text after error correction t v (w).

[0076] 205. With the goal of minimizing the target loss function, the preset encoder and the preset decoder are optimized using the sample texts in the target data set to obtain a trained encoder and a trained decoder.

[0077] Among them, the target loss function includes a part-of-speech sequence loss term constructed based on the part-of-speech sequence generation probability function, and a character sequence loss term constructed based on the character sequence generation probability function.

[0078] For example, first, the target loss function Loss is constructed. As shown in Formula 6, the part-of-speech sequence generation probability function shown in Formula 4 can be used to construct the part-of-speech loss term L by minimizing the negative log-likelihood of the part-of-speech sequence generation probability function. CRF As shown in Formula 7, the character sequence generation probability function shown in Formula 5 can be used to construct the part-of-speech loss term L by minimizing the negative log-likelihood of the character sequence generation probability function. ce ; As shown in Formula 8, using the part-of-speech loss term L CRF and character sequence loss term L ce ; Construct the target loss function Loss.

[0079]

[0080] Loss = L ce +L CRF Formula 8

[0081] Then, with the goal of minimizing the target loss function Loss, the preset encoder and the preset decoder are optimized using the sample texts in the target data set to obtain a trained encoder and a trained decoder.

[0082] Through steps 201 to 205, the part-of-speech components are integrated into the neural network model (i.e., the encoder and the decoder). In addition to using the neural network model (i.e., the encoder and the decoder) to implement the character sequence (i.e., the corrected text) prediction task, an auxiliary task is also introduced, namely, the POS sequence (i.e., the part-of-speech tag sequence corresponding to each character in the corrected text) prediction, in which a conditional random field (CRF) layer is added to ensure the validity of the generated POS sequence and stimulate the model to learn grammatical level corrections.

[0083] On the one hand, in this embodiment, the characteristic of "the parts of speech of the correct characters and the wrong characters are obviously different in the case of wrong characters" is utilized to introduce the POS sequence prediction task (i.e., introducing the post-correction part-of-speech sequence prediction task, which is used to mark the part of speech corresponding to each character of the text after the correction). Since the wrong POS part-of-speech tag part is usually close to the wrong part of the sentence, the token-level (i.e., character prediction) error correction and the POS-level (i.e., part-of-speech prediction) error correction have the same error correction target, and various types of token-level errors may be mapped to the same error at the POS level. Therefore, POS-level errors are more common. Through the post-correction part-of-speech sequence prediction task, which is used to mark the part of speech corresponding to each character of the text after the correction, the part-of-speech features can be fully utilized to improve the ability to detect text character errors, thereby improving the text error correction ability. Therefore, the part-of-speech features can help the model better understand the grammatical structure and semantic information in the text, so as to more accurately identify errors and make corrections.

[0084] Secondly, by using the part-of-speech sequence loss term to construct the target loss function for training, the POS sequence is applied to the text error correction task, and the reliability and sensitivity of the POS sequence to spelling errors are fully utilized, so that the encoder and decoder can effectively capture the part-of-speech information, improve the feature expression ability of the encoder and decoder, and thus improve the text error correction ability.

[0085] Thirdly, by introducing part-of-speech transition parameters, the dependency relationship between adjacent POS part-of-speech tags can be modeled to improve the ability to capture part-of-speech information.

[0086] Furthermore, in order to improve the error correction capability of the model, the error correction learning is performed from easy to difficult in this embodiment. Please refer to Figure 4 , specifically steps 401 to 404 are as follows:

[0087] 401. Perform difficulty evaluation on each sample in the total training data set according to a preset sample difficulty evaluation index to obtain an error correction difficulty score for each sample in the total training data set.

[0088] The preset sample difficulty evaluation index includes at least one of sample context similarity and the number of sample error characters.

[0089] 402. Divide each sample in the total training data set according to the error correction difficulty score to obtain k training data subsets.

[0090] Among them, the sample context similarity refers to the similarity between the wrong characters and the correct characters in the sample. The sample context similarity is positively correlated with the error correction difficulty score of the sample. The higher the sample context similarity, the higher the error correction difficulty score of the corresponding sample, and the higher the difficulty level of the training data subset that the sample falls into; conversely, the lower the sample context similarity, the lower the error correction difficulty score of the corresponding sample, and the lower the difficulty level of the training data subset that the sample falls into.

[0091] The number of sample error characters refers to the number of error characters in the sample. The number of sample error characters is positively correlated with the error correction difficulty score of the sample. The larger the number of sample error characters, the higher the error correction difficulty score of the corresponding sample, and the higher the difficulty level of the training data subset that the sample falls into; conversely, the smaller the number of sample error characters, the lower the error correction difficulty score of the corresponding sample, and the lower the difficulty level of the training data subset that the sample falls into.

[0092] 403. According to the difficulty level of each training data subset in the k training data subsets from low to high, each training data subset is used as the target data set for stage training in turn.

[0093] 404. When k stages of training are completed using the k training data subsets, the total training data set is used as the target data set to perform the k+1th stage of training.

[0094] Specifically, in this embodiment, the model (including encoder and decoder) learning framework is divided into two parts: learning difficulty assessment and learning course arrangement. 1) Learning difficulty assessment, through sample context similarity and sample error character number, the difficulty of each sample in the total training data set is assessed, and according to the error correction difficulty score of each sample in the total training data set, each sample in the total training data set is divided to obtain k training data subsets and the difficulty level of each training data subset in the k training data subsets, so that the context similarity of characters (the more similar the error character is to the correct character, the more difficult it is to correct the error) and the number of errors (the more errors, the more difficult it is to correct at one time) can be used as indicators to measure sample difficulty, so as to organize the scattered samples in the total training data set into ordered samples for model training. First, the samples in the total training data set are sorted in ascending order of the error correction difficulty score, and divided into k training data subsets {S1; S2; ...; Sk}; in this embodiment, these k training data subsets are non-overlapping to prevent overfitting and improve generalization performance. 2) Learning course arrangement, which includes k+1 training stages. For the first k stages, the training data subsets are put into the training model from easy to difficult to guide the model from easy learning to difficult learning. In the last stage (i.e., the k+1 stage), the model is trained on the total training data set S to fit the original data distribution. In this way, the model can perform error correction learning from easy to difficult, improving the error correction generalization ability of the model.

[0095] This embodiment designs part-of-speech features and course learning to improve ASR (Automatic Speech Recognition) text error correction to achieve better error correction results. Part-of-speech features can help the model better understand the properties of words, such as the word's part of speech, word form, word meaning, etc., thereby improving the error correction effect. Learning difficulty assessment and learning course arrangement can provide more detailed error correction suggestions to help the model better understand the contextual information of errors.

[0096] 103. Perform feature decoding processing based on the first character-associated coding vector through the trained decoder to obtain a decoding state vector of the t-th decoding time step of the trained decoder.

[0097] The trained decoder is learned using a target loss function, and the target loss function includes a part-of-speech sequence loss term and a character sequence loss term.

[0098] Specifically, through the trained decoder, based on the first character association coding vector and the decoding state vectors of the trained decoder from the 1st to the t-1th decoding time steps, the decoding state vector of the trained decoder at the tth decoding time step is determined.

[0099] For example, first, input the first character-associated encoding vector into the trained decoder; then, at the 1st decoding time step of the trained decoder, calculate using the first character-associated encoding vector and the initialized decoding state vector to obtain the decoding state vector at the 1st decoding time step of the trained decoder; at the t-th decoding time step of the trained decoder, calculate using the first character-associated encoding vector and the decoding state vectors of each decoding time step before the t-th decoding time step of the trained decoder (for example, the decoding state vectors of each decoding time step before the t = 3rd decoding time step include: the decoding state vector at the 1st decoding time step the decoding state vector at the 2nd decoding time step ) to obtain the decoding state vector at the t-th decoding time step of the trained decoder

[0100] 104. Based on the decoding state vector at the t-th decoding time step of the trained decoder, predict the t-th character of the output character sequence to obtain the corrected text of the text to be processed.

[0101] For example, take the character sequence X w ‘I want to go to Shenzhen this year’ which is composed of an incorrect sentence as an example. Through steps 102 - 103, the decoding state vectors of m (m = 7 in this case) decoding time steps can be output: the decoding state vector at the t = 1st decoding time step the decoding state vector at the t = 2nd decoding time step …, the decoding state vector at the t = mth decoding time step Then, use the decoding state vector at the t = 1st decoding time step to predict that the t = 1st character of the output character sequence is “I”; use the decoding state vector at the t = 2nd decoding time step to predict that the t = 1st character of the output character sequence is “want”; …; and so on; use the decoding state vector at the t = 7th decoding time step to predict that the t = 7th character of the output character sequence is “zhen”. When the decoding state vectors of each decoding time step of the trained decoder have been processed, the predicted characters combined to output the character sequence ‘I want to go to Shenzhen this year’ as the corrected text of the text to be processed.

[0102] From the above content, it can be seen that, through the trained decoder, the first character associated coding vector of the text to be processed is subjected to feature decoding processing, and the decoding state vector of the t-th decoding time step of the trained decoder is obtained, which is used to predict the t-th character of the output character sequence, and obtain the corrected text of the text to be processed. Since the trained decoder is learned using the target loss function, and on the one hand, the target loss function includes the part-of-speech sequence loss term, the decoder can fully capture the part-of-speech features of the text to be processed, and apply the part-of-speech features to the text correction task. The part-of-speech features can be used to capture character errors in the text to be processed, and improve the ability to detect text character errors, thereby more accurately correcting typos in the text and improving the text correction ability. On the other hand, the target loss function also includes the character sequence loss term, so that the decoder can fully capture the contextual information of the text to be processed and apply it to the text correction task, thereby improving the text correction ability.

[0103] In addition, in order to better implement the text error correction method based on artificial intelligence in the embodiment of the present application, on the basis of the text error correction method based on artificial intelligence, the embodiment of the present application also provides a text error correction device based on artificial intelligence, such as Figure 5 As shown, it is a schematic diagram of the structure of an embodiment of an artificial intelligence-based text error correction device provided in an embodiment of the present application. The artificial intelligence-based text error correction device 500 includes:

[0104] An acquisition unit 501 is used to acquire a text to be processed;

[0105] The encoding unit 502 is used to perform feature encoding processing on the text to be processed by using a trained encoder to obtain a first character associated encoding vector of the text to be processed;

[0106] A decoding unit 503 is used to perform feature decoding processing based on the first character association encoding vector through a trained decoder to obtain a decoding state vector of the t-th decoding time step of the trained decoder, wherein the trained decoder is learned using a target loss function, and the target loss function includes a part-of-speech sequence loss term and a character sequence loss term;

[0107] The prediction unit 504 is used to predict the t-th character of the output character sequence based on the decoding state vector of the trained decoder at the t-th decoding time step to obtain the error-corrected text of the text to be processed.

[0108] In some embodiments, the text error correction device based on artificial intelligence further includes a training unit (not shown in the figure), which is specifically used to:

[0109] Encode each sample text in the target data set by a preset encoder to obtain a second character associated encoding vector of the sample text;

[0110] By using a preset decoder, feature decoding processing is performed based on the second character associated coding vector to obtain a decoding state vector of each decoding time step of the preset decoder;

[0111] Predicting according to the decoding state vector of each decoding time step of the preset decoder to obtain the error-corrected text and the error-corrected part-of-speech sequence of the sample text;

[0112] Obtaining a part-of-speech sequence generation probability function and a character sequence generation probability function, wherein the part-of-speech sequence generation probability function is used to predict the generation probability of the part-of-speech sequence of the sample text after error correction, and the character sequence generation probability function is used to predict the generation probability of the sample text after error correction;

[0113] With the goal of minimizing the target loss function, the preset encoder and the preset decoder are optimized using the sample texts in the target data set to obtain a trained encoder and a trained decoder, wherein the target loss function includes a part-of-speech sequence loss item constructed based on the part-of-speech sequence generation probability function, and a character sequence loss item constructed based on the character sequence generation probability function.

[0114] In some embodiments, the part-of-speech sequence generation probability function includes a normalization parameter and a part-of-speech transition parameter, the part-of-speech transition parameter being used to indicate the transition score between the tth part-of-speech in the corrected part-of-speech sequence of the sample text and the t-1th part-of-speech in the corrected part-of-speech sequence of the sample text; the part-of-speech sequence generation probability function is used to predict the generation probability of the corrected part-of-speech sequence of the sample text based on the generation probability of each part-of-speech in the corrected part-of-speech sequence of the sample text, the normalization transition parameter and the part-of-speech transition parameter.

[0115] In some embodiments, the training unit is specifically configured to:

[0116] Fitting is performed according to the decoding state vector of the preset decoder at the t-th decoding time step to obtain the generation probability of the t-th part of speech in the part-of-speech sequence of the sample text after error correction.

[0117] In some embodiments, the training unit is specifically configured to:

[0118] According to the decoding state vector of the preset decoder at the t-th decoding time step, predict the t-th character of the output character sequence to obtain the error-corrected text of the sample text;

[0119] Perform part-of-speech tagging on the t-th character of the error-corrected text of the sample text to obtain a part-of-speech sequence of the error-corrected text of the sample text.

[0120] In some embodiments, the training unit is specifically configured to:

[0121] According to a preset sample difficulty evaluation index, a difficulty evaluation is performed on each sample in the total training data set to obtain an error correction difficulty score for each sample in the total training data set, wherein the preset sample difficulty evaluation index includes at least one of sample context similarity and the number of sample error characters;

[0122] According to the error correction difficulty score, each sample in the total training data set is divided to obtain k training data subsets;

[0123] According to the difficulty level of each training data subset in the k training data subsets from low to high, each training data subset is used as the target data set for stage training in turn;

[0124] When k stages of training are completed using the k training data subsets, the total training data set is used as the target data set to perform the k+1th stage of training.

[0125] In some embodiments, the decoding unit 503 is specifically used to:

[0126] Through the trained decoder, based on the first character association coding vector and the decoding state vectors of the trained decoder from the 1st to the t-1th decoding time steps, the decoding state vector of the trained decoder at the tth decoding time step is determined.

[0127] In specific implementation, the above-mentioned units can be implemented as independent entities, or can be arbitrarily combined to be implemented as the same or several entities. The specific implementation of the above-mentioned units can be referred to the above-mentioned text correction method based on artificial intelligence, which will not be repeated here.

[0128] See also Figure 6 , Figure 6 A schematic block diagram of the structure of a computer device provided in an embodiment of the present application. The computer device may be a server.

[0129] like Figure 6 As shown, the computer device includes a processor, a memory, and a network interface connected via a system bus, wherein the memory may include a non-volatile storage medium and an internal memory.

[0130] The non-volatile storage medium can store an operating system and a computer program. The computer program includes program instructions, and when the program instructions are executed, the processor can execute any text error correction method based on artificial intelligence.

[0131] The processor is used to provide computing and control capabilities and support the operation of the entire computer equipment.

[0132] The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium. When the computer program is executed by the processor, the processor can execute any text error correction method based on artificial intelligence.

[0133] The network interface is used for network communication, such as sending assigned tasks, etc. Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0134] It should be understood that the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0135] In one embodiment, the processor is used to run a computer program stored in the memory to implement the following steps:

[0136] Acquire a text to be processed; perform feature encoding processing on the text to be processed through a trained encoder to obtain a first character-associated encoding vector of the text to be processed; perform feature decoding processing based on the first character-associated encoding vector through a trained decoder to obtain a decoding state vector of the t-th decoding time step of the trained decoder, wherein the trained decoder is learned using a target loss function, and the target loss function includes a part-of-speech sequence loss term and a character sequence loss term; based on the decoding state vector of the t-th decoding time step of the trained decoder, predict the t-th character of the output character sequence to obtain an error-corrected text of the text to be processed.

[0137] In one embodiment, the processor is used to run a computer program stored in the memory to implement the following steps:

[0138] By means of a preset encoder, each sample text in a target data set is encoded to obtain a second character-associated encoding vector of the sample text; by means of a preset decoder, feature decoding processing is performed based on the second character-associated encoding vector to obtain a decoding state vector of each decoding time step of the preset decoder; prediction is performed based on the decoding state vector of each decoding time step of the preset decoder to obtain a corrected text and a corrected part-of-speech sequence of the sample text; a part-of-speech sequence generation probability function and a character sequence generation probability function are obtained, wherein the part-of-speech sequence generation probability function is used to predict the generation probability of the part-of-speech sequence of the sample text after correction, and the character sequence generation probability function is used to predict the generation probability of the corrected text of the sample text; with the goal of minimizing a target loss function, the preset encoder and the preset decoder are optimized using each sample text in a target data set to obtain a trained encoder and a trained decoder, wherein the target loss function includes a part-of-speech sequence loss term constructed based on the part-of-speech sequence generation probability function, and a character sequence loss term constructed based on the character sequence generation probability function.

[0139] In one embodiment, the processor is used to run a computer program stored in the memory to implement the following steps:

[0140] The part-of-speech sequence generation probability function includes a normalization parameter and a part-of-speech transition parameter, wherein the part-of-speech transition parameter is used to indicate a transition score between the tth part-of-speech of the corrected part-of-speech sequence of the sample text and the t-1th part-of-speech of the corrected part-of-speech sequence of the sample text; the part-of-speech sequence generation probability function is used to predict the generation probability of the corrected part-of-speech sequence of the sample text based on the generation probability of each part-of-speech in the corrected part-of-speech sequence of the sample text, the normalization transition parameter and the part-of-speech transition parameter.

[0141] In one embodiment, the processor is used to run a computer program stored in the memory to implement the following steps:

[0142] Fitting is performed according to the decoding state vector of the preset decoder at the t-th decoding time step to obtain the generation probability of the t-th part of speech in the part-of-speech sequence of the sample text after error correction.

[0143] In one embodiment, the processor is used to run a computer program stored in the memory to implement the following steps:

[0144] According to the decoding state vector of the preset decoder at the t-th decoding time step, the t-th character of the output character sequence is predicted to obtain the corrected text of the sample text; the t-th character of the corrected text of the sample text is tagged with a part-of-speech tag to obtain the corrected part-of-speech sequence of the sample text.

[0145] In one embodiment, the processor is used to run a computer program stored in the memory to implement the following steps:

[0146] According to a preset sample difficulty evaluation index, difficulty evaluation is performed on each sample in the total training data set to obtain an error correction difficulty score for each sample in the total training data set, wherein the preset sample difficulty evaluation index includes at least one of sample context similarity and the number of sample error characters; according to the error correction difficulty score, each sample in the total training data set is divided to obtain k training data subsets; according to the difficulty level of each training data subset in the k training data subsets from low to high, each training data subset is used as the target data set for stage training in turn; until k stages of training are completed using the k training data subsets, the total training data set is used as the target data set to perform the k+1th stage training.

[0147] In one embodiment, the processor is used to run a computer program stored in the memory to implement the following steps:

[0148] Through the trained decoder, based on the first character association coding vector and the decoding state vectors of the trained decoder from the 1st to the t-1th decoding time steps, the decoding state vector of the trained decoder at the tth decoding time step is determined.

[0149] A person of ordinary skill in the art will appreciate that all or part of the steps in the above-mentioned artificial intelligence-based text correction method can be completed by instructions, or by controlling related hardware through instructions. The instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0150] To this end, an embodiment of the present application provides a computer-readable storage medium, in which multiple computer programs are stored. The computer programs can be loaded by a processor to execute any one of the artificial intelligence-based text correction methods provided in the embodiments of the present application.

[0151] The computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0152] In the above-mentioned text error correction device based on artificial intelligence and computer-readable storage medium embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, refer to the relevant description of other embodiments. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process and beneficial effects of the above-described text error correction device based on artificial intelligence, computer-readable storage medium and its corresponding units can refer to the description of the text error correction method based on artificial intelligence in the above embodiment, and will not be repeated here.

[0153] The above is a detailed introduction to an artificial intelligence-based text correction method, device, computer equipment and computer-readable storage medium provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the ideas of the present application, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A text error correction method based on artificial intelligence, It is characterized in that The method comprises: Get the text to be processed; Performing feature encoding processing on the text to be processed by using a trained encoder to obtain a first character associated encoding vector of the text to be processed; By means of a trained decoder, feature decoding processing is performed based on the first character association coding vector to obtain a decoding state vector of the trained decoder at the t-th decoding time step, wherein the trained decoder is learned using a target loss function, and the target loss function includes a part-of-speech sequence loss term and a character sequence loss term; Based on the decoding state vector of the trained decoder at the t-th decoding time step, the t-th character of the output character sequence is predicted to obtain the error-corrected text of the text to be processed.

2. The text error correction method based on artificial intelligence according to claim 1, It is characterized in that The method further comprises: Encode each sample text in the target data set by a preset encoder to obtain a second character associated encoding vector of the sample text; By using a preset decoder, feature decoding processing is performed based on the second character associated coding vector to obtain a decoding state vector of each decoding time step of the preset decoder; Predicting according to the decoding state vector of each decoding time step of the preset decoder to obtain the error-corrected text and the error-corrected part-of-speech sequence of the sample text; Obtaining a part-of-speech sequence generation probability function and a character sequence generation probability function, wherein the part-of-speech sequence generation probability function is used to predict the generation probability of the part-of-speech sequence of the sample text after error correction, and the character sequence generation probability function is used to predict the generation probability of the sample text after error correction; With the goal of minimizing the target loss function, the preset encoder and the preset decoder are optimized using the sample texts in the target data set to obtain a trained encoder and a trained decoder, wherein the target loss function includes a part-of-speech sequence loss item constructed based on the part-of-speech sequence generation probability function, and a character sequence loss item constructed based on the character sequence generation probability function.

3. The text error correction method based on artificial intelligence according to claim 2, It is characterized in that The part-of-speech sequence generation probability function includes a normalization parameter and a part-of-speech transition parameter, wherein the part-of-speech transition parameter is used to indicate the tth part-of-speech Y of the part-of-speech sequence after error correction of the sample text. t p The t-1th part of speech of the corrected part of speech sequence of the sample text The transition score between The part-of-speech sequence generation probability function is used to predict the generation probability of the part-of-speech sequence of the sample text after error correction based on the generation probability of each part-of-speech in the part-of-speech sequence of the sample text after error correction, the normalized transition parameter and the part-of-speech transition parameter.

4. The text error correction method based on artificial intelligence according to claim 3, It is characterized in that The method further comprises: Fitting is performed according to the decoding state vector of the preset decoder at the t-th decoding time step to obtain the generation probability of the t-th part of speech in the part-of-speech sequence of the sample text after error correction.

5. The text error correction method based on artificial intelligence according to claim 2, It is characterized in that The method of predicting according to the decoding state vector of each decoding time step of the preset decoder to obtain the error-corrected text and the error-corrected part-of-speech sequence of the sample text includes: According to the decoding state vector of the preset decoder at the t-th decoding time step, predict the t-th character of the output character sequence to obtain the error-corrected text of the sample text; Perform part-of-speech tagging on the t-th character of the error-corrected text of the sample text to obtain a part-of-speech sequence of the error-corrected text of the sample text.

6. The text error correction method based on artificial intelligence according to claim 2, It is characterized in that The method further comprises: According to a preset sample difficulty evaluation index, a difficulty evaluation is performed on each sample in the total training data set to obtain an error correction difficulty score for each sample in the total training data set, wherein the preset sample difficulty evaluation index includes at least one of sample context similarity and the number of sample error characters; According to the error correction difficulty score, each sample in the total training data set is divided to obtain k training data subsets; According to the difficulty level of each training data subset in the k training data subsets from low to high, each training data subset is used as the target data set for stage training in turn; When k stages of training are completed using the k training data subsets, the total training data set is used as the target data set to perform the k+1th stage of training.

7. The text error correction method based on artificial intelligence according to claim 1, It is characterized in that The method of performing feature decoding processing based on the first character association coding vector by the trained decoder to obtain a decoding state vector of the t-th decoding time step of the trained decoder includes: Through the trained decoder, based on the first character association coding vector and the decoding state vectors of the trained decoder from the 1st to the t-1th decoding time steps, the decoding state vector of the trained decoder at the tth decoding time step is determined.

8. A text error correction device based on artificial intelligence, It is characterized in that The text error correction device based on artificial intelligence comprises: An acquisition unit, used for acquiring the text to be processed; An encoding unit, used for performing feature encoding processing on the text to be processed by using a trained encoder to obtain a first character associated encoding vector of the text to be processed; A decoding unit, configured to perform feature decoding processing based on the first character association encoding vector through a trained decoder to obtain a decoding state vector of the t-th decoding time step of the trained decoder, wherein the trained decoder is learned using a target loss function, and the target loss function includes a part-of-speech sequence loss term and a character sequence loss term; The prediction unit is used to predict the t-th character of the output character sequence based on the decoding state vector of the t-th decoding time step of the trained decoder to obtain the error-corrected text of the text to be processed.

9. A computer device, It is characterized in that The computer device includes a memory and a processor; The memory is used to store computer programs; The processor is used to execute the computer program and implement the artificial intelligence-based text error correction method as described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, It is characterized in that A computer program is stored thereon, and the computer program is loaded by a processor to execute the artificial intelligence-based text error correction method described in any one of claims 1 to 7.

Images