Text error correction method and device based on artificial intelligence recurrent neural network

[0028] The embodiments of the present invention are described in detail below. Examples of the embodiments are shown in the accompanying drawings, in which the same or similar reference numerals indicate the same or similar elements or elements with the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary, and are intended to explain the present invention, but should not be construed as limiting the present invention.

[0029] Specifically, each embodiment of the present invention addresses the problem of text error correction in the prior art, which mainly relies on feature design, is data-driven, lacks contextual syntax information, and has poor error correction accuracy. Artificial intelligence cyclic neural network text error correction method. According to an artificial intelligence-based cyclic neural network text error correction method provided by the embodiment of the present invention, after obtaining the text data to be corrected, the trained cyclic neural network model is used to perform error correction processing on the text data to be corrected to generate error corrections. Text data. By adopting the cyclic neural network model and combining the syntactic information of the context, text error correction is performed, which improves the accuracy of error correction, can better meet the needs of users and improve user experience.

[0030] The following describes the method and device for text error correction based on artificial intelligence-based cyclic neural network in the embodiments of the present invention with reference to the accompanying drawings.

[0031] figure 1 It is a flowchart of an artificial intelligence-based cyclic neural network text error correction method according to an embodiment of the present invention.

[0032] Such as figure 1 As shown, the method for text error correction based on artificial intelligence-based cyclic neural network includes:

[0033] Step 101: Obtain text data to be corrected.

[0034] Among them, the executor of the method for text error correction based on the artificial intelligence-based cyclic neural network provided by the embodiment of the present invention is the text error correction device based on the artificial intelligence-based cyclic neural network provided by the embodiment of the present invention, and the device can be configured in any In the terminal device, error correction processing is performed on the text data to be corrected.

[0035] In specific implementation, the text data to be corrected may be a search sentence entered by the user in a search engine, or a sentence entered by the user in interactive applications such as WeChat and QQ, or service applications such as DuMi.

[0036] In addition, the text data to be corrected may be text data obtained by voice recognition after the user enters a voice-type sentence, or text data directly input by the user, which is not limited here.

[0037] Step 102, using the trained recurrent neural network model to perform error correction processing on the text data to be corrected to generate error-corrected text data.

[0038] Specifically, the corpus of text to be trained can be used to train and generate a recurrent neural network model, so that after obtaining the text data to be corrected, the recurrent neural network model generated by pre-training can be used to correct the text to be corrected deal with.

[0039] It is understandable that using the trained cyclic neural network model, after performing error correction processing on the text data to be corrected, multiple text data may be generated. In the embodiment of the present invention, the text editing distance can be changed by using the language model. And text pronunciation similarity and other methods, the generated multiple text data are scored and sorted according to the scores, so that the text data with the highest score is used as the final corrected text data to improve the reliability of text error correction.

[0040] Combine below figure 2 , The recurrent neural network model used in the embodiment of the present invention will be specifically described.

[0041] Such as figure 2 As shown, the recurrent neural network can be expanded in time to make it more visual. From figure 2 It can be seen that the structure of the recurrent neural network includes the input vector of the input layer {x 1 , X 2 …X n }, the output vector of the output layer {o 1 , O 2 …O n }, the hidden unit node h, the parameter weight matrix U connecting the input layer to the hidden layer node, the parameter weight matrix W connecting the hidden layer nodes, and the parameter weight matrix V from the hidden layer node to the output layer node, each of which is Node sharing.

[0042] It can be seen from the structure of the recurrent neural network that the hidden layer nodes of the recurrent neural network are connected, and the input of the hidden layer includes not only the output of the input layer, but also the output of the previous hidden layer. Therefore, the output of the output layer It is not only related to the input of the input layer at the current moment, but also related to the input of the input layer at the previous moment.

[0043] Specifically, the hidden layer node t step h t Can be determined by the formula h t =f(Ux t +Wh t-1 ) Get, output layer o t Can be determined by the formula o t =softmax(Vh t )get. Among them, h in the formula t-1 Is the hidden layer state at step t-1, f is generally a nonlinear activation function, such as hyperbolic tangent function tanh or modified linear unit ReLU.

[0044] It is understandable that by using the above cyclic neural network model to perform error correction processing on the text data to be corrected, not only the corrected text data is related to a single word in the text data to be corrected, but also the context is integrated. Syntactic information. Therefore, the text error correction method based on the artificial intelligence-based cyclic neural network model provided by the embodiment of the present invention has higher accuracy than the prior art, can better meet the needs of users, and improve user experience.

[0045] In a possible implementation form of the embodiment of the present invention, before performing error correction processing on the text data to be corrected through the trained cyclic neural network model, it is also possible to make a preliminary judgment on whether there are errors in the text data to be corrected to reduce The probability of correct text data being corrected is to improve the accuracy of text error correction. That is, in step 101, it may further include:

[0046] Step 101a: Determine that the language model score corresponding to the text data input by the user is less than a preset value;

[0047] Among them, the language model can be an n-gram model, a maximum entropy model, a maximum entropy Markov model, a neural network model, and so on.

[0048] It is understandable that by using language models, the probability that a sentence is correct can be determined. Therefore, in the embodiment of the present invention, a language model can be used to score text data input by the user. If the score is less than a preset value, it can be determined that there is a greater possibility of error in the sentence. Therefore, it is possible to use the trained recurrent neural network model to perform error correction processing on the text data input by the user only when there is a greater possibility of errors in the sentence, so as to avoid correcting the correct text data into wrong text data and improve the text The accuracy of error correction.

[0049] Among them, the preset value can be determined according to factors such as the type of the language model, the performance of the language model, and the data volume of the text data to be corrected.

[0050] Step 101b, using a preset classification model, determine that the text data input by the user is the text data to be corrected.

[0051] Among them, the classification model may be a classification model such as a decision tree classifier, a selection tree classifier, an evidence classifier, a Bayesian text classifier, a neural network classifier, and so on.

[0052] In specific implementation, a large number of wrong texts and corresponding accurate texts can be used to train the classification model in advance, and when the classification model is used to classify the text data input by the user, if the classification result of the text data is determined to be accurate text, output "1" ", if it is determined that the classification result of the text data is an error text, output "0". After obtaining the text data input by the user, you can input the text data input by the user into the classification model, and when the output result of the classification model is "0", that is, when the classification model determines that the text data input by the user is wrong text, use The trained recurrent neural network model performs error correction processing on the text data input by the user, thereby avoiding correcting the correct text data into wrong text data and improving the accuracy of text error correction.

[0053] It should be noted that in the embodiment of the present invention, a language model and a classification model can also be used comprehensively to make a preliminary judgment on the text data to be corrected. That is, the text data input by the user can be scored using the language model first, and then the text data input by the user can be classified using the preset classification model to determine that the text data input by the user is the text data to be corrected. When it is determined that the language model score corresponding to the user data input by the user is less than the preset value, and the classification model determines that the user data input by the user is incorrect text, the trained recurrent neural network model is used to correct the text data input by the user. Error processing, so as to avoid correcting the correct text data into wrong text data, and improve the accuracy of text error correction.

[0054] The method for text error correction based on artificial intelligence-based cyclic neural network of the embodiment of the present invention first obtains the text data to be corrected, and then uses the trained cyclic neural network model to perform error correction processing on the text data to be corrected to generate the corrected text data. text data. Therefore, by adopting the recurrent neural network model and combining the syntactic information of the context, error correction processing is performed on the text data with errors, which improves the accuracy of error correction, better meets the needs of users, and improves user experience.

[0055] From the above analysis, it can be known that the trained recurrent neural network model can be used to perform error correction processing on the text data to be corrected to generate corrected text data to improve the accuracy of error correction. Combine below image 3 In the text error correction method based on artificial intelligence-based cyclic neural network provided by the embodiment of the present application, the training generation method of the cyclic neural network model is described in detail.

[0056] image 3 It is a flowchart of a method for training and generating a recurrent neural network model according to another embodiment of the present invention.

[0057] Such as image 3 As shown, the method includes:

[0058] Step 301: Obtain a text pair corpus to be trained, the text pair includes an error text and an accurate text corresponding to the error text.

[0059] In specific implementation, the following methods can be used to obtain the text pair corpus to be trained.

[0060] method one

[0061] Recognize two consecutive voices input by the user within a preset time, and generate a first text pair.

[0062] Among them, the preset time can be set as needed. For example, when the user’s speaking rate is slow, or the duration of the user’s input voice is longer, the preset time can be set to a longer time; when the user’s speaking rate is faster, or the user’s input voice has a shorter duration , The preset time can be set to a shorter time, and so on.

[0063] It is understandable that when users input speech in search engines, application software and other applications, the input speech may not be the result they expected due to verbal errors, low voice, ambiguity, etc. At this time, the user will usually After entering the voice once, enter the voice you want to express again. In the embodiment of the present invention, it is possible to recognize the speech input twice consecutively by the user within the preset time, and use the recognition result of the two times as the first text pair. Among them, the recognition result of the voice input for the first time is the wrong text, and the recognition result of the voice input for the second time is the accurate text corresponding to the wrong text.

[0064] Or, in some applications with voice recognition function, after the user inputs the voice for the first time, the application will recognize the voice input by the user and show the recognition result to the user. If the user determines that the voice recognition result is consistent with what he wants to express The sentence is different, the user usually re-enters the voice until the recognition result is the same as the sentence he wants to express. In the embodiment of the present invention, among the N voices continuously input by the user within a preset time, the recognition results of the N-1th and the Nth input voices may be determined as the first text pair. Among them, the recognition result of the voice input at the N-1th time is an erroneous text, and the recognition result of the voice input at the Nth time is an accurate text corresponding to the erroneous text.

[0065] Method Two

[0066] The second text pair is determined according to two text retrieval sentences continuously input by the user within a preset time.

[0067] It is understandable that in the search engine, the user may directly enter the retrieval sentence in text form. When the text retrieval sentence entered for the first time is incorrect, the user will usually enter the correct retrieval sentence again. In the embodiment of the present invention, two text retrieval sentences continuously input by the user within a preset time can be directly determined as the second text pair. Among them, the text search sentence input for the first time is an error text, and the text search sentence input for the second time is an accurate text corresponding to the error text.

[0068] Step 302: Use the text to train the preset cyclic neural network model on the corpus, and determine the trained cyclic neural network model.

[0069] It should be noted that the more corpus of the text to be trained, the better the performance of the generated training recurrent neural network model for training the preset recurrent neural network model. Therefore, in the embodiment of the present invention, a large amount of text pair corpus can be used to train the preset recurrent neural network model.

[0070] During specific implementation, step 302 may specifically include:

[0071] Step 302a: Use a preset cyclic neural network to perform encoding processing on the first error text, and determine a vector matrix corresponding to the error text.

[0072] Step 302b, decode the vector matrix, and output the first text.

[0073] Step 302c, according to the difference between the first text and the accurate text, revise the preset weight coefficient of the recurrent neural network to determine the first revision model.

[0074] It is understandable that using a preset cyclic neural network to encode the first error text refers to using the preset cyclic neural network to process the vector corresponding to the first error text.

[0075] In specific implementation, the weight coefficients of the cyclic neural network model can be preset, and the first error text is input into the preset cyclic neural network model, and then the first error text is encoded and the generated vector matrix is ​​decoded. Processing can generate the corresponding error correction result, that is, the first text.

[0076] By comparing the first text with the accurate text corresponding to the first erroneous text, the first correction coefficient can be determined according to the difference between the first text and the accurate text, thereby revising the weight coefficient and determining the first revision model.

[0077] After that, input the second error text into the preset cyclic neural network model. Through the encoding process of the second error text and the decoding process of the generated vector matrix, the corresponding error correction result can be generated, that is, the second text .

[0078] By comparing the second text with the accurate text corresponding to the second wrong text, the second correction coefficient can be determined according to the difference between the second text and the accurate text, so that the revised weight coefficient can be revised continuously to determine the second Revise the model.

[0079] Repeat the above process, by using a large amount of text to revise the preset recurrent neural network several times, then the final weight coefficient can be determined, and the trained recurrent neural network model can be generated.

[0080] Understandably, figure 2 The recurrent neural network shown can utilize contextual information in the process of mapping between input and output vectors. However, figure 2 The recurrent neural network shown has a limited range of context information that can be stored, which makes the influence of the input of the hidden layer on the output of the network decay as the network loop continues to recurse.

[0081] Therefore, in a better implementation form of the present invention, the cyclic neural network model may be a Long Short-Term Memory (LSTM) cyclic neural network model. That is, step 302 may include:

[0082] The text is used to train the long and short-term memory (LSTM) recurrent neural network model on the corpus to generate the trained long and short-term memory recurrent neural network model.

[0083] Specifically, in the LSTM recurrent neural network model, the connection between hidden layer nodes is a conventional recurrent neural network hidden layer interconnection method, and each node unit of the hidden layer has a linear self-circulation operation by introducing a gate structure.

[0084] Such as Figure 4 As shown in the hidden layer node structure diagram, the gates in the node include control gates, forget gates and output gate structures. Through a unique design structure, LSTM can store a wide range of contextual information. Therefore, using the LSTM cyclic neural network model to perform error correction processing on the text data to be corrected can make the error correction results more accurate.

[0085] In addition, figure 2 The recurrent neural network model shown can process data based on past contextual information, but often ignores future contextual information. In another preferred implementation form of the present invention, the cyclic neural network model can also be a bidirectional cyclic neural network model, such as Figure 5 Shown. That is, step 302 may include:

[0086] Use the text to train the bidirectional cyclic neural network model on the corpus to generate the trained bidirectional cyclic neural network model.

[0087] Specific, such as Figure 5 As shown, the bidirectional cyclic neural network model can be divided into two cyclic neural networks, forward and backward. The forward cyclic neural network processes the input sentence from left to right, and the backward cyclic neural network processes the input sentence from right to right. The input sentence is processed in the reverse direction on the left, and the two recurrent neural networks are connected to an output layer.

[0088] Through the above structure, the output layer can be provided with complete past and future contextual information for each point in the input layer. Therefore, in the embodiment of the present invention, using the trained bidirectional cyclic neural network model to perform error correction processing on the text data to be corrected, not only makes the corrected text data related to a single word in the text data to be corrected, At the same time, it also integrates complete syntactic information of past and future contexts, so that the accuracy of the error correction results is higher, which can better meet the needs of users and improve user experience.

[0089] In the method for text error correction based on artificial intelligence-based recurrent neural network in the embodiment of the present invention, first obtain the text pair corpus to be trained, and then use the text pair corpus to train a preset recurrent neural network model to determine the trained recurrent neural network model . Therefore, by training the recurrent neural network model and using the trained recurrent neural network model, combined with the syntactic information of the context, error correction processing is performed on the text data with errors, which improves the accuracy of error correction and better meets It meets the needs of users and improves user experience.

[0090] Image 6 It is a schematic structural diagram of an artificial intelligence-based cyclic neural network text error correction device according to an embodiment of the present invention.

[0091] Such as Image 6 As shown, the artificial intelligence-based cyclic neural network text error correction device includes:

[0092] The first obtaining module 61 is configured to obtain text data to be corrected;

[0093] The processing module 62 is configured to use the trained cyclic neural network model to perform error correction processing on the text data to be corrected to generate error-corrected text data.

[0094] In a possible implementation form of the embodiment of the present application, the above-mentioned first obtaining module 61 is specifically configured to:

[0095] Determine that the language model score corresponding to the text data input by the user is less than the preset value;

[0096] and / or,

[0097] Using a preset classification model, it is determined that the text data input by the user is the text data to be corrected.

[0098] Among them, the text error correction device based on the artificial intelligence-based cyclic neural network provided in this embodiment can be configured in any terminal device for executing the text error correction method based on the artificial intelligence-based cyclic neural network provided by the embodiment of the present invention , To perform error correction processing on the text data to be corrected.

[0099] It should be noted that the foregoing explanation of the embodiment of the text error correction method based on the artificial intelligence-based cyclic neural network is also applicable to the text error correction device based on the artificial intelligence-based cyclic neural network of this embodiment, and will not be repeated here.

[0100] The text error correction device based on artificial intelligence-based cyclic neural network of the embodiment of the present invention first obtains the text data to be corrected, and then uses the trained cyclic neural network model to perform error correction processing on the text data to be corrected to generate the corrected text data. text data. Therefore, by adopting the recurrent neural network model and combining the syntactic information of the context, error correction processing is performed on the text data with errors, which improves the accuracy of error correction, better meets the needs of users, and improves user experience.

[0101] Figure 7 It is a schematic structural diagram of an artificial intelligence-based cyclic neural network text error correction device according to another embodiment of the present invention.

[0102] Such as Figure 7 Shown in Image 6 On the basis of, this artificial intelligence-based cyclic neural network text error correction device also includes:

[0103] The second obtaining module 71 is configured to obtain a text pair corpus to be trained, the text pair including an error text and an accurate text corresponding to the error text;

[0104] The determining module 72 is configured to use the textual corpus to train a preset recurrent neural network model and determine the trained recurrent neural network model.

[0105] In a possible implementation form of the embodiment of the present application, the above determining module 72 is specifically used for:

[0106] Use the text to corpus to train the long- and short-term memory cyclic neural network model;

[0107] or,

[0108] The text pair corpus is used to train the bidirectional cyclic neural network model.

[0109] In another possible implementation form of the embodiment of the present application, the above-mentioned second obtaining module 71 is specifically configured to:

[0110] Recognize two consecutive voices input by the user within a preset time, and generate the first text pair;

[0111] or,

[0112] The second text pair is determined according to the two text retrieval sentences continuously input by the user within a preset time.

[0113] It should be noted that the foregoing explanation of the embodiment of the text error correction method based on the artificial intelligence-based cyclic neural network is also applicable to the text error correction device based on the artificial intelligence-based cyclic neural network of this embodiment, and will not be repeated here.

[0114] The text error correction device based on artificial intelligence-based cyclic neural network of the embodiment of the present invention first obtains the text data to be corrected, and then uses the trained cyclic neural network model to perform error correction processing on the text data to be corrected to generate the corrected text data. text data. Therefore, by adopting the recurrent neural network model and combining the syntactic information of the context, error correction processing is performed on the text data with errors, which improves the accuracy of error correction, better meets the needs of users, and improves user experience.

[0115] To achieve the foregoing objective, an embodiment of the third aspect of the present invention provides a terminal device, including:

[0116] A memory, a processor, and a computer program stored on the memory and capable of running on the processor, when the above-mentioned processor executes the program, the artificial intelligence-based cyclic neural network text error correction method in the foregoing embodiment is implemented.

[0117] In order to achieve the above objective, an embodiment of the fourth aspect of the present invention provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the artificial intelligence-based recurrent neural network as in the foregoing embodiment is realized. The text error correction method of the network.

[0118] In order to achieve the above objective, an embodiment of the fifth aspect of the present invention proposes a computer program product. When the instruction processor in the computer program product executes, it executes the text of the artificial intelligence-based recurrent neural network in the foregoing embodiment. Error correction method.

[0119] In the description of this specification, descriptions with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" etc. mean specific features described in conjunction with the embodiment or example , Structure, materials or features are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the described specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine the different embodiments or examples and the characteristics of the different embodiments or examples described in this specification without contradicting each other.

[0120] In addition, the terms "first" and "second" are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Therefore, the features defined with "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise specifically defined.

[0121] Any process or method description in the flowchart or described in other ways herein can be understood as a module, segment or part of code that includes one or more executable instructions for implementing custom logic functions or steps of the process , And the scope of the preferred embodiment of the present invention includes additional implementations, which may not be in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order according to the functions involved. It is understood by those skilled in the art to which the embodiments of the present invention belong.

[0122] The logic and/or steps represented in the flowchart or described in other ways herein, for example, can be considered as a sequenced list of executable instructions for implementing logic functions, and can be embodied in any computer-readable medium, For use by instruction execution systems, devices, or equipment (such as computer-based systems, systems including processors, or other systems that can fetch instructions from instruction execution systems, devices, or equipment and execute instructions), or combine these instruction execution systems, devices Or equipment. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transmit a program for use by an instruction execution system, device, or device or in combination with these instruction execution systems, devices, or devices. More specific examples (non-exhaustive list) of computer readable media include the following: electrical connections (electronic devices) with one or more wiring, portable computer disk cases (magnetic devices), random access memory (RAM), Read only memory (ROM), erasable and editable read only memory (EPROM or flash memory), fiber optic devices, and portable compact disk read only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable media on which the program can be printed, because it can be used, for example, by optically scanning the paper or other media, and then editing, interpreting, or other suitable media if necessary. The program is processed in a manner to obtain the program electronically and then stored in the computer memory.

[0123] It should be understood that each part of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if it is implemented by hardware, as in another embodiment, it can be implemented by any one or a combination of the following technologies known in the art: a logic gate circuit for implementing logic functions on data signals Discrete logic circuits, application-specific integrated circuits with suitable combinational logic gates, programmable gate array (PGA), field programmable gate array (FPGA), etc.

[0124] Those of ordinary skill in the art can understand that all or part of the steps carried in the method of the foregoing embodiments can be implemented by a program instructing relevant hardware to complete, and the program can be stored in a computer-readable storage medium. When executed, it includes one of the steps of the method embodiment or a combination thereof.

[0125] In addition, the functional units in the various embodiments of the present invention may be integrated into one processing module, or each unit may exist alone physically, or two or more units may be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware or software functional modules. If the integrated module is implemented in the form of a software function module and sold or used as an independent product, it may also be stored in a computer readable storage medium.

[0126] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. A person of ordinary skill in the art can comment on the above-mentioned embodiments within the scope of the present invention. The embodiment undergoes changes, modifications, replacements and modifications.