A sentence error correction method and device
By using the associated probability recognition model to identify and replace the abnormal participle in the statement, the problems of high cost and low efficiency in the prior art are solved, and automatic error correction and efficient statement error correction are achieved.
Patent Information
- Application Number
- CN201911157807.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2019-11-22
- Publication Date
- 2025-04-25
- Estimated Expiration
- 2039-11-22
AI Technical Summary
The existing statement error correction technology requires high labor and time costs, and the proofreading process is low and cannot meet the verification needs of today's huge amount of electronic data.
The preset correlation probability identification model recognizes abnormal participle in the target statement and replaces it with legal participle to generate an error-corrected statement.
Automatic error correction of target statements is achieved, the efficiency of statement error correction is improved, the accuracy of output statements is ensured, and the verification needs of large amounts of electronic data is met.
Smart Images

Figure CN112836495B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of data processing, and in particular relates to a statement error correction method and device. Background Art
[0002] With the continuous advancement of the electronic process, more and more information is stored in an electronic way. For example, voice signals can be converted into text data through voice recognition, and the content that users need to write can be converted into electronic documents through input methods. When users input text information through voice signals or input methods, sentence errors often appear in the output text data. For example, when the voice signal is converted into text data, dialects are included, resulting in the inability to accurately recognize the text data expressed by the user; for example, when users use input methods to input electronic documents, pinyin errors or similar character errors may occur, resulting in the output electronic documents containing typos.
[0003] For the above situation, the existing sentence error correction technology mainly relies on manual proofreading of massive amounts of data, which requires high manpower and time costs. Although it is more professional and accurate, the proofreading process is less efficient. Especially in today's world where electronic data is growing at an exponential rate, manual proofreading is obviously unable to meet actual verification needs. Summary of the invention
[0004] In view of this, an embodiment of the present invention provides a sentence error correction method and device to solve the problem that the existing sentence error correction technology requires high manpower and time costs and the proofreading process is inefficient.
[0005] A first aspect of an embodiment of the present invention provides a sentence error correction method, comprising:
[0006] Identify abnormal word segmentations contained in the target sentence through a preset association probability recognition model;
[0007] The abnormal participles in the target sentence are replaced with legal participles to obtain a corrected sentence.
[0008] A second aspect of an embodiment of the present invention provides a sentence error correction device, including:
[0009] An abnormal word segmentation recognition unit is used to recognize abnormal word segmentations contained in a target sentence through a preset association probability recognition model;
[0010] The corrected sentence output unit is used to replace the abnormal participles in the target sentence with legal participles to obtain a corrected sentence.
[0011] A third aspect of an embodiment of the present invention provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the first aspect when executing the computer program.
[0012] A fourth aspect of an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the first aspect are implemented.
[0013] The sentence error correction method and device provided by the embodiment of the present invention have the following beneficial effects:
[0014] The embodiment of the present invention imports the target sentence that needs to be corrected into the association probability recognition module, identifies whether the target sentence contains abnormal participles with low correlation with the entire sentence, and replaces all abnormal participles in the target sentence with corresponding legal participles, generates and outputs the corrected sentence, and achieves the purpose of automatically correcting the target sentence. Compared with the existing sentence correction technology, the present invention does not need to rely on manual sentence correction, but can perform sentence correction on the target sentence through the association probability recognition model, which ensures the accuracy of the output sentence while also improving the efficiency of sentence correction, meeting the current verification needs of huge amounts of electronic data. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0016] Figure 1 is a flow chart of an implementation of a sentence error correction method provided by the first embodiment of the present invention;
[0017] Figure 2 This is a specific implementation flow chart of a sentence error correction method S101 provided in the second embodiment of the present invention;
[0018] Figure 3 It is a specific implementation flow chart of a sentence error correction method provided by the third embodiment of the present invention;
[0019] Figure 4 is a specific implementation flow chart of a sentence error correction method provided by the fourth embodiment of the present invention;
[0020] Figure 5 This is a specific implementation flow chart of a sentence error correction method S401 provided in the fifth embodiment of the present invention;
[0021] Figure 6 This is a specific implementation flow chart of a sentence error correction method S4011 provided in the sixth embodiment of the present invention;
[0022] Figure 7 This is a specific implementation flow chart of a sentence error correction method S4013 provided in the seventh embodiment of the present invention;
[0023] Figure 8 is a specific implementation flow chart of a sentence error correction method S102 provided in the eighth embodiment of the present invention;
[0024] Fig. 9 It is a structural block diagram of a sentence error correction device provided by an embodiment of the present invention;
[0025] Fig.10 It is a schematic diagram of a terminal device provided by another embodiment of the present invention. DETAILED DESCRIPTION
[0026] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.
[0027] The embodiment of the present invention imports the target sentence that needs to be corrected into the association probability recognition module, identifies whether the target sentence contains abnormal participles with a low correlation with the entire sentence, extracts legal participles corresponding to the abnormal participles from the corpus, and replaces all abnormal participles in the target sentence with corresponding legal participles, generates and outputs corrected sentences, thereby achieving the purpose of automatically correcting the target sentence and solving the problem that the existing sentence correction technology requires high manpower and time costs and the proofreading process has low efficiency.
[0028] In the embodiment of the present invention, the execution subject of the process is a terminal device, which includes but is not limited to: a server, a computer, a smart phone, a smart TV, a tablet computer and other devices capable of executing statement error correction operations. Figure 1 The following is a flowchart of the implementation of the sentence error correction method provided by the first embodiment of the present invention, which is described in detail as follows:
[0029] In S101, abnormal word segmentations included in the target sentence are identified through a preset association probability recognition model.
[0030] In this embodiment, the target sentence can be a text, or it can be text data obtained by converting a voice signal through a voice recognition module. The terminal device can import the target sentence into an association probability recognition module, and the association probability recognition model is used to detect the association between different character strings in the target sentence. If the association between a certain character string and all other characters in the target sentence is less than a preset association threshold, it is recognized that the character string contains abnormal participles, and the abnormal participles in the character string are extracted.
[0031] Optionally, in this embodiment, the specific method of identifying abnormal word segmentation can be: the association probability recognition model is configured with a sentence topic detection network and a topic association calculation network, and the terminal device first imports the target sentence into the sentence topic detection network, determines the sentence topic of the target sentence, and the sentence topic can be represented by multiple keywords, and marks the contribution weight of each keyword to the sentence topic. After determining the sentence topic of the target sentence, the target sentence can be segmented, and the association between each word segmentation and the sentence topic can be identified respectively. If the association between a word segmentation and the sentence topic is lower than a preset association threshold, the word segmentation is identified as an abnormal word segmentation, and based on the sentence topic and the abnormal word segmentation, the corresponding legal word segmentation is output.
[0032] Optionally, in this embodiment, the terminal device is provided with a corpus, which can be constructed based on a general dictionary containing phrases. The terminal device can select a phrase related to the abnormal participle in the general dictionary from the corpus as a candidate participle, for example, a phrase with the same pinyin, similar pronunciation, or similar glyph as the abnormal participle as a candidate participle, and detect the correlation between each candidate participle and the target sentence, and select a candidate participle with the highest correlation as a legal participle.
[0033] Optionally, in this embodiment, in addition to constructing a main prediction library based on a general dictionary, the corpus in the terminal device can also construct multiple auxiliary word lists, for example, corresponding auxiliary word lists are created based on three different types of Chinese confusion sets, professional vocabulary sets, and spoken recognition sets. The above different sets can be manually configured by the user, or extracted from the original sentence obtained by recognition, and imported into the corresponding auxiliary word list based on the type of word segmentation in the original sentence, thereby achieving the purpose of automatically expanding the auxiliary word list. It should be emphasized that in addition to being used when selecting legal word segmentation, the auxiliary word list can also be applied to the initial error correction model, thereby improving the accuracy of the initial error correction model when performing the operation of replacing abnormal word segmentation and the range of identifiable abnormal word segmentations.
[0034] In S102, the abnormal participles in the target sentence are replaced with legal participles to obtain a corrected sentence.
[0035] In this embodiment, the terminal device can mark each abnormal participle on the target sentence, and replace the legal participle corresponding to the abnormal participle according to the position of the abnormal participle in the target sentence, and identify the target sentence after the abnormal participle is replaced as a corrected sentence, and output the corrected sentence.
[0036] Optionally, in this embodiment, after the terminal device obtains the corrected sentence, it can execute other business logic based on the corrected sentence. For example, the terminal device is a smart TV, which can receive the user's voice signal and search for the corresponding TV program. In this case, the terminal device can search for the corresponding TV program based on the corrected sentence after the sentence is corrected, thereby improving the accuracy of the TV search operation.
[0037] From the above, it can be seen that a sentence correction method provided by an embodiment of the present invention imports a target sentence that needs to be corrected into an associated probability recognition module, identifies whether the target sentence contains abnormal participles with a low degree of correlation with the entire sentence, and replaces all abnormal participles in the target sentence with corresponding legal participles, generates and outputs corrected sentences, and achieves the purpose of automatically correcting the target sentence. Compared with the existing sentence correction technology, the present invention does not need to rely on manual sentence correction, but can perform sentence correction on the target sentence through an associated probability recognition model, which ensures the accuracy of the output sentence while also improving the efficiency of sentence correction, meeting the current verification needs of large amounts of electronic data.
[0038] Figure 2 FIG. 1 shows a specific implementation flow chart of a sentence error correction method S101 provided in the second embodiment of the present invention. Figure 1 , relative to Figure 1 In the embodiment, a sentence error correction method S101 provided in this embodiment includes: S1011 to S1014, which are described in detail as follows:
[0039] Furthermore, identifying abnormal word segmentations contained in the target sentence by using a preset association probability recognition model includes:
[0040] In S1011, a plurality of consecutive characters are selected from the target sentence to obtain a character string.
[0041] In this embodiment, for abnormal word segmentation, each character in the word segmentation often has a low correlation with other characters. Therefore, the terminal device can select multiple consecutive characters in the target sentence to form a string, and identify the correlation between different characters in the string, so as to determine whether the string contains abnormal word segmentation.
[0042] In this embodiment, the terminal device can randomly configure the number of character strings, and each time a character string is selected from the target sentence, the number of the character string can be obtained based on the random algorithm configuration. For example, in the first selection, the number of character strings is 5, and a character string consisting of 5 characters is selected from the target sentence; and in the next selection operation, the number of character strings can be configured as 6, and a character string consisting of 6 characters is selected from the target sentence. The above selection method can be performed until every character in the target sentence is selected and constitutes a character string, and then the operation of S1012 is performed to obtain multiple target characters. It should be noted that when selecting consecutive characters to generate a character string, the consecutive number can be determined by a random algorithm.
[0043] Further, as another embodiment of the present invention, S1011 may specifically be:
[0044] In S201, a preset number of consecutive characters are framed in the target sentence to obtain the character string.
[0045] In this embodiment, the terminal device is provided with a fixed number of characters, that is, the preset number mentioned above, and selects a character string containing multiple consecutive characters on the target sentence according to the preset number. The characters contained in each character string are consistent with the preset number mentioned above. For example, if the preset number is 4, 4 characters are extracted from the target sentence to generate a character string. Of course, in this embodiment, the terminal device can also perform frame extraction in units of segmentation. For example, if the preset number is 2, the terminal device can first divide the target sentence into multiple different segmentations, and select 2 consecutive segmentations each time to generate a character string consisting of 2 segmentations.
[0046] For example, if the target sentence is "I ate bread today" and the preset number is 3, then in the process of traversal and selection, five character strings "Today I", "Today I eat", "I ate", "Ate noodles" and "Eat bread" can be generated, and each character string contains 3 characters. The number of characters in the string is consistent with the size of the sliding window.
[0047] It should be noted that the terminal device can extract multiple character strings from the target sentence by configuring a preset number, and perform operations S1012 to S1014 on each character string respectively.
[0048] In 1012, the association degree between each of the characters and the remaining characters in the character string is calculated respectively.
[0049] In this embodiment, after extracting the character string, the terminal device can calculate the correlation between each character in the character string and other characters. For example, if the character string is [x] m ={x1,x2,…,x m}, where the value of m can be the size of the sliding window. The terminal device calculates the correlation degree between the i-th character in the string and other characters, denoted as P i (x i 丨x1, x2, …, x m ).
[0050] Optionally, the calculation method of the correlation degree can be: The terminal device can obtain multiple standard documents, such as articles published in magazines, news, textbooks and other documents with high grammar accuracy. The terminal device can detect the number of occurrences of the character and other characters in the standard document, and based on the number of occurrences, import a preset hash conversion function, and identify the output hash value as the correlation degree between the character and other characters.
[0051] In S1013, according to the correlation degrees corresponding to the respective characters, determine the word segmentation correlation coefficient of the string.
[0052] In this embodiment, after the terminal device calculates the correlation degrees between each character in the string and other characters, it can multiply the respective correlation degrees, and identify the product as the word segmentation correlation coefficient of the character, that is . If the value of the word segmentation correlation coefficient is larger, it indicates that the correlation between the respective characters in the above string is higher, that is, each character is used to represent the same theme, and the probability of a grammar error at this time is smaller; on the contrary, if the value of the word segmentation correlation coefficient is smaller, it indicates that the correlation between the respective characters in the string is lower, and there may be characters in the string that are irrelevant to the theme of the entire string, and the probability of a grammar error at this time is higher.
[0053] In this embodiment, if the word segmentation correlation coefficient is greater than or equal to the correlation threshold, it is recognized that the string does not contain abnormal word segmentation; on the contrary, if the word segmentation correlation coefficient is less than the correlation threshold, the operation of S1014 is performed.
[0054] Optionally, in this embodiment, the correlation threshold is specifically obtained by training with the KenLM tool to obtain an empirical threshold of Chinese language perplexity, so as to improve the accuracy of the correlation threshold.
[0055] In S1014, if the word segmentation correlation coefficient is less than the preset correlation threshold, it is recognized that the string contains abnormal word segmentation.
[0056] In this embodiment, if the terminal device detects that the word segmentation correlation coefficient of the string is less than the correlation threshold, it is recognized that the string has a grammar error, that is, there is abnormal word segmentation.
[0057] In an embodiment of the present invention, a target sentence is divided into multiple strings through a sliding window, and the correlation degree of each character in each string is calculated respectively, the word segmentation correlation coefficient of the string is calculated, and abnormal word segmentations are extracted based on the word segmentation correlation coefficient being less than the correlation threshold, thereby realizing the recognition of abnormal word segmentations and improving the recognition accuracy.
[0058] Figure 3 The following is a flowchart showing a specific implementation of a sentence error correction method provided by the third embodiment of the present invention. Figure 3 , relative to Figure 1 The embodiment, a sentence correction method provided by the embodiment, further includes: S301 to S307, which are described in detail as follows:
[0059] Furthermore, before identifying abnormal participles contained in the target sentence to be corrected by using a preset association probability recognition model and selecting legal participles of the abnormal participles from the corpus, it also includes:
[0060] In S301, model learning parameters are obtained; the model learning parameters include: initial efficiency, minimum efficiency, efficiency adjustment step length, and number of learning cycles.
[0061] In this embodiment, before executing the sentence error correction operation, the terminal device can train and learn the associated probability recognition model, so as to improve the accuracy of subsequent recognition operations. Of course, the terminal device can also be set with a parameter update cycle, and the terminal device is provided with an update timer. If it is detected that the update timer reaches the preset parameter update cycle, the operation of S301 is performed, and the learning parameters in the associated probability recognition model can be updated regularly, thereby improving the accuracy of the parameters of the associated probability recognition model.
[0062] In this embodiment, the model learning parameters can be set by the system default or manually configured by the user. The model learning parameters include initial efficiency, minimum efficiency, efficiency adjustment step size, and number of learning cycles, and the learning efficiency is dynamically adjusted during the training and learning process based on the above four parameters. Since the lower the learning efficiency, the slower the change rate of the loss function, although the use of low learning efficiency can ensure that no local minimum is missed, it also means that it will take longer to converge the training. Therefore, the embodiment of the present invention adopts a method of dynamically adjusting the learning efficiency, which takes into account the training time while ensuring the accuracy of the training.
[0063] In S302 , the learning efficiency of the Nth learning cycle is set according to the initial efficiency; the initial value of N is 1.
[0064] In this embodiment, the terminal device will perform multiple training and learning on the association probability recognition model based on the number of learning cycles. In the first learning cycle, the terminal device will set the learning efficiency of this time to the initial efficiency.
[0065] In S303, the association probability recognition model is trained and learned using a plurality of training samples at the Nth learning cycle learning efficiency until an output value of a loss function of the association probability recognition model is less than a preset convergence threshold.
[0066] In this embodiment, the terminal device stores a training database, and the terminal device can extract multiple training samples from the training database to train the association probability recognition model based on the learning efficiency of the current cycle. During the training process, the terminal device will adjust the learning parameters in the association probability recognition model, and detect the loss function of the association probability recognition model based on the learning parameters. When the output value of the loss function is less than the convergence threshold, the association probability recognition model is recognized to converge, that is, the training of this learning cycle has been completed.
[0067] Optionally, the training database may be training data manually configured by a user, or may be obtained based on the original sentence with the corrected sentence packaged. It should be noted that the training sample includes the original sample before the correction, and the legal sample after the correction.
[0068] In S304, the value of N is incremented by 1, and the learning efficiency of the N-1th learning cycle is reduced based on the efficiency adjustment step, and the adjusted learning efficiency of the N-1th learning cycle is identified as the learning efficiency of the Nth learning cycle.
[0069] In this embodiment, after completing the training and learning operation of this cycle, the terminal device needs to adjust the learning efficiency of the next cycle. At this time, the value of N will be increased by 1 to jump to the next learning cycle. The terminal device will adjust the efficiency step length to reduce the learning efficiency of the current cycle. The efficiency adjustment step length can be an adjustment ratio or an absolute value of the learning efficiency. For example, the efficiency adjustment step length is an adjustment ratio, such as 50%. If the current learning efficiency is 0.01, the learning efficiency of the next learning cycle is 0.005.
[0070] In this embodiment, after calculating the learning efficiency of the Nth learning cycle, the terminal device will detect whether the learning efficiency is legal, that is, whether the learning efficiency is less than the minimum efficiency. If so, the learning efficiency is identified as illegal learning efficiency, and operation S305 is executed; otherwise, if the learning efficiency is greater than or equal to the minimum efficiency, the learning efficiency is identified as legal learning efficiency, and operation S306 is executed.
[0071] In S305, if the learning efficiency is less than the minimum efficiency, the learning efficiency is set to the initial efficiency, and the operation of training and learning the association probability recognition model through multiple training samples at the Nth learning cycle learning efficiency is returned to be executed.
[0072] In this embodiment, if the learning efficiency is less than the minimum efficiency, the terminal device will set the learning efficiency of the Nth learning cycle as the initial efficiency, and train the above-mentioned association probability recognition model based on the learning efficiency of this cycle until the output value of the loss function of the association probability recognition model is less than the convergence threshold.
[0073] In S306, if the learning efficiency is greater than or equal to the minimum efficiency, the operation of training and learning the association probability recognition model using multiple training samples at the Nth learning cycle learning efficiency is returned to be executed.
[0074] In this embodiment, if the learning efficiency is greater than or equal to the minimum efficiency, the association probability recognition model can be trained and learned based on the adjusted learning efficiency until the output value of the loss function of the association probability recognition model is less than the convergence threshold.
[0075] See Table 1, which is a schematic diagram of a dynamic adjustment of learning efficiency provided by this embodiment. The initial efficiency is 0.01, the minimum efficiency is 0.001, and the adjustment step is 50%. When it is detected that the learning efficiency of the current cycle is less than the minimum efficiency, it will be set as the initial efficiency.
[0076]
[0077] Table 1
[0078] In S307, if the value of N is greater than the number of learning cycles, it is determined that the association probability recognition model has been trained.
[0079] In this embodiment, if the terminal device detects that the value of N is greater than the number of learning cycles, it recognizes that the training learning operation has been completed and the learning parameters of the association probability recognition model have been adjusted. At this time, the original sentence can be corrected through the adjusted association probability recognition model.
[0080] See Table 2, which is a comparison table of training effects of fixed learning efficiency and dynamically adjusted learning efficiency provided by an embodiment of the present invention. It can be determined from Table 2 that by dynamically adjusting the learning efficiency, not only the loss rate and accuracy can be improved, but also the training time can be reduced.
[0081]
[0082] Table 2
[0083] In the embodiment of the present invention, in the process of training and learning the association probability recognition model, the learning efficiency of different learning cycles is dynamically adjusted, which can ensure the accuracy of training while taking into account the training duration.
[0084] Figure 4 FIG. 4 shows a specific implementation flow chart of a sentence error correction method provided by the fourth embodiment of the present invention. Figure 4 , relative to Figure 1-3 The embodiment, a sentence correction method provided by the embodiment, before identifying abnormal segmentation words contained in the target sentence to be corrected by using a preset association probability recognition model and selecting legal segmentation words of the abnormal segmentation words from the corpus, further includes: S401, which is described in detail as follows:
[0085] In S401, the original sentence to be corrected is imported into the initial error correction model to generate a target sentence of the original sentence.
[0086] In this embodiment, the terminal device can be an intelligent voice terminal, such as an intelligent voice TV, an intelligent voice mobile phone, or other devices equipped with an intelligent voice assistant. In this case, the intelligent voice device has a built-in automatic speech recognition (ASR) module, which is used to collect voice signals in the environment where the intelligent voice terminal is located, and convert the voice signals into text information, that is, the original sentence mentioned above, and perform sentence correction on the original sentence through S401 to S403. Optionally, the intelligent voice terminal is associated with an upper server, which stores voice libraries of various languages, such as Mandarin voice libraries, English voice libraries, and can also be provided with local dialect voice libraries, such as Sichuan dialect voice libraries, Shanghai dialect voice libraries, etc. The terminal device can receive the user's input command, download the voice library corresponding to the input command from the upper server to the local memory, and then convert the voice signal into text through the voice library corresponding to the user. Preferably, the terminal device can collect the user's voice information, for example, prompt the user to read the standard paragraph in the target language, and identify the language type of the target language based on the voice signal corresponding to the standard paragraph, and then download the voice library corresponding to the language type from the upper server. Of course, the intelligent voice terminal can store multiple different voice libraries in the local memory at the same time.
[0087] In this embodiment, in addition to converting the user's voice signal into original sentences, the terminal device can also receive electronic documents sent by the user, divide the electronic documents into sentences, divide the text information in the electronic documents into multiple original sentences, and perform sentence correction operations on each original sentence.
[0088] In this embodiment, the terminal device can also be an error correction server, which can receive error correction requests sent by each user terminal, and each error correction request carries the original sentence to be corrected. The error correction server can store each error correction request in the error correction task pool based on the reception time of each error correction request, and create multiple error correction threads, obtain the error correction request from the error correction task pool through the error correction thread, perform error correction operations on the original original sentence, and return the corrected sentence to the user terminal after error correction. In this way, the pressure of the data processing module in the user terminal can be reduced, and all error correction operations are integrated into the error correction server for processing by handing over the error correction operation to the upper server, further reducing the cost of the user terminal, and realizing the function of sentence correction.
[0089] In this embodiment, the terminal device is configured with an initial error correction model, which can identify sentence errors in the original sentence. The sentence errors include two types: grammatical errors and typos. Among them, grammatical errors include but are not limited to: word order errors, word usage errors, missing word errors and multiple word errors; typos include but are not limited to: homophonic errors (such as: admire eyes / match glasses), confusing sound errors (such as: wandering niece / Cowherd and Weaver Girl), similar characters (such as: sorghum / sorghum), pinyin errors (such as: keshu- cough number / kesou- cough), dialect errors, etc. The initial error correction model can mark the sentence errors in the original sentence, and based on the error type of the sentence error, repair the error and output the above-mentioned corrected sentence.
[0090] Optionally, in this embodiment, the initial error correction model can be a multiple recurrent neural network. The terminal device performs a segmentation operation on the original sentence, that is, divides the original sentence into multiple words, i.e., segmentations, and imports each segmentation into the multiple recurrent neural network in turn based on the order of appearance of each segmentation in the original sentence. The multiple recurrent neural network is specifically an N-layer recurrent neural network, and N is a positive integer greater than 2. When the multiple recurrent neural network performs abnormal identification on the t-th segmentation, it will determine whether the t-th segmentation is an abnormal segmentation based on the correlation between the t-th segmentation to the t+N-th segmentation. If the correlation is lower than the preset correlation threshold, the t-th segmentation is identified as an abnormal segmentation and the abnormal segmentation is replaced; wherein t is a positive integer less than or equal to the number of segmentations contained in the original sentence.
[0091] In this embodiment, the initial error correction model is connected in series with an associated probability recognition model, that is, the target sentence output by the initial error correction model is used as the input sentence of the associated probability recognition model to achieve secondary sentence error correction.
[0092] In S402, abnormal segmentations contained in the target sentence to be corrected are identified through a preset association probability recognition model, and legal segmentations of the abnormal segmentations are selected from the corpus.
[0093] In S403, the abnormal participles in the target sentence are replaced with the legal participles, the target sentence after the replacement is identified as a corrected sentence, and the corrected sentence is output.
[0094] Since the specific implementation method of S402 and S403 is the same as that of S102 and S103, the specific description can refer to the relevant description of S102 and S103, which will not be repeated here.
[0095] Figure 5 FIG. 5 shows a specific implementation flow chart of a sentence error correction method S401 provided in the fifth embodiment of the present invention. Figure 5 , relative to Figure 4 In the embodiment, the sentence error correction method provided in this embodiment includes S401: S4011 to S4013, which are described in detail as follows:
[0096] Furthermore, the step of importing the original sentence to be corrected into the initial error correction model to generate a target sentence of the original sentence includes:
[0097] In S4011, the original sentence is encoded by a preset sentence encoding network, and an encoding sequence of the original sentence is output.
[0098] In this embodiment, the initial error correction model includes three modules, namely, a sentence encoding network, an attention calculation model, and a sentence decoding network, which are connected in series by the above three modules. Among them, the sentence encoding network can be constructed by two bidirectional GRU networks, and the sentence decoding network can be obtained by connecting two unidirectional long-term and short-term LSTM networks in series. Among them, the input of the initial error correction model is a Chinese word sequence, and the output is also a Chinese word sequence. Therefore, the terminal device can perform word segmentation operations on the original sentence to generate a Chinese sequence corresponding to the original sentence. Among them, the sentence encoding network is used to convert a variable-length input sequence into a fixed-length vector; and the sentence decoding network is used to decode a fixed-length vector into a variable-length output sequence. Through the above conversion, the sentence errors contained in the original sentence can be filtered out.
[0099] In this embodiment, the terminal device imports the Chinese sequence after the word segmentation operation on the original sentence into the sentence encoding network, and outputs the word vector of the Chinese sequence. Preferably, the terminal device can identify the standard vector size and detect whether the size of the currently output word vector is consistent with the standard vector size. If not, the output word vector is adjusted to the standard vector by supplementing invalid data or compressing the word vector.
[0100] In S4012, the coding sequence is introduced into an attention calculation model to calculate the sentence weight of each element in the coding sequence in the original sentence.
[0101] In this embodiment, inspired by the human brain's use of the attention mechanism to solve the problem of information overload, the attention mechanism is used in the artificial neural network model to improve information processing efficiency. The terminal device is provided with an attention calculation model, which can identify the attention of each element in the coding sequence. For elements with higher attention, more resources can be used for error correction; and for elements with lower attention, fewer resources can be used for error correction, thereby avoiding information overload. The attention calculation model can be implemented using an Attention network.
[0102] Optionally, the attention calculation model can be implemented by the following code:
[0103] def call(self, inputs):
[0104] q, v, v_mask = inputs
[0105] k = v
[0106] mv = SAT.max(v - (1. - v_mask) * 1e10, axis=1, keepdims=True)
[0107] mv = mv + SAT.zeros_like(q[:, :, :1])
[0108] qw = SAT.dot(q, self.kernel)
[0109] a = SAT.batch_dot(qw, k, [2, 2]) / 10.
[0110] a = (1- SAT.permute_dimensions(v_mask, [0, 2, 1])) * 1e10
[0111] a = SAT.softmax(a)
[0112] o = SAT.batch_dot(a, v, [2, 1])
[0113] return SAT.concatenate([o, q, mv], 2)
[0114] In S4013, the sentence weight and the encoding sequence are imported into a preset sentence decoding network to generate a target sentence corresponding to the original sentence.
[0115] In this embodiment, after calculating the sentence weights of each element in the coding sequence, the terminal device can create an attention matrix and import the attention matrix and the coding sequence into the sentence decoding network, which can convert the word vectors in the coding sequence into Chinese sequences, and re-integrate the Chinese sequences to output the target sentence.
[0116] Optionally, in this embodiment, a fully connected layer can be cascaded at the output end of the sentence decoding network, and the fully connected layer uses tanh as the activation function, cross entropy as the loss function, and Adam as the optimizer. Here, the loss function is specially explained. Since the sentence decoding network outputs the probability distribution of each word, the cross entropy loss function can be used as the loss function.
[0117] In an embodiment of the present invention, the original sentence is converted into a coding sequence through a sentence encoding network, the input data can be standardized, and an attention calculation model is added between the sentence encoding network and the sentence decoding network, which can improve the output efficiency of the target sentence. While improving the processing efficiency, it can also reduce the sentence errors contained in the target sentence.
[0118] Figure 6 FIG. 4 shows a specific implementation flow chart of a sentence error correction method S4011 provided in the sixth embodiment of the present invention. Figure 6 , relative to Figure 5 In the embodiment, a sentence error correction method S4011 provided in this embodiment includes: S601 to S603, which are described in detail as follows:
[0119] Furthermore, the original sentence is encoded by a preset sentence encoding network, and an encoding sequence of the original sentence is output, including
[0120] In S601, the original sentence is divided into at least one short sentence.
[0121] In this embodiment, in order to ensure the accuracy of sentence error correction and the computing efficiency of the processor, the terminal device needs to standardize the original sentence to ensure that the data input into the sentence encoding network is consistent with the standard format. Based on this, the terminal device can parse the original sentence and divide the original sentence into at least one short sentence according to the semantics of the original sentence, wherein the actual number of characters in the short sentence is not greater than the preset standard number.
[0122] In this embodiment, if it is detected that the actual number of characters of any short sentence obtained by division is equal to the preset standard number, the short sentence is identified as a standard sentence and the operation S603 is directly executed; conversely, if it is detected that the actual number of characters of any short sentence is less than the preset standard number, the operation S602 is executed.
[0123] In S602, if the actual number of characters in any of the short sentences is less than the preset standard number, invalid characters are filled into the short sentence to generate a standard sentence.
[0124] In this embodiment, in order to make full use of the computing resources within the terminal device, the data input to the sentence encoding network needs to be converted into a unified data format. In this case, data filling technology can be used, that is, by supplementing invalid characters on the basis of short sentences with fewer characters, so that the data input to the sentence encoding network remains in a unified format, thereby improving processor efficiency.
[0125] In this embodiment, the terminal device can add invalid characters at the beginning or end of the short sentence, or determine multiple insertion points based on the actual number of characters in the short sentence, and fill in invalid characters at each insertion point. The invalid character can be 0 or other agreed invalid characters, which are not limited here.
[0126] In S603, all the standard sentences are encoded by a preset sentence encoding network, and an encoding sequence of the original sentence is output.
[0127] In this embodiment, the terminal device can import each calibrated standard tool into the sentence coding network and output the coding sequence of the original sentence. The specific steps of outputting the coding sequence can be referred to the description of S4011 and will not be repeated here.
[0128] In the embodiment of the present invention, by dividing the original sentence into multiple short sentences and filling invalid characters in the short sentences to standardize each short sentence, the data processing efficiency can be improved, and the response speed of sentence error correction is further improved.
[0129] Figure 7 FIG. 4 shows a specific implementation flow chart of a sentence error correction method S4013 provided in the seventh embodiment of the present invention. Figure 7 , relative to Figure 6 In the embodiment, a sentence error correction method S4013 provided in this embodiment includes: S701-S702, which are described in detail as follows:
[0130] Furthermore, the step of importing the sentence weight and the encoding sequence into a preset sentence decoding network to generate a target sentence corresponding to the original sentence includes:
[0131] In S701, an invalid character filtering matrix of the original sentence is generated based on the invalid characters contained in each of the standard sentences.
[0132] In this embodiment, since invalid characters are filled in the process of generating the standard sentence, and the invalid character part is meaningless in the process of performing the error correction operation, it is not necessary to calculate the loss rate for the prediction of the invalid character position, so it is necessary to construct an invalid character filter matrix according to the invalid character position to eliminate the influence of this part in calculating the loss rate.
[0133] In S702, the invalid character filtering matrix, the sentence weight and the encoding sequence are imported into a preset sentence decoding network to generate a target sentence corresponding to the original sentence.
[0134] In this embodiment, the terminal device creates an invalid character filter matrix based on the addition position and number of each invalid character in the short sentence, and imports the invalid character filter matrix simultaneously in the process of converting the coding sequence into the target sentence to mask the loss value of the corresponding position, thereby reducing the impact of invalid characters on the final output result.
[0135] Optionally, the creation process of the invalid character filter matrix can be implemented by the following statement:
[0136] x_mask = Lambda(lambda x:SAT.cast(SAT.greater(SAT.expand_dims(x, 2),0), 'float32'))(x)
[0137] y_mask = Lambda(lambda x: SAT.cast(SAT.greater(SAT.expand_dims(x, 2),0), 'float32'))(y)
[0138] cross_entropy = SAT.loss=sparse_categorical_crossentropy(y_in[:, 1:],xy[:, :-1])
[0139] loss = SAT.sum(cross_entropy * y_mask[:, 1:, 0]) / SAT.sum(y_mask[:,1:, 0])
[0140] In the embodiment of the present invention, an invalid character matrix about invalid characters is constructed, thereby avoiding the influence of invalid characters in calculating the loss amount, thereby improving the error correction accuracy.
[0141] Figure 8 FIG. 8 is a flowchart showing a specific implementation of a sentence error correction method S102 provided in the eighth embodiment of the present invention. Figure 8 , relative to Figure 1-4 In any of the embodiments, a sentence error correction method S102 provided in this embodiment includes: S1021 to S1023, which are described in detail as follows:
[0142] In S1021, candidate participles associated with the abnormal participles are selected from a preset corpus.
[0143] In this embodiment, the terminal device has a built-in corpus, and the corpus has associations between different phrases. The terminal device can query the association corresponding to the abnormal participle and determine the candidate participle associated with the abnormal participle. Among them, the candidate participle is specifically a participle determined according to the participle attributes such as the pronunciation or shape of the abnormal participle, and the candidate participle can be a participle that can be extracted from a dictionary or a common sentence.
[0144] In S1022, the matching degree between each candidate word segment and the target sentence is calculated respectively.
[0145] In this embodiment, since the abnormal participle can be associated with multiple candidate participles, in order to determine the legal participle that matches the target sentence, the terminal device can calculate the matching degree between the candidate participle and the target sentence, wherein the matching degree calculation process can be specifically as follows: replace the abnormal participle in the target sentence with the candidate participle, and import the replaced target sentence into the association probability recognition model, determine whether the target sentence still contains abnormal participles, if so, identify the candidate participle as an illegal participle, that is, the matching degree is 0, if not, identify the candidate participle as a legal participle, that is, the matching degree is 100%. Of course, the method provided in the second embodiment can also be used to replace the candidate participle with the target sentence, select a string consisting of multiple characters, and calculate the association degree between each character and other characters in the string, so that the matching degree corresponding to the candidate keyword can be determined.
[0146] In S1023, a legal participle of the abnormal participle is determined based on the matching degree, and the abnormal participle in the target sentence is replaced with the legal participle.
[0147] In this embodiment, the terminal device may select a candidate segmentation with the highest matching degree from multiple candidate segmentations as a legal segmentation corresponding to the abnormal segmentation, and perform a replacement operation.
[0148] In an embodiment of the present invention, candidate participles corresponding to abnormal participles are determined through a corpus, and legal participles are determined based on the correlation between the candidate participles and the target sentence, thereby improving the accuracy of legal participle selection and the accuracy of sentence error correction.
[0149] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.
[0150] Fig. 9 The structure block diagram of a sentence correction device provided by an embodiment of the present invention is shown. The sentence correction device includes units for executing Figure 1 For details, please refer to the steps in the corresponding embodiment. Figure 1 and Figure 1 For the convenience of explanation, only the parts related to this embodiment are shown.
[0151] See also Fig. 9 , the error correction device of the statement includes:
[0152] An abnormal word segmentation recognition unit 91 is used to recognize abnormal word segmentations 9 included in the target sentence through a preset association probability recognition model;
[0153] The corrected sentence output unit 92 is used to replace the abnormal segmented words in the target sentence with legal segmented words to obtain the corrected sentence.
[0154] Optionally, the abnormal word segmentation identification unit 91 includes:
[0155] A character string frame extracting unit, used for selecting a plurality of consecutive characters in the target sentence to obtain a character string;
[0156] A correlation calculation unit, used to calculate the correlation between each of the characters and the remaining characters in the character string;
[0157] An association coefficient calculation unit, used to determine the word segmentation association coefficient of the character string according to the association degree corresponding to each of the characters;
[0158] The abnormal word segmentation identification unit is used to identify that the character string contains abnormal word segmentation if the word segmentation association coefficient is less than a preset association threshold.
[0159] Optionally, the character string framing unit is specifically configured to obtain the character string by framing a preset number of consecutive characters in the target sentence.
[0160] Optionally, the statement error correction device further includes:
[0161] A learning parameter acquisition unit, used to acquire model learning parameters; the model learning parameters include: initial efficiency, minimum efficiency, efficiency adjustment step length and number of learning cycles;
[0162] A learning efficiency initialization unit, used for setting the learning efficiency of the Nth learning cycle according to the initial efficiency; the initial value of N is 1;
[0163] A training and learning execution unit, used to train and learn the association probability recognition model through multiple training samples at the Nth learning cycle learning efficiency until the output value of the loss function of the association probability recognition model is less than a preset convergence threshold;
[0164] A learning efficiency adjustment unit, configured to add 1 to the value of N, reduce the learning efficiency of the N-1th learning cycle based on the efficiency adjustment step, and identify the adjusted learning efficiency of the N-1th learning cycle as the learning efficiency of the Nth learning cycle;
[0165] A learning efficiency resetting unit, configured to set the learning efficiency to an initial efficiency if the learning efficiency is less than the minimum efficiency, and return to perform the operation of training and learning the association probability recognition model through multiple training samples at the Nth learning cycle learning efficiency;
[0166] A cyclic training learning unit, configured to return to executing the operation of training and learning the association probability recognition model through multiple training samples at the Nth learning cycle learning efficiency if the learning efficiency is greater than or equal to the minimum efficiency;
[0167] The training completion determination unit is used to identify that the association probability recognition model has been trained if the value of N is greater than the number of learning cycles.
[0168] Optionally, the statement error correction device further includes:
[0169] The target sentence output unit is used to import the original sentence to be corrected into the initial error correction model to generate a target sentence of the original sentence.
[0170] Optionally, the target sentence output unit includes:
[0171] A coding sequence output unit, used to encode the original sentence through a preset sentence encoding network, and output a coding sequence about the original sentence;
[0172] A sentence weight calculation unit, used for importing the coding sequence into an attention calculation model, and calculating the sentence weight of each element in the coding sequence in the original sentence;
[0173] The sentence weighting unit is used to import the sentence weight and the encoding sequence into a preset sentence decoding network to generate a target sentence corresponding to the original sentence.
[0174] Optionally, the coding sequence output unit includes:
[0175] A short sentence division unit, used for dividing the original sentence into at least one short sentence;
[0176] A standard sentence construction unit, for filling invalid characters into any of the short sentences to generate a standard sentence if the actual number of characters in the short sentences is less than a preset standard number;
[0177] The standard sentence importing unit is used to encode all the standard sentences through a preset sentence encoding network and output an encoding sequence of the original sentences.
[0178] Optionally, the statement weighting unit includes:
[0179] An invalid character filter matrix construction unit, used to generate an invalid character filter matrix of the original sentence based on the invalid characters contained in each of the standard sentences;
[0180] The invalid character filter matrix importing unit is used to import the invalid character filter matrix, the sentence weight and the encoding sequence into a preset sentence decoding network to generate a target sentence corresponding to the original sentence.
[0181] Optionally, the corrected sentence output unit 92 includes:
[0182] A candidate segmentation selection unit, used to select candidate segmentations associated with the abnormal segmentation from a preset corpus;
[0183] A matching degree calculation unit, used to respectively calculate the matching degree between each of the candidate participles and the target sentence;
[0184] A legal word segmentation selection unit is used to determine the legal word segmentation of the abnormal word segmentation based on the matching degree, and replace the abnormal word segmentation in the target sentence with the legal word segmentation.
[0185] Therefore, the sentence correction device provided by the embodiment of the present invention can also import the target sentence that needs to be corrected into the association probability recognition module, identify whether the target sentence contains abnormal participles with low correlation with the entire sentence, extract the legal participles corresponding to the abnormal participles from the corpus, and replace all abnormal participles in the target sentence with corresponding legal participles, generate and output the corrected sentence, and achieve the purpose of automatic correction of the target sentence. Compared with the existing sentence correction technology, the present invention does not need to rely on manual sentence correction, but can perform sentence correction on the target sentence through the association probability recognition model, while ensuring the accuracy of the output sentence, it can also improve the efficiency of sentence correction, and meet the current verification needs of large amounts of electronic data.
[0186] Fig.10 FIG. 1 is a schematic diagram of a terminal device provided by another embodiment of the present invention. Fig.10 As shown, the terminal device 10 of this embodiment includes: a processor 100, a memory 101, and a computer program 102 stored in the memory 101 and executable on the processor 100, such as a statement error correction program. When the processor 100 executes the computer program 102, the steps of the above-mentioned statement error correction method embodiments are implemented, such as Figure 1 Alternatively, when the processor 100 executes the computer program 102, the functions of each unit in the above-mentioned device embodiments are realized, for example, Fig. 9 The functions of modules 91 to 92 are shown.
[0187] Exemplarily, the computer program 102 may be divided into one or more units, which are stored in the memory 101 and executed by the processor 100 to complete the present invention. The one or more units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program 102 in the terminal device 10. For example, the computer program 102 may be divided into an abnormal word segmentation recognition unit and a corrected sentence output unit, and the specific functions of each unit are as described above.
[0188] The terminal device 10 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The terminal device may include, but is not limited to, a processor 100 and a memory 101. Those skilled in the art will appreciate that Fig.10 It is only an example of the terminal device 10 and does not constitute a limitation of the terminal device 10. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the terminal device may also include input and output devices, network access devices, buses, etc.
[0189] The processor 100 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.
[0190] The memory 101 may be an internal storage unit of the terminal device 10, such as a hard disk or memory of the terminal device 10. The memory 101 may also be an external storage device of the terminal device 10, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal device 10. Further, the memory 101 may also include both an internal storage unit of the terminal device 10 and an external storage device. The memory 101 is used to store the computer program and other programs and data required by the terminal device. The memory 101 may also be used to temporarily store data that has been output or is to be output.
[0191] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0192] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.
Claims
1. A sentence error correction method, characterized in that: include: Importing the original sentence to be corrected into the initial error correction model to generate a target sentence corresponding to the original sentence includes: The original sentence is encoded by a preset sentence encoding network, and an encoding sequence of the original sentence is outputted. The coding sequence is introduced into the attention calculation model to calculate the sentence weight of each element in the coding sequence in the original sentence. Importing the sentence weight and the encoding sequence into a preset sentence decoding network to generate a target sentence corresponding to the original sentence; wherein the sentence encoding network is constructed by two bidirectional GRU networks, and the sentence decoding network is obtained by connecting two unidirectional long-short-term LSTM networks in series; Identify abnormal word segmentations contained in the target sentence through a preset association probability recognition model; including: Selecting a plurality of consecutive characters in the target sentence to obtain a character string; Detecting the number of occurrences of each character and the remaining characters in the string in the standard document, importing a preset hash conversion function based on the number of occurrences, and identifying the output hash value as the correlation between each character and the remaining characters in the string; Determining a word segmentation correlation coefficient of the character string according to the correlation degree corresponding to each of the characters; If the word segmentation correlation coefficient is less than a preset correlation threshold, identifying that the character string contains abnormal word segmentation; Replacing the abnormal participles in the target sentence with legal participles to obtain a corrected sentence; The association probability recognition model is configured with a sentence topic detection network and a topic association calculation network. The terminal device imports the target sentence into the sentence topic detection network to determine the sentence topic of the target sentence; performs word segmentation on the target sentence, and identifies abnormal word segmentations based on the association between each word segmentation and the sentence topic, and outputs corresponding legal word segmentations based on the sentence topic and the abnormal word segmentation.
2. The error correction method according to claim 1, characterized in that: The step of selecting a plurality of consecutive characters from the target sentence to obtain a character string includes: A preset number of consecutive characters are framed in the target sentence to obtain the character string.
3. The error correction method according to claim 1, characterized in that: Before identifying abnormal word segments contained in the target sentence by using a preset association probability recognition model, the method further includes: Obtaining model learning parameters; the model learning parameters include: initial efficiency, minimum efficiency, efficiency adjustment step length, and number of learning cycles; Setting the learning efficiency of the Nth learning cycle according to the initial efficiency; the initial value of N is 1; Training the association probability recognition model with the Nth learning cycle learning efficiency through multiple training samples until the output value of the loss function of the association probability recognition model is less than a preset convergence threshold; Add 1 to the value of N, and reduce the learning efficiency of the N-1th learning cycle based on the efficiency adjustment step size, and identify the adjusted learning efficiency of the N-1th learning cycle as the learning efficiency of the Nth learning cycle; If the learning efficiency is less than the minimum efficiency, the learning efficiency is set as the initial efficiency, and the operation of training and learning the association probability recognition model through multiple training samples at the Nth learning cycle learning efficiency is returned to be executed; If the learning efficiency is greater than or equal to the minimum efficiency, returning to perform the operation of training and learning the association probability recognition model through multiple training samples at the Nth learning cycle learning efficiency; If the value of N is greater than the number of learning cycles, it is determined that the association probability recognition model has been trained.
4. The error correction method according to claim 1, characterized in that: The encoding of the original sentence by a preset sentence encoding network and outputting a coding sequence of the original sentence includes: Dividing the original sentence into at least one short sentence; If the actual number of characters in any of the short sentences is less than the preset standard number, then the invalid characters are filled into the short sentence to generate a standard sentence; All the standard sentences are encoded through a preset sentence encoding network, and an encoding sequence of the original sentences is output.
5. The error correction method according to claim 4, characterized in that: The step of importing the sentence weight and the encoding sequence into a preset sentence decoding network to generate a target sentence corresponding to the original sentence includes: Based on the invalid characters contained in each of the standard sentences, generating an invalid character filter matrix of the original sentence; The invalid character filtering matrix, the sentence weights and the encoding sequence are imported into a preset sentence decoding network to generate a target sentence corresponding to the original sentence.
6. The error correction method according to any one of claims 1 to 3, characterized in that: The step of replacing the abnormal participles in the target sentence with legal participles to obtain a corrected sentence includes: Selecting candidate participles associated with the abnormal participles from a preset corpus; Calculating the matching degree between each candidate word segment and the target sentence respectively; A legal participle of the abnormal participle is determined based on the matching degree, and the abnormal participle in the target sentence is replaced with the legal participle.
7. A sentence error correction device, based on the error correction method according to any one of claims 1 to 6, characterized in that: include: An abnormal word segmentation recognition unit is used to recognize abnormal word segmentations contained in a target sentence through a preset association probability recognition model; The corrected sentence output unit is used to replace the abnormal participles in the target sentence with legal participles to obtain a corrected sentence.
8. A terminal device, characterized in that: The terminal device comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the method according to any one of claims 1 to 6 when executing the computer program.
9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.
Citation Information
Patent Citations
An LSTM-based Chinese text grammar error correction model method
CN109948152A
Chinese error correction method, device and equipment
CN110276077A