Text error correction language model training method, text error correction method and related products

Abstract

The invention relates to a text error correction language model training method, a text error correction method and a related product, and relates to the technical field of computers, the method comprises the following steps: obtaining use scene data and initial training data obtained based on the use scene data; determining context information of the information before reform in the use scene data, performing information retrieval on the use scene data based on the context information and the information before reform to obtain a retrieval result, and generating target training data according to the retrieval result and the information after reform; and training the text error correction language model by using the target training data. The error distribution of the target training data is closer to the error distribution of the actual use scene, the quality of the target training data is improved, the effect of a text error correction language model can be improved, in addition, data enhancement is realized based on information retrieval, and the efficiency of generating the target training data can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a text error correction language model training method, a text error correction method and related products. Background Art

[0002] There are two ways to improve the effect of the text error correction language model, one is the change of the model structure, and the other is data enhancement. Among them, data plays a very important role in the training of the text error correction model, so the data enhancement scheme is very important. The text error correction language model requires a large amount of high-quality training data. Among them, high quality includes two aspects: 1. High text quality, the errors in the text must conform to the errors in the actual usage scenario; 2. The distribution of error types must conform to the actual distribution of text errors in the actual usage scenario. At present, the existing data enhancement scheme 1 is a manually annotated data enhancement scheme, which requires a lot of manpower and professional knowledge, is very costly, and is not conducive to generating a large amount of training data. The existing data enhancement scheme 2 is a data enhancement scheme that performs random replacement based on the actual distribution of global text errors. On the one hand, the generated text quality is low, and only features such as similar sound and similar shape are considered; on the other hand, the error distribution of the generated text may be inconsistent with the distribution of the actual usage scenario, resulting in bias in model training. Summary of the invention

[0003] The text error correction language model training method, text error correction method and related products provided by the embodiments of the present invention can at least improve the quality of target training data, enhance the effect of the text error correction language model, and improve the efficiency of generating target training data.

[0004] According to one aspect of the present application, a method for training a text correction language model is also provided, comprising: obtaining usage scenario data and initial training data obtained based on the usage scenario data; wherein the initial training data includes pre-correction information and post-correction information corresponding to the pre-correction information; determining context information of the pre-correction information in the usage scenario data, performing information retrieval on the usage scenario data based on the context information and the pre-correction information to obtain a retrieval result, and generating target training data based on the retrieval result and the post-correction information; wherein the context information is associated with the pre-correction information, and the target training data includes target pre-correction information and the post-correction information; using the target training data to train the text correction language model; wherein the text correction language model is used to modify the target pre-correction information to the post-correction information.

[0005] According to another aspect of the present application, a text correction language model training device is also provided, including: a data module, used to obtain usage scenario data and initial training data obtained based on the usage scenario data; wherein the initial training data includes pre-correction information and post-correction information corresponding to the pre-correction information; an enhancement module, used to determine the context information of the pre-correction information in the usage scenario data, based on the context information and the pre-correction information, perform information retrieval on the usage scenario data to obtain a retrieval result, and generate target training data according to the retrieval result and the post-correction information; wherein the context information is associated with the pre-correction information, and the target training data includes target pre-correction information and the post-correction information; a training module, used to train the text correction language model using the target training data; wherein the text correction language model is used to modify the target pre-correction information to the post-correction information.

[0006] According to another aspect of the present application, a text correction method is also provided, comprising: obtaining a text to be processed; inputting the text to be processed into a text correction language model to perform text correction processing on the text to be processed; wherein the text correction language model is obtained after training using the above method.

[0007] According to another aspect of the present application, a text correction device is also provided, including: an acquisition module for acquiring a text to be processed; a processing module for inputting the text to be processed into a text correction language model to perform text correction processing on the text to be processed; wherein the text correction language model is obtained after training using the above-mentioned method.

[0008] According to another aspect of the present application, a text review system is also provided, comprising the above-mentioned text error correction device.

[0009] According to another aspect of the present application, an electronic device is provided, comprising: a processor; and a memory storing a program, wherein the program comprises instructions, and when the instructions are executed by the processor, the processor executes the above method.

[0010] According to another aspect of the present application, a non-transitory computer-readable storage medium storing computer instructions is also provided, wherein the computer instructions are used to enable the computer to execute the steps of the above method.

[0011] According to another aspect of the present application, a computer program product is also provided. The computer program product includes a computer program. When the computer program is executed by a processor, the above method steps are implemented.

[0012] Beneficial effects of the embodiments of the present invention:

[0013] In an embodiment of the present invention, use scenario data and initial training data obtained based on the use scenario data are obtained; wherein the initial training data includes information before change and information after change corresponding to the information before change; context information of the information before change is determined in the use scenario data, and information retrieval is performed on the use scenario data based on the context information and the information before change to obtain a retrieval result, and target training data is generated according to the retrieval result and the information after change; wherein the context information is associated with the information before change, and the target training data includes target information before change and target information after change; the text error correction language model is trained using the target training data; wherein the text error correction language model is used to modify the target information before change to the information after change. In an embodiment of the present invention, the context information of the information before change is determined based on the existing initial training data, and information retrieval is performed on the use scenario data in combination with the information before change and its context information, and then the target training data is constructed based on the retrieval result, and the error distribution of the target training data is closer to the error distribution of the actual use scenario, and the quality of the target training data is improved, so that the effect of the text error correction language model can be improved, and in addition, data enhancement is achieved based on information retrieval, and the efficiency of generating target training data can also be improved.

[0014] The details of one or more embodiments of the invention are set forth in the following drawings and description so that other features, objects, and advantages of the invention are more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other embodiments can be obtained based on these drawings without creative work.

[0016] Figure 1 is a flow chart of a text error correction language model training method according to an embodiment of the present invention;

[0017] Figure 2 is a flowchart of an implementation method of a text error correction language model training method according to an embodiment of the present invention;

[0018] Figure 3 Schematic diagram of the structure of the electronic device of this embodiment. DETAILED DESCRIPTION

[0019] Embodiments of the present embodiment will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present embodiment are shown in the accompanying drawings, it should be understood that the present embodiment can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein, which are instead provided for a more thorough and complete understanding of the present embodiment. It should be understood that the drawings and embodiments of the present embodiment are only for exemplary purposes and are not intended to limit the scope of protection of the present embodiment.

[0020] The terms involved are explained below.

[0021] Text Correction: Point out errors in a text and give suggestions for revisions to make it error-free.

[0022] Data augmentation: is a technique that increases the amount and diversity of training data by performing a series of transformations and expansions on the original training data. The purpose of data augmentation is to improve the generalization ability, robustness, and performance of the model.

[0023] Embedding: A technique for mapping discrete symbols or objects into a continuous vector space.

[0024] The impact of training data on model training is very important, for example, data quality: The quality of training data directly affects the performance of the model. If there are errors, noise, or missing values ​​in the training data, the model may learn the wrong patterns or make inaccurate predictions. Therefore, it is very important to ensure the quality of the training data. Data quantity: The amount of training data also has an impact on the performance of the model. Generally speaking, more training data can provide more samples to learn the model, thereby improving the generalization ability of the model. However, if the amount of data is too small, the model may overfit the training data, resulting in poor performance on new data. Data distribution: The distribution of training data also affects the performance of the model. If the distribution of training data is inconsistent with the distribution of actual application scenarios, the model may perform poorly on new data. Therefore, it is important to ensure that the distribution of training data is similar to the distribution of actual application scenarios. Data label: For supervised learning tasks, the label of training data is crucial to the training of the model. Accurate labels can help the model learn the correct patterns and relationships. However, if the labels are wrong or inaccurate, the model may learn the wrong patterns.

[0025] Random replacement based on the actual distribution of global text errors is a data augmentation method used to improve the performance of the model when processing text data. In this method, first, the error distribution in the real text data is observed and analyzed, and then the text in the training data is randomly replaced based on this error distribution information.

[0026] Specifically, by collecting and analyzing error samples in real text data, such as spelling errors, grammatical errors, punctuation errors, etc., we can understand the distribution of these errors in the entire text data set. Then, based on this error distribution information, we can perform random replacement operations on the text in the training data to simulate the errors in the real text. For example, suppose the training data is a set of email texts, and it is observed that in real email texts, spelling errors are a more common type of error. The distribution of spelling errors in real email texts can be counted, such as the frequency of a word being misspelled. Then, randomly select some words in the training data, and randomly replace the spelling of these words based on the spelling error distribution information to generate new training samples.

[0027] Based on this data augmentation method, the generated training data not only has low text quality, but also the error types are generated based on the actual distribution of global text errors, which may not meet the actual demand scenarios.

[0028] Based on the above background, the present invention provides a text correction language model training method, a text correction method and related products. The method performs data enhancement processing based on information retrieval, and can generate a large amount of high-quality training data. Using this part of data for model training can improve the model effect.

[0029] In an embodiment of the present invention, a text error correction language model training method is provided. The device for implementing the method can be selected according to the type and source of the usage scenario data. For example, the method can be implemented on devices such as personal computers, servers, cloud platforms, mobile devices, and Internet of Things devices. Figure 1 is a flowchart of a text error correction language model training method according to an embodiment of the present application. Figure 1 The method steps involved are described.

[0030] Step S101, obtaining usage scenario data and initial training data obtained based on the usage scenario data; wherein the initial training data includes pre-modification information and post-modification information corresponding to the pre-modification information.

[0031] In this step, the usage scenario data is text data about different fields or languages. The fields may include but are not limited to: social media error correction, academic writing assistance, machine translation post-editing, automatic error correction, and grammatical error correction. The languages ​​may include but are not limited to: Chinese and English.

[0032] Initial training data can be obtained by annotating data processing based on the usage scenario data. For example, existing error correction tools or manual annotation can be used. The initial training data can be structured data (such as tabular data) or unstructured data (such as text, images, audio, etc.). In an embodiment of the present invention, in the initial training data, each training sample includes pre-change information and post-change information corresponding to the pre-change information, wherein the pre-change information can be erroneous information before correction, and the post-change information is correct information after correction corresponding to the pre-change information.

[0033] When generating initial training data, the word segmentation algorithm can be used to process the usage scenario data, determine the pre-change information based on the processing result, and then determine the corresponding post-change information based on the pre-change information. The pre-change information and post-change information can be any one of characters, words, and punctuation marks, or a combination thereof.

[0034] It should be noted that the initial training data is a data set consisting of a plurality of initial training samples. Each initial training sample is a data point in the training set, including pre-modification information and post-modification information corresponding to the pre-modification information.

[0035] In this step, the applicable field and applicable language of the text error correction language model can be determined by selecting the usage scenario data. The initial training data is high-quality training data obtained based on the usage scenario data. The subsequent steps perform further data enhancement processing based on the initial training data to obtain higher-quality training data.

[0036] Step S102, determining the context information of the information before change in the usage scenario data, performing information retrieval on the usage scenario data based on the context information and the information before change to obtain a retrieval result, and generating target training data according to the retrieval result and the information after change; wherein the context information is correlated with the information before change, and the target training data includes the target information before change and the information after change.

[0037] In this step, any information other than the information before change in the usage scenario data can be used as the context information of the information before change. Preferably, the information in the usage scenario data that has some correlation with the information before change can be used as the context information of the information before change. For example, this correlation can be measured by dimensions such as similarity, association rule mining, network analysis, semantic association, and time association, that is, the context information is correlated with the information before change.

[0038] A search is performed in the usage scenario data to find content including the context information and the pre-modification information, that is, based on the context information and the pre-modification information, information retrieval is performed on the usage scenario data to obtain a retrieval result.

[0039] The retrieval results or the adjustment results obtained based on the retrieval results and the modified information are combined into training samples, that is, the target training data is generated according to the retrieval results and the modified information. The adjustment results include the adjusted context information and / or the adjusted pre-modification information. The pre-modification information or the adjusted pre-modification information is used as the target pre-modification information, so that the trained text error correction language model can adjust the target pre-modification information to the corresponding post-modification information, that is, the target training data includes the target pre-modification information and the post-modification information.

[0040] It should be noted that the target training data is a data set consisting of multiple target training samples. Each target training sample is a data point in the training set, which contains context information or modified context information, as well as target pre-change information and post-change information corresponding to the target pre-change information. Compared with the solution that only uses the glyph and pronunciation information of the error, in this step, more consideration is given to the context of the error and the combined information of the error, so the generated training data text can be of higher quality.

[0041] In this step, information retrieval can more efficiently find retrieval results including pre-modification information and its context information in the usage scenario data, thereby improving the efficiency of generating target training data. Then, target training data is generated based on the retrieval results, so that the error distribution of the target training data is closer to the error distribution of the actual usage scenario.

[0042] Step S103, using the target training data to train the text error correction language model; wherein the text error correction language model is used to modify the target pre-correction information into the post-correction information.

[0043] In this step, when the number of target training samples in the target training data is large enough, for example, after exceeding a preset sample threshold, the target training data is used to train the text correction language model, so that the text correction language model can be used to combine contextual information to modify the target pre-correction information into the post-correction information, thereby improving the effect of the model.

[0044] For example, if the text error correction language model is trained with the initial training data, it can be used to change the word "burst" in the text to "burst". When the text is corrected based on the trained model, "upstart" may be changed to "burst", resulting in a correction error. However, when the text is corrected based on the trained model after training with the target training data, "burst" will be changed to "burst" only when there is context information "volcano", thus avoiding the phenomenon of correction errors.

[0045] In an embodiment of the present invention, use scenario data and initial training data obtained based on the use scenario data are obtained; wherein the initial training data includes information before change and information after change corresponding to the information before change; context information of the information before change is determined in the use scenario data, and information retrieval is performed on the use scenario data based on the context information and the information before change to obtain a retrieval result, and target training data is generated according to the retrieval result and the information after change; wherein the context information is associated with the information before change, and the target training data includes target information before change and target information after change; the text error correction language model is trained using the target training data; wherein the text error correction language model is used to modify the target information before change to the information after change. In an embodiment of the present invention, the context information of the information before change is determined based on the existing initial training data, and information retrieval is performed on the use scenario data in combination with the information before change and its context information, and then the target training data is constructed based on the retrieval result, and the error distribution of the target training data is closer to the error distribution of the actual use scenario, and the quality of the target training data is improved, so that the effect of the text error correction language model can be improved, and in addition, data enhancement is achieved based on information retrieval, and the efficiency of generating target training data can also be improved.

[0046] Considering that the text information in the adjacent position of the information before being changed may have a significant impact on the correct use of the information before being changed, therefore, in a possible implementation, determining the context information of the information before being changed in the usage scenario data can be performed according to the following steps: determining the first position of the information before being changed in the usage scenario data; and using the first text information corresponding to the first target position adjacent to the first position as the context information of the information before being changed.

[0047] In this possible implementation, the usage scenario data may include multiple pre-modification information, and the location of each pre-modification information is called the first location. The location adjacent to the first location is used as the first target location, and the first target location may be before the first location and / or after the first location. The first text information corresponding to the first target location may be a word, a phrase, or a combination of words. The first text information is used as the context information of the pre-modification information.

[0048] In specific implementation, the information before change can be expanded forward and backward by several words, and the expanded words can be used as the context information of the information before change. For example, the information before change "eruption" can be expanded forward to obtain "volcanic eruption", where "volcano" is the context information of "eruption".

[0049] In this possible implementation, context information is obtained by selecting information that is adjacent to the information before modification in position, so as to obtain target training data based on the context information and optimize the effect of the model.

[0050] In a possible implementation, determining the context information of the information before being changed in the usage scenario data may be performed according to the following steps: determining a second position of the information before being changed in the usage scenario data; determining a second target position in the usage scenario data based on the second position; wherein the distance between the second target position and the second position is less than a distance threshold; calculating an association value between the second text information corresponding to the second target position and the information before being changed; determining the context information of the information before being changed in the second text information according to the association value; wherein the association value corresponding to the context information is greater than an association value corresponding to any second text information in the second text information except the context information.

[0051] In this possible implementation, the usage scenario data may include multiple pre-modification information, and the location of each pre-modification information is called the second location, and the location whose distance from the second location meets the preset distance range is used as the second target location. A plurality of second target locations can be selected by a preset distance range, for example, the distance range is: the interval between the second target location and the second location is more than M characters and less than N characters, and the second text information corresponding to each second target location is calculated, and the correlation value between the information before the modification is obtained, thereby obtaining a plurality of correlation values. The second text information corresponding to the larger correlation value is used as the context information of the information before the modification, that is, the context information of the information before the modification is determined in the second text information according to the correlation value, wherein the correlation value corresponding to the context information is greater than the correlation value corresponding to any second text information in the second text information except the context information.

[0052] It should be noted that in this step, the relevance value can be used to evaluate the degree of relevance, for example, to characterize the frequency of co-occurrence, the degree of similarity, the degree of mutual importance, etc. Preferably, the relevance value can be calculated using the TF-IDF (Term Frequency-Inverse Document Frequency) algorithm.

[0053] In a specific implementation, for example, the usage scenario data includes information such as "volcanic eruption", "volcanoes often erupt", and "volcanoes occasionally erupt", among which the second text information includes "volcano", "of", "often", and "occasionally", and after calculation, the correlation value of "volcano" is higher, so "volcano" is used as the context information of the information "eruption" before the change.

[0054] In this possible implementation, context information is obtained by selecting information that is correlated with the information before modification in dimensions such as usage habits and usage frequency, so as to obtain target training data based on the context information and optimize the effect of the model.

[0055] In a possible implementation, information retrieval is performed on the usage scenario data based on the context information and the information before modification to obtain a retrieval result, which can be performed according to the following steps: target context information is determined in the usage scenario data based on the context information; wherein the similarity between the target context information and the context information satisfies a preset similarity threshold range; based on the target context information and the information before modification, information retrieval is performed on the usage scenario data to obtain multiple groups of retrieval contents; wherein the positions of each group of retrieval contents in the usage scenario data are different; and the retrieval contents including the target context information and the information before modification are used as retrieval results.

[0056] In this possible implementation, context information, information similar to the context information, or information different from the context information is used as target context information, that is, the target context information is determined in the usage scenario data based on the context information; wherein, the similarity between the target context information and the context information satisfies a preset similarity threshold range. Through the similarity threshold range, the degree of similarity between the context information and the target context information can be controlled. The similarity threshold range can be set according to actual needs and experience, and the embodiment of the present invention does not specifically limit this.

[0057] The target context information and the information before the change are used as keywords to search the usage scenario data, that is, based on the target context information and the information before the change, the usage scenario data is searched to obtain multiple groups of search contents; wherein each group of search contents has a different position in the usage scenario data. Considering that the information search contents may include the target context information and / or the information before the change, or the search contents may include information similar to the target context information and / or the information before the change, in order to obtain the target training data including the target context information, the search contents including the target context information and the information before the change are used as the search results.

[0058] In a specific implementation, for example, for the context information "volcano" of the pre-modified information "eruption", the target context information similar to it may be "spring volcano", "summer volcano" and "winter volcano", the target context information identical to it may be "volcano", and the target context information different from it may be "conflict". The retrieval results obtained based on the target context information and the pre-modified information may include "spring volcano eruption", "volcanic eruption" and "conflict eruption", etc., and then, the target training data is constructed based on the retrieval results, and then the model is trained based on the target training data, so that the obtained model can change "eruption" to the corresponding post-modified information when the context is "spring volcano", "volcano" or "conflict".

[0059] In this possible implementation, the target context information is obtained by expanding the context information, so as to achieve data enhancement of the target training data based on the target context information, so that the error distribution of the target training data is closer to the error distribution of the actual usage scenario, and the efficiency of generating the target training data is improved.

[0060] In order to further expand the training data, in a possible implementation, target training data is generated based on the retrieval results and the changed information, which can be performed according to the following steps: determining a plurality of other pre-change information corresponding to the changed information in the initial training data, and determining a first pre-change information among the plurality of other pre-change information; modifying the pre-change information in the retrieval results to the first pre-change information to obtain target pre-change information; and generating target training data using the modified retrieval results and the changed information.

[0061] In this possible implementation, it is taken into account that in the initial training data, there may be a situation where the post-change information corresponding to multiple pre-change information is the same, for example, error A is changed to B, error C is changed to B, and error D is changed to B. The pre-change information corresponding to the post-change information B includes pre-change information C and pre-change information D in addition to pre-change information A. Among C and D, one can be selected, for example, C is selected as the first pre-change information, that is, multiple other pre-change information corresponding to the post-change information is determined in the initial training data, and the first pre-change information is determined among the multiple other pre-change information.

[0062] The retrieval result includes the target context information and the information before the change, the information before the change is modified to the first information before the change, and the first information before the change in the modified retrieval result is used as the target information before the change, that is, the information before the change in the retrieval result is modified to the first information before the change, and the target information before the change is obtained. For example, the retrieval result includes the target context information and A, and the modified retrieval result includes the target context information and C, where C is the target information before the change,

[0063] The target training data is generated using the modified retrieval results and the modified information. Thus, the model obtained based on the target training data can be used to modify the target pre-modification information in the text information into the modified information, so that the error distribution of the target training data is closer to the error distribution of the actual usage scenario.

[0064] In the specific implementation, for example, the initial training data includes samples in which "outbreak" is changed to "eruption", and there are also samples in which "explosion" is changed to "eruption". The changed information "outbreak" corresponds to "eruption" and "explosion". "Explosion" is used as the first information before change, and the retrieval results include "volcanic eruption". The modified retrieval results include "volcanic explosion". The target training data obtained using the retrieval results can be used to train the model to change the "explosion" in "volcanic explosion" to "eruption".

[0065] In a possible implementation, generating target training data based on the retrieval results and the modified information can be performed according to the following steps: using the pre-modification information in the retrieval results as target pre-modification information; and generating target training data using the retrieval results and the modified information.

[0066] In this possible implementation, the information before change in the retrieval results can be directly used as the target information before change, and the target training data can be generated using the retrieval results and the information after change. The target training data thus obtained can be used to train the model to modify the information before change into the information after change in combination with context information or target context information, so that the error distribution of the target training data is closer to the error distribution of the actual usage scenario.

[0067] Taking into account the limited types of pre-correction information in the initial training data, in order to make the text error correction language model applicable to more types of error correction scenarios, in a possible implementation, generating target training data based on the retrieval results and the post-correction information can be performed according to the following steps: obtaining a plurality of similar pre-correction information corresponding to the pre-correction information, and determining second pre-correction information among the plurality of similar pre-correction information; modifying the pre-correction information in the retrieval results into the second pre-correction information to obtain target pre-correction information; and using the modified retrieval results and the post-correction information to generate target training data.

[0068] In this possible implementation, similar pre-modification information corresponding to the pre-modification information may be obtained or constructed in advance through multiple channels and stored. In executing this step, multiple similar pre-modification information corresponding to the pre-modification information is obtained from the stored data. Multiple similar pre-modification information corresponding to the pre-modification information may also be obtained in real time through the network. The embodiment of the present invention does not specifically limit the channels for obtaining the multiple similar pre-modification information.

[0069] The information before the change that is more similar to the information before the change is taken as the second information before the change, that is, the second information before the change is determined from the multiple similar information before the change. It should be noted that various similar information before the change and the information before the change can be embedded to obtain vectors, and then similarity information can be obtained by calculating the vector distance, and the second information before the change is determined from the multiple similar information before the change based on the similarity information.

[0070] After obtaining the second information before change, the information before change in the search result is modified to the second information before change to obtain the target information before change; the target training data is generated using the modified search result and the information after change. The specific implementation method can refer to the above-mentioned steps of modification based on the first information before change, which will not be repeated here.

[0071] In this step, similar expansion is performed based on the information before modification to obtain the target information before modification. Then, the target training data is generated using the modified retrieval results and the information after modification. Thus, the model obtained based on the target training data can be used to modify the target information before modification in the text information into the information after modification, so that the error distribution of the target training data is closer to the error distribution of the actual usage scenario.

[0072] In specific implementation, for example, the initial training data includes the pre-change information "outbreak", and the similar pre-change information obtained includes "eruption", "burst", etc. It should be noted that "eruption" and "burst" and other similar pre-change information are not included in the initial training data, and are information similar to the pre-change information obtained by construction or acquisition. After calculation, for example, if the similarity between "eruption" and "burst" is higher, "eruption" will be used as the second pre-change information. The retrieval results include "volcanic eruption", and the modified retrieval results include "volcanic eruption". The target training data obtained using the retrieval results can be used to train the model to change the "eruption" in "volcanic eruption" to "burst".

[0073] The following is a specific example to illustrate the text error correction language model training method. Figure 2 The text error correction language model training method implementation flow chart shown in the figure can be implemented in the following steps:

[0074] 1. Based on the existing high-quality training data, determine the errors in the text, recorded as A->B, where A represents the information before the change and B represents the corresponding information after the change.

[0075] 2. Based on A->B, new high-quality training data is generated through information retrieval under the collected full text of the application scenario, and added to the existing high-quality training data.

[0076] 3. For the information retrieval implementation in 2, the following method is adopted: the method of same error in different contexts, that is, the text in the full text of the application scenario that matches A->B but is not exactly the same as the text where A->B is located is taken out, and based on this, new training data whose error is A->B is constructed.

[0077] 4. There are two methods for determining the match between A->B in 3: 1. Continuous fragment method, that is, expand the error A by several words to form a fragment, and the text containing this fragment is matched. 2. Word co-occurrence method, that is, score the word word_a where the error A is located and several words before and after it, and select the word word_s with a higher score (which can be implemented using the TFIDF algorithm). The text containing both word_a and word_s is matched.

[0078] 5. Repeat the above generation scheme until the existing high-quality training data reaches the expected quantity requirement.

[0079] 6. Use the generated data for model training.

[0080] For the above information retrieval steps, the following methods can also be used: 1) Same context different error method, that is, take out the text with the same context as the text where A->B is located in the full text of the application scenario, and combine it with the error C-

[0081] >B, based on this, construct new training data with error C->B. 2) Similar context different error method, that is, take out the text with similar context to the text where A->B is located in the full text of the application scenario, combine it with the error C->B in the existing training data, and based on this, construct new training data with error C->B. (It can be implemented by Embedding algorithm) 3) Different context similar error method, that is, take out the text in the full text of the application scenario that matches A->B but is not exactly the same as the text where A->B is located, and based on this, construct new training data of error AA->B similar to error A->B.

[0082] The above steps can be based on the existing high-quality training data as seeds, combined with the text in the application scenario, through information retrieval, using the continuous fragment method or the word co-occurrence method to construct a large amount of high-quality training data. The obtained training data conforms to the error type distribution in the application scenario, thereby improving the model effect.

[0083] The present invention provides a text error correction language model training method, a text error correction method and related products. The method is an automatic data enhancement scheme based on information retrieval. The data enhancement method based on information retrieval is applied to text error correction, and a large amount of high-quality training data can be generated, and the high quality can be guaranteed without manual intervention. The training data generated by the method, on the one hand, has high text quality, because it uses the existing high-quality training data as seeds, and reasonably retrieves similar texts in the full text of the application scenario according to the errors and context therein, and constructs training data containing the existing errors; on the other hand, the method is also aimed at the errors in the existing high-quality training data, and is extended in the application scenario, and the generated error type distribution conforms to the scenario.

[0084] Based on the above-mentioned text error correction language model training method provided by an embodiment of the present invention, an embodiment of the present invention further provides a text error correction language model training device, which includes: a data module, used to obtain usage scenario data and initial training data obtained based on the usage scenario data; wherein the initial training data includes pre-change information and post-change information corresponding to the pre-change information; an enhancement module, used to determine the context information of the pre-change information in the usage scenario data, based on the context information and the pre-change information, perform information retrieval on the usage scenario data to obtain a retrieval result, and generate target training data according to the retrieval result and the post-change information; wherein the context information is associated with the pre-change information, and the target training data includes target pre-change information and the post-change information; a training module, used to train the text error correction language model using the target training data; wherein the text error correction language model is used to modify the target pre-change information to the post-change information.

[0085] In an embodiment of the present invention, use scenario data and initial training data obtained based on the use scenario data are obtained; wherein the initial training data includes information before change and information after change corresponding to the information before change; context information of the information before change is determined in the use scenario data, and information retrieval is performed on the use scenario data based on the context information and the information before change to obtain a retrieval result, and target training data is generated according to the retrieval result and the information after change; wherein the context information is associated with the information before change, and the target training data includes target information before change and target information after change; the text error correction language model is trained using the target training data; wherein the text error correction language model is used to modify the target information before change to the information after change. In an embodiment of the present invention, the context information of the information before change is determined based on the existing initial training data, and information retrieval is performed on the use scenario data in combination with the information before change and its context information, and then the target training data is constructed based on the retrieval result, and the error distribution of the target training data is closer to the error distribution of the actual use scenario, and the quality of the target training data is improved, so that the effect of the text error correction language model can be improved, and in addition, data enhancement is achieved based on information retrieval, and the efficiency of generating target training data can also be improved.

[0086] In a possible implementation, the enhancement module is specifically used to: determine a first position of the information before modification in the usage scenario data; and use first text information corresponding to a first target position adjacent to the first position as context information of the information before modification.

[0087] In one possible implementation, the enhancement module is specifically used to: determine a second position of the information before modification in the usage scenario data; determine a second target position in the usage scenario data based on the second position; wherein the distance between the second target position and the second position satisfies a preset distance range; calculate a correlation value between the second text information corresponding to the second target position and the information before modification; determine context information of the information before modification in the second text information according to the correlation value; wherein the correlation value corresponding to the context information is greater than the correlation value corresponding to any second text information in the second text information except the context information.

[0088] In one possible implementation, the enhancement module is specifically used to: determine target context information in the usage scenario data based on the context information; wherein the similarity between the target context information and the context information satisfies a preset similarity threshold range; based on the target context information and the pre-modification information, perform information retrieval on the usage scenario data to obtain multiple groups of retrieval content; wherein the positions of each group of retrieval content in the usage scenario data are different; and use the retrieval content including the target context information and the pre-modification information as the retrieval result.

[0089] In one possible implementation, the enhancement module is specifically used to: determine a plurality of other pre-change information corresponding to the post-change information in the initial training data, and determine a first pre-change information among the plurality of other pre-change information; modify the pre-change information in the retrieval result to the first pre-change information to obtain target pre-change information; and generate target training data using the modified retrieval result and the post-change information.

[0090] In a possible implementation, the enhancement module is specifically used to: use the pre-modification information in the retrieval result as the target pre-modification information; and generate target training data using the retrieval result and the post-modification information.

[0091] In one possible implementation, the enhancement module is specifically used to: obtain multiple similar pre-change information corresponding to the pre-change information, and determine second pre-change information among the multiple similar pre-change information; modify the pre-change information in the retrieval result to the second pre-change information to obtain target pre-change information; and generate target training data using the modified retrieval result and the post-change information.

[0092] Based on the above-mentioned text error correction language model training method provided by the embodiment of the present invention, the embodiment of the present invention also provides a text error correction method, including: obtaining a text to be processed; inputting the text to be processed into a text error correction language model to perform text error correction processing on the text to be processed; wherein the text error correction language model is obtained after training by the above-mentioned method.

[0093] Based on the above-mentioned text error correction language model training method provided by an embodiment of the present invention, an embodiment of the present invention also provides a text error correction device, including: an acquisition module, used to acquire a text to be processed; a processing module, used to input the text to be processed into a text error correction language model to perform text error correction processing on the text to be processed; wherein the text error correction language model is obtained after training by the above-mentioned method.

[0094] Based on the above-mentioned text error correction language model training method provided by the embodiment of the present invention, the embodiment of the present invention also provides a text review system, which includes the above-mentioned text error correction device.

[0095] In an embodiment of the present invention, the text review system can be used as a media-integrated APP (Application) editing and review platform system.

[0096] An embodiment of the present invention further provides an electronic device, comprising: at least one processor; and a memory in communication with the at least one processor. The memory stores a computer program executable by the at least one processor, and when the computer program is executed by the at least one processor, the electronic device executes the method of the embodiment of the present invention.

[0097] An embodiment of the present invention further provides a non-transitory machine-readable medium storing a computer program, wherein the computer program, when executed by a processor of a computer, is used to cause the computer to execute the method of the embodiment of the present invention.

[0098] An embodiment of the present invention further provides a computer program product, including a computer program, wherein the computer program, when executed by a processor of a computer, is used to enable the computer to execute the method of the embodiment of the present invention.

[0099] refer to Figure 3 , a block diagram of an electronic device that can be used as a server or client of an embodiment of the present invention will now be described, which is an example of a hardware device that can be applied to various aspects of the present invention. Electronic devices are intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0100] like Figure 3 As shown, the electronic device includes a computing unit 301, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 302 or a computer program loaded from a storage unit 308 into a random access memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the electronic device can also be stored. The computing unit 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0101] Multiple components in the electronic device are connected to the I / O interface 305, including: an input unit 306, an output unit 307, a storage unit 308, and a communication unit 309. The input unit 306 can be any type of device that can input information to the electronic device, and the input unit 306 can receive input digital or character information, and generate key signal input related to user settings and / or function control of the electronic device. The output unit 307 can be any type of device that can present information, and can include but is not limited to a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The storage unit 308 can include but is not limited to a disk, an optical disk. The communication unit 309 allows the electronic device to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks, and can include but is not limited to a modem, a network card, an infrared communication device, a wireless communication transceiver, and / or a chipset, such as a Bluetooth device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.

[0102] The computing unit 301 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 301 include, but are not limited to, a CPU, a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 301 performs the various methods and processes described above. For example, in some embodiments, the method embodiments of the present invention may be implemented as a computer program, which is tangibly contained in a machine-readable medium, such as a storage unit 308. In some embodiments, part or all of the computer program may be loaded and / or installed on an electronic device via a ROM 302 and / or a communication unit 309. In some embodiments, the computing unit 301 may be configured to perform the above-described method in any other appropriate manner (e.g., by means of firmware).

[0103] The computer programs for implementing the methods of the embodiments of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer programs are executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer programs may be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0104] In the context of an embodiment of the present invention, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable signal medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0105] It should be noted that the term "including" and its variations used in the embodiments of the present invention are open inclusions, that is, "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one other embodiment"; the term "some embodiments" means "at least some embodiments". The modifications of "one" and "multiple" mentioned in the embodiments of the present invention are illustrative rather than restrictive. Those skilled in the art should understand that unless the context clearly indicates otherwise, it should be understood as "one or more".

[0106] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of the present invention are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0107] The various steps described in the method implementation methods provided in the embodiments of the present invention may be performed in different orders and / or in parallel. In addition, the method implementation methods may include additional steps and / or omit the steps shown. The scope of protection of the present invention is not limited in this respect.

[0108] The term "embodiment" in this specification refers to specific features, structures or characteristics described in conjunction with the embodiment that can be included in at least one embodiment of the present invention. The appearance of this phrase in various places in the specification does not necessarily mean the same embodiment, nor does it mean that it is mutually exclusive with other embodiments and is independent or optional. The various embodiments in this specification are described in a related manner, and the same or similar parts between the various embodiments refer to each other. In particular, for the device, equipment, and system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts refer to the partial description of the method embodiment.

[0109] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of patent protection. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the attached claims.

Claims

1. A text error correction language model training method, characterized in that: include: Acquire usage scenario data and initial training data obtained based on the usage scenario data; wherein the initial training data includes pre-modification information and post-modification information corresponding to the pre-modification information; Determine the context information of the information before the change in the usage scenario data, perform information retrieval on the usage scenario data based on the context information and the information before the change to obtain a retrieval result, and generate target training data according to the retrieval result and the information after the change; wherein the context information is associated with the information before the change, and the target training data includes the target information before the change and the information after the change; The target training data is used to train the text error correction language model; wherein the text error correction language model is used to modify the target pre-correction information into the post-correction information.

2. The method according to claim 1, characterized in that Determining the context information of the pre-modification information in the usage scenario data includes: Determining a first position of the pre-modification information in the usage scenario data; The first text information corresponding to the first target position adjacent to the first position is used as the context information of the pre-modified information.

3. The method according to claim 1, characterized in that Determining the context information of the pre-modification information in the usage scenario data includes: Determining a second position of the pre-modification information in the usage scenario data; Determine a second target position in the usage scenario data based on the second position; wherein the distance between the second target position and the second position satisfies a preset distance range; Calculating a correlation value between the second text information corresponding to the second target position and the information before the modification; The context information of the information before the modification is determined in the second text information according to the association value; wherein the association value corresponding to the context information is greater than the association value corresponding to any second text information in the second text information except the context information.

4. The method according to claim 1, characterized in that: Based on the context information and the pre-modification information, information retrieval is performed on the usage scenario data to obtain a retrieval result, including: Determining target context information in the usage scenario data based on the context information; wherein the similarity between the target context information and the context information satisfies a preset similarity threshold range; Based on the target context information and the pre-modification information, information retrieval is performed on the usage scenario data to obtain multiple groups of retrieval contents; wherein each group of retrieval contents has a different position in the usage scenario data; The search content including the target context information and the information before modification is used as the search result.

5. The method according to claim 4, characterized in that Generating target training data according to the search result and the modified information includes: Determine a plurality of other pre-modification information corresponding to the post-modification information in the initial training data, and determine the first pre-modification information among the plurality of other pre-modification information; Modify the information before modification in the search result into the first information before modification to obtain target information before modification; The target training data is generated using the modified retrieval results and the modified information.

6. The method according to claim 4, characterized in that Generating target training data according to the search result and the modified information includes: Using the information before modification in the search result as target information before modification; The target training data is generated using the retrieval result and the modified information.

7. The method according to claim 4, characterized in that Generating target training data according to the search result and the modified information includes: Acquire multiple similar pre-modification information corresponding to the pre-modification information, and determine second pre-modification information from the multiple similar pre-modification information; Modify the information before modification in the search result into the second information before modification to obtain target information before modification; The target training data is generated using the modified retrieval results and the modified information.

8. A text error correction method, characterized in that: include: Get the text to be processed; The text to be processed is input into a text error correction language model to perform text error correction on the text to be processed; wherein the text error correction language model is obtained by training using the method described in any one of claims 1 to 7.

9. A text error correction language model training device, characterized in that: include: A data module, used to obtain usage scenario data and initial training data obtained based on the usage scenario data; wherein the initial training data includes pre-modification information and post-modification information corresponding to the pre-modification information; an enhancement module, configured to determine context information of the information before the change in the usage scenario data, perform information retrieval on the usage scenario data based on the context information and the information before the change to obtain a retrieval result, and generate target training data according to the retrieval result and the information after the change; wherein the context information is associated with the information before the change, and the target training data includes the target information before the change and the information after the change; A training module is used to train the text error correction language model using the target training data; wherein the text error correction language model is used to modify the target pre-correction information into the post-correction information.

10. A text error correction device, characterized in that: include: An acquisition module is used to obtain the text to be processed; A processing module is used to input the text to be processed into a text error correction language model to perform text error correction on the text to be processed; wherein the text error correction language model is obtained after training by the method described in any one of claims 1-7.

11. A text review system, characterized in that: Including the text correction device as described in claim 10.

12. An electronic device, characterized in that: include: A processor, and a memory storing a program, wherein the program comprises instructions which, when executed by the processor, cause the processor to perform the method according to any one of claims 1 to 8.

13. A non-transitory machine-readable medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-8.

14. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images