Data enhancement method and device
Through the combination of text mask and prediction model, a data augmented text with consistent semantics and controllable transformation is generated, which solves the problems of semantic in the prior art that enhancement text is not smooth and the degree of change is difficult to control, and achieves high-quality data augmentation effect.
Patent Information
- Application Number
- CN202411195055.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-28
- Publication Date
- 2024-12-03
AI Technical Summary
Existing data augmentation methods are prone to generate semantic uneven enhanced texts or cannot control the degree of transformation changes, especially in professional fields, text data is difficult to guarantee.
The enhanced text is treated in a masking manner, and the prediction model is used to obtain the replacement text, and enhancement data is generated based on semantic similarity and prediction probability to ensure the semantic consistency between the enhanced data and the original text and the controllability of the modified changes.
High-quality data enhancement is achieved, ensuring that the enhanced data is consistent with the original text semantics, controlling the degree of transformation and change, and improving the reliability and effectiveness of data enhancement.
Smart Images

Figure CN119067111A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer technology, and in particular to a method and device for data enhancement. Background Art
[0002] Data enhancement is to use existing data and adopt certain methods to construct data of the same type that is as similar as possible to the existing data. This data should be as similar as possible to the original data, but not completely the same. Only in this way can it produce positive effects when using these data for training, improve model accuracy, and reduce model overfitting. There are two main ways to implement data enhancement. One is to directly perform data enhancement by editing the original text by randomly adding or deleting some words, and the other is to perform data enhancement by back-translating the sentences (using two different translation tools to translate the sentences into foreign languages and then translate the foreign languages into Chinese).
[0003] In the process of implementing the present invention, the inventors found that the prior art has the following problems:
[0004] The existing methods of directly editing the original text are prone to generating enhanced texts that are semantically incoherent or have completely deviated from the semantics of the original sentence, thus affecting the enhancement effect. The back-translation method cannot control the degree of modification and change to the original text, cannot achieve accurate control of the scope of change of the enhanced sample, and is heavily dependent on the quality of the translation tool. If the quality of the translation tool cannot be determined, the quality of the back-translated text cannot be guaranteed, especially for text data in professional fields, where the quality is even more difficult to guarantee. Summary of the invention
[0005] In view of this, an embodiment of the present invention provides a method and device for data enhancement, which realizes high-quality data enhancement. The use of text masking can not only ensure the semantic consistency between the enhanced data and the original text, but also control the degree of modification and change of the original text. Combined with the excellent replacement text prediction ability of the prediction model, the high quality of the enhanced data is guaranteed.
[0006] To achieve the objective, according to one aspect of an embodiment of the present invention, a method for data enhancement is provided, comprising:
[0007] Acquire text data to be enhanced, and perform masking processing on the text data to be enhanced to obtain masked text data;
[0008] Inputting the masked text data into a pre-built prediction model to obtain replacement text output by the prediction model;
[0009] According to the text data to be enhanced, a target text is selected from the replacement text, and according to the target text and the mask text data, enhanced data of the text data to be enhanced is generated.
[0010] Optionally, mask processing is performed on the text data to be enhanced to obtain masked text data, including: segmenting the text to be enhanced, performing part-of-speech analysis on the segmentation result, and determining masked participles in the text to be enhanced based on the result of the part-of-speech analysis; and masking the masked participles to obtain masked text data.
[0011] Optionally, the masked text data is input into a pre-built prediction model to obtain replacement text output by the prediction model, including: inputting the masked text data into the prediction model, and based on a pre-built text vocabulary, using the prediction model to predict replacement text that is semantically similar to the text data to be enhanced and the prediction probability corresponding to the replacement text; using the prediction model, sorting the replacement text according to the corresponding prediction probability, and obtaining the sorted replacement text output by the prediction model.
[0012] Optionally, according to the text data to be enhanced, a target text is selected from the replacement text, including: searching for masked word participles in the text data to be enhanced from the sorted replacement text, and determining a selection range of the target text to be selected in the sorted replacement text according to the position of the masked word participles in the sorted replacement text; and selecting the target text from the sorted replacement text according to the selection range.
[0013] Optionally, the target text has a corresponding prediction probability; based on the target text and the mask text data, generating enhanced data of the text data to be enhanced, including: calculating the recommendation probability of the enhanced data of the text data to be enhanced generated by the target text based on the prediction probability of the target text, and marking the recommendation probability on the enhanced data.
[0014] Optionally, in response to receiving a request to construct a training set using the enhanced data, the method further includes: using a label corresponding to the text data to be enhanced as a label of the enhanced data, and adding the enhanced data marked with the label to the training set.
[0015] Optionally, the prediction model is constructed based on an artificial neural network model pre-trained using a masked language model task.
[0016] According to a second aspect of an embodiment of the present invention, a data enhancement device is provided, including:
[0017] A mask processing module, used for acquiring text data to be enhanced, performing mask processing on the text data to be enhanced, and obtaining masked text data;
[0018] A text acquisition module, used for inputting the masked text data into a pre-built prediction model to obtain a replacement text output by the prediction model;
[0019] The data enhancement module is used to select a target text from the replacement text according to the text data to be enhanced, and generate enhanced data of the text data to be enhanced according to the target text and the mask text data.
[0020] Optionally, the mask processing module is further used to: perform word segmentation on the text to be enhanced, perform part-of-speech analysis on the word segmentation results, and determine masked word segmentations in the text to be enhanced based on the results of the part-of-speech analysis; and perform mask processing on the masked word segmentations to obtain masked text data.
[0021] Optionally, the text acquisition module is also used to: input the masked text data into the prediction model, and based on a pre-constructed text vocabulary, predict replacement texts that are semantically similar to the text data to be enhanced and the prediction probabilities corresponding to the replacement texts through the prediction model; sort the replacement texts according to the corresponding prediction probabilities through the prediction model, and obtain the sorted replacement texts output by the prediction model.
[0022] Optionally, the data enhancement module is also used to: search for masked word participles in the text data to be enhanced from the sorted replacement text, determine a selection range of the target text to be selected in the sorted replacement text according to the position of the masked word participles in the sorted replacement text; and select the target text from the sorted replacement text according to the selection range.
[0023] Optionally, the target text has a corresponding prediction probability; the data enhancement module is further used to: calculate the recommendation probability of the enhanced data of the text data to be enhanced generated by the target text according to the prediction probability of the target text, and mark the recommendation probability on the enhanced data.
[0024] Optionally, in response to receiving a request to construct a training set using the enhanced data, the data enhancement device also includes a label determination module, which is used to: use the label corresponding to the text data to be enhanced as the label of the enhanced data, and add the enhanced data marked with the label to the training set.
[0025] Optionally, the prediction model is constructed based on an artificial neural network model pre-trained using a masked language model task.
[0026] According to a third aspect of an embodiment of the present invention, there is provided a data enhanced electronic device, including:
[0027] one or more processors;
[0028] a storage device for storing one or more programs,
[0029] When the one or more programs are executed by the one or more processors, the one or more processors implement the method provided in the first aspect of the embodiment of the present invention.
[0030] According to a fourth aspect of an embodiment of the present invention, a computer-readable medium is provided, on which a computer program is stored, and when the program is executed by a processor, the method provided by the first aspect of the embodiment of the present invention is implemented.
[0031] According to a fifth aspect of an embodiment of the present invention, there is provided a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the method provided in the first aspect of the embodiment of the present invention is implemented.
[0032] An embodiment of the invention has the following advantages or beneficial effects: by obtaining text data to be enhanced, masking the text data to be enhanced to obtain masked text data; inputting the masked text data into a pre-built prediction model to obtain replacement text output by the prediction model; selecting target text from the replacement text according to the text data to be enhanced, and generating enhanced data of the text data to be enhanced according to the target text and the masked text data. The technical solution realizes a high-quality data enhancement method. The use of text masking can not only ensure the semantic consistency of the enhanced data and the original text, but also control the degree of modification and change of the original text. Combined with the excellent replacement text prediction ability of the prediction model, the high quality of the enhanced data is guaranteed. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] The accompanying drawings are used to better understand the present invention and do not constitute an improper limitation of the present invention.
[0034] Figure 1 is a schematic diagram of the main process of the data enhancement method according to an embodiment of the present invention;
[0035] Figure 2 is a flowchart of a method for data enhancement according to a reference embodiment of the present invention;
[0036] Figure 3 is a flowchart of a method for data enhancement according to another reference embodiment of the present invention;
[0037] Figure 4 is a detailed flowchart of a data enhancement method according to an embodiment of the present invention;
[0038] Figure 5 is a schematic diagram of main modules of a data enhancement device according to an embodiment of the present invention;
[0039] Figure 6 is an exemplary system architecture diagram to which embodiments of the present invention may be applied;
[0040] Figure 7 It is a schematic diagram of the structure of a computer system of a terminal device or a server suitable for implementing an embodiment of the present invention. DETAILED DESCRIPTION
[0041] It should be noted that the acquisition, storage and application of user personal information involved in the technical solution of the present disclosure are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.
[0042] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.
[0043] The existing methods of directly editing the original text are prone to generate enhanced texts that are semantically incoherent or have completely deviated from the semantics of the original sentence, affecting the enhancement effect; and the back-translation method cannot control the degree of modification and change to the original text, cannot achieve accurate control of the scope of change of the enhanced sample, and is heavily dependent on the quality of the translation tool. If the quality of the translation tool cannot be determined, the quality of the back-translated text cannot be guaranteed, especially for text data in professional fields, and cannot meet practical applications well.
[0044] In order to solve the above problems existing in the prior art, the present invention proposes a method for data enhancement, which adopts a text masking method to perform mask processing on text data to be enhanced, uses a pre-built prediction model to obtain a replacement text of the masked part in the masked text data, selects a target text therefrom, and uses the target text to generate enhanced text data, thereby realizing high-quality data enhancement. The text masking method can not only ensure the semantic consistency between the enhanced data and the original text, but also control the degree of modification and change of the original text. Combined with the excellent replacement text prediction ability of the prediction model, the high quality of the enhanced data is ensured.
[0045] Figure 1 is a schematic diagram of the main process of the data enhancement method according to an embodiment of the present invention, such as Figure 1 As shown, the data enhancement method of the embodiment of the present invention includes the following steps S101 to S103.
[0046] Step S101: Acquire text data to be enhanced, and perform mask processing on the text data to be enhanced to obtain masked text data.
[0047] Specifically, in artificial intelligence applications, data is the basis of model training. However, obtaining a large amount of high-quality annotated data is often a time-consuming and expensive task. Data enhancement technology generates more training samples by changing the original data, thereby improving the generalization ability of the model. In the fields of natural language processing and speech recognition, during the construction of training samples, based on the annotated text data, the data enhancement task is started, the text data to be enhanced is packaged and encapsulated as a data enhancement request, and the data enhancement request is sent to the enhancement system; of course, when the annotated text data reaches a preset quantity threshold, the annotated text data can be used as a data enhancement task, and the data enhancement task is automatically triggered and sent to the enhancement system for data enhancement. When the enhancement system receives a data enhancement task or a data enhancement request, it parses the text data to be enhanced according to the preset format, or obtains the text data to be enhanced from the specified storage path.
[0048] Furthermore, for the acquired text data to be enhanced, first use the word segmentation tool to segment the text data, and then determine the number of masks of the current text data to be enhanced according to the preset mask ratio and the number of words in the word segmentation result. For example, if the mask ratio is 20%, and there are 5 words after the text data to be enhanced is segmented, then the number of masks is 1 word. Then, according to the number of masks, randomly extract 1 word from the word segmentation result of the text data to be enhanced as the masked word, marked as MASK, and then obtain the masked text data. For example: the text data to be enhanced is "this movie is very good", and the result after word segmentation is: this, movie, is, very, good. Randomly select "movie" as the masked word, then the masked text data is "this [MASK] is very good".
[0049] Step S102: input the masked text data into a pre-built prediction model to obtain a replacement text output by the prediction model.
[0050] Specifically, the embodiment of the present invention uses an artificial neural network model to perform text prediction on masked word segmentation in masked text data, and outputs replacement text corresponding to the masked word segmentation in the masked text data. It can be understood that the construction of the prediction model requires the collection of various masked training texts in advance, and the marking of replacement texts corresponding to the masked word segmentation, the formation of a masked training set of the prediction model, and the use of the training set to train the prediction model. After the training converges, a trained prediction model is obtained. As long as the training samples of the masked training set are accurate enough, the prediction model will become a masked word segmentation prediction model with excellent performance after continuous learning, prediction and iterative updating.
[0051] Step S103: selecting a target text from the replacement text according to the text data to be enhanced, and generating enhanced data of the text data to be enhanced according to the target text and the mask text data.
[0052] Specifically, based on the replacement text output by the obtained prediction model, the masked word segmentation of [MASK] in the text data to be enhanced is obtained by comparing the text data to be enhanced with the masked text data, or when marking the MASK, the corresponding masked word segmentation is directly recorded. For example, the masked word segmentation of MASK in the above-mentioned "This [MASK] is very good" is "movie", and then the target text similar to the masked word segmentation is selected from the replacement text, such as "film" and "film". Finally, the target text is used to replace the masked word segmentation in the masked text data to generate enhanced text data with similar semantics to the text data to be enhanced, but with different expressions.
[0053] According to another embodiment of the present invention, in response to receiving a request to construct a training set using the enhanced data, the method further includes: using a label corresponding to the text data to be enhanced as a label of the enhanced data, and adding the enhanced data marked with the label to the training set.
[0054] Specifically, considering that data enhancement mainly serves the construction of the training set, the enhanced text data is only generated based on the text data to be enhanced itself and has no corresponding labels. Therefore, after generating the enhanced text data to be enhanced, or when receiving a request to use the enhanced data to construct a training set, trace the text data to be enhanced corresponding to the enhanced data, obtain the label corresponding to the text data to be enhanced, and use the label corresponding to the text data to be enhanced as the label of the enhanced data to construct a complete enhanced data that can be directly used to construct a training set. Finally, add the enhanced data with the label to the training set.
[0055] Figure 2 is a flow chart of a method for data enhancement according to a reference embodiment of the present invention. As another embodiment of the present invention, Figure 2 As shown, the data enhancement method may include:
[0056] Step S201: Acquire text data to be enhanced.
[0057] Step S202: segment the text to be enhanced, perform part-of-speech analysis on the segmentation results, and determine masked segmentations in the text to be enhanced according to the results of the part-of-speech analysis.
[0058] Specifically, although the semantic consistency between the enhanced text data and the original text (text data to be enhanced) can be effectively controlled by the preset mask ratio, over-masking of the text data to be enhanced can be avoided. Over-masking means that the ratio of masked word segmentation is too large, causing the predicted replacement text to deviate from the semantics of the original text. However, in order to further ensure the quality of data enhancement, the embodiment of the present invention also provides a preferred mask processing method.
[0059] First, the text to be enhanced is segmented to obtain each segmented text after segmentation. Based on the mask ratio mentioned above, the part of speech of each segmented text in the text to be enhanced is analyzed using a part-of-speech analysis tool to determine the part of speech of each segmented text (noun, verb, adverb, adjective, etc.), according to the preset mask segmentation priority, for example, the mask segmentation priority determined based on common parts of speech: auxiliary word → adjective → adverb → pronoun → verb → name. According to the part of speech of each segmented text, the segmented text with priority masking is selected from the masked segmentation priority, such as the above-mentioned "this", "film", "very", and "good". First, the number of masks is determined to be 1 according to the mask ratio, and then the text segmentation corresponding to this mask is determined. Since the text data has no auxiliary words, the highest priority is the segmented text "good" with the part of speech being an adjective, so the masked segmentation is "good".
[0060] Step S203: performing mask processing on the masked word segmentation to obtain masked text data.
[0061] Specifically, based on the masked word segmentation determined by taking into account the part of speech of each word in the text data to be enhanced, the text word at that position is masked according to the position of the masked word segmentation in the text data to be enhanced, and is marked with MASK to explicitly indicate that mask processing has been performed, and the text data to be enhanced marked with MASK is used as masked text data.
[0062] The embodiment of the present invention combines the mask ratio and the part of speech of the text segmentation to determine the mask segmentation of the text data to be enhanced, which can ensure to the greatest extent that the semantics of the enhanced text data and the semantics of the original text (the text data to be enhanced) do not deviate, thereby ensuring the subsequent generation quality of the target text and the enhanced text data.
[0063] It can be understood that if the number of enhanced data of the generated text data to be enhanced based on the above MASK does not meet the quantity requirements in the data enhancement task, the remaining text segmentations can be screened for masked segmentations according to the mask ratio and the part of speech of the text segmentation to determine a new round of masked segmentations. For example, in the above "This movie is very good", if the number of enhanced text data obtained after data enhancement based on the masked segmentation "good" does not meet the quantity requirements in the data enhancement task, then according to the above preset masked segmentation priority, the adverb "very" is selected from the remaining "this", "part", "movie", and "very" as the masked segmentation for the new round of data enhancement.
[0064] Step S204: input the masked text data into a pre-built prediction model to obtain a replacement text output by the prediction model.
[0065] Step S205: Select a target text from the replacement text according to the text data to be enhanced, and generate enhanced data of the text data to be enhanced according to the target text and the mask text data.
[0066] Figure 3 FIG. 1 is a flow chart of a method for data enhancement according to another reference embodiment of the present invention. As another embodiment of the present invention, Figure 3 As shown, the data enhancement method may include:
[0067] Step S301: Acquire text data to be enhanced, and perform mask processing on the text data to be enhanced to obtain masked text data.
[0068] Step S302: input the masked text data into the prediction model, and based on a pre-constructed text vocabulary, predict replacement texts with semantic similarity to the text data to be enhanced and prediction probabilities corresponding to the replacement texts through the prediction model.
[0069] According to one embodiment of the present invention, the prediction model is constructed based on an artificial neural network model pre-trained using a masked language model task.
[0070] Specifically, the prediction model of an embodiment of the present invention selects an artificial neural network model pre-trained using a masked language model task (Masked Language Model), or an improved model based on the artificial neural network model pre-trained using the masked language model task, such as the pre-trained language representation model BERT, the pre-trained language model roBERTa, and the lightweight pre-trained language model ALBERT.
[0071] The main reason for choosing an artificial neural network model that uses the masked language model task for pre-training is that this type of model will randomly replace some tokens with [MASK] tags during pre-training, and then let the model predict these replaced tokens. This process simulates language noise and uncertainty in the real world, such as typos, slips of the tongue, or missing information. When we also use masked [MASK] tags to perform data augmentation on the augmented text data, the model can treat these masked text data as tasks similar to the model pre-training data. Since the model has learned how to process [MASK] tags and determine the replacement text corresponding to the [MASK] tags during the pre-training stage, it can more efficiently and conveniently utilize the ability of this type of model to process [MASK]-tagged text data during pre-training without spending too much energy on model training, thus achieving the purpose of saving time and effort. With the help of this type of model, the replacement text corresponding to the masked word segmentation in the masked text data can be efficiently and accurately obtained.
[0072] In contrast, if we use other forms of data augmentation, such as inserting, deleting, or replacing arbitrary tokens, without using the mask marking method, the model may need to spend more time and resources to learn to adapt to this new augmentation method, because during the pre-training stage, the model has not been exposed to these forms of text data and may be confused by them.
[0073] In other words, using the mask [MASK] tag for data augmentation can make the augmented data obtained during data augmentation more credible, and also enable the subsequent prediction model training to have a certain degree of anti-interference ability and backbone information extraction ability, thereby improving the overall generalization performance of the prediction model.
[0074] Furthermore, the masked file data is input into a prediction model constructed based on an artificial neural network model pre-trained using a masked language model task, and the prediction model calls a pre-constructed text vocabulary, and uses the prediction model to predict a replacement text that is semantically similar to the original text (text data to be enhanced) of the input masked text data, as well as a prediction probability corresponding to the replacement text. The pre-constructed text vocabulary of the embodiment of the present invention is equivalent to a general or business scenario word knowledge base, which stores various commonly used words or text fragments, and provides a data source for prediction for the prediction model.
[0075] The embodiment of the present invention uses masked text data as the input form of the prediction model, which not only achieves an input form consistent with the artificial neural network model pre-trained using the masked language model task, but is also more conducive to the prediction efficiency and prediction effect of the prediction model, but also can ensure semantic consistency, avoiding the semantics of the enhanced text data generated according to the output replacement text from deviating from the semantics of the original text.
[0076] Step S303: sorting the replacement texts according to the corresponding prediction probabilities through the prediction model, and obtaining the sorted replacement texts output by the prediction model.
[0077] Specifically, based on the replacement text predicted by the above-mentioned prediction model and the predicted probability corresponding to the replacement text, in order to facilitate the subsequent determination of the target text, the replacement text is sorted according to the corresponding predicted probability, which can be in ascending or descending order. Usually, descending order is selected to obtain the sorted replacement text, and the sorted replacement text is used as the final output of the prediction model.
[0078] Step S304: Find the masked word segmentation in the text data to be enhanced from the sorted replacement text, and determine the selection range of the target text to be selected in the sorted replacement text according to the position of the masked word segmentation in the sorted replacement text.
[0079] Specifically, according to the masked word segmentation recorded when the text data to be enhanced is masked, or the masked word segmentation determined by comparing the masked text data and the text data to be enhanced, the masked word segmentation in the text data to be enhanced is found from the sorted replacement text, and according to the sorting position of the masked word segmentation in the sorted replacement text, when the replacement text is sorted in descending order according to the predicted probability, the next position of the sorting position is used as the starting point, and the predicted probability corresponding to the masked word segmentation is referred to to determine the corresponding lower limit probability threshold, and the position of the lower limit probability threshold in the sorted replacement text is used as the end point to obtain the selection range of the target text to be selected in the sorted replacement text. It can be understood that the next position of the sorting position is used as the starting point because the masked word segmentation cannot be used to generate the enhanced text data, and then it is still the original text without the enhancement effect. Of course, a mark can also be made at the masked word segmentation to prompt that the word does not participate in the subsequent generation of enhanced text data.
[0080] For example, the following table 1 shows the results of partial replacement text output by the prediction model. Taking the above-mentioned mask text data: this movie is [MASK] as an example, after the prediction model, the sorted replacement text output is shown in Table 1.
[0081] As can be seen from Table 1, the position of the masked word "good" is found, and "good" is marked as not participating in sampling, indicating that it does not participate in the subsequent generation of enhanced text data. According to the predicted probability of "good" 0.31 and the predicted probabilities of other texts around the position of "good", the lower limit probability threshold is determined to be 0.3, then the starting point of the selection range of the target text is determined to be the next position of "good", and the end point is "beautiful" corresponding to 0.3, that is, the target text is "good", "like", and "beautiful".
[0082] It can be understood that, based on the premise that the replacement text is sorted in descending order according to the predicted probability, the sorted replacement text is defined as the sorted replacement text. Then, when the difference between the predicted probability of the partial replacement text before the sorting position of the masked word in the sorted replacement text and the corresponding predicted probability of the masked word in the sorted replacement text is less than a preset comparison threshold, these partial replacement texts are considered to be similar texts to the masked word, and the positions of these similar texts can also be added to the selection range.
[0083] To summarize, the selection range of the target text to be selected in the sorted replacement text is flexibly determined based on the predicted probability corresponding to the masked word segmentation in the sorted replacement text, and the distribution of the predicted probabilities corresponding to other texts in the sorted replacement text except the masked word segmentation.
[0084] Of course, if the prediction model directly outputs the predicted replacement text and the corresponding prediction probability of the replacement text without sorting it, then the masked word segmentation in the text data to be enhanced and the prediction probability corresponding to the masked word segmentation are found from the sorted replacement text, and the screening threshold is determined based on the prediction probability, and the target text is screened out from the replacement text according to the determined screening threshold. For example, the prediction probability of the masked word segmentation "good" mentioned above is 0.31, so the screening threshold is selected as 0.3, that is, the replacement text with a prediction probability between 0.31 and 0.3 is regarded as the target text. This screening method is relatively slow compared to the above-mentioned method of sorting first and then determining the selection range, and the effect is not particularly good, because the replacement text is disordered, and it is impossible to understand the distribution of the prediction probability of each text, which is not conducive to determining a reasonable screening threshold.
[0085] Step S305: Select the target text from the sorted replacement texts according to the selection range.
[0086] Specifically, according to the selection range determined above, target texts are selected from the starting point to the end point. These target texts are the texts predicted by the prediction model to be closest or relatively close to the mask text under the premise of being semantically similar to the original text. These target texts can be used to replace the mask text in the mask text data to generate enhanced text data for the text data to be enhanced.
[0087] Step S306: Generate enhanced data of the text data to be enhanced according to the target text and the mask text data.
[0088] According to one embodiment of the present invention, the target text has a corresponding prediction probability; based on the target text and the mask text data, enhanced data of the text data to be enhanced is generated, including: based on the prediction probability of the target text, calculating the recommendation probability of the enhanced data of the text data to be enhanced generated by the target text, and marking the recommendation probability on the enhanced data.
[0089] Specifically, based on the above-mentioned enhanced data generation method, multiple target texts will generate multiple enhanced text data of text data to be enhanced. Since the prediction probability of each target text is different, but the multiple enhanced text data generated are indistinguishable to the user, in order to facilitate the user to use these enhanced text data to construct a high-quality training sample set, the embodiment of the present invention uses the prediction probability of each target text output by the prediction model, and calculates the ratio of the prediction probability of each target text to the sum of the prediction probabilities of each target text to obtain the recommended probability of the enhanced text data of the text data to be enhanced generated by the target text, and marks the calculated recommended probability on the corresponding enhanced text data to assist the user to selectively or focus on using these enhanced text data to construct a training sample set.
[0090] For example, still taking "This movie is good" as the text data to be enhanced, for each target text "good", "like", and "beautiful" in Table 1 above, corresponding enhanced text data is generated, and the corresponding recommendation probabilities calculated are shown in Table 2 below.
[0091] The recommendation probability of each enhanced text data calculated by the embodiment of the present invention establishes a reference standard for selecting, screening or using each enhanced text data, which is more conducive to the construction of a training sample set.
[0092] Figure 4 Detailed flowchart of the data enhancement method of the embodiment of the present invention. Obtain the text data to be enhanced "This movie is very good", segment the text data to be enhanced, use the preset mask ratio, and analyze the part of speech of the segmentation result to determine that the mask text is "good", and obtain the mask text data "This movie is [MASK]".
[0093] Furthermore, the masked text data "This movie is [MASK]" is input into a prediction model constructed based on an artificial neural network model pre-trained using a masked language model task. The prediction model calls a pre-constructed text vocabulary to encode each character in the masked text data to obtain a corresponding representation vector, and predicts a replacement text at [MASK] that is semantically similar to the text data to be enhanced based on the position of each character in the masked text data. The replacement text and the prediction probability corresponding to the replacement text are then output through a classifier at the back end of the prediction model.
[0094] Finally, according to the masked word "good" in the text data to be enhanced, the text with a predicted probability after "good" greater than 0.3 is selected from the replacement text as the target text, such as "good", "like" and "beautiful" in the figure, and then the enhanced text data of the corresponding three text data to be enhanced are generated.
[0095] It can be understood that based on the above method of data enhancement through masking, we can perform data enhancement M times. Assuming that the number of texts to be enhanced is N, the total amount of data after enhancement is (M+1)*N, where M*N data are all enhanced data with masks. Usually, M is an integer between 2 and 4. If the amount of data N is small, the M value can be appropriately increased to between 4 and 6.
[0096] Figure 5 Schematic diagram of the main modules of the data enhancement device according to an embodiment of the present invention. Figure 5 As shown, the data enhancement device 500 mainly includes a mask processing module 501, a text acquisition module 502 and a data enhancement module 503.
[0097] The mask processing module 501 is used to obtain text data to be enhanced, and perform mask processing on the text data to be enhanced to obtain masked text data;
[0098] A text acquisition module 502, used for inputting the masked text data into a pre-built prediction model to acquire a replacement text output by the prediction model;
[0099] The data enhancement module 503 is used to select a target text from the replacement text according to the text data to be enhanced, and generate enhanced data of the text data to be enhanced according to the target text and the mask text data.
[0100] According to one embodiment of the present invention, the mask processing module 501 is also used to: perform word segmentation on the text to be enhanced, and perform part-of-speech analysis on the word segmentation results, and determine the masked word segmentations in the text to be enhanced according to the results of the part-of-speech analysis; perform mask processing on the masked word segmentations to obtain masked text data.
[0101] According to another embodiment of the present invention, the text acquisition module 502 is also used to: input the masked text data into the prediction model, and based on a pre-constructed text vocabulary, predict replacement texts with similar semantics to the text data to be enhanced and the prediction probabilities corresponding to the replacement texts through the prediction model; sort the replacement texts according to the corresponding prediction probabilities through the prediction model, and obtain the sorted replacement texts output by the prediction model.
[0102] According to another embodiment of the present invention, the data enhancement module 503 is further used to: search for masked word participles in the text data to be enhanced from the sorted replacement text, determine a selection range of the target text to be selected in the sorted replacement text according to the position of the masked word participles in the sorted replacement text; and select the target text from the sorted replacement text according to the selection range.
[0103] According to another embodiment of the present invention, the target text has a corresponding prediction probability; the data enhancement module 503 is further used to: calculate the recommendation probability of the enhanced data of the text data to be enhanced generated by the target text according to the prediction probability of the target text, and mark the recommendation probability on the enhanced data.
[0104] According to another embodiment of the present invention, in response to receiving a request to construct a training set using the enhanced data, the data enhancement device 500 also includes a label determination module (not shown in the figure), which is used to: use the label corresponding to the text data to be enhanced as the label of the enhanced data, and add the enhanced data marked with the label to the training set.
[0105] According to yet another embodiment of the present invention, the prediction model is constructed based on an artificial neural network model pre-trained using a masked language model task.
[0106] Figure 6 is an exemplary system architecture diagram to which embodiments of the present invention can be applied.
[0107] like Figure 6 As shown, system architecture 600 may include terminal devices 601, 602, 603, network 604 and server 605. Network 604 is used to provide a medium for communication links between terminal devices 601, 602, 603 and server 605. Network 604 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.
[0108] Users can use terminal devices 601, 602, 603 to interact with server 605 through network 604 to receive or send messages, etc. Various communication client applications, such as data enhancement applications, etc., can be installed on terminal devices 601, 602, 603 (only as an example).
[0109] The terminal devices 601 , 602 , and 603 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers, etc.
[0110] The server 605 may be a server that provides various services, such as a background management server that provides support for data enhancement performed by users using the terminal devices 601, 602, and 603 (only as an example). The background management server may obtain the text data to be enhanced, perform masking processing on the text data to be enhanced, and obtain masked text data; input the masked text data into a pre-built prediction model to obtain replacement text output by the prediction model; select a target text from the replacement text according to the text data to be enhanced, generate enhanced data of the text data to be enhanced according to the target text and the masked text data, and feed back the processing result (such as enhanced data, etc. - only as an example) to the terminal device.
[0111] It should be noted that the data enhancement method provided in the embodiment of the present invention is generally executed by the server 605 , and accordingly, the data enhancement device is generally set in the server 605 .
[0112] It should be understood that Figure 6 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.
[0113] Reference below Figure 7 , which shows a schematic diagram of the structure of a computer system of a terminal device or a server suitable for implementing an embodiment of the present invention. Figure 7 The terminal device or server shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.
[0114] like Figure 7As shown, the computer system 700 includes a central processing unit (CPU) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage part 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the system 700 are also stored. The CPU 701, the ROM 702, and the RAM 703 are connected to each other through a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.
[0115] The following components are connected to the I / O interface 705: an input section 706 including a keyboard, a mouse, etc.; an output section 707 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN card, a modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as needed. A removable medium 711, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 710 as needed, so that a computer program read therefrom is installed into the storage section 708 as needed.
[0116] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 709, and / or installed from the removable medium 711. When the computer program is executed by the central processing unit (CPU) 701, the above-mentioned functions defined in the system of the present invention are executed.
[0117] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination thereof.
[0118] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0119] The units involved in the embodiments of the present invention may be implemented by software or hardware. The units described may also be arranged in a processor, for example, it may be described as follows: a processor includes: a mask processing module, a text acquisition module and a data enhancement module.
[0120] Among them, the names of these modules do not constitute a limitation on the modules themselves in some cases. For example, the mask processing module can also be described as "a module for obtaining text data to be enhanced, performing mask processing on the text data to be enhanced, and obtaining masked text data."
[0121] On the other hand, the present invention also provides a computer-readable medium, which may be included in the device described in the embodiment; or may exist independently without being assembled into the device. The computer-readable medium carries one or more programs, and when the one or more programs are executed by a device, the device includes: obtaining text data to be enhanced, performing mask processing on the text data to be enhanced, and obtaining masked text data; inputting the masked text data into a pre-built prediction model, and obtaining replacement text output by the prediction model; selecting a target text from the replacement text according to the text data to be enhanced, and generating enhanced data of the text data to be enhanced according to the target text and the masked text data.
[0122] The technical solution according to the embodiment of the present invention has the following advantages or beneficial effects: by obtaining the text data to be enhanced, masking the text data to be enhanced to obtain masked text data; inputting the masked text data into a pre-built prediction model to obtain the replacement text output by the prediction model; according to the text data to be enhanced, selecting the target text from the replacement text, and generating the enhanced data of the text data to be enhanced according to the target text and the masked text data. The technical solution realizes a high-quality data enhancement method. The use of text masking can not only ensure the semantic consistency of the enhanced data and the original text, but also control the degree of modification and change of the original text. Combined with the excellent replacement text prediction ability of the prediction model, the high quality of the enhanced data is guaranteed.
[0123] The specific implementation methods described herein do not constitute limitations on the scope of protection of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention shall be included in the scope of protection of the present invention.
Claims
1. A method of data enhancement, It is characterized in that include: Acquire text data to be enhanced, and perform masking processing on the text data to be enhanced to obtain masked text data; Inputting the masked text data into a pre-built prediction model to obtain replacement text output by the prediction model; According to the text data to be enhanced, a target text is selected from the replacement text, and according to the target text and the mask text data, enhanced data of the text data to be enhanced is generated.
2. The method according to claim 1, It is characterized in that Performing mask processing on the text data to be enhanced to obtain masked text data, including: Performing word segmentation on the text to be enhanced, and performing part-of-speech analysis on the word segmentation results, and determining masked word segmentations in the text to be enhanced according to the results of the part-of-speech analysis; Masking is performed on the masked word segmentation to obtain masked text data.
3. The method according to claim 1, It is characterized in that Inputting the masked text data into a pre-built prediction model to obtain replacement text output by the prediction model includes: Inputting the masked text data into the prediction model, and predicting replacement texts with semantic similarity to the text data to be enhanced and prediction probabilities corresponding to the replacement texts through the prediction model based on a pre-constructed text vocabulary; The replacement texts are sorted according to the corresponding prediction probabilities by using the prediction model, and the sorted replacement texts output by the prediction model are obtained.
4. The method according to claim 3, It is characterized in that Selecting a target text from the replacement text according to the text data to be enhanced includes: Finding masked participles in the text data to be enhanced from the sorted replacement texts, and determining a selection range of the target text to be selected in the sorted replacement texts according to the positions of the masked participles in the sorted replacement texts; The target text is selected from the sorted replacement texts according to the selection range.
5. The method according to claim 1, It is characterized in that The target text has a corresponding prediction probability; Generating enhanced data of the text data to be enhanced according to the target text and the mask text data, comprising: According to the predicted probability of the target text, the recommendation probability of the enhanced data of the text data to be enhanced generated from the target text is calculated, and the recommendation probability is marked on the enhanced data.
6. The method according to claim 1, It is characterized in that In response to receiving a request to construct a training set using the augmented data, the method further includes: The label corresponding to the text data to be enhanced is used as the label of the enhanced data, and the enhanced data marked with the label is added to the training set.
7. The method according to claim 1, It is characterized in that The prediction model is constructed based on an artificial neural network model pre-trained using a masked language model task.
8. A data enhancement device, It is characterized in that include: A mask processing module, used for acquiring text data to be enhanced, performing mask processing on the text data to be enhanced, and obtaining masked text data; A text acquisition module, used for inputting the masked text data into a pre-built prediction model to obtain a replacement text output by the prediction model; The data enhancement module is used to select a target text from the replacement text according to the text data to be enhanced, and generate enhanced data of the text data to be enhanced according to the target text and the mask text data.
9. A mobile electronic device terminal, It is characterized in that include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.
10. A computer readable medium having a computer program stored thereon, It is characterized in that When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.
11. A computer program product comprising a computer program, It is characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.
