Error sentence generation method and device for model training, computer device and medium

Abstract

This invention relates to the field of artificial intelligence technology, and more particularly to a method, apparatus, computer device, and medium for generating incorrect sentences for model training. The method maps the number of terms in the text to be processed to an adjustment quantity, samples target terms from the text to be processed, adjusts the target terms to obtain augmented text, occludes the text to be processed, inputs it into a reconstruction model to obtain reconstructed text, trains a generative model based on the text to be processed, the augmented text, and the reconstructed text, inputs real-time text into the trained generative model to obtain generated incorrect sentence text, processes the text to be processed using multiple data augmentation methods to obtain text containing rich error types as labels, trains an end-to-end generative model, allowing real-time text to be directly input into the trained generative model to obtain generated incorrect sentence text without frequent switching of data augmentation methods, greatly improving the efficiency of incorrect sentence generation while ensuring high quality.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to a wrong sentence generation method and device for model training, a computer device and a medium. BACKGROUND

[0002] At present, a text recognition model based on a neural network usually needs a large amount of text data with labels, but the acquisition cost and labeling cost of such text data are relatively high, and therefore, the existing method usually adopts a data enhancement method to expand the text data.

[0003] However, the error types and error qualities of the text data generated by different data enhancement methods are different, and if text data with rich error types and high error qualities is needed, multiple data enhancement methods need to be executed in parallel, resulting in a cumbersome processing process of data enhancement, and a data enhancement strategy based on the fusion of multiple data enhancement methods is difficult to quickly migrate and deploy. Therefore, how to improve the efficiency of wrong sentence generation while ensuring high quality of generated wrong sentences has become a problem to be solved. SUMMARY

[0004] Therefore, the embodiments of the present application provide a wrong sentence generation method and device for model training, a computer device and a medium to solve the problem of low efficiency of wrong sentence generation while ensuring high quality of generated wrong sentences.

[0005] In a first aspect, the embodiments of the present application provide a wrong sentence generation method for model training, which comprises:

[0006] performing word segmentation processing on the to-be-processed text to obtain at least two word items, performing statistics on all the word items to obtain a number of word items, and mapping the number of word items to an adjusted number using a preset mapping function;

[0007] sampling all the word items N times to obtain N target word items, and adjusting the target word items in the to-be-processed text using a preset adjustment strategy for any target word item to obtain augmented text, wherein N is the adjusted number;

[0008] performing occlusion on the to-be-processed text using a preset sliding window, inputting the occluded to-be-processed text into a reconstructed model trained to perform reconstruction, and obtaining a reconstructed text;

[0009] training the generation model according to the to-be-processed text, the augmented text and the reconstructed text to obtain a trained generation model;

[0010] inputting the obtained real-time text into the trained generation model to obtain generated wrong sentence text.

[0011] In a second aspect, an embodiment of the present application provides an incorrect sentence generation device for model training, the incorrect sentence generation device comprising:

[0012] a text segmentation module, configured to perform text segmentation on the to-be-processed text to obtain at least two word items, count all the word items to obtain a word item quantity, and map the word item quantity to an adjusted quantity using a preset mapping function;

[0013] a text augmentation module, configured to sample all the word items N times to obtain N target word items, and adjust any target word item in the to-be-processed text using a preset adjustment strategy to obtain augmented text, where N is the adjusted quantity;

[0014] a text reconstruction module, configured to shield the to-be-processed text using a preset sliding window, and input the shielded to-be-processed text into a trained reconstruction model to obtain reconstructed text;

[0015] a model training module, configured to train the generation model according to the to-be-processed text, the augmented text, and the reconstructed text to obtain a trained generation model;

[0016] an incorrect sentence generation module, configured to input real-time text obtained to the trained generation model to obtain generated incorrect sentence text.

[0017] In a third aspect, an embodiment of the present application provides a computer device, the computer device comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, and the processor implements the incorrect sentence generation method of the first aspect when executing the computer program.

[0018] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, the computer readable storage medium storing a computer program, and the computer program implements the incorrect sentence generation method of the first aspect when executed by a processor.

[0019] Compared with the prior art, the embodiment of the present application has the following beneficial effects:

[0020] The text to be processed is segmented to obtain at least two word items, statistics are obtained for all the word items to obtain the number of word items, a preset mapping function is used to map the number of word items to an adjusted number, N times of sampling are performed on all the word items to obtain N target word items, for any target word item, a preset adjustment strategy is used to adjust the target word item in the text to be processed to obtain an augmented text, a preset sliding window is used to shield the text to be processed, and the shielded text to be processed is input into the trained reconstruction model for reconstruction to obtain a reconstructed text. The generated model is trained according to the text to be processed, the augmented text and the reconstructed text to obtain the trained generated model, the real-time text obtained is input into the trained generated model to obtain the generated incorrect sentence text, the text to be processed is processed by using the multi-class data augmentation method to obtain the generated model label containing rich error types, and the end-to-end generated model is trained based on the generated model label, so that the real-time text can be directly input into the trained generated model to obtain the generated incorrect sentence text, without frequent switching of the data augmentation method, the incorrect sentence generation efficiency is greatly improved under the condition that the quality of the generated incorrect sentence is high. BRIEF DESCRIPTION OF DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0022] Figure 1 is an application environment schematic diagram of a incorrect sentence generation method for model training provided by the first embodiment of the present application;

[0023] Figure 2 is a flowchart of a incorrect sentence generation method for model training provided by the first embodiment of the present application;

[0024] Figure 3 is a flowchart of a incorrect sentence generation method for model training provided by the second embodiment of the present application;

[0025] Figure 4 is a structure schematic diagram of a incorrect sentence generation device for model training provided by the third embodiment of the present application;

[0026] Figure 5 is a structure schematic diagram of a computer device provided by the fourth embodiment of the present application. DETAILED DESCRIPTION

[0027] In the following description, for purposes of explanation and not limitation, specific details are set forth such as particular architectures, techniques, etc. in order to provide a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application can be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known methods, devices, circuits, and

[0028] It will be understood that the terms "comprises" and / or "comprising," when used in this specification, include the presence of one or more features, integers, steps, operations, elements, and / or components described in the specification, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0029] It will be understood that the term "and / or," when used in the specification and in the following claims, is intended to mean one or more of the associated listed items can be present, and, if more than one, any combination of the associated listed items can be present.

[0030] As used in the description of the application and the following claims, the term "if" can be interpreted to mean "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [a described condition or event] is detected" can be interpreted to mean "upon determining" or "in response to determining" or "upon detecting [the described condition or event]" or "in response to detecting [the described condition or event]," depending on the context.

[0031] In addition, the terms "first," "second," "third," etc. are used herein only to describe different instances, and do not imply relative importance.

[0032] The description in the specification of a reference to "one embodiment" or "some embodiments" etc. means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the application. The appearances of the phrase "in one embodiment" or "in some embodiments" in various places in the specification are not necessarily all referring to the same embodiment, although it can be. Furthermore, the terms "comprises," "comprising," "includes," "including," "has," "having" and the like are intended to be open-ended terms that specifically permit the presence of one or more other features, integers, steps, operations, elements, and / or components in the example of the application.

[0033] The embodiments of this invention can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that utilize digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0034] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0035] It should be understood that the sequence number of each step in the following embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0036] To illustrate the technical solution of the present invention, specific embodiments are described below.

[0037] The method for generating incorrect sentences for model training provided in Embodiment 1 of this invention can be applied to, for example... Figure 1 In this application environment, the client communicates with the server. Clients include, but are not limited to, handheld computers, desktop computers, laptops, ultra-mobile personal computers (UMPCs), netbooks, cloud terminal devices, and personal digital assistants (PDAs). The server can be a standalone server or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0038] See Figure 2 This is a flowchart illustrating a method for generating incorrect sentences for model training provided in Embodiment 1 of the present invention. The above-described method for generating incorrect sentences can be applied to... Figure 1The client, a computer device connected to the server, receives text to be processed and real-time text. A generative model is deployed within the client; after training, this model can generate text with incorrect sentences. The text to be processed provides training samples for the generative model, and the real-time text refers to the text received by the server that needs to be used to generate incorrect sentences. For example... Figure 2 As shown, this method for generating incorrect sentences may include the following steps:

[0039] Step S201: Perform word segmentation on the text to be processed to obtain at least two terms. Count all terms to obtain the term count. Use a preset mapping function to map the term count to an adjustment quantity.

[0040] The text to be processed can be real text without errors. This text can be obtained through speech recognition in actual business scenarios, and speech recognition can be achieved using a speech recognition model. Word segmentation can be implemented using dictionary models, recurrent neural network models, or other methods.

[0041] A term can refer to the basic processing unit composed of characters or words in the text to be processed. The number of terms can refer to the total number of terms in the text to be processed. The mapping function can include the mapping relationship between the number of terms and the number of adjustments. The number of adjustments can refer to the number of terms that need to be adjusted.

[0042] The above steps involve segmenting the text to be processed to obtain at least two terms, counting all terms to obtain the term count, and using a preset mapping function to map the term count to the adjustment quantity. This allows the adjustment quantity to be adaptively determined based on the length of the text to be processed, thereby making the subsequently adjusted text closer to the real erroneous text and improving the quality of the subsequent erroneous text generation.

[0043] Step S202: Sample all terms N times to obtain N target terms. For any target term, use a preset adjustment strategy to adjust the target term in the text to be processed to obtain the augmented text.

[0044] Where N represents the number of adjustments, the adjustment strategy can refer to a preset term adjustment method, and the augmented text can refer to text containing the adjusted terms.

[0045] Specifically, in this embodiment, the sampling process can adopt random sampling. If the total number of terms is M, then the probability of each term being sampled is 1 / M.

[0046] In one embodiment, the sampling process can also employ probability sampling, which means assigning different sampling probabilities to different terms. For example, the part of speech corresponding to each term can be obtained from a preset dictionary, and different probabilities can be assigned to different parts of speech. For example, the probabilities of nouns, verbs, and adjectives can be set to a higher probability, while the probabilities of pronouns, prepositions, and quantifiers can be set to a lower probability. It should be noted that the sum of the probabilities of all terms is still 1.

[0047] Optionally, all terms are sampled N times to obtain N target terms, including:

[0048] For any given term, obtain the term's position in the text to be processed, assign a sequence number to the term based on its position, and obtain a sequence number sequence.

[0049] In the sequence of numbers, N random numbers are generated to obtain N generated numbers. The terms corresponding to the N generated numbers are determined as the N target terms.

[0050] Here, "position" can refer to the location of a term in the text to be processed, and "serial number" can refer to the position information of a term according to the text order. The serial number sequence includes M serial numbers, where M is the number of terms.

[0051] The generated number can refer to a random number generated within the sequence of numbers, and the target term can refer to the term to be adjusted.

[0052] Specifically, for a text containing M terms to be processed, the position of the m-th term can be represented by the index m according to the text order, where m is an integer in the range [1, M]. The index sequence can be represented as [1, 2, ..., M]. Random numbers are generated in the index sequence. For example, if the generated number is 5, then the term with index 5 is determined as the target term.

[0053] In this embodiment, the generated numbers can be repeated. For example, among N generated numbers, there can be two cases where the generated numbers are both 5.

[0054] In one implementation, the generated numbers cannot be repeated. If the newly randomly generated number is the same as the existing generated number, the number is randomly re-randomized. This rule of not allowing the generated numbers to be repeated is usually applied when the adjustment strategy is fixed. If the generated numbers can be repeated, two identical augmented texts will be generated, which will reduce the richness of the labels when the augmented text is used as the label of the generation model.

[0055] This embodiment determines the target terms by generating random numbers, so that each term has the possibility of being adjusted. This increases the error richness of the augmented text after adjustment, thereby improving the label richness of the generation model and effectively improving the error quality of the erroneous sentence text generated by the subsequently trained generation model.

[0056] Optionally, for any target term, a preset adjustment strategy is used to adjust the target term in the text to be processed, resulting in augmented text including:

[0057] For any target term, a preset set of adjustment strategies is sampled to obtain the sampling results;

[0058] The target terms are adjusted in the text to be processed using the sampling results to obtain the amplified text.

[0059] The preset set of adjustment strategies can contain a variety of adjustment strategies, such as deletion, replacement, swapping, and insertion. Sampling can refer to sampling all the adjustment strategies in the set of adjustment strategies, and the sampling result can refer to the adjustment strategy sampled from the set of adjustment strategies.

[0060] Specifically, when sampling the preset set of adjustment strategies, random sampling can be used, meaning that the sampling probability of each adjustment strategy is the same.

[0061] Similarly, implementers can refer to the above sampling method based on random numbers, assign a corresponding sequence number to each adjustment strategy, and determine the sampling result based on the sequence number corresponding to the generated random number.

[0062] In this embodiment, the adjustment strategy is sampled from a set of adjustment strategies that includes multiple adjustment strategies, so that the augmented text can have different error types, thereby increasing the error richness of the augmented text after adjustment, and thus improving the label richness of the generation model, which can effectively improve the error quality of the erroneous sentence text generated by the subsequently trained generation model.

[0063] Optionally, the set of adjustment strategies includes deletion strategies, replacement strategies, and insertion strategies;

[0064] The preset set of adjustment strategies is sampled, and the sampled adjustment strategies obtained for the text to be processed include:

[0065] The deletion, replacement, and insertion strategies are sampled with equal probability to obtain the sampling results.

[0066] Accordingly, the target terms in the text to be processed are adjusted using the sampling results to obtain the amplified text, which includes:

[0067] If the sampling result is a deletion strategy, the target terms are deleted from the text to be processed to obtain the amplified text;

[0068] If the sampling result is a replacement strategy, the target term is replaced with a preset first term in the text to be processed to obtain the augmented text;

[0069] If the sampling result is an insertion strategy, a preset second term is inserted after the target term in the text to be processed to obtain the amplified text.

[0070] Among them, the deletion strategy can refer to deleting the target term from the text to be processed, the replacement strategy can refer to replacing the target term with other terms in the text to be processed, and the insertion strategy can refer to adding a new term at the position of the target term in the text to be processed.

[0071] The first preset term can be a synonym, near-synonym, or similar-looking term obtained from a preset dictionary.

[0072] The pre-defined second term can refer to the related term of the target term obtained based on the pre-defined dictionary. The related term can refer to the term that frequently appears at the same time and adjacent to the target term in the existing corpus database.

[0073] Specifically, when the deletion strategy is adopted, the target terms are deleted from the text to be processed. The number of terms in the text to be processed before adjustment is M, and the number of terms in the expanded text after adjustment is M-1.

[0074] When a replacement strategy is used, the target term is replaced with other terms in the text to be processed. The number of terms in the text to be processed before adjustment is M, and the number of terms in the expanded text after adjustment is still M.

[0075] When using the insertion strategy, a new term is added at the target term position in the text to be processed. The number of terms in the text to be processed before adjustment is M, and the number of terms in the expanded text after adjustment is M+1.

[0076] In this embodiment, by default, related terms of the target term are inserted after the target term. It should be noted that the determination of related terms only considers terms that appear at the same time as the target term and in the position after the target term.

[0077] In one implementation, the implementer can set a related term to be inserted before the target term. Accordingly, the determination of the related term only considers the term that appears at the same time as the target term and in the position before the target term. The implementer can also flexibly determine the insertion position of the related term based on the positional relationship between the related term and the target term.

[0078] In this embodiment, by adjusting the text to be processed according to different adjustment strategies, the erroneous sentences in the augmented text can be made to better match the real erroneous sentences, thereby improving the quality of the augmented text and thus improving the label quality of the generation model. This can effectively improve the error quality of the erroneous sentence text generated by the subsequently trained generation model.

[0079] The above steps involve sampling all terms N times to obtain N target terms. For any target term, a preset adjustment strategy is used to adjust the target term in the text to be processed to obtain augmented text. The augmented text obtained through term adjustment is used as error text. This method conforms to existing data augmentation methods, has high generation efficiency, and can effectively save data preparation costs.

[0080] Step S203: Use a preset sliding window to occlude the text to be processed, and input the occluded text to be processed into the trained reconstruction model for reconstruction to obtain the reconstructed text.

[0081] The preset sliding window can refer to a vector of size 1*L containing L elements, all of which are zero. Occlusion can refer to multiplying each element of the preset sliding window with the corresponding term in the text to be processed. Since the elements in the preset sliding window are all zero, the multiplication operation is also a zeroing operation. The text to be processed after occlusion can refer to the text containing the zeroed terms.

[0082] A well-trained reconstruction model can be a well-trained autoencoder model, a well-trained generator model, or the like.

[0083] Reconstruction can refer to reconstructing obscured terms, and reconstructed text can refer to text containing reconstructed terms.

[0084] Specifically, the number of elements L in the preset sliding window should be less than the number of terms M in the text to be processed. In this embodiment, L is set to 1.

[0085] A well-trained reconstruction model can reconstruct the occluded word based on the context words of the occluded word, and it needs to reconstruct a different text word from the one to be processed corresponding to the occluded word, thus achieving the effect of error generation.

[0086] Optionally, using a preset sliding window to obscure the text to be processed includes:

[0087] Align the first element of the preset sliding window with the first word of the text to be processed, and move the preset sliding window on the text to be processed according to the random walk algorithm to obtain the target sliding window;

[0088] Multiply the target sliding window by its corresponding term in the text to be processed to obtain the occluded text.

[0089] The first element of the preset sliding window can refer to the first element of the preset sliding window. In the vector corresponding to the preset sliding window, the coordinates of the first element can be represented as (1, 1). The random walk algorithm can refer to making the preset sliding window slide on the text to be processed according to the preset walk method. The target sliding window can refer to the sliding window after the preset sliding window stops. The difference between the target sliding window and the preset sliding window is only that the corresponding terms in the text to be processed are different.

[0090] Specifically, the text to be processed can be regarded as a vector of size 1*M, where an element in the vector is a term, and the position of the first term can also be represented as (1, 1).

[0091] After the preset sliding window stops moving, the word position corresponding to the first element of the target sliding window can be represented as (1, m). Then the word corresponding to the target sliding window is from the m-th word to the (m+L-1)-th word. It should be noted that if m+L-1 is greater than M, then the word corresponding to the target sliding window is determined to be from the m-th word to the M-th word.

[0092] In this embodiment, the word corresponding to the sliding window in the text to be processed is determined by random walk. This can balance the probability of each word in the text being occluded, thereby obtaining a richer reconstructed text. This improves the label richness of the generation model and can effectively improve the error quality of the erroneous sentence text generated by the subsequently trained generation model.

[0093] Optionally, based on a random walk algorithm, a preset sliding window is moved across the text to be processed to obtain the target sliding window, which includes:

[0094] Obtain the reference position of the first element of the preset sliding window in the text to be processed;

[0095] Based on the preset dictionary, count the number of terms belonging to the non-entity term category among all terms corresponding to the preset sliding window on the text to be processed, and obtain the reference number;

[0096] Based on the reference position and quantity, determine the leftward movement probability, rightward movement probability, and stop probability of the preset sliding window;

[0097] The walking patterns of the preset sliding window are sampled based on the left-shift probability, right-shift probability, and stop probability. The preset sliding window is then moved according to the sampled walking patterns to obtain the target sliding window.

[0098] The preset dictionary may include terms and their corresponding terms categories. In this embodiment, the terms categories include entity terms categories and non-entity terms categories. Non-entity terms are more prone to errors in real-world scenarios and are easier to identify, while entity terms are difficult to identify based on text alone, even if errors occur. Therefore, this embodiment counts the number of terms belonging to the non-entity terms category among all terms corresponding to the preset sliding window on the text to be processed as a reference number.

[0099] Similarly, in a real-world text, the probability of errors occurring at different locations is also different. For example, errors usually do not occur at the beginning of the text. Therefore, the first element of the preset sliding window is the reference position of the corresponding word in the text to be processed.

[0100] Specifically, the left-moving probability, right-moving probability, and stopping probability of the preset sliding window can be obtained through a preset mapping table, which contains the mapping relationship between the reference position and the reference quantity and the left-moving probability, right-moving probability, and stopping probability, respectively.

[0101] The left-shift, right-shift, and stop operations of the sliding window are sampled based on the left-shift probability, right-shift probability, and stop probability, and the preset sliding window is moved according to the sampled operations.

[0102] When the terms corresponding to the moved sliding window are different from those of the original sliding window, the moved sliding window is determined to be a temporary sliding window, and an iterative process is executed. The iterative process includes:

[0103] Obtain the reference position of the first element of the temporary sliding window corresponding to the term in the text to be processed. According to the preset dictionary, count the number of terms belonging to the non-entity term category among all terms corresponding to the temporary sliding window in the text to be processed, and obtain the reference number. Based on the reference position and the number, determine the left-moving probability, right-moving probability, and stopping probability of the temporary sliding window. Sample the walking mode of the temporary sliding window based on the left-moving probability, right-moving probability, and stopping probability. Move the temporary sliding window according to the sampled walking mode to obtain a new temporary sliding window, until the sampled walking mode is stopped, and determine the final temporary sliding window as the target sliding window.

[0104] This embodiment determines the sampling probability of the random walk method based on the prior knowledge of the text error distribution and iteratively moves the sliding window. While preserving the diversity of errors, it makes the error distribution as consistent as possible with the real situation, thereby improving the error quality of the reconstructed text and thus improving the label quality of the generative model. This can effectively improve the error quality of the erroneous sentence text generated by the subsequently trained generative model.

[0105] The above steps involve using a preset sliding window to occlude the text to be processed, inputting the occluded text into a trained reconstruction model for reconstruction, and obtaining the reconstructed text. By generating the reconstructed text through the trained reconstruction model, the reconstructed terms are reconstructed based on the context of the text to be processed, thus making the reconstructed terms more consistent with the error forms in real text, ensuring the error quality of the reconstructed text, and thereby improving the label quality of the generation model. This can effectively improve the error quality of the erroneous sentence text generated by the subsequently trained generation model.

[0106] Step S204: Based on the text to be processed, the amplified text, and the reconstructed text, train the generation model to obtain the trained generation model.

[0107] Among them, the generative model can adopt the autoencoder model, the generator model in the generative adversarial network, etc.

[0108] Optionally, based on the text to be processed, the augmented text, and the reconstructed text, a generative model is trained to obtain the following trained generative models:

[0109] The text to be processed is input into the generation model to obtain the generation result. The similarity of the generation result with the amplified text and the reconstructed text is calculated to obtain the first similarity and the second similarity.

[0110] The larger value between the first similarity and the second similarity is determined, and the result of subtracting the larger value from the preset value is used as the generation loss;

[0111] Based on the generation loss, a generative model is trained to obtain a well-trained generative model.

[0112] The similarity calculation method can include cosine similarity, Euclidean distance, etc. The larger value between the first similarity and the second similarity can characterize the degree of similarity between the generated result and the closest label, that is, the generated result only needs to be sufficiently similar to any label in the label set composed of the amplified text and the reconstructed text.

[0113] This embodiment eliminates the need to label each text to be processed by constructing a tag set, greatly reducing data preparation costs. At the same time, it enables the trained generative model to output erroneous sentences of different error categories for the same input, improving the efficiency of erroneous sentence generation.

[0114] The above steps involve training a generative model based on the text to be processed, the augmented text, and the reconstructed text to obtain a trained generative model. The augmented text and the reconstructed text are used as labels, and the text to be processed is used as samples to train an end-to-end generative model, which enables the trained generative model to greatly improve the efficiency of generating incorrect sentences.

[0115] Step S205: Input the acquired real-time text into the trained generative model to obtain the generated text of incorrect sentences.

[0116] Among them, real-time text can refer to text that needs to be generated for error correction in real time, and error correction text can refer to text containing errors obtained through a trained generation model.

[0117] The above steps, which involve inputting the acquired real-time text into the trained generative model to obtain the generated erroneous sentence text, can directly generate erroneous sentences using the trained generative model. Furthermore, after multiple inputs, different types of erroneous sentence texts can be obtained, greatly improving the richness and efficiency of erroneous sentence generation.

[0118] This embodiment uses multiple data augmentation methods to process the text to be processed, obtaining generative model labels containing rich error types, and trains an end-to-end generative model based on the generative model labels. This allows real-time text input to be directly used with the trained generative model to obtain generated erroneous text without frequently switching data augmentation methods. While ensuring high quality of generated erroneous sentences, it greatly improves the efficiency of erroneous sentence generation.

[0119] See Figure 3 This is a flowchart illustrating a method for generating incorrect sentences for model training provided in Embodiment 2 of the present invention. In this method, the training process of reconstructing the model includes:

[0120] Step S301: Obtain text samples and sample sliding windows, occlude the text samples with the sample sliding windows to obtain occluded samples, and input the occluded samples into the reconstruction model to obtain reconstructed samples;

[0121] Step S302: Occlude the reconstructed samples using a sample sliding window to obtain the occlusion result. Calculate the reconstruction loss based on the occlusion result, the occluded samples, and the preset first loss function.

[0122] Step S303: Calculate the filling loss based on the occluded sample, the reconstructed sample, and the preset second loss function;

[0123] Step S304: Based on the sum of reconstruction loss and filling loss, train the reconstruction model to obtain the trained reconstruction model.

[0124] Here, text sample can refer to text with no errors obtained in advance, sample sliding window can refer to a sliding window of preset length, and occluded sample can refer to text obtained after occluding the text sample using the sample sliding window.

[0125] Reconstructed samples can refer to the output results obtained by inputting occluded samples into a reconstruction model. The reconstruction model can be an autoencoder model, an adversarial generative model, etc.

[0126] The occlusion result can refer to the text obtained after occluding the reconstructed sample using a sample sliding window. The preset first loss function can be used to measure the difference between the occlusion result and the occluded sample. The first loss function can be the mean squared error loss function, the Euclidean distance loss function, etc. The reconstruction loss can be used to measure whether other terms, except for the terms at the occlusion position, are effectively reconstructed.

[0127] The preset second loss function can be used to measure the difference between the occluded sample and the reconstructed sample. For ease of calculation, cosine similarity is used as the second loss function in this embodiment. The filling loss can be used to measure whether the word at the occluded position is different from the word in the text sample after reconstruction.

[0128] Specifically, if a text sample contains A terms, then the text sample can be regarded as a vector of size 1*A, and one element in the vector is a term in the text sample. If the sample window is a vector of size 1*B, then all elements in the sample window are zero elements.

[0129] An initial position is randomly determined from the text sample vector, denoted by coordinates (1, a), where a can be an integer in the range [1, A]. The first element of the sample sliding window vector is aligned with this initial position, that is, the element with coordinates (1, 1) in the sample sliding window vector corresponds to the element with coordinates (1, a) in the text sample vector. Correspondingly, the element with coordinates (1, B) in the sample sliding window vector corresponds to the element with coordinates (1, a+B-1) in the text sample vector.

[0130] Multiply the elements at coordinates (1, a) to (1, a+B-1) in the text sample vector with the zero element, which is also called zeroing, to obtain the occluded sample.

[0131] In this embodiment, the reconstruction model adopts a variational autoencoder model. The occluded sample is input into the reconstruction model to obtain the reconstruction sample, which is also a vector of size 1*A.

[0132] The reconstructed samples are occluded using a sliding window method. This means that the elements from coordinate (1, a) to coordinate (1, a+B-1) in the reconstructed sample vector are all set to zero to obtain the occlusion result.

[0133] Reconstruction loss L r It can be represented as L r =SE(V1) ′ ,2 ′ ), where V1 ′ This could refer to occluded samples, V2 ′ It can refer to the occlusion result, and MSE can refer to the mean square error function.

[0134] Filling loss L f It can be represented as L f = α -1, where α can refer to the cosine similarity between the occluded sample and the reconstructed sample. The cosine similarity ranges from [0, 1]. The closer the cosine similarity is to 1, the more similar the occluded sample and the reconstructed sample are; the closer the cosine similarity is to 0, the less similar the occluded sample and the reconstructed sample are. Since in this embodiment, the reconstruction model is used to generate text with erroneous sentences, that is, the output of the reconstruction model should be different from the input text sample with no erroneous terms. Therefore, the closer the cosine similarity is to 1, the lower the imputation loss L. f The larger the value, the more constrained the reconstruction model will be to output reconstructed terms that differ from the text sample terms at the occluded locations.

[0135] In this embodiment, the reconstruction model is trained using reconstruction loss and filling loss. This enables the trained reconstruction model to generate reconstructed terms that are different from the input terms only for occluded positions, thereby achieving high accuracy in generating incorrect sentences. Compared with the direct replacement method, the trained reconstruction model can consider more contextual information of terms and obtain generated incorrect sentences that are more in line with reality. Using the reconstructed text generated by the trained reconstruction model as the label of the generation model can ensure high quality of generated incorrect sentences.

[0136] Corresponding to the error sentence generation method for model training in the above embodiment, Figure 4 A structural block diagram of the error sentence generation device for model training provided in Embodiment 3 of the present invention is shown. The error sentence generation device is applied to a client. The computer device corresponding to the client connects to the server to obtain the text to be processed and the real-time text. A generation model is deployed within the client. After training, the generation model can be used to generate error sentence text. The text to be processed can be used to provide training samples for the training of the generation model. The real-time text can refer to the text received by the server that needs to be used for error sentence generation. For ease of explanation, only the parts related to the embodiments of the present invention are shown.

[0137] See Figure 4 The error sentence generation device includes:

[0138] The text segmentation module 41 is used to segment the text to be processed, obtain at least two terms, count all terms to obtain the term count, and use a preset mapping function to map the term count to an adjustment quantity;

[0139] The text augmentation module 42 is used to sample all terms N times to obtain N target terms. For any target term, a preset adjustment strategy is used to adjust the target term in the text to be processed to obtain augmented text, where N is the number of adjustments.

[0140] The text reconstruction module 43 is used to occlude the text to be processed using a preset sliding window, and input the occluded text to be processed into the trained reconstruction model for reconstruction to obtain the reconstructed text.

[0141] The model training module 44 is used to train the generative model based on the text to be processed, the augmented text, and the reconstructed text, and obtain the trained generative model.

[0142] The error sentence generation module 45 is used to input the acquired real-time text into the trained generation model to obtain the generated error sentence text.

[0143] Optionally, the text augmentation module 42 mentioned above includes:

[0144] The sequence number allocation unit is used to obtain the position of any term in the text to be processed, and assign a sequence number to the term according to the position to obtain a sequence number sequence, which includes M sequence numbers, where M is the number of terms;

[0145] The random number generation unit is used to generate N random numbers in the sequence of numbers to obtain N generated numbers, and to determine the N terms corresponding to the generated numbers as N target terms.

[0146] Optionally, the text augmentation module 42 mentioned above includes:

[0147] The strategy sampling unit is used to sample a preset set of adjustment strategies for any target term and obtain the sampling result.

[0148] The term adjustment unit is used to adjust the target terms in the text to be processed using the sampling results, so as to obtain the augmented text.

[0149] Optionally, the set of adjustment strategies includes deletion strategies, replacement strategies, and insertion strategies;

[0150] The above-mentioned strategy sampling unit includes:

[0151] The probability sampling subunit is used to perform equal-probability sampling on deletion, replacement, and insertion strategies to obtain sampling results.

[0152] Accordingly, the aforementioned term adjustment units include:

[0153] The first amplification subunit is used to delete the target term in the text to be processed if the sampling result is a deletion strategy, so as to obtain the amplified text;

[0154] The second amplification subunit is used to replace the target term with a preset first term in the text to be processed if the sampling result is a replacement strategy, so as to obtain the amplified text;

[0155] The third amplification subunit is used to insert a preset second term after the target term in the text to be processed if the sampling result is an insertion strategy, so as to obtain the amplified text.

[0156] Optionally, the text reconstruction module 43 mentioned above includes:

[0157] The random walk unit is used to align the first element of the preset sliding window with the first word of the text to be processed. According to the random walk algorithm, the preset sliding window is moved on the text to be processed to obtain the target sliding window.

[0158] The term occlusion unit is used to multiply the target sliding window by its corresponding term in the text to be processed, so as to obtain the occluded text.

[0159] Optionally, the above-mentioned random walk unit includes:

[0160] The position acquisition word unit is used to obtain the reference position of the first element of the preset sliding window and the corresponding word in the text to be processed;

[0161] The quantity statistics subunit is used to count the number of terms belonging to the non-entity term category among all terms corresponding to the preset sliding window on the text to be processed, based on the preset dictionary, and to obtain the reference quantity;

[0162] The probability determination subunit is used to determine the leftward movement probability, rightward movement probability, and stopping probability of the preset sliding window based on the reference position and quantity.

[0163] The sliding window movement subunit is used to sample the walking mode of the preset sliding window based on the left-moving probability, right-moving probability, and stopping probability, and move the preset sliding window according to the sampled walking mode to obtain the target sliding window.

[0164] Optionally, the above model training module 44 includes:

[0165] The similarity calculation unit is used to input the text to be processed into the generation model, obtain the generation result, and calculate the similarity between the generation result and the amplified text and the reconstructed text respectively to obtain the first similarity and the second similarity.

[0166] The loss calculation unit is used to determine the larger value between the first similarity and the second similarity, and uses the difference between the preset value and the larger value as the generation loss;

[0167] The iterative training unit is used to train the generative model based on the generation loss to obtain a trained generative model.

[0168] It should be noted that the information interaction and execution process between the above modules, units, and sub-units are based on the same concept as the method embodiments of the present invention. For details on their specific functions and technical effects, please refer to the method embodiments section, which will not be repeated here.

[0169] Figure 5 This is a schematic diagram of the structure of a computer device provided in Embodiment 4 of the present invention. Figure 5 As shown, the computer device of this embodiment includes: at least one processor ( Figure 5 Only one is shown in the diagram), a memory, and a computer program stored in the memory and capable of running on at least one processor, wherein the processor executes the computer program to implement the steps in any of the above-described error generation method embodiments.

[0170] This computer device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that... Figure 5The examples of computer devices are merely examples and do not constitute a limitation on computer devices. Computer devices may include more or fewer components than shown in the illustration, or combinations of certain components, or different components, such as network interfaces, displays, and input devices.

[0171] The processor referred to can be a CPU, but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0172] Memory includes readable storage media, internal memory, etc., wherein internal memory can be the RAM of a computer device, providing an environment for the operation of the operating system and computer-readable instructions stored in the readable storage media. The readable storage media can be the hard drive of a computer device, or in other embodiments, it can be an external storage device of the computer device, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, memory can include both internal storage units and external storage devices of a computer device. Memory is used to store the operating system, applications, bootloader, data, and other programs, such as program code for computer programs. Memory can also be used to temporarily store data that has been output or will be output.

[0173] Those skilled in the art will understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the functions described above can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this invention. The specific working process of the units and modules in the above device can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here. If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention can implement all or part of the processes in the methods of the above embodiments by instructing related hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the above method embodiments. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. A computer-readable medium can include at least: any entity or device capable of carrying computer program code, a recording medium, a computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0174] The present invention can implement all or part of the processes in the methods of the above embodiments, or it can be accomplished by a computer program product. When the computer program product is run on a computer device, the computer device executes the steps in the above method embodiments.

[0175] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0176] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0177] In the embodiments provided by this invention, it should be understood that the disclosed apparatus / computer devices and methods can be implemented in other ways. For example, the apparatus / computer device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0178] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0179] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for generating incorrect sentences for model training, characterized in that, The method for generating incorrect sentences includes: The text to be processed is segmented into words to obtain at least two terms. All terms are counted to obtain the number of terms. The number of terms is then mapped to an adjustment quantity using a preset mapping function. All terms are sampled N times to obtain N target terms. For any target term, a preset adjustment strategy is used to adjust the target term in the text to be processed to obtain augmented text. N is the number of adjustments. The text to be processed is occluded by a preset sliding window, and the occluded text is input into a trained reconstruction model for reconstruction to obtain the reconstructed text. Based on the text to be processed, the augmented text, and the reconstructed text, a generation model is trained to obtain a trained generation model. The acquired real-time text is input into the trained generation model to obtain the generated text of incorrect sentences; The step of using a preset sliding window to obscure the text to be processed includes: Align the first element of the preset sliding window with the first term of the text to be processed, and move the preset sliding window on the text to be processed according to the random walk algorithm to obtain the target sliding window; Multiply the target sliding window by its corresponding term on the text to be processed to obtain the occluded text; The first element of the preset sliding window refers to the first element of the preset sliding window. The random walk algorithm refers to making the preset sliding window slide on the text to be processed according to a preset walk method. The target sliding window refers to the sliding window after the preset sliding window stops.

2. The method for generating incorrect sentences according to claim 1, characterized in that, The process of sampling all terms N times to obtain N target terms includes: For any term, the position of the term in the text to be processed is obtained, and a serial number is assigned to the term according to the position to obtain a serial number sequence. The serial number sequence includes M serial numbers, where M is the number of terms. In the sequence of numbers, N random numbers are generated to obtain N generated numbers, and the N terms corresponding to the generated numbers are determined as the N target terms.

3. The method for generating incorrect sentences according to claim 1, characterized in that, For any target term, a preset adjustment strategy is used to adjust the target term in the text to be processed, resulting in augmented text, including: For any target term, a preset set of adjustment strategies is sampled to obtain the sampling results; The target terms are adjusted in the text to be processed using the sampling results to obtain the amplified text.

4. The method for generating incorrect sentences according to claim 3, characterized in that, The set of adjustment strategies includes deletion strategies, replacement strategies, and insertion strategies; The sampling of the preset set of adjustment strategies to obtain the sampled adjustment strategies includes: The deletion strategy, the replacement strategy, and the insertion strategy are sampled with equal probability to obtain the sampling results; Accordingly, the step of adjusting the target terms in the text to be processed using the sampling results to obtain the amplified text includes: If the sampling result is the deletion strategy, the target term is deleted from the text to be processed to obtain the amplified text; If the sampling result is the replacement strategy, the target term is replaced with a preset first term in the text to be processed to obtain the augmented text; If the sampling result is the insertion strategy, a preset second term is inserted into the text to be processed after the target term to obtain the amplified text.

5. The method for generating incorrect sentences according to claim 1, characterized in that, The step of moving the preset sliding window over the text to be processed according to the random walk algorithm to obtain the target sliding window includes: Obtain the reference position of the first element of the preset sliding window corresponding to the word in the text to be processed; Based on a preset dictionary, the number of terms belonging to the non-entity term category among all terms corresponding to the preset sliding window on the text to be processed is counted to obtain a reference number; Based on the reference position and the quantity, determine the leftward movement probability, rightward movement probability, and stop probability of the preset sliding window; The walking pattern of the preset sliding window is sampled based on the left-shift probability, the right-shift probability, and the stop probability. The preset sliding window is then moved according to the sampled walking pattern to obtain the target sliding window.

6. The method for generating incorrect sentences according to any one of claims 1 to 5, characterized in that, The step of training the generation model based on the text to be processed, the augmented text, and the reconstructed text to obtain the trained generation model includes: The text to be processed is input into the generation model to obtain the generation result. The similarity of the generation result with the amplified text and the reconstructed text is calculated to obtain the first similarity and the second similarity. The larger of the first similarity and the second similarity is determined, and the result of subtracting the larger value from the preset value is used as the generation loss; The generation model is trained based on the generation loss to obtain the trained generation model.

7. A device for generating incorrect sentences for model training, characterized in that, The error sentence generation device includes: The text segmentation module is used to segment the text to be processed, obtain at least two terms, count all terms to obtain the term count, and use a preset mapping function to map the term count to an adjustment quantity; The text augmentation module is used to sample all terms N times to obtain N target terms. For any target term, a preset adjustment strategy is used to adjust the target term in the text to be processed to obtain augmented text, where N is the number of adjustments. The text reconstruction module is used to occlude the text to be processed using a preset sliding window, and then input the occluded text to be processed into a trained reconstruction model for reconstruction to obtain the reconstructed text. The model training module is used to train a generative model based on the text to be processed, the augmented text, and the reconstructed text, and obtain a trained generative model. The error sentence generation module is used to input the acquired real-time text into the trained generation model to obtain the generated error sentence text; The text reconstruction module includes: The random walk unit is used to align the first element of the preset sliding window with the first word of the text to be processed. According to the random walk algorithm, the preset sliding window is moved on the text to be processed to obtain the target sliding window. The term occlusion unit is used to multiply the target sliding window by its corresponding term in the text to be processed to obtain the occluded text; The first element of the preset sliding window refers to the first element of the preset sliding window. The random walk algorithm refers to making the preset sliding window slide on the text to be processed according to a preset walk method. The target sliding window refers to the sliding window after the preset sliding window stops.

8. A computer device, characterized in that, The computer device includes a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the error sentence generation method as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the error sentence generation method as described in any one of claims 1 to 6.

Images