Parallel corpus set construction method and device and text conversion model training method and device
By automatically building parallel corpus sets, the problem of high cost and low accuracy of corpus labeling in the prior art is solved, and efficient and accurate text conversion model training is achieved.
Patent Information
- Application Number
- CN202411104037.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2023-08-11
- Filing Date
- 2024-08-12
- Publication Date
- 2024-11-19
AI Technical Summary
In existing text conversion technology, in order to ensure the accuracy of the model, massive corpus pairs need to be marked in advance, resulting in high labeling costs and low accuracy.
By obtaining the initial label-free corpus, using pre-trained grammar models, multi-label classification models and sequence annotation models, parallel corpus sets are automatically constructed, reducing the cost of corpus annotation and improving accuracy.
It realizes automatic construction of parallel corpus sets, reduces the cost of corpus labeling, avoids errors caused by manual annotation, improves the accuracy of corpus labeling, and improves the performance of text conversion models.
Smart Images

Figure CN118982004A_ABST
Abstract
Description
[0001] This application claims the priority of the Chinese patent application filed with the China Patent Office on August 11, 2023, with application number "202311014588.1" and invention name "Parallel corpus construction method, text conversion model training method and device", all contents of which are recorded in this application by reference. Technical Field
[0002] The present application relates to the field of text conversion technology, and in particular to a parallel corpus construction method, a text conversion model training method, and a text conversion method. The present application also relates to a parallel corpus construction device, a text conversion model training device, a text conversion device, a computing device, and a computer-readable storage medium. Background Art
[0003] With the development of Internet technology and Natural Language Processing (NLP) technology, people participate in online activities more and more frequently, for example, people go shopping and seek medical treatment online. Online activities often involve text processing. In order to improve the effect of human-computer interaction, it is generally necessary to convert the text input by the user into a text with a unified text structure that is more in line with user habits, so as to facilitate further text processing.
[0004] Currently, text conversion usually adopts a neural network-based method. Using a parallel corpus, a preset neural network is trained to obtain a text conversion model. The text to be converted is input into the trained text conversion model to obtain the converted text. In order to ensure the accuracy of the text conversion model, the parallel corpus needs to include a large number of pre-annotated corpus pairs.
[0005] In the above method, in order to ensure the accuracy of the text conversion model, a large number of corpus pairs need to be annotated in advance, and the annotation of the corpus pairs is generally done manually, which leads to excessively high annotation costs and low annotation accuracy. Summary of the invention
[0006] In view of this, the embodiments of the present application provide a parallel corpus construction method, a text conversion model training method and a text conversion method to solve the technical defects existing in the prior art. The embodiments of the present application also provide a parallel corpus construction device, a text conversion model training device, a text conversion device, a computing device, and a computer-readable storage medium.
[0007] According to a first aspect of an embodiment of the present application, a method for constructing a parallel corpus is provided, comprising:
[0008] Obtaining an initial corpus set, wherein the initial corpus set includes a plurality of unlabeled initial corpora;
[0009] Inputting each initial corpus into a pre-trained grammar model to obtain a grammar recognition result of each initial corpus, and selecting first corpus with a correct grammar recognition result from the initial corpus to form a first corpus set;
[0010] Input each first corpus into a pre-trained multi-label classification model to obtain a classification label of each first corpus, and delete the corpus whose classification label is a non-modification class label from the first corpus to obtain a second corpus;
[0011] Using the pre-trained sequence labeling model, perform sequence labeling on each corpus in the second corpus to obtain a third corpus;
[0012] According to the sequence annotation label of each corpus in the third corpus, each corpus in the third corpus is processed to obtain a fourth corpus;
[0013] Based on the second corpus and the fourth corpus, a parallel corpus is constructed.
[0014] According to a second aspect of an embodiment of the present application, a text conversion model training method is provided, comprising:
[0015] Obtaining a parallel corpus, wherein the parallel corpus is constructed using the parallel corpus construction method according to the first aspect of the embodiment of the present application;
[0016] Based on the parallel corpus, the preset neural network is trained to obtain the text conversion model.
[0017] According to a third aspect of an embodiment of the present application, a text conversion method is provided, comprising:
[0018] Get the text to be converted;
[0019] The text to be converted is input into a text conversion model to obtain the converted text, wherein the text conversion model is trained according to the text conversion model training method according to the second aspect of the embodiment of the present application.
[0020] According to a fourth aspect of an embodiment of the present application, there is provided a parallel corpus construction device, comprising:
[0021] A first acquisition module is configured to acquire an initial corpus set, wherein the initial corpus set includes a plurality of unlabeled initial corpora;
[0022] A screening module is configured to input each initial corpus into a pre-trained grammar model, obtain a grammar recognition result of each initial corpus, and screen out first corpus with a correct grammar recognition result from the initial corpus set to form a first corpus set;
[0023] A deletion module is configured to input each first corpus into a pre-trained multi-label classification model to obtain a classification label of each first corpus, and delete the corpus whose classification label is a non-modification class label from the first corpus set to obtain a second corpus set;
[0024] A sequence labeling module is configured to use a pre-trained sequence labeling model to perform sequence labeling on each corpus in the second corpus to obtain a third corpus;
[0025] a processing module configured to process each corpus in the third corpus according to the sequence annotation label of each corpus in the third corpus to obtain a fourth corpus;
[0026] The construction module is configured to construct a parallel corpus based on the second corpus and the fourth corpus.
[0027] According to a fifth aspect of an embodiment of the present application, a text conversion model training device is provided, comprising:
[0028] A second acquisition module is configured to acquire a parallel corpus, wherein the parallel corpus is constructed using the parallel corpus construction method of the first aspect of the embodiment of the present application;
[0029] The training module is configured to train a preset neural network based on a parallel corpus to obtain a text conversion model.
[0030] According to a sixth aspect of an embodiment of the present application, a text conversion device is provided, including:
[0031] A third acquisition module is configured to acquire the text to be converted;
[0032] The conversion module is configured to input the text to be converted into a text conversion model to obtain the converted text, wherein the text conversion model is trained according to the text conversion model training method of the second aspect of the embodiment of the present application.
[0033] According to a seventh aspect of an embodiment of the present application, there is provided a computing device, including:
[0034] Memory and processor;
[0035] The memory is used to store computer-executable instructions, and when the processor executes the computer-executable instructions, it can implement the parallel corpus construction method of the first aspect of the embodiment of the present application, or the text conversion model training method of the second aspect of the embodiment of the present application, or the text conversion method of the third aspect of the embodiment of the present application.
[0036] According to a fourth aspect of the embodiments of the present application, a computer-readable storage medium is provided, which stores computer-executable instructions. When the instructions are executed by a processor, the parallel corpus construction method of the first aspect of the embodiments of the present application, or the text conversion model training method of the second aspect of the embodiments of the present application, or the text conversion method of the third aspect of the embodiments of the present application is implemented.
[0037] According to the fifth aspect of the embodiments of the present application, a chip is provided, which stores a computer program. When the computer program is executed by the chip, it implements the parallel corpus construction method of the first aspect of the embodiments of the present application, or the text conversion model training method of the second aspect of the embodiments of the present application, or the text conversion method of the third aspect of the embodiments of the present application.
[0038] The present application provides a parallel corpus construction method, a text conversion model training method and a device, wherein, in the parallel corpus construction method, by inputting the acquired initial corpus into a pre-trained grammar model, a first corpus consisting of grammatically correct corpora is screened, and then the first corpus is input into a pre-trained multi-label classification model, and the corpus of the unmodified type is deleted to obtain a second corpus, and then the pre-trained sequence annotation model is used to perform sequence annotation on the second corpus to obtain a third corpus, and the corpus in the third corpus is processed according to the sequence annotation label of each corpus in the third corpus to obtain a fourth corpus. The corpus in the second corpus can be used as the original corpus, and the corpus in the fourth corpus can be used as the target corpus. Based on the second corpus and the fourth corpus, a parallel corpus can be constructed. Through the embodiment of the present application, a parallel corpus can be automatically constructed using a neural network-based method, without the need for pre-annotation of corpus, which reduces the cost of corpus annotation, and the automatic acquisition of a parallel corpus can avoid the errors caused by manual annotation, thereby improving the accuracy of corpus annotation. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] Figure 1 is a structural diagram of a text conversion system provided by an embodiment of the present application;
[0040] Figure 2 is a flow chart of a method for constructing a parallel corpus provided in one embodiment of the present application;
[0041] Figure 3 It is a flow chart of a method for training a grammar model in a method for constructing a parallel corpus provided in one embodiment of the present application;
[0042] Figure 4 It is a flow chart of a training method for a multi-label classification model in a parallel corpus construction method provided in one embodiment of the present application;
[0043] Figure 5It is a flow chart of a training method for a sequence annotation model in a parallel corpus construction method provided in one embodiment of the present application;
[0044] Figure 6 It is a flow chart of a training method for a multi-label classification model and a sequence annotation model in another parallel corpus construction method provided in one embodiment of the present application;
[0045] Figure 7 It is a flow chart of a method for updating a training set in a parallel corpus construction method provided in an embodiment of the present application;
[0046] Figure 8 is a flow chart of a method for corpus processing in a parallel corpus construction method provided in an embodiment of the present application;
[0047] Fig. 9 It is a flow chart of a training method for a character addition model in a parallel corpus construction method provided in an embodiment of the present application;
[0048] Fig.10 is a flowchart of a text conversion model training method provided by an embodiment of the present application;
[0049] Fig.11 is a flowchart of a text conversion method provided by an embodiment of the present application;
[0050] Fig.12 is a processing flow chart of a text conversion method provided by an embodiment of the present application;
[0051] Fig.13 It is a structural schematic diagram of a parallel corpus construction device provided in one embodiment of the present application;
[0052] Fig.14 It is a structural schematic diagram of a text conversion model training device provided by an embodiment of the present application;
[0053] Fig.15 is a structural schematic diagram of a text conversion device provided by an embodiment of the present application;
[0054] Fig.16 It is a structural block diagram of a computing device provided in one embodiment of the present application. DETAILED DESCRIPTION
[0055] Many specific details are described in the following description to facilitate a full understanding of the present application. However, the present application can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of the present application, so the present application is not limited by the specific implementation disclosed below.
[0056] The terms used in one or more embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of the present application. The singular forms of "a", "said" and "the" used in one or more embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of the present application refers to and includes any or all possible combinations of one or more associated listed items.
[0057] It should be understood that, although the terms first, second, etc. may be used to describe various information in one or more embodiments of the present application, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of the present application, the first may also be referred to as the second, and similarly, the second may also be referred to as the first.
[0058] First, the terms involved in one or more embodiments of the present application are explained.
[0059] Parallel corpus: different text corpora of a language that can be converted into each other, for example, simplified and traditional Chinese, spoken and written language, text corpora that are translations of each other, etc.
[0060] Neural Machine Translation (NMT): End-to-end neural machine translation based on encoder-decoder is the current mainstream research direction of machine translation. It uses the encoder to encode the text to be translated into a fixed-length semantic representation, and the decoder uses the semantic representation to generate the corresponding translation text word by word.
[0061] The BERT (Bidirectional Encoder Representation from Transformers) model is a pre-trained language representation model. It emphasizes that it no longer uses the traditional unidirectional language model or the shallow concatenation of two unidirectional language models for pre-training as in the past, but instead uses a new Masked Language Model (MLM) to generate deep bidirectional language representations.
[0062] RoBERTa model: is an improved version of BERT, which achieves state-of-the-art results by improving training tasks and data generation methods, training longer, using larger batches, and using more data; it can be loaded directly with BERT.
[0063] F1-Score: It is a criterion that integrates the model's precision and recall. It is the harmonic mean of the precision and recall predicted by the language representation model.
[0064] Precision: refers to the proportion of positive samples among the positive samples determined by the model prediction classifier.
[0065] Recall: refers to the proportion of positive examples predicted to be positive.
[0066] Positive sample (TP, True Positive): The sample whose predicted answer is correct.
[0067] BLEU (Bilingual Evaluation Understudy) value: is an evaluation method and evaluation value used for machine-implemented text tasks. It can be used to evaluate the accuracy of characters or words, and can also be used to further evaluate the fluency of text conversion results.
[0068] In the present application, a parallel corpus construction method, a text conversion model training method and a text conversion method are provided. The present application also relates to a parallel corpus construction device, a text conversion model training device, a text conversion device, a computing device, and a computer-readable storage medium, which are described in detail one by one in the following embodiments.
[0069] Figure 1 A structural schematic diagram of a text conversion system provided by an embodiment of the present application is shown. The execution subject of the parallel corpus construction method provided by the embodiment of the present application is a server 101 having a function of constructing a parallel corpus; the execution subject of the text conversion model training method provided by the embodiment of the present application can be a training terminal 102, and the training terminal 102 can specifically be a training server or a terminal with a model training function; the execution subject of the text conversion method provided by the embodiment of the present application is a terminal 103 with a text conversion function. The above-mentioned terminal can be any electronic product that can perform human-computer interaction with a user, such as a PC (Personal Computer), a mobile phone, a Pocket PC, a tablet computer, etc.
[0070] The server 101, the training terminal 102 and the terminal 103 in the text conversion system can be integrated into one device or can be independent devices. In the case of independent devices, the server 101 and the training terminal 102 can be a server, a server cluster composed of multiple servers, or a cloud computing service center, which is not limited in the embodiments of the present application.
[0071] The server 101 with the function of constructing a parallel corpus inputs the acquired initial corpus into a pre-trained grammar model, screens out a first corpus consisting of grammatically correct corpora, then inputs the first corpus into a pre-trained multi-label classification model, deletes the corpora of unmodified types, and obtains a second corpus, then uses a pre-trained sequence annotation model to perform sequence annotation on the second corpus to obtain a third corpus, and processes each corpus in the third corpus according to the sequence annotation label of each corpus in the third corpus to obtain a fourth corpus. The corpus in the second corpus can be used as the original corpus, and the corpus in the fourth corpus can be used as the target corpus, then based on the second corpus and the fourth corpus, a parallel corpus can be constructed.
[0072] The training terminal 102 can obtain a parallel corpus constructed by a server having a function of constructing a parallel corpus, and train a preset neural network based on the parallel corpus to obtain a text conversion model. The terminal 103 receives the text conversion model trained by the training server, and after obtaining the text to be converted, inputs the text to be converted into the text conversion model to obtain the converted text.
[0073] The specific parallel corpus construction method, text conversion model training method and text conversion method will be described in detail in the subsequent embodiments.
[0074] Figure 2 A flowchart of a parallel corpus construction method provided in an embodiment of the present application is shown, which specifically includes the following steps:
[0075] Step 202: Obtain an initial corpus, wherein the initial corpus includes a plurality of unlabeled initial corpora.
[0076] The initial corpus is a corpus composed of multiple initial corpora. The initial corpus is unlabeled, that is, the initial corpus refers to the corpus of unknown quality, and there is no need to annotate these corpora in advance. The initial corpus can be chat records on chat software, article messages on news media, etc., or it can be randomly obtained from other open source corpora. Among them, open source corpora store a huge amount of corpora. The current mainstream open source corpora include the United Nations Corpus, the TED Corpus (a multilingual parallel corpus, including multilingual parallel corpora and monolingual corpora), etc.
[0077] Step 204: Input each initial corpus into the pre-trained grammar model to obtain the grammar recognition result of each initial corpus, and select the first corpus with correct grammar recognition result from the initial corpus set to form a first corpus set.
[0078] The grammar model is a pre-trained neural network model that can identify whether the grammar of the corpus is correct. Whether the grammar of the corpus is correct means whether the grammar of the corpus conforms to the normal grammatical structure. The normal grammatical structure includes subject-verb-object structure, inversion structure, imperative sentence structure, past tense, present tense, and future tense structure in English, etc.
[0079] The grammar model is trained based on sample corpus. The sample corpus is a corpus that carries grammar labels indicating whether the grammar is correct or not. The sample corpus with grammar correct labels is used as positive samples, and the sample corpus with grammar incorrect labels is used as negative samples. The grammar model can be trained by training a binary classification neural network model (the neural network model can be a language representation model, such as the RoBERTa model). The grammar model can directly perform grammar recognition on the input corpus to obtain the result of whether the grammar of the corpus is correct or not. The specific training process of the grammar model will be described in Figure 3 This is described in detail in the illustrated embodiment and will not be repeated here.
[0080] By inputting each initial corpus in the initial corpus into the trained grammar model, the grammar recognition results of each initial corpus can be obtained, that is, whether the grammar of each initial corpus is correct can be identified. For initial corpus with grammatical errors, it does not conform to traditional grammatical habits. The original grammatical errors may affect the accuracy of subsequent text conversion. Therefore, it is not easy to be used as the original corpus in the parallel corpus. It is necessary to screen out the first corpus with correct grammar recognition results from the initial corpus to form the first corpus. For example, the initial corpus obtained includes "I want to go to school", which obviously does not conform to the normal subject-predicate-object grammatical structure. The probability of such a sentence appearing in real applications is also extremely low. In order to be more in line with the actual situation, such initial corpus can be screened out.
[0081] Step 206: Input each first corpus into a pre-trained multi-label classification model to obtain a classification label of each first corpus, delete the corpus with a classification label of no modification class label from the first corpus set, and obtain a second corpus set.
[0082] The multi-label classification model is a pre-trained multi-classification neural network model that can classify corpora. Corpora can be divided into two categories: unmodified and modified. The modified category can be further divided into multiple subcategories such as character addition, character duplication, and character deletion.
[0083] The multi-label classification model is trained based on sample corpus. The sample corpus is a corpus carrying corpus category labels. The unmodified sample corpus carries an unmodified label (for example, marked with 000), the sample corpus with added characters carries an added character label (for example, marked with 001), the sample corpus with repeated characters carries a repeated character label (for example, marked with 010), and the sample corpus with deleted characters carries a deleted character label (for example, marked with 100). Of course, for the sample corpus divided into two categories, the unmodified and modified categories, the sample corpus with the unmodified label can be labeled with 0, and the sample corpus with the modified label can be labeled with 1. Using the above sample corpus, by training a multi-classification neural network model (the neural network model can be a language representation model, such as a RoBERTa model), a multi-label classification model can be trained. The multi-label classification model can directly classify the input corpus to obtain the classification label of the corpus. The specific training process of the multi-label classification model will be described in. Figure 4 This is described in detail in the illustrated embodiment and will not be repeated here.
[0084] Although the corpora in the first corpus are all grammatically correct corpora, not all corpora need to be used as the original corpora in the parallel corpus. Generally, in order to realize text conversion normally, it is necessary to delete the corpora with the classification label of no modification class label from the first corpus, and then convert the corpora with modification class label. In an embodiment of the present application, text conversion can be the conversion of spoken corpus and written corpus, the spoken corpus is the original corpus, and the written corpus is the target corpus. Compared with the written corpus, there are generally phenomena such as character omission, character duplication, and character increase in the spoken corpus. In order to realize the conversion from spoken corpus to written corpus, the embodiment of the present application needs to delete the corpus with the classification label of no modification class label from the first corpus, and retain the corpus with the classification label of modification class label (i.e., the spoken corpus with phenomena such as character omission, character duplication, and character increase).
[0085] By inputting each first corpus in the first corpus set into the trained multi-label classification model respectively, the classification label of each first corpus can be obtained, that is, the category of each first corpus is identified, and the corpus with the classification label of no modification class label is deleted. The remaining corpus with the classification label of modification class label constitutes the second corpus set.
[0086] Step 208: Use the pre-trained sequence labeling model to perform sequence labeling on each corpus in the second corpus to obtain a third corpus.
[0087] The sequence labeling model is a pre-trained neural network model that can perform sequence labeling on corpus. Corpus consists of various characters (words, words). For corpus with modifications, in order to more accurately record and identify the modifications of the corpus, each character can be sequence labeled according to the actual modification situation. For example, the newly added characters can be labeled as D, and the deleted characters can be labeled as I.
[0088] The sequence labeling model is trained based on sample corpus. The sample corpus is a corpus with sequence labels. The original character can be labeled as "O", the newly added character can be labeled as "D", and the next character of the deleted character can be labeled as "I". Using the above sample corpus, by training a preset neural network model (the neural network model can be a language representation model, such as the RoBERTa model), a sequence labeling model can be trained. The sequence labeling model can directly perform sequence labeling on the input corpus. The specific training process of the sequence labeling model will be described in Figure 5 This is described in detail in the illustrated embodiment and will not be repeated here.
[0089] In order to facilitate the subsequent corpus conversion processing, it is necessary to use the trained sequence labeling model to perform sequence labeling on each corpus in the second corpus, that is, to mark which characters in each corpus are unchanged and which characters are added or deleted, so as to obtain the third corpus.
[0090] For example, the corpus “The weather is nice today” is annotated, and the result is [O,O,O,O,O,O,O]; after adding characters, the corpus “The weather is nice today” is annotated, and the result is [O,O,D,D,O,O,O,O,O]; after adding repeated characters, the corpus “The weather is nice today” is annotated, and the result is [O,D,D,O,O,O,O,O,O]; after deleting characters, the corpus “The air is nice today” is annotated, and the result is [O,I,O,O,O,O].
[0091] When training the grammar model, multi-label classification model, and sequence labeling model, the preset training set used can be the same training set or different training sets, and they can also be included in each other. For example, the training set for training the grammar model includes the training sets for training the multi-label classification model and the sequence labeling model.
[0092] Step 210: Process each corpus in the third corpus according to the sequence annotation label of each corpus in the third corpus to obtain a fourth corpus.
[0093] In step 208, sequence annotation is performed on the second corpus to obtain a third corpus after sequence annotation. In step 210, each corpus in the third corpus is processed corresponding to the sequence annotation label to obtain a fourth corpus. For example, if a text sequence of a corpus in the third corpus is annotated with a character increase class label D, the character or word annotated with the character increase class label D is deleted. After all corpuses in the third corpus are processed accordingly, a fourth corpus is obtained.
[0094] Step 212: construct a parallel corpus based on the second corpus and the fourth corpus.
[0095] In step 210, the third corpus is processed to obtain a fourth corpus. In step 212, each corpus in the second corpus and each corresponding corpus in the fourth corpus are constructed into a corpus pair, and the set constructed by the corpus pair is determined as a parallel corpus. There is a one-to-one correspondence between the corpuses in the parallel corpus. The corpus in the second corpus can be used as the original corpus, and the corresponding corpus in the fourth corpus can be used as the target corpus, based on which a one-to-one correspondence relationship is established.
[0096] In an implementation of the embodiment of the present application, step 212 can be specifically implemented in the following manner: the corpus in the second corpus set is matched one-to-one with the corpus in the fourth corpus set to obtain a parallel corpus set.
[0097] Specifically, the features of the corpus in the second corpus set are compared with each corpus in the fourth corpus set, and the corpus with the highest degree of correspondence is determined as a corpus pair, that is, the corpus in the second corpus set corresponds one-to-one with the corpus in the fourth corpus set, to obtain a parallel corpus set.
[0098] By using the embodiment of the present application, the acquired initial corpus is input into a pre-trained grammar model, and a first corpus consisting of grammatically correct corpora is screened, and then the first corpus is input into a pre-trained multi-label classification model, and the corpus of the unmodified type is deleted to obtain a second corpus, and then the pre-trained sequence annotation model is used to perform sequence annotation on the second corpus to obtain a third corpus, and each corpus in the third corpus is processed according to the sequence annotation label of each corpus in the third corpus to obtain a fourth corpus. The corpus in the second corpus can be used as the original corpus, and the corpus in the fourth corpus can be used as the target corpus. Based on the second corpus and the fourth corpus, a parallel corpus can be constructed. Through the embodiment of the present application, a parallel corpus can be automatically constructed using a neural network-based method, without the need for pre-annotation of corpus, which reduces the cost of corpus annotation, and the automatic acquisition of a parallel corpus can avoid the errors caused by manual annotation, thereby improving the accuracy of corpus annotation.
[0099] Figure 3A flowchart of a method for training a grammar model in a parallel corpus construction method provided in an embodiment of the present application is shown, which specifically includes the following steps:
[0100] Step 302: Obtain a preset training set, where the preset training set includes a plurality of sample corpora, and the plurality of sample corpora include unmodified sample corpora and modified sample corpora.
[0101] The preset training set is a training set composed of multiple sample corpora. The sample corpora carry grammatical labels indicating whether the grammar is correct. The sample corpora can be chat records on chat software, article messages on news media, etc., or can be randomly obtained from other open source corpora.
[0102] The sample corpus can be divided into unmodified sample corpus and modified sample corpus. Optionally, the unmodified sample corpus is a negative sample corpus without grammatical errors correction, and the modified corpus is a positive sample corpus with grammatical errors correction. For example, the input corpus "did you eat?" is corrected for grammatical errors, and the corpus "did you eat?" is obtained. Correcting grammatical errors is generally a function of intelligent correction software developed by developers for grammatical errors. Correcting grammatical errors can also be manual correction of grammatical errors, which is not limited here.
[0103] Optionally, the unmodified sample corpus may be a positive sample corpus with correct grammar without modification, and the modified corpus may be a negative sample corpus with modified grammar errors. For example, the input corpus "did you eat?" is changed to "did you eat?" after making an error change to the grammar. The changed grammar may be random, that is, the position of the expected words may be randomly adjusted to achieve an error change in the grammar.
[0104] Step 304: Use multiple sample corpora in a preset training set to train a first preset neural network to obtain a grammar model, wherein the first preset neural network is a neural network with a grammar recognition function.
[0105] The first preset neural network is a neural network with a grammar recognition function, for example, a BERT model, a Transformer model, a RoBERTa model, etc. The first preset neural network model used in the embodiment of the present application is a binary classification neural network model, that is, a neural network model that outputs only 0 (indicating correct grammar) and 1 (indicating grammar errors).
[0106] Specifically, the obtained preset training set is input into the first preset neural network, the neural network is trained, the training is terminated when a preset training termination condition is met, and the neural network is determined to be a grammar model.
[0107] Exemplarily, the specific training process of the grammar model may be: extracting a sample corpus from a preset training set, inputting the sample corpus into a first preset neural network (e.g., a RoBERTa model with a grammar recognition function), and identifying the grammar recognition result of the sample corpus after operation by the first preset neural network. Since the sample corpus carries a grammar label indicating whether the grammar is accurate, the grammar recognition result is compared with the grammar label of the sample corpus to obtain a loss value between the two. The network parameters of the first preset neural network may be adjusted based on the loss value, thus completing one training iteration. Then, a sample corpus is extracted from the preset training set again, and training iterations are continued according to the above steps until the calculated loss value is less than a preset threshold, or the number of training iterations reaches a preset number. In this case, it can be determined that the training is completed, and the first preset neural network obtained at the end of the training is the grammar model.
[0108] By applying the embodiments of the present application, the first preset neural network is trained with a preset training set carrying labels to obtain a grammatical model, thereby obtaining a more accurate first sample set, and further constructing a more accurate parallel corpus. In addition, the construction of a parallel sample set using the pre-trained grammatical model can be completed automatically, which reduces costs and improves efficiency.
[0109] Figure 4 A flowchart of a method for training a multi-label classification model in a parallel corpus construction method provided in an embodiment of the present application is shown, which specifically includes the following steps:
[0110] Step 402: Obtain a preset training set, where the preset training set includes a plurality of sample corpora, and the plurality of sample corpora include unmodified sample corpora and modified sample corpora.
[0111] The preset training set is a training set composed of multiple sample corpora. The sample corpora carry corpus category labels. The sample corpora can be chat records on chat software, article messages on news media, etc., or can be randomly obtained from other open source corpora.
[0112] The sample corpus can be divided into unmodified sample corpus and modified sample corpus. Optionally, the unmodified sample corpus is a sample corpus with a modification label (i.e., the correct original corpus), and the modified corpus is a sample corpus with a modification type (i.e., the corpus modified from the original corpus), wherein the modification may be to add characters, repeat characters, delete characters, etc., and the corresponding label will be added to the sample corpus after the modification, and the details can be referred to the above step 206. For example, the unmodified sample corpus is "Have you eaten yet", which carries the modification label "000", and after the character repetition modification is performed (the character "no" is repeated at the end of the above corpus), the modified sample corpus "Have you eaten yet" is obtained, which carries the character repetition label "010".
[0113] Optionally, the unmodified sample corpus may also be a corpus with errors, and the modified sample corpus is a corpus obtained by modifying the corpus with errors. For example, if the unmodified corpus is "Have you eaten yet", and the modified sample corpus is "Have you eaten yet", it means that the unmodified corpus has a problem of repeated characters, and carries the repeated character label "010", and the corpus "Have you eaten yet" carries the unmodified label "000".
[0114] Step 404: using a plurality of sample corpora in a preset training set, the second preset neural network is trained to obtain a multi-label classification model, wherein the second preset neural network is a neural network with corpus classification function.
[0115] The second preset neural network is a neural network that can perform multi-label classification on the corpus, for example, a BERT model, a Transformer model, a RoBERTa model, etc. The second preset neural network model used in the embodiment of the present application is a multi-classification neural network model, that is, a neural network model that can output multiple categories such as 000 (no modification class), 001 (character addition class), 010 (character duplication class), and 100 (character deletion class).
[0116] Specifically, the obtained preset training set is input into the second preset neural network, the neural network is trained, the training is terminated when the preset training termination condition is met, and the neural network is determined to be a multi-label classification model.
[0117] Exemplarily, the specific training process of the multi-label classification model can be: extract a sample corpus from a preset training set, input the sample corpus into a second preset neural network (for example, a RoBERTa model with corpus classification function), and identify the category recognition result of the sample corpus after the operation of the second preset neural network. Since the sample corpus carries a category label in advance, the category recognition result is compared with the category label of the sample corpus to obtain a loss value between the two. The network parameters of the second preset neural network can be adjusted based on the loss value, and one training iteration is completed. Then, a sample corpus is extracted from the preset training set again, and the training iteration is continued according to the above steps until the calculated loss value is less than the preset threshold, or the number of training iterations reaches the preset number, then it can be determined that the training is completed, and the second preset neural network obtained after the training is completed is the multi-label classification model.
[0118] By applying the embodiments of the present application, the second preset neural network is trained by carrying a preset training set with labels to obtain a multi-label classification model, so that a more accurate second sample set can be obtained, and a more accurate parallel corpus can be further constructed. In addition, the construction of a parallel sample set using a pre-trained multi-label classification model can be completed automatically, which reduces costs and improves efficiency.
[0119] Figure 5 A flow chart of a method for training a sequence annotation model in a parallel corpus construction method provided in an embodiment of the present application is shown, which specifically includes the following steps:
[0120] Step 502: Obtain a preset training set, where the preset training set includes a plurality of sample corpora, and the plurality of sample corpora include unmodified sample corpora and modified sample corpora.
[0121] The preset training set is a training set composed of multiple sample corpora. The sample corpora carry sequence annotations. The sample corpora can be chat records on chat software, article messages on news media, etc., or can be randomly obtained from other open source corpora.
[0122] The sample corpus is sequence-annotated, that is, each character in the sample corpus is annotated. For example, if the character is an original character, it can be annotated as "O"; if the character is a newly added character, it can be annotated as "D"; if the original character is deleted, the next character of the original character can be annotated as "I". For example, the sequence of the sample corpus "Have you eaten?" is annotated as "[O,O,O,O]"; the sequence of the sample corpus "Have you eaten?" is annotated as "[O,O,O,O,D]".
[0123] Step 504: using a plurality of sample corpora in a preset training set to train a third preset neural network to obtain a sequence labeling model, wherein the third preset neural network is a neural network with a sequence labeling function.
[0124] The third preset neural network is a neural network that can perform sequence annotation on corpus, for example, BERT model, Transformer model, RoBERTa model, etc. The third preset neural network model used in the embodiment of the present application is a text annotation model.
[0125] Specifically, the obtained preset training set is input into the third preset neural network, the neural network is trained, the training is terminated when the preset training termination condition is met, and the neural network is determined to be a sequence labeling model.
[0126] Exemplarily, the specific training process of the sequence labeling model can be: extract a sample corpus from a preset training set, input the sample corpus into a third preset neural network (for example, a RoBERTa model with a sequence labeling function), and obtain a sequence labeling result for the sample corpus after operation by the third preset neural network. Since the sample corpus carries sequence labeling information in advance, the sequence labeling result is compared with the sequence labeling information of the sample corpus to obtain a loss value between the two. The network parameters of the third preset neural network can be adjusted based on the loss value, and one training iteration is completed. Then, a sample corpus is extracted from the preset training set again, and training iterations are continued according to the above steps until the calculated loss value is less than a preset threshold, or the number of training iterations reaches a preset number. It can be determined that the training is completed, and the third preset neural network obtained after the training is completed is the sequence labeling model.
[0127] By applying the embodiments of the present application, the third preset neural network is trained by a preset training set to obtain a sequence labeling model, thereby obtaining a more accurate third sample set, and further constructing a more accurate parallel corpus. In addition, the construction of a parallel sample set using the pre-trained sequence labeling model can be completed automatically, thereby reducing costs and improving efficiency.
[0128] It should be noted that the above Figure 3 - Figure 5 The preset training set in the embodiment may be the same training set, or different parts of the same training set, or a combination of different parts of the same training set, which is not limited here.
[0129] Figure 6 A flow chart of a training method for a multi-label classification model and a sequence annotation model in another parallel corpus construction method provided in an embodiment of the present application is shown, which specifically includes the following steps:
[0130] Step 602: Obtain a preset training set, where the preset training set includes a plurality of sample corpora, and the plurality of sample corpora include unmodified sample corpora and modified sample corpora.
[0131] The preset training set is a training set composed of multiple sample corpora. The sample corpora carry corpus category labels and sequence annotations. The sample corpora can be chat records on chat software, article messages on news media, etc., or they can be randomly obtained from other open source corpora.
[0132] Sample corpora can be divided into unmodified sample corpora and modified sample corpora. Unmodified sample corpora are sample corpora with modification labels and sequence annotations. For example, the sample corpus "Have you eaten?" carries the category label "000" and the sequence annotation "[O,O,O,O]"; for another example, the sample corpus "Have you eaten?" carries the category label "010" and the sequence annotation "[O,O,O,O,D]".
[0133] Step 604: Split the preset training set to obtain a first preset training set and a second preset training set.
[0134] By splitting the preset training set to obtain two training sets, a first preset training set and a second preset training set, part of the sample corpus can be used to simultaneously train the multi-label classification model and the sequence annotation model.
[0135] Specifically, the splitting can be performed according to the average splitting method, or according to the preset neural network attributes, training scenarios and requirements to be trained. For example, if the model architecture of the multi-label classification model is more complex, the sample corpus contained in the first preset training set to be split is more. In an embodiment of the present application, the second preset neural network and the third preset neural network can both use the RoBERTa model, and the model architecture is similar, then the average splitting method is preferably used for splitting.
[0136] Step 606: Use multiple sample corpora in the first preset training set to train the second preset neural network to obtain a multi-label classification model, wherein the second preset neural network is a neural network with corpus classification function.
[0137] Specifically, the first preset training set is input into the second preset neural network, the neural network is trained, the training is terminated when the preset training termination condition is met, and the neural network is determined to be a multi-label classification model. For more specific training process, please refer to the above Figure 4 The method for training a multi-label classification model in step 404 of the illustrated embodiment.
[0138] Step 608: Using the multiple sample corpora in the second preset training set, train the third preset neural network to obtain a sequence labeling model, wherein the third preset neural network is a neural network with a sequence labeling function.
[0139] Specifically, the second preset training set is input into the third preset neural network, the neural network is trained, the training is terminated when the preset training termination condition is met, and the neural network is determined to be a sequence labeling model. For more specific training process, please refer to the above Figure 5 The method for training the sequence labeling model in step 504 of the illustrated embodiment.
[0140] By applying the embodiments of the present application, by splitting the preset training set to obtain two training sets, a first preset training set and a second preset training set, it is possible to simultaneously train the multi-label classification model and the sequence labeling model using partial sample corpus, thereby improving the training efficiency.
[0141] Figure 7 A flow chart of a method for updating a training set in a parallel corpus construction method provided in an embodiment of the present application is shown, which specifically includes the following steps:
[0142] Step 702: Input multiple sample corpora in the first preset training set into the sequence labeling model respectively to obtain the inference sequence labeling label of each sample corpus in the first preset training set.
[0143] The multiple sample corpora in the first preset training set are respectively input into the sequence labeling model. The sequence labeling model performs sequence labeling on each character of the multiple sample corpora in the first preset training set to obtain the inference sequence labeling label of each sample corpus in the first preset training set, so as to provide a reference for subsequent comparison.
[0144] The inference sequence labeling label is a sequence labeling result of the sample corpus in the first preset training set obtained by the sequence labeling model in the current training process.
[0145] Step 704: input the multiple sample corpora in the second preset training set into the multi-label classification model respectively to obtain the inference classification label of each sample corpus in the second preset training set.
[0146] The multiple sample corpora in the second preset training set are respectively input into the multi-label classification model. The multi-label classification model classifies each character of the multiple sample corpora in the second preset training set to obtain the inferred multi-classification label of each sample corpus in the second preset training set to provide a reference for subsequent comparison.
[0147] The inferred classification label is a multi-label classification result of the sample corpus in the second preset training set by the multi-label classification model obtained during the current training process.
[0148] Step 706: Compare the inference sequence annotation label of each sample corpus in the first preset training set with the preset sequence annotation label pre-annotated for each sample corpus in the first preset training set, add the sample corpus with the same label to the second preset training set, and obtain an updated second preset training set.
[0149] The inference sequence annotation labels of each sample corpus in the first preset training set are compared with the preset sequence annotation labels pre-annotated by each sample corpus in the first preset training set, and the sample corpus with the same label is cross-added to the second preset training set to obtain an updated second preset training set. That is, the trained sequence annotation model is verified using the first preset training set, and the sample corpus that has passed the verification is added to the second preset training set, so that the sample quality of the second preset training set is higher.
[0150] Step 708: Compare the inferred classification label of each sample corpus in the second preset training set with the preset classification label pre-labeled for each sample corpus in the second preset training set, add the sample corpus with the same label to the first preset training set, and obtain an updated first preset training set.
[0151] The inferred classification labels of each sample corpus in the second preset training set are compared with the preset classification labels pre-labeled for each sample corpus in the second preset training set, and the sample corpus with the same label is cross-added to the first preset training set to obtain an updated first preset training set. That is, the trained multi-label classification model is verified using the second preset training set, and the sample corpus that has passed the verification is added to the first preset training set, so that the sample quality of the first preset training set is higher.
[0152] Step 710: Continue to train the multi-label classification model using the updated multiple sample corpora in the first preset training set, and continue to train the sequence labeling model using the updated multiple sample corpora in the second preset training set.
[0153] The multi-label classification model and the sequence labeling model are further trained using the updated multiple sample corpora in the first preset training set and the multiple sample corpora in the second preset training set. Because the above steps 706 and 708 cross-add sample corpora with the same labels, the first preset training set and the second preset training set are updated, and the updated training sets are of higher quality. Therefore, the training effect of the multi-label classification model and the sequence labeling model is better, the obtained multi-label classification model and sequence labeling model are more accurate, and the quality of the subsequently constructed parallel corpus is higher.
[0154] The specific process of continuing training is the same or similar to the training process of the above embodiment, and will not be repeated here. Optionally, the condition for the end of training can also be a preset threshold value for the model performance of the two models. In the embodiment of the present application, F1-Score is used as the preset threshold value. When the two models reach the preset F1-Score respectively, the training of the two models can be considered to be completed.
[0155] By applying the embodiments of the present application, the first preset training set and the second preset training set are updated to improve the sample quality, thereby improving the accuracy of the trained multi-label classification model and sequence annotation model, thereby ensuring that the quality of the subsequently constructed parallel corpus is higher.
[0156] Figure 8 A flowchart of a method for corpus processing in a parallel corpus construction method provided in an embodiment of the present application is shown, which specifically includes the following steps:
[0157] Step 802: For any corpus in the third corpus set, if the sequence annotation label of the corpus includes a deletion label, the corpus is input into a pre-trained character addition model, and the character addition model is used to perform character addition processing on the corpus.
[0158] The character addition model is a pre-trained neural network model that can add characters to the corpus. The character addition model is trained based on sample corpus, which is a corpus lacking certain characters and can also carry a label (the label is a character added at a certain position). Using the above sample corpus, by training a preset neural network (the neural network model can be a language representation model, such as a RoBERTa model), the character addition model can be trained.
[0159] The specific character addition model training process will be described in Fig. 9 This is described in detail in the illustrated embodiment and will not be repeated here.
[0160] For any corpus in the third corpus, the tags carried by the third corpus can be identified. If the sequence annotation tags of the corpus include the deletion tag "I", the character adding model is input to directly add characters to the corpus.
[0161] Step 804: For any corpus in the third corpus set, if the sequence annotation tag of the corpus includes an addition tag, the characters with the addition tag in the corpus are deleted.
[0162] For any corpus in the third corpus, the tags carried by the third corpus can be identified. If the sequence annotation tags of the corpus include the added tag "D", the characters with the added tag "D" in the corpus can be directly deleted.
[0163] Step 806: Determine a third corpus obtained by processing each corpus as a fourth corpus.
[0164] After adding or deleting characters to all the corpora in the third corpus, the fourth corpus can be obtained.
[0165] By applying the embodiment of the present application, the character addition model adds or deletes the corpus in the third corpus including the deleted tags or the added tags, restores the corpus with the modified tags (i.e., the corpus with the characters omitted, repeated, added, etc.), and obtains the corpus without the characters omitted, repeated, added, etc., thereby providing a corpus basis for constructing a parallel corpus.
[0166] Fig. 9 A flow chart of a method for training a character addition model in a parallel corpus construction method provided in an embodiment of the present application is shown, which specifically includes the following steps:
[0167] Step 902: Obtain a preset training subset, where the preset training subset includes a plurality of modified sample corpora, where the modified sample corpora are sample corpora obtained by deleting characters from a preset unmodified sample corpora.
[0168] The preset training subset is a training set composed of multiple modified sample corpora. The sample corpora carry deletion labels. The sample corpora can be chat records on chat software, article messages on news media, etc., or can be randomly obtained from other open source corpora.
[0169] Step 904: Utilize a plurality of modified sample corpora in a preset training subset to train a fourth preset neural network to obtain a character adding model, wherein the fourth preset neural network is a neural network with a character adding function.
[0170] The fourth neural network is a neural network that can add or delete characters to the modified corpus, for example, a BERT model, a Transformer model, a RoBERTa model, etc.
[0171] Specifically, a preset training subset is input into a fourth preset neural network, the neural network is trained, the training is terminated when a preset training termination condition is met, and the neural network is determined to be a character adding model.
[0172] Exemplarily, the specific training process of the character addition model can be: extract a modified sample corpus from a preset training subset, input the modified sample corpus into a fourth preset neural network (for example, a RoBERTa model with a character adding function), and obtain the character addition result for the modified sample corpus after the operation of the fourth preset neural network. Since the modified sample corpus carries a deletion label (the position of the deleted character and the deleted character) in advance, the character addition result is compared with the deletion label of the sample corpus to obtain a loss value between the two. The network parameters of the fourth preset neural network can be adjusted based on the loss value, and one training iteration is completed. Then, a modified sample corpus is extracted from the preset training subset again, and the training iteration is continued according to the above steps until the calculated loss value is less than the preset threshold, or the number of training iterations reaches the preset number, then it can be determined that the training is completed, and the fourth preset neural network obtained after the training is completed is the character addition model.
[0173] By applying the embodiments of the present application, the fourth preset neural network is trained by a preset training subset to obtain a character addition model, thereby obtaining a more accurate fourth sample set, and further constructing a more accurate parallel corpus. In addition, the construction of a parallel sample set using the pre-trained character addition model can be completed automatically, thereby reducing costs and improving efficiency.
[0174] Fig.10 A flowchart of a text conversion model training method provided by an embodiment of the present application is shown, which specifically includes the following steps:
[0175] Step 1002: Obtain a parallel corpus.
[0176] Among them, the parallel corpus is based on the above Figure 2 - Fig. 9 The parallel corpus construction method of the embodiment is constructed.
[0177] Taking the training of spoken-written translation model as an example, the second corpus (spoken corpus) included in the parallel corpus is the initial corpus, and the fourth corpus (written corpus) is the target corpus.
[0178] Step 1004: Based on the parallel corpus, a preset neural network is trained to obtain a text conversion model.
[0179] The preset neural network is a neural network that can perform text conversion on the initial corpus in the initial corpus set, for example, a BERT model, a Transformer model, a RoBERTa model, etc.
[0180] Specifically, multiple initial corpora in the second corpus are input into the preset neural network to obtain the target corpus corresponding to each initial corpus, and then the loss value is calculated based on the obtained target corpus and the corresponding target corpus in the first corpus, and the network parameters of the preset neural network are adjusted based on the loss value. The training is terminated when the preset training end conditions are met, and the preset neural network is determined to be a text conversion model.
[0181] Exemplarily, the specific training process of the text conversion model can be: extract an initial corpus from the second corpus in the parallel corpus, input the initial corpus into a preset neural network (for example, a RoBERTa model with text processing function), and obtain the text conversion result of the initial corpus after the operation of the preset neural network. The text conversion result is compared with the corresponding target corpus in the fourth corpus to obtain the loss value between the two (the loss value can be calculated using the traditional BLEU), and the network parameters of the preset neural network can be adjusted based on the loss value. At this point, one training iteration is completed, and then an initial corpus is extracted from the second corpus again, and the training iteration is continued according to the above steps until the calculated loss value is less than the preset threshold, or the number of training iterations reaches the preset number, then it can be determined that the training is completed, and the preset neural network obtained after the training is completed is the text conversion model.
[0182] The training method of the text conversion model provided in the embodiment of the present application is as follows: Figure 2 - Fig. 9 The parallel corpus constructed by the parallel corpus construction method of the embodiment is used to train the text conversion model. Since the parallel corpus has high accuracy and high quality, training the text conversion model based on the parallel corpus can improve the text conversion performance of the obtained text conversion model with higher accuracy.
[0183] Fig.11 A flowchart of a text conversion method provided by an embodiment of the present application is shown, which specifically includes the following steps:
[0184] Step 1102: Obtain the text to be converted.
[0185] Step 1104: input the text to be converted into the text conversion model to obtain the converted text.
[0186] Among them, the text conversion model is based on the above Fig.10 The text conversion model is trained using the text conversion model training method of the illustrated embodiment.
[0187] The text conversion method provided in the embodiment of the present application, the text conversion model is trained by the above-mentioned text conversion model training method, and the text conversion model is trained by the parallel corpus constructed by the above-mentioned method. Since the parallel corpus has high accuracy and high quality, training the text conversion model based on the parallel corpus can improve the text conversion performance of the obtained text conversion model and improve the accuracy of the text conversion model. Therefore, the converted text obtained is more accurate.
[0188] The following is combined with Fig.12 Taking the application of constructing a parallel corpus of spoken language and book language as an example, the parallel corpus construction method, text conversion model training method and text conversion method provided in the embodiment of the present application are described in detail. Among them, Fig.12 A processing flow chart of a text conversion method provided by an embodiment of the present application is shown, which specifically includes the following steps:
[0189] Step 1202: Obtain corpus A, and divide corpus A into four parts.
[0190] The corpus in corpus A is a corpus of Chinese texts with coherent sentences, which is divided into four parts: corpus A1 without any processing; using word segmentation tools such as pkuseg and jieba to segment the original corpus, and then adding random modal particles at random positions in the corpus to form corpus A2, where the random modal particles are pre-constructed; using word segmentation tools such as pkuseg and jieba to segment the original corpus, and then selecting one or more random characters (characters or words) to repeat one or more times to form corpus A3; using word segmentation tools such as pkuseg and jieba to segment the original corpus, and then selecting one or more random characters (characters or words) to delete, to form corpus A4.
[0191] Step 1204: Mix the corpus in the corpus sets A1, A2, A3, and A4 and divide them equally into two parts C1 and C2.
[0192] Step 1206: Use corpus C1 to train the Chinese pre-trained model RoBERTa to obtain a multi-label text classification model Model_C.
[0193] Step 1208: Use corpus C2 to train the Chinese pre-training model RoBERTa to obtain the sequence labeling model Model_S.
[0194] Step 1210: Use corpus A4 to train the Chinese pre-trained model RoBERTa to obtain a character adding model Model_M. The character adding model Model_M is used to implement the function of filling in words according to the context.
[0195] Step 1212: Use data set A to train Chinese and training model RoBERTa to obtain grammar model Model_G.
[0196] The model training in the above steps can refer to the above Figure 3-9 The illustrated embodiments will not be described in detail here.
[0197] Step 1214: Use Model_S to infer corpus C1, use Model_C to infer corpus C2, add the correctly inferred corpus to each other's corpus, and then continue to train Model_C and Model_S.
[0198] Among them, the correctly inferred corpus refers to the corpus whose inferred sequence labeling labels obtained by Model_S are the same as the pre-annotated preset sequence labeling labels, and the corpus whose inferred classification labels obtained by Model_C are the same as the pre-annotated preset classification labels. After continuous iteration, until the training standard values of Model_C and Model_S reach a certain threshold, the training standard value can be calculated based on the precision and recall rate.
[0199] Step 1216: Obtain a corpus set D including a large amount of corpus of unknown quality, wherein the corpus of unknown quality refers to corpus without labels.
[0200] Step 1218: Use Model_G to filter the corpus D to obtain the grammatically correct corpus D1.
[0201] Step 1220: Use Model_C to classify corpus D1, delete the corpus without modification class labels, and obtain corpus D2.
[0202] Step 1222: Use Model_S to perform sequence labeling on corpus D2 to obtain corpus D3.
[0203] Step 1224: Identify the sequence annotation labels of each corpus in corpus D3. If the corpus has an I label, input the corpus into Model_M for automatic word filling; delete the characters with the D label to obtain corpus D4.
[0204] Step 1226: Based on corpora D2 and D4, a parallel corpus E is constructed.
[0205] Step 1228: Use the parallel corpus E to train the text conversion model Model_T. When training Model_T, BLEU can be used as a training standard.
[0206] Step 1230: Obtain the spoken text to be converted, input the spoken text into Model_T, and obtain the converted book language text.
[0207] By using the embodiment of the present application, the acquired corpus D is input into the pre-trained grammar model Model_G, and the grammatically correct corpus D1 is screened, and then the corpus D1 is input into the pre-trained multi-label classification model Model_C, and the corpus of the unmodified type is deleted to obtain the corpus D2, and then the pre-trained sequence annotation model Model_S is used to perform sequence annotation on the corpus D2 to obtain the corpus D3, and the corpus in the corpus D3 is processed according to the sequence annotation label of each corpus in the corpus D3 to obtain the corpus D4. The corpus in the corpus D2 is used as the original corpus, and the corpus in the corpus D4 is used as the target corpus. Based on the corpus D2 and the corpus D4, a parallel corpus E can be constructed. Through the embodiment of the present application, a parallel corpus can be automatically constructed by using a neural network-based method, without the need for pre-annotation of corpus, which reduces the cost of corpus annotation, and the automatic acquisition of parallel corpus can avoid the errors caused by manual annotation, thereby improving the accuracy of corpus annotation. Then, the accuracy of the text conversion model Model_T obtained by training with the parallel corpus E is improved. After obtaining the spoken text to be converted, the spoken text can be input into Model_T to obtain the accurate book language text with high conversion accuracy.
[0208] Corresponding to the above method embodiment, the present application also provides a parallel corpus construction device embodiment, Fig.13 FIG. 1 is a schematic diagram showing the structure of a parallel corpus construction device provided by an embodiment of the present application. Fig.13 As shown, the device comprises:
[0209] A first acquisition module 1302 is configured to acquire an initial corpus set, where the initial corpus set includes a plurality of unlabeled initial corpora;
[0210] The screening module 1304 is configured to input each initial corpus into a pre-trained grammar model, obtain a grammar recognition result of each initial corpus, and screen out first corpus with a correct grammar recognition result from the initial corpus set to form a first corpus set;
[0211] The deletion module 1306 is configured to input each first corpus into a pre-trained multi-label classification model to obtain a classification label of each first corpus, and delete the corpus with a classification label of no modification class label from the first corpus set to obtain a second corpus set;
[0212] The sequence labeling module 1308 is configured to use a pre-trained sequence labeling model to perform sequence labeling on each corpus in the second corpus to obtain a third corpus;
[0213] The processing module 1310 is configured to process each corpus in the third corpus according to the sequence annotation label of each corpus in the third corpus to obtain a fourth corpus;
[0214] The construction module 1312 is configured to construct a parallel corpus based on the second corpus and the fourth corpus.
[0215] Optionally, the device further comprises: a grammar model training module;
[0216] The grammar model training module is configured to obtain a preset training set, which includes multiple sample corpora, and the multiple sample corpora include unmodified sample corpora and modified sample corpora; use the multiple sample corpora in the preset training set to train a first preset neural network to obtain a grammar model, wherein the first preset neural network is a neural network with grammar recognition function.
[0217] Optionally, the device further comprises: a multi-label classification model training module;
[0218] The multi-label classification model training module is configured to obtain a preset training set, which includes multiple sample corpora, and the multiple sample corpora include unmodified sample corpora and modified sample corpora; use the multiple sample corpora in the preset training set to train the second preset neural network to obtain a multi-label classification model, wherein the second preset neural network is a neural network with corpus classification function.
[0219] Optionally, the device further comprises: a sequence labeling model training module;
[0220] The sequence labeling model training module is configured to obtain a preset training set, which includes multiple sample corpora, and the multiple sample corpora include unmodified sample corpora and modified sample corpora; use the multiple sample corpora in the preset training set to train a third preset neural network to obtain a sequence labeling model, wherein the third preset neural network is a neural network with sequence labeling function.
[0221] Optionally, the device further comprises: a multi-label classification model and a sequence annotation model training module;
[0222] The multi-label classification model and sequence labeling model training module is configured to obtain a preset training set, which includes multiple sample corpora, and the multiple sample corpora include unmodified sample corpora and modified sample corpora; split the preset training set to obtain a first preset training set and a second preset training set; use the multiple sample corpora in the first preset training set to train the second preset neural network to obtain a multi-label classification model, wherein the second preset neural network is a neural network with a corpus classification function; use the multiple sample corpora in the second preset training set to train the third preset neural network to obtain a sequence labeling model, wherein the third preset neural network is a neural network with a sequence labeling function.
[0223] Optionally, the multi-label classification model and sequence labeling model training module are further configured to input multiple sample corpora in the first preset training set into the sequence labeling model respectively to obtain the inference sequence labeling label of each sample corpus in the first preset training set; input multiple sample corpora in the second preset training set into the multi-label classification model respectively to obtain the inference classification label of each sample corpus in the second preset training set; compare the inference sequence labeling label of each sample corpus in the first preset training set with the preset sequence labeling label pre-labeled for each sample corpus in the first preset training set, add the sample corpus with the same label to the second preset training set, and obtain an updated second preset training set; compare the inference classification label of each sample corpus in the second preset training set with the preset classification label pre-labeled for each sample corpus in the second preset training set, add the sample corpus with the same label to the first preset training set, and obtain an updated first preset training set; use the multiple sample corpora in the updated first preset training set to continue training the multi-label classification model, and use the multiple sample corpora in the updated second preset training set to continue training the sequence labeling model.
[0224] Optionally, the processing module 1310 is further configured to, for any corpus in the third corpus set, if the sequence annotation label of the corpus includes a deletion label, input the corpus into a pre-trained character addition model, and use the character addition model to add characters to the corpus; for any corpus in the third corpus set, if the sequence annotation label of the corpus includes an addition label, delete the characters with the addition label in the corpus; and determine the third corpus set after processing each corpus as the fourth corpus set.
[0225] Optionally, the device further comprises: a character adding model training module;
[0226] The character adding model training module is configured to obtain a preset training subset, which includes multiple modified sample corpora, where the modified sample corpora are sample corpora obtained by deleting characters from a preset unmodified sample corpus; and train a fourth preset neural network using the multiple modified sample corpora in the preset training subset to obtain a character adding model, wherein the fourth preset neural network is a neural network with a function of adding characters.
[0227] Optionally, the construction module 1312 is further configured to make a one-to-one correspondence between the corpus in the second corpus and the corpus in the fourth corpus to obtain a parallel corpus.
[0228] By using the embodiment of the present application, the acquired initial corpus is input into a pre-trained grammar model, and a first corpus consisting of grammatically correct corpora is screened, and then the first corpus is input into a pre-trained multi-label classification model, and the corpus of the unmodified type is deleted to obtain a second corpus, and then the pre-trained sequence annotation model is used to perform sequence annotation on the second corpus to obtain a third corpus, and each corpus in the third corpus is processed according to the sequence annotation label of each corpus in the third corpus to obtain a fourth corpus. The corpus in the second corpus can be used as the original corpus, and the corpus in the fourth corpus can be used as the target corpus. Based on the second corpus and the fourth corpus, a parallel corpus can be constructed. Through the embodiment of the present application, a parallel corpus can be automatically constructed using a neural network-based method, without the need for pre-annotation of corpus, which reduces the cost of corpus annotation, and the automatic acquisition of a parallel corpus can avoid the errors caused by manual annotation, thereby improving the accuracy of corpus annotation.
[0229] The above is a schematic scheme of a parallel corpus construction device of the present embodiment. It should be noted that the technical scheme of the parallel corpus construction device and the technical scheme of the above-mentioned parallel corpus construction method belong to the same concept. For the details not described in detail in the technical scheme of the parallel corpus construction device, please refer to the description of the technical scheme of the above-mentioned parallel corpus construction method. In addition, each component in the device embodiment should be understood as a functional module that must be established to implement each step of the program flow or each step of the method, and each functional module is not an actual functional division or separation definition. The device claim defined by such a group of functional modules should be understood as a functional module architecture that mainly implements the solution through the computer program recorded in the specification, and should not be understood as a physical device that mainly implements the solution through hardware.
[0230] Corresponding to the above method embodiment, the present application also provides a text conversion model training device embodiment, Fig.14 FIG. 1 is a schematic diagram showing the structure of a text conversion model training device provided by an embodiment of the present application. Fig.14As shown, the device comprises:
[0231] The second acquisition module 1402 is configured to acquire a parallel corpus. The parallel corpus uses the present application Figure 2 - Fig. 9 The method of any one embodiment is constructed;
[0232] The training module 1404 is configured to train a preset neural network based on a parallel corpus to obtain a text conversion model.
[0233] Applying the embodiment of the present application, through the above Figure 2 - Fig. 9 The parallel corpus constructed by the parallel corpus construction method of the embodiment is used to train the text conversion model. Since the parallel corpus has high accuracy and high quality, training the text conversion model based on the parallel corpus can improve the text conversion performance of the obtained text conversion model with higher accuracy.
[0234] The above is a schematic scheme of a text conversion model training device of the present embodiment. It should be noted that the technical scheme of the text conversion model training device and the technical scheme of the above-mentioned text conversion model training method belong to the same concept. For the details not described in detail in the technical scheme of the text conversion model training device, please refer to the description of the technical scheme of the above-mentioned text conversion model training method. In addition, each component in the device embodiment should be understood as a functional module that must be established to implement each step of the program flow or each step of the method, and each functional module is not an actual functional division or separation definition. The device claim defined by such a group of functional modules should be understood as a functional module architecture that mainly implements the solution through the computer program recorded in the specification, and should not be understood as a physical device that mainly implements the solution through hardware.
[0235] Corresponding to the above method embodiment, the present application also provides a text conversion device embodiment, Fig.15 FIG. 1 is a schematic diagram showing the structure of a text conversion device provided by an embodiment of the present application. Fig.15 As shown, the device comprises:
[0236] The third acquisition module 1502 is configured to acquire the text to be converted;
[0237] The conversion module 1504 is configured to input the text to be converted into a text conversion model to obtain the converted text. The text conversion model is based on the present application. Fig.10 The method of the embodiment is trained.
[0238] Using the embodiments of the present application, a text conversion model is obtained by training through the above-mentioned text conversion model training method. The text conversion model is trained through the parallel corpus constructed by the above-mentioned method. Since the parallel corpus has high accuracy and high quality, training the text conversion model based on the parallel corpus can improve the text conversion performance of the obtained text conversion model and improve the accuracy of the text conversion model. Therefore, the converted text obtained is more accurate.
[0239] The above is a schematic scheme of a text conversion device of this embodiment. It should be noted that the technical scheme of the text conversion device and the technical scheme of the above-mentioned text conversion method belong to the same concept. For the details not described in detail in the technical scheme of the device, please refer to the description of the technical scheme of the above-mentioned text conversion method. In addition, the various components in the device embodiment should be understood as functional modules that must be established to implement each step of the program flow or each step of the method, and each functional module is not an actual functional division or separation definition. The device claim defined by such a group of functional modules should be understood as a functional module architecture that mainly implements the solution through the computer program recorded in the specification, and should not be understood as a physical device that mainly implements the solution through hardware.
[0240] Fig.16 The block diagram of a computing device 1600 provided in one embodiment of the present application is shown. The components of the computing device 1600 include but are not limited to a memory 1610 and a processor 1620. The processor 1620 is connected to the memory 1610 via a bus 1630, and the database 1650 is used to store data.
[0241] The computing device 1600 also includes an access device 1640 that enables the computing device 1600 to communicate via one or more networks 1660. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 1640 may include one or more of any type of network interface (e.g., a network interface card (NIC)) of wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a world microwave interconnection access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, and the like.
[0242] In one embodiment of the present application, the above components of the computing device 1600 and Fig.16 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Fig.16 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of the present application. Those skilled in the art may add or replace other components as needed.
[0243] Computing device 1600 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or PC. Computing device 1600 may also be a mobile or stationary server.
[0244] The processor 1620 is used to execute computer executable instructions of the parallel corpus construction method, the text conversion model training method or the text conversion method.
[0245] The above is a schematic scheme of a computing device of this embodiment. It should be noted that the technical scheme of the computing device and the technical schemes of the above-mentioned parallel corpus construction method, text conversion model training method and text conversion method belong to the same concept, and the details not described in detail in the technical scheme of the computing device can be referred to the description of the technical schemes of the above-mentioned parallel corpus construction method, text conversion model training method or text conversion method.
[0246] An embodiment of the present application also provides a computer-readable storage medium storing computer instructions, which, when executed by a processor, are used for a parallel corpus construction method, a text conversion model training method, or a text conversion method.
[0247] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium and the technical schemes of the above-mentioned parallel corpus construction method, text conversion model training method and text conversion method belong to the same concept, and the details not described in detail in the technical scheme of the storage medium can be referred to the description of the technical schemes of the above-mentioned parallel corpus construction method, text conversion model training method or text conversion method.
[0248] An embodiment of the present application also provides a chip storing a computer program, which, when executed by the chip, implements the steps of the parallel corpus construction method, the text conversion model training method or the text conversion method.
[0249] The above describes specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
[0250] The computer instructions include computer program codes, which may be in source code form, object code form, executable files or some intermediate forms, etc. The computer readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc.
[0251] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described action sequence, because according to the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.
[0252] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0253] The preferred embodiments of the present application disclosed above are only used to help explain the present application. The optional embodiments do not describe all the details in detail, nor do they limit the invention to the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of the present application. The present application selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present application, so that those skilled in the art can understand and use the present application well. The present application is only limited by the claims and their full scope and equivalents.
Claims
1. A parallel corpus construction method, It is characterized in that include: Acquire an initial corpus set, wherein the initial corpus set includes a plurality of unlabeled initial corpora; Inputting each initial corpus into a pre-trained grammar model respectively to obtain a grammar recognition result of each initial corpus, and selecting a first corpus with a correct grammar recognition result from the initial corpus set to form a first corpus set; Input each first corpus into a pre-trained multi-label classification model to obtain a classification label of each first corpus, and delete corpuses whose classification labels are no modification labels from the first corpus set to obtain a second corpus set; Using a pre-trained sequence labeling model, sequence labeling is performed on each corpus in the second corpus to obtain a third corpus; Processing each corpus in the third corpus according to the sequence annotation label of each corpus in the third corpus to obtain a fourth corpus; A parallel corpus is constructed based on the second corpus and the fourth corpus.
2. The method according to claim 1, It is characterized in that Before the step of inputting each initial corpus into the pre-trained grammar model to obtain the grammar recognition result of each initial corpus, the step further includes: Acquire a preset training set, wherein the preset training set includes a plurality of sample corpora, and the plurality of sample corpora include unmodified sample corpora and modified sample corpora; The first preset neural network is trained using the plurality of sample corpora in the preset training set to obtain a grammar model, wherein the first preset neural network is a neural network with a grammar recognition function.
3. The method according to claim 1, It is characterized in that Before the step of inputting each first corpus into a pre-trained multi-label classification model to obtain a classification label of each first corpus, the method further includes: Acquire a preset training set, wherein the preset training set includes a plurality of sample corpora, and the plurality of sample corpora include unmodified sample corpora and modified sample corpora; The second preset neural network is trained using the multiple sample corpora in the preset training set to obtain a multi-label classification model, wherein the second preset neural network is a neural network with corpus classification function.
4. The method according to claim 1, It is characterized in that Before the step of using the pre-trained sequence labeling model to perform sequence labeling on each corpus in the second corpus to obtain the third corpus, the method further includes: Acquire a preset training set, wherein the preset training set includes a plurality of sample corpora, and the plurality of sample corpora include unmodified sample corpora and modified sample corpora; The third preset neural network is trained using the multiple sample corpora in the preset training set to obtain a sequence labeling model, wherein the third preset neural network is a neural network with a sequence labeling function.
5. The method according to claim 1, It is characterized in that Before the step of inputting each first corpus into a pre-trained multi-label classification model to obtain a classification label of each first corpus, the method further includes: Acquire a preset training set, wherein the preset training set includes a plurality of sample corpora, and the plurality of sample corpora include unmodified sample corpora and modified sample corpora; Splitting the preset training set to obtain a first preset training set and a second preset training set; Using a plurality of sample corpora in the first preset training set, training a second preset neural network to obtain a multi-label classification model, wherein the second preset neural network is a neural network with a corpus classification function; The third preset neural network is trained using the plurality of sample corpora in the second preset training set to obtain a sequence labeling model, wherein the third preset neural network is a neural network with a sequence labeling function.
6. The method according to claim 5, It is characterized in that After the step of training the third preset neural network using the plurality of sample corpora in the second preset training set to obtain a sequence labeling model, the method further includes: Inputting a plurality of sample corpora in the first preset training set into the sequence labeling model respectively to obtain an inference sequence labeling label of each sample corpus in the first preset training set; Inputting the plurality of sample corpora in the second preset training set into the multi-label classification model respectively to obtain the inference classification label of each sample corpus in the second preset training set; Compare the inference sequence annotation label of each sample corpus in the first preset training set with the preset sequence annotation label pre-annotated for each sample corpus in the first preset training set, and add the sample corpus with the same label to the second preset training set to obtain an updated second preset training set; Compare the inferred classification label of each sample corpus in the second preset training set with the preset classification label pre-labeled for each sample corpus in the second preset training set, and add the sample corpus with the same label to the first preset training set to obtain an updated first preset training set; The multi-label classification model is continuously trained using the multiple sample corpora in the updated first preset training set, and the sequence labeling model is continuously trained using the multiple sample corpora in the updated second preset training set.
7. The method according to claim 1, It is characterized in that The step of processing each corpus in the third corpus according to the sequence annotation label of each corpus in the third corpus to obtain the fourth corpus includes: For any corpus in the third corpus set, if the sequence annotation label of the corpus includes a deletion label, the corpus is input into a pre-trained character addition model, and the character addition model is used to perform character addition processing on the corpus; For any corpus in the third corpus set, if the sequence annotation tag of the corpus includes an addition tag, deleting the characters with the addition tag in the corpus; A third corpus obtained after processing each corpus is determined as a fourth corpus.
8. The method according to claim 7, It is characterized in that Before the step of inputting the corpus into the pre-trained character adding model and using the character adding model to perform character adding processing on the corpus, the method further includes: Acquire a preset training subset, wherein the preset training subset includes a plurality of modified sample corpora, wherein the modified sample corpora are sample corpora obtained by deleting characters from a preset unmodified sample corpora; The fourth preset neural network is trained using the multiple modified sample corpora in the preset training subset to obtain a character adding model, wherein the fourth preset neural network is a neural network with a character adding function.
9. The method according to claim 1, It is characterized in that The step of constructing a parallel corpus based on the second corpus and the fourth corpus comprises: The corpus in the second corpus set is matched one-to-one with the corpus in the fourth corpus set to obtain a parallel corpus set.
10. A text conversion model training method, It is characterized in that include: Obtaining a parallel corpus, wherein the parallel corpus is constructed using the method described in any one of claims 1 to 9; Based on the parallel corpus, a preset neural network is trained to obtain a text conversion model.
11. A text conversion method, It is characterized in that include: Get the text to be converted; The text to be converted is input into a text conversion model to obtain a converted text, and the text conversion model is trained according to the method according to claim 10.
12. A parallel corpus construction device, It is characterized in that include: A first acquisition module is configured to acquire an initial corpus set, wherein the initial corpus set includes a plurality of unlabeled initial corpora; A screening module is configured to input each initial corpus into a pre-trained grammar model, obtain a grammar recognition result of each initial corpus, and screen out first corpuses with correct grammar recognition results from the initial corpus set to form a first corpus set; a deletion module configured to input each first corpus into a pre-trained multi-label classification model, obtain a classification label of each first corpus, and delete corpuses whose classification labels are no modification class labels from the first corpus set to obtain a second corpus set; A sequence labeling module is configured to use a pre-trained sequence labeling model to perform sequence labeling on each corpus in the second corpus to obtain a third corpus; a processing module configured to process each corpus in the third corpus according to the sequence annotation label of each corpus in the third corpus to obtain a fourth corpus; A construction module is configured to construct a parallel corpus based on the second corpus and the fourth corpus.
13. A text conversion model training device, It is characterized in that include: A second acquisition module is configured to acquire a parallel corpus, wherein the parallel corpus is constructed using the method according to any one of claims 1 to 9; The training module is configured to train a preset neural network based on the parallel corpus to obtain a text conversion model.
14. A text conversion device, It is characterized in that include: A third acquisition module is configured to acquire the text to be converted; The conversion module is configured to input the text to be converted into a text conversion model to obtain the converted text, and the text conversion model is trained according to the method according to claim 10.
15. A computing device, It is characterized in that include: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the parallel corpus construction method as described in any one of claims 1 to 9, or the text conversion model training method as described in claim 10, or the text conversion method as described in claim 11.
16. A computer-readable storage medium storing computer instructions, It is characterized in that When the instruction is executed by the processor, the parallel corpus construction method as described in any one of claims 1 to 9, or the text conversion model training method as described in claim 10, or the text conversion method as described in claim 11 is implemented.
