A text regularization method, device, equipment and storage medium

Abstract

The present application provides a text regularization method, device, equipment and storage medium, wherein the method includes: obtaining a text to be regularized; regularizing the text to be regularized using a pre-constructed text regularization model to obtain a regularized text corresponding to the text to be regularized; wherein the text regularization model is trained using training samples, and the training samples include multiple training texts with the same semantics but different expressions and annotated with the same regularized text, and when the training samples are used to train the text regularization model, the semantic consistency of the multiple training texts contained in the training samples is constrained. The text regularization method provided by the present application can not only regularize the spoken text to be regularized into a written text, but also ensure that the semantics of the regularized text remain unchanged.

Description

Technical Field

[0001] The present application relates to the field of text processing technology, and in particular to a text regularization method, device, equipment and storage medium. Background Art

[0002] With the rapid development of Internet technology, people are faced with more and more information every day, and most of this information appears in the form of text.

[0003] Due to the complexity and diversity of specific application scenarios, there is often a large amount of noise data in text information, such as meaningless words that appear in large numbers or words with similar semantics that appear repeatedly. The existence of noise data makes the description of text data more arbitrary and cannot be used directly in many scenarios. For example, the text data obtained after voice recognition of interview data in journalists' work often contains unsmooth phenomena such as modal particles, responses, catchphrases or repeated descriptions, which makes it impossible to directly use the relevant descriptions in the recognized text to edit it into a manuscript for publication.

[0004] Therefore, it is of great significance to regularize text data and make the regularized text data more written. Summary of the invention

[0005] In view of this, the present application provides a text regularization method, device, equipment and storage medium for regularizing text so that the regularized text is more written and convenient to use. The technical solution is as follows:

[0006] A text regularization method, comprising:

[0007] Get the text to be regulated;

[0008] Regularizing the text to be regularized using a pre-built text regularization model to obtain a regularized text corresponding to the text to be regularized;

[0009] Among them, the text regularization model is trained using training samples, and the training samples include multiple training texts with the same semantics, different expressions and annotated with the same regularized text. When the training samples are used to train the text regularization model, the semantic consistency of the multiple training texts contained in the training samples is constrained.

[0010] Optionally, the training objectives of the text regularization model include:

[0011] Make the latent variable posterior distributions corresponding to the multiple training texts included in the training sample consistent, wherein the latent variable posterior distribution is the posterior distribution of the latent variable that is obtained according to the corresponding training text and can represent the semantic space;

[0012] And, the regularized texts respectively corresponding to the multiple training texts included in the training sample are made consistent with the annotated regularized texts.

[0013] Optionally, the training texts included in the training samples are obtained from a pre-constructed training text set, and the construction process of the training text set includes:

[0014] Acquire an original training text set, wherein the original training text set includes a plurality of original training texts annotated with regularized texts;

[0015] For each original training text in the original training text set: based on the regularized text annotated by the original training text, the original training text is processed into a text with the same semantics as the original training text but different expressions, and the processed text is annotated with the regularized text annotated by the original training text to obtain an extended training text corresponding to the original training text, and the original training text and the extended training text corresponding to the original training text form a set of training texts;

[0016] The training text set is composed of a plurality of sets of obtained training texts.

[0017] Optionally, the regularized text annotated based on the original training text, processing the original training text into a text with the same semantics as the original training text but different expressions, includes:

[0018] Performing word segmentation processing on the original training text and the regularized text annotated by the original training text respectively to obtain two word sequences;

[0019] Determining the deletable words in the original training text by aligning the two word sequences;

[0020] Some or all of the deletable words in the original training text are deleted to obtain a text with the same semantics but different expressions as the original training text.

[0021] Optionally, deleting some or all of the deletable words in the original training text to obtain a text with the same semantics but different expressions as the original training text includes:

[0022] Determining a deletion strategy or a plurality of different deletion strategies according to the deletable words and the number of the deletable words in the original training text, wherein each deletion strategy is used to indicate the number of words actually deleted and the words actually deleted from the deletable words;

[0023] The original training text is deleted according to the determined deletion strategy to obtain one or more texts with the same semantics but different expressions as the original training text.

[0024] Optionally, the step of regularizing the text to be regularized by using a pre-built text regularization model to obtain a regularized text corresponding to the text to be regularized includes:

[0025] Using the text regularization model, obtain the context vector of each word in the text to be regularized and the semantic representation vector of the text to be regularized; or, using the text regularization model, obtain the context vector of each word in the text to be regularized, the semantic representation vector of the text to be regularized and the semantic auxiliary vector of the text to be regularized, wherein the semantic auxiliary vector is a common feature of the text to be regularized and other texts in the semantic dimension;

[0026] The text regularization model and the obtained vector are used to determine the regularized text corresponding to the text to be regularized.

[0027] Optionally, the using the text regularization model to obtain a context vector of each word in the text to be regularized, a semantic representation vector of the text to be regularized, and a semantic auxiliary vector of the text to be regularized includes:

[0028] Using the encoding module of the text regularization model, each word in the text to be regularized is encoded to obtain a context vector of each word in the text to be regularized;

[0029] Utilizing the semantic representation acquisition module of the text regularization model, based on the context vector of each word in the text to be regularized, a semantic representation vector of the text to be regularized is acquired;

[0030] Using the semantic representation generation module of the text regularization model, a semantic auxiliary vector of the text to be regularized is generated;

[0031] The step of using the text regularization model and the obtained vector to determine the regularized text corresponding to the text to be regularized includes:

[0032] The decoding module of the text regularization model is used to decode the context vector of each word in the text to be regularized, the semantic representation vector of the text to be regularized, and the semantic auxiliary vector of the text to be regularized to obtain the regularized text corresponding to the text to be regularized.

[0033] Optionally, the process of training the text regularization model using the training sample includes:

[0034] Using a text regularization model, obtaining a context vector of each word in each training text contained in the training sample, a semantic representation vector of each training text, and a posterior distribution of latent variables corresponding to each training text;

[0035] Using a text regularization model, based on the context vector of each word in each training text contained in the training sample and the semantic representation vector of each training text, obtain a text regularization result corresponding to each training text contained in the training sample;

[0036] The parameters of the text regularization model are updated according to the posterior distribution of latent variables corresponding to each training text contained in the training sample, the text regularization result corresponding to each training text, and the regularized text annotated by each training text.

[0037] Optionally, the process of training the text regularization model using the training sample further includes:

[0038] For each training text included in the training sample: using the text regularization model, based on the posterior distribution of latent variables corresponding to the training text, reconstructing the semantic representation vector of the training text to obtain a reconstructed vector corresponding to the training text, and using the part of the text regularization model used for reconstructing the semantic representation vector, based on the set prior distribution of latent variables, obtaining the semantic auxiliary vector of the training text;

[0039] The step of obtaining a text regularization result corresponding to each training text included in the training sample based on the context vector of each word in each training text included in the training sample and the semantic representation vector of each training text includes:

[0040] Based on the context vector of each word in each training text contained in the training sample, the semantic representation vector of each training text and the semantic auxiliary vector of each training text, a text regularization result corresponding to each training text contained in the training sample is obtained.

[0041] Optionally, updating parameters of the text regularization model according to the posterior distribution of latent variables corresponding to each training text included in the training sample, the text regularization result corresponding to each training text, and the regularized text annotated by each training text includes:

[0042] The parameters of the text regularization model are updated according to the posterior distribution of latent variables corresponding to each training text contained in the training sample, the text regularization result corresponding to each training text, the regularized text annotated by each training text, the semantic representation vector of each training text, the reconstruction vector corresponding to each training text and the set prior distribution of latent variables.

[0043] Optionally, the text regularization model is updated according to the hidden variable posterior distribution corresponding to each training text contained in the training sample, the text regularization result corresponding to each training text, the regularized text annotated by each training text, the semantic representation vector of each training text, the reconstruction vector corresponding to each training text and the set hidden variable prior distribution, including:

[0044] Determining, according to a posterior distribution of a latent variable corresponding to each training text included in the training sample, a first prediction loss capable of characterizing the semantic consistency of each training sample included in the training sample;

[0045] Determine, according to the text regularization result corresponding to each training text included in the training sample and the regularized text annotated by each training text, a second prediction loss capable of characterizing the expression consistency between the regularized text corresponding to each training sample included in the training sample and the annotated regularized text;

[0046] Determine, according to the semantic representation vector of each training text included in the training sample and the reconstruction vector corresponding to each training text, a third prediction loss capable of representing the reconstruction error of the semantic representation vector of each training text included in the training sample;

[0047] Determine, according to the latent variable posterior distribution corresponding to each training text included in the training sample and the set latent variable prior distribution, a fourth prediction loss that can characterize the degree of proximity between the latent variable posterior distribution of each training sample included in the training sample and the set latent variable prior distribution;

[0048] The parameters of the text regularization model are updated according to the first prediction loss, the second prediction loss, the third prediction loss and the fourth prediction loss.

[0049] A text regularization device, comprising: a text acquisition module and a text regularization module;

[0050] The text acquisition module is used to acquire the text to be regularized;

[0051] The text regularization module is used to regularize the text to be regularized using a pre-built text regularization model to obtain a regularized text corresponding to the text to be regularized;

[0052] Among them, the text regularization model is trained using training samples, and the training samples include multiple training texts with the same semantics, different expressions and annotated with the same regularized text. When the training samples are used to train the text regularization model, the semantic consistency of the multiple training texts contained in the training samples is constrained.

[0053] Optionally, the training objectives of the text regularization model include:

[0054] Make the latent variable posterior distributions corresponding to the multiple training texts included in the training sample consistent, wherein the latent variable posterior distribution is the posterior distribution of the latent variable that is obtained according to the corresponding training text and can represent the semantic space;

[0055] And, the regularized texts respectively corresponding to the multiple training texts included in the training sample are made consistent with the annotated regularized texts.

[0056] A text regularization device, comprising: a memory and a processor;

[0057] The memory is used to store programs;

[0058] The processor is used to execute the program to implement each step of the text regularization method described in any one of the above items.

[0059] A computer-readable storage medium having a computer program stored thereon, characterized in that when the computer program is executed by a processor, each step of the text regularization method described in any one of the above items is implemented.

[0060] It can be seen from the above scheme that the text regularization method, device, equipment and storage medium provided by the present application can use the pre-trained text regularization model to regularize the spoken text to be regularized into written text, that is, the text regularization method provided in the embodiment of the present application can realize text regularization. In addition, the regularization of the text in the present application is realized by using the pre-trained text regularization model. Since in the process of training the text regularization model with training samples, the semantic consistency of multiple training texts contained in the training samples is constrained, therefore, the trained text regularization model not only has the ability to regularize the spoken text into written text, but also has the ability to keep the semantics of the regularized text unchanged, that is, the text obtained after regularizing the text to be regularized using the text regularization model is consistent with the semantics of the text before regularization. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0062] Figure 1 A flowchart of a text regularization method provided in an embodiment of the present application;

[0063] Figure 2 A schematic diagram of a process for constructing a training text set provided in an embodiment of the present application;

[0064] Figure 3 A schematic diagram of a process for training a text regularization model using training samples provided in an embodiment of the present application;

[0065] Figure 4 A schematic diagram of the structure of a text regularization model provided in an embodiment of the present application;

[0066] Figure 5 A schematic diagram of the structure of a semantic representation generation module of a text regularization model provided in an embodiment of the present application;

[0067] Figure 6 A schematic diagram of the structure of a text regularization device provided in an embodiment of the present application;

[0068] Figure 7 A schematic diagram of the structure of the text regularization device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0069] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0070] In order to be able to regularize spoken text into written text, the applicant conducted research. The initial idea was: to pre-train a text regularization model using training text annotated with regularized text. During the training process, the model optimization goal was to minimize the cross entropy loss between the regularized text corresponding to the training text and the regularized text annotated with the training text. After the text regularization model was obtained through training, the text regularization model obtained through training was used to regularize the text to be regularized to obtain the regularized text corresponding to the text to be regularized.

[0071] During the research process, the applicant found that the above idea has defects: minimizing the cross entropy loss between the regularized text corresponding to the training text and the regularized text annotated with the training text as the model optimization goal can only constrain the gap between the regularized text corresponding to the training text and the regularized text annotated with the training text in terms of textual expression, and cannot ensure that the semantics of the regularized text and the text before regularization are consistent. For example, the original text is "Let my daughter talk to you about her good things", and the annotated text is "Let my daughter talk to you about her good things". Suppose there are two candidate results, candidate result 1 is "Let my daughter talk to you about her good things", and candidate result 2 is "Let my daughter talk to you about her good things". If the model is only based on cross entropy optimization, the model will select candidate result 2, but subjectively candidate result 1 is more acceptable. This is because, at the semantic level, the semantics of candidate result 1 is obviously more consistent with the original text than the semantics of candidate result 2.

[0072] In view of the defects of the above-mentioned ideas, the inventor of this case continued to study and finally proposed a text regularization method with better effect through continuous research. This text regularization method can be applied to any application scenario that requires text regularization. This text regularization method overcomes the defects of the above-mentioned ideas, that is, this text regularization method can not only regularize spoken text into written text, but also ensure that the regularized text is consistent with the text before regularization, that is, the semantics of the regularized text remains unchanged.

[0073] The text regularization method provided by the present application can be applied to electronic devices with data processing capabilities, which can be servers on the network side (can be a single server, or multiple servers or server clusters), or terminals used by the user side, such as PCs, notebooks, PADs, smart phones, etc. Next, the text regularization method provided by the present application is introduced through the following embodiments.

[0074] First embodiment

[0075] See also Figure 1 , shows a flow chart of a text regularization method provided in an embodiment of the present application, which may include:

[0076] Step S101: Obtain the text to be regularized.

[0077] The text to be regularized is a spoken text, for example, a speech recognition result of spoken voice data. The purpose of this application is to regularize the spoken text into a written text.

[0078] Step S102: regularize the text to be regularized using a pre-built text regularization model to obtain a regularized text corresponding to the text to be regularized.

[0079] Among them, the text regularization model is trained using a large number of training samples. Each training sample used to train the text regularization model includes multiple training texts with the same semantics but different expressions and annotated with the same regularized text. Optionally, the training sample includes two training texts with the same semantics but different expressions and annotated with the same regularized text. When the training samples are used to train the text regularization model, the semantic consistency of the multiple training texts contained in the training samples is constrained.

[0080] It should be noted that the training texts contained in the training samples are spoken texts, and the regularized texts annotated with the training texts are written texts. When the training samples are used to train the text regularization model, the multiple training texts contained in the training samples are input into the text regularization model together.

[0081] It should be noted that the first principle of text regularization is to perform appropriate regularization operations on the original spoken text while ensuring that the semantics of the regularized text remains unchanged.

[0082] In view of this, when training the text regularization model, this embodiment not only makes the regularized text corresponding to the training text contained in the training sample and the regularized text annotated by the training text consistent in textual expression, but also constrains the semantic consistency of multiple training texts contained in the training sample during the training process, thereby ensuring that the semantics of the regularized text remain unchanged.

[0083] The training objectives of the text regularization model in this embodiment include: making the posterior distributions of latent variables corresponding to the multiple training texts included in the training sample consistent, and making the regularized texts corresponding to the multiple training texts included in the training sample (the regularized text corresponding to the training text refers to the text obtained after regularizing the training text using the text regularization model) and the annotated regularized text consistent. Among them, the latent variable is an unobservable random variable. The latent variable in this application represents an abstract semantic space, and the posterior distribution of the latent variable is the posterior distribution of the latent variable that can characterize the semantic space obtained according to the corresponding training text.

[0084] Assuming that the training samples include two training texts, a first training text and a second training text, the training objectives of the text regularization model include: making the posterior distribution of latent variables corresponding to the first training text consistent with the posterior distribution of latent variables corresponding to the second training text, and making the regularized text corresponding to the first training text consistent with the regularized text annotated by the first training text, and making the regularized text corresponding to the second training text consistent with the regularized text annotated by the second training text.

[0085] It should be noted that in order to constrain the semantic consistency of each training text contained in the training sample, this embodiment proposes to obtain the posterior distribution of the latent variable that can characterize the semantic space according to each training text contained in the training sample, so as to obtain the posterior distribution of the latent variable corresponding to each training sample contained in the training sample, and then constrain the semantic consistency of each training sample contained in the training sample by making the posterior distribution of the latent variable corresponding to each training sample contained in the training sample tend to be consistent.

[0086] In this embodiment, there are multiple implementation methods for regularizing the text to be regularized using the pre-built text regularization model to obtain the regularized text corresponding to the text to be regularized:

[0087] In one possible implementation, a pre-built text regularization model can be first used to obtain the context vector of each word in the text to be regularized and the semantic representation vector of the text to be regularized (the semantic representation vector is a vector that can represent the semantics of the text to be regularized), and then the text regularization model, the context vector of each word in the text to be regularized, and the semantic representation vector of the text to be regularized are used to determine the regularized text corresponding to the text to be regularized.

[0088] In order to obtain a better regularization effect, in a possible implementation method, a pre-built text regularization model can be first used to obtain the context vector of each word in the text to be regularized, the semantic representation vector of the text to be regularized, and the semantic auxiliary vector of the text to be regularized, and then the text regularization model, the context vector of each word in the text to be regularized, the semantic representation vector of the text to be regularized, and the semantic auxiliary vector of the text to be regularized are used to determine the regularized text corresponding to the text to be regularized. Among them, the semantic auxiliary vector is the common feature of the text to be regularized and other texts in the semantic dimension. In this implementation method, the semantic auxiliary vector is introduced, and the semantic auxiliary vector is used to enhance the semantics of the text to be regularized, so that the text regularization model can better understand the semantics of the text to be regularized, thereby improving the text regularization effect.

[0089] The text regularization method provided in the embodiment of the present application can use a pre-constructed text regularization model to regularize the spoken text to be regularized into a written text, that is, the text regularization method provided in the embodiment of the present application can realize the regularization of the spoken text into a written text. In addition, the regularization of the text in the present application is realized by using a pre-trained text regularization model. In the process of training the text regularization model with training samples, the semantic consistency of multiple training texts contained in the training samples is constrained. Therefore, the trained text regularization model not only has the ability to regularize the spoken text into a written text, but also has the ability to keep the semantics of the regularized text unchanged, that is, the semantics of the text obtained after regularizing the text to be regularized by using the text regularization model remains unchanged.

[0090] Second embodiment

[0091] The above embodiment mentioned that the text regularization model used to regularize the text to be regularized is trained using training samples, and the training samples are trained with multiple training texts with the same semantics but different expressions and annotated with the same regularized text. This embodiment introduces the process of obtaining the training samples.

[0092] In this embodiment, multiple training texts with the same semantics, different expressions, and annotated with the same regularized text can be obtained from a pre-constructed training text set (for example, two training texts with the same semantics, different expressions, and annotated with the same regularized text) to form training samples.

[0093] The training text set includes multiple groups of training texts, each group of training texts includes multiple training texts with the same semantics, different expressions, and annotated with the same regularized texts, and different groups of training texts have different semantics, different expressions, and different annotated regularized texts.

[0094] From the above content, it can be seen that the training samples are composed of training texts obtained from a pre-constructed training text set. Next, the process of constructing the training text set is introduced.

[0095] See also Figure 2 , showing a schematic diagram of the process of constructing a training text set, which may include:

[0096] Step S201: Obtain an original training text set.

[0097] Among them, the original training text set includes multiple original training texts annotated with regularized texts. The original training texts in the original training text set are obtained by annotating directly collected texts (such as recognition results of spoken speech, spoken description texts of characters in novels, etc.).

[0098] Step S202: for each original training text T in the original training text set i ,implement:

[0099] Step S2021: Based on the original training text T i Expand at least one text with the same semantics, different expressions, and the same annotations as the original training text Ti to obtain the original training text T i The corresponding extended training text T i ′.

[0100] Specifically, based on the original training text T i Expand at least one of the original training texts T i The process of texts with the same semantics, different expressions and the same annotations may include: iThe annotated regularized text is converted into the original training text T i Processed into the original training text T i Texts with the same semantics but different expressions, and annotating the original training text T for the processed text i The annotated regularized text is obtained to obtain the original training text T i The corresponding extended training text T i ′.

[0101] Among them, according to the original training text T i The annotated regularized text is converted into the original training text T i Processed into the original training text T i The process of texts with the same meaning but different expressions includes:

[0102] Step a1: respectively analyze the original training text T i and the original training text T i The annotated regularized text is segmented to obtain the original training text T i The corresponding word sequence and the original training text T i The word sequence corresponding to the annotated regularized text.

[0103] For example, the original training text T i For example, for “this high-tech enterprise has twenty-seven people, right?”, the annotated regularized text is “there are twenty-seven people in a high-tech enterprise”, and the word sequence obtained after word segmentation for “this high-tech enterprise has twenty-seven people, right?” is “this / high / is / high-tech / enterprise / has / twenty-seven / people / right?”, and the word sequence obtained after word segmentation for “there are twenty-seven people in a high-tech enterprise” is “high-tech / enterprise / has / twenty-seven / people”.

[0104] Step a2: By converting the original training text T i The corresponding word sequence and the original training text T i Align the word sequences corresponding to the annotated regularized text to determine the original training text T i Words that can be deleted.

[0105] The following table shows the alignment of the two word sequences in the above example: "This / high / is / High-tech / enterprise / has / twenty-seven / people / bar" and "High-tech / enterprise / has / twenty-seven / people":

[0106] Table 1 Alignment results of two word sequences and words that can be deleted based on the alignment results

[0107] Serial number 1 2 3 4 5 6 7 8 9 10 Aligned original training text this high that is High-tech enterprise have XXVII indivual people Bar Aligned regularized text High-tech enterprise have XXVII indivual people Label D D D K K K K K K D

[0108] Among them, "D" in the above table represents the words that need to be deleted, and "K" represents the words that need to be retained, that is, the words "this", "high", "that is", and "bar" in the original training text are the words that need to be deleted.

[0109] It should be noted that by converting the original training text T i The corresponding word sequence and the original training text T i The word sequences corresponding to the annotated regularized text are aligned, and the deletable words determined are words that do not affect the semantics of the text.

[0110] Step a3: The original training text T i Delete some or all of the words that can be deleted in the training text T i Texts with the same semantics but different expressions are compared with the original training text T i Text annotations with the same semantics but different expressions Original training text T i The annotated regularized text is used to obtain the original training text T i The corresponding extended training text T i ′.

[0111] Among them, the original training text T i Delete some or all of the words that can be deleted from the original training text T i The process of texts with the same semantics but different expressions may include: i The words that can be deleted and the number of words that can be deleted are determined, and a deletion strategy or multiple different deletion strategies are determined. Each deletion strategy is used to indicate the number of words that are actually deleted and the words that are actually deleted among the words that can be deleted. The original training text T is processed according to each deletion strategy. i Delete the original training text T i Texts with the same meaning but different expressions.

[0112] Assume that step a2 determines that there are N words that can be deleted, namely c1, c2, ..., c N Optionally, N groups of deletion strategies can be determined, each group of deletion strategies includes at least one deletion strategy, and the first group of deletion strategies (strategies for deleting a word) is {(1, c1), (1, c2), ..., (1, c N )}, the second group of deletion strategies (strategy for deleting two words) is {(2, c1, c2), (2, c1, c3), ...}, ..., the Nth group of deletion strategies (strategy for deleting N words) is {(N, c1, c2, ..., c N )}, where the deletion strategy (2, c1, c3) means that the number of words actually deleted is 2, and the words actually deleted are c1 and c3. Other strategies are similar.

[0113] For the above original training text "This Gao is a high-tech enterprise with 27 people", the deletable words include "this", "high", "is", and "is", so the following four groups of deletion strategies can be determined: {(1, this), (1, high), (1, is), (1, is)}, {(2, this, high), (2, this, is), (2, this, is), (2, high, is), (2, high, is), (2, is, is)}, {(3, this, high, is), (3, this, high, is), (3, high, is, is), (3, this, is, is)}, {(4, this, high, is)}. After deleting the original training text "This Gao is a high-tech enterprise with 27 people", the following text can be obtained:

[0114] Table 2 Texts with the same semantics but different expressions as the original training texts

[0115]

[0116] For the original training text "This Gao means a high-tech enterprise has 27 people", after deleting "This Gao means a high-tech enterprise has 27 people" according to 15 deletion strategies, 15 texts with the same semantics and different expressions as the original training text "This Gao means a high-tech enterprise has 27 people" can be obtained, namely "Gao means a high-tech enterprise has 27 people", "This is a high-tech enterprise has 27 people", ..., "It is a high-tech enterprise with 27 people", "Gao Gaoke Enterprise has 27 people", ..., "It is a high-tech enterprise with 27 people", "It is a high-tech enterprise with 27 people", ..., "It is a high-tech enterprise with 27 people". After obtaining the text with the same semantics and different expressions as the original training text "This Gao means a high-tech enterprise has 27 people", the text with the same semantics and different expressions as the original training text "This Gao means a high-tech enterprise has 27 people" is annotated as "High-tech enterprise has 27 people". It should be noted that the original training text can also be deleted according to some of the possible deletion strategies.

[0117] Step S2022: The original training text T i The extended training text T corresponding to the original training text i ′ forms a set of training texts.

[0118] Through step S202, multiple groups of training texts can be finally obtained, each group of training texts includes multiple training texts with the same semantics, different expressions, and annotated with the same regularized text. After the multiple groups of training texts are obtained, the multiple groups of training texts form a training text set. It should be noted that each training sample used to train the text regularization model is composed of multiple training texts obtained from a group of training texts in the training text set. For example, two training texts can be randomly obtained from a group of training texts in the training text set to form a training sample.

[0119] Third embodiment

[0120] The above embodiment provides a method for obtaining training samples. After obtaining the training samples in the manner provided in the above embodiment, the obtained training samples can be used to train the text regularization model. This embodiment focuses on introducing the process of using the training samples to train the text regularization model.

[0121] See also Figure 3 , showing a schematic diagram of a process of training a text regularization model using training samples, which may include:

[0122] Step S301: using a text regularization model, obtaining a context vector of each word in each training text contained in a training sample and a semantic representation vector of each training text.

[0123] Optional, such as Figure 4 As shown, the text regularization model in this embodiment may include an encoding module 401 and a semantic representation acquisition module 402. Each training text contained in the training sample is input into the encoding module 401 of the text regularization model for encoding, and a context vector of each word in each training text contained in the training sample can be obtained. The context vector of each word in each training text contained in the training sample is input into the semantic representation acquisition module 402, and the semantic representation vector of each training text contained in the training sample can be obtained.

[0124] Figure 4 The schematic diagram of processing two training texts is shown. Assume that the training sample includes two training texts, X and X', respectively. X and X' are input into the encoding module 401 of the text regularization model. The encoding module 401 outputs the hidden layer features with context information of each word in X (i.e., the context vector mentioned above). (h i represents the context vector of the i-th word in the training text X), and the hidden features with context information for each word in X' (i.e. the context vector mentioned above) (h' i represents the context vector of the i-th word in the training text X'), N x and N x'Respectively represent the number of words contained in the two training texts X and X', and H X and H X The input semantic representation acquisition module 402 can obtain the semantic representation vector x of the training text X and the semantic representation vector x' of the training text X'.

[0125] Step S302: For each training text included in the training sample: use the text regularization model to obtain the latent variable posterior distribution corresponding to the training text, and reconstruct the semantic representation vector of the training text based on the latent variable posterior distribution corresponding to the training text to obtain the reconstructed vector corresponding to the training text, and use the part of the text regularization model used to reconstruct the semantic representation vector to obtain the semantic auxiliary vector of the training text based on the set latent variable prior distribution.

[0126] like Figure 4 As shown, the text regularization model can also include a semantic representation generation module 403. For each training text included in the training sample, the semantic representation generation module 403 of the text regularization model can be used to obtain the latent variable posterior distribution corresponding to the training text, and the semantic representation vector of the training text is reconstructed based on the latent variable posterior distribution corresponding to the training text to obtain the reconstructed vector corresponding to the training text, and the semantic representation generation module 403 of the text regularization model is used to obtain the semantic auxiliary vector of the training text based on the set latent variable prior distribution. It should be noted that the semantic representation generation module 403 is a latent variable generative model (Latent Variable Generative Model), and the present application uses latent variables to automatically learn semantic information in natural language understanding.

[0127] Further, such as Figure 5 As shown, the semantic representation generation module 403 of the text regularization model may include a posterior distribution acquisition submodule 501a, a posterior sample acquisition submodule 502a, a priori distribution acquisition submodule 501b, a priori sample acquisition submodule 502b and a decoding submodule 503.

[0128] For each training text included in the training sample: using the posterior distribution acquisition submodule 501a, based on the semantic representation vector of the training text, the latent variable posterior distribution corresponding to the training text is obtained, that is, the semantic representation vector of the training text is input into the posterior distribution acquisition submodule 501a, and the posterior distribution acquisition submodule 501a obtains the latent variable posterior distribution based on the input semantic representation vector and outputs it; using the posterior sample acquisition submodule 502a to sample the latent variable posterior distribution corresponding to the training text to obtain the posterior sample corresponding to the training text; using the decoding submodule 503, based on the posterior sample corresponding to the training text, the semantic representation vector of the training text is reconstructed to obtain the reconstructed vector corresponding to the training text; using the prior distribution acquisition submodule 501b to obtain the set latent variable prior distribution; using the prior sample acquisition submodule 502b to sample the set latent variable prior distribution to obtain the prior sample corresponding to the training sample; input the prior sample corresponding to the training sample into the decoding submodule 503, and use the vector output by the decoding submodule 503 as the semantic auxiliary vector of the training sample.

[0129] It should be noted that the latent variables in the present application are unobservable random variables, which represent abstract semantic space. The posterior distribution acquisition submodule 501a infers the posterior distribution of the latent variables based on the semantic representation vector of the training text. The posterior distribution of the latent variables represents the distribution of the latent variables inferred after observing the semantic representation vector of the training text, while the prior distribution of the latent variables represents the distribution of the abstract semantic space that naturally conforms to a certain rule, which is prior knowledge. The set prior distribution of the latent variables can be but is not limited to standard normal distribution, Bernoulli distribution, etc.

[0130] It should also be noted that the posterior distribution acquisition submodule 501a and the posterior sample acquisition submodule 502a are only used in the training phase. In the actual application phase, only the prior distribution acquisition submodule 501b, the prior sample acquisition submodule 502b and the decoding submodule 503 are used.

[0131] Step S303: using the text regularization model, based on the context vector of each word in each training text contained in the training sample, the semantic representation vector of each training text and the semantic auxiliary vector of each training text, obtain the text regularization result corresponding to each training text contained in the training sample.

[0132] like Figure 4As shown, the text regularization model also includes a decoding module 404. After obtaining the context vector of each word in each training text contained in the training sample, the semantic representation vector of each training text and the semantic auxiliary vector of each training text, these vectors are input into the decoding module 404 for decoding. The decoding module 404 outputs the text regularization result corresponding to each training text contained in the training sample.

[0133] Step S304: Update the parameters of the text regularization model according to the posterior distribution of latent variables corresponding to each training text contained in the training sample, the text regularization result corresponding to each training text, the regularized text annotated by each training text, the semantic representation vector of each training text, the reconstruction vector corresponding to each training text and the set prior distribution of latent variables.

[0134] Specifically, the implementation process of step S304 may include:

[0135] Step S3041: Determine a first prediction loss that can characterize the semantic consistency of each training text included in the training sample according to the latent variable posterior distribution corresponding to each training text included in the training sample.

[0136] It should be noted that the first prediction loss is used to keep the semantics of the regularized text output by the text regularization model unchanged. Assuming that the training sample includes two training texts X and X', the first prediction loss can be expressed as:

[0137] l 语义 (x,x')=-KL[q(z|x)||q(z'|x')] (1)

[0138] Among them, q(z|x) represents the posterior distribution of the latent variable corresponding to the training text X, that is, the posterior distribution of the latent variable z after the semantic representation vector x of the training text X is known. Similarly, q(z'|x') represents the posterior distribution of the latent variable corresponding to the training text X', that is, the posterior distribution of the latent variable z after the semantic representation vector x' of the training text X' is known. KL[q(z|x)||q(z'|x')] represents the divergence of the posterior distribution of the latent variables corresponding to the two training texts. It should be noted that if the training sample includes more than two training texts, all the training texts contained in the training sample can be combined in pairs, and for each combination, the prediction loss is calculated according to the above formula.

[0139] Step S3042: Determine a second prediction loss that can characterize the expression consistency between the regularized text corresponding to each training sample contained in the training sample and the annotated regularized text based on the text regularization result corresponding to each training text contained in the training sample and the annotated regularized text.

[0140] Optionally, the second prediction loss may be a cross entropy loss. The method for determining the cross entropy loss is a prior art and will not be described in detail in this embodiment.

[0141] Step S3043: Determine a third prediction loss that can characterize the reconstruction error of the semantic representation vector of each training text included in the training sample according to the semantic representation vector of each training text included in the training sample and the reconstruction vector corresponding to each training text.

[0142] Optionally, for each training text included in the training sample, the error between the reconstructed vector corresponding to the training sample and the semantic representation vector of the training text may be calculated as the third prediction loss.

[0143] Step S3044: Determine a fourth prediction loss that can characterize the degree of proximity between the latent variable posterior distribution of each training sample contained in the training sample and the set latent variable prior distribution according to the latent variable posterior distribution corresponding to each training text contained in the training sample and the set latent variable prior distribution.

[0144] Assuming that the latent variable posterior distribution of a training sample X is represented by q(z|x), and the set latent variable prior distribution is represented by p(z), then the degree of closeness between the latent variable posterior distribution q(z|x) of the training sample X and the set latent variable prior distribution p(z) can be represented by the divergence KL[q(z|x)||p(z)] between the latent variable posterior distribution q(z|x) of the training sample and the set latent variable prior distribution p(z).

[0145] It should be noted that this embodiment does not limit the execution order of steps S3041 to S3044, and these four steps can be performed in any order.

[0146] Step S3045: update the parameters of the text regularization model according to the first prediction loss, the second prediction loss, the third prediction loss and the fourth prediction loss.

[0147] The text regularization model is iteratively trained multiple times according to the above process until the training end conditions are met. The model obtained after the training is the constructed text regularization model. The text regularization model can not only regularize the spoken text into written text, but also ensure that the regularized text is semantically consistent with the text before regularization, that is, the semantics of the regularized text remains unchanged.

[0148] After constructing the text regularization model, it can be used to regularize the text to be regularized. Specifically, the encoding module of the text regularization model can be used to encode each word in the text to be regularized to obtain the context vector of each word in the text to be regularized. Then, the semantic representation acquisition module of the text regularization model can be used to obtain the semantic representation vector of the text to be regularized based on the context vector of each word in the text to be regularized. The semantic representation generation module of the text regularization model can be used to generate the semantic auxiliary vector of the text to be regularized. Finally, the decoding module of the text regularization model can be used to decode the context vector of each word in the text to be regularized, the semantic representation vector of the text to be regularized, and the semantic auxiliary vector of the text to be regularized to obtain the regularized text corresponding to the text to be regularized.

[0149] Among them, the process of generating a semantic auxiliary vector of the text to be regularized by using the semantic representation generation module of the text regularization model includes: using the prior distribution acquisition submodule in the semantic representation generation module to obtain the set prior distribution of latent variables, using the prior sample acquisition submodule in the semantic representation generation module to sample the set prior distribution of latent variables to obtain the prior samples corresponding to the text to be regularized, and using the decoding submodule in the semantic representation generation module to determine the semantic auxiliary vector of the text to be regularized based on the prior samples corresponding to the text to be regularized.

[0150] It should be noted that when using the text regularization model constructed in the above manner to regularize the text to be regularized, it is also possible not to obtain the semantic auxiliary vector of the text to be regularized, and only obtain the regularized text corresponding to the text to be regularized based on the context vector of each word in the text to be regularized and the semantic representation vector of the text to be regularized.

[0151] In addition, this embodiment does not limit the training method of the text regularization model to the above method only, and other methods can also be used to train the text regularization model. For example, the text regularization model can be trained in the following manner:

[0152] Step b1: Using the text regularization model, obtain the context vector of each word in each training text contained in the training sample, the semantic representation vector of each training text, and the latent variable posterior distribution corresponding to each training text.

[0153] The implementation method of using the text regularization model to obtain the context vector of each word in each training text contained in the training sample, the semantic representation vector of each training text, and the posterior distribution of the latent variables corresponding to each training text can be referred to the relevant part of the first training method mentioned above, and this embodiment will not be repeated here.

[0154] Step b2: using the text regularization model, based on the context vector of each word in each training text contained in the training sample and the semantic representation vector of each training text, obtain the text regularization result corresponding to each training text contained in the training sample.

[0155] Step b3: update the parameters of the text regularization model according to the posterior distribution of the latent variables corresponding to each training text contained in the training sample, the text regularization result corresponding to each training text and the regularized text annotated by each training text.

[0156] Specifically, according to the posterior distribution of latent variables corresponding to each training text contained in the training sample, a first prediction loss that can characterize the semantic consistency of each training sample contained in the training sample is determined; according to the text regularization result corresponding to each training text contained in the training sample and the regularized text annotated by each training text, a second prediction loss that can characterize the expression consistency between the text regularization result corresponding to each training sample contained in the training sample and the annotated regularized text is determined; and the parameters of the text regularization model are updated according to the first prediction loss and the second prediction loss.

[0157] The text regularization model is iteratively trained multiple times according to the process of steps b1 to b3 above until the training end conditions are met. The model obtained after the training is the constructed text regularization model. The text regularization model can not only regularize the spoken text into written text, but also ensure that the regularized text is semantically consistent with the text before regularization, that is, the semantics of the regularized text remains unchanged.

[0158] After constructing the text regularization model through the second training method mentioned above, the encoding module of the text regularization model can be used to encode each word in the text to be regularized to obtain the context vector of each word in the text to be regularized. Then, the semantic representation acquisition module of the text regularization model can be used to obtain the semantic representation vector of the text to be regularized based on the context vector of each word in the text to be regularized. Finally, the decoding module of the text regularization model can be used to decode the context vector of each word in the text to be regularized and the semantic representation vector of the text to be regularized to obtain the regularized text corresponding to the text to be regularized.

[0159] Fourth embodiment

[0160] The embodiment of the present application also provides a text regularization device. The text regularization device provided in the embodiment of the present application is described below. The text regularization device described below and the text regularization method described above can be referenced to each other.

[0161] See also Figure 6 , showing a schematic diagram of the structure of the text regularization device provided in an embodiment of the present application, which may include: a text acquisition module 601 and a text regularization module 602.

[0162] The text acquisition module 601 is used to acquire the text to be regularized.

[0163] The text regularization module 602 is used to regularize the text to be regularized using a pre-built text regularization model to obtain a regularized text corresponding to the text to be regularized.

[0164] Among them, the text regularization model is trained using training samples, and the training samples include multiple training texts with the same semantics, different expressions and annotated with the same regularized text. When the training samples are used to train the text regularization model, the semantic consistency of the multiple training texts contained in the training samples is constrained.

[0165] Optionally, the training objectives of the text regularization model include:

[0166] Make the latent variable posterior distributions corresponding to the multiple training texts included in the training sample consistent, wherein the latent variable posterior distribution is the posterior distribution of the latent variable that is obtained according to the corresponding training text and can represent the semantic space;

[0167] And, the regularized texts respectively corresponding to the multiple training texts included in the training sample are made consistent with the annotated regularized texts.

[0168] Optionally, the training texts included in the training samples are obtained from a pre-constructed training text set. The text regularization device provided in the embodiment of the present application may also include: a training text set construction module.

[0169] The training text set construction module may include: an original training text set acquisition submodule, a training text group acquisition submodule and a training text set acquisition submodule.

[0170] An original training text set acquisition submodule is used to acquire an original training text set, wherein the original training text set includes a plurality of original training texts annotated with regularized texts;

[0171] A training text group acquisition submodule is used for: for each original training text in the original training text set: based on the regularized text annotated by the original training text, processing the original training text into a text with the same semantics as the original training text but different expressions, and annotating the processed text with the regularized text annotated by the original training text to obtain an extended training text corresponding to the original training text, and forming a group of training texts by the original training text and the extended training text corresponding to the original training text;

[0172] The training text set acquisition submodule is used to form the training text set from the obtained multiple groups of training texts.

[0173] Optionally, when the training text group acquisition submodule processes the original training text into a text with the same semantics but different expressions as the original training text based on the regularized text annotated by the original training text, it is specifically used to:

[0174] The original training text and the regularized text annotated with the original training text are respectively segmented to obtain two word sequences; the deletable words in the original training text are determined by aligning the two word sequences; some or all of the deletable words in the original training text are deleted to obtain a text with the same semantics but different expressions as the original training text.

[0175] Optionally, when the training text group acquisition submodule deletes some or all of the deletable words in the original training text to obtain a text with the same semantics but different expressions as the original training text, it is specifically used to:

[0176] According to the deletable words and the number of deletable words in the original training text, determine a deletion strategy or multiple different deletion strategies, wherein each deletion strategy is used to indicate the number of words actually deleted and the words actually deleted among the deletable words; the original training text is deleted according to the determined deletion strategy to obtain one or more texts with the same semantics but different expressions as the original training text.

[0177] Optionally, the text regularization module 602 is specifically used for:

[0178] Using the text regularization model, obtain the context vector of each word in the text to be regularized and the semantic representation vector of the text to be regularized; or, using the text regularization model, obtain the context vector of each word in the text to be regularized, the semantic representation vector of the text to be regularized and the semantic auxiliary vector of the text to be regularized, wherein the semantic auxiliary vector is the common feature of the text to be regularized and other texts in the semantic dimension; using the text regularization model and the obtained vector, determine the regularized text corresponding to the text to be regularized.

[0179] Optionally, the text regularization module 602 includes: an encoding module, a semantic representation acquisition module, a semantic representation generation module and a decoding module.

[0180] The encoding module is used to encode each word in the text to be regularized to obtain a context vector for each word in the text to be regularized.

[0181] The semantic representation acquisition module is used to utilize the semantic representation acquisition module of the text regularization model to acquire the semantic representation vector of the text to be regularized based on the context vector of each word in the text to be regularized.

[0182] The semantic representation generation module is used to generate a semantic auxiliary vector of the text to be regularized by using the semantic representation generation module of the text regularization model.

[0183] The decoding module decodes the context vector of each word in the text to be regularized, the semantic representation vector of the text to be regularized, and the semantic auxiliary vector of the text to be regularized to obtain the regularized text corresponding to the text to be regularized.

[0184] Optionally, the text regularization device provided in the embodiment of the present application may also include: a model training module.

[0185] The model training module is used to:

[0186] Utilizing a text regularization model, the context vector of each word in each training text contained in the training sample, the semantic representation vector of each training text, and the posterior distribution of latent variables corresponding to each training text are obtained; utilizing a text regularization model, based on the context vector of each word in each training text contained in the training sample and the semantic representation vector of each training text, the text regularization result corresponding to each training text contained in the training sample is obtained; and the parameters of the text regularization model are updated according to the posterior distribution of latent variables corresponding to each training text contained in the training sample, the text regularization result corresponding to each training text, and the regularized text annotated by each training text.

[0187] Optionally, the model training module is further used to:

[0188] For each training text included in the training sample: using the text regularization model, based on the posterior distribution of latent variables corresponding to the training text, reconstructing the semantic representation vector of the training text to obtain a reconstructed vector corresponding to the training text, and using the part of the text regularization model used for reconstructing the semantic representation vector, based on the set prior distribution of latent variables, obtaining the semantic auxiliary vector of the training text;

[0189] The model training module is specifically used to obtain the text regularization result corresponding to each training text contained in the training sample based on the context vector of each word in each training text contained in the training sample and the semantic representation vector of each training text:

[0190] Based on the context vector of each word in each training text contained in the training sample, the semantic representation vector of each training text and the semantic auxiliary vector of each training text, a text regularization result corresponding to each training text contained in the training sample is obtained.

[0191] Optionally, when the model training module updates the parameters of the text regularization model according to the posterior distribution of the latent variables corresponding to each training text contained in the training sample, the text regularization result corresponding to each training text, and the regularized text annotated by each training text, the model training module is specifically used to:

[0192] The parameters of the text regularization model are updated according to the posterior distribution of latent variables corresponding to each training text contained in the training sample, the text regularization result corresponding to each training text, the regularized text annotated by each training text, the semantic representation vector of each training text, the reconstruction vector corresponding to each training text and the set prior distribution of latent variables.

[0193] Optionally, when the model training module updates the parameters of the text regularization model according to the hidden variable posterior distribution corresponding to each training text contained in the training sample, the text regularization result corresponding to each training text, the regularized text annotated by each training text, the semantic representation vector of each training text, the reconstruction vector corresponding to each training text and the set hidden variable prior distribution, it is specifically used to:

[0194] According to the posterior distribution of latent variables corresponding to each training text contained in the training sample, a first prediction loss capable of characterizing the semantic consistency of each training sample contained in the training sample is determined; according to the text regularization result corresponding to each training text contained in the training sample and the regularized text annotated by each training text, a second prediction loss capable of characterizing the expression consistency between the regularized text corresponding to each training sample contained in the training sample and the annotated regularized text is determined; according to the semantic representation vector of each training text contained in the training sample and the reconstruction vector corresponding to each training text, a third prediction loss capable of characterizing the reconstruction error of the semantic representation vector of each training text contained in the training sample is determined; according to the posterior distribution of latent variables corresponding to each training text contained in the training sample and the set prior distribution of latent variables, a fourth prediction loss capable of characterizing the degree of proximity between the posterior distribution of latent variables of each training sample contained in the training sample and the set prior distribution of latent variables is determined;

[0195] The parameters of the text regularization model are updated according to the first prediction loss, the second prediction loss, the third prediction loss and the fourth prediction loss.

[0196] The text regularization device provided in the embodiment of the present application can use a pre-constructed text regularization model to regularize the spoken text to be regularized into a written text, that is, the text regularization device provided in the embodiment of the present application can realize the regularization of the spoken text into a written text. In addition, the regularization of the text in the present application is realized by using a pre-trained text regularization model. In the process of training the text regularization model with training samples, the semantic consistency of multiple training texts contained in the training samples is constrained. Therefore, the trained text regularization model not only has the ability to regularize the spoken text into a written text, but also has the ability to keep the semantics of the regularized text unchanged, that is, the semantics of the text obtained after regularizing the text to be regularized by using the text regularization model remains unchanged.

[0197] Fifth embodiment

[0198] The present application also provides a text regularization device, see Figure 7 , showing a schematic diagram of the structure of the text regularization device, the text regularization device may include: at least one processor 701, at least one communication interface 702, at least one memory 703 and at least one communication bus 704;

[0199] In the embodiment of the present application, the number of the processor 701, the communication interface 702, the memory 703, and the communication bus 704 is at least one, and the processor 701, the communication interface 702, and the memory 703 communicate with each other through the communication bus 704;

[0200] The processor 701 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention;

[0201] The memory 603 may include a high-speed RAM memory, and may also include a non-volatile memory, such as at least one disk memory;

[0202] The memory stores a program, and the processor can call the program stored in the memory, wherein the program is used to:

[0203] Get the text to be regulated;

[0204] Regularizing the text to be regularized using a pre-built text regularization model to obtain a regularized text corresponding to the text to be regularized;

[0205] Among them, the text regularization model is trained using training samples, and the training samples include multiple training texts with the same semantics, different expressions and annotated with the same regularized text. When the training samples are used to train the text regularization model, the semantic consistency of the multiple training texts contained in the training samples is constrained.

[0206] Optionally, the detailed functions and extended functions of the program may refer to the above description.

[0207] Sixth embodiment

[0208] The embodiment of the present application further provides a readable storage medium, which may store a program suitable for execution by a processor, wherein the program is used to:

[0209] Get the text to be regulated;

[0210] Regularizing the text to be regularized using a pre-built text regularization model to obtain a regularized text corresponding to the text to be regularized;

[0211] Among them, the text regularization model is trained using training samples, and the training samples include multiple training texts with the same semantics, different expressions and annotated with the same regularized text. When the training samples are used to train the text regularization model, the semantic consistency of the multiple training texts contained in the training samples is constrained.

[0212] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0213] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0214] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A text regularization method, characterized in that: include: Get the text to be regulated; Regularizing the text to be regularized using a pre-built text regularization model to obtain a regularized text corresponding to the text to be regularized; The text regularization model is trained using training samples, wherein the training samples include multiple training texts with the same semantics but different expressions and annotated with the same regularized texts. When the training samples are used to train the text regularization model, the semantic consistency of the multiple training texts included in the training samples is constrained; The training objectives of the text regularization model include: Make the latent variable posterior distributions corresponding to the multiple training texts included in the training sample consistent, wherein the latent variable posterior distribution is the posterior distribution of the latent variable that is obtained according to the corresponding training text and can represent the semantic space; And, the regularized texts respectively corresponding to the multiple training texts included in the training sample are made consistent with the annotated regularized texts.

2. The text regularization method according to claim 1, characterized in that: The training texts included in the training samples are obtained from a pre-constructed training text set, and the construction process of the training text set includes: Acquire an original training text set, wherein the original training text set includes a plurality of original training texts annotated with regularized texts; For each original training text in the original training text set: based on the regularized text annotated by the original training text, the original training text is processed into a text with the same semantics as the original training text but different expressions, and the processed text is annotated with the regularized text annotated by the original training text to obtain an extended training text corresponding to the original training text, and the original training text and the extended training text corresponding to the original training text form a set of training texts; The training text set is composed of a plurality of sets of obtained training texts.

3. The text regularization method according to claim 2, characterized in that: The regularized text annotated based on the original training text, processing the original training text into a text with the same semantics but different expressions as the original training text, includes: Performing word segmentation processing on the original training text and the regularized text annotated by the original training text respectively to obtain two word sequences; Determining the deletable words in the original training text by aligning the two word sequences; Some or all of the deletable words in the original training text are deleted to obtain a text with the same semantics but different expressions as the original training text.

4. The text regularization method according to claim 3, characterized in that: The method of deleting some or all of the deletable words in the original training text to obtain a text with the same semantics but different expressions as the original training text includes: Determining a deletion strategy or a plurality of different deletion strategies according to the deletable words and the number of the deletable words in the original training text, wherein each deletion strategy is used to indicate the number of words actually deleted and the words actually deleted from the deletable words; The original training text is deleted according to the determined deletion strategy to obtain one or more texts with the same semantics but different expressions as the original training text.

5. The text regularization method according to claim 1, characterized in that: The step of regularizing the text to be regularized by using a pre-built text regularization model to obtain a regularized text corresponding to the text to be regularized includes: Using the text regularization model, obtain the context vector of each word in the text to be regularized and the semantic representation vector of the text to be regularized; or, using the text regularization model, obtain the context vector of each word in the text to be regularized, the semantic representation vector of the text to be regularized and the semantic auxiliary vector of the text to be regularized, wherein the semantic auxiliary vector is a common feature of the text to be regularized and other texts in the semantic dimension; The text regularization model and the obtained vector are used to determine the regularized text corresponding to the text to be regularized.

6. The text regularization method according to claim 5, characterized in that: The method of using the text regularization model to obtain a context vector of each word in the text to be regularized, a semantic representation vector of the text to be regularized, and a semantic auxiliary vector of the text to be regularized includes: Using the encoding module of the text regularization model, each word in the text to be regularized is encoded to obtain a context vector of each word in the text to be regularized; Utilizing the semantic representation acquisition module of the text regularization model, based on the context vector of each word in the text to be regularized, a semantic representation vector of the text to be regularized is acquired; Using the semantic representation generation module of the text regularization model, a semantic auxiliary vector of the text to be regularized is generated; The step of using the text regularization model and the obtained vector to determine the regularized text corresponding to the text to be regularized includes: The decoding module of the text regularization model is used to decode the context vector of each word in the text to be regularized, the semantic representation vector of the text to be regularized, and the semantic auxiliary vector of the text to be regularized to obtain the regularized text corresponding to the text to be regularized.

7. The text regularization method according to claim 1, characterized in that: The process of training the text regularization model using the training sample includes: Using a text regularization model, obtaining a context vector of each word in each training text contained in the training sample, a semantic representation vector of each training text, and a posterior distribution of latent variables corresponding to each training text; Using a text regularization model, based on the context vector of each word in each training text contained in the training sample and the semantic representation vector of each training text, obtain a text regularization result corresponding to each training text contained in the training sample; The parameters of the text regularization model are updated according to the posterior distribution of latent variables corresponding to each training text contained in the training sample, the text regularization result corresponding to each training text, and the regularized text annotated by each training text.

8. The text regularization method according to claim 7, characterized in that: The process of using the training sample to train the text regularization model also includes: For each training text included in the training sample: using the text regularization model, based on the posterior distribution of latent variables corresponding to the training text, reconstructing the semantic representation vector of the training text to obtain a reconstructed vector corresponding to the training text, and using the part of the text regularization model used for reconstructing the semantic representation vector, based on the set prior distribution of latent variables, obtaining the semantic auxiliary vector of the training text; The step of obtaining a text regularization result corresponding to each training text included in the training sample based on the context vector of each word in each training text included in the training sample and the semantic representation vector of each training text includes: Based on the context vector of each word in each training text contained in the training sample, the semantic representation vector of each training text and the semantic auxiliary vector of each training text, a text regularization result corresponding to each training text contained in the training sample is obtained.

9. The text regularization method according to claim 8, characterized in that: The updating of parameters of the text regularization model according to the hidden variable posterior distribution corresponding to each training text contained in the training sample, the text regularization result corresponding to each training text and the regularized text annotated by each training text includes: The parameters of the text regularization model are updated according to the posterior distribution of latent variables corresponding to each training text contained in the training sample, the text regularization result corresponding to each training text, the regularized text annotated by each training text, the semantic representation vector of each training text, the reconstruction vector corresponding to each training text and the set prior distribution of latent variables.

10. The text regularization method according to claim 9, characterized in that: The method updates the parameters of the text regularization model according to the hidden variable posterior distribution corresponding to each training text included in the training sample, the text regularization result corresponding to each training text, the regularized text annotated by each training text, the semantic representation vector of each training text, the reconstruction vector corresponding to each training text and the set hidden variable prior distribution, including: Determining, according to a posterior distribution of a latent variable corresponding to each training text included in the training sample, a first prediction loss capable of characterizing the semantic consistency of each training sample included in the training sample; Determine, according to the text regularization result corresponding to each training text included in the training sample and the regularized text annotated by each training text, a second prediction loss capable of characterizing the expression consistency between the regularized text corresponding to each training sample included in the training sample and the annotated regularized text; Determine, according to the semantic representation vector of each training text included in the training sample and the reconstruction vector corresponding to each training text, a third prediction loss capable of representing the reconstruction error of the semantic representation vector of each training text included in the training sample; Determine, according to the latent variable posterior distribution corresponding to each training text included in the training sample and the set latent variable prior distribution, a fourth prediction loss that can characterize the degree of proximity between the latent variable posterior distribution of each training sample included in the training sample and the set latent variable prior distribution; The parameters of the text regularization model are updated according to the first prediction loss, the second prediction loss, the third prediction loss and the fourth prediction loss.

11. A text regularization device, characterized in that: include: Text acquisition module and text regularization module; The text acquisition module is used to acquire the text to be regularized; The text regularization module is used to regularize the text to be regularized using a pre-built text regularization model to obtain a regularized text corresponding to the text to be regularized; The text regularization model is trained using training samples, wherein the training samples include multiple training texts with the same semantics but different expressions and annotated with the same regularized texts. When the training samples are used to train the text regularization model, the semantic consistency of the multiple training texts included in the training samples is constrained; The training objectives of the text regularization model include: Make the latent variable posterior distributions corresponding to the multiple training texts included in the training sample consistent, wherein the latent variable posterior distribution is the posterior distribution of the latent variable that is obtained according to the corresponding training text and can represent the semantic space; And, the regularized texts respectively corresponding to the multiple training texts included in the training sample are made consistent with the annotated regularized texts.

12. A text regularization device, characterized in that: include: Memory and processor; The memory is used to store programs; The processor is used to execute the program to implement each step of the text regularization method according to any one of claims 1 to 10.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, each step of the text regularization method according to any one of claims 1 to 10 is implemented.

Images