A text style transfer method and system based on BERT model

Abstract

This application discloses a text style transfer method and system based on the BERT model, the method comprising: extracting corpus from different platforms and screening out qualified corpus; using the qualified corpus to train a BERT-based model; using the trained model to rewrite the text to obtain a style text that meets the requirements. Through this application, a model can be trained using corpus of a specific style, the style of the text can be converted, and the generated text can be migrated to the required style.

Description

Technical Field

[0001] The present invention relates to the fields of computer application and natural language processing. More specifically, the present invention relates to a text style transfer method and system based on a BERT model. Background Art

[0002] At present, machine-generated text has become a very popular and convenient technology. However, the text generated by the machine itself is rigid and boring. In order to make the text automatically generated by the machine more interesting and positive, it is necessary to control the style of the generated text. Since the style itself is abstract, it is undoubtedly very difficult to directly control the style of the generated text, but the difficulty of changing from one style to another is reduced. Therefore, the study of text style transfer technology has high application value in academia and industry.

[0003] Text style transfer has a wide range of application prospects in the fields of text generation and data enhancement. For example, when the corpus of a certain style is not enough to support the training of a text generation model, you can use enough general corpus to train the text generation model to generate text, and then use a smaller number of samples to train the model to transfer the generated text to the required style. At present, most of the work is to map the sentence to a hidden semantic space, and then convert it to the semantic space of the target style, but there are still the following problems:

[0004] 1. This conversion method makes it difficult to separate the textual information and style information of the sentence;

[0005] 2. It is difficult to control the quality of generated text, which may cause incoherent sentences or even contextual semantic conflicts;

[0006] 3. Manually annotated aligned corpora are required, but the annotation cost is high, and there is a lack of parallel corpora for text style transfer tasks. For example, for emotional style, there are many positive emotional corpora and many negative emotional corpora, but they are not one-to-one corresponding. In other words, there is a lack of two expressions with both positive and negative emotions for a sentence. Summary of the invention

[0007] The embodiment of the present application provides a text style transfer method based on the BERT model to at least solve the problem of subjective factors in the related art.

[0008] The present invention provides a text style transfer method based on a BERT model, the method comprising the following steps:

[0009] Extraction steps: extract corpus from different platforms and screen out qualified corpus;

[0010] Model training step: using the qualified corpus to train the BERT-based model;

[0011] Rewriting step: rewrite the text using the trained model to obtain text in a style that meets the requirements.

[0012] As a further improvement of the present invention, the model training step specifically includes the following steps:

[0013] The first training step: using the qualified corpus to train the Mask LM model;

[0014] The second training step: using the qualified corpus to train the Next Sentence Prediction model.

[0015] As a further improvement of the present invention, the rewriting step specifically includes the following steps:

[0016] Sentence segmentation step: segmenting the text into sentences;

[0017] First replacement step: using the Next Sentence Prediction model to perform sentence replacement on the text after sentence processing to obtain replacement text;

[0018] Word segmentation step: performing word segmentation processing on the replacement text;

[0019] The second replacement step: using the Mask LM model to perform word replacement on the replacement text after the word segmentation process to obtain the style text.

[0020] As a further improvement of the present invention, the first replacing step specifically includes the following steps:

[0021] The first selection step: select the statement to be replaced;

[0022] The second selection step: selecting a set of replacement sentences that meet the replacement conditions in the target style text library;

[0023] The third selection step: using the Next Sentence Prediction model to select a replacement sentence from the replacement sentence set.

[0024] As a further improvement of the present invention, the replacement condition includes sentence length and sentence similarity.

[0025] As a further improvement of the present invention, the second replacement step specifically includes the following steps:

[0026] Replacement steps: Select the word to be replaced and use [MASK] to replace it;

[0027] Input step: inputting the text replaced by [MASK] into the Mask LM model;

[0028] Prediction step: Use the Mask LM model to predict replacement words.

[0029] Based on the same inventive concept, the present invention also discloses a text style transfer system based on the BERT model based on the text style transfer method based on the BERT model disclosed in any one of the inventions.

[0030] The text style transfer system based on the BERT model includes:

[0031] The extraction module extracts corpus from different platforms and selects qualified corpus;

[0032] A model training module, using the qualified corpus to train a BERT-based model;

[0033] The rewriting module uses the trained model to rewrite the text to obtain a text with a style that meets the requirements.

[0034] As a further improvement of the present invention, the model training module specifically includes:

[0035] A first training unit, using the qualified corpus to train the Mask LM model;

[0036] The second training unit uses the qualified corpus to train the Next Sentence Prediction model.

[0037] As a further improvement of the present invention, the rewriting module specifically includes:

[0038] A sentence division unit, which performs sentence division on the text;

[0039] A first replacement unit, using the Next Sentence Prediction model to perform sentence replacement on the text after the sentence processing to obtain a replacement text;

[0040] A word segmentation unit, performing word segmentation processing on the replacement text;

[0041] The second replacement unit uses the Mask LM model to perform word replacement on the replacement text after the word segmentation process to obtain the style text.

[0042] As a further improvement of the present invention, the first replacement unit specifically includes:

[0043] The first selection unit selects the sentence to be replaced;

[0044] The second selection unit selects a set of replacement sentences that meet the replacement conditions from the text library of the target style;

[0045] The third selection unit selects a replacement sentence from the replacement sentence set using the Next Sentence Prediction model.

[0046] Compared with the prior art, the present invention has the following beneficial effects:

[0047] 1. A text style transfer method based on the BERT model is proposed. The model is trained using corpus of a specific style to convert the style of the text. The training model rewrites some words and sentences in the text. After the text is rewritten, the text is smooth and the effect of text style transfer can be achieved.

[0048] 2. By stripping the text information and style information of the sentence, the model can understand the semantics of the context and does not change the basic semantics of the text during text migration, making the text context smooth after the style conversion;

[0049] 3. No manually annotated parallel corpus is required during model training, which reduces the cost of manual annotation, and the generated style-transferred sentences are closer to real sentences;

[0050] 4. Realize rewriting at the word granularity. When predicting the rewritten words, it is possible to not rewrite the meaning of the original text, but at the same time achieve the transfer of text style.

[0051] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0053] Figure 1 This is an overall flow chart of a text style transfer method based on a BERT model provided by an embodiment of the present invention;

[0054] Figure 2 is a flow chart of model training disclosed in this embodiment;

[0055] Figure 3 yes Figure 1 The overall flow chart of step S2 disclosed;

[0056] Figure 4 is a flow chart of text style transfer provided by an embodiment of the present invention;

[0057] Figure 5 yes Figure 1 The disclosed overall flow chart of step S3;

[0058] Figure 6 yes Figure 5 The overall flow chart of step S32 disclosed;

[0059] Figure 7 yes Figure 5 The overall flow chart of step S34 disclosed;

[0060] Figure 8 This is a structural framework diagram of a text style transfer system based on a BERT model provided in this embodiment;

[0061] Fig. 9 is a framework diagram of a computer device according to an embodiment of the present invention.

[0062] In the above picture:

[0063] 100, extraction module; 200, model training module; 300, rewriting module; 201, first training unit; 202, second training unit; 301, sentence unit; 302, first replacement unit; 3021, first selection unit; 3022, second selection unit; 3023, third selection unit; 303, word segmentation unit; 304, second replacement unit; 3041, replacement unit; 3042, input unit; 3043, prediction unit; 80, bus; 81, processor; 82, memory; 83, communication interface. DETAILED DESCRIPTION

[0064] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments provided in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0065] Obviously, the drawings described below are only some examples or embodiments of the present application. For ordinary technicians in this field, the present application can also be applied to other similar scenarios based on these drawings without creative work. In addition, it can also be understood that although the efforts made in this development process may be complicated and lengthy, for ordinary technicians in this field related to the content disclosed in this application, some changes in design, manufacturing or production based on the technical content disclosed in this application are just conventional technical means, and should not be understood as insufficient content disclosed in this application.

[0066] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those of ordinary skill in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0067] Unless otherwise defined, the technical terms or scientific terms involved in this application should be understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the" and the like involved in this application do not indicate a quantitative limitation, and may represent the singular or plural. The terms "include", "comprise", "have" and any of their variations involved in this application are intended to cover non-exclusive inclusions; for example, a process, method, system, product or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units that are not listed, or may also include other steps or units inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there may be three relationships, for example, "A and / or B" can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects before and after are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects, and do not represent a specific ordering of the objects.

[0068] The present invention is described in detail below in conjunction with the various embodiments shown in the accompanying drawings, but it should be noted that these embodiments are not limitations of the present invention, and any equivalent transformations or substitutions in functions, methods, or structures made by ordinary technicians in the field based on these embodiments are all within the scope of protection of the present invention.

[0069] Before elaborating on each embodiment of the present invention in detail, the core inventive concept of the present invention is summarized and elaborated in detail through the following several embodiments.

[0070] The present invention can transfer the text style based on the BERT model, use the corpus of a specific style to train the model, convert the style of the text, and migrate the generated text to the required style.

[0071] Embodiment 1:

[0072] Reference Figures 1 to 7 As shown, this example discloses a specific implementation method of a text style transfer method based on the BERT model (hereinafter referred to as the “method”).

[0073] Specifically, refer to Figure 1 As shown, the method disclosed in this embodiment mainly includes the following steps:

[0074] Step S1: extract corpus from different platforms and screen out qualified corpus.

[0075] Specifically, in some of the embodiments, it is necessary to screen out the corpus required for training the model before model training. The style differences between the corpuses of different platforms are quite obvious. For example, corpus can be extracted from Weibo, Xiaohongshu, Baijiahao and other platforms respectively.

[0076] Then, step S2 is performed to train the BERT-based model using the qualified corpus.

[0077] Specifically, in some of the embodiments, reference is made to Figure 2 and Figure 3 The step S2 shown specifically includes the following steps:

[0078] S21, using the qualified corpus to train the Mask LM model;

[0079] S22: Use the qualified corpus to train the Next Sentence Prediction model.

[0080] Specifically, the role of the Mask LM model is to replace some words in the text with [MASK], and then after the text is input into the Mask LM model, the Mask LM model can predict what the word replaced by [MASK] is. The role of the Next Sentence Prediction model is to input two sentences into the Next Sentence Prediction model at the same time, and the Next Sentence Prediction model can determine whether the two sentences are two consecutive sentences.

[0081] Then, step S3 is performed to rewrite the text using the trained model to obtain a text with a style that meets the requirements.

[0082] Specifically, in some of the embodiments, reference is made to Figure 4 and Figure 5 The step S3 shown specifically includes the following steps:

[0083] S31, performing sentence processing on the text;

[0084] S32, using the Next Sentence Prediction model to perform sentence replacement on the text after the sentence processing, to obtain a replacement text;

[0085] S33, performing word segmentation processing on the replacement text;

[0086] S34. Use the Mask LM model to perform word replacement on the replacement text after the word segmentation process to obtain the style text.

[0087] Specifically, in some of the embodiments, reference is made to Figure 6 The step S32 shown specifically includes the following steps:

[0088] S321, select the statement to be replaced;

[0089] S322, selecting a replacement statement set that meets the replacement condition in the text library of the target style;

[0090] S323: Use the Next Sentence Prediction model to select a replacement sentence from the replacement sentence set.

[0091] Specifically, the Next Sentence Prediction model is used to replace sentences in the text, and the replacement conditions include sentence length and sentence similarity. For example, the Next Sentence Prediction model is used to replace 30% of the sentences in the text, and the replacement process is as follows: there are three consecutive sentences "a, b, c" in the original text. When we need to rewrite sentence "b", we need to select sentences that meet the following conditions in the text library of the target style: 1. The length is between 80% and 120% of the length of sentence "b"; 2. The similarity with sentence b is between 85% and 75%. Let the set of sentences that meet the above conditions be N, select sentences Ni from the set N, and input sentence pairs a-Ni and Ni-c into the model respectively. When the model determines that these two sentence pairs are continuous sentences, Ni is the sentence that replaces sentence b.

[0092] Specifically, sentence similarity is calculated using TF-IDF. TF-IDF (Term Frequency-Inverse Document Frequency) is a commonly used weighting technique for information retrieval and text mining.

[0093] Specifically, in some of the embodiments, reference is made to Figure 7 The step S34 shown specifically includes the following steps:

[0094] S341. Select the word to be replaced and use [MASK] to replace it;

[0095] S342, inputting the text replaced by [MASK] into the Mask LM model;

[0096] S343: Use the Mask LM model to predict replacement words.

[0097] Specifically, the Mask LM model is used to replace words in the text. For example, the Mask LM model is used to rewrite and replace 30% of the words in the text. The replacement process is as follows: the original text is: "Friends! I'm here to recommend it to you again"; the input to the model is: "[MASK][MASK]! I'm here to give you [MASK][MASK] again!; the output of the model is: "Jimeis! I'm here to give you some recommendations again~", "friends" is rewritten as "Jimeis", and "recommendation" is rewritten as "recommendation".

[0098] A text style transfer method based on the BERT model disclosed in the embodiment of the present application uses a corpus training model in a specific style to convert the style of the text. The training model rewrites some words and sentences in the text. After the text rewriting, the text is smooth and the effect of text style transfer can be achieved. The text information and style information of the sentence are stripped away, and the model can understand the semantics of the context. The basic semantics of the text is not changed during text transfer, so that the text context after the style conversion is smooth. No manually annotated parallel corpus is required during the model training process, which reduces the cost of manual annotation. The generated style-transferred sentences are closer to real sentences, and rewriting at the word granularity is achieved. When predicting the rewritten words, the meaning of the original text can be kept without rewriting, but at the same time, the text style transfer is achieved.

[0099] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent.

[0100] Embodiment 2:

[0101] In combination with the text style transfer method based on the BERT model disclosed in Example 1, this embodiment discloses a specific implementation example of a text style transfer system based on the BERT model (hereinafter referred to as the "system").

[0102] Reference Figure 8 As shown, the system comprises:

[0103] The extraction module 100 extracts corpus from different platforms and selects qualified corpus;

[0104] A model training module 200, using the qualified corpus to train a BERT-based model;

[0105] The rewriting module 300 rewrites the text using the trained model to obtain a text with a style that meets the requirements.

[0106] Specifically, in some embodiments, the model training module 200 specifically includes:

[0107] A first training unit 201 uses the qualified corpus to train the Mask LM model;

[0108] The second training unit 202 uses the qualified corpus to train the Next Sentence Prediction model.

[0109] Specifically, in some embodiments, the rewriting module 300 specifically includes:

[0110] Sentence division unit 301, performs sentence division on the text;

[0111] A first replacement unit 302 uses the Next Sentence Prediction model to perform sentence replacement on the text after the sentence processing to obtain a replacement text;

[0112] A word segmentation unit 303 performs word segmentation processing on the replacement text;

[0113] The second replacement unit 304 uses the Mask LM model to perform word replacement on the replacement text after the word segmentation process to obtain the style text.

[0114] Specifically, in some embodiments, the first replacement unit 302 specifically includes:

[0115] The first selection unit 3021 selects the statement to be replaced;

[0116] The second selection unit 3022 selects a set of replacement sentences that meet the replacement condition in the text library of the target style;

[0117] The third selection unit 3023 selects a replacement sentence from the replacement sentence set using the Next Sentence Prediction model.

[0118] Specifically, in some embodiments, the second replacement unit 304 specifically includes:

[0119] Replacement unit 3041, select the word to be replaced, and use [MASK] to replace it;

[0120] An input unit 3042 inputs the text replaced by [MASK] into the Mask LM model;

[0121] The prediction unit 3043 predicts the replacement word using the Mask LM model.

[0122] For the technical solutions of the other identical parts of the text style transfer system based on the BERT model disclosed in this embodiment and the text style transfer method based on the BERT model disclosed in the first embodiment, please refer to the description in the first embodiment, which will not be repeated here.

[0123] Embodiment three:

[0124] Combination Fig. 9 As shown, this embodiment discloses a specific implementation of a computer device. The computer device may include a processor 81 and a memory 82 storing computer program instructions.

[0125] Specifically, the processor 81 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0126] Among them, the memory 82 may include a large-capacity memory for data or instructions. By way of example and not limitation, the memory 82 may include a hard disk drive (HDD), a floppy disk drive, a solid-state drive (SSD), a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 82 may include a removable or non-removable (or fixed) medium. Where appropriate, the memory 82 may be inside or outside the data processing device. In a specific embodiment, the memory 82 is a non-volatile memory. In a specific embodiment, the memory 82 includes a read-only memory (ROM) and a random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (Programmable Read-Only Memory, PROM for short), an erasable PROM (Erasable ProgrammableRead-Only Memory, EPROM for short), an electrically erasable PROM (Electrically Erasable ProgrammableRead-Only Memory, EEPROM for short), an electrically alterable ROM (Electrically Alterable Read-Only Memory, EAROM for short) or a flash memory (FLASH) or a combination of two or more of these. Under appropriate circumstances, the RAM can be a static random access memory (SRAM) or a dynamic random access memory (DRAM), wherein the DRAM can be a fast page mode dynamic random access memory (FPMDRAM), an extended data output dynamic random access memory (EDODRAM), a synchronous dynamic random access memory (SDRAM), etc.

[0127] The memory 82 may be used to store or cache various data files required for processing and / or communication, as well as possible computer program instructions executed by the processor 81 .

[0128] The processor 81 implements any one of the text style transfer methods based on the BERT model in the above embodiments by reading and executing computer program instructions stored in the memory 82.

[0129] In some of these embodiments, the computer device may further include a communication interface 83 and a bus 80. Fig. 9 As shown, the processor 81, the memory 82, and the communication interface 83 are connected via a bus 80 and communicate with each other.

[0130] The communication interface 83 is used to implement communication between the modules, devices, units and / or equipment in the embodiment of the present application. The communication interface 83 can also implement data communication with other components such as: external devices, image / data acquisition equipment, databases, external storage, and image / data processing workstations.

[0131] The bus 80 includes hardware, software or both, and couples the components of the computer device to each other. The bus 80 includes but is not limited to at least one of the following: a data bus, an address bus, a control bus, an expansion bus, and a local bus. By way of example and not limitation, bus 80 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses or a combination of two or more of these. Where appropriate, bus 80 may include one or more buses. Although embodiments of the present application describe and illustrate a particular bus, the present application contemplates any suitable bus or interconnect.

[0132] The computer device can realize text style transfer based on the BERT model, thereby realizing the combination Figure 1 Describe the method.

[0133] In addition, in combination with the text style transfer method based on the BERT model in the above embodiment, the embodiment of the present application can provide a computer-readable storage medium for implementation. The computer-readable storage medium stores computer program instructions; when the computer program instructions are executed by the processor, any one of the text style transfer methods based on the BERT model in the above embodiment is implemented.

[0134] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0135] In summary, the beneficial effect of the present invention is that a text style transfer method based on the BERT model is proposed, a corpus training model is used to convert the style of the text, and the training model rewrites some words and sentences in the text. After the text rewriting, the text is smooth, and the effect of text style transfer can be achieved at the same time. The text information and style information of the sentence are stripped, and the model can understand the semantics of the context. The basic semantics of the text is not changed during text migration, so that the text context after the style conversion is smooth. No manually annotated parallel corpus is required during the model training process, which reduces the cost of manual annotation. The generated style-transferred sentences are closer to real sentences, and rewriting at the word granularity is achieved. When predicting the rewritten words, the meaning of the original text can be kept without rewriting, but at the same time, the text style transfer is achieved.

[0136] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A text style transfer method based on the BERT model, characterized in that: The steps include: Extraction steps: extract corpus from different platforms and screen out qualified corpus; Model training step: using the qualified corpus to train the BERT-based model; The model training step specifically includes the following steps: The first training step: using the qualified corpus to train the Mask LM model; Second training step: using the qualified corpus to train the Next Sentence Prediction model; Rewriting step: rewriting the text using the trained model to obtain text in a style that meets the requirements; The rewriting step specifically comprises the following steps: Sentence segmentation step: segmenting the text into sentences; First replacement step: using the Next Sentence Prediction model to perform sentence replacement on the text after sentence processing to obtain replacement text; Word segmentation step: performing word segmentation processing on the replacement text; The second replacement step: using the Mask LM model to perform word replacement on the replacement text after the word segmentation process to obtain the style text.

2. The text style transfer method based on the BERT model as claimed in claim 1, characterized in that The first replacement step specifically includes the following steps: The first selection step: select the statement to be replaced; The second selection step: selecting a set of replacement sentences that meet the replacement conditions in the target style text library; The third selection step: using the Next Sentence Prediction model to select a replacement sentence from the replacement sentence set.

3. The text style transfer method based on the BERT model as claimed in claim 2, characterized in that The replacement conditions include sentence length and sentence similarity.

4. The text style transfer method based on the BERT model as claimed in claim 1, characterized in that The second replacement step specifically includes the following steps: Replacement steps: Select the word to be replaced and use [MASK] to replace it; Input step: inputting the text replaced by [MASK] into the Mask LM model; Prediction step: Use the Mask LM model to predict replacement words.

5. A text style transfer system based on the BERT model, characterized in that: include: The extraction module extracts corpus from different platforms and selects qualified corpus; A model training module, using the qualified corpus to train a BERT-based model; The model training module specifically includes: A first training unit, using the qualified corpus to train the Mask LM model; The second training unit uses the qualified corpus to train the Next Sentence Prediction model; A rewriting module rewrites the text using the trained model to obtain a text with a style that meets the requirements; The rewriting module specifically includes: A sentence division unit, which performs sentence division on the text; A first replacement unit, using the Next Sentence Prediction model to perform sentence replacement on the text after the sentence processing to obtain a replacement text; A word segmentation unit, performing word segmentation processing on the replacement text; The second replacement unit uses the Mask LM model to perform word replacement on the replacement text after the word segmentation process to obtain the style text.

6. The text style transfer system based on the BERT model as claimed in claim 5, characterized in that The first replacement unit specifically includes: The first selection unit selects the sentence to be replaced; The second selection unit selects a set of replacement sentences that meet the replacement conditions from the text library of the target style; The third selection unit selects a replacement sentence from the replacement sentence set using the Next Sentence Prediction model.

Images