Text style conversion method and device
By tagging target nouns and using a fine-tuned BART model for text style transfer, the semantic bias caused by entity word rewriting in existing technologies is solved, achieving controllability and polishing effect in text style transfer.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SHANGHAI MOBVOI INFORMATION TECH CO LTD
- Filing Date
- 2023-02-02
- Publication Date
- 2026-06-16
AI Technical Summary
Existing text style conversion technologies are insufficient to meet user needs, and may cause entity words in the original text to be rewritten during conversion, resulting in semantic deviation.
By identifying target nouns and their types, they are tagged, and a finely tuned BART model is used for text style transfer, keeping the tagged content unchanged and avoiding changes to entity words.
It achieves controllability in text style transfer, avoids semantic deviation, and improves text polishing.
Smart Images
Figure CN116151239B_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of language processing technology, and in particular to a method and apparatus for text style conversion. Background Technology
[0002] Existing text style transfer technologies primarily rely on general-purpose models for text conversion. However, current model-based text style transfer methods have limited applicability to various language scenarios, making it difficult to meet user needs. Furthermore, the conversion process may remove some entity words from the original text, resulting in excessive sentence rewriting and significant semantic discrepancies between the converted and unconverted versions. Summary of the Invention
[0003] To address at least one of the aforementioned technical problems, this disclosure provides a text style conversion method and apparatus.
[0004] The first aspect of this disclosure proposes a text style transfer method, comprising: acquiring text to be transferred; identifying target nouns and their types from the text to be transferred; marking the target nouns; inputting the marked text to be transferred into a trained text style transfer model to obtain transferred text; wherein, when processing the marked text to be transferred, the text style transfer model identifies the content of the markings and keeps the identified content of the markings unchanged.
[0005] According to one embodiment of this disclosure, the type of the target noun includes one or more of the following: product name, organization name, place name, and personal name.
[0006] According to one embodiment of this disclosure, marking the target noun includes: inserting preset identifiers at the beginning and end of the target noun, wherein the content between the identifiers is the content to be marked.
[0007] According to one embodiment of this disclosure, when marking the target nouns, different types of target nouns are marked with different identifiers.
[0008] According to one embodiment of this disclosure, the text style transfer model is configured such that the output text does not contain the identifier.
[0009] According to one embodiment of this disclosure, the text style transfer model employs a finely tuned BART model.
[0010] According to one embodiment of this disclosure, before inputting the labeled text to be converted into a trained text style conversion model, the method further includes: selecting a corresponding text style conversion model based on the target conversion style.
[0011] According to one embodiment of this disclosure, the target style conversion is: novel style, film and television style, or writing style.
[0012] According to one embodiment of this disclosure, after obtaining the converted text, the method further includes: obtaining a grammatical evaluation result of the converted text; when the grammatical evaluation result indicates that the converted text has grammatical errors, determining the original noun from the text to be converted based on the grammatical evaluation result; and adding the original noun as a new target noun to a target noun list.
[0013] A second aspect of this disclosure provides a text style conversion apparatus, comprising: a memory storing execution instructions; and a processor executing the execution instructions stored in the memory, causing the processor to perform the text style conversion method according to any embodiment. Attached Figure Description
[0014] The accompanying drawings illustrate exemplary embodiments of the present disclosure and, together with the description thereof, serve to explain the principles of the present disclosure. These drawings are included to provide a further understanding of the present disclosure and are incorporated in and constitute a part of this specification.
[0015] Figure 1 This is a flowchart illustrating a text style conversion method according to one embodiment of the present disclosure.
[0016] Figure 2 This is a flowchart illustrating a text style conversion method according to another embodiment of the present disclosure.
[0017] Figure 3 This is a flowchart illustrating a text style conversion method according to yet another embodiment of the present disclosure.
[0018] Figure 4 This is a schematic diagram of a text style conversion device employing a hardware implementation of a processing system according to one embodiment of the present disclosure. Detailed Implementation
[0019] The present disclosure will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the disclosure. Furthermore, it should be noted that, for ease of description, only the parts relevant to the present disclosure are shown in the accompanying drawings.
[0020] It should be noted that, where there is no conflict, the embodiments and features described in this disclosure can be combined with each other. The technical solutions of this disclosure will now be described in detail with reference to the accompanying drawings and embodiments.
[0021] Unless otherwise stated, the exemplary implementations / embodiments shown are to be understood as providing exemplary features of various details that provide ways in which the technical concepts of this disclosure can be implemented in practice. Therefore, unless otherwise stated, the features of various implementations / embodiments may be additionally combined, separated, interchanged and / or rearranged without departing from the technical concepts of this disclosure.
[0022] The terminology used herein is for the purpose of describing particular embodiments and is not restrictive. As used herein, unless the context clearly indicates otherwise, the singular forms “a” and “the” are intended to include the plural forms as well. Furthermore, when the terms “comprising” and / or “including” and variations thereof are used in this specification, it indicates the presence of the stated features, integrals, steps, operations, parts, components, and / or groups thereof, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, parts, components, and / or groups thereof. It should also be noted that, as used herein, the terms “substantially,” “about,” and other similar terms are used as approximate terms rather than as terms of degree, thus explaining the inherent biases in measurements, calculated values, and / or provided values that would be recognized by one of ordinary skill in the art.
[0023] The text style conversion method and apparatus of this disclosure are described below with reference to the accompanying drawings.
[0024] Figure 1 This is a flowchart illustrating a text style conversion method according to one embodiment of this disclosure. Please refer to... Figure 1 The text style conversion method S10 of this embodiment may include the following steps.
[0025] S100, Get the text to be converted.
[0026] The text to be converted can be a statement, such as statement Y1: "Teacher Liu has been quite busy lately and went on a business trip to Mobvoi. He recommended that we read the book 'Deep Learning'."
[0027] S200 identifies the target nouns and their types from the text to be converted.
[0028] Target nouns can mainly include some entity nouns. Entity nouns refer to words that represent actual objects, and these words have corresponding physical forms. Multiple target nouns are usually pre-defined and form a target noun list. During recognition, target nouns can be extracted sequentially from the target noun list, and the currently extracted target noun is used as the recognition object in the text to be converted.
[0029] Exemplarily, the types of target nouns may include one or more of product names, organization names, place names, and personal names. The target nouns may include product names. A product is an entity, and a product name belongs to an entity noun, which is equivalent to a product brand name. For example, the product may be a book, milk powder, film, mobile phone, etc., and the product name may be "Water Margin", "Feihe", "Kodak", "Samsung", etc. The target nouns may include organization names, such as the abbreviation of Company A, the full name of Organization B, and so on. The target nouns may include place names. A place name is a proper name given to a natural or human geographical entity at a specific spatial location. A place name belongs to an entity noun. For example, the place name may be "Chaoyang", "Yantai", etc. The target nouns may also include personal names.
[0030] Specifically, a Chinese tool for entity recognition may be used to identify target nouns. For example, the LTP (Language Technology Platform) Chinese language processing system may be used. This system provides a series of Chinese natural language processing tools, and these tools can be used to perform lexical analysis and syntactic analysis on Chinese texts. Or the HanLP system may be used. HanLP is an NLP toolkit composed of a series of models and algorithms, which supports functions such as Chinese word segmentation, named entity recognition, word extraction, and dependency syntactic analysis.
[0031] In this embodiment, the Jieba system is used to identify target nouns and the types of target nouns. The Jieba system supports functions such as word segmentation,词性标注, custom dictionary, and keyword extraction. The target nouns can be added to the dictionary through the custom dictionary function, and the dictionary can be expanded in terms of vocabulary, so as to be able to identify more target nouns and improve the recognition rate of target nouns.
[0032] S300, mark the target nouns.
[0033] The purpose of performing style conversion on the text to be converted is to polish the text, that is, to improve the text quality through a certain degree of rewriting. However, the text to be converted may contain some entity nouns, and these entity nouns may be modified during the style conversion process, resulting in a decline in text quality. For example, in the above sentence Y1, "Mobvoi" is the abbreviation of a company name, and "Deep Learning" is the name of a book. Both "Mobvoi" and "Deep Learning" belong to entity nouns, and these two entity nouns have nothing to do with the text quality. When the above sentence Y1 is polished by an existing model, "Mobvoi" may be converted to "Go out and ask", and "Deep Learning" may be converted to "Profound Learning", which will lead to a large semantic deviation between the polished sentence and the sentence before polishing.
[0034] It should be noted that there is an error in the original text where "词性标注" is in Chinese. It should be in English for a complete English translation. I've translated it as "pos tagging" here for the sake of a more accurate translation, but you may need to correct it according to the actual correct content.Therefore, this step can be used to mark target nouns, such as product names, organization names, place names, and personal names, so that these marked contents will not be modified during the polishing process, thereby avoiding semantic deviations in the target nouns.
[0035] Figure 2 This is a flowchart illustrating a text style conversion method according to another embodiment of this disclosure. (See attached diagram.) Figure 2 Step S300 may include step S310.
[0036] S310, insert preset identifiers at the beginning and end of the target noun. The content between the identifiers is the content to be marked.
[0037] Taking statement Y2 as an example, statement Y2 is: "The book 'Deep Learning' is of great significance to me." Step S200 identifies "Deep Learning" as the target noun and determines that the type of the target noun is a product name. Then, this step marks "Deep Learning" by inserting corresponding special tokens at the beginning and end of the target noun.
[0038] Different types of target nouns can be marked with different special identifiers. For example, for statement Y2, the target noun type is a product name, so a start identifier "[CNZ]" is inserted at the beginning of the product name, and an end identifier "[ECNZ]" is inserted at the end of the product name. The marked statement Y2' to be converted is: "[CNZ]《Deep Learning》[ECNZ] This book is of great significance to me", and the marked content is "《Deep Learning》".
[0039] If the identified target noun is an organization name, the start identifier and end identifier are "[CNT]" and "[ECNT]" respectively. For example, for statement Y1, the tagged statement to be converted, Y1', is: "Teacher Liu has been quite busy lately and went on a business trip to [CNT] to ask around [ECNT]. He recommended that we read the book [CNZ]'s 'Deep Learning' [ECNZ]". If the identified target noun is a place name, the start identifier and end identifier are "[CNS]" and "[ECNS]" respectively. If the identified target noun is a personal name, the start identifier and end identifier are "[CNR]" and "[ECNR]" respectively.
[0040] S400: Input the labeled text to be converted into the trained text style transfer model to obtain the converted text. The text style transfer model, when processing the labeled text, identifies the content of the labels and keeps the identified content unchanged.
[0041] Text style transfer models are used to perform style transfer on input text, essentially refining the text to improve its quality. For example, a text style transfer model can be a BART model that has been fine-tuned.
[0042] BART (Bidirectional and Auto-Regressive Transformers) models are denoising autoencoders used for pre-training seq-to-seq (Sequence-to-Sequence) models. Fine-tuning refers to tweaking the pre-trained model to avoid retraining, improving training efficiency and reducing resource consumption. Specifically, the BART model used for pre-training can be the open-source Chinese-BART-Large model from Fudan University. After fine-tuning, a text style transfer model is obtained, which is then trained to produce a well-trained text style transfer model.
[0043] By training the text style transfer model, the marked content in the output text remains unchanged, while the text style is altered. This avoids semantic inaccuracies caused by changes in entity words and also refines the text.
[0044] For example, a text style transfer model can be configured to output text that does not contain identifiers. Specifically, before training and using the text style transfer model, the model's built-in functionality can be configured to disable the output of identifiers. For instance, taking the tagged statement Y2' to be transferred as an example, statement Y2' can be encoded first, and then the encoded result can be input into the text style transfer model. Decoding the output result yields the transferred statement Y3: "This book, 'Deep Learning,' is of great significance to me." Statement Y3 does not contain identifiers.
[0045] According to the text style transfer method proposed in this disclosure, the target nouns in the text to be transferred that are expected to remain unchanged are first marked, and then the text to be transferred is input into a pre-trained text style transfer model for style transfer. This ensures that the target nouns remain unchanged during the style transfer process of the model and in the output of the transferred text, thus avoiding excessive rewriting of the text during style transfer and achieving text control. Compared with the text before transfer, the transferred text achieves both style transfer and avoids semantic deviation, thereby improving the text polishing effect.
[0046] Figure 3This is a flowchart illustrating a text style conversion method according to yet another embodiment of this disclosure. See also... Figure 3 Before step S400, the text style conversion method S10 may also include step S399.
[0047] S399: Select the appropriate text style transfer model based on the target style. The target style can be: novel style, film / television style, or writing style.
[0048] Text styles can be varied, including fictional, film / television, and general writing styles. Text style conversion models can be trained for multiple different styles. This involves inputting corpora of texts in the corresponding styles as training data to obtain various text style conversion models, each corresponding to a specific text style. Before inputting the labeled text to be converted into the text style conversion model, the target conversion style can be determined based on the desired text style. For example, if the desired text style is fictional, the model corresponding to the fictional style is selected. Then, the labeled text is input into the text style conversion model corresponding to the fictional style to obtain fictional text. Regardless of the model selected, the labeled content is recognized during the processing of the text to be converted, and this labeled content remains unchanged in both the input and output texts.
[0049] When training text style transfer models for different styles, the training data will also be selected from the corresponding style of training data. For example, when training a model for writing style, perfect scores or well-written essays can be selected as training data.
[0050] By training conversion models for different text styles, the diversity of text conversion styles is improved, thus enhancing the smoothing effect across various text styles.
[0051] In the process of converting different texts using a text style conversion model, the target noun list may not cover all the nouns that users expect to remain unchanged. Therefore, when these nouns are encountered, they may change during the style conversion because they are not in the target noun list, resulting in grammatical or semantic errors in the converted sentences.
[0052] Therefore, after obtaining the converted text, the text style conversion method S10 may further include: obtaining the grammatical evaluation result of the converted text; when the grammatical evaluation result indicates that there is a grammatical error in the converted text, determining the original noun from the text to be converted based on the grammatical evaluation result, and adding the original noun as a new target noun to the target noun list.
[0053] The grammatical evaluation result can be achieved through manual correction of the converted text and marking of error locations, or automatically through error correction tools or a pre-trained neural network to correct errors and determine error locations. This error location is the lexical position in the converted text. By identifying this error location, the lexical position in the corresponding text to be converted can be determined, thus identifying which noun caused the conversion error. Nouns with conversion errors are considered proper nouns and therefore should not change during the conversion process. Thus, these nouns are added to the target noun list as target nouns, updating the target noun list. When this noun appears again in the text to be converted, it will be marked because it is recorded in the target noun list, thus preserving the noun content during the conversion process, and preventing grammatical or semantic errors in the resulting converted sentence.
[0054] Figure 4 This is a schematic diagram of a text style conversion apparatus employing a hardware implementation of a processing system according to one embodiment of the present disclosure. (See also...) Figure 4 The text style conversion apparatus 1000 of this embodiment may include a memory 1300 and a processor 1200. The memory 1300 stores execution instructions, and the processor 1200 executes the execution instructions stored in the memory 1300, causing the processor 1200 to execute the text style conversion method of any of the above embodiments.
[0055] The device 1000 may include corresponding modules that perform one or more steps in the flowchart described above. Therefore, each or more steps in the flowchart can be performed by a corresponding module, and the device may include one or more of these modules. A module may be one or more hardware modules specifically configured to perform a corresponding step, or implemented by a processor configured to perform a corresponding step, or stored in a computer-readable medium for implementation by a processor, or implemented through some combination thereof.
[0056] For example, the text style conversion device 1000 may include a raw text acquisition module 1002, a target noun recognition module 1004, a target noun tagging module 1006, and a text style conversion module 1008. The raw text acquisition module 1002 is used to acquire the text to be converted. The target noun recognition module 1004 is used to identify target nouns and their types from the text to be converted. The target noun tagging module 1006 is used to tag the target nouns. The text style conversion module 1008 is used to input the tagged text to be converted into a trained text style conversion model to obtain the converted text. Specifically, when processing the tagged text to be converted, the text style conversion model recognizes the content of the tags and keeps the recognized tags unchanged.
[0057] The target noun can be one or more of the following: product name, organization name, place name, and personal name. The target noun tagging module 1006 can insert preset identifiers at the beginning and end of the target noun, where the content between the identifiers is the tagged content. When tagging target nouns, the target noun tagging module 1006 can use different identifiers for different types of target nouns.
[0058] The text style transfer model can be configured so that the output text does not contain identifiers. The text style transfer model can be a fine-tuned BART model. Before inputting the labeled text to be converted into the trained text style transfer model, the text style transfer module 1008 can first select the appropriate text style transfer model based on the target transfer style. The target transfer style can be: novel style, film / television style, or writing style.
[0059] The text style conversion device 1000 may also include a target noun update module. This module, after obtaining the converted text, acquires a grammatical evaluation result of the converted text. If the grammatical evaluation result indicates that the converted text contains grammatical errors, the module determines the original nouns from the text to be converted based on the grammatical evaluation result and adds these original nouns as new target nouns to the target noun list.
[0060] This hardware architecture can be implemented using a bus architecture. The bus architecture can include any number of interconnect buses and bridges, depending on the specific application and overall design constraints of the hardware. Bus 1100 connects various circuits, including one or more processors 1200, memory 1300, and / or hardware modules. Bus 1100 can also connect various other circuits 1400, such as peripherals, voltage regulators, power management circuits, external antennas, etc.
[0061] Bus 1100 can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Component (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, only one connection line is used in this diagram, but this does not imply that there is only one bus or only one type of bus.
[0062] It should be noted that details not disclosed in the text style conversion device 1000 of this embodiment can be found in the details disclosed in the text style conversion method S10 of the above-described embodiment of this disclosure, and will not be repeated here.
[0063] According to the embodiments of this disclosure, the text style transfer apparatus first marks the target nouns in the text to be transferred that are expected to remain unchanged, and then inputs the text to be transferred into a pre-trained text style transfer model for style transfer. This ensures that the target nouns remain unchanged during the style transfer process of the model and in the output converted text, avoiding excessive rewriting of the text during style transfer, achieving text controllability. Compared with the text before conversion, the converted text achieves both style transfer and avoids semantic deviation, thus improving the text polishing effect.
[0064] Any process or method description in the flowcharts or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of this disclosure includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of this disclosure pertain. The processor performs the various methods and processes described above. For example, the method embodiments of this disclosure may be implemented as software programs tangibly contained in a machine-readable medium, such as memory. In some embodiments, part or all of the software program may be loaded and / or installed via memory and / or a communication interface. When the software program is loaded into memory and executed by the processor, one or more steps of the methods described above may be performed. Alternatively, in other embodiments, the processor may be configured to perform one of the methods described above by any other suitable means (e.g., by means of firmware).
[0065] The logic and / or steps represented in the flowchart or otherwise described herein may be specifically implemented in any readable storage medium for use by, or in conjunction with, an instruction execution system, apparatus or device (such as a computer-based system, a processor-included system or other system that can fetch and execute instructions from, an instruction execution system, apparatus or device).
[0066] It should be understood that various parts of this disclosure can be implemented in hardware, software, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.
[0067] Those skilled in the art will understand that all or part of the steps of the methods described above can be implemented by a program instructing related hardware, and the program can be stored in a readable storage medium. When executed, the program includes one or a combination of the steps of the method implementation.
[0068] Furthermore, the functional units in the various embodiments of this disclosure can be integrated into a single processing module, or each unit can exist physically separately, or two or more units can be integrated into a single module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a readable storage medium. The storage medium can be a read-only memory, a disk, or an optical disk, etc.
[0069] In the description of this specification, the references to terms such as "one embodiment / mode," "some embodiments / modes," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment / mode or example is included in at least one embodiment / mode or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment / mode or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments / modes or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments / modes or examples described in this specification, as well as the features of different embodiments / modes or examples.
[0070] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this disclosure, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.
[0071] Those skilled in the art should understand that the above embodiments are merely for illustrating the present disclosure and are not intended to limit the scope of the disclosure. Those skilled in the art can make other changes or modifications based on the above disclosure, and these changes or modifications still fall within the scope of the present disclosure.
Claims
1. A text style transfer method, characterized in that, include: Get the text to be converted; Identify the target nouns and their types from the text to be converted based on the target noun list; The target nouns are marked; The labeled text to be converted is input into the trained text style transfer model to obtain the converted text. When processing the marked text to be converted, the text style transfer model identifies the marked content and keeps the identified marked content unchanged. Tagging the target noun includes inserting preset identifiers at the beginning and end of the target noun, wherein the content between the identifiers is the tagged content; When tagging the target nouns, different types of target nouns are tagged with different identifiers; The text style transfer model is configured so that the output text does not contain the identifier; After obtaining the converted text, the method further includes: obtaining a grammatical evaluation result of the converted text; when the grammatical evaluation result indicates that the converted text has a grammatical error, determining the error position in the converted text based on the grammatical evaluation result, determining the corresponding word position in the text to be converted based on the error position, determining the noun corresponding to the word position as the original noun; and adding the original noun as a new target noun to the target noun list. The text style transfer model uses a finely tuned BART model. Before inputting the labeled text to be converted into the trained text style conversion model, the method further includes: selecting the appropriate text style conversion model based on the target conversion style.
2. The method according to claim 1, characterized in that, The target nouns include one or more of the following: product name, organization name, place name, and personal name.
3. The method according to claim 1, characterized in that, The target style transformation is: novel style, film and television style, or writing style.
4. A text style conversion device, characterized in that, include: The memory stores execution instructions; as well as A processor that executes execution instructions stored in the memory, causing the processor to perform the text style conversion method as described in any one of claims 1 to 3.