A method and device for generating stylized text, a storage medium, and an electronic device

Abstract

The present invention discloses a method and device for generating stylistic text, a storage medium, and an electronic device. The method includes: constructing a syntactic template according to feature words and opinion words; extracting a text feature tag combination according to the syntactic template; determining the target writing style of the target stylistic text, embedding the style parameters of the target writing style as generation conditions into a Bert generation model to generate a target Bert language representation model; using the text feature tag combination as the input of the target Bert language representation model to generate conditional text corresponding to the target writing style. Through the present invention, the technical problem that the text styles generated by related technologies are single is solved. This solution can be used to generate more high-quality and personalized content during information dissemination, improve the richness of the text, and enhance the dissemination influence.

Description

Technical Field

[0001] The present invention relates to the field of computers, and more particularly, to a method and apparatus for generating stylistic text, a storage medium, and an electronic device. Background Art

[0002] In related technologies, text generation tasks are a very important part of natural language processing. Due to various different special requirements in practical applications, there are inevitably many constraints in the process of text generation. Therefore, conditional text generation has been widely used. Conditional text generation generates target text according to some specific conditions. The constraints of the conditions are roughly divided into two categories. One is soft constraints, which usually restrict the text through specific attributes (such as sentiment, theme), and the generated results need to be semantically related to these attributes. Generally, it can be converted into a style transfer type problem to solve; the other is hard constraints. Generally, keywords are given, and it is required that the generated text must contain or block these keywords. To solve this type of problem, on the one hand, optimization needs to be done at the data level, and on the other hand, the model structure needs to be optimized so that the model can learn the target features. According to the different specific tasks, the diversity, personalization, etc. of the generated text also need to be considered in practical applications.

[0003] In related technologies, most of the widely used neural network-based text generation methods are data-driven. However, in practical applications, there is a lack of high-quality labeled data, facing the problem of "data hunger", which can be alleviated to a certain extent by pre-trained models. However, the text generated by the model is too single in style and lacks personality, and most pre-trained models are difficult to generate text for specific fields or specific words, and cannot solve the problem of text generation with conditional constraints.

[0004] In view of the above problems existing in related technologies, no effective solution has been found yet. Summary of the Invention

[0005] To solve the deficiencies of the prior art, the present invention provides a method and apparatus for generating stylistic text, a storage medium, and an electronic device.

[0006] According to one aspect of the embodiments of the present application, a method for generating stylistic text is provided, including: constructing a syntactic template according to feature words and opinion words; extracting a text feature label combination according to the syntactic template; determining the target writing style of the target stylistic text, embedding the style parameters of the target writing style as generation conditions into a Bert generation model to generate a target Bert language representation model; using the text feature label combination as the input of the target Bert language representation model to generate conditional text corresponding to the target writing style.

[0007] Further, constructing a syntactic template based on feature words and opinion words includes: constructing a syntactic template according to the part-of-speech rules and dependency relationships of feature words and opinion words, where the dependency relationships include: direct positive relationships, direct negative relationships, and indirect relationships.

[0008] Further, extracting a text feature label combination according to the syntactic template includes: obtaining text x from the source data i ; identifying the part-of-speech of each word in the text x i , as well as the dependency relationships between adjacent words; inputting the part-of-speech tagging of the part-of-speech and the dependency relationships between adjacent words into the syntactic template, and outputting a text feature label combination W i , where represents the j-th feature of the text x i , represents the opinion corresponding to the j-th feature of the text x i .

[0009] Further, embedding the style parameters of the target writing style as generation conditions into the Bert generation model to generate a target Bert language representation model includes: extracting an embedding representation of the target writing style through an embedding matrix; determining a Bert generation model, and fusing the embedding representation into the normalization layer of the Bert generation model to obtain a target Bert language representation model.

[0010] Further, fusing the embedding representation into the normalization layer of the Bert generation model includes: extracting a first parameter and a second parameter in the normalization layer of the Bert generation model, where both the first parameter and the second parameter are unconditional parameters; using a first transformation matrix to transform the embedding representation to the same dimension as the first parameter and then adding it to the first parameter to obtain a first parameter with fusion conditions, using a second transformation matrix to transform the embedding representation to the same dimension as the second parameter and then adding it to the second parameter to obtain a second parameter with fusion conditions; adding the fusion conditions to the normalization layer of the Bert generation model.

[0011] Further, using the text feature label combination as the input of the target Bert language representation model to generate a conditional text corresponding to the target writing style includes: concatenating the text feature label combination and the target style text to obtain a model input feature; inputting the model input feature into the target Bert language representation model, and performing bidirectional attention encoding on the text feature label combination, and generating a conditional text corresponding to the target writing style through beam search decoding.

[0012] Further, before embedding the style parameters of the target writing style as generation conditions into the Bert generation model, the method further includes: obtaining sample feature tags of a sample style file; splicing the sample feature tags with the sample style file to obtain the following input data Input i : Input i =[CLS]W i [SEP]x i [SEP]; using the input data Input i to train an initial Bert language representation model to obtain a Bert generation model, where the input part Input i of the initial Bert language representation model uses bidirectional attention encoding, and the output part x i [SEP]of the initial Bert language representation model uses unidirectional attention decoding, and the objective function only calculates the loss of the output part, where [CLS] is the flag bit at the beginning of the sentence, and [SEP] is the separator flag bit between adjacent sentences.

[0013] According to another aspect of the embodiments of the present application, there is also provided a device for generating style text, including: a construction module for constructing a syntactic template according to feature words and opinion words; an extraction module for extracting a combination of text feature tags according to the syntactic template; an embedding module for determining the target writing style of the target style text, embedding the style parameters of the target writing style as generation conditions into the Bert generation model to generate a target Bert language representation model; a generation module for using the combination of text feature tags as the input of the target Bert language representation model to generate conditional text corresponding to the target writing style.

[0014] Further, the construction module includes: a construction unit for constructing a syntactic template according to the part-of-speech rules and dependency relationships of feature words and opinion words, where the dependency relationships include: direct positive relationships, direct negative relationships, and indirect relationships.

[0015] Further, the extraction module includes: an extraction unit for obtaining text x i from source data; an identification unit for identifying the part of speech of each word in the text x i and the dependency relationship between adjacent words; an output unit for inputting the part-of-speech tagging of the part of speech and the dependency relationship between adjacent words into the syntactic template and outputting a combination of text feature tags W i , where, represents the j-th feature of the text x i , represents the opinion corresponding to the j-th feature of the text x i .

[0016] Furthermore, the embedding module includes: an extraction unit for extracting an embedding representation of the target writing style through an embedding matrix; a fusion unit for determining a Bert generation model and fusing the embedding representation into the normalization layer of the Bert generation model to obtain a target Bert language representation model.

[0017] Furthermore, the fusion unit includes: an extraction subunit for extracting a first parameter and a second parameter in the normalization layer of the Bert generation model, where both the first parameter and the second parameter are unconditional parameters; an operation subunit for transforming the embedding representation to the same dimension as the first parameter using a first transformation matrix and then adding it to the first parameter to obtain a first parameter with a fusion condition, and transforming the embedding representation to the same dimension as the second parameter using a second transformation matrix and then adding it to the second parameter to obtain a second parameter with a fusion condition; an addition subunit for adding the fusion condition to the normalization layer of the Bert generation model.

[0018] Furthermore, the generation module includes: a splicing unit for splicing the text feature label combination and the target style text to obtain model input features; a generation unit for inputting the model input features into the target Bert language representation model, performing bidirectional attention encoding on the text feature label combination, and generating conditional text corresponding to the target writing style through beam search decoding.

[0019] Furthermore, the apparatus further includes: an acquisition module for acquiring sample feature labels of a sample style file before the embedding module embeds the style parameters of the target writing style as generation conditions into the Bert generation model; a splicing module for splicing the sample feature labels and the sample style file to obtain the following input data Input i : Input i = [CLS]W i [SEP]x i [SEP]; a training module for training an initial Bert language representation model using the input data Input i to obtain a Bert generation model, where the input part Input of the initial Bert language representation model i uses bidirectional attention encoding, and the output part x of the initial Bert language representation model i [SEP]uses unidirectional attention decoding, and the objective function only calculates the loss of the output part, where [CLS] is a flag bit at the beginning of a sentence and [SEP] is a separator flag bit for adjacent sentences.

[0020] According to another aspect of the embodiments of the present application, a storage medium is further provided. The storage medium includes a stored program, and when the program runs, it executes the above method steps.

[0021] According to another aspect of the embodiments of the present application, an electronic device is further provided, including a processor, a communication interface, a memory, and a communication bus. Among them, the processor, the communication interface, and the memory complete communication with each other through the communication bus. Among them: the memory is used to store a computer program; the processor is used to execute the above method steps by running the program stored on the memory.

[0022] The embodiments of the present application also provide a computer program product containing instructions. When it runs on a computer, it causes the computer to execute the steps in the above method.

[0023] Through the present invention, a syntactic template is constructed according to feature words and opinion words, a combination of text feature tags is extracted according to the syntactic template, the target writing style of the target style text is determined, the style parameters of the target writing style are embedded into the Bert generation model as generation conditions, a target Bert language representation model is generated, the combination of text feature tags is used as the input of the target Bert language representation model, and conditional text corresponding to the target writing style is generated. By designing a syntactic template to extract text feature tags and integrating them with the writing style as conditions into the pre-trained model in different ways, personalized text that meets the features is generated, and text with diverse styles is automatically generated, solving the technical problem that the text styles generated by the related technology using a network model are single. This solution can be used to generate more high-quality and personalized content during the information dissemination process, improve the richness of the text, and enhance the dissemination influence. Description of the Drawings

[0024] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The schematic embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation to the present invention. In the drawings:

[0025] Figure 1 is a hardware structure block diagram of a computer according to an embodiment of the present invention;

[0026] Figure 2 is a flowchart of a method for generating a style text according to an embodiment of the present invention;

[0027] Figure 3 is an implementation flowchart of an embodiment of the present invention;

[0028] Figure 4 is a structure block diagram of a device for generating a style text according to an embodiment of the present invention. Detailed Embodiments

[0029] In order to enable those skilled in the art to better understand the solution of this application, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of this application. It should be noted that, without conflict, the embodiments in this application and the features in the embodiments can be combined with each other.

[0030] It should be noted that the terms "preset", "again", etc. in the specification and claims of this application and the above-mentioned accompanying drawings are used to distinguish similar objects, and do not necessarily need to be used to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances, so that the embodiments of this application described here can be implemented in an order different from those illustrated or described here. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, product or device including a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0031] Embodiment 1

[0032] The method embodiment provided in the first embodiment of this application can be executed on a server, a computer, or a similar computing device. Taking running on a computer as an example, Figure 1 is a hardware structure block diagram of a computer according to an embodiment of the present invention. As Figure 1 shown, the computer may include one or more ( Figure 1 only one is shown in the figure) processors 102 (the processors 102 may include, but are not limited to, processing devices such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data. Optionally, the above computer may further include a transmission device 106 for communication functions and an input / output device 108. Those of ordinary skill in the art can understand that Figure 1 the structure shown is only schematic and does not limit the structure of the above computer. For example, the computer may further include more or fewer components than those shown in Figure 1 the figure, or have a different configuration from that shown in Figure 1 the figure.

[0033] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the method for generating a stylistic text in an embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, the above-mentioned method is implemented. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some instances, the memory 104 may further include a memory remotely disposed relative to the processor 102, and these remote memories may be connected to the computer through a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0034] The transmission device 106 is used to receive or send data via a network. Specific examples of the above-mentioned network may include a wireless network provided by a communication provider of a computer. In one instance, the transmission device 106 includes a network adapter (Network Interface Controller, abbreviated as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one instance, the transmission device 106 may be a radio frequency (Radio Frequency, abbreviated as RF) module, which is used to communicate with the Internet wirelessly.

[0035] In this embodiment, a method for generating a stylistic text is provided. Figure 2 is a flowchart of a method for generating a stylistic text according to an embodiment of the present invention, as Figure 2 shown, and the process includes the following steps:

[0036] Step S202, constructing a syntactic template according to feature words and opinion words;

[0037] Optionally, the feature words are nouns or noun phrases (n), and the opinion words are adjective (adj) phrases.

[0038] Step S204, extracting a text feature label combination according to the syntactic template;

[0039] Among them, the text feature label combination includes multiple text feature labels, for example, multiple text feature labels are extracted from sample data by using the syntactic template;

[0040] Step S206, determining the target writing style of the target stylistic text, embedding the style parameters of the target writing style as generation conditions into the Bert generation model, and generating a target Bert language representation model;

[0041] Optionally, the target writing style can be one or more styles, such as humorous, serious, innocent, etc. The number of target writing styles is the same as the number of conditional texts output. A writing style knowledge base can be preset, which includes all writing styles.

[0042] Optionally, the target style text is the same as the source text combined with the extracted text feature tags. Rules can be set up based on the writing style knowledge base to analyze the writing style of each text in the corpus text and obtain the target writing style.

[0043] Step S208: Use the text feature tag combination as the input of the target Bert language representation model to generate conditional texts corresponding to the target writing style.

[0044] Through the above steps, a syntactic template is constructed according to the feature words and opinion words, the text feature tag combination is extracted according to the syntactic template, the target writing style of the target style text is determined, the style parameters of the target writing style are used as generation conditions and embedded into the Bert generation model to generate the target Bert language representation model. Using the text feature tag combination as the input of the target Bert language representation model, conditional texts corresponding to the target writing style are generated. By designing a syntactic template to extract text feature tags and incorporating them with the writing style as conditions into the pre-trained model in different ways, personalized texts that meet the features are generated, and texts with diverse styles are automatically generated, solving the technical problem that the texts generated by the related technology have a single style. This solution can be used to generate more high-quality and personalized content during the information dissemination process, improve the richness of the text, and enhance the dissemination influence.

[0045] In an implementation manner of this embodiment, constructing a syntactic template according to the feature words and opinion words includes: constructing a syntactic template according to the part-of-speech rules and dependency relationships of the feature words and opinion words, where the dependency relationships include: direct positive relationship, direct negative relationship, and indirect relationship.

[0046] In this embodiment, effective syntactic rules are designed according to the corpus and a template is constructed. The text is subjected to part-of-speech tagging and dependency syntactic analysis, and the core words and dependent words that conform to the syntactic rules are extracted from it. The original text is condensed into feature tags in the form of "feature - opinion words", and at the same time, training data is provided for the generation model.

[0047] First, construct the syntactic templates for text feature tags. After performing dependency syntactic analysis and part-of-speech tagging on the corpus and summarizing, it is found that the feature words are nouns or noun phrases (n), while the opinion words are mostly adjectives (adj). According to the correspondence between the core word and the feature word and opinion word in the dependency relationship, they can be divided into direct positive relationship, direct negative relationship, and indirect relationship. For example, the direct positive relationship is defined as: the feature word is the dependent word, the opinion word is the core word, and the relationship between them is the subject-predicate relationship, and the dependency path is "n-SBV-adj". Therefore, templates can be constructed based on the part-of-speech rules of the core word and the dependent word and the dependency relationship.

[0048] In one example, extracting the text feature tag combination according to the syntactic template includes: obtaining text x from the source data i ; identifying the part-of-speech of each word in text x i and the dependency relationship between adjacent words; inputting the part-of-speech tagging and the dependency relationship between adjacent words into the syntactic template, and outputting the text feature tag combination W i , where represents the j-th feature of text x i , represents the opinion corresponding to the j-th feature of text x i .

[0049] According to the syntactic template, comprehensively utilize the part-of-speech tagging and dependency syntactic analysis tools to extract the feature tag combination W i from text x i .

[0050] In one example, embed the style parameters of the target writing style as generation conditions into the Bert generation model to generate the target Bert language representation model, including:

[0051] S11, extract the embedding representation of the target writing style through the embedding matrix;

[0052] The writing styles of people with different personalities are different. In order to enable the model to generate more personalized content closer to human expression, this solution constructs a writing style knowledge base, including common function words, emotion words, punctuation marks, emoticons, etc. under different styles. Then, based on the writing style knowledge base, establish rules to analyze and obtain the writing style of each text in the corpus Extract the embedding representation of the target writing style.

[0053] S12, determine the Bert generation model, and fuse the embedding representation into the normalization layer of the Bert generation model to obtain the target Bert language representation model.

[0054] In one example, integrating the embedding representation into the normalization layer of the Bert generation model includes: extracting the first parameter and the second parameter in the normalization layer of the Bert generation model, where both the first parameter and the second parameter are unconditional parameters; adding the embedding representation to the first parameter after transforming the embedding representation to the same dimension as the first parameter using a first transformation matrix to obtain a first parameter with a fused condition, and adding the embedding representation to the second parameter after transforming the embedding representation to the same dimension as the second parameter using a second transformation matrix to obtain a second parameter with a fused condition; adding the fused condition to the normalization layer of the Bert generation model.

[0055] In models based on the Transformer architecture such as BERT (Bidirectional Encoder Representation from Transformers), the main Normalization method is LayerNormalization (conditional normalization layer). In this embodiment, based on the conditional batch normalization method in the field of image generation, the parameters are made into functions of writing style variables to control the generation behavior of the model. The specific approach is as follows:

[0056] First, obtain the text x i through an embedding matrix i to get the embedding representation C i of the writing style of x, and then fuse the writing style C i into the original parameters β and γ of LayerNormalization. Since there are already existing, unconditional β and γ in the pre-trained Bert, two fully connected layers can be added, that is, transform C i to the same dimension as β and γ through two different transformation matrices, and then add the transformed results to β and γ, so as to achieve the purpose of controlling personalized text generation through the writing style. The structure of the conditional normalization layer is as follows:

[0057]

[0058] γ(C i ) = γ + W γ C i

[0059] β(C i ) = β + W β C i

[0060]

[0061] where CLN() represents the conditional normalization layer, and a i is the text x iThe input vector corresponding to this layer, Emb() is the writing style embedding layer, W γ 、W β are parameters, μ represents the mean of this layer, σ represents the standard deviation, and ∈ is an infinitesimal value to prevent the denominator from being zero.

[0062] In order to avoid interfering with the original weights of the model during the training process of this embodiment, the two transformation matrices are initialized to all zeros, so that the model is consistent with the original pre-trained model in the initial state.

[0063] In an implementation manner of this embodiment, taking the text feature label combination as the input of the target Bert language representation model, and generating conditional text corresponding to the target writing style, including: concatenating the text feature label combination and the target style text to obtain the model input feature; inputting the model input feature into the target Bert language representation model, and performing bidirectional attention encoding on the text feature label combination, and generating conditional text corresponding to the target writing style through beam search decoding.

[0064] Optionally, before using the style parameters of the target writing style as the generation condition to be embedded in the Bert generation model, it further includes: obtaining the sample feature labels of the sample style file; concatenating the sample feature labels with the sample style file to obtain the following input data Input i :Input i =[CLS]W i [SEP]x i [SEP]; using the input data Input i to train the initial Bert language representation model to obtain the Bert generation model, where the input part Input of the initial Bert language representation model i uses bidirectional attention encoding, and the output part x of the initial Bert language representation model i [SEP]uses unidirectional attention decoding, and the objective function only calculates the loss of the output part, where [CLS] is the flag bit at the beginning of the sentence, and [SEP] is the separator flag bit for adjacent sentences.

[0065] In order to ensure the fluency and readability of the generation result, this solution uses a pre-trained Bert model and uses the prefix bidirectional attention mechanism to enable Bert to complete the Seq2Seq task.

[0066] During the training stage, the text feature label W i is concatenated with the text x i to obtain the input Input of the Bert model i ;

[0067] Input i =[CLS]W i[SEP]x i [SEP]

[0068] For the original input part [CLS]W i [SEP]Bidirectional attention is used, while for the output part x i [SEP]Unidirectional attention is used. The objective function only calculates the loss of the output part. Thus, the Bert model can complete the work of both the encoder and the decoder. [CLS] and [SEP] are flag bits in the Bert input, where [CLS] is placed at the beginning of the first sentence, and [SEP] is used to separate two input sentences.

[0069] In the prediction stage and the model application stage of this embodiment, the BeamSearch method is used for decoding, so that the generated results are more diverse.

[0070] Figure 3 is the implementation flowchart of the embodiment of the present invention. The input data of the model in this embodiment is a large-scale corpus, a syntactic template, and a writing style knowledge base, and the output result is a text that meets the user's expressed views and conforms to the user's writing style. The process includes: extracting text feature tags based on the syntactic template; embedding the writing style based on conditional layer normalization; generating conditional text based on prefix bidirectional attention.

[0071] In the process of extracting text feature tags based on the syntactic template, effective syntactic rules are designed according to the corpus and templates are constructed. The text is subjected to part-of-speech tagging and dependency syntactic analysis, and the core words and dependent words that conform to the syntactic rules are extracted, condensing the original text into feature tags in the form of "feature-viewpoint words", and at the same time providing training data for the generation model.

[0072] In the process of embedding the writing style based on conditional layer normalization, a writing style knowledge base is constructed, rules are set to analyze the writing style of the text, and its vectorized representation is obtained by training the writing style embedding matrix, so that it can be used as a condition to be fused into the layer normalization parameters to control text generation.

[0073] In the process of generating conditional text based on prefix attention, the input of the Bert model is reconstructed according to the text feature tags, and the prefix bidirectional visible attention mechanism is used to fine-tune on the basis of the Bert pre-trained weights, and finally the BeamSearch method is used for decoding to ensure the diversity of the generated text.

[0074] Adopting the solution of this embodiment, a method for extracting text feature tags based on syntactic templates is proposed. By constructing templates, feature tags in the form of "feature-opinion words" are extracted from the text, and the original text is condensed into a more concise and formally structured feature description without losing the opinions and main ideas. A writing style embedding method based on conditional layer normalization is proposed. By embedding the writing style into the parameters of layer normalization to control text generation, the generated results become more personalized. The input of the Bert model is reconstructed using the extracted text feature tags, and combined with the prefix bidirectional attention mechanism, the Bert model can generate text according to the feature tags. At the same time, Beam Search decoding ensures the diversity of the generated results. By designing syntactic templates to extract text feature tags and incorporating them together with the writing style as conditions into the pre-trained model in different ways to generate personalized text that conforms to the features, this solution can be used to generate high-quality content during information dissemination and enhance the dissemination influence.

[0075] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by means of software plus necessary general mechanical equipment. Of course, it can also be implemented by hardware, but in many cases, the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of software controlling mechanical equipment. The software is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disc) and includes several instructions for causing a mechanical equipment to execute the methods described in various embodiments of the present invention.

[0076] Embodiment 2

[0077] In this embodiment, a device for generating stylized text is also provided to implement the above embodiments and preferred implementation manners, and those that have been described will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that realizes a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware is also possible and contemplated.

[0078] Figure 4 is a structural block diagram of a device for generating stylized text according to an embodiment of the present invention, as Figure 4 shown, the device includes: a construction module 40, an extraction module 42, an embedding module 44, and a generation module 46, wherein,

[0079] The construction module 40 is used to construct a syntactic template according to feature words and opinion words;

[0080] The extraction module 42 is used to extract a combination of text feature tags according to the syntactic template;

[0081] An embedding module 44 is configured to determine the target writing style of the target-style text, embed the style parameters of the target writing style into the Bert generation model as generation conditions, and generate a target Bert language representation model;

[0082] A generation module 46 is configured to use the text feature label combination as the input of the target Bert language representation model to generate conditional text corresponding to the target writing style.

[0083] Optionally, the construction module includes: a construction unit configured to construct a syntactic template according to the part-of-speech rules and dependency relationships of feature words and opinion words, where the dependency relationships include: direct positive relationships, direct negative relationships, and indirect relationships.

[0084] Optionally, the extraction module includes: an extraction unit configured to obtain text x from the source data i ; an identification unit configured to identify the part of speech of each word in the text x i and the dependency relationship between adjacent words; an output unit configured to input the part-of-speech tagging of the part of speech and the dependency relationship between adjacent words into the syntactic template and output a text feature label combination W i , where represents the jth feature of the text x i , represents the opinion corresponding to the jth feature of the text x i .

[0085] Optionally, the embedding module includes: an extraction unit configured to extract the embedding representation of the target writing style through an embedding matrix; a fusion unit configured to determine the Bert generation model, and fuse the embedding representation into the normalization layer of the Bert generation model to obtain a target Bert language representation model.

[0086] Optionally, the fusion unit includes: an extraction subunit configured to extract a first parameter and a second parameter in the normalization layer of the Bert generation model, where both the first parameter and the second parameter are unconditional parameters; an operation subunit configured to use a first transformation matrix to transform the embedding representation to the same dimension as the first parameter and then add it to the first parameter to obtain a first parameter with fusion conditions, use a second transformation matrix to transform the embedding representation to the same dimension as the second parameter and then add it to the second parameter to obtain a second parameter with fusion conditions; an addition subunit configured to add the fusion conditions to the normalization layer of the Bert generation model.

[0087] Optionally, the generation module includes: a splicing unit configured to splice the text feature label combination and the target style text to obtain model input features; a generation unit configured to input the model input features into the target Bert language representation model, perform bidirectional attention encoding on the text feature label combination, and generate conditional text corresponding to the target writing style through beam search decoding.

[0088] Optionally, the apparatus further includes: an acquisition module configured to acquire sample feature labels of a sample style file before the embedding module embeds style parameters of the target writing style into the Bert generation model as generation conditions; a splicing module configured to splice the sample feature labels and the sample style file to obtain the following input data Input i : Input i =[CLS]W i [SEP]x i [SEP]; a training module configured to train an initial Bert language representation model by using the input data Input i to obtain a Bert generation model, wherein an input part Input of the initial Bert language representation model i uses bidirectional attention encoding, and an output part x of the initial Bert language representation model i [SEP]uses unidirectional attention decoding, and a target function only calculates a loss of the output part, where [CLS] is a flag bit at the beginning of a sentence, and [SEP] is a separator flag bit for adjacent sentences.

[0089] It should be noted that the above-mentioned modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited thereto: the above-mentioned modules are all located in the same processor; or, the above-mentioned modules are separately located in different processors in any combination form.

[0090] Embodiment 3

[0091] An embodiment of the present invention further provides a storage medium, in which a computer program is stored, and the computer program is configured to execute the steps in any one of the above method embodiments when running.

[0092] Optionally, in this embodiment, the above storage medium can be configured to store a computer program for executing the following steps:

[0093] S1, constructing a syntactic template according to feature words and opinion words;

[0094] S2, extracting a text feature label combination according to the syntactic template;

[0095] S3. Determine the target writing style of the target style text, embed the style parameters of the target writing style into the Bert generation model as generation conditions, and generate a target Bert language representation model;

[0096] S4. Use the text feature label combination as the input of the target Bert language representation model to generate conditional text corresponding to the target writing style.

[0097] Optionally, in this embodiment, the above storage medium may include, but is not limited to: USB flash drive, read-only memory (ROM for short), random access memory (RAM for short), mobile hard disk, magnetic disk, or optical disc, etc., all kinds of media that can store computer programs.

[0098] An embodiment of the present invention also provides an electronic device, including a memory and a processor. A computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0099] Optionally, the above electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the above processor, and the input / output device is connected to the above processor.

[0100] Optionally, in this embodiment, the above processor may be configured to execute the following steps through a computer program:

[0101] S1. Construct a syntactic template according to feature words and opinion words;

[0102] S2. Extract the text feature label combination according to the syntactic template;

[0103] S3. Determine the target writing style of the target style text, embed the style parameters of the target writing style into the Bert generation model as generation conditions, and generate a target Bert language representation model;

[0104] S4. Use the text feature label combination as the input of the target Bert language representation model to generate conditional text corresponding to the target writing style.

[0105] Optionally, specific examples in this embodiment may refer to the examples described in the above embodiments and optional implementation manners, and will not be elaborated herein.

[0106] The serial numbers of the above embodiments of the present application are only for description and do not represent the advantages or disadvantages of the embodiments.

[0107] In the above embodiments of the present application, the descriptions of the respective embodiments have their own emphases. For parts not detailed in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

[0108] In several embodiments provided by the present application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection to each other can be through some interfaces. The indirect coupling or communication connection of units or modules can be in an electrical or other form.

[0109] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0110] In addition, in each embodiment of the present application, the functional units can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0111] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to enable a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the methods described in each embodiment of the present application. And the aforementioned storage medium includes: USB flash drive, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disc and other various media that can store program codes.

[0112] The above is only the preferred embodiment of the present application. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present application, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present application.

Claims

1. A method for generating stylistic text, characterized in that, including: Constructing a syntactic template according to feature words and opinion words; Extracting a text feature label combination according to the syntactic template, where the text feature label combination includes multiple text feature labels; Determining the target writing style of the target style text, embedding the style parameters of the target writing style as generation conditions into the Bert generation model, and generating a target Bert language representation model; Using the text feature label combination as the input of the target Bert language representation model to generate conditional text corresponding to the target writing style; Among them, the extraction of the text feature label combination according to the syntactic template includes: obtaining text x from the source data i ; identifying the part-of-speech of each word in the text x i , as well as the dependency relationship between adjacent words; inputting the part-of-speech tagging of the part-of-speech and the dependency relationship between adjacent words into the syntactic template, and outputting the text feature label combination W i , where represents the j-th feature of the text x i , represents the view corresponding to the j-th feature of the text x i ; Among them, embedding the style parameters of the target writing style as generation conditions into the Bert generation model to generate a target Bert language representation model includes: extracting the embedding representation of the target writing style through an embedding matrix; determining the Bert generation model, and fusing the embedding representation into the normalization layer of the Bert generation model to obtain a target Bert language representation model.

2. The method according to claim 1, wherein Constructing a syntactic template according to feature words and opinion words includes: Constructing a syntactic template according to the part-of-speech rules and dependency relationships of feature words and opinion words, where the dependency relationship includes: a direct positive relationship, where the direct positive relationship is defined as: the feature word is the dependent word, the opinion word is the core word, and the relationship between the two is a subject-predicate relationship.

3. The method according to claim 1, wherein Fusing the embedding representation into the normalization layer of the Bert generation model includes: Extracting the first parameter and the second parameter in the normalization layer of the Bert generation model, where both the first parameter and the second parameter are unconditional parameters; Using a first transformation matrix to transform the embedding representation to the same dimension as the first parameter and then adding it to the first parameter to obtain a first parameter with fusion conditions, and using a second transformation matrix to transform the embedding representation to the same dimension as the second parameter and then adding it to the second parameter to obtain a second parameter with fusion conditions; Adding the fusion conditions to the normalization layer of the Bert generation model.

4. The method according to claim 1, wherein Using the text feature label combination as the input of the target Bert language representation model to generate conditional text corresponding to the target writing style includes: Concatenating the text feature label combination and the target style text to obtain model input features; Inputting the model input features into the target Bert language representation model, performing bidirectional attention encoding on the text feature label combination, and generating conditional text corresponding to the target writing style through beam search decoding.

5. The method according to claim 1, characterized in that, Before embedding the style parameters of the target writing style as generation conditions into the Bert generation model, the method further includes: Obtaining the sample feature labels of the sample style file; After splicing the sample feature label and the sample style file, the following input data Input is obtained i : Input i =[CLS]W i [SEP]x i [SEP]; Adopt the input data Input i Train the initial Bert language representation model to obtain a Bert generation model, where the input part Input of the initial Bert language representation model i Use bidirectional attention encoding, and the output part x of the initial Bert language representation model i [SEP]Use unidirectional attention decoding, and the objective function only calculates the loss of the output part, where [CLS] is the flag bit at the beginning of the sentence, [SEP] is the separator flag bit for adjacent sentences, and W i is the text feature label combination.

6. An apparatus for generating stylized text, characterized in that, including: A construction module for constructing a syntactic template according to feature words and opinion words; An extraction module for extracting a text feature label combination according to the syntactic template, where the text feature label combination includes multiple text feature labels; An embedding module for determining the target writing style of the target style text, embedding the style parameters of the target writing style as generation conditions into the Bert generation model, and generating a target Bert language representation model; A generation module, configured to use the text feature label combination as the input of the target Bert language representation model, and generate conditional text corresponding to the target writing style; Among them, the extraction module includes: an extraction unit for obtaining text x from source data i ; an identification unit for identifying the part-of-speech of each word in the text x i and the dependency relationship between adjacent words; an output unit for inputting the part-of-speech tagging of the part-of-speech and the dependency relationship between adjacent words into the syntactic template and outputting a text feature label combination W i , where represents the j-th feature of the text x i , represents the view corresponding to the j-th feature of the text x i ​ Wherein, the embedding module includes: an extraction unit, configured to extract the embedding representation of the target writing style through an embedding matrix; a fusion unit, configured to determine a Bert generation model, and fuse the embedding representation into the normalization layer of the Bert generation model to obtain a target Bert language representation model.

7. A storage medium, characterized in that, The storage medium includes a stored program, wherein when the program runs, it executes the method steps described in any one of claims 1 to 5 above.

8. An electronic device, comprising a processor, a communication interface, a memory, and a communication bus, wherein, A processor, a communication interface, and a memory complete mutual communication through a communication bus; wherein: The memory is used for storing a computer program; The processor is configured to execute the method steps described in any one of claims 1 to 5 by running the program stored on the memory.

Images