Controllable style-based text transformation

Abstract

The present invention provides a method, system and computer program product for multi-style text conversion. The computer-implemented method includes obtaining an input text and selecting a set of style specifications for converting the input text. The set of style specifications includes one or more target writing style fields selected from a plurality of writing style fields, a weight for each target writing style field in the target writing style fields, and a weight for each linguistic aspect in a set of linguistic aspects for conversion of the input text. The weight for the target writing style field represents the relative influence of the target writing style field on the conversion of the input text. The computer-implemented method also includes generating one or more style-converted output texts based at least in part on the set of style specifications using an unsupervised neural network.

Description

Technical Field

[0001] The present invention relates to information technology, and more particularly, to text conversion. Background Art

[0002] Natural language generation technology is used in a variety of application areas (including translators, summarizers, and dialog generators). Automated dialog systems or conversational systems can be used, for example, to create various workspaces with different dialog flows for implementing chatbots for customer support, user interaction, etc. A chatbot is a computer program or other software that is capable of performing conversational communications via auditory and / or text processing. Natural language generation technology can be used to generate parts of such conversational communications. Summary of the invention

[0003] Embodiments of the present invention provide a controllable style-based text conversion technology.

[0004] In one embodiment, a computer-implemented method includes the following steps: obtaining input text; selecting a set of style specifications for transforming the input text, the set of style specifications including: one or more target writing style domains selected from a plurality of writing style domains; a weight for each of the target writing style domains, the weight representing a relative influence of the target writing style domain on the transformation of the input text; and a weight for each of a set of linguistic aspects for the transformation of the input text; and generating one or more style-transformed output texts based at least in part on the set of style specifications using an unsupervised neural network. The step is performed by at least one processing device.

[0005] In another embodiment, a computer-implemented method includes the steps of receiving an input text segment to be style-converted according to a set of style control parameters specified in a control vector; passing the input text segment to an unsupervised neural network including a plurality of gated recurrent units; determining a hidden representation of the input text segment using a first subset of the plurality of gated recurrent units arranged in a stack provided to an encoder; and generating a style-converted output text segment using a second subset of the plurality of gated recurrent units provided to a decoder, the decoder generating each word in the style-converted output text segment using a nonlinear function that outputs a probability distribution for a given word in the style-converted output text segment based on (i) an embedding of a previously generated word of the style-converted output text segment, (ii) the control vector, (iii) a vector obtained by focusing on the hidden representation of the input text segment, and (iv) a hidden state of a decoder of the unsupervised neural network. The steps are performed by at least one processing device.

[0006] Another embodiment of the present invention or an element thereof may be implemented in the form of an article of manufacture tangibly containing computer-readable instructions, which when implemented causes a computer to perform a plurality of method steps as described herein. In addition, another embodiment of the present invention or an element thereof may be implemented in the form of an apparatus comprising a memory and at least one processor coupled to the memory and configured to perform the proposed method steps. Still further, another embodiment of the present invention or an element thereof may be implemented in the form of an apparatus that performs the method steps or an element thereof described herein; the apparatus may include a hardware module or a combination of hardware and software modules, wherein the software module is stored in a tangible computer-readable storage medium (or multiple such media).

[0007] These and other objects, features and advantages of the present invention will become apparent from the following detailed description of illustrative embodiments thereof, which is to be read in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 An adjustable multi-style text conversion system according to an exemplary embodiment of the present invention is shown.

[0009] Figure 2 The use of an exemplary embodiment of the present invention is shown Figure 1 Example of a systematic text transformation.

[0010] Figure 3 Another adjustable rich text conversion system according to an exemplary embodiment of the present invention is shown.

[0011] Figure 4 A process for controllable style-based text conversion according to an exemplary embodiment of the present invention is shown.

[0012] Figure 5 Another process for controllable style-based text conversion according to an exemplary embodiment of the present invention is shown.

[0013] Figure 6 A computer system according to an exemplary embodiment of the present invention is shown, upon which one or more components / steps of the techniques of the present invention may be implemented.

[0014] Figure 7 A cloud computing environment according to an exemplary embodiment of the present invention is shown.

[0015] Figure 8 An abstract model layer according to an exemplary embodiment of the present invention is shown. DETAILED DESCRIPTION

[0016] Illustrative embodiments of the present invention may be described herein in the context of illustrative methods, systems, and devices for controllable style-based text conversion. However, it should be understood that embodiments of the present invention are not limited to these illustrative methods, systems, and devices, but are more broadly applicable to other suitable methods, systems, and devices.

[0017] In various application scenarios, it is desirable to transform text. For example, text transformation is an important problem in the field of natural language generation (NLG). Text transformation systems such as translators, summarizers, dialog generators, etc. can be used. However, techniques are needed for performing text transformation in the field of "style", in which it is desirable to generate style variations of a given input text according to a set of defined input style specifications.

[0018] The illustrative embodiments enable conversion of a piece of text into its stylistic variants according to a plurality of adjustable stylistic specifications, wherein each stylistic specification or requirement may be defined vaguely (e.g., the target text should follow a domain-specific unique style of a particular domain to a certain level or degree specified by the user) or clearly defined (e.g., the target text should exhibit certain linguistic properties related to linguistic aspects such as emotional intensity, formality, tone, etc. specified by the user).

[0019] For example, consider the following scenario. A user wishes to convert a text so that the converted text focuses on specific aspects of the input and borrows a unique writing style from multiple fields (e.g., the advertising field and the legal field). Further, the user wants a stronger influence from the advertising field compared to the legal field. The user opens an interface where the user inputs or otherwise provides a text fragment to be converted. The user also uses the interface to select a set of N fields that will affect the style of the target text. Because a pop-up window or other notification or request is sent to the user to provide the desired degree of influence of the style of each field in the specific N fields on the target text, the interface can be interactive. The interface can subsequently ask the user via additional input fields to allow the user to provide general language-specific controls on the tone, formality, intensity, etc. of the target text. The system receives these various inputs and generates style variants of the input text fragment.

[0020] Conventional techniques fail to provide this type of multi-style text conversion. For example, the input text may be passed through multiple text generators, where random sampling is used according to the output word probability (e.g., 80% from a first text generator of a first style, 20% from a second text generator of a second style). In this case, the sampling is random, and so may blend words from each text generator in a way that reduces fluency and appropriateness. Further, there is no guarantee that the output will conform to any provided style-specific controls.

[0021] Figure 1A system 100 capable of controllable style text conversion is shown. In some embodiments, the system 100 uses an unsupervised neural network and is suitable for the addition and removal of various parameters for different style requirements or specifications. The system 100 includes a text input module 102, in which a user can type or otherwise input or provide an input text fragment that will convert style. The system 100 also includes a field selection module 104, in which a user can select various fields of output text. Based on the selected target field, the system 100 uses an interactive interface that allows the user to appropriately weight different fields in the selected target writing style field to utilize a field-based style determination module 106 to obtain a unique writing style based on the field (for example, it can be fuzzily defined). The system 100 also uses an interactive interface that allows the user to appropriately weight different linguistic aspects to utilize a linguistic aspect determination module 108 to obtain the linguistic aspects of the target text (for example, it can be clearly defined). The input text segments are input to the text conversion module 110 , along with weights for domain-based unique writing styles and linguistic aspects, which generates converted text output via the text output module 112 .

[0022] We will now describe Figure 1Example of use of the system. Consider a software developer who has created a text editor "Editor 1" and wants to distribute the editor for free for non-commercial use. The software developer has written a basic product description of the text editor: "Editor 1 is a free text editor. It supports RegEx-based searches. It is easy to integrate Editor 1 with Editor 2." However, the software developer may think that this description is too general to get people to use the new text editor. The software developer therefore provides the basic product description of the text editor as an input text snippet 202 (e.g., provided as input to the text input module 102 of the system 100). The input text cutter 202 is combined with a selection of a target writing style domain 204 (e.g., provided as input via the domain selection module 104 of the system 100) according to which the generated text should be processed. In this example, the target writing style domain 204 includes a "default" domain, an "academic" or "technical" domain, an "advertising" domain, and a "legal" domain. It should be understood that these domains are introduced by way of example only, and a variety of other domains can be selected, including but not limited to e-commerce, health care, travel, etc. The software developer utilizes the interactive interface 206 (e.g., provided by the domain-based style determination module 106 of the system 100) to control the relative influence of the target writing style domains 204 as shown by adjusting the sliders for each target writing style domain 204. The software developer further utilizes the interactive interface 208 (e.g., provided by the linguistic aspect determination module 108 of the system 100) to control the emotional intensity and tone or formality of the wording of the generated text as shown by adjusting the sliders for the different linguistic aspects.

[0023] Various information is provided to a text conversion module 210 (e.g., the text conversion module 110 of the system 100), which provides an output text 212 "Would you like to use a text editor with RegEx-based search capabilities? Try the excellent editor 1 now. It is completely free for non-commercial use under the GNU-GPL license." It should be understood that the generated output text 212 will vary depending on the selected target domain 204, the domain-based unique writing style specification 206, and the linguistic aspect specification 208. For example, if the "legal" domain is not part of the target domain 204, the generated output text 212 may be "Would you like to use a text editor with RegEx search capabilities? Try the excellent editor 1 for free." However, the software developer may be concerned that this product description will lead other users to believe that the new text editor is free for use in commercial settings and the legal aspects are unclear. By adding the "legal" domain to the target domain 204 with relatively low impact as specified by the domain-based unique style specification 206, the generated output text 212 is Figure 2As shown, the output text 212 includes additional language regarding being licensed under the GNU-GPL and indicating that the text editor is free for non-commercial use.

[0024] In some embodiments, a system such as system 100 is provided that is configured to convert a text into its stylistic variants according to a plurality of adjustable style specifications. The adjustable style specifications may be defined vaguely, such as in the case of a unique domain-specific writing style, where the target or output text should follow the unique domain-specific writing style in a particular proportion specified by a user at runtime. The adjustable style specifications may also be defined explicitly, such as in the case of explicitly defining linguistic aspects, where the target or output text should be styled according to linguistic aspect parameters specified by a user at runtime. For example, in the case where the proportions of the various style specifications are provided as real number inputs (e.g., see Figure 2 Elements 206 and 208), the style specifications or requirements are adjustable.

[0025] The system in some embodiments takes as input: (i) a text snippet; (ii) a set of N domains from a set of D domains; (iii) N domains of options s style strengths; and (iv) L parameters of linguistic aspects. This input is used to generate an output text, which is a style-transformed version of (i) the input text fragment and conforms to the style specifications (ii)-(iv). The system of some embodiments may be characterized by the following ability: by considering a large-scale unlabeled corpus representing each domain D and for each of the L linguistic aspect parameters by relying on a natural language processing (NLP) tool that can measure the linguistic aspect parameter, learning to perform style transfer for each domain in an unsupervised setting at a user-specified suitable ratio. The system in some embodiments is also characterized by universality, because as long as domain-specific unlabeled corpora and NLP tools are available, the system can be trained for any finite set of writing style domains and linguistic parameters.

[0026] For D different domains and L different linguistic parameters, the system in some embodiments utilizes D domain-specific unlabeled corpora (e.g., believed to contain unique domain-specific writing styles for the D domains) and L NLP tools. The system can be viewed as comprising four main components: a shared encoder and decoder, a domain unique-style strength scorer, a language aspect scorer, and a domain classifier. Now referring to Figure 3 Describe examples of these components, Figure 3 Shows the workflow of our system for generating style text transfer.

[0027] During or during the training phase, Figure 3 The system trains a domain-unique style strength scorer module 306, a language aspect scorer synthesis module 310, and a domain classifier module 312 using a corpus 301 and NLP tools. A shared encoder module 314, decoders 316-1, 316-2, ... 316-D (collectively referred to as decoders 316), and a shared decoder module 318 are trained with a specific corpus, and the entire network is trained by minimizing the composite loss of fluency, appropriateness of conversion, and semantic relevance of the output text segment 319 relative to the input text segment 313 and style norms. During or during the test phase, the domain of the input text is determined in real time using the domain classifier module 312, and an appropriate one of the decoders 316 is selected based on the domain of the input text. The input text is encoded using the shared encoder module 314, and the encoded information is passed through the selected decoder 316 together with the input style parameters and until the shared decoder module 318, which generates a transformed output (e.g., an output text segment 319).

[0028] The techniques for preparing the domain-unique style strength scorer module 306, the language aspect scorer synthesis module 310, and the domain classifier module 312 will now be described. The domain-unique style strength scorer module 306 takes text as input and provides a list of scores, each score indicating how well the text follows the style of the domain. In some embodiments, the domain-unique style strength scorer module 306 utilizes a neural regressor. For training the domain-unique style strength scorer module 306, a one-hot list of tags is prepared for each sentence using a DS-based tagging module 302 that outputs a corpus 303 of domain strength (DS) tags from an input corpus 301 (e.g., if the sentence belongs to a certain domain, the corresponding score is 1, otherwise the score is 0). The correlation network-based training module 304 obtains the DS-tagged corpus 303 and provides training for the domain-unique style strength scorer module 306.

[0029] Language aspect scorer synthesis module 310 uses text as input, and calculates the score corresponding to each in a group of language or linguistic aspect. Language aspect scorer synthesis module 310 can utilize multiple regressors or classifiers (for example, depending on the nature of output), and each regressor or classifier is responsible for measuring a certain language aspect. In certain embodiments, use the technology of for example support vector regression (Support Vector Regressions), train the regressor or classifier of language aspect scorer synthesis module 310 under the setting of being supervised, for this technology, by the NLP tool that provides with NLP module 308, process multiple input corpus 301, generate label data. For example, if the user wishes to measure two linguistic aspects (for example, emotional intensity and formality), then can use the corresponding NLP tool from NLP tool library to generate the label data with language aspect intensity label corpus 309 form, these data are provided to language aspect scorer synthesis module 310 for training.

[0030] The domain classifier module 312 takes text as input and determines the domain to which the text belongs. For training, the domain classifier module 312 uses the input corpus 301 of different domains together with their associated domain tags. Various classification techniques can be utilized by the domain classifier module 312, including unsupervised machine learning techniques.

[0031] The core encoder and decoder (e.g., shared encoder module 314, decoder 316, and shared decoder module 318) are configured as follows. For D domains, the core encoder and decoder should have D input text segments 313, shared encoder modules 314, D decoders 316, and shared decoder modules 318. The input of the shared encoder module 314 is the input text segment 313 C 1 , C 2 , ...C D The shared encoder module encodes these segments into corresponding embeddings, which are provided to the appropriate decoder 316. The decoder 316 can take additional inputs as shown, which correspond to the so-called The domain-specific ("dynamic" or "dyn") specification and is called In some embodiments, the shared encoder module 314, the decoder module 316, and the shared decoder module 318 utilize recurrent neural units.

[0032] To train the core encoder and decoder, an iterative process is used in which one input text segment from a domain-specific corpus is considered at a time. In each step, a text segment C from a corpus is selected iIn the following description, it is assumed that a text segment C from domain 1 is selected. 1 . The default dynamic specification Set to 1 and set the rest (where i≠1) is set to 0. The NLP module 308 is used to calculate the fragment default language or linguistic aspects, and the rest (where i≠1) is set to a 0 vector. C 1 The shared encoder module 314 is used to encode the data and the shared encoder module 314 is used to pass the data through the D decoders 316. The D decoders 316 also receive the data. and The output of decoder 316 is passed through a shared decoder module 318, which combines the different decoder outputs to generate an output text segment GC. 1 . Enable output GC 1 The domain-unique style strength scorer module 306 and the language aspect scorer synthesis module 310 generate the following scores for all domains and all language aspects, respectively: and The score vector 321 of .

[0033] Learning the transformation is achieved using a set of losses 323, 325, and 327. Loss 323 (called reconstruction loss) is defined as L 1 (GC i ,C i ) and ensures that the model can generate fluent sentences that follow the fluency of the input text. The loss 325 (called back-translation loss) is defined as L 2 (GC i ,C i ) and is a composite loss (including D loss terms for D domains). Each loss term is calculated by performing back translation via all decoders except the current decoder. The loss 327 is referred to as a control loss and includes a dynamic control loss and a static control loss. The dynamic control loss (defined as ) indicates the gap between the input domain strength specification and the measured strength of the output text. The static control loss (defined as ) indicates the gap between the input linguistic aspect norm and the measured linguistic aspect of the output text. Various differentiable loss functions can be used to calculate losses 323, 325, and 327, including categorical cross-entropy and squared loss. For optimization and validation, standard neural settings can be used.

[0034] To test the core encoders and decoders, the input text C is preprocessed (e.g., normalized) and passed through the domain classifier module 312 to detect the default domain of the input text. For each domain specified by the user, the corresponding decoder 316 is enabled, and the other decoders are disabled (e.g., they receive zero-valued inputs). The input text C is input into the system along with the domain strength and language aspect values ​​as appropriate, and the output GC is returned.

[0035] Figure 3 The system utilizes a domain-specific corpus 301 for each domain involved. The core encoder and decoder (e.g., shared encoder module 314, decoder 316, and shared decoder module 318) are assumed to be implemented using a deep learning execution platform (e.g., Tensorflow, Pytorch, keras, caffe, etc.) or other types of deep learning or machine learning platforms. For each tunable language aspect parameter, it is assumed that the NLP module 308 includes an NLP tool (e.g., a classifier, a regressor, a metric calculation system, etc.) for measuring that parameter. Further, Figure 3 Various aspects of the system assume the availability and use of NLP frameworks for text normalization, tokenization, learning embeddings, measuring semantic relatedness, etc.

[0036] To expand Figure 3 The system can add or remove domain or language aspect parameters using the following techniques. To add a new domain unique style, the associated domain specific corpus for that domain should be obtained. A separate decoder 316 for the newly added domain should be attached to the system and the system retrained. To add a new language aspect parameter, the corresponding NLP framework for measuring that parameter is obtained and the system is trained again with the existing dataset. For removing domain or language aspect parameters, the corresponding component is removed from Figure 3 Removing and retraining the system is not necessary, but can be performed if desired.

[0037] The system for multi-style text conversion described in this article can be used in a variety of application areas, including various NLG systems, such as intelligent dialogue systems, persuasive NLG systems, reinforcement services, and other types of NLG-related services, including automatic specification, manual creators, automatic content writers, etc.

[0038] In some embodiments, a framework for controllable natural language conversion is provided. Recognizing that the requirement for parallel corpora is in fact unsustainable for controllable generation tasks, an unsupervised training scheme is introduced in some embodiments. The framework utilizes a deep neural encoder-decoder, which is enhanced with text conversion knowledge through an auxiliary module, which is also referred to as a scorer in this article. Based on NLP tools, the scorer determines the learning scheme of the encoder-decoder based on its actions. The framework can be applied to various text conversion tasks, where the type and degree of style-based conversion can be controlled by the user at runtime.

[0039] Automatic text style conversion is a key goal of text-to-text NLG. Conventional approaches to this text-to-text conversion are often supervised, using statistical machine translation models, or are based on templates and rules. Supervised NLG requires a large-scale parallel corpus for training, which is a major obstacle to expansion to a variety of use cases. For example, in the context of a separate automatic dialogue system, there are multiple scenarios in which system replies (which may come from a database) need to be converted for their tone (e.g., politeness, excitement, etc.), formality level (e.g., informal, formal, etc. based on user personality), or complexity (e.g., such as simplifying linguistics or field-specific terms in the legal or medical fields). As such requirements and use cases continue to grow, obtaining a large-scale parallel corpus for such text conversion tasks is in fact unsustainable.

[0040] From a scientific perspective, all of this supervised processing using multiple parallel corpora attempts to learn both language conversion (e.g., to maintain semantics) and style conversion for each task at the same time. Many observations have been made about this. First, since maintaining language semantics is necessary for conversion, and only the properties or style of the text need to be changed, it should be possible to disconnect these two aspects. Second, it is more computationally economical to independently verify these aspects at the output stage (e.g., using NLP techniques) compared to specifying the required conversion of each input text with its output examples. Third, the degree or amplitude of the expected properties required for the output (e.g., readability level, politeness level, etc.) should be controllable. These observations are taken into account when developing an unsupervised approach to style-based text conversion.

[0041] In some embodiments, the framework relies only on unlabeled text for initialization with the entirety of the NLP module. The framework described herein can be used for a variety of style transfer tasks controlled by the user. For example, again referring to Figure 2 , the user-selected controls or style specifications may include different target writing style domain attributes (e.g., academic, advertising, legal) with their corresponding levels, strengths, or other weights (e.g., as described above). Figure 2). Advantageously, the framework described herein is able to perform style-based text transformation with fuzzily defined target writing style domain features (possibly combined with clearly defined linguistic aspects). As an example of linguistic style transformation, consider the task of "formalization" of input text, which is relevant to multiple NLG applications, such as formal conversation generation in the field of compliance and adaptation, email response composition, summary document generation, etc. Moreover, such a system for text formalization can assist professional authors in a similar way to computer-assisted translation (CAT) systems assisting human translators, paving the way for cost-effective and time-effective solutions for content creation.

[0042] In some embodiments, a framework is established using an encoder-decoder module pre-trained with unlabeled text. (For example, for unique styles and linguistic aspects based on domains) the decoder may additionally use user-specific control parameters as input. The knowledge of the desired conversion is obtained by using an auxiliary module or a scorer, which determines the learning scheme of the encoder-decoder based on the action of the encoder-decoder. The score may be based on an NLP tool that generates a score, and the score indicates, for example, how formal the generated text is, whether the generated text is fluent, and whether the generated text has semantics similar to the input, whether the generated text corresponds to the selected target writing style field, etc. The framework may be trained in multiple iterations, wherein each iteration includes two stages of exploration and exploitation. In the exploration stage, the decoder randomly samples the candidate text for a given input and automatically generates training data for controllable generation by means of a scorer. In the exploitation stage, the encoder-decoder is retrained with the example generated in this way.

[0043] Unsupervised NLG is challenging due to the following facts: (i) the output space is more complex and structured, making unsupervised learning more difficult; (ii) the metrics used to evaluate NLG systems without reference to output text are elusive. In some systems, architectures for unsupervised language translation using unsupervised autoencoder-based dictionary induction techniques are used. This approach focuses primarily on cross-language conversion and requires multiple unlabeled corpora from different languages. As a result, these approaches are difficult to scale to the goal of controllable text conversion within a single language, and further there is no concept of style-based control in language translation. Techniques for controllable generation can utilize or take into account control parameters such as sentiment, tense, etc., and generate random sentences that conform to the control. However, these systems do not convert a given input text, but simply randomly sample the text based on user-provided controls.

[0044] In other ways, a variational autoencoder (VAE) is trained together with an achievement prediction module to correct the input so that the output has a higher expected result. However, the VAE / achievement prediction system may not be able to maintain the semantic relevance between the input and output, and relevance is necessary for many text conversion tasks. Further, the VAE / achievement prediction system does not consider external control parameters. Other ways can be based on: unsupervised style transformation based on emotions and attributes, semi-supervised transformation by back translation using a translation corpus, formal-informal text classification using linguistic features, polite conversation generation using an encoder-decoder model, etc. However, these ways do not perform controllable text conversion. Similarly, the generation framework for formal text generation and interpretation generation is template-based or supervised, and is not controllable. The language generation system that incorporates a scorer based on NLP is unsupervised, but encounters convergence problems during training.

[0045] The framework used in some embodiments is designed to receive text input (eg, a sentence) and a set of control parameters (eg, vaguely defined domain-based unique style parameters and explicitly defined linguistic aspect parameters).

[0046] Figure 3 The core encoder and decoder of the proposed algorithm can utilize a framework and learning scheme that includes pre-training and multiple iterations of exploration and exploitation. Figure 3 As shown, each decoder 316 includes a control input in the target writing style area) and (For linguistics.) Control input can go through the embedding layer.

[0047] Figure 3 The core encoder and decoder of can be pre-trained so that in the first iteration of exploration, the shared decoder module 318 generates the same output GC as the input C. Since the sampled sentences are considered to have better scores (or they would not be selected), the first iteration of exploration ensures that the output of the synthesized data GC is different from the input C and has a better cumulative score (e.g., the score vector GV dyn and GV stat During exploitation, the shared encoder module 314, decoder 316, and shared decoder module 318 are trained using data generated during exploration.

[0048] The shared encoding module 314, decoder 316, and shared decoder module 318 may be trained in a classification setting, where batches of labeled data are fed into multiple iterations and various losses (e.g., L 1 , L 2 , L 3 and L 4 ) minimized.

[0049] Figure 4 A process flow 400 for controllable style-based text conversion is depicted. The process flow 400 begins at step 402, where input text to be converted is obtained. In step 404, a set of style specifications is selected for converting the input text obtained in step 402. The set of style specifications includes: one or more target writing style domains selected from a plurality of writing style domains; a weight for each target writing style domain, the weight representing the relative impact of the target writing style domain on the conversion of the input text; and a weight for each linguistic aspect in a set of linguistic aspects for the conversion of the input text. The plurality of writing style domains may include a default domain, an academic domain, a technical domain, an advertising domain, a legal domain, a medical domain, etc. The set of linguistic aspects may include formality, emotional intensity, tone, etc.

[0050] In some embodiments, step 404 includes providing a real number input proportional to the expected relative impact of the selected target writing style domain on the style-converted output text as the weight of the selected target writing style domain. Step 404 may also include providing a real number input corresponding to the expected impact of the linguistic aspect on the style-converted output text as the weight of each linguistic aspect in the set of linguistic aspects.

[0051] Process flow 400 continues at step 406 by generating one or more style-transferred output texts based at least in part on the set of style specifications using an unsupervised neural network. The unsupervised neural network may include a deep learning network including a plurality of GRUs.

[0052] In some embodiments, step 406 includes: generating an embedding for the input text, generating domain-specific style-converted output text using each decoder of a subset of multiple decoders, the subset of the multiple decoders being associated with the selected target writing style domain and provided with the embedding for the input text, the weight of the selected target writing style domain, and the weight of the set of linguistic aspects; generating a given style-converted output text in the style-converted output text as a combination of the domain-specific style-converted output text from each decoder in the subset of the multiple decoders.

[0053] Process flow 400 may further include training an unsupervised neural network. Training the unsupervised neural network may include: generating an embedding for a given training text from a given domain in a plurality of domains; generating domain-specific style-transferred training output text using a plurality of decoders associated with the plurality of writing style domains, a given one of the decoders corresponding to the given domain provided with a set of training weights for a set of training writing style domains and the set of linguistic aspects, other ones of the decoders corresponding to other domains in the plurality of domains provided with zero-valued weights for the training writing style domains and the set of linguistic aspects; and generating the given style-transferred training output text as a combination of the domain-specific style-transferred training output texts from the plurality of decoders.

[0054] Training the unsupervised neural network may further include: determining a domain style score vector for a given style-converted training output text, the domain style score vector comprising a list of scores, each score representing the extent to which the given style-converted training output text follows the style of one of a set of training writing style domains; and determining a linguistic aspect score vector for the given style-converted output text, the linguistic aspect score vector comprising a score corresponding to each linguistic aspect in the set of linguistic aspects, the score being calculated using a natural language processing tool trained for the linguistic aspect. Training the unsupervised neural network may further include: calculating a plurality of loss metrics and minimizing such loss metrics. The loss metric may include: a first control loss indicating the gap between (i) training weights for the set of training writing style domains and (ii) scores in the domain style score vector; a second control loss indicating the gap between (i) training weights for the set of linguistic aspects and (ii) scores in the linguistic aspect score vector; a reconstruction loss calculated by comparing the fluency of the training input text with the fluency of the given style-transferred training output text; and a back-translation loss calculated by performing a back-translation of the given style-transferred training output text by the multiple decoders.

[0055] Figure 5A process flow 500 for controllable style-based text conversion is described. The process flow 500 begins at step 502, receiving an input text segment to be style converted according to a set of style control parameters specified in a control vector. In step 504, the input text segment is passed to an unsupervised neural network including a plurality of GRUs. In step 506, a hidden representation of the input text segment is determined using a first subset of the plurality of GRUs arranged in a stack that provide an encoder. In step 508, a style-converted output text segment is generated using a second subset of the plurality of GRUs that provide a decoder. The decoder generates each word in the style-converted output text segment using a nonlinear function that outputs a probability distribution of a given word in the style-converted output text segment based on (i) an embedding of previously generated words of the style-converted output text segment, (ii) the control vector, (iii) a vector obtained by focusing on the hidden representation of the input text segment, and (iv) a hidden state of the decoder of the unsupervised neural network.

[0056] Processing flow 500 may further include training the unsupervised neural network by repeating steps 504, 506, and 508 using a plurality of training input text segments. For each generated style-converted output text segment corresponding to a given one of the training input text segments, training may include: generating a set of variants of the generated style-converted output text segment; selecting a given one of the variants of the generated style-converted output text segment based at least in part on the semantic relevance of the given training input text segment, the fluency of the given variant of the generated style-converted output text segment as measured using a specified language model, and a readability rating score of the given variant of the generated style-converted output text segment; and determining a set of style control parameters associated with the given variant of the generated style-converted output text segment. The encoder and decoder of the unsupervised neural network are trained using the given variant of the generated style-converted output text segment and its associated set of style control parameters as labeled training data.

[0057] The input received in step 502 may include text encoded in an embedded form, such as by an embedding lookup operation. The embeddings during training may be further encoded into a deep representation using a long short-term memory (LSTM) unit. Overall, this provides a shared encoder (e.g., Figure 3 For each domain considered (e.g., the target writing style domain), there is a decoder (e.g., Figure 3Such LSTM units may be arranged in a bidirectional network. The decoder takes as input the encoded deep representation, and the control vector or a portion thereof. For example, the control vector may have a set of dynamic (e.g., specific to the target writing style domain) and static (e.g., specific to language or linguistics) control values. The outputs of all such decoders from multiple target writing style domains are combined and sent to a shared decoder (e.g., Figure 3 The shared decoder module 318 in . The shared decoder then predicts the sentence for output. Using the decoder output, different scores can be calculated. These scores can be real numbers representing various aspects of the output, including (i) how fluent or grammatically correct the output is, (ii) how appropriate or relevant the output is to the input, and (iii) to what extent the output conforms to the control vectors (e.g., dynamic and static control values) given as input. The inverse of the score can be considered the loss. For the calculation of the score, a linguistic or linguistic aspect scorer (e.g., Figure 3 310) and domain strength scorers (e.g., Figure 3 Domain-unique style strength scorer module 306 in.

[0058] During training, the goal is to minimize the loss. During testing or real-time use, the input text is fed into the shared encoder and encoded by the shared encoder. The control vector (e.g., including dynamic and static control values) and the encoded output from the shared encoder are provided to the decoder, whose outputs are concatenated and passed through the shared decoder. The output text predicted by the shared decoder is a sentence or other text fragment with the desired characteristics specified by the control values ​​in the control vector.

[0059] Embodiments of the present invention include systems, methods and / or computer program products at any possible level of technical detail integration. The computer program product may include a computer-readable storage medium (or multiple computer-readable storage media) having computer-readable program instructions thereon for causing a processor to perform various aspects of the present invention.

[0060] Computer readable storage medium can be a tangible device that can hold and store instructions used by an instruction execution device. Computer readable storage medium can be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing devices. A non-exhaustive list of more specific examples of computer readable storage medium includes the following: portable computer disk, hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, mechanical encoding device, such as a punch card or a convex structure in a groove on which instructions are stored, and any suitable combination of the foregoing devices. The computer readable storage medium used herein is not interpreted as a transient signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated by a waveguide or other transmission medium (e.g., a light pulse by a fiber optic cable), or an electrical signal transmitted by a wire.

[0061] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to a corresponding computing / processing device or downloaded to an external computer or external storage device via a network such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical transmission fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in the corresponding computing / processing device.

[0062] The computer-readable program instructions for performing the operation of the present invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, integrated circuit configuration data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and procedural programming languages, such as "C" programming language or similar programming languages. The computer-readable program instructions may be executed entirely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, electronic circuits including, for example, programmable logic circuits, field programmable gate arrays (FPGAs), or programmable logic arrays (PLAs) may execute computer-readable program instructions by utilizing state information of the computer-readable program instructions to personalize the electronic circuits, thereby performing various aspects of the present invention.

[0063] Various aspects of the present invention are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to embodiments of the present invention. It should be understood that each box in the flowchart and / or block diagram, and the combination of boxes in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0064] These computer-readable program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device is created to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram. These computer-readable program instructions may also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to operate in a specific manner, so that the computer-readable storage medium in which the instructions are stored includes an article of manufacture, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0065] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device, so that a series of operational steps are performed on the computer, other programmable apparatus, or other device to produce computer-implemented processing, thereby causing the instructions executed on the computer, other programmable apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0066] Flowcharts and block diagrams in the accompanying drawings show the possible architecture, function, and operation of the system, method, and computer program product according to multiple embodiments of the present invention. In this regard, each square block in the flow chart or block diagram may represent a part of a module, a program segment, or an instruction, and a part of this module, a program segment, or an instruction includes one or more executable instructions for realizing the logical function of the specification. In some alternative implementations, the function marked in the square block may occur in a sequence different from that marked in the accompanying drawings. For example, two square blocks shown in succession may actually be performed substantially in parallel, or sometimes these square blocks may be performed in reverse order, depending on the function involved. It should be noted that each square block in the block diagram and / or flow chart, and the combination of the square blocks in the block diagram and / or flow chart may be realized by a system based on dedicated hardware that performs a specified function or action or implements a combination of dedicated hardware and computer instructions.

[0067] One or more embodiments may utilize software running on a general purpose computer or workstation. Figure 6 , there is a computer system / server 612 in computing node 610, which can operate under a variety of other general or special computer system environments or configurations. Examples of well-known computing systems, environments and / or configurations that may be suitable for computer system / server 612 include, but are not limited to, personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, small computer systems, mainframe computer systems, mobile and wearable devices, and distributed cloud computing environments including any of the above systems or devices, etc.

[0068] Computer system / server 612 may be described in the general context of computer system executable instructions (such as program modules) executed by a computer system. In general, program modules may include routines, programs, object programs, components, logic, data structures, etc. that perform specific tasks or implement specific abstract data types. Computer system / server 612 may be practiced in a distributed cloud computing environment, where tasks are performed by remote processing devices linked through a communication network. In a distributed cloud computing environment, program modules may be located in both local and remote computer system storage media, including memory storage devices.

[0069] like Figure 6 As shown, the computer system / server 612 in the computing node 610 is shown in the form of a general-purpose computing device. The components of the computer system / server 612 may include, but are not limited to: one or more processors or processing units 616, a system memory 628, and a bus 618 that couples various system components including the system memory 628 to the processor 616.

[0070] Bus 618 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. Such architectures include, by way of example, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MCA) bus, an Enhanced ISA (EISA) bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0071] Computer system / server 612 typically includes a variety of computer system readable media. Such media can be any available media that can be accessed by computer system / server 612, and it includes both volatile and nonvolatile media, removable and non-removable media.

[0072] System memory 628 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 630 and / or cache memory 632. Computer system / server 612 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, memory system 634 may be used to read and write to non-removable, non-volatile magnetic media (not shown, commonly referred to as a "hard drive"). Although not shown, a magnetic disk drive for reading and writing to a removable non-volatile magnetic disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (such as a CD-ROM, DVD-ROM or other optical media) may be provided. In these cases, each drive may be connected to bus 618 via one or more data media interfaces. As depicted and described herein, memory 628 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of an embodiment of the present invention. A program / utility 640 having a set (at least one) of program modules 642 may be stored in, by way of example and not limitation, memory 628, along with an operating system, one or more application programs, other program modules, and program data. Each or a combination of the operating system, one or more application programs, other program modules, and program data may include an implementation of a networking environment. Program modules 642 generally perform the functions and / or methods of embodiments of the invention described herein.

[0073] The computer system / server 612 may also communicate with one or more external devices 614 (such as a keyboard, pointing device, display 624, etc.), may also communicate with one or more devices that enable a user to interact with the computer system / server 612, and / or communicate with any device that enables the computer system / server 612 to communicate with one or more other computing devices (e.g., a network card, a modem, etc.). Such communication may be performed via an I / O interface 622. However, the computer system / server 612 may communicate with one or more networks (such as a LAN, a general WAN, and / or a public network (e.g., the Internet)) via a network adapter 620. As shown, the network adapter 620 communicates with other components of the computer system / server 612 via a bus 618. It should be understood that, although not shown in the figure, other hardware and / or software components may be used in conjunction with the computer system / server 612. Examples include, but are not limited to, microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archive storage systems.

[0074] First, it should be understood that although the present disclosure includes a detailed description about cloud computing, the implementation of the technical solutions recorded therein is not limited to a cloud computing environment, but can be implemented in combination with any other type of computing environment now known or later developed.

[0075] Cloud computing is a service delivery model for convenient, on-demand network access to a shared pool of configurable computing resources. Configurable computing resources are resources that can be quickly deployed and released with minimal management cost or interaction with the service provider, such as networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services. This cloud model can include at least five characteristics, at least three service models, and at least four deployment models.

[0076] Features include:

[0077] On-demand self-service: Cloud consumers can unilaterally and automatically deploy computing capabilities such as server time and network storage on demand without human interaction with the service provider.

[0078] Broad network access: Computing power can be accessed over the network through standard mechanisms that facilitate the use of the cloud through different types of thin-client or thick-client platforms (e.g., mobile phones, laptops, personal digital assistants (PDAs)).

[0079] Resource pool: The computing resources of the provider are grouped into resource pools and serve multiple consumers through a multi-tenant model, where different physical and virtual resources are dynamically allocated and reallocated on demand. Generally, consumers cannot control or even know the exact location of the provided resources, but can specify the location (such as country, state or data center) at a higher level of abstraction, so it is location-independent.

[0080] Rapid elasticity: The ability to quickly and elastically (sometimes automatically) deploy computing power to scale up quickly, and quickly release it to scale down quickly. To consumers, the computing power available for deployment often appears to be unlimited, and any amount of computing power can be accessed at any time.

[0081] Measurable services: Cloud systems automatically control and optimize resource usage by leveraging metering capabilities at a level of abstraction appropriate to the type of service (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported, providing transparency to both service providers and consumers.

[0082] The service model is as follows:

[0083] Software as a Service (SaaS): The capability provided to consumers is to use the provider's applications running on a cloud infrastructure. Applications can be accessed from a variety of client devices through a thin client interface such as a web browser (e.g., web-based email). Aside from limited user-specific application configuration settings, consumers neither manage nor control the underlying cloud infrastructure including networks, servers, operating systems, storage, or even individual application capabilities.

[0084] Platform as a Service (PaaS): The capability provided to consumers is to deploy applications created or acquired by consumers on the cloud infrastructure, which are created using programming languages ​​and tools supported by the provider. Consumers neither manage nor control the underlying cloud infrastructure including networks, servers, operating systems or storage, but have control over the applications they deploy and may also have control over the configuration of the application hosting environment.

[0085] Infrastructure as a Service (IaaS): The capabilities provided to consumers are processing, storage, network, and other basic computing resources on which consumers can deploy and run arbitrary software including operating systems and applications. Consumers neither manage nor control the underlying cloud infrastructure, but have control over the operating system, storage, and applications they deploy, and may have limited control over selected network components (such as host firewalls).

[0086] The deployment model is as follows:

[0087] Private Cloud: Cloud infrastructure is run solely for an organization. The cloud infrastructure can be managed by the organization or a third party and can exist inside or outside the organization.

[0088] Community Cloud: A cloud infrastructure is shared by several organizations and supports a specific community with common interests (e.g., mission, security requirements, policies, and compliance considerations). A community cloud can be managed by multiple organizations within the community or by a third party and can exist inside or outside the community.

[0089] Public cloud: Cloud infrastructure is provided to the public or to large industry groups and is owned by an organization that sells cloud services.

[0090] Hybrid cloud: A cloud infrastructure consisting of two or more clouds (private, community or public) in a deployment model that remain distinct entities but are bound together by standardized or proprietary technologies (such as cloud burst sharing for load balancing between clouds) that enable data and application portability.

[0091] The cloud computing environment is service-oriented, with characteristics centered on statelessness, low coupling, modularity, and semantic interoperability. The core of cloud computing is the infrastructure that includes a network of interconnected nodes.

[0092] Reference now Figure 7 , in which an exemplary cloud computing environment 750 is shown. As shown, the cloud computing environment 750 includes one or more cloud computing nodes 710 with which a local computing device used by a cloud computing consumer can communicate, and the local computing device can be, for example, a personal digital assistant (PDA) or a mobile phone 754A, a desktop computer 754B, a laptop computer 754C and / or an automobile computer system 754N. The cloud computing nodes 710 can communicate with each other. The cloud computing nodes 710 can be physically or virtually grouped in one or more networks including but not limited to private clouds, community clouds, public clouds or hybrid clouds, or a combination thereof as described above (not shown in the figure). In this way, cloud consumers can request infrastructure as a service (IaaS), platform as a service (PaaS) and / or software as a service (SaaS) provided by the cloud computing environment 750 without having to maintain resources on local computing devices. It should be understood that Figure 7 The various types of computing devices 54A-N shown are merely illustrative, and cloud computing node 710 and cloud computing environment 750 can communicate with any type of computing device on any type of network and / or network addressable connection (eg, using a web browser).

[0093] Reference now Figure 8 , which shows a cloud computing environment 750 ( Figure 7 ) provides a set of functional abstraction layers. First of all, it should be understood that Figure 8The components, layers, and functions shown are only exemplary and the embodiments of the present invention are not limited thereto. Figure 8 As shown, the following layers and corresponding functions are provided:

[0094] The hardware and software layer 860 includes hardware and software components. Examples of hardware components include: host 861; server 862 based on RISC (Reduced Instruction Set Computer) architecture; server 863; blade server 864; storage device 865; network and network components 866. Examples of software components include: network application server software 867 and database software 868.

[0095] The virtualization layer 870 provides an abstraction layer that can provide examples of the following virtual entities: virtual servers 871 , virtual storage 872 , virtual networks 873 (including virtual private networks), virtual applications and operating systems 874 , and virtual clients 875 .

[0096] In one example, the management layer 880 may provide the following functions: Resource provisioning function 881: Provide dynamic acquisition of computing resources and other resources for performing tasks in a cloud computing environment; Metering and pricing function 882: Track the cost of resource usage within the cloud computing environment and provide bills and invoices for this purpose. In one example, the resource may include an application software license. Security function: Provide identity authentication for cloud consumers and tasks, and provide protection for data and other resources. User portal function 883: Provide access to the cloud computing environment for consumers and system administrators. Service level management function 884: Provide allocation and management of cloud computing resources to meet the required service levels. Service level agreement (SLA) planning and fulfillment function 885: Provide pre-arrangement and provision for future demand for cloud computing resources predicted according to the SLA.

[0097] The workload layer 890 provides examples of functions that may be implemented in a cloud computing environment. In this layer, examples of workloads or functions that may be provided include: mapping and navigation 891; software development and lifecycle management 892; virtual classroom teaching provision 893; data analysis processing 894; transaction processing 895; and

[0098] The embodiments of the present invention have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terms used herein are selected to best explain the principles of the embodiments, practical applications, or technical improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A computer-implemented method comprising the following steps: Get the input text; selecting a set of style specifications for converting the input text, the set of style specifications comprising: one or more target writing style domains selected from a plurality of writing style domains; a weight for each of the target writing style domains, the weight representing a relative influence of the target writing style domain on the conversion of the input text; and a weight for each of a set of linguistic aspects for the conversion of the input text; and generating, using an unsupervised neural network, one or more style-transferred output texts based at least in part on the set of style specifications; Wherein, the steps are performed by at least one processing device, wherein the plurality of writing style domains comprises two or more of: a default domain; an academic domain; a technical domain; an advertising domain; a legal domain; and a medical domain, and wherein the set of linguistic aspects comprises at least one of: formality, emotional intensity, and tone.

2. The computer-implemented method of claim 1 , wherein: The selecting the set of style specifications includes providing a real number input as a weight for the selected target writing style domain that is proportional to an expected relative influence of the selected target writing style domain on the style-converted output text.

3. The computer-implemented method of claim 1 , wherein: The selecting the set of style specifications includes providing a real number input corresponding to a desired impact of the set of linguistic aspects on the style-converted output text as a weight for each linguistic aspect of the set of linguistic aspects.

4. The computer-implemented method of claim 1 , wherein: The unsupervised neural network includes a deep learning network, and the deep learning network includes a plurality of gated recurrent units.

5. The computer-implemented method of claim 1 , wherein: Generating the one or more style-converted output texts comprises: generating an embedding for the input text; generating a domain-specific style-transferred output text using each decoder of a subset of a plurality of decoders, the subset of the plurality of decoders being associated with the selected target writing style domain and provided with the embedding for the input text, the weight for the selected target writing style domain, and the set of linguistic aspect weights; and A given one of the style-transferred output texts is generated as a combination of the domain-specific style-transferred output texts from each decoder in the subset of the plurality of decoders.

6. The computer-implemented method of claim 1 , comprising: Training the unsupervised neural network, wherein training the unsupervised neural network comprises: generating an embedding for a given training text from a given domain among the plurality of domains; generating domain-specific style-transferred training output text using a plurality of decoders associated with the plurality of writing style domains, a given one of the decoders corresponding to a given domain provided with a set of training weights for a set of training writing style domains and the set of linguistic aspects, and other ones of the decoders corresponding to other domains of the plurality of domains provided with zero-valued weights for the training writing style domains and the set of linguistic aspects; and A given style-transferred training output text is generated as a combination of the domain-specific style-transferred training output texts from the plurality of decoders.

7. The computer-implemented method of claim 6, wherein: The training of the unsupervised neural network comprises: determining a domain style score vector for the given style-transferred training output text, the domain style score vector comprising a list of scores, wherein each score indicates to what extent the given style-transferred training output text follows the style of one of a set of training writing style domains; and A linguistic aspect score vector is determined for the given style-transferred output text, the linguistic aspect score vector comprising a score corresponding to each linguistic aspect of the set of linguistic aspects, the score being calculated using a natural language processing tool trained on the linguistic aspect.

8. The computer-implemented method of claim 7, wherein: The training of the unsupervised neural network comprises: calculating a first control loss indicating a difference between (i) the training weights for the set of training writing style domains and (ii) the scores in the domain style score vector; calculating a second control loss indicating a difference between (i) the set of trained weights for the linguistic aspects and (ii) the scores in the linguistic aspect score vector; calculating a reconstruction loss by comparing the fluency of the training input text with the fluency of the given style-transferred training output text; and A back-translation loss is calculated by performing a back-translation of the given style-transferred training output text through the plurality of decoders.

9. The computer-implemented method of claim 8, wherein: The training of the unsupervised neural network includes minimizing the first control loss, the second control loss, the reconstruction loss, and the back translation loss.

10. A computer program product, comprising program instructions executable by at least one computing device to cause the at least one computing device to perform the steps of any one of claims 1 to 9.

11. A system comprising: Memory; as well as At least one processor coupled to the memory and configured to perform the steps of any one of claims 1 to 9.

12. A computer-implemented method comprising the steps of: receiving an input text segment to be style-transferred according to a set of style control parameters specified in a control vector; Passing the input text segment to an unsupervised neural network comprising a plurality of gated recurrent units; determining a hidden representation of the input text segment using a first subset of the plurality of gated recurrent units arranged in a stack provided to an encoder; as well as generating a style-converted output text segment using a second subset of the plurality of gated recurrent units provided to a decoder, the decoder generating each word in the style-converted output text segment using a non-linear function, the non-linear function outputting a probability distribution for a given word in the style-converted output text segment based on (i) embeddings of previously generated words of the style-converted output text segment, (ii) the control vector, (iii) a vector obtained by attending to the hidden representation of the input text segment, and (iv) a hidden state of a decoder of the unsupervised neural network; Wherein, the steps are performed by at least one processing device, wherein the control vector has a set of control values ​​specific to a target writing style domain and control values ​​specific to linguistic aspects, wherein the target writing style domain includes two or more of: a default domain; an academic domain; a technical domain; an advertising domain; a legal domain; and a medical domain, and wherein the linguistic aspects include at least one of: formality, emotional intensity, and tone.

13. The computer-implemented method of claim 12, comprising: The unsupervised neural network is trained by repeating the transferring, determining, and generating steps for each training input text segment in a plurality of training input text segments, and for each generated style-transferred output text segment corresponding to a given training input text segment in the training input text segments: generating a set of variants of the generated style-transferred output text segment; selecting a given variant of the variants of the generated style-transferred output text segment based at least in part on the semantic relevance of the given training input text segment, the fluency of the given variant of the generated style-transferred output text segment as measured using a specified language model, and a readability grade score of the given variant of the generated style-transferred output text segment; and A set of style control parameters associated with the given variant of the generated style-transferred output text segment is determined.

14. The computer-implemented method of claim 13, comprising: The encoder and decoder of the unsupervised neural network are trained using the given variant of the generated style-transferred output text segment and its associated set of style control parameters as labeled training data.

Images