Text generation method and device based on dynamic style, equipment and medium

Abstract

The invention relates to the technical field of natural language processing, can be applied to business system platforms of financial science and technology, medical health and the like, and discloses a text generation method, device and equipment based on a dynamic style and a medium, and the method comprises the following steps: encoding an input text into a vector sequence, and extracting potential vectors of the vector sequence; dynamic style types in the potential space corresponding to the potential vectors are extracted, and target style vectors corresponding to the dynamic style types are analyzed according to the potential vectors and the temperature parameters; converting the text demand style into a style demand vector, and calculating a style weight corresponding to the style demand vector; aggregating the target style vector according to the style weight to obtain a style aggregation vector, and projecting the style aggregation vector to a preset style perception decoder; and decoding the vector sequence by using the projected style perception decoder to obtain a target style text corresponding to the input text. And the generation accuracy of the style text of the user input content is improved.

Description

Technical Field

[0001] The present invention relates to the field of natural language processing technology, and in particular to a text generation method, device, equipment and medium based on dynamic style. Background Art

[0002] In order to meet the personalized needs of users, the style text corresponding to the user needs should be input based on the user input content. Therefore, in order to meet the personalized needs of users, it is necessary to accurately analyze the style of text generation to improve the accuracy of style text generation.

[0003] In the healthcare sector, users' diverse expression styles necessitate a diverse range of styles in generated text. For example, in scenarios like medical record summary generation, patient medication guidance, and customized doctor-patient communication scripts, using pre-set style templates to generate text content in a single style lacks flexible adaptation and fine-grained control over user-specified styles, resulting in low accuracy in generating text in the style desired by the user.

[0004] In the fintech sector, users' diverse expression styles place higher demands on text generation systems. For example, in scenarios like automated customer service replies, financial information push notifications, and personalized presentation of insurance contract terms, existing systems often only support a small number of pre-set style templates and lack flexible adaptation and fine-grained control over user-specified styles. This results in a monotonous generated content style and low accuracy in generating text tailored to user needs.

[0005] Existing technologies primarily rely on static latent space modeling or global style embedding methods. For example, models based on variational autoencoders (VAEs) introduce a disentanglement loss function to force the alignment of latent dimensions with specific semantic attributes. However, these methods require predefined style dimensions and are difficult to adapt to dynamic style combinations. They also fail to address the issue of automatically identifying style-specific dimensions, resulting in low accuracy when generating style text based on user input. Summary of the Invention

[0006] The present invention provides a method, device, equipment and medium for generating text based on dynamic style, so as to solve the technical problem of low accuracy when generating style text based on user input content.

[0007] In a first aspect, a method for generating text based on a dynamic style is provided, comprising:

[0008] Obtaining input text from a target user, encoding the input text into a vector sequence, and extracting a latent vector from the vector sequence using a preset encoder;

[0009] Extracting a dynamic style type in a latent space corresponding to the latent vector, and analyzing a target style vector corresponding to each dynamic style type based on the latent vector and a preset temperature parameter;

[0010] Obtaining the text requirement style of the target user, converting the text requirement style into a style requirement vector, and calculating the style weight corresponding to the style requirement vector;

[0011] Aggregating the target style vector according to the style weight to obtain a style aggregation vector, and projecting the style aggregation vector into a preset style-aware decoder;

[0012] The projected style-aware decoder is used to decode the vector sequence to obtain a target style text corresponding to the input text.

[0013] In a second aspect, a text generation device based on a dynamic style is provided, comprising:

[0014] A latent vector analysis module is used to obtain input text from a target user, encode the input text into a vector sequence, and extract a latent vector from the vector sequence using a preset encoder;

[0015] a target style vector analysis module, configured to extract the dynamic style type in the latent space corresponding to the latent vector, and analyze the target style vector corresponding to each dynamic style type based on the latent vector and a preset temperature parameter;

[0016] A style weight calculation module is used to obtain the text requirement style of the target user, convert the text requirement style into a style requirement vector, and calculate the style weight corresponding to the style requirement vector;

[0017] a style aggregation vector projection module, configured to aggregate the target style vector according to the style weight to obtain a style aggregation vector, and project the style aggregation vector into a preset style-aware decoder;

[0018] The target style text generation module is used to decode the vector sequence using the projected style-aware decoder to obtain the target style text corresponding to the input text.

[0019] In a third aspect, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned method for generating text based on a dynamic style when executing the computer program.

[0020] In a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned method for generating text based on dynamic style are implemented.

[0021] In the scheme implemented by the above-mentioned dynamic style-based text generation method, device, equipment and medium, the input text of the target user can be obtained through the client, the input text can be encoded into a vector sequence, and the latent vector of the vector sequence can be extracted using a preset encoder; the dynamic style type in the latent space corresponding to the latent vector is extracted, and the target style vector corresponding to each dynamic style type is analyzed based on the latent vector and a preset temperature parameter; the text requirement style of the target user is obtained, the text requirement style is converted into a style requirement vector, and the style weight corresponding to the style requirement vector is calculated; the target style vector is aggregated according to the style weight to obtain a style aggregation vector, and the style aggregation vector is projected into a preset style-aware decoder; the vector sequence is decoded using the projected style-aware decoder to obtain the target style text corresponding to the input text, and the target style text is fed back to the client. In the present invention, by simultaneously maintaining multiple style paths in the latent space, infinite style mixing is supported through linear combination; soft partitioning is achieved through a learnable temperature parameter to support continuous style adjustment; the gated latent representation is injected into the attention calculation of the decoder to achieve fine-grained style control; the style, content and reconstruction goals are jointly optimized to balance the style and content, thereby solving the technical problem of low accuracy in the generation of style text based on user input content. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0023] Figure 1 This is a schematic diagram of an application environment of a method for generating text based on dynamic style according to an embodiment of the present invention;

[0024] Figure 2 This is a flow chart of a method for generating text based on dynamic style in one embodiment of the present invention;

[0025] Figure 3 yes Figure 2 A schematic flow chart of a specific implementation of step S1;

[0026] Figure 4 yes Figure 2 A schematic flow chart of a specific implementation of step S3;

[0027] Figure 5 is a structural diagram of a text generation device based on dynamic style in one embodiment of the present invention;

[0028] Figure 6is a structural diagram of a computer device in one embodiment of the present invention;

[0029] Figure 7 FIG. 2 is another structural diagram of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0030] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0031] The text generation method based on dynamic style provided by the embodiment of the present invention can be applied in Figure 1 In an application environment, a client communicates with a server via a network. The server can obtain the input text of a target user through the client, encode the input text into a vector sequence, and extract the latent vector of the vector sequence using a preset encoder; extract the dynamic style type in the latent space corresponding to the latent vector, and analyze the target style vector corresponding to each dynamic style type based on the latent vector and a preset temperature parameter; obtain the target user's text requirement style, convert the text requirement style into a style requirement vector, and calculate the style weight corresponding to the style requirement vector; aggregate the target style vector according to the style weight to obtain a style aggregation vector, and project the style aggregation vector into a preset style-aware decoder; decode the vector sequence using the projected style-aware decoder to obtain the target style text corresponding to the input text, and feed the target style text back to the client. In the present invention, by simultaneously maintaining multiple style paths in the latent space, an infinite number of style mixtures can be supported through linear combination; soft partitioning can be achieved through a learnable temperature parameter to support continuous style adjustment; the gated latent representation is injected into the attention calculation of the decoder to achieve fine-grained style control; and the style, content, and reconstruction goals are jointly optimized to balance style and content, thereby solving the technical problem of low accuracy in generating style text based on user input content. The client can be, but is not limited to, various personal computers, laptops, smartphones, tablet computers, and portable wearable devices. The server can be implemented as an independent server or a server cluster consisting of multiple servers. The present invention is described in detail below through specific embodiments.

[0032] See also Figure 2 As shown, Figure 2 A flowchart of a method for generating text based on a dynamic style according to an embodiment of the present invention includes the following steps:

[0033] S1. Obtain input text from a target user, encode the input text into a vector sequence, and use a preset encoder to extract a latent vector from the vector sequence.

[0034] In the embodiment of the present invention, the input text refers to the original text data that needs to be style-transformed, and is the starting input of the entire style transfer process, such as "This product works well, I recommend trying it."

[0035] In detail, the input text of the target user can be obtained from a pre-stored storage area through a computer statement with data crawling function (such as Java statement, Python statement, etc.), where the storage area includes but is not limited to a database and a blockchain.

[0036] Furthermore, the input text is converted into a vector sequence, which can be converted into a vector sequence by a vector conversion model, wherein the vector conversion model includes but is not limited to a BERT model and a word2vec model, and the vector sequence refers to the vector representation of the input text, that is, the text x={x1,x2,...,x n}After tokenization, it is encoded into a vector sequence.

[0037] Furthermore, in order to decouple the content and style of the input text and provide a semantic basis for the style conversion of the input text, it is necessary to analyze the potential representation of the vector sequence.

[0038] In the embodiment of the present invention, the latent vector is a dense encoding of the input text in a high-dimensional semantic space, that is, the grammatical and semantic information of the original text is compressed into a fixed dimension, redundancy is eliminated, and the encoding goes beyond the deep semantics of the surface vocabulary.

[0039] In the embodiment of the present invention, referring to Figure 3 As shown, the extracting the latent vector of the vector sequence by using a preset encoder includes:

[0040] S31, inputting the vector sequence into the hidden layer of the encoder to obtain a hidden representation vector;

[0041] S32, extracting a target hidden vector corresponding to the last hidden layer in the encoder according to the hidden representation vector;

[0042] S33. Perform average pooling on the target hidden vector to obtain a latent vector of the vector sequence.

[0043] In detail, the encoder is a Transformer, which consists of multiple layers of self-attention modules and feedforward networks. Each layer receives an input sequence and outputs a higher-level semantic representation. The input vector sequence x∈R n×d, n is the sequence length, d is the vector dimension, after the lth hidden layer, the hidden representation Z∈R n×d , where the vector at each position encodes the contextual semantics of the Token, that is, the hidden representation is obtained using the Transformer encoder: Z = TransformerEncoder(x)∈R n×d .

[0044] Specifically, different hidden layers capture features at different levels. The shallow layer captures basic features such as lexical and syntactic features, such as part of speech and phrase structure, while the deep layer captures advanced features such as semantics and context, such as entity relationships and sentiment tendencies. The last hidden layer is selected as the target layer because it integrates global semantic information. The sequence dimension of the target hidden vector is average pooled, that is, the average pooled vector of the last layer is taken as the overall potential vector, that is, z = MeanPool(Z)∈R d , compress the variable-length sequence into a fixed-length vector, ignore the position information, and retain the overall semantics, thereby obtaining the latent vector corresponding to the vector sequence.

[0045] Furthermore, style-specific gating masks are generated through a style-sensitive gating network, and the dimensions related to a specific style in the latent space are dynamically identified through the attention mechanism.

[0046] For example, in a fintech scenario, the input text is "This investment product is very effective, I recommend trying it." The semantics of "investment product" and "effect" are parsed from the input, and a vector containing potential dimensions such as "yield" and "risk level" is generated through the encoder. In addition, in a medical scenario, the input text is "Instructions for use of the drug." The potential dimensions of the pharmacological effects of the drug are parsed from the input.

[0047] S2. Extract the dynamic style type in the latent space corresponding to the latent vector, and analyze the target style vector corresponding to each dynamic style type according to the latent vector and a preset temperature parameter.

[0048] In the embodiment of the present invention, the dynamic style type refers to a text style pattern in a latent space, represented by a latent vector, which changes dynamically with the context, semantic logic, or user needs, such as a formal style, a humorous style, etc.

[0049] In detail, the latent vector is a low-dimensional abstract feature vector to which the encoder maps the input text. The abstract space in which it is located is called the latent space. Each point in the latent space, that is, the latent vector, can be regarded as a compressed representation of the semantics, style and other features of the input text. Points at different positions correspond to different combinations of text attributes. The feature dimensions related to style are separated through a preset style parsing mechanism. Clustering algorithms such as K-means, DBSCAN or deep learning models such as autoencoders and Transformers are used to group the style features in the latent space. Each group corresponds to a dynamic style type, and each clustering result or style change pattern is mapped to a predefined dynamic style type label.

[0050] Furthermore, the multi-head attention mechanism is combined with temperature-adjusted gating and a multi-channel parallel processing architecture to achieve fine-grained style control and flexible style mixing capabilities.

[0051] In an embodiment of the present invention, the target style vector refers to mapping the dynamic style type into a specific numerical vector in the vector space, which is used to guide the subsequent decoder to generate the target style text. The style-sensitive gating network (SSGN) analyzes the potential representation and generates style-related gating masks. Each SSGN specializes in processing a basic style to achieve parallel processing of multiple style pathways.

[0052] In the embodiment of the present invention, analyzing the target style vector corresponding to each dynamic style type according to the latent vector and the preset temperature parameter includes:

[0053] Extract the weight matrix and bias parameters corresponding to each dynamic style type;

[0054] Performing a nonlinear transformation on the latent vector according to a first weight matrix in the weight matrix and a first bias parameter in the bias parameters to obtain a first transformed vector;

[0055] Performing a linear transformation on the first transformation vector according to a second weight matrix in the weight matrix and a second bias parameter in the bias parameter to obtain a second transformation vector;

[0056] The temperature parameter is used to control the sharpness of the preset activation function, and the activation function after the sharpness control is used to convert the second transformation vector into a target style vector corresponding to each dynamic style type.

[0057] In detail, the preset weight matrix and bias parameters are parameters learned after model training, which are used to map the latent vector to the style-related feature space. The weight matrix defines the correlation strength between each dimension of the latent vector and the style feature, and the bias parameter is used to adjust the reference position of the mapping. Given the latent representation output by the encoder, SSGN first calculates the style vectors under different styles. For K different styles, for each style: That is, after z passes through two layers of full connection, the vector z of each style is obtained (k) ,in, and is a learnable weight matrix, is the bias term, σ represents the sigmoid activation function, then the nonlinear transformation of the latent vector enables the model to learn the complex mapping relationship between the latent vector and the style features, breaking the expression limitations of the linear model; the linear transformation of the first transformation vector maps the features after nonlinear transformation to the dimensional space of the target style vector, adjusting the weight ratio of each style feature.

[0058] Specifically, to enhance the flexibility of gating, the system introduces a temperature parameter τ to control the sharpness of the gating mask. The temperature parameter τ is usually used to soften or sharpen the output distribution of the activation function, that is, The temperature parameter τ is initially set to 1 and is automatically adjusted during training as a learnable parameter. When τ approaches 0, the probability distribution of the style vector is more concentrated, the generated style is more certain, and the gated mask approaches the one-hot vector, achieving hard partitioning. When τ is larger, a smooth soft mask is formed, and the style vector incorporates more types of features, supporting progressive style adjustment.

[0059] Furthermore, in order to meet the user's demand for style text, it is necessary to adjust the style of the input text according to the target audience, enhance the dissemination and acceptance of the text, and determine the degree of fit between the text style generation results and user expectations.

[0060] S3. Obtain the text requirement style of the target user, convert the text requirement style into a style requirement vector, and calculate the style weight corresponding to the style requirement vector.

[0061] In the embodiment of the present invention, the text requirement style refers to the target user's requirements on the language expression form and emotional tendency of the output text and the style of the input text.

[0062] In detail, the text requirement style of the target user can be obtained from a pre-stored storage area through a computer statement with data crawling function (such as Java statement, Python statement, etc.), where the storage area includes but is not limited to a database and a blockchain.

[0063] Furthermore, the user inputs a specified style, such as “formal+humorous”, and encodes it into a specified style vector s user =MLP(concat(s 风格1 ,s 风格2)), the MLP (multi-layer perceptron) maps the input vector suser∈Rs to the style weight space through two layers of linear transformation and a nonlinear activation function, thereby obtaining the style requirement vector, where the style requirement vector converts the user's abstract text requirement style into a numerical vector representation that can be calculated by the machine learning model.

[0064] Furthermore, user demands usually contain multiple style dimensions. Directly using the original demand vector may lead to conflicts between dimensions. By calculating the weights, multidimensional demands can be converted into a set of coordinated control parameters.

[0065] For example, in the fintech scenario, the target user's text demand style is to transform from colloquial to rigorous and professional or from formal to humorous; in the medical scenario, the target user's text demand style is to transform from professional to caring or from popular to rigorous.

[0066] In the embodiment of the present invention, the style weight refers to converting the user's style requirement vector into a set of parameterized weight values ​​that can be used to control text generation, and adjusting the weights to achieve precise control of the style of the generated text.

[0067] In the embodiment of the present invention, referring to Figure 4 As shown, the calculating of the style weight corresponding to the style requirement vector includes:

[0068] S41, projecting the dimensional space corresponding to the style requirement vector to a preset hidden space;

[0069] S42, performing nonlinear activation on the style demand vector projected into the latent space;

[0070] S43, mapping the style requirement vector after nonlinear activation to a preset style weight space;

[0071] S44 . Perform a normalization operation on the style requirement vector in the style weight space to obtain a style weight corresponding to the style requirement vector.

[0072] In detail, the parallel gated path (PGP) is constructed by specifying the style vector s user , get the weight of each style, Then the user-specified style vector s user It is the semantic encoding of multiple style labels (such as the vector of "formal + humorous" after MLP mapping), which contains the user's mixed intentions for the target style. The PGP module transforms s user Converted into weights of each style, that is, extracted by MLP user The features related to each style in are normalized using Softmax to ensure that the sum of the weights is 1, forming a probability distribution. The MLP usually contains a linear layer + activation function, such as ReLU, using two layers of MLP to s user Perform nonlinear transformation and map the hidden layer output to K-style weight space, then use Softmax to perform normalization operation to obtain the style weight corresponding to each style.

[0073] Furthermore, the style weight reflects the user's specific preference for text style. The aggregation process adjusts the target style vector through the weight so that the final style vector is more in line with the user's actual needs.

[0074] S4. Aggregate the target style vector according to the style weight to obtain a style aggregation vector, and project the style aggregation vector into a preset style-aware decoder.

[0075] In an embodiment of the present invention, the style aggregation vector refers to a comprehensive vector obtained by weighted aggregation of the target style vector using style weights, which integrates the user's preference intensity for different style dimensions (determined by the style weights) and the numerical representation of specific style features (determined by the target style vector).

[0076] In the embodiment of the present invention, aggregating the target style vectors according to the style weights to obtain a style aggregation vector includes:

[0077] Extracting a target style weight corresponding to each vector dimension in the style requirement vector according to the style weight;

[0078] Extracting a vector value corresponding to each vector dimension in the target style vector;

[0079] Mapping the target style weight to the vector value to obtain a mapping relationship;

[0080] A weighted operation is performed on the target style weight and the vector value according to the mapping relationship to obtain a style aggregation vector.

[0081] Specifically, based on the calculated style weights, the weight values ​​corresponding to each dimension of the style demand vector are extracted to clarify the importance of each style dimension in user needs. The target style vector is a numerical vector converted from the dynamic style type. The specific values ​​of each dimension are extracted and corresponded one-to-one with the style weights. The style weights of the same dimension are associated with the vector values ​​to form weight-eigenvalue pairs. The proportion in which each eigenvalue needs to be weighted is determined to ensure that the aggregation results meet user preferences.

[0082] Specifically, different style vectors z (k) polymerization: For the specified style vector s userAfter full connection, softmax is performed to obtain a k-dimensional vector. Each value of the vector represents the weight of the corresponding style. The k style vectors z (k) Weighted to get the final style vector

[0083] Furthermore, the full-link transmission of style information from user requirements to decoder parameters can ensure that the style of the generated text is consistent with the user's intention. Figure 1 It retains the language generation ability of the original self-attention parameters and only superimposes style control through style projection, avoiding training the model from scratch, thereby generating text content with unique style.

[0084] In the embodiment of the present invention, the text x to be changed in style = {x1, x2, ..., x n}, and the final style vector Input Transformer model, based on Transformer decoder, introduce style injection mechanism before each layer of attention module.

[0085] In the embodiment of the present invention, projecting the style aggregation vector into a preset style-aware decoder includes:

[0086] extracting self-attention parameters of the attention layer of the style-aware decoder;

[0087] updating the self-attention parameters according to the style aggregation vector and a preset learning matrix to obtain style-aware projection parameters;

[0088] The style-aware projection parameters are projected into a self-attention layer of the style-aware decoder.

[0089] In detail, the Transformer self-attention layer contains the projection matrices of query (Q), key (K), value (V), and output projection matrix, which can be learned as matrix U Q ,U K ,U V ∈R d×d Used to establish the mapping relationship between the style aggregation vector and the self-attention parameter, the projected style information (style injection) is defined as follows: Among them U Q ,U K ,U V ∈R d×d is a learnable matrix, and the updated Q ′ , K ′ 、V ′ That is, the style-aware projection parameters, which include the original language generation capability and the target style information. In the style adapter layer, other style projection matrices are disabled and only the U and V matrices corresponding to the target style are activated.

[0090] Specifically, the updated parameters are directly replaced with the original parameters of the self-attention layer, or fused through residual connections. Ensure that each step in the decoding process is aware of the current style state.

[0091] Furthermore, the vector sequence (such as the latent vector) output by the encoder is an abstract representation of semantics, which needs to be converted into a text form understandable to humans through decoding.

[0092] S5. Decode the vector sequence using the projected style-aware decoder to obtain a target style text corresponding to the input text.

[0093] In the embodiment of the present invention, the target style text refers to the output text that retains the original semantic content but has the target style characteristics after the input text undergoes style transfer, which meets the style requirements specified by the user, such as formal, humorous, ancient style, etc.

[0094] In an embodiment of the present invention, decoding the vector sequence using the projected style-aware decoder to obtain a target style text corresponding to the input text includes:

[0095] Performing position encoding on the vector sequence to obtain a coding sequence, and extracting mask information corresponding to the vector sequence;

[0096] Calculating self-attention parameters of the vector sequence using the self-attention layer in the projected style-aware decoder and the mask information;

[0097] performing layer normalization processing on the self-attention parameters and the encoding sequence to obtain normalized data, and determining an output result of the style-aware decoder based on the normalized data;

[0098] Analyzing the vocabulary probability distribution of the output result according to a preset vocabulary table, and generating a mask sequence corresponding to the vector sequence according to the vocabulary probability distribution;

[0099] The mask sequence is converted into target style text corresponding to the input text.

[0100] Specifically, position information is added to the vector sequence x to solve the position independence problem of Transformer, and the effective length mask in the sequence is extracted, such as the padding position is 0 and the effective token is 1, to ensure that the self-attention calculation only focuses on the effective content, and then use the updated projection parameter Q ′ , K ′ 、V ′ Calculate the attention weight, where the mask information is used to mask the attention score of invalid positions. ′ , K′ 、V ′ By integrating the style aggregation vector, attention will be biased towards semantic information related to the target style. For example, in the humorous style, attention to metaphorical words is enhanced.

[0101] Specifically, the self-attention output and position encoding sequence are layer-normalized, and the normalized data is processed through a feedforward network and a multi-layer Transformer block to obtain the final decoder output. The vector representation of each position predicts the semantic information of the next token. The decoder output is projected into the vocabulary space, and the generation probability of each token is calculated through softmax. The token index is sampled or greedily selected according to the probability distribution to generate a mask sequence. Then, each index in the mask sequence is mapped to the corresponding token according to the vocabulary and spliced ​​into the target text.

[0102] For example, a neutral text is converted into two styles: ["informal", "humorous"] and ["formal", "persuasive"]. The input text is "This product works well. I recommend trying it." The target style input is: ["informal", "humorous"], and the generated output is: "This thing is ridiculously easy to use. I don't want to return it after using it. Go for it!" Target style input 2 is: ["formal", "persuasive"], and the generated output is: Given that this product performed well in multiple tests, we recommend that you try it as soon as possible.

[0103] Furthermore, the output combines informal, colloquial expressions (such as "this thing" and "Go for it") with humorous, exaggerated rhetoric (such as "I don't even want to give it back after using it"), conveying the recommendation intent in a relaxed tone. The system first obtains the latent representation z through an encoder, and then SSGN generates different gating masks based on the target style. For formal styles, the system strengthens latent dimensions related to rigorous wording and complex sentence structures; for informal styles, it activates dimensions related to colloquial expressions and sentiment words. The PGP module can handle both style requirements simultaneously, generating outputs with distinct styles but consistent content.

[0104] In addition, the system is trained end-to-end using multiple loss functions weighted in a weighted manner: Reconstruction loss Cross entropy loss is used to ensure the fluency of generated text; style loss is the cross entropy loss of the style classifier to ensure style accuracy; content loss Using contrast loss, we constrain the similarity between input and output in the content encoding space. The cross entropy loss of the token output by the Transformer decoder, in, is the true style one-hot vector, K is the number of categories, is the final style vector The k-dimensional vector obtained after a fully connected MLP; This loss is a contrast loss. In the same batch, the similarity between the output of the current text and the input is as large as possible compared to the similarity between the input of the current text and the output of other texts, where x is the current input text. is the corresponding output text with changed style, It is the output of other texts in the same batch after changing the style. f(x) is the output of the CLS token corresponding to the current text after pre-training BERT. Its dimension is 1*768. Similarly, They are the output corresponding to the current text, the output corresponding to other texts, the τ temperature variable, and the hyperparameters.

[0105] For example, in the fintech scenario, the user-input text "This investment product works well, I recommend you try it" is converted from colloquial to rigorous, and the output content is that the annualized return of this investment product in the past 12 months is 6.5% (the benchmark index in the same period is 4.2%), and the risk-adjusted return performance is better than the median of similar products. It is recommended to configure it based on personal risk tolerance; in the medical scenario, the user-input text "Instructions for use of the drug" is converted from professional to caring, and the output content is that the drug is usually used to relieve headaches and fevers, and it is recommended to take one tablet (100 mg) on ​​an empty stomach every morning. If stomach pain or bleeding gums occur, remember to stop taking the drug in time and tell your doctor.

[0106] It can be seen that in the above scheme, by simultaneously maintaining multiple style channels in the latent space, infinite style mixtures are supported through linear combination; soft partitioning is achieved through learnable temperature parameters to support continuous style adjustment; the gated latent representation is injected into the decoder's attention calculation to achieve fine-grained style control; the style, content and reconstruction goals are jointly optimized to balance the style and content, thereby solving the technical problem of low accuracy in generating style text based on user input content.

[0107] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0108] In one embodiment, a text generation device based on dynamic style is provided, which corresponds to the text generation method based on dynamic style in the above embodiment. Figure 5As shown, the text generation device based on dynamic style includes a latent vector analysis module 101, a target style vector analysis module 102, a style weight calculation module 103, a style aggregation vector projection module 104 and a target style text generation module 105. The functional modules are described in detail as follows:

[0109] The latent vector analysis module 101 is used to obtain input text from a target user, encode the input text into a vector sequence, and extract latent vectors from the vector sequence using a preset encoder;

[0110] a target style vector analysis module 102 for extracting the dynamic style type in the latent space corresponding to the latent vector, and analyzing the target style vector corresponding to each dynamic style type based on the latent vector and a preset temperature parameter;

[0111] The style weight calculation module 103 is used to obtain the text requirement style of the target user, convert the text requirement style into a style requirement vector, and calculate the style weight corresponding to the style requirement vector;

[0112] A style aggregation vector projection module 104 is configured to aggregate the target style vectors according to the style weights to obtain a style aggregation vector, and project the style aggregation vector into a preset style-aware decoder;

[0113] The target style text generation module 105 is configured to decode the vector sequence using the projected style-aware decoder to obtain a target style text corresponding to the input text.

[0114] In one embodiment, the latent vector analysis module 101, when extracting the latent vector of the vector sequence using a preset encoder, is configured to:

[0115] Inputting the vector sequence into the hidden layer of the encoder to obtain a hidden representation vector;

[0116] Extracting a target hidden vector corresponding to the last hidden layer in the encoder according to the hidden representation vector;

[0117] Average pooling is performed on the target hidden vector to obtain a latent vector of the vector sequence.

[0118] In one embodiment, the target style vector analysis module 102, when analyzing the target style vector corresponding to each dynamic style type based on the latent vector and the preset temperature parameter, is configured to:

[0119] Extract the weight matrix and bias parameters corresponding to each dynamic style type;

[0120] Performing a nonlinear transformation on the latent vector according to a first weight matrix in the weight matrix and a first bias parameter in the bias parameters to obtain a first transformed vector;

[0121] Performing a linear transformation on the first transformation vector according to a second weight matrix in the weight matrix and a second bias parameter in the bias parameter to obtain a second transformation vector;

[0122] The temperature parameter is used to control the sharpness of the preset activation function, and the activation function after the sharpness control is used to convert the second transformation vector into a target style vector corresponding to each dynamic style type.

[0123] In one embodiment, the style weight calculation module 103, when calculating the style weight corresponding to the style requirement vector, is configured to:

[0124] Projecting the dimensional space corresponding to the style requirement vector to a preset latent space;

[0125] Perform nonlinear activation on the style requirement vector projected into the latent space;

[0126] Map the style requirement vector after nonlinear activation to the preset style weight space;

[0127] A normalization operation is performed on the style requirement vector in the style weight space to obtain a style weight corresponding to the style requirement vector.

[0128] In one embodiment, the style aggregation vector projection module 104 , when aggregating the target style vectors according to the style weights to obtain the style aggregation vector, is configured to:

[0129] Extracting a target style weight corresponding to each vector dimension in the style requirement vector according to the style weight;

[0130] Extracting a vector value corresponding to each vector dimension in the target style vector;

[0131] Mapping the target style weight to the vector value to obtain a mapping relationship;

[0132] A weighted operation is performed on the target style weight and the vector value according to the mapping relationship to obtain a style aggregation vector.

[0133] In one embodiment, the style aggregation vector projection module 104, when performing the projection of the style aggregation vector to a preset style-aware decoder, is configured to:

[0134] extracting self-attention parameters of the attention layer of the style-aware decoder;

[0135] updating the self-attention parameters according to the style aggregation vector and a preset learning matrix to obtain style-aware projection parameters;

[0136] The style-aware projection parameters are projected into a self-attention layer of the style-aware decoder.

[0137] In one embodiment, the target style text generation module 105 , when decoding the vector sequence using the projected style-aware decoder to obtain the target style text corresponding to the input text, is configured to:

[0138] Performing position encoding on the vector sequence to obtain a coding sequence, and extracting mask information corresponding to the vector sequence;

[0139] Calculating self-attention parameters of the vector sequence using the self-attention layer in the projected style-aware decoder and the mask information;

[0140] performing layer normalization processing on the self-attention parameters and the encoding sequence to obtain normalized data, and determining an output result of the style-aware decoder based on the normalized data;

[0141] Analyzing the vocabulary probability distribution of the output result according to a preset vocabulary table, and generating a mask sequence corresponding to the vector sequence according to the vocabulary probability distribution;

[0142] The mask sequence is converted into target style text corresponding to the input text.

[0143] The present invention provides a text generation device based on dynamic style, which simultaneously maintains multiple style paths in the latent space and supports infinite style mixing through linear combination; realizes soft partitioning through learnable temperature parameters and supports continuous style adjustment; injects the gated latent representation into the attention calculation of the decoder to achieve fine-grained style control; jointly optimizes style, content and reconstruction goals, and balances style with content, thereby solving the technical problem of low accuracy in generating style text based on user input content.

[0144] For the specific definition of the text generation device based on dynamic style, please refer to the definition of the text generation method based on dynamic style above, which will not be repeated here. The various modules in the above-mentioned text generation device based on dynamic style can be implemented in whole or in part by software, hardware, or a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0145] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 6 As shown. The computer device includes a processor, a memory, a network interface and a database connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile and / or volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external client via a network connection. When the computer program is executed by the processor, it realizes the functions or steps on the server side of a text generation method based on a dynamic style.

[0146] In one embodiment, a computer device is provided. The computer device may be a client, and its internal structure diagram may be as follows: Figure 7 As shown. The computer device includes a processor, memory, network interface, display screen and input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external server via a network connection. When the computer program is executed by the processor, it implements the functions or steps on the client side of a text generation method based on dynamic style.

[0147] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the following steps are performed:

[0148] Obtaining input text from a target user, encoding the input text into a vector sequence, and extracting a latent vector from the vector sequence using a preset encoder;

[0149] Extracting a dynamic style type in a latent space corresponding to the latent vector, and analyzing a target style vector corresponding to each dynamic style type based on the latent vector and a preset temperature parameter;

[0150] Obtaining the text requirement style of the target user, converting the text requirement style into a style requirement vector, and calculating the style weight corresponding to the style requirement vector;

[0151] Aggregating the target style vector according to the style weight to obtain a style aggregation vector, and projecting the style aggregation vector into a preset style-aware decoder;

[0152] The projected style-aware decoder is used to decode the vector sequence to obtain a target style text corresponding to the input text.

[0153] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0154] Obtaining input text from a target user, encoding the input text into a vector sequence, and extracting a latent vector from the vector sequence using a preset encoder;

[0155] Extracting a dynamic style type in a latent space corresponding to the latent vector, and analyzing a target style vector corresponding to each dynamic style type based on the latent vector and a preset temperature parameter;

[0156] Obtaining the text requirement style of the target user, converting the text requirement style into a style requirement vector, and calculating the style weight corresponding to the style requirement vector;

[0157] Aggregating the target style vector according to the style weight to obtain a style aggregation vector, and projecting the style aggregation vector into a preset style-aware decoder;

[0158] The projected style-aware decoder is used to decode the vector sequence to obtain a target style text corresponding to the input text.

[0159] It should be noted that the above functions or steps that can be implemented by the computer-readable storage medium or computer device can be found in the relevant descriptions of the server side and the client side in the aforementioned method embodiment. To avoid repetition, they will not be described one by one here.

[0160] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0161] Those skilled in the art will clearly understand that for the sake of convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0162] It should be noted that if software tools or components other than those of our company appear in the embodiments of this application, they are only used for illustration and do not represent actual use.

[0163] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention, and should all be included in the scope of protection of the present invention.

Claims

1. A text generation method based on dynamic style, characterized in that: include: Obtaining input text from a target user, encoding the input text into a vector sequence, and extracting a latent vector from the vector sequence using a preset encoder; Extracting a dynamic style type in a latent space corresponding to the latent vector, and analyzing a target style vector corresponding to each dynamic style type based on the latent vector and a preset temperature parameter; Obtaining the text requirement style of the target user, converting the text requirement style into a style requirement vector, and calculating the style weight corresponding to the style requirement vector; Aggregating the target style vector according to the style weight to obtain a style aggregation vector, and projecting the style aggregation vector into a preset style-aware decoder; The projected style-aware decoder is used to decode the vector sequence to obtain a target style text corresponding to the input text.

2. The text generation method based on dynamic style according to claim 1, characterized in that: The extracting the latent vector of the vector sequence by using a preset encoder includes: Inputting the vector sequence into the hidden layer of the encoder to obtain a hidden representation vector; Extracting a target hidden vector corresponding to the last hidden layer in the encoder according to the hidden representation vector; Average pooling is performed on the target hidden vector to obtain a latent vector of the vector sequence.

3. The text generation method based on dynamic style according to claim 1, characterized in that: Analyzing the target style vector corresponding to each dynamic style type according to the latent vector and the preset temperature parameter includes: Extract the weight matrix and bias parameters corresponding to each dynamic style type; Performing a nonlinear transformation on the latent vector according to a first weight matrix in the weight matrix and a first bias parameter in the bias parameters to obtain a first transformed vector; Performing a linear transformation on the first transformation vector according to a second weight matrix in the weight matrix and a second bias parameter in the bias parameter to obtain a second transformation vector; The temperature parameter is used to control the sharpness of the preset activation function, and the activation function after the sharpness control is used to convert the second transformation vector into a target style vector corresponding to each dynamic style type.

4. The method for generating text based on dynamic style according to claim 1, wherein: The calculating the style weight corresponding to the style requirement vector includes: Projecting the dimensional space corresponding to the style requirement vector to a preset latent space; Perform nonlinear activation on the style requirement vector projected into the latent space; Map the style requirement vector after nonlinear activation to the preset style weight space; A normalization operation is performed on the style requirement vector in the style weight space to obtain a style weight corresponding to the style requirement vector.

5. The method for generating text based on dynamic style according to claim 1, wherein: The aggregating the target style vectors according to the style weights to obtain a style aggregation vector includes: Extracting a target style weight corresponding to each vector dimension in the style requirement vector according to the style weight; Extracting a vector value corresponding to each vector dimension in the target style vector; Mapping the target style weight to the vector value to obtain a mapping relationship; A weighted operation is performed on the target style weight and the vector value according to the mapping relationship to obtain a style aggregation vector.

6. The method for generating text based on dynamic style according to claim 1, wherein: The projecting the style aggregation vector into a preset style-aware decoder includes: extracting self-attention parameters of the attention layer of the style-aware decoder; updating the self-attention parameters according to the style aggregation vector and a preset learning matrix to obtain style-aware projection parameters; The style-aware projection parameters are projected into a self-attention layer of the style-aware decoder.

7. The method for generating text based on dynamic style according to claim 1, wherein: The method of decoding the vector sequence using the projected style-aware decoder to obtain a target style text corresponding to the input text includes: Performing position encoding on the vector sequence to obtain a coding sequence, and extracting mask information corresponding to the vector sequence; Calculating self-attention parameters of the vector sequence using the self-attention layer in the projected style-aware decoder and the mask information; performing layer normalization processing on the self-attention parameters and the encoding sequence to obtain normalized data, and determining an output result of the style-aware decoder based on the normalized data; Analyzing the vocabulary probability distribution of the output result according to a preset vocabulary table, and generating a mask sequence corresponding to the vector sequence according to the vocabulary probability distribution; The mask sequence is converted into target style text corresponding to the input text.

8. A text generation device based on dynamic style, characterized in that: include: A latent vector analysis module is used to obtain input text from a target user, encode the input text into a vector sequence, and extract a latent vector from the vector sequence using a preset encoder; a target style vector analysis module, configured to extract the dynamic style type in the latent space corresponding to the latent vector, and analyze the target style vector corresponding to each dynamic style type based on the latent vector and a preset temperature parameter; A style weight calculation module is used to obtain the text requirement style of the target user, convert the text requirement style into a style requirement vector, and calculate the style weight corresponding to the style requirement vector; a style aggregation vector projection module, configured to aggregate the target style vector according to the style weight to obtain a style aggregation vector, and project the style aggregation vector into a preset style-aware decoder; The target style text generation module is used to decode the vector sequence using the projected style-aware decoder to obtain the target style text corresponding to the input text.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method for generating a text based on a dynamic style of a quotation according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for generating text based on dynamic style according to any one of claims 1 to 7 is implemented.

Images