Text style conversion method and device based on semantic and style double condition injection

Abstract

This invention provides a text style transfer method and apparatus based on dual-conditional injection of semantics and style. A pre-trained semantic extraction model extracts semantic vectors from user-source text, while simultaneously retrieving standard style vectors for the target type from a built-in standard style library. Based on task instructions, the semantic vectors and standard style vectors are fed into a ChatGLM-based text style transfer model, which has undergone low-rank adaptive fine-tuning on a single-style sample set. This generates target text that is semantically consistent and style-matched to the source text through autoregression. During the fine-tuning phase, the text style transfer model uses only samples of the same style, applying random masks to the samples to obtain incomplete semantic vectors. These incomplete semantic vectors are then combined with the task instructions and style vectors to reconstruct the complete text. The reconstructed result is categorized by a style recognizer, and multiple losses are constructed based on semantic deviation, style deviation, and reconstruction error for backpropagation and updating. This invention effectively guides the model to generate high-quality text that conforms to the target style, achieving a balance between semantic preservation and style transfer.

Description

Technical Field

[0001] This invention relates to the fields of natural language processing and deep learning, and in particular to a text style transfer method and apparatus based on semantic and style dual-condition injection. Background Technology

[0002] Language style transfer is an important task in the field of natural language processing, aiming to transform a piece of text from one style to another while preserving the core semantics of the original text. For example, transforming a plain narrative text into a poetic style, an ornate style, or a news style. With the development of deep learning technology, especially the emergence of pre-trained language models, new technological possibilities have been provided for language style transfer. Existing language style transfer methods mainly include rule-based methods, statistical methods, and neural network-based methods. Rule-based methods achieve style transfer through manually defined transfer rules, but the rule design is complex and has limited coverage; statistical methods learn style transfer patterns through statistical models, but struggle to capture deep semantics; neural network-based methods directly learn style transfer mappings using deep learning models, which, while achieving good results, often suffer from the following problems: a balance between semantic preservation and style transfer is difficult to achieve, and the converted text may deviate from the original semantics; style features are difficult to quantify, leading to inaccurate style transfer; and the quality of the generated text is unstable, potentially resulting in disjointed or incoherent text. Therefore, a new text style transfer scheme is urgently needed. Summary of the Invention

[0003] In view of this, embodiments of the present invention provide a text style transfer method and apparatus based on semantic and style dual-condition injection, so as to eliminate or improve one or more defects existing in the prior art, and solve the problems of poor text style transfer effect, missing semantic preservation and poor quality of generated text in the prior art.

[0004] One aspect of the present invention provides a text style transfer method based on semantic and style dual-condition injection, the method comprising the following steps:

[0005] The system obtains the source text and target text style types provided by the user, extracts the source text semantic feature vector of the first set dimension using a preset semantic extraction model, and queries a predefined style database based on the target text style type to obtain the standard style vector of the second set dimension; the semantic extraction model includes a frozen BERT model and a CLS pooling layer.

[0006] The task instructions based on natural language, the semantic feature vector of the source text, and the normalized style vector are input into a preset text style transfer model through a two-layer injection mechanism to output target text that is semantically consistent with the source text and conforms to the target text style type. The task instructions are used to instruct the execution of the text style transfer task. The two-layer injection mechanism includes injecting the semantic feature vector into one of the first 1 / 3 to 1 / 2 layers of the text style transfer model and injecting the normalized style vector into one of the last 1 / 3 layers of the text style transfer model.

[0007] The text style transfer model is obtained by low-rank adaptive fine-tuning of the ChatGLM large language model using a single-style text sample set. Each sample in the single-style text sample set contains a task instruction based on natural language, a sample text, and its corresponding sample text style label. The semantic extraction model is used to extract incomplete semantic feature vectors from the randomly corrupted sample text. The random corruption includes masking, lexical deletion, and similar character replacement implemented according to a preset ratio. The style vector is obtained by querying the style database based on the sample text style label. The task instruction, the incomplete semantic feature vector, and the sample style vector are input into the text style transfer model through the two-layer injection mechanism to perform an autoregressive task and output reconstructed text. The reconstructed text style is identified based on a preset text style recognition model. The text reconstruction bias loss, semantic bias loss, and style type bias loss are calculated based on the reconstructed text and the sample text, and a joint loss is constructed. The joint loss function is used to perform low-rank adaptive parameter updates on the text style transfer model.

[0008] In some embodiments, the semantic extraction model adopts the BERT model based on the Transformer architecture and extracts 768-dimensional semantic feature vectors based on CLS pooling; the text style transfer model is the ChatGLM model; during the low-rank adaptive parameter update process, the rank parameter is set to 8, the scaling parameter is set to 32, the dropout rate is set to 0.1, the main parameters of the ChatGLM model are frozen, and the parameters are updated for the query layer and the key-value layer.

[0009] In some embodiments, the two-layer injection mechanism includes injecting the semantic feature vector into one of the first 1 / 3 to 1 / 2 layers of the text style transfer model, and injecting the canonical style vector into one of the last 1 / 3 layers of the text style transfer model, including:

[0010] The source text semantic feature vector is mapped through a projection layer and then injected into the 6th hidden state of the ChatGLM model; the canonical style vector is mapped through a projection layer and then injected into the 18th hidden state of the ChatGLM model; the fusion ratio between the injected vector and the original hidden state is controlled by a gating mechanism.

[0011] In some embodiments, the steps for constructing the predefined style database include:

[0012] Representative samples of various text styles are input into a pre-trained style encoder, which outputs a representative style vector of the representative samples. The style encoder includes a BERT model and a projection mapping layer, which outputs a 256-dimensional representative style vector.

[0013] The average value of multiple representative style vectors corresponding to each text style is calculated to obtain the standard style vector of the corresponding text style.

[0014] In some embodiments, the text style recognition model includes a text style encoder and a text style classifier. The text style encoder consists of a BERT model and a projection mapping layer. The projection mapping layer includes a linear layer, a ReLU activation function, a Dropout layer, and a LayerNorm layer. The projection mapping layer maps the 768-dimensional BERT output to a 256-dimensional style vector. The text style classifier consists of multiple single-linearly connected layers, mapping the 256-dimensional style vector to classification logits representing the number of style categories. The pre-training steps of the text style recognition model include:

[0015] Obtain a text style classification training dataset containing multiple style categories and multiple samples, where each sample contains sample text and its corresponding style classification label;

[0016] The text style recognition model, which is randomly initialized, is trained using the text style classification training dataset. The text style encoder extracts sample style vectors from the sample texts and outputs style predictions for the sample texts via the text style classifier. A style classification loss is constructed based on the deviation between the style predictions and the style classification labels. An intra-style contrast loss is constructed using the sample style vectors corresponding to each sample text to reduce the differences between styles of the same type and increase the differences between styles of different types. The text style recognition model is updated by combining the style classification loss and the intra-style contrast loss.

[0017] In some embodiments, the style classification loss is calculated as follows:

[0018] ;

[0019] in, This represents the style classification loss. For cross-entropy loss, This indicates the style prediction. This indicates the style category label.

[0020] The formula for calculating the intra-style contrast loss is:

[0021]

[0022] in, This represents the intra-style contrast loss. This represents the anchor style vector of the anchor samples selected within a batch. This represents the style vector of the j-th positive sample within the batch that has the same style as the anchor sample. Represents the set of all positive samples; This represents the style vector of the k-th negative sample within the batch that has a style different from the anchor sample. Let represent the set of all negative samples; sim represents the cosine similarity calculation; exp represents the exponential function. Temperature is a hyperparameter; InfoNCE contrastive learning is used to enhance the aggregation of similar style vectors and the discriminative power of dissimilar style vectors.

[0023] In updating the parameters of the text style recognition model by combining the style classification loss and the intra-style contrast loss, the first joint loss calculation formula is as follows:

[0024] ;

[0025] in, This represents the first joint loss. and These are the weighting coefficients.

[0026] In some embodiments, the text style transfer model is updated with parameters based on a loss constructed from the reconstructed text style and the semantic deviation, style type deviation, and text reconstruction deviation of the sample text, including:

[0027] The semantic extraction model is used to extract semantic vectors from the reconstructed text style and the sample text respectively, and the semantic bias loss is calculated based on cosine similarity.

[0028] Calculate the cross-entropy between the reconstructed text style and the sample text style label as the style type bias loss;

[0029] The cross-entropy between the predicted distribution of each word in the reconstructed text by the text style transfer model and the true distribution of each word in the sample text is calculated as the text reconstruction bias loss.

[0030] The semantic bias loss, the style type bias loss, and the text reconstruction bias loss are weighted and summed to construct a second joint loss. The parameters of the text style transfer model are updated by minimizing the second joint loss.

[0031] On the other hand, the present invention also provides a text style transfer apparatus that integrates semantics and style vectors, including a processor, a memory, and a computer program / instructions stored in the memory, wherein the processor is used to execute the computer program / instructions, and when the computer program / instructions are executed, the apparatus implements the steps of the above method.

[0032] On the other hand, the present invention also provides a computer-readable storage medium having a computer program / instructions stored thereon, which, when executed by a processor, implement the steps of the above-described method.

[0033] On the other hand, the present invention also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described method.

[0034] The text style transfer method and apparatus based on dual-conditional injection of semantics and style, as described in this invention, extracts semantic vectors from user source text using a pre-trained semantic extraction model, while simultaneously retrieving the standard style vectors of the target type from a built-in standard style library. Based on task instructions, the semantic vectors and standard style vectors are fed into a ChatGLM-based text style transfer model, which has undergone low-rank adaptive fine-tuning on a single style sample set, via a two-layer injection mechanism. This autoregressive process generates target text that is semantically consistent and style-matched to the source text. During the fine-tuning phase, the text style transfer model uses only samples of the same style, applying random masks to the samples to obtain incomplete semantic vectors. These incomplete semantic vectors are then combined with the task instructions and style vectors to allow the model to reconstruct the complete text. The reconstructed result is categorized by a style recognizer, and multiple losses are constructed based on semantic deviation, style deviation, and reconstruction error for backpropagation and updating. This invention effectively guides the model to generate high-quality text that conforms to the target style, achieving a balance between semantic preservation and style transfer.

[0035] Additional advantages, objects, and features of the invention will be set forth in part in the description which follows, and will also become apparent in part to those skilled in the art upon studying the description, or may be learned by practice of the invention. The objects and other advantages of the invention can be realized and obtained by means of the structures specifically pointed out in the description and drawings.

[0036] Those skilled in the art will understand that the objectives and advantages achievable with the present invention are not limited to those specifically described above, and that the above and other objectives achievable with the present invention will become clearer from the following detailed description. Attached Figure Description

[0037] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, are not intended to limit the scope of the invention. In the drawings:

[0038] Figure 1 This is a flowchart illustrating a text style transfer method based on semantic and style dual-condition injection, as described in an embodiment of the present invention.

[0039] Figure 2 This is a logical diagram illustrating the fine-tuning of the text style transfer model in a text style transfer method based on semantic and style dual-condition injection, as described in an embodiment of the present invention.

[0040] Figure 3 This is a schematic diagram of the pre-training logic of the text style recognition model in a text style transfer method based on semantic and style dual-condition injection according to an embodiment of the present invention.

[0041] Figure 4 This is a schematic diagram of the style database construction process in a text style transfer method based on semantic and style dual-condition injection according to an embodiment of the present invention.

[0042] Figure 5 This is a schematic diagram of the preprocessing flow of a single-style text sample set in a text style transfer method based on semantic and style dual-condition injection according to an embodiment of the present invention.

[0043] Figure 6 This is a semantic similarity comparison diagram between the single-style fine-tuning method used in this invention and the style transfer model obtained by the traditional pairwise sample method.

[0044] Figure 7 This is a comparison chart of the style transfer accuracy between the single-style fine-tuning method used in this invention and the style transfer model obtained by the traditional paired-sample method. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the embodiments and accompanying drawings. Here, the illustrative embodiments and descriptions of this invention are used to explain the invention, but are not intended to limit the invention.

[0046] It should also be noted that, in order to avoid obscuring the invention with unnecessary details, only the structures and / or processing steps closely related to the solution according to the invention are shown in the accompanying drawings, while other details that are not closely related to the invention are omitted.

[0047] It should be emphasized that the term "including / comprises" as used herein refers to the presence of a feature, element, step, or component, but does not exclude the presence or addition of one or more other features, elements, steps, or components.

[0048] Existing text style transfer technologies have three main drawbacks: First, it is difficult to balance semantic preservation and style transfer, and the converted text often deviates from the original meaning; second, style features lack quantifiable representation, resulting in low conversion accuracy; and third, the generation quality is unstable, easily leading to unfluent sentences and incoherent context.

[0049] To address the problems existing in the prior art, this application provides a text style transfer method based on semantic and stylistic dual-condition injection, such as... Figure 1 As shown, the method includes the following steps S101 and S102:

[0050] Step S101: Obtain the source text and target text style types provided by the user, use a preset semantic extraction model to extract the source text semantic feature vector of the first set dimension, and query the predefined style database based on the target text style type to obtain the standard style vector of the second set dimension; the semantic extraction model includes a frozen BERT model and a CLS pooling layer.

[0051] Step S102: Input the natural language-based task instructions, source text semantic feature vectors, and normative style vectors into the preset text style transfer model through a two-layer injection mechanism to output target text that is semantically consistent with the source text and conforms to the target text style type; the task instructions are used to instruct the execution of the text style transfer task; the two-layer injection mechanism includes injecting the semantic feature vectors into one of the layers from the first 1 / 3 to the first 1 / 2 of the text style transfer model, and injecting the normative style vectors into one of the layers from the last 1 / 3 of the text style transfer model.

[0052] In step S101, the source text provided by the user is the text to be converted, which has an original text style, while the target text style is the style type that the user wants to generate. For example, the user inputs the source text "The weather is so nice today, the sun is shining brightly, I've decided to go for a walk in the park", and specifies the target style as "poetry".

[0053] To ensure that semantics are not lost during text style transfer, a semantic extraction model can be invoked first to compress the sentence into a source text semantic feature vector of the first predetermined dimension. Simultaneously, the "poetry" entry is retrieved from a predefined style database to extract the corresponding canonical style vector of the second predetermined dimension. The canonical style vector is obtained by extracting features from representative samples of various text styles based on a pre-defined style encoder.

[0054] like Figure 4As shown, the steps for constructing the predefined style database include: inputting representative samples of various text styles into a pre-trained style encoder, and outputting representative style vectors of the representative samples; the style encoder includes a BERT model and a projection mapping layer, used to output the 256-dimensional representative style vectors; calculating the average value of multiple representative style vectors corresponding to each text style to obtain the canonical style vector of the corresponding text style.

[0055] In step S102, a natural language task instruction—"Please rewrite the following text as a modern poem"—is concatenated. Then, based on this task instruction, the source text semantic feature vector (first defined dimension) and the canonical style vector (second defined dimension) are fed into the finely tuned ChatGLM style transfer model via a two-layer injection mechanism. The two-layer injection mechanism involves injecting the semantic feature vector into one of the first 1 / 3 to 1 / 2 layers of the style transfer model, and injecting the canonical style vector into one of the last 1 / 3 layers. For example, the source text semantic feature vector is injected into the 6th hidden state of the ChatGLM model after being mapped through a projection layer; the canonical style vector is injected into the 18th hidden state of the ChatGLM model after being mapped through a projection layer; and a gating mechanism controls the fusion ratio between the injected vector and the original hidden state.

[0056] The style transfer model uses semantic vectors as the content skeleton and normalized style vectors as style cues during autoregressive generation, ultimately outputting the target text. For example, the sentence "The sky is clear and bright, golden light weaves a tapestry; I tread on the green shade, my heart dancing lightly with the fragrance of flowers" retains the core semantics of the original sentence, "Good weather → going for a walk," while also presenting the poem's rhythmic flow and lyrical vocabulary, thus completing the style transfer.

[0057] In some embodiments, the semantic extraction model adopts the BERT model based on the Transformer architecture and extracts 768-dimensional semantic feature vectors based on CLS pooling; the text style transfer model is the ChatGLM model; during the low-rank adaptive parameter update process, the rank parameter is set to 8, the scaling parameter is set to 32, the dropout rate is set to 0.1, the main parameters of the ChatGLM model are frozen, and the parameters are updated for the query layer and the key-value layer.

[0058] Furthermore, such as Figure 2As shown, the text style transfer model is obtained by low-rank adaptive fine-tuning of the ChatGLM large language model using a single-style text sample set. Each sample in the single-style text sample set contains a task instruction based on natural language, the sample text, and its corresponding sample text style label. A semantic extraction model is used to extract incomplete semantic feature vectors from the randomly corrupted sample text. Random corruption includes masking, word deletion, and similar character replacement implemented according to a preset ratio. The style vector of the sample text is obtained by querying the style database based on the sample text style label. The task instruction, incomplete semantic feature vector, and sample style vector are input into the text style transfer model through a two-layer injection mechanism to perform an autoregressive task and output the reconstructed text. The reconstructed text style is identified based on a preset text style recognition model. The text reconstruction bias loss, semantic bias loss, and style type bias loss are calculated based on the reconstructed text and the sample text, and a joint loss is constructed. The joint loss function is used to perform low-rank adaptive parameter updates on the text style transfer model.

[0059] Existing text style transfer models struggle to balance semantic preservation and text transfer effectiveness. Typically, training or fine-tuning a text style transfer model requires paired texts before and after style transfer. This necessitates labeling a large number of samples, consuming significant storage space and computational resources, resulting in extremely high costs. This application employs a more refined text style transfer model fine-tuning scheme. The core idea is to enable the text style transfer model, through enhanced training with single-style, masked, and multi-task fine-tuning, to learn to complete sentences into the target style even with only a fragmented semantic skeleton, thereby significantly improving semantic preservation and style accuracy in real-world scenarios.

[0060] The single-style text sample described in this invention refers to each sample data containing only one style of text and its tag, rather than two pairs of texts with the same semantics but different styles. Each sample contains three parts: the first is the task instruction, which indicates the text style conversion task; the second is the sample text, which records the original text to be converted; and the third is the sample text style tag, which is also the text type of the original text. For example... Figure 5As shown, after loading and parsing single-style text samples, a style-to-ID mapping is created for each sample text. Based on text destruction processing, the sample text is randomly hollowed out, replaced, or deleted according to a set ratio to generate destroyed text, similar to BERT's Mask. Then, the BERT semantic extractor mentioned earlier is used to extract vectors to obtain incomplete semantic feature vectors, simulating the situation where real user input may have missing information or incomplete expression. This forces the model to rely more on style vectors during reconstruction rather than memorizing the original text, which is actually a style transfer process that fits the actual use scenario. The sample labels are used to look up the pre-built style database to retrieve the corresponding canonical style vectors, which serve as prompts to instruct the model to generate text of the corresponding style. The sample fields are reassembled to obtain the training set and validation set, and the style mapping file is saved. In fact, the fine-tuning stage is an autoregressive task, which can be understood as the reconstruction of the masked sample text.

[0061] For reconstructed text, style recognition is performed using a pre-defined text style recognition model, and the text style transfer model is fine-tuned by combining the semantic bias of the reconstructed text style with that of the sample text and the text reconstruction bias.

[0062] In some embodiments, the text style transfer model is updated with parameters based on a loss constructed from the semantic bias, style type bias, and text reconstruction bias of the reconstructed text style and the sample text, including steps S201-S204:

[0063] Step S201: Use a preset semantic extraction model to extract semantic vectors from the reconstructed text style and sample text respectively, and calculate semantic bias loss based on cosine similarity.

[0064] Step S202: Calculate the cross-entropy between the reconstructed text style and the sample text style label as the style type bias loss.

[0065] Step S203: Calculate the cross-entropy between the predicted distribution of each word in the reconstructed text by the text style transfer model and the true distribution of each word in the sample text as the text reconstruction bias loss.

[0066] Step S204: The semantic bias loss, style type bias loss, and text reconstruction bias loss are weighted and summed to construct a second joint loss. The parameters of the text style transfer model are updated by minimizing the second joint loss.

[0067] In steps S201-204, the same BERT model can be used to extract a 768-dimensional semantic vector from both the reconstructed text and the original sample text. Then, the similarity between the two is calculated to construct the loss, ensuring that the rewritten sentence stays on topic and its core meaning remains consistent with the original. The reconstructed sentence is then fed into a text style recognition model to obtain the logit distribution for K style classes. This logit distribution is then cross-entropyed with the true labels (one-hot). If the recognizer misclassifies the reconstructed sentence as a different style, the penalty is amplified. For each training sample, the model performs autoregression, outputting a probability distribution across all words at each step. This distribution can be used to calculate the token-level cross-entropy with the one-hot distribution of the next true word, and then averaged over sentence length to penalize incorrect guesses, ensuring the sentence is fluent and grammatically correct.

[0068] This three-loss joint training improves the BLEU (Bilingual Evaluation Understudy) metric by 4.1 points compared to using only text reconstruction bias loss, improves style recognition accuracy by 11%, and improves human fluency score by 0.8 points. Figure 6 and Figure 7 As shown, compared to the traditional pairwise sample method for fine-tuning text transfer models, the semantic similarity and style accuracy of the text transfer process in this invention are significantly improved.

[0069] In this fine-tuning process, the text style recognition model used includes a text style encoder and a text style classifier. The text style encoder consists of a BERT model and a projection mapping layer. The projection mapping layer includes a linear layer, a ReLU activation function, a Dropout layer, and a LayerNorm layer; the projection mapping layer maps the 768-dimensional BERT output to a 256-dimensional style vector; the text style classifier consists of multiple single-linearly connected layers, mapping the 256-dimensional style vector to the number of style category logits; as shown... Figure 3 As shown, the pre-training steps of the text style recognition model include steps S301 and S302:

[0070] Step S301: Obtain a text style classification training dataset containing multiple style categories and multiple samples. Each sample contains sample text and corresponding style classification label.

[0071] Step S302: The text style recognition model is trained using a text style classification training dataset. The text style encoder extracts the sample style vector from the sample text and outputs the style prediction of the sample text by the text style classifier. The style classification loss is constructed based on the deviation between the style prediction and the style classification label. The intra-style contrast loss is constructed with the sample style vector corresponding to each sample text to reduce the difference between the same style and increase the difference between different styles. The text style recognition model is updated by combining the style classification loss and the intra-style contrast loss.

[0072] In steps S301 and S302, the text style encoder includes a continuous BERT layer and a projection mapping layer. For example, the continuous BERT layer compresses text of arbitrary length into a 768-dimensional context vector; the projection mapping layer then reduces it to 256 dimensions, obtaining a compact sample style vector specifically designed to capture style information such as sentence length, punctuation, part-of-speech, and sentiment. The text style classifier includes multiple single-linear connection layers that perform local perception on the 256-dimensional style vector, extracting high-order style patterns; finally, a fully connected layer outputs K-dimensional logits, corresponding to K text style classes. Step S301 first prepares a text style classification training dataset, where each sentence has manually labeled style tags, covering all target categories and ensuring a balanced sample size for each class.

[0073] In step S302, the sample text is fed into the text style encoder each time to obtain a 256-dimensional style vector. The text style classifier outputs predicted logits and calculates two losses at the same time: style classification loss and intra-style contrast loss.

[0074] Style classification loss uses cross-entropy to measure the difference between the predicted distribution and the true labels, ensuring accurate classification. Intra-style contrastive loss, for all style vectors in the same batch, moves them closer together if the labels are the same, and further apart if they are different.

[0075] In some embodiments, the style classification loss is calculated as follows:

[0076] ;

[0077] in, Indicates style classification loss, For cross-entropy loss, Indicates style prediction, Indicates style category tags.

[0078] The formula for calculating intrastyle contrast loss is:

[0079]

[0080] in, This represents the intra-style contrast loss. This represents the anchor style vector of the anchor samples selected within a batch. This represents the style vector of the j-th positive sample within the batch that has the same style as the anchor sample. Represents the set of all positive samples; This represents the style vector of the k-th negative sample within the batch that has a style different from the anchor sample. represents the set of all negative samples; sim represents the cosine similarity calculation, and exp represents the exponential function. The temperature hyperparameter is set to 0.07. InfoNCE contrastive learning is used to enhance the aggregation of similar style vectors and the discriminative power of dissimilar style vectors.

[0081] In updating the parameters of the text style recognition model using the joint style classification loss and intra-style contrast loss, the first joint loss calculation formula is as follows:

[0082] ;

[0083] in, Indicates the first joint loss, and These are the weighting coefficients.

[0084] Driven by joint loss, the encoder gradually maps texts of the same style to a compact cluster in the vector space, and texts of different styles to a more distant cluster. The classifier only needs to draw boundaries on the clusters that have been separated, resulting in more robust discrimination. Experiments show that after adding contrastive loss, the cosine similarity of vectors of the same style increases by 18%, the distance between dissimilar classes increases by 25%, and the overall style recognition accuracy increases from 89.3% to 94.7%, with a significant enhancement in generalization ability for short texts and cross-domain texts.

[0085] On the other hand, the present invention also provides a text style transfer apparatus that integrates semantics and style vectors, including a processor, a memory, and a computer program / instructions stored in the memory, wherein the processor is used to execute the computer program / instructions, and when the computer program / instructions are executed, the apparatus implements the steps of the above method.

[0086] On the other hand, the present invention also provides a computer-readable storage medium having a computer program / instructions stored thereon, which, when executed by a processor, implement the steps of the above-described method.

[0087] On the other hand, the present invention also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described method.

[0088] The present invention will now be described with reference to a specific embodiment:

[0089] The purpose of this embodiment is to provide a language style conversion method and system based on deep learning. Through an innovative dual-vector separation and dual-layer injection mechanism, it achieves effective separation and extraction of semantic and style information, preserving the original semantics while realizing high-quality text style conversion. The language style conversion method provided in this embodiment includes the following steps: receiving input text and a target style; extracting semantic vectors from the input text; obtaining style vectors corresponding to the target style; and, based on task instructions, fusing the semantic and style vectors and injecting them into different layers of a pre-trained language model to generate target style text, guiding the generation process of the pre-trained language model. This invention also provides a language style conversion system based on deep learning, including: a semantic vector extraction module for extracting semantic vectors from the input text; a style vector acquisition module for obtaining a canonical style vector corresponding to the target style; and a conditional generation module for using a pre-trained language model to generate target style text based on task instructions, semantic vectors, and canonical style vectors. This embodiment solves the problem of balancing semantic preservation and style conversion by extracting the semantic and style information of the text into independent vectors and injecting them layer by layer into the semantic and style vectors. Simultaneously, it provides canonical style vectors for different text styles, improving the accuracy of style conversion.

[0090] The language style transfer system provided in this embodiment adopts a modular design and consists of five main components: a semantic vector extractor, a style vector acquirer, a conditional generator, a style transfer trainer, and an evaluator. These components work together to form a complete text style transfer process.

[0091] The system's workflow is as follows: First, it receives the source text and target style input from the user; then, it uses a semantic vector extractor to extract semantic vectors from the source text, and simultaneously obtains the canonical style vectors of the target style from the style vector library; the semantic vectors and canonical style vectors are then injected hierarchically into the style transfer model of the conditional generator to generate text in the target style. The system also includes an evaluation module to calculate semantic similarity and style similarity, and to evaluate the conversion effect.

[0092] The semantic vector extractor uses a pre-trained BERT model to extract semantic vectors from the input text. The specific implementation is as follows:

[0093] First, the input text is segmented and encoded using BERT's tokenizer to generate an input ID sequence. Then, this input ID sequence is fed into the BERT model to obtain its output representation. Finally, a pooling strategy is used to extract semantic vectors from the BERT output. This embodiment supports two pooling strategies: CLS pooling and average pooling. CLS pooling uses the hidden state corresponding to the [CLS] marker in the BERT output as the semantic vector, while average pooling calculates the average of the hidden states of all tokens as the semantic vector.

[0094] The style vector generator extracts canonical style vectors from representative samples of various text styles, forming a style database. In subsequent applications, these canonical style vectors are queried to guide text migration.

[0095] The conditional generator uses a pre-trained ChatGLM model. Based on the task instructions, semantic vectors and canonical style vectors are input into the ChatGLM model in layers. The semantic vectors are injected into the 6th hidden state of the ChatGLM model after being mapped through the projection layer, and the canonical style vectors are injected into the 18th hidden state of the ChatGLM model after being mapped through the projection layer to generate text in the target style.

[0096] To improve the style transfer performance, this embodiment provides a function for fine-tuning the ChatGLM model using a single-style text sample set. Each sample in the single-style text sample set contains a natural language-based task instruction, sample text, and its corresponding sample text style label. A pre-trained BERT model is used to extract incomplete semantic feature vectors from the sample text after 15% random disruption. This 15% random disruption includes a 60% probability of replacing with a mask marker, a 20% probability of deleting a word, and a 20% probability of replacing with a similar character. The style vector is obtained by querying a style database based on the sample text style label. The task instruction, incomplete semantic feature vector, and sample style vector are input into the ChatGLM model through a two-layer injection mechanism to perform an autoregressive task and output reconstructed text. A pre-defined text style recognition model identifies the reconstructed text style. Based on the reconstructed text and the sample text, text reconstruction bias loss, semantic bias loss, and style type bias loss are calculated and a joint loss is constructed. The joint loss function is used to update the LoRA parameters of the text style transfer model. This embodiment provides a comprehensive evaluation system for evaluating the style transfer performance. Evaluation metrics include: semantic similarity, used to assess the degree of semantic preservation of the text before and after conversion, calculated using the BERT model to obtain cosine similarity; style similarity, used to assess the similarity between the converted text and the target style, calculated by comparing style vectors; fluency assessment, used to assess the language fluency of the converted text, using the perplexity metric of a language model; and a comprehensive score, which comprehensively considers semantic preservation, style transfer, and fluency, calculating a weighted average score.

[0097] The following specific example illustrates the application effect of this invention. Assume the input text is: "The weather is so nice today, the sun is shining brightly, I've decided to go for a walk in the park." The target style is "poetry."

[0098] First, the semantic vector of the input text is extracted using the BERT model. Then, the canonical style vector of the "poetry" style is obtained. The semantic vector is mapped and injected into layer 6 of the ChatGLM model, and the canonical style vector is mapped and injected into layer 18 of the ChatGLM model to generate text in the target style. The generated poetry-style text might be: "The sky is clear and bright, the golden light is dazzling, strolling in the green shade, feeling refreshed and relaxed."

[0099] The evaluation showed that the semantic similarity of the conversion result was 0.85, the style similarity was 0.92, the fluency was 0.88, and the overall score was 0.88, indicating that the system successfully achieved high-quality style conversion while maintaining the semantics of the original text.

[0100] The language style conversion method and system provided by this invention have the following beneficial effects:

[0101] (1) Through the two-layer vector injection mechanism, the effective separation and feature injection of semantic information and style information are realized. The gating mechanism is used to inject semantic vector and target style vector into ChatGLM, which solves the problem of balancing semantic preservation and style transfer.

[0102] (2) A standardized style vector is constructed by deep learning method. A pre-trained style encoder (BERT + projection layer) is used to extract style features from representative samples and average them, providing a quantifiable representation method for different text styles. The InfoNCE contrast loss is combined to enhance the discriminativeness of style vectors and improve the accuracy of style transfer.

[0103] (3) Through the innovative two-layer vector injection mechanism, semantic vectors and style vectors are injected into different levels of ChatGLM through the projection layer, and combined with LoRA efficient fine-tuning technology, the model is effectively guided to generate high-quality text that conforms to the target style.

[0104] (4) By controlling the vector injection intensity through the gating mechanism and adjusting the weights of the triple loss function, the balance between semantic preservation and style transfer can be precisely adjusted according to different needs.

[0105] (5) A comprehensive evaluation system, including using frozen BERT to calculate semantic consistency (cosine similarity) and using pre-trained style encoders to calculate style accuracy, can objectively measure the effect of style transfer and provide a basis for system optimization.

[0106] (6) The system adopts a modular design and includes five core components: semantic vector extractor, style vector acquirer, condition generator, style transfer trainer and evaluator. It has good scalability and can easily add new style types, adjust model parameters or improve existing components.

[0107] (7) A three-stage training strategy is adopted: style encoder pre-training, style vector library construction and ChatGLM fine-tuning, which ensures the stability and effectiveness of training and avoids the instability problem of end-to-end training.

[0108] (8) The LoRA technique is used for efficient parameter fine-tuning, requiring only about 0.1% of the model parameters to be adjusted, which significantly reduces training costs and memory requirements while maintaining good style transfer results. This embodiment can be widely applied to content creation, education and training, marketing copywriting, literary translation and personalized content, providing users with high-quality text style transfer services.

[0109] Corresponding to the above method, the present invention also provides an apparatus / system including a computer device, the computer device including a processor and a memory, the memory storing computer instructions, the processor executing the computer instructions stored in the memory, and when the computer instructions are executed by the processor, the apparatus / system performs the steps of the method as described above.

[0110] This invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the aforementioned edge computing server deployment method. The computer-readable storage medium can be a tangible storage medium, such as random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, floppy disks, hard disks, removable storage disks, CD-ROMs, or any other form of storage medium known in the art.

[0111] In summary, the text style transfer method and apparatus based on dual-conditional injection of semantics and style described in this invention employs a frozen BERT model to extract semantic vectors from the original text and obtains 256-dimensional quantized style vectors of the target style from a pre-constructed style vector library. Through a gating mechanism, the semantic vectors are injected into layer 6 of ChatGLM, and the style vectors into layer 18. Combined with natural language instructions and LoRA fine-tuning technology, text generation achieves semantic preservation and accurate style transfer. During fine-tuning, a three-stage training strategy is adopted: first, a style encoder is trained using classification loss and InfoNCE contrastive loss; then, a style vector library is constructed; finally, a 15% random mask is applied to single-style samples, and ChatGLM is trained under incomplete semantic conditions to reconstruct complete sentences. The style classification loss is calculated using the pre-trained style encoder, the semantic consistency loss is calculated using frozen BERT, and the language modeling loss is calculated. A multi-task triple loss function is used for joint optimization. This method significantly improves the accuracy and fluency of style transfer, achieving an effective balance between semantic preservation and style consistency.

[0112] Those skilled in the art will understand that the exemplary components, systems, and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software, or a combination of both. Whether implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this invention. When implemented in hardware, it can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this invention are programs or code segments used to perform the desired tasks. The programs or code segments can be stored in a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave.

[0113] It should be clarified that the present invention is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of the present invention.

[0114] In this invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with or in place of features of other embodiments.

[0115] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various modifications and variations of the embodiments of the present invention are possible. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A text style transfer method based on semantic and stylistic dual-condition injection, characterized in that, The method includes the following steps: The system obtains the source text and target text style types provided by the user, extracts the source text semantic feature vector of the first set dimension using a preset semantic extraction model, and queries a predefined style database based on the target text style type to obtain the standard style vector of the second set dimension; the semantic extraction model includes a frozen BERT model and a CLS pooling layer. The task instructions based on natural language, the semantic feature vector of the source text, and the normalized style vector are input into a preset text style transfer model through a two-layer injection mechanism to output target text that is semantically consistent with the source text and conforms to the target text style type. The task instructions are used to instruct the execution of the text style transfer task. The two-layer injection mechanism includes injecting the semantic feature vector into one of the first 1 / 3 to 1 / 2 layers of the text style transfer model and injecting the normalized style vector into one of the last 1 / 3 layers of the text style transfer model. The text style transfer model is obtained by low-rank adaptive fine-tuning of the ChatGLM large language model using a single-style text sample set. Each sample in the single-style text sample set contains a task instruction based on natural language, a sample text, and its corresponding sample text style label. The semantic extraction model is used to extract incomplete semantic feature vectors from the randomly corrupted sample text. The random corruption includes masking, lexical deletion, and similar character replacement implemented according to a preset ratio. The style vector is obtained by querying the style database based on the sample text style label. The task instruction, the incomplete semantic feature vector, and the sample style vector are input into the text style transfer model through the two-layer injection mechanism to perform an autoregressive task and output reconstructed text. The reconstructed text style is identified based on a preset text style recognition model. The text reconstruction bias loss, semantic bias loss, and style type bias loss are calculated based on the reconstructed text and the sample text, and a joint loss is constructed. The joint loss function is used to perform low-rank adaptive parameter updates on the text style transfer model.

2. The text style transfer method based on semantic and stylistic dual-condition injection according to claim 1, characterized in that, The semantic extraction model adopts the BERT model based on the Transformer architecture and extracts 768-dimensional semantic feature vectors based on CLS pooling; the text style transfer model is the ChatGLM model; during the low-rank adaptive parameter update process, the rank parameter is set to 8, the scaling parameter is set to 32, the dropout rate is set to 0.1, the main parameters of the ChatGLM model are frozen, and the parameters are updated for the query layer and the key-value layer.

3. The text style transfer method based on semantic and stylistic dual-condition injection according to claim 1, characterized in that, The two-layer injection mechanism includes injecting the semantic feature vector into one of the first 1 / 3 to 1 / 2 layers of the text style transfer model, and injecting the normalized style vector into one of the last 1 / 3 layers of the text style transfer model, including: The source text semantic feature vector is mapped through a projection layer and then injected into the 6th hidden state of the ChatGLM model; the canonical style vector is mapped through a projection layer and then injected into the 18th hidden state of the ChatGLM model; the fusion ratio between the injected vector and the original hidden state is controlled by a gating mechanism.

4. The text style transfer method based on semantic and style dual-condition injection according to claim 1, characterized in that, The steps for constructing the predefined style database include: Representative samples of various text styles are input into a pre-trained style encoder, which outputs a representative style vector of the representative samples. The style encoder includes a BERT model and a projection mapping layer, which outputs a 256-dimensional representative style vector. The average value of multiple representative style vectors corresponding to each text style is calculated to obtain the standard style vector of the corresponding text style.

5. The text style transfer method based on semantic and stylistic dual-condition injection according to claim 1, characterized in that, The text style recognition model includes a text style encoder and a text style classifier. The text style encoder consists of a BERT model and a projection mapping layer. The projection mapping layer includes a linear layer, a ReLU activation function, a Dropout layer, and a LayerNorm layer. The projection mapping layer maps the 768-dimensional BERT output to a 256-dimensional style vector. The text style classifier consists of multiple single linear connection layers, which map the 256-dimensional style vector to the number of style categories (logits). The pre-training steps of the text style recognition model include: Obtain a text style classification training dataset containing multiple style categories and multiple samples, where each sample contains sample text and its corresponding style classification label; The text style recognition model, which is randomly initialized, is trained using the text style classification training dataset. The text style encoder extracts sample style vectors from the sample texts and outputs style predictions for the sample texts via the text style classifier. A style classification loss is constructed based on the deviation between the style predictions and the style classification labels. An intra-style contrast loss is constructed using the sample style vectors corresponding to each sample text to reduce the differences between styles of the same type and increase the differences between styles of different types. The text style recognition model is updated by combining the style classification loss and the intra-style contrast loss.

6. The text style transfer method based on semantic and style dual-condition injection according to claim 5, characterized in that, The formula for calculating the style classification loss is: ； in, This represents the style classification loss. For cross-entropy loss, This indicates the style prediction. This indicates the style category label; The formula for calculating the intra-style contrast loss is: in, This represents the intra-style contrast loss. This represents the anchor style vector of the anchor samples selected within a batch. This represents the style vector of the j-th positive sample within the batch that has the same style as the anchor sample. Represents the set of all positive samples; This represents the style vector of the k-th negative sample within the batch that has a style different from the anchor sample. Let represent the set of all negative samples; sim represents the cosine similarity calculation; exp represents the exponential function. This refers to temperature hyperparameters. In updating the parameters of the text style recognition model by combining the style classification loss and the intra-style contrast loss, the first joint loss calculation formula is as follows: ； in, This represents the first joint loss. and These are the weighting coefficients.

7. The text style transfer method based on semantic and stylistic dual-condition injection according to claim 5, characterized in that, The text style transfer model is updated with parameters based on the loss constructed from the reconstructed text style and the semantic, style type, and text reconstruction biases of the sample text, including: The semantic extraction model is used to extract semantic vectors from the reconstructed text style and the sample text respectively, and the semantic bias loss is calculated based on cosine similarity. Calculate the cross-entropy between the reconstructed text style and the sample text style label as the style type bias loss; The cross-entropy between the predicted distribution of each word in the reconstructed text by the text style transfer model and the true distribution of each word in the sample text is calculated as the text reconstruction bias loss. The semantic bias loss, the style type bias loss, and the text reconstruction bias loss are weighted and summed to construct a second joint loss. The parameters of the text style transfer model are updated by minimizing the second joint loss.

8. A text style transfer apparatus that integrates semantic and style vectors, comprising a processor, a memory, and a computer program / instructions stored in the memory, characterized in that, The processor is configured to execute the computer program / instructions, and when the computer program / instructions are executed, the device implements the steps of the method as described in any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method as described in any one of claims 1 to 7.

10. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method according to any one of claims 1 to 7.

Images