Controllable text generation method, device and equipment

Abstract

The invention provides a controllable text generation method, device and equipment, and the method comprises the steps: obtaining an input text, segmenting the input text, and obtaining a plurality of input words; inputting an input word into the trained vertical classifier, and outputting a category probability corresponding to the input word; constructing a word-level attribute graph according to all the input words and the category probability corresponding to each input word; processing the word-level attribute graph by utilizing a webpage ranking algorithm to obtain a target keyword; generating a semantic-level attribute graph according to the attribute graph generation prompt; determining a target combination prompt according to the input text, the original prompt, the target keyword, the context information and the semantic-level attribute graph; and inputting the target combination prompt into the large language model, processing the target combination prompt by the large language model, and outputting a controllable text. The word-level attribute graph and the semantic-level attribute graph are combined to guide the large model to better understand the input text, so that the high-quality response is generated, and the controllable generation of the vertical drive is realized.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing, and in particular, to a controllable text generation method, apparatus, and device. Background Art

[0002] Controllable text generation refers to adding control over certain attributes, styles, key information, etc. of the generated text on the basis of traditional text generation. On the premise of maintaining the quality of the original text, the control conditions are integrated into the text generation process to make the generated text meet specific expectations or requirements.

[0003] Currently, for the generation of controllable text, the PPLM, MAGIC, or PCTG-X method is usually adopted. PPLM realizes the control of the generated text by optimizing in the activation space, and this method may cause fluctuations in the quality of the generated text in different situations. MAGIC improves the effect of multi-angle control through target-guided counterfactual enhancement in the inference stage, but its generation ability of attribute combinations may be limited. PCTG-X integrates multiple components, including a single-attribute discriminator based on prompt learning, GASTE, STASN, and TKE, etc. These components need to work together to achieve precise control of text attributes, resulting in a complex model structure. Summary of the Invention

[0004] In view of this, the purpose of the present disclosure is to propose a controllable text generation method, apparatus, and device to solve or partially solve the above problems.

[0005] Based on the above purpose, the first aspect of the present disclosure provides a controllable text generation method, and the method includes:

[0006] Obtain an input text, segment the input text to obtain a plurality of input words;

[0007] For each input word, input the input word into a trained stance classifier, and after being processed by the stance classifier, output the class probability corresponding to the input word;

[0008] Construct a word-level attribute graph according to all the input words and the class probability corresponding to each input word;

[0009] Process the word-level attribute graph by using a web ranking algorithm to obtain target keywords;

[0010] Determine a preset attribute graph generation prompt, and generate a semantic-level attribute graph according to the attribute graph generation prompt;

[0011] Obtain a preset original prompt and preset context information, and determine a target combined prompt according to the input text, the original prompt, the target keywords, the context information, and the semantic-level attribute graph;

[0012] Input the target combined prompt into a large language model, and after being processed by the large language model, a controllable text is output, where the controllable text is generated by the large language model according to the input text and the original prompt, and has the same property stance as the target keyword, context information, and semantic-level attribute graph.

[0013] Based on the same inventive concept, a second aspect of the present disclosure provides a controllable text generation device, including:

[0014] A text segmentation module, configured to obtain an input text, segment the input text, and obtain a plurality of input words;

[0015] A category probability determination module, configured to input each input word into a trained stance classifier, and after being processed by the stance classifier, output the category probability corresponding to the input word;

[0016] A word-level attribute graph determination module, configured to construct a word-level attribute graph according to all input words and the category probability corresponding to each input word;

[0017] A target keyword determination module, configured to process the word-level attribute graph by using a web ranking algorithm to obtain a target keyword;

[0018] A semantic-level attribute graph determination module, configured to determine a preset attribute graph generation prompt, and generate a semantic-level attribute graph according to the attribute graph generation prompt;

[0019] A target combined prompt determination module, configured to obtain a preset original prompt and preset context information, and determine a target combined prompt according to the input text, the original prompt, the target keyword, the context information, and the semantic-level attribute graph;

[0020] A controllable text generation module, configured to input the target combined prompt into a large language model, and after being processed by the large language model, output a controllable text, where the controllable text is generated by the large language model according to the input text and the original prompt, and has the same property stance as the target keyword, the context information, and the semantic-level attribute graph.

[0021] Based on the same inventive concept, a third aspect of the present disclosure provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable by the processor, where the processor implements the controllable text generation method as described above when executing the computer program.

[0022] Based on the same inventive concept, a fourth aspect of the present disclosure proposes a non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute the controllable text generation method as described above.

[0023] As can be seen from the above, the present disclosure proposes a controllable text generation method, apparatus and device, which obtain an input text, segment the input text to obtain a plurality of input words. For each input word, the input word is input into a trained stance classifier, and after being processed by the stance classifier, the class probability corresponding to the input word is output. A word-level attribute graph is constructed according to all the input words and the class probability corresponding to each input word. The word-level attribute graph is processed by using a web ranking algorithm to obtain target keywords, and by identifying keywords related to stances and other attributes, the control and quality of subsequent large language model-generated controllable text are improved. At the same time, the target keywords will be used to emphasize (or avoid) in the final prompt, so as to achieve precise control of text attributes while ensuring the semantic integrity of the text, and determine a preset attribute graph generation prompt. A semantic-level attribute graph is generated according to the attribute graph generation prompt, a preset original prompt and preset context information are obtained, and a target combined prompt is determined according to the input text, the original prompt, the target keywords, the context information and the semantic-level attribute graph. The target combined prompt is input into a large language model, and after being processed by the large language model, a controllable text is output. By combining the word-level attribute graph and the semantic-level attribute graph to identify semantic structures and keywords related to the stance attribute dimension, the large model is guided to better understand the input text, so as to generate high-quality responses and achieve stance-driven controllable generation. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the present disclosure or related technologies, the following will briefly introduce the drawings required for use in the embodiments or related technology descriptions. Obviously, the drawings in the following description are only embodiments of the present disclosure. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0025] Figure 1 It is a flowchart of the controllable text generation method according to an embodiment of the present disclosure;

[0026] Figure 2 It is a schematic diagram of the controllable text generation framework system according to an embodiment of the present disclosure;

[0027] Figure 3 It is a structural block diagram of the controllable text generation apparatus according to an embodiment of the present disclosure;

[0028] Figure 4 It is a schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure. Detailed implementation manners

[0029] To make the objectives, technical solutions and advantages of the present disclosure clearer and more understandable, the following further describes the present disclosure in detail with reference to specific embodiments and the accompanying drawings.

[0030] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present disclosure should be the ordinary meanings understood by those of ordinary skill in the art to which the present disclosure belongs. The terms "first", "second" and similar terms used in the embodiments of the present disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. The terms such as "including" or "comprising" mean that the elements or objects appearing before this word cover the elements or objects listed after this word and their equivalents, without excluding other elements or objects. The terms such as "connected" or "coupled" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The terms such as "upper", "lower", "left" and "right" are only used to represent relative positional relationships, and when the absolute position of the object being described changes, the relative positional relationship may also change accordingly.

[0031] The following are the explanations of the terms related to the present disclosure:

[0032] PPLM: PPLM is a conditional generation method for generating text. It is customized and controlled based on a pre-trained language model (such as GPT-2) without retraining the model. This method uses "external guidance" to regulate the output of the pre-trained language model, rather than re-training or fine-tuning the model. PPLM introduces an additional control signal (which can be an emotion label, a topic label, etc.), and these signals affect the generation process of the language model in a specific way. These control signals can be used to guide the generation process through a designed additional model (such as a discriminant model or a gradient guidance model). At the same time, by introducing an adversarial optimization method in the language model generation process, that is, by optimizing the objective of the generated sample to meet the requirements of the external control signal. Finally, the generated sample is adjusted multiple times to finally obtain the text that meets the given conditions. This process is dynamic, and each generated word is guided by the external control signal.

[0033] MAGIC: MAGIC is a multi - perspective controllable text generation method aimed at solving the problem of unbalanced attribute correlations by decoupling counterfactual enhancement. Specifically, MAGIC balances the correlations between different attributes by introducing counterfactual feature vectors in the attribute latent space. During the training phase, MAGIC uses counterfactual latent vectors to construct a semantically more balanced attribute latent space to mitigate the impact of unbalanced attribute correlations. During the inference phase, MAGIC further improves the effect of multi - perspective control through target - guided counterfactual enhancement. This method generates latent vectors with counterfactual features through an attribute decoupling module and adopts an iterative intersection retrieval algorithm to generate texts that conform to the target attribute combination.

[0034] PCTG - X: PCTG - X is a method for topic - oriented controllable text generation in social network scenarios. This model controls the decoding process of a pre - trained language model through a single - attribute discriminator based on prompt learning to achieve separate control over stance, style, and topic attributes. Specifically, the PCTG - X model controls the stance, style, and topic attributes of social text responses by custom - analyzing social topics. This model uses GASTE to extract sentiment triples in the text as prompt information for the stance attribute; uses STASN to identify the style attributes of social network texts; and introduces knowledge graph technology through TKE to enhance the topic relevance of the generated text. This method performs well in precisely controlling the three key attributes of text: stance, style, and topic.

[0035] Controllable text generation refers to adding control over certain attributes, styles, key information, etc. of the generated text on the basis of traditional text generation. On the premise of maintaining the quality of the original text, the control conditions are integrated into the text generation process so that the generated text meets specific expectations or requirements. Controllable text generation can be regarded as an ability dimension orthogonal to the objective knowledge ability of large language models. Although large language models perform well in objective abilities such as logical reasoning, text analysis, or problem - solving, controllable text generation emphasizes the way of expressing and presenting these objective information. It focuses on the way of conveying information. The main challenge of controllable text generation lies in seamlessly integrating these control conditions into the generation process without compromising the inherent quality of the output generated by large language models.

[0036] Stance controllability refers to the ability to precisely control the stance or attitude of the generated text during the text generation process, ensuring that the text is consistent with the given stance.

[0037] An attribute graph is a structured semantic representation method that can effectively capture the attributes in text and their relationships. In text generation tasks, the use of attribute graphs can guide the generation process to ensure that the generated content is closely aligned with the desired attribute specifications. Attribute graphs usually contain entities, attributes, and relationships. Attribute graphs can be word-level or semantic-level, corresponding to different levels of abstraction and detail. Word-level attribute graphs focus on attributes at the lexical and phrasal levels, while semantic-level attribute graphs contain deeper semantic information and relationships between entities.

[0038] Combinatorial prompting learning, combinatorial prompting is a prompting method that guides large language models to generate text consistent with a specific stance by combining different prompts and attribute graphs. The key advantage of combinatorial prompting is that it can combine the structured representation of attribute graphs and natural language prompts to improve the control ability and semantic relevance of large language models when generating text. This method does not require fine-tuning of the model, but rather achieves control over the generated text through carefully designed prompts.

[0039] Chain of thought is a type of prompting method that requires the model to gradually show the intermediate steps of its thinking and reasoning before generating the final answer. When faced with complex problems, chain-of-thought prompting can help the model break down the problem into smaller, more manageable parts, and then gradually solve these sub-problems, finally synthesizing them into a complete answer to the original problem. Chain-of-thought prompting helps the model understand the problem more deeply by providing more context and reasoning steps, and generates more accurate and coherent outputs. By explicitly showing the reasoning process, the chain-of-thought method improves the interpretability of the model's output, enabling users to understand how the model arrives at a specific answer.

[0040] Currently, for the generation of controllable text, the PPLM, MAGIC, or PCTG-X methods are usually adopted. PPLM achieves control over the generated text by optimizing in the activation space, and this method may cause fluctuations in the quality of the generated text in different situations. MAGIC improves the effect of multi-angle control through target-guided counterfactual enhancement in the inference stage, but its ability to generate attribute combinations may be limited. PCTG-X integrates multiple components, including a single-attribute discriminator based on prompting learning, GASTE, STASN, and TKE, etc. These components need to work together to achieve precise control over text attributes, resulting in a complex model structure.

[0041] Based on the above description, this embodiment proposes a method for generating controllable text, as Figure 1 shown, the method includes:

[0042] Step 101, obtain the input text, segment the input text to obtain multiple input words;

[0043] Step 102: For each input word, input the input word into the trained stance classifier. After being processed by the stance classifier, output the class probability corresponding to the input word.

[0044] Step 103: Construct a word-level attribute graph based on all the input words and the class probability corresponding to each input word.

[0045] Step 104: Process the word-level attribute graph using the PageRank algorithm to obtain the target keyword.

[0046] Step 105: Determine the preset attribute graph generation prompt, and generate a semantic-level attribute graph according to the attribute graph generation prompt.

[0047] Step 106: Obtain the preset original prompt and preset context information, and determine the target combined prompt according to the input text, the original prompt, the target keyword, the context information, and the semantic-level attribute graph.

[0048] Step 107: Input the target combined prompt into the large language model. After being processed by the large language model, output the controllable text, where the controllable text is generated by the large language model according to the input text and the original prompt, and has the same stance as the target keyword, the context information, and the semantic-level attribute graph.

[0049] Specifically in implementation, obtain the input text, where the input text is the text obtained according to the text that the user wants to input into the large language model. Segment the input text to obtain multiple input words.

[0050] Exemplarily, the input text is T. Segment the input text to obtain multiple input words, denoted as S = {s1, s2,..., s n}, where S is the set of segmented input words, and s i represents the i-th input word.

[0051] Pre-train the stance classifier in advance to obtain the trained stance classifier. For each input word, input the input word into the trained stance classifier. After the stance classifier processes the input word, output the class probability corresponding to the input word.

[0052] In this embodiment, the class probability is represented by the formula:

[0053] score(s i ) = σ(classifier(s i ))

[0054] where score(s i) is the class probability, σ is the Sigmoid function, and σ(classifier(s i )) is the output of the stance classifier.

[0055] Construct a word-level attribute graph based on all input words and the class probabilities corresponding to each input word. Specifically, each input word s i is defined as the node v i of the word-level attribute graph. Each node represents a word, and the edges between nodes represent the relationships between words. The characteristics of the node include the content of the sentence (e.g., text string) and the sentiment score score(v i ), where the sentiment score of the node is the class probability score(s i ) corresponding to the input word.

[0056] Process the word-level attribute graph using the PageRank algorithm to obtain the target keyword. The PageRank algorithm is a widely used method for analyzing the importance of nodes and is particularly suitable for analyzing the global influence between nodes. The importance of a node depends not only on its direct connections but also on the importance of the nodes connected to it.

[0057] In this embodiment, the specific process of processing the word-level attribute graph using the PageRank algorithm to obtain the target keyword includes:

[0058] Use the PageRank algorithm to determine the PageRank value of each node in the word-level attribute graph, select the node with the highest PageRank value as the target node, and use the input word corresponding to the target node as the target keyword. The PageRank value is represented by the formula:

[0059]

[0060] where PR(v i ) is the PageRank value corresponding to node v i , d is the damping factor, is the set of all nodes pointing to node v i , and L(v j ) is the out-degree of node v j .

[0061] Determine a preset attribute graph generation prompt, which is used to prompt that the generated graph must include Entities, Attributes, and Relationships, and then subsequently guide the large language model to systematically construct a semantic-level attribute graph. Among them, Entities are the key entities in the input text (such as person names, events, organizations, etc.). Attributes are the attributes related to the Entities (such as stance, tone, etc.). Relationships are the relationships between Entities and Attributes.

[0062] Through the prompts generated by the foregoing process, combine all the prompts to form a final combined prompt for generating a stance-driven response. Specifically, obtain a preset original prompt and preset context information, and determine a target combined prompt according to the input text, the original prompt, the target keyword, the context information, and the semantic-level attribute graph. The target combined prompt is represented by the formula:

[0063]

[0064] Among them, is the target combined prompt, x is the input text, P in is the original prompt, W is the target keyword, which is used to remind the large language model of the words that should be emphasized and avoided in the lexical-level attribute graph. C is the context information, which briefly instructs the large language model to use the provided context, and G g is the generated semantic-level attribute graph.

[0065] Input the target combined prompt into the large language model, and after being processed by the large language model, output a controllable text. Among them, the controllable text is represented by the formula:

[0066]

[0067] Among them, R is the controllable text, that is, the stance control statement for the input text and the task prompt.

[0068] In this embodiment, the large language model is an LLM (Large Language Model, LLM) model, which refers to a deep learning model trained with a large amount of text data, so that the model can generate natural language text or understand the meaning of language text.

[0069] The input text is encoded into a shared embedding space, and the language model f θ (·) (parameterized by θ) can perform reasoning in this space, and the task prompt is tokenized and encoded using a fixed language embedding l. The language model (usually an LLM) will output a text response G g , that is, obtain the semantic-level attribute graph.

[0070] Specifically, the target combined prompt is input into the large language model. The target combined prompt is tokenized and converted into word embedding vectors. The word embedding vectors are processed through multiple encoder layers to obtain encoded vectors. Each encoder layer contains a self-attention mechanism and a feed-forward neural network. The self-attention mechanism determines the dependencies between words, and the words in the encoded vectors are adjusted according to the original prompt, target keywords, context, and semantic-level attribute graph, so that the stance attribute of the encoded vectors better meets the user's needs. The encoded vectors are decoded through multiple decoder layers to obtain the output text. Using the Softmax layer, it is converted into the final word sequence, and the word sequence is the controllable text.

[0071] Through the above solution, after the target combined prompt is input into the large language model, the large language model generates corresponding text according to the original prompt by understanding the meaning of the input text. At the same time, the target keywords remind the large language model to pay attention to the words that should be emphasized or avoided during the text generation process. The context information improves the coherence and consistency between the text generated by the large language model and the context. Finally, the semantic-level attribute graph enhances the large language model's understanding ability of the input text, further improving the accuracy of the obtained controllable text.

[0072] Through the above solution, the input text is obtained, and the input text is segmented to obtain multiple input words. For each input word, the input word is input into the trained stance classifier. After being processed by the stance classifier, the class probability corresponding to the input word is output. The word-level attribute graph is constructed according to all the input words and the class probability corresponding to each input word. The PageRank algorithm is used to process the word-level attribute graph to obtain the target keywords. By identifying the keywords related to the stance and other attributes, the control and quality of the subsequent large language model to generate controllable text are improved. At the same time, the target keywords will be used to emphasize (or avoid) in the final prompt, so as to achieve precise control of the text attributes while ensuring the semantic integrity of the text. Determine the preset attribute graph generation prompt. The semantic-level attribute graph is generated according to the attribute graph generation prompt. The preset original prompt and preset context information are obtained. The target combined prompt is determined according to the input text, original prompt, target keywords, context information, and semantic-level attribute graph. The target combined prompt is input into the large language model. After being processed by the large language model, the controllable text is output. The combination of the word-level attribute graph and the semantic-level attribute graph is used to identify the semantic structure and keywords related to the stance attribute dimension, guiding the large model to better understand the input text, thereby generating high-quality responses and realizing stance-driven controllable generation.

[0073] In some embodiments, step 101 specifically includes:

[0074] Step 1011, obtain the initial input text, input the initial input text into a large language model, and after being processed by the large language model, output the input text;

[0075] Step 1012, perform segmentation processing on the input text to obtain multiple input words.

[0076] Specifically in implementation, obtain the initial input text, where the initial input text is the text that the user wants to input into the large language model. Use the initial input text as a prompt text and input it into the large language model. After being processed by the large language model, obtain the input text to form a corpus.

[0077] Exemplarily, by using the input text as a prompt text and using the large language model to generate multiple candidate sentences, finally form a corpus X = {X1, X2, …, X m}. Each sentence X j is a token sequence {x 1j , x 2j , …, x njj}, where n j represents the number of tokens in the sentence.

[0078] This process depends on the context understanding ability of the language model and sampling techniques to ensure the diversity of the generated content (where the parameters are set as "top_k": 100, "top_p": 0.85, "temperature": 0.7).

[0079] Assume that it is necessary to generate a text sequence y = [y1, y2,..., y T , and the generation probability is:

[0080]

[0081] where x is the input prompt text, and P(y t |y1, y2,..., y t-1 , x) is the probability that the model generates the word y t under the given context.

[0082] In some embodiments, the training process of the stance classifier required in step 102 specifically includes:

[0083] Step 10A, obtain a text data set and an initial stance classifier, where the initial stance classifier includes a classification head, the text data set includes a training data set and a test data set, the training data set includes training texts and the training true labels corresponding to the training texts, and the test data set includes test texts and the test true labels corresponding to the test texts;

[0084] Step 10B: Map the training text to the classification space using a classification head to obtain training class probabilities.

[0085] Step 10C: Determine the average loss function corresponding to the initial stance classifier according to the training class probabilities and the training true labels.

[0086] Step 10D: In response to the average loss function converging to a preset convergence threshold, input the test text into the initial stance classifier to output predicted class probabilities.

[0087] Step 10E: Determine the accuracy of the initial stance classifier according to the predicted class probabilities and the test true labels.

[0088] Step 10F: In response to the accuracy being greater than a preset accuracy threshold, determine that the training of the initial stance classifier is completed to obtain a stance classifier.

[0089] In specific implementation, obtain a text dataset and an initial stance classifier, where the initial stance classifier includes a classification head, and the text dataset includes a training dataset and a test dataset.

[0090] To prevent the impact of class imbalance on model training, first calculate the number of labels with fewer positive stance labels and negative stance labels, then perform random sampling with equal sample numbers for each class, and finally achieve a balanced dataset. The text dataset is loaded in JSON format, contains text and corresponding sentiment labels (0 or 1), and select the text and label columns for analysis.

[0091] Randomly divide the text dataset to obtain a training dataset and a test dataset. Exemplarily, the number of data in the training dataset accounts for 90% of the text dataset, and the number of data in the test dataset accounts for 10% of the text dataset. The training dataset includes training text and the training true labels corresponding to the training text, and the test dataset includes test text and the test true labels corresponding to the test text.

[0092] Initialize the initial stance classifier, use a pre-trained Transformer model such as BERT, and load the model configuration and weights:

[0093]

[0094] Connect a linear classification head (classification_head) after the output of the basic Transformer model for subsequent use of the linear classification head to map the text embedding to the classification space.

[0095] The training text is mapped using the classification head in the initial stance classifier to map the training text to the classification space, obtaining the training class probabilities. An average loss function corresponding to the initial stance classifier is determined based on the training class probabilities and the training true labels.

[0096] When the average loss function converges to a preset convergence threshold, it indicates that the initial training of the initial stance classifier is completed. The test text is input into the initial stance classifier, and after being processed by the initial stance classifier, the predicted class probabilities are output.

[0097] The accuracy rate corresponding to the initial stance classifier is determined based on the predicted class probabilities and the test true labels, where the accuracy rate represents the proportion of the class probabilities predicted by the model that match the test true labels. If the accuracy rate is greater than the preset accuracy rate threshold, it is determined that the training of the initial stance classifier is completed, and the stance classifier is obtained.

[0098] In this embodiment, the preset accuracy rate threshold is preferably 98%.

[0099] Through the above solution, during the training process of the initial stance classifier, the minimization of the loss value and the maximization of the classification accuracy rate are mainly concerned. When the accuracy rate reaches the preset accuracy rate threshold, it can be considered that the training of the stance classifier is completed, and the model parameters are saved for subsequent input of the input word into the trained stance classifier. After being processed by the stance classifier, the class probabilities corresponding to the input word are output.

[0100] In some embodiments, step 10A specifically includes:

[0101] Step 10A1, obtaining an initial text dataset, and processing the data in the initial text dataset using a preset tokenizer to obtain the identification sequence of each word in the initial text dataset;

[0102] Step 10A2, determining the validity of each word in the initial text dataset, and generating an attention mask corresponding to the initial text dataset according to the validity;

[0103] Step 10A3, constructing a text dataset according to the identification sequence, the attention mask, and token_type_ids.

[0104] Specifically in implementation, an initial text dataset is obtained, and the data in the initial text dataset is processed using a preset tokenizer to obtain the identification sequence of each word in the initial text dataset.

[0105] In this embodiment, the preset tokenizer is tokenizer, and the initial text dataset is x = {x1, x2, …, x T}, where T is the number of words in the initial text dataset, and the identification sequence is represented by the formula:

[0106] input ids = {i1, i2, …, i T}

[0107] where input ids is the identification sequence.

[0108] Determine the validity of each word in the initial text dataset, and generate the attention mask corresponding to the initial text dataset according to the validity. In this embodiment, if the word is valid, the attention mask is 1, and if the word is invalid, the attention mask is 0. The attention mask is represented by the formula:

[0109] attention_mask = {m1, m2, …, m T}

[0110] where attention_mask is the attention mask corresponding to the initial text dataset, m i ∈ {0, 1}, and m i represents the validity of each word.

[0111] Construct the text dataset according to the identification sequence and the attention mask, that is, the text dataset is represented by the formula (input_ids, attention_mask).

[0112] In some embodiments, step 10B specifically includes:

[0113] Step 10B1, input the training text into the sequence labeling model, and obtain the token-level embedding through the processing of the sequence labeling model;

[0114] Step 10B2, perform weighted pooling on the token-level embedding to obtain the sentence-level embedding, where the sentence-level embedding is represented by the formula:

[0115]

[0116] where e is the sentence-level embedding, H = [h1, h2, …, h T , H is the token-level embedding, h i is a d-dimensional vector, T is the number of words corresponding to the training text, m i is the i-th value of the attention mask, indicating whether the i-th word in the training text is valid, and ∈ is a constant;

[0117] Step 10B3, calculate the initial score value according to the sentence-level embedding, where the initial score value is represented by the formula:

[0118]

[0119] Among them, is the initial score value, is the weight matrix of the classification head, is the bias term;

[0120] Step 10B4, process the initial score value using an activation function to obtain the training class probability, where the training class probability is expressed by the formula:

[0121]

[0122] Among them, is the training class probability.

[0123] In specific implementation, the sequence labeling model is a Transformer model, and the Transformer model is a sequence labeling model based on the self-attention mechanism. Input the training text into the sequence labeling model, and after being processed by the sequence labeling model, obtain the token-level embedding, where the token-level embedding is expressed by the formula:

[0124]

[0125] Among them, H = [h1, h2,..., h T , h i is a d-dimensional vector.

[0126] Perform weighted pooling on the token-level embedding to obtain the sentence-level embedding, where the sentence-level embedding is expressed by the formula:

[0127]

[0128] Among them, e is the sentence-level embedding, H = [h1, h2,..., h T , H is the token-level embedding, h i is a d-dimensional vector, T is the number of words corresponding to the training text, m i is the i-th value of the attention mask, indicating whether the i-th word in the training text is valid, ∈ is a constant.

[0129] In this embodiment, ∈ is a very small constant (such as 1e-9), and the purpose is to prevent division by zero.

[0130] Calculate the initial score value based on the sentence-level embedding, where the initial score value represents the classification result of the text, and the initial score value is expressed by the formula:

[0131]

[0132] Among them, is the initial score value, i.e., logits, is the weight matrix of the classification head, is the bias term. The calculated logits are scores used to represent each class. In a classification task, logits are the raw scores output by the model and are passed to an activation function (such as softmax) to generate the final class probabilities, that is, class probabilities;

[0133] The activation function is used to process the initial score value to obtain the training class probabilities, where the training class probabilities are expressed by the formula:

[0134]

[0135] where, is the training class probability.

[0136] In some embodiments, step 10C specifically includes:

[0137] Step 10C1, for each training data in the training dataset, determine the initial loss function corresponding to the training data according to the training class probability corresponding to the training data and the training true label corresponding to the training data, where the initial loss function is expressed by the formula:

[0138]

[0139] where, is the initial loss function corresponding to the i-th training data in the training dataset, N is the number of training data in the training dataset, y i is the training true label corresponding to the i-th training data;

[0140] Step 10C2, determine the average loss function corresponding to the initial stance classifier according to the initial loss functions corresponding to all training data, where the average loss function is expressed by the formula:

[0141]

[0142] where, rain Loss is the average loss function.

[0143] Specifically in implementation, for each training data in the training dataset, determine the initial loss function corresponding to the training data according to the training class probability corresponding to the training data and the training true label corresponding to the training data, where the initial loss function is expressed by the formula:

[0144]

[0145] where, is the initial loss function corresponding to the i-th training data in the training dataset, N is the number of training data in the training dataset, and y i is the training true label corresponding to the i-th training data.

[0146] After determining the initial loss function corresponding to each training data in all the training data, the average loss function corresponding to the initial stance classifier is calculated according to all the loss functions, where the average loss function is expressed by the formula:

[0147]

[0148] where, rain Loss is the average loss function.

[0149] In some embodiments, step 10E specifically includes:

[0150] Step 10E1, obtaining a preset sentiment threshold, and performing a conversion process on the predicted class probability according to the preset sentiment threshold to obtain a predicted class label corresponding to the predicted class probability, where the predicted class label is expressed by the formula:

[0151]

[0152] where, is the predicted class label, is the predicted class probability, and 0.5 is the preset sentiment threshold;

[0153] Step 10E2, determining the accuracy rate of the initial stance classifier according to the predicted class label and the test true label, where the accuracy rate is expressed by the formula:

[0154]

[0155] where, Accuracy is the accuracy rate, M is the number of test texts in the test dataset, is the indicator function, which is 1 when the predicted class label and the test true label are the same, and 0 otherwise.

[0156] In specific implementation, a preset sentiment threshold is obtained. In this embodiment, the preset sentiment threshold is preferably 0.5.

[0157] Performing a conversion process on the predicted class probability according to the preset sentiment threshold to obtain a predicted class label corresponding to the predicted class probability, where the predicted class label is expressed by the formula:

[0158]

[0159] where, is the predicted class label, For predicting the class probability, 0.5 is the preset sentiment threshold.

[0160] Determine the accuracy rate corresponding to the initial stance classifier according to the predicted class label and the test true label, where the accuracy rate is expressed by the formula:

[0161]

[0162] Among them, Accuracy is the accuracy rate, M is the number of test texts in the test dataset, is the indicator function, which is 1 when the predicted class label and the test true label are the same, and 0 otherwise.

[0163] In some embodiments, step 105 specifically includes:

[0164] Step 1051, obtain the preset original prompt and the initial attribute graph generation prompt, and construct the attribute graph generation prompt according to the input text, the original prompt and the initial attribute graph generation prompt, where the attribute graph generation prompt is expressed by the formula:

[0165]

[0166] Among them, is the attribute graph generation prompt, x is the input text, P in is the original prompt, G in is the initial attribute graph generation prompt;

[0167] Step 1052, input the attribute graph generation prompt into the large language model, and after being processed by the large language model, output the semantic-level attribute graph, where the semantic-level attribute graph is expressed by the formula:

[0168]

[0169] Among them, G g is the semantic-level attribute graph.

[0170] Specifically in implementation, obtain the preset original prompt and the initial attribute graph generation prompt, and construct the attribute graph generation prompt according to the input text, the original prompt and the initial attribute graph generation prompt. The format of the attribute graph generation prompt is in JSON format, and the initial attribute graph generation prompt is used to guide the large language model to comprehensively construct a semantic graph with three key features (entities, attributes, and their relationships).

[0171] In this embodiment, the attribute graph generation prompt is expressed by the formula:

[0172]

[0173] Among them, Generate prompts for the attribute graph, where x is the input text, and P in is the original prompt, G in Generate prompts for the initial attribute graph, where [·] represents the slots for inserting the various elements of the question.

[0174] Input the generated prompts for the attribute graph into a large language model, and after being processed by the large language model, output a semantic-level attribute graph, where the semantic-level attribute graph is represented by the formula:

[0175]

[0176] where, G g is the semantic-level attribute graph.

[0177] In this embodiment, the format of the semantic-level attribute graph is the JSON format, which is convenient for subsequent parsing and processing by the large language model. By organizing information structurally, it avoids generating responses that are too concise or lack semantics.

[0178] The controllable text generation method proposed in this embodiment aims to generate controllable and consistent statements, helping users express their views more effectively and promoting constructive conversations. It uses a combined attribute graph to identify semantic structures and keywords related to the dimension of the stance attribute, extracts word-level and semantic-level attribute graphs from the input text. On the one hand, it identifies keywords related to the stance and other attributes from the input text, thereby improving the control and quality of the sentences generated by the LLM. On the other hand, by extracting entities, attributes, and their relationships, it encapsulates the deep structural semantic information of the input text using the zero-shot chain-of-thought prompting of the LLMs.

[0179] Combining the attribute graph in the prompt eliminates the need for fine-tuning and prevents forgetting, thereby improving the stance controllability and topic relevance of the generated text. Finally, the attribute graphs are combined as a compact language representation of the input text, making it an efficient prompting method.

[0180] The prompts derived from the aforementioned steps are combined with the input text and task prompts to form the final combined prompt, which is then input into the LLMs to obtain the final response as the generated statement. This combination method ensures that the generated statement maintains consistency and relevance of the stance, while achieving a relatively low perplexity, thus producing high-quality responses.

[0181] Based on the same inventive concept, another embodiment of the present disclosure provides a controllable text generation framework system, which can provide text generation services with more explicit stance and topic relevance according to the semantic structure and keywords of the input text. The system is as Figure 2 shown, and specifically includes:

[0182] The word-level attribute graph construction module includes a corpus generation unit, a classifier discrimination unit, and an attribute graph construction unit. It aims to identify key attribute-related words from the input sentence, which enables the LLM to more precisely control the controllability and quality of text generation. The word-level attribute graph is achieved by constructing a pair of graphs that display the positive and negative characteristics of the input sentence, and the PageRank algorithm is applied to screen out the most important nodes in the graph. The corpus generation unit is used to send the input text data into the large model to generate a corpus. The specific steps are as follows: Use the large model to generate a text sequence from the specified prompt, generate a series of corpora, and for each input text, generate 50 text sequences respectively to form a corpus library.

[0183] The classifier discrimination unit is used to learn to distinguish the nature of the input sentence in order to accurately evaluate and quantify the existence and intensity of the stance. It includes classifier training and generates a series of scores. The specific steps are as follows:

[0184] Train a text classifier, which is fine-tuned on a specific dataset; for the given text, generate a set of basic sentences through simple rules, denoted as x = {x1, x2, …, x m}. Use the trained classifier to score these sentences and generate a series of probability scores. This process can be expressed as:

[0185] S(x i ) = ClassifierScore(x i )

[0186] The attribute graph construction unit is used to construct an attribute graph at the lexical level that displays the input sentence. The specific steps are as follows:

[0187] For the task of generating a positive stance, directly use the score; for the task of generating a negative stance, use 1 - S(x i ). Each sentence is tokenized into discrete tokens. The transformed scores are then added to the graph. The order of the tokens will also be considered.

[0188] Then apply the PageRank algorithm to calculate the importance of each node in the graph. PageRank quantifies the importance of each node by considering the edge weights between nodes, and identifies the tokens that most significantly affect the result as the words related to the stance attribute, denoted as W.

[0189] S2 Semantic attribute graph construction module, uses the attribute graph generation prompt to guide the LLM to systematically construct a semantic-level attribute graph. The semantic-level attribute graph generation prompt G in guides the LLM to comprehensively construct a semantic graph with three key characteristics (entities, attributes, and their relationships). In the attribute graph generation prompt, it is further required that its format must be JSON. The input text x, the task prompt P in and the attribute graph generation prompt Gin , together constitute the first-round combined prompts for the LLM The complete prompt is as follows:

[0190]

[0191] Among them, [·] represents the slots for inserting the various elements of the question. The LLM thus generates a Semantic-Level Attribute Graph (SLAG), as shown below:

[0192]

[0193] The S3 combined prompt generation module uses the prompts generated by the previous two modules to combine all the prompts to form a combined prompt. Therefore, the final prompt will incorporate the relevant information from the original task prompt, the input text, and the generated semantic-level and lexical-level attribute graphs.

[0194] The overall input prompt for generating the response is as follows:

[0195]

[0196] where x is the input text, P in is the original task prompt, G g is the generated attribute graph. In addition, C briefly indicates that the LLM uses the provided context, and W is used to remind the LLM of the words to be emphasized and avoided in the lexical-level attribute graph. W and G g can effectively achieve lexical-level and semantic-level control of the input text. This makes the stance of the generated response more controllable and enhances its consistency. Therefore, the LLM generates the final response R, as a stance control statement for the input text and the task prompt, as shown below:

[0197]

[0198] It should be noted that the method of the embodiments of the present disclosure can be executed by a single device, such as a computer or a server, etc. The method of this embodiment can also be applied to a distributed scenario, and multiple devices cooperate with each other to complete it. In the case of such a distributed scenario, one of the multiple devices can only execute one or more steps of the method of the embodiments of the present disclosure, and these multiple devices will interact with each other to complete the described method.

[0199] It should be noted that some embodiments of the present disclosure have been described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the above embodiments and still achieve the desired results. Additionally, the processes depicted in the drawings do not necessarily require the particular order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0200] Based on the same inventive concept, corresponding to any of the above method embodiments, the present disclosure also provides a controllable text generation device.

[0201] Referring to Figure 3 , Figure 3 the controllable text generation device for the embodiment, includes:

[0202] A text segmentation module 301, configured to obtain an input text, segment the input text, and obtain a plurality of input words;

[0203] A category probability determination module 302, configured to, for each input word, input the input word into a trained stance classifier, and output the category probability corresponding to the input word after being processed by the stance classifier;

[0204] A word-level attribute graph determination module 303, configured to construct a word-level attribute graph according to all input words and the category probability corresponding to each input word;

[0205] A target keyword determination module 304, configured to process the word-level attribute graph using a web ranking algorithm to obtain a target keyword;

[0206] A semantic-level attribute graph determination module 305, configured to determine a preset attribute graph generation prompt, and generate a semantic-level attribute graph according to the attribute graph generation prompt;

[0207] A target combination prompt determination module 306, configured to obtain a preset original prompt and preset context information, and determine a target combination prompt according to the input text, the original prompt, the target keyword, the context information, and the semantic-level attribute graph;

[0208] A controllable text generation module 307, configured to input the target combination prompt into a large language model, and output a controllable text after being processed by the large language model, where the controllable text is generated by the large language model according to the input text and the original prompt, and has the same stance attribute as the target keyword, the context information, and the semantic-level attribute graph.

[0209] In some embodiments, the text segmentation module 301 is specifically configured to:

[0210] Obtain an initial input text, input the initial input text into a large language model, and output the input text after being processed by the large language model;

[0211] Perform a segmentation process on the input text to obtain multiple input words.

[0212] In some embodiments, the device further includes a model training module, and the model training module specifically includes:

[0213] A data acquisition unit, configured to acquire a text data set and an initial stance classifier, where the initial stance classifier includes a classification head, the text data set includes a training data set and a test data set, the training data set includes training texts and training true labels corresponding to the training texts, and the test data set includes test texts and test true labels corresponding to the test texts;

[0214] A training class probability determination unit, configured to map the training texts to a classification space by using the classification head to obtain training class probabilities;

[0215] An average loss function determination unit, configured to determine an average loss function corresponding to the initial stance classifier according to the training class probabilities and the training true labels;

[0216] A predicted class probability determination unit, configured to, in response to the average loss function converging to a preset convergence threshold, input the test texts into the initial stance classifier and output predicted class probabilities;

[0217] An accuracy determination unit, configured to determine the accuracy corresponding to the initial stance classifier according to the predicted class probabilities and the test true labels;

[0218] A stance classifier determination unit, configured to, in response to the accuracy being greater than a preset accuracy threshold, determine that the training of the initial stance classifier is completed and obtain a stance classifier.

[0219] In some embodiments, the data acquisition unit is specifically configured to:

[0220] Obtain an initial text data set, process the data in the initial text data set by using a preset tokenizer to obtain an identification sequence of each word in the initial text data set;

[0221] Determine the validity of each word in the initial text data set, and generate an attention mask corresponding to the initial text data set according to the validity;

[0222] Construct a text data set according to the identification sequence and the attention mask.

[0223] In some embodiments, the training category probability determination unit is specifically configured to:

[0224] Input the training text into a sequence labeling model, and obtain token-level embeddings through the processing of the sequence labeling model;

[0225] Perform weighted pooling on the token-level embeddings to obtain sentence-level embeddings, where the sentence-level embeddings are represented by the formula:

[0226]

[0227] where e is the sentence-level embedding, H = [h1, h2,..., h T , H is the token-level embedding, h i is a d-dimensional vector, T is the number of words in the training text, m i is the i-th value of the attention mask, indicating whether the i-th word in the training text is valid, and ∈ is a constant;

[0228] Calculate an initial score value based on the sentence-level embedding, where the initial score value is represented by the formula:

[0229]

[0230] where, is the initial score value, is the weight matrix of the classification head, is the bias term;

[0231] Process the initial score value using an activation function to obtain the training category probability, where the training category probability is represented by the formula:

[0232]

[0233] where, is the training category probability.

[0234] In some embodiments, the average loss function determination unit is specifically configured to:

[0235] For each training data in the training dataset, determine the initial loss function corresponding to the training data according to the training category probability corresponding to the training data and the training true label corresponding to the training data, where the initial loss function is represented by the formula:

[0236]

[0237] where, is the initial loss function corresponding to the i-th training data in the training dataset, N is the number of training data in the training dataset, y iis the training true label corresponding to the i-th training data;

[0238] Determine the average loss function of the initial stance classifier according to the initial loss function corresponding to all training data, where the average loss function is expressed by the formula:

[0239]

[0240] where rain Loss is the average loss function.

[0241] In some embodiments, the accuracy determination unit is specifically configured to:

[0242] Obtain a preset sentiment threshold, perform a conversion process on the predicted class probability according to the preset sentiment threshold, and obtain a predicted class label corresponding to the predicted class probability, where the predicted class label is expressed by the formula:

[0243]

[0244] where, is the predicted class label, is the predicted class probability, and 0.5 is the preset sentiment threshold;

[0245] Determine the accuracy of the initial stance classifier according to the predicted class label and the test true label, where the accuracy is expressed by the formula:

[0246]

[0247] where Accuracy is the accuracy, M is the number of test texts in the test dataset, is the indicator function, which is 1 when the predicted class label and the test true label are the same, and 0 otherwise.

[0248] In some embodiments, the semantic-level attribute graph determination module 305 is specifically configured to:

[0249] Obtain a preset original prompt and an initial attribute graph generation prompt, and construct an attribute graph generation prompt according to the input text, the original prompt, and the initial attribute graph generation prompt, where the attribute graph generation prompt is expressed by the formula:

[0250]

[0251] where, is the attribute graph generation prompt, x is the input text, P in is the original prompt, G in is the initial attribute graph generation prompt;

[0252] Input the prompt for generating the attribute graph into the large language model. After being processed by the large language model, a semantic-level attribute graph is output, where the semantic-level attribute graph is represented by the formula:

[0253]

[0254] where G g is the semantic-level attribute graph.

[0255] For the convenience of description, when describing the above device, it is divided into various modules according to functions for separate description. Of course, when implementing the present disclosure, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0256] The device of the above embodiment is used to implement the corresponding controllable text generation method in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be elaborated here.

[0257] Based on the same inventive concept, corresponding to the method of any of the above embodiments, the present disclosure also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, it implements the controllable text generation method described in any of the above embodiments.

[0258] Figure 4 FIG. shows a more specific schematic diagram of the hardware structure of the electronic device provided in this embodiment. The device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. Among them, the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are communicatively connected to each other inside the device through the bus 1050.

[0259] The processor 1010 may be implemented in a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, etc., and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0260] The memory 1020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage devices, dynamic storage devices, etc. The memory 1020 can store an operating system and other application programs. When implementing the technical solutions provided in the embodiments of this specification through software or firmware, the relevant program codes are stored in the memory 1020 and are called and executed by the processor 1010.

[0261] The input / output interface 1030 is used to connect to the input / output module to achieve information input and output. The input / output module can be configured as a component in the device (not shown in the figure) or can be externally connected to the device to provide corresponding functions. Among them, the input device can include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device can include a display, a speaker, a vibrator, an indicator light, etc.

[0262] The communication interface 1040 is used to connect to a communication module (not shown in the figure) to achieve communication interaction between this device and other devices. Among them, the communication module can achieve communication through wired means (such as USB, network cable, etc.) or can also achieve communication through wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0263] The bus 1050 includes a path for transmitting information between various components of the device (such as the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040).

[0264] It should be noted that although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040, and the bus 1050, in the specific implementation process, this device may also include other components necessary for normal operation. In addition, those skilled in the art can understand that the above device may also only include the components necessary to implement the solutions of the embodiments of this specification, and do not have to include all the components shown in the figure.

[0265] The electronic device in the above embodiments is used to implement the corresponding controllable text generation method in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be elaborated here.

[0266] Based on the same inventive concept, corresponding to the method in any of the above embodiments, the present disclosure also provides a non-transitory computer-readable storage medium storing computer instructions for causing the computer to execute the controllable text generation method as described in any of the foregoing embodiments.

[0267] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette tapes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible by a computing device.

[0268] The computer instructions stored in the storage medium of the above embodiment are used to cause the computer to execute the controllable text generation method described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be elaborated here.

[0269] It can be understood that before using the technical solutions of the various embodiments of the present disclosure, the types, usage scopes, usage scenarios, etc. of the personal information involved will be informed to the user in an appropriate manner, and the user's authorization will be obtained.

[0270] For example, in response to receiving an active request from the user, a prompt message is sent to the user to clearly prompt the user that the operation requested by the user will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application program, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0271] As an optional but non-limiting implementation manner, the way of sending a prompt message to the user in response to receiving an active request from the user can be, for example, in the form of a pop-up window, and the prompt message can be presented in text in the pop-up window. In addition, the pop-up window can also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0272] It can be understood that the above process of notifying and obtaining the user's authorization is only illustrative and does not limit the implementation manner of the present disclosure. Other ways that meet relevant laws and regulations can also be applied to the implementation manner of the present disclosure.

[0273] Those of ordinary skill in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of the present disclosure (including the claims) is limited to these examples; under the concept of the present disclosure, the technical features in the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations in different aspects of the embodiments of the present disclosure as described above, and they are not provided in detail for the sake of brevity.

[0274] In addition, for simplicity of explanation and discussion, and in order not to make the embodiments of the present disclosure difficult to understand, the well-known power / ground connections to integrated circuit (IC) chips and other components may or may not be shown in the provided drawings. Further, the devices may be shown in block diagram form in order to avoid making the embodiments of the present disclosure difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform on which the embodiments of the present disclosure are to be implemented (i.e., these details should be fully within the understanding of those skilled in the art). In cases where specific details (such as circuits) are set forth to describe exemplary embodiments of the present disclosure, it will be apparent to those skilled in the art that the embodiments of the present disclosure may be implemented without these specific details or with variations of these specific details. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0275] Although the present disclosure has been described in connection with specific embodiments thereof, many alternatives, modifications, and variations of these embodiments will be apparent to those of ordinary skill in the art based on the foregoing description. For example, other memory architectures (such as dynamic RAM (DRAM)) may be used with the embodiments discussed.

[0276] The embodiments of the present disclosure are intended to cover all such alternatives, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present disclosure shall be included within the protection scope of the present disclosure.

Claims

1. A controllable text generation method, characterized in that: include: Obtaining an input text, and segmenting the input text to obtain a plurality of input words; For each input word, the input word is input into a trained stance classifier, and the stance classifier processes the input word to output a category probability corresponding to the input word; Construct a word-level attribute graph based on all input words and the category probability corresponding to each input word; Processing the word-level attribute graph using a web page ranking algorithm to obtain target keywords; Determine a preset attribute graph generation prompt, and generate a semantic-level attribute graph according to the attribute graph generation prompt; Obtaining a preset original prompt and preset context information, and determining a target combination prompt according to the input text, the original prompt, the target keyword, the context information, and the semantic-level attribute graph; The target combination prompt is input into a large language model, processed by the large language model, and a controllable text is output, wherein the controllable text is generated by the large language model according to the input text and the original prompt, and has the same attribute stance as the target keywords, context information, and semantic-level attribute graph.

2. The method according to claim 1, characterized in that: The step of obtaining an input text and segmenting the input text to obtain a plurality of input words includes: Acquire initial input text, input the initial input text into a large language model, process the large language model, and output the input text; The input text is segmented to obtain multiple input words.

3. The method according to claim 1, characterized in that The training process of the stance classifier includes: Acquire a text data set and an initial stance classifier, wherein the initial stance classifier includes a classification head, the text data set includes a training data set and a test data set, the training data set includes training texts and training true labels corresponding to the training texts, and the test data set includes test texts and test true labels corresponding to the test texts; Use the classification head to map the training text to the classification space to obtain the training category probability; Determine an average loss function corresponding to the initial stance classifier according to the training category probability and the training true label; In response to the average loss function converging to a preset convergence threshold, inputting the test text into an initial stance classifier and outputting a predicted category probability; Determine the accuracy rate corresponding to the initial stance classifier according to the predicted category probability and the test true label; In response to the accuracy being greater than a preset accuracy threshold, it is determined that the initial stance classifier training is completed, and a stance classifier is obtained.

4. The method according to claim 3, characterized in that The step of obtaining a text dataset includes: Acquire an initial text data set, and use a preset word segmenter to process the data in the initial text data set to obtain an identification sequence of each word in the initial text data set; Determine the validity of each word in the initial text data set, and generate an attention mask corresponding to the initial text data set according to the validity; A text dataset is constructed according to the identification sequence and the attention mask.

5. The method according to claim 4, characterized in that The classification head is used to map the training text to the classification space to obtain the training category probability, including: Inputting the training text into a sequence labeling model, and processing the sequence labeling model to obtain a word-level embedding; The word-level embedding is weighted pooled to obtain a sentence-level embedding, where the sentence-level embedding is expressed as: Where e is the sentence-level embedding, H = [h1,h2,…,h T ], H is the word level embedding, h i is a d-dimensional vector, T is the number of words corresponding to the training text, m i is the i-th value of the attention mask, indicating whether the i-th word in the training text is valid, and ∈ is a constant; The initial score value is calculated based on the sentence-level embedding, where the initial score value is expressed using the formula: in, is the initial score value, is the weight matrix of the classification head, is the bias term; The initial score value is processed using an activation function to obtain a training category probability, where the training category probability is expressed using the formula: in, is the training category probability.

6. The method according to claim 5, characterized in that The determining the average loss function corresponding to the initial stance classifier according to the training category probability and the training true label includes: For each training data in the training data set, the initial loss function corresponding to the training data is determined according to the training category probability corresponding to the training data and the training true label corresponding to the training data, wherein the initial loss function is expressed by the formula: in, is the initial loss function corresponding to the i-th training data in the training data set, N is the number of training data in the training data set, y i is the true training label corresponding to the i-th training data; The average loss function corresponding to the initial stance classifier is determined according to the initial loss function corresponding to all training data, wherein the average loss function is expressed by the formula: Among them, rain Loss is the average loss function.

7. The method according to claim 3, characterized in that The determining the accuracy rate corresponding to the initial stance classifier according to the predicted category probability and the test true label includes: The preset emotion threshold is obtained, and the predicted category probability is transformed according to the preset emotion threshold to obtain the predicted category label corresponding to the predicted category probability, wherein the predicted category label is expressed by the formula: in, To predict the class label, is the predicted category probability, and 0.5 is the preset sentiment threshold; The accuracy rate corresponding to the initial stance classifier is determined according to the predicted category label and the test true label, wherein the accuracy rate is expressed by the formula: Among them, Accuracy is the accuracy, M is the number of test texts in the test data set, It is an indicator function, which is 1 when the predicted category label and the test true label are consistent, otherwise it is 0.

8. The method according to claim 1, characterized in that The step of determining a preset attribute graph generation prompt and generating a semantic-level attribute graph according to the attribute graph generation prompt includes: Obtain a preset original prompt and an initial attribute graph generation prompt, and construct an attribute graph generation prompt according to the input text, the original prompt and the initial attribute graph generation prompt, wherein the attribute graph generation prompt is expressed by the formula: in, Generate prompts for the attribute graph, x is the input text, P in is the original prompt, G in Generate hints for the initial property graph; The attribute graph generation prompt is input into the large language model, and after being processed by the large language model, a semantic level attribute graph is output, wherein the semantic level attribute graph is expressed by the formula: Among them, G g is a semantic level attribute graph.

9. A controllable text generation device, characterized in that: include: A text segmentation module is configured to obtain an input text and segment the input text to obtain a plurality of input words; A category probability determination module is configured to input each input word into a trained stance classifier, and output a category probability corresponding to the input word after being processed by the stance classifier; A word-level attribute graph determination module is configured to construct a word-level attribute graph according to all input words and the category probability corresponding to each input word; a target keyword determination module, configured to process the word-level attribute graph using a web page ranking algorithm to obtain a target keyword; A semantic-level property graph determination module is configured to determine a preset property graph generation prompt, and generate a semantic-level property graph according to the property graph generation prompt; A target combination prompt determination module is configured to obtain a preset original prompt and preset context information, and determine a target combination prompt according to the input text, the original prompt, the target keyword, the context information and the semantic level attribute graph; The controllable text generation module is configured to input the target combination prompt into a large language model, process it through the large language model, and output controllable text, wherein the controllable text is generated by the large language model based on the input text and the original prompt, and has the same attribute stance as the target keywords, context information, and semantic-level attribute graph.

10. An electronic device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method according to any one of claims 1 to 8 is implemented.

Images