Text generation method and device, storage medium and electronic equipment

Abstract

The invention discloses a text generation method and device, a storage medium and electronic equipment. The method comprises the steps that a request text sequence is obtained, and the request text sequence comprises a non-forbidden word lexical element and a forbidden word lexical element carrying a forbidden word mark; the request text sequence is input into a first model for text generation, a reply text is obtained, in the first model, a first generation weight is adopted for the forbidden word lexical elements, a second generation weight is adopted for the non-forbidden word lexical elements, and the first generation weight is smaller than or equal to a preset threshold value; the reply text has a semantic association relationship with the semantics of the forbidden word lexical elements and the semantics of the non-forbidden word lexical elements. According to the text generation method and device, the technical problem that the accuracy of the text generated by the model is relatively low due to the limitation of an existing text generation method on a processing mode of the abnormal text in the input text of the model is solved.

Description

Technical Field

[0001] This application relates to the field of text processing, and more specifically, to a text generation method and apparatus, a storage medium, and an electronic device. Background Technology

[0002] In recent years, with the rapid development of artificial intelligence technology, Large Language Models (LLMs) have been increasingly widely used in dialogue systems, bringing significant cost savings and efficiency improvements to enterprises. However, these scenarios are often accompanied by complex emotion management and strict compliance requirements, making it a significant challenge to train models to generate responses that meet both business needs and industry standards.

[0003] Current technologies for generating responses directly remove dialogue turns containing non-compliant or emotionally charged content in an attempt to obtain more compliant model output. However, while this approach avoids the model learning incorrect information to some extent, it reveals some limitations in practical applications. For example, it ignores the semantic coherence of the context and the impact of emotional progression during multi-turn dialogues, resulting in generated responses that are inconsistent with the specific dialogue situation and leading to inaccurate response results. Summary of the Invention

[0004] This application provides a text generation method, apparatus, storage medium, and electronic device to at least solve the technical problem that existing text generation methods have limitations in handling abnormal text in the input text of the model, resulting in low accuracy of the text generated by the model.

[0005] According to one aspect of the embodiments of this application, a text generation method is provided, comprising: obtaining a request text sequence, wherein the request text sequence includes non-forbidden words and forbidden words carrying forbidden word markers; inputting the request text sequence into a first model for text generation to obtain a response text, wherein in the first model, a first generation weight is applied to the forbidden words and a second generation weight is applied to the non-forbidden words, the first generation weight is less than or equal to a preset threshold, and the response text has a semantic association with both the semantics of the forbidden words and the semantics of the non-forbidden words.

[0006] According to another aspect of the embodiments of this application, a text generation apparatus is also provided, comprising: an acquisition unit, configured to acquire a request text sequence, wherein the request text sequence includes non-forbidden words and forbidden words carrying forbidden word markers; and a generation unit, configured to input the request text sequence into a first model for text generation to obtain a response text, wherein in the first model, a first generation weight is applied to the forbidden words and a second generation weight is applied to the non-forbidden words, the first generation weight is less than or equal to a preset threshold, and the response text has a semantic association with both the semantics of the forbidden words and the semantics of the non-forbidden words.

[0007] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, wherein a computer program is stored in the computer program, and the computer program is configured to execute the above-described text generation method at runtime.

[0008] According to another aspect of the embodiments of this application, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the text generation method as described above.

[0009] According to another aspect of the embodiments of this application, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the above-described text generation method through the computer program.

[0010] In this embodiment, a request text sequence including non-forbidden words and forbidden words carrying forbidden word markers is obtained, and the request text sequence is input into a first model. Different generation weights are applied to forbidden words and non-forbidden words to generate text, so as to obtain response text that has semantic association with both forbidden and non-forbidden words. By configuring different generation weights for abnormal text and normal text in the input text of the text generation model, the content of abnormal text in the output text of the model can be reduced without changing the input text. This not only preserves the text content in the input text completely, thus avoiding the loss of content or semantic omission in the input text of the model, which leads to poor accuracy of the output text of the model, but also enables the processing of abnormal text in the input text of the model by using different generation weights. This allows the model to completely identify the input text while avoiding the generation of abnormal text, thereby improving the accuracy of the generated text. This solves the technical problem that the existing text generation methods have limitations in handling abnormal text in the input text of the model, resulting in low accuracy of the generated text. Attached Figure Description

[0011] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a schematic diagram of an application environment for an optional text generation method according to an embodiment of this application; Figure 2 This is a flowchart of an optional text generation method according to an embodiment of this application; Figure 3 This is a flowchart of another optional text generation method according to an embodiment of this application; Figure 4 This is a schematic diagram of an optional text generation method according to an embodiment of this application; Figure 5 This is a schematic diagram of another optional text generation method according to an embodiment of this application; Figure 6 This is a schematic diagram of the structure of an optional text generation model according to an embodiment of this application; Figure 7 This is a flowchart of another optional text generation method according to an embodiment of this application; Figure 8 This is a flowchart of another optional text generation method according to an embodiment of this application; Figure 9 This is a schematic diagram of an optional text generation device according to an embodiment of this application; Figure 10 This is a schematic diagram of the structure of an optional electronic device according to an embodiment of this application. Detailed Implementation

[0012] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0013] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0014] With the rapid development of artificial intelligence technology, large-scale language models are playing an increasingly important role in online dialogue scenarios, especially in customer service and financial consultation. They can replace human agents, automatically generating corresponding response texts based on user input to achieve real-time, rapid automated dialogue. However, the input text for existing large-scale language models sometimes contains prohibited language elements such as threatening phrases and non-compliant expressions. The presence of these prohibited elements may cause the model to generate response texts containing such content, resulting in poor accuracy and a reduced user experience.

[0015] Traditional methods for handling forbidden language in input text often employ a "detect-then-remove" text processing strategy. This involves using pre-defined forbidden language filtering rules or pre-trained machine learning models to identify input text containing forbidden language, and then removing entire sentences or paragraphs containing forbidden language from the input text. This prevents the model from learning the expressions of forbidden language and thus avoids the presence of forbidden language in the model's output text. However, directly deleting forbidden language from the input text results in the loss of content, leading to semantic incoherence. This can cause the model to generate responses with lower accuracy and less semantic understanding due to the semantic incoherence of the input text.

[0016] To address the aforementioned issues, this application provides a text generation method. After obtaining a request text sequence including non-forbidden words and forbidden words carrying forbidden language markers, the request text sequence is input into a first model for text generation to obtain a response text. In the aforementioned text generation process, the text generation method provided in this application uses different generation weights for forbidden and non-forbidden words. This not only processes the forbidden parts of the input text when they exist, thus preventing the response text generated by the first model from carrying forbidden content, but also completely preserves the content of the input text, avoiding the problem of semantic incoherence. This allows the first model to learn complete text expression and semantics, thereby generating a more accurate response text, improving the accuracy of text generation. This solves the technical problem that existing text generation methods have limitations in handling abnormal text in the model's input text, leading to low accuracy in the generated text.

[0017] Optionally, in this embodiment, the text generation method can be applied to large language models in various text generation scenarios, enabling the large language model to quickly and accurately generate automatic dialogue and response texts. For example, in an online customer service scenario, the above text generation method can be applied to a large language model so that, when the input text contains illegal text, it can learn the semantics of the input text while completely preserving it, and avoid generating response texts containing illegal texts similar to or identical to those in the input text. This allows it to output response content without illegal content while fully learning the semantics of the user's input text, thus improving the user experience. As another example, in an online psychological counseling scenario, when the user's input text contains overly emotional content, the above text generation method can be applied to a large language model so that, while learning the complete semantics of the input text to accurately learn the user's emotions or psychological state, the large language model can avoid generating response texts containing overly emotional or negative content, improving the effectiveness and accuracy of psychological counseling.

[0018] In the embodiments of this application, the above-described text generation method can be applied to, but is not limited to, methods such as... Figure 1 The text generation system in the hardware environment shown may include, but is not limited to, terminal device 102 and text generation server 104. As shown in steps S102 to S108, after the terminal device sends a request text sequence to the text generation server, the obtained request text sequence is input into the first model for text generation. Forbidden and non-forbidden words in the request text sequence are assigned first and second generation weights respectively to obtain a response text that has semantic association with both forbidden and non-forbidden words. Subsequently, the text generation server sends the response text to the terminal device so that the terminal device can display the response text to the user.

[0019] Optionally, in this embodiment, the aforementioned terminal device may be a terminal device configured with a first client, which may include, but is not limited to, at least one of the following: mobile phone (such as Android phone, iOS phone, etc.), laptop computer, tablet computer, PDA, MID (Mobile Internet Devices), PAD, desktop computer, smart TV, etc. The first client may be a video client, instant messaging client, browser client, educational client, etc. The aforementioned server may be a single server, a server cluster composed of multiple servers, or a cloud server. The above is merely an example, and no limitation is made in this embodiment.

[0020] As an optional implementation method, such as Figure 2 As shown, the above text generation method includes: S202, Obtain the request text sequence, wherein the request text sequence includes non-forbidden word elements and forbidden word elements carrying forbidden word markers; Optionally, in this embodiment, the above-mentioned request text sequence may be, but is not limited to, a lexical sequence generated based on the request text. Here, the request text may be, but is not limited to, the text instruction input by the user to guide the large language model to generate specific content. Here, the lexical may be, but is not limited to, the basic unit that divides the text into text fragments during the text processing.

[0021] Optionally, in this embodiment, the aforementioned forbidden word marker may be, but is not limited to, an identifier used to mark forbidden word elements. The aforementioned forbidden word elements may be word elements generated after processing forbidden text in the request text, and the aforementioned non-forbidden word elements may be, but are not limited to, word elements generated after processing text in the request text that does not contain forbidden parts. Here, forbidden words may refer to prohibited text, such as text with excessive emotional intensity, text containing malicious attack content, text containing false information, etc.

[0022] S204, the request text sequence is input into the first model for text generation to obtain the response text. In the first model, a first generation weight is applied to forbidden words and a second generation weight is applied to non-forbidden words. The first generation weight is less than or equal to a preset threshold. The response text has semantic association with both forbidden and non-forbidden words.

[0023] Optionally, in this embodiment, the first model mentioned above may refer to, but is not limited to, a model that can learn the patterns and structures of natural language and generate coherent and relevant natural language response text based on the text input by the user. This model can be widely applied in multiple fields such as text generation, translation, and question answering. The text generation process mentioned above may refer to, but is not limited to, the process by which the first model learns the expression methods of each word in the request text sequence and the semantics between each word, thereby generating response text similar to the expression methods of each word in the request text sequence.

[0024] Optionally, in this embodiment, the above-mentioned response text may be, but is not limited to, text content generated by the first model after learning the semantics of forbidden and non-forbidden words in the request text sequence. This text content is related to the needs described by the user's input request text and is semantically coherent. It can be used to provide users with useful information, answer questions, or complete specific language tasks.

[0025] Optionally, in this embodiment, the aforementioned generation weight may be, but is not limited to, the weight assigned to each word in the request text sequence, indicating the probability value of each word appearing in the response text sequence; the response text sequence here may be, but is not limited to, the word sequence generated by the first model, and the text obtained after converting the words in the response text sequence into text format is the aforementioned response text. Optionally, the aforementioned first generation weight and the aforementioned second generation weight may be different, and the aforementioned first generation weight may be less than the second generation weight.

[0026] Optionally, in this embodiment, the semantic association can refer to the semantic connection between each word in the request text sequence and each word in the response text sequence. This connection can take various forms such as synonym, antonym, hierarchical relationship, causal relationship, and parallel relationship, to reflect the interaction and connection of language units at the semantic level.

[0027] The embodiments provided in this application obtain a request text sequence including non-forbidden words and forbidden words carrying forbidden word markers. This request text sequence is then input into a first model. Different generation weights are applied to forbidden and non-forbidden words to generate text, resulting in response text that has semantic relevance to both forbidden and non-forbidden words. By configuring different generation weights for abnormal and normal text in the input text of the text generation model, the content of abnormal text in the model's output text is reduced without changing the input text. This not only preserves the complete text content of the input text, thus avoiding content loss or semantic omission that leads to poor accuracy in the model's output text, but also allows for the processing of abnormal text in the input text through the differentiated use of generation weights. This enables the model to fully recognize the input text while avoiding the generation of abnormal text, thereby improving the accuracy of the generated text. This solves the technical problem that existing text generation methods have limitations in handling abnormal text in the model's input text, resulting in low accuracy of the generated text.

[0028] As an optional implementation, the above text generation includes: Determine the generation weight for each term in the request text sequence; The determination of generation weights includes: in response to identifying a word as a forbidden word carrying a forbidden word marker, configuring a first generation weight for the forbidden word; in response to identifying a word as a non-forbidden word, configuring a second generation weight for the non-forbidden word. For each word, a weighted sum is calculated according to the determined generation weight to obtain the response text that matches the request text sequence.

[0029] Optionally, in this embodiment, the weighted summation described above may be, but is not limited to, a process of weighting and summing each word in the request text sequence according to the generation weight of each word in the request text sequence to obtain a new word sequence.

[0030] Optionally, the response text may, but is not limited to, be the text content obtained by combining the lexical units in the new lexical unit sequence with the lexical units in the response text sequence generated by the first model and converting them into text format. During text generation, the generation weight of forbidden word units may, but is not limited to, be configured below a low threshold (e.g., 0.01, 0.05, etc.) to reduce the probability of forbidden content appearing in the response text, thereby improving the accuracy of the response text. For example, the request text entered by the user may be: "I uploaded a report today, why haven't I received feedback yet?" The first generation weight of the forbidden word unit corresponding to the forbidden phrase "why haven't I received feedback yet" in the request text can be set to a low value to prevent the first model from including the aforementioned forbidden content in the request text when generating the response text.

[0031] The embodiments provided in this application enable semantic association between the response text generated by the first model and the words in the request text sequence by configuring different generation weights for the words in the request text sequence, thereby avoiding the situation where the response text contains prohibited content in the request text, and improving the accuracy of text generation and user experience.

[0032] As an optional implementation method, such as Figure 3 As shown, before obtaining the request text sequence, the process also includes: S302, the obtained sample text is segmented to obtain sample text phrases, wherein the sample text includes a first sample text carrying a forbidden word tag and a second sample text not carrying a forbidden word tag; Optionally, in this embodiment, the above-mentioned word segmentation processing may refer to the process of dividing the sample text into multiple independent word units or substrings according to certain rules. Further, the above-mentioned sample text phrases may be the result of word segmentation processing of the sample text, including multiple text fragments obtained after segmenting the sample text.

[0033] Optionally, in this embodiment, the segmentation operation may be to segment only the text portion of the sample text, or it may be to segment both the text portion and the punctuation portion of the sample text. For example, as Figure 4 As shown, the request text is: "I want to apply for a credit line of 100,000 yuan. What procedures do I need to go through?". The text phrase obtained by segmenting the text portion of the request text is: "I / want / to / apply / for / a / credit line of 100,000 yuan / need / to / go / with / what / procedures". The text phrase obtained by segmenting both the text portion and the punctuation portion of the request text is: "I / want / to / apply / for / a / credit line of 100,000 yuan / , / need / to / go / with / what / procedures / ?".

[0034] S304, map sample text phrases to sample text sequences; Optionally, in this embodiment, the above mapping process may refer to, but is not limited to, the process of mapping each text segment in the sample text phrase to the corresponding codeword ID in the codebook. Wherein, such as Figure 5 As shown, a codebook can be a predefined mapping table used to map tokens (such as words, characters, etc.) in text to unique identifiers, facilitating the conversion of text into numerical format. Codeword IDs can be, but are not limited to, identifiers used in the mapping table to indicate individual codewords. Each codeword corresponds to a codeword ID, and different codewords can have different codeword IDs. A codeword can refer to a token or symbol mapped in the codebook; it is the basic unit that needs to be encoded in text data. For example, in the above request text phrase “I / want / apply / for / a / credit / limit / of / ten thousand yuan / need / to / do / what / procedures”, “I”, “want”, “ten thousand yuan”, and “credit / limit” are all codewords. It can be understood that the multiple sample tokens included in the above sample text sequence can be the codewords corresponding to the multiple codeword IDs obtained after mapping the text fragments in the sample text phrase.

[0035] S306, The initial model is trained based on the sample text sequence to obtain the first model; wherein, the first weight combination corresponding to multiple first sample word units obtained based on the first sample text mapping is different from the second weight combination corresponding to multiple second sample word units obtained based on the second sample text mapping.

[0036] Optionally, in this embodiment, the aforementioned initial model may refer to, but is not limited to, a large language model capable of learning patterns and structures of natural language and generating coherent and relevant natural language response text based on user-inputted text. This model can be widely applied in multiple fields such as text generation, translation, and question answering. For example, the model structure of the initial model may be as follows: Figure 6 As shown, the initial model includes: an input layer 602, a self-attention layer 604, a feedforward neural network layer 806, and an output layer 808. The input layer receives user sample text sequences and normalizes the format of each word in the sequence. The self-attention layer calculates the attention score between each word and other words in the normalized text sequence, and performs a weighted sum of these scores to obtain the global representation of each word, thus capturing global dependencies. The feedforward neural network layer uses an activation function to perform a non-linear transformation on the global representation of each word, increasing the model's expressive power and enabling it to capture more complex semantic information. The output layer maps the output of the feedforward neural network layer to the target space to obtain the text content generated by the model.

[0037] Optionally, the above model training process may be, but is not limited to, training an initial model until the similarity between the response text generated by the initial model and the sample response text corresponding to the sample text reaches a threshold. This similarity can be represented by a loss function value. Optionally, in this embodiment, the weights corresponding to the first and second sample words may be, but are not limited to, the weights of each sample word when calculating the loss function value.

[0038] It is understandable that, during the text generation process, before using the first model to generate text, the process of obtaining the request text sequence can be achieved through word segmentation and mapping operations as described in steps S302 and S304 above. That is, each word element in the request text sequence can be obtained by segmenting the user-input request text to obtain request text phrases containing multiple text segments, and mapping each text segment in the request text phrases to codewords in a codebook. Here, the request text sequence includes codewords mapped from each text segment in the request text phrases. It is also understandable that the user-input request text can undergo pre-processing forbidden language identification and labeling to obtain forbidden request texts containing forbidden language labels and non-forbidden language request texts without forbidden language labels. Furthermore, among the word elements included in the request text sequence, non-forbidden language word elements can be codewords obtained by mapping forbidden language text segments in the request text phrases, and forbidden language word elements carrying forbidden language labels can be codewords obtained by mapping non-forbidden language text segments in the request text phrases.

[0039] The embodiments provided in this application pre-process the sample text with word segmentation and mapping to obtain a sample text sequence, which improves the data processing efficiency of the initial model and thus improves the model training efficiency. During the model training process, different weights are configured for different words in the sample text sequence to process forbidden words in the sample text without deleting forbidden content. This allows the trained first model to not only learn the complete semantics of the sample text and generate text with semantic association with the sample text, but also to avoid the generated response text from containing forbidden content, thus improving the accuracy of the model's text generation.

[0040] As an optional implementation method, such as Figure 7 As shown, before training the initial model based on the sample text sequence to obtain the first model, the following steps are also included: S702, Identify sample forbidden word elements carrying forbidden word tags from multiple first sample word elements; Optionally, in this embodiment, the sample forbidden word units mentioned above can be word units mapped from forbidden words. For example, the sample text mentioned above is a dialogue text, including: the text entered by the user: "I failed the exam again, why is studying so difficult!"; and the reply text generated by the online customer service: "Don't be discouraged, learning is a process of continuous effort. Do you need me to generate an error analysis for you?" In the above sample text, "I failed the exam again, why is studying so difficult!" is the first sample text, and "Don't be discouraged, learning is a process of continuous effort. Do you need me to generate an error analysis for you?" is the second sample text. Further, in the first sample word units mapped from the first sample text mentioned above, the word units mapped from the forbidden text fragment "why is it so difficult!" are sample forbidden word units carrying forbidden word tags.

[0041] S704, assign a first weight to the sample forbidden word element, and assign a second weight to the sample word elements other than the sample forbidden word element among the multiple first sample word elements, wherein the first weight is greater than a first preset threshold, and the first weight and the second weight constitute a first weight combination.

[0042] Optionally, in this embodiment, the first weight and the second weight mentioned above can be, but are not limited to, the weights corresponding to each sample word when calculating the loss function. Here, the loss function can be used to calculate the similarity between the text generated by the initial model and the reply text generated by the online customer service in the sample text. The training convergence condition for training the initial model to obtain the first model can be, but is not limited to, the function value of the loss function being greater than a preset threshold. As an optional scheme, the loss function can be as shown in formula (1): (1) Where L is the loss function, Output is the text content generated by the initial model during training, token is the word element obtained by the text mapping generated by the initial model during training, and the length of the token in Output is the total number of word elements obtained by the text mapping generated by the initial model during training. The i-th lexical unit in the text mapping generated for the initial model during training; The weights assigned to each sample word are as follows: the first sample word marked as a forbidden word is assigned a first weight of 1, and the first sample word marked as a non-forbidden word is assigned a second weight of 0.

[0043] For example, the sample text includes: the user's input text: "I applied for an account upgrade last week, when will it be completed?"; and the online customer service's generated reply text: "This service is being processed, what's the rush? We will try our best to reply to you today." Here, "what's the rush?" is marked as prohibited content, and the word elements mapped to "what's the rush?" are marked as prohibited word elements. The loss function value can then be expressed as formula (2): (2) Through the embodiments provided in this application, during model training, different weights are assigned to forbidden and non-forbidden word units in the sample word units when calculating the loss function value. This reduces the loss function value when the text generated by the initial model during training contains forbidden content. By selectively changing the loss function value, an indirect penalty mechanism is implemented on the model when the text generated by the initial model during training contains forbidden content. Under this indirect penalty mechanism, during the initial model training process, the generation weights corresponding to forbidden word units can be reduced when updating model parameters based on the loss function value, thereby indirectly penalizing the model and reducing the proportion of forbidden content in the model-generated text. During text generation, the first model assigns lower generation weights to forbidden word units in the request text sequence when generating the response text, thereby reducing the proportion of forbidden content in the response text generated by the first model and improving the accuracy of the response text.

[0044] As an optional implementation, before training the initial model based on the sample text sequence to obtain the first model, the following steps are also included: S1, Identify from multiple first sample word units the sample forbidden word unit carrying the forbidden word tag, and the reference word unit located before and adjacent to the sample forbidden word unit; S2, assign a third weight to the reference word, and assign a fourth weight to the sample word other than the reference word among the multiple first sample word, wherein the third weight is less than or equal to the second preset threshold, and the third weight and the fourth weight constitute the first weight combination.

[0045] Optionally, in this embodiment, the aforementioned reference lexical unit may be one or more sample lexical units whose position in the sample lexical unit sequence precedes the aforementioned sample forbidden lexical unit. The aforementioned third weight may be the weight of the loss function value corresponding to each lexical unit when calculating the total loss function value, and the total loss function value may be the value obtained by weighting and summing the loss function values ​​corresponding to each lexical unit according to their respective weights.

[0046] Optionally, in this embodiment, the aforementioned third weight can be zero to reduce the loss function value of the first few sample words adjacent to the forbidden word word, thereby indirectly preventing the model from generating forbidden text by avoiding the generation of text content before the forbidden word. For example, the sample text includes: the user-inputted text: "I applied for an account upgrade last week, when will it be completed?"; and the online customer service's reply text: "This service is being processed, what's the rush? I'll try my best to reply to you today." Here, "what's the rush?" is marked as forbidden content, the word word mapped to "what's the rush?" is marked as a forbidden word word, and the word word adjacent to the forbidden word word and arranged before the forbidden word word word is the word word mapped to "processing". By configuring the weight of the word word word mapped to "processing" to 0, the loss function value of the word word word mapped to "processing" is 0 when calculating the total loss function value. This reduces the loss function value of the first few sample words adjacent to the forbidden word word, thereby preventing the model from generating content before the forbidden word when generating text, and thus achieving the effect of preventing the model from generating forbidden content.

[0047] The embodiments provided in this application reduce the loss function value of the first few sample words adjacent to the forbidden word word to avoid the model generating content before the forbidden word word when generating text, thereby achieving the effect of avoiding the model generating forbidden content. This enables the trained first model to reduce the generation weight of non-forbidden word words that are adjacent to the forbidden word word word and whose position in the request text sequence is before the forbidden word word word word when generating response text. In this way, the response text avoids including content before the forbidden word word, thereby improving the quality of the response text.

[0048] As an optional implementation, before training the initial model based on the sample text sequence to obtain the first model, the following steps are also included: The preset weight combination is determined as the second weight combination.

[0049] Optionally, in this embodiment, the values ​​of the second weights configured for each second sample word can be the same.

[0050] The embodiments provided in this application use preset weights for non-forbidden words in the sample words, thereby avoiding the accidental deletion or misprocessing of non-forbidden words during model training, and improving the training efficiency and accuracy of the first model.

[0051] Specific combination Figure 8 To illustrate the process of applying the above text generation method to an online dialogue scenario: S802, Obtain sample text; the request text includes user input text and agent response text.

[0052] S804, perform word segmentation on the sample text to obtain sample text phrases; perform word segmentation on the text content of the sample text to obtain sample text phrases containing multiple text fragments.

[0053] S806, map sample text phrases to sample text sequences; map each text segment in the sample text phrases to a codeword ID in a predefined codebook, where the codeword ID corresponds to the word element corresponding to each text segment in the sample text phrases.

[0054] S808: The initial model is trained using the sample text sequence to obtain the first model; the sample text sequence is used as the input of the initial model, and different weights are applied to forbidden and non-forbidden words in the sample text sequence to calculate the similarity between the output text of the model and the agent's response text in the sample text, and the calculated similarity value is used as the loss function value; if the loss function value is less than the threshold, the training convergence condition is determined to be met, and the trained model is determined as the first model.

[0055] S810, retrieve the request text.

[0056] S812, convert the request text into a request text sequence; perform word segmentation on the request text to obtain a request text phrase containing multiple request text fragments; map each text fragment in the request text phrase to a codeword ID in a predefined codebook, and the codeword corresponding to the codeword ID is the word element corresponding to each text fragment in the request text phrase.

[0057] S814, input the request text sequence into the first model to generate the response text; apply different weights to forbidden and non-forbidden words in the request text sequence, and perform a weighted summation of each word according to its weight; and fuse the weighted summation result of the words in the request text sequence with the output text of the first model to obtain the response text.

[0058] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0059] According to another aspect of the embodiments of this application, a text generation apparatus for implementing the above-described text generation method is also provided. For example... Figure 9 As shown, the device includes: The acquisition unit 902 is used to acquire a request text sequence, wherein the request text sequence includes non-forbidden word elements and forbidden word elements carrying forbidden word markers; The generation unit 904 is used to input the request text sequence into the first model to generate text and obtain the response text. In the first model, a first generation weight is applied to forbidden words and a second generation weight is applied to non-forbidden words. The first generation weight is less than or equal to a preset threshold. The response text has a semantic relationship with both forbidden and non-forbidden words.

[0060] As an optional solution, the above-mentioned generation unit 904 includes: The determination module is used to determine the generation weight for each word element contained in the request text sequence; wherein, determining the generation weight includes: in response to identifying a word element as a forbidden word element carrying a forbidden word marker, configuring a first generation weight for the forbidden word element; in response to identifying a word element as a non-forbidden word element, configuring a second output weight for the non-forbidden word element; The calculation module is used to perform weighted summation on each word according to the determined generation weight, so as to obtain the response text that matches the request text sequence.

[0061] Optionally, the embodiments in this solution may be, but are not limited to, the embodiments of the reference method, which will not be described in detail here.

[0062] As an optional solution, the above-mentioned device further includes: The word segmentation unit is used to segment the obtained sample text into word groups. The sample text includes a first sample text carrying a forbidden word tag and a second sample text that does not carry a forbidden word tag. A mapping unit is used to map sample text phrases to sample text sequences. The training unit is used to train the initial model based on the sample text sequence to obtain the first model; wherein, the first weight combination corresponding to the multiple first sample words obtained based on the first sample text mapping is different from the second weight combination corresponding to the multiple second sample words obtained based on the second sample text mapping.

[0063] Optionally, the embodiments in this solution may be, but are not limited to, the embodiments of the reference method, which will not be described in detail here.

[0064] As an optional solution, the above-mentioned device further includes: The first determining unit is used to determine the sample forbidden word character carrying the forbidden word tag from multiple first sample word characters; The first configuration unit is used to configure a first weight for sample forbidden word elements and configure a second weight for sample word elements other than sample forbidden word elements among a plurality of first sample word elements, wherein the first weight is greater than a first preset threshold and the first weight and the second weight constitute a first weight combination.

[0065] Optionally, the embodiments in this solution may be, but are not limited to, the embodiments of the reference method, which will not be described in detail here.

[0066] As an optional solution, the above-mentioned device further includes: The second determining unit is used to determine, from multiple first sample word elements, a sample forbidden word element carrying a forbidden word tag, and a reference word element located before and adjacent to the sample forbidden word element. The second configuration unit is used to configure a third weight for the reference word and a fourth weight for the sample word other than the reference word among the multiple first sample word, wherein the third weight is less than or equal to a second preset threshold, and the third weight and the fourth weight constitute a first weight combination.

[0067] Optionally, the embodiments in this solution may be, but are not limited to, the embodiments of the reference method, which will not be described in detail here.

[0068] As an optional solution, the above-mentioned device further includes: The third determining unit is used to determine the preset weight combination as the second weight combination.

[0069] Optionally, the embodiments in this solution may be, but are not limited to, the embodiments of the reference method, which will not be described in detail here.

[0070] According to one aspect of this application, a computer-readable storage medium is provided, from which a processor of a computer device reads computer instructions, and the processor executes the computer instructions, causing the computer device to perform the methods provided in various alternative implementations of the above-described text generation method.

[0071] Optionally, in this embodiment, the computer-readable storage medium may be configured to store a computer program for performing the following steps: S1, obtain the request text sequence, wherein the request text sequence includes non-forbidden word elements and forbidden word elements carrying forbidden word markers; S2, input the request text sequence into the first model to generate text and obtain the response text. In the first model, the forbidden word elements are given a first generation weight and the non-forbidden word elements are given a second generation weight. The first generation weight is less than or equal to a preset threshold. The response text has a semantic relationship with both the forbidden word elements and the non-forbidden word elements.

[0072] According to one aspect of this application, a computer program product is provided, comprising a computer program / instructions containing program code for performing the text generation method described above.

[0073] According to another aspect of the embodiments of this application, an electronic device for implementing the above-described text generation method is also provided, the electronic device being... Figure 1 The terminal device or server shown. This embodiment uses the electronic device as a server as an example for illustration. Figure 10 As shown, the electronic device includes a memory 1002 and a processor 1004. The memory 1002 stores a computer program, and the processor 1004 is configured to execute the steps of any of the above method embodiments via the computer program.

[0074] Optionally, in this embodiment, the aforementioned electronic device may be located in at least one of a plurality of network devices in a computer network.

[0075] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program: S1, obtain the request text sequence, wherein the request text sequence includes non-forbidden word elements and forbidden word elements carrying forbidden word markers; S2, input the request text sequence into the first model to generate text and obtain the response text. In the first model, the forbidden word elements are given a first generation weight and the non-forbidden word elements are given a second generation weight. The first generation weight is less than or equal to a preset threshold. The response text has a semantic relationship with both the forbidden word elements and the non-forbidden word elements.

[0076] Alternatively, as those skilled in the art will understand, Figure 10 The structure shown is for illustrative purposes only. Electronic devices can also be smartphones (such as Android phones, iOS phones, etc.), tablets, PDAs, mobile internet devices (MIDs), PADs, and other terminal devices. Figure 10 This does not limit the structure of the aforementioned electronic devices. For example, the electronic device may also include components that are more... Figure 10 The more or fewer components shown (such as network interfaces, etc.), or having the same Figure 10 The different configurations shown.

[0077] The memory 1002 can be used to store software programs and modules, such as the program instructions / modules corresponding to the text generation method and apparatus in this embodiment. The processor 1004 executes various functional applications and data processing by running the software programs and modules stored in the memory 1002, thereby implementing the aforementioned text generation method. The memory 1002 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 1002 may further include memory remotely located relative to the processor 1004, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. Specifically, the memory 1002 may be used, but is not limited to, to store the aforementioned request text sequence. As an example, such as... Figure 10 As shown, the memory 1002 may include, but is not limited to, the acquisition unit 902 and the generation unit 904 in the text generation device. Furthermore, the memory 1002 may also include, but is not limited to, other module units in the text generation device, which will not be elaborated upon in this example.

[0078] Optionally, the transmission device 1006 described above is used to receive or send data via a network. Specific examples of the network described above may include wired networks and wireless networks. In one example, the transmission device 1006 includes a Network Interface Controller (NIC), which can be connected to other network devices and a router via a network cable to communicate with the Internet or a local area network. In another example, the transmission device 1006 is a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0079] In addition, the aforementioned electronic device also includes: a display 1008 for displaying the aforementioned response text; and a connection bus 1010 for connecting the various module components in the aforementioned electronic device.

[0080] In other embodiments, the aforementioned terminal device or server can be a node in a distributed system, wherein the distributed system can be a blockchain system, which is a distributed system formed by connecting multiple nodes through network communication. The nodes can form a point-to-point network, and any form of computing device, such as a server, terminal, or other electronic device, can become a node in the blockchain system by joining this point-to-point network.

[0081] Optionally, in embodiments of this application, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0082] Optionally, in this embodiment, those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0083] If the integrated units in the above embodiments are implemented as software functional units and sold or used as independent products, they can be stored in the aforementioned computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause one or more computer devices (which may be personal computers, servers, or network devices, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.

[0084] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0085] In the several embodiments provided in this application, it should be understood that the disclosed client can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between units or modules, and may be electrical or other forms.

[0086] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0087] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0088] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A text generation method, characterized in that, The method includes: Obtain a request text sequence, wherein the request text sequence includes non-forbidden word elements and forbidden word elements carrying forbidden word markers; The request text sequence is input into a first model for text generation to obtain a response text. In the first model, a first generation weight is applied to the forbidden word elements, and a second generation weight is applied to the non-forbidden word elements. The first generation weight is less than or equal to a preset threshold. The response text has a semantic relationship with both the forbidden word elements and the non-forbidden word elements.

2. The method according to claim 1, characterized in that, The text generation includes: A generation weight is determined for each term in the request text sequence; The determination of generation weights includes: in response to identifying the word as a forbidden word carrying the forbidden word marker, configuring the first generation weight for the forbidden word; in response to identifying the word as a non-forbidden word, configuring the second generation weight for the non-forbidden word. Each lexical unit is weighted and summed according to the determined generation weight to obtain the response text that matches the request text sequence.

3. The method according to claim 1, characterized in that, Before obtaining the requested text sequence, the following is also included: The obtained sample text is segmented to obtain sample text phrases, wherein the sample text includes a first sample text carrying a forbidden word tag and a second sample text not carrying the forbidden word tag; Map the sample text phrases to sample text sequences; The initial model is trained based on the sample text sequence to obtain the first model; The first weight combination corresponding to the plurality of first sample words obtained based on the first sample text mapping is different from the second weight combination corresponding to the plurality of second sample words obtained based on the second sample text mapping.

4. The method according to claim 3, characterized in that, Before training the initial model based on the sample text sequence to obtain the first model, the method further includes: From the plurality of first sample word units, determine the sample forbidden word units carrying the forbidden word tag; A first weight is assigned to the sample forbidden word unit, and a second weight is assigned to the sample word units other than the sample forbidden word units among the plurality of first sample word units, wherein the first weight is greater than a first preset threshold, and the first weight and the second weight constitute the first weight combination.

5. The method according to claim 3, characterized in that, Before training the initial model based on the sample text sequence to obtain the first model, the method further includes: From the plurality of first sample word units, a sample forbidden word unit carrying a forbidden word tag is determined, as well as a reference word unit located before and adjacent to the sample forbidden word unit. A third weight is configured for the reference word, and a fourth weight is configured for the sample words other than the reference word among the plurality of first sample words, wherein the third weight is less than or equal to a second preset threshold, and the third weight and the fourth weight constitute the first weight combination.

6. The method according to claim 3, characterized in that, Before training the initial model based on the sample text sequence to obtain the first model, the method further includes: The preset weight combination is determined as the second weight combination.

7. A text generation device, characterized in that, include: An acquisition unit is used to acquire a request text sequence, wherein the request text sequence includes non-forbidden word elements and forbidden word elements carrying forbidden word markers; The generation unit is used to input the request text sequence into a first model to generate text and obtain a response text. In the first model, a first generation weight is applied to the forbidden word elements and a second generation weight is applied to the non-forbidden word elements. The first generation weight is less than or equal to a preset threshold. The response text has a semantic relationship with both the semantics of the forbidden word elements and the semantics of the non-forbidden word elements.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein the program is executed by a processor to perform the method described in any one of claims 1 to 6.

9. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method according to any one of claims 1 to 6.

10. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to execute the method described in any one of claims 1 to 6 through the computer program.

Images