Spoken language text generation method and device, equipment and storage medium

Abstract

The application relates to the technical field of artificial intelligence and the field of financial technology, and discloses a spoken language text generation method, which comprises the following steps: written text is divided into multiple sentences, and a pre-trained language model is used to extract the text features of each sentence; based on a context attention mechanism, the text features of each sentence are combined with spoken language instructions to generate context tokens; the context tokens and the text features of each sentence are projected to a unified feature space through a linear projector to generate content tokens; the text features of each sentence are input into a self-recurrent language model, and the content tokens are combined to generate spoken language text; through the context attention mechanism, the application can more effectively combine the correlation and semantic information between sentences, and the naturalness and fluency of the spoken language text are improved; the text features and the context tokens are processed through the linear projector, so that the coherence and consistency of the generated text are ensured; the problem of incoherent text generation is overcome, and the naturalness and accuracy of the spoken language text are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence and the technical field of financial technology, and in particular to a spoken language text generation method and device, equipment and a storage medium. BACKGROUND

[0002] With the wide application of intelligent customer service and dialogue generation technology in the financial field, especially in customer service and risk consultation scenarios, spoken language transcription systems have gradually become a key tool for financial institutions to optimize user experience and improve service efficiency. However, the existing spoken language transcription technology still has many shortcomings in dealing with the complexity and diversity of the financial field.

[0003] The existing spoken language transcription system has high computational complexity when processing long text dialogues, especially in the aspect of context understanding. In the financial dialogue scenario, customers often ask multi-level and multi-background questions, and the system needs to accurately understand the user's intention and generate corresponding replies based on the context information. However, the existing technology lacks deep understanding of context information in the generation process, which easily leads to context disconnection and incoherent dialogue text, reducing the quality of customer interaction.

[0004] At the same time, the existing spoken language transcription system performs poorly when processing multi-modal data (such as text and audio), and cannot fully utilize the tone, pause and other information in the audio data to enhance the naturalness and context coherence of the text. In the financial field, customer communication often involves voice characteristics such as emotional expression and tone changes, which are crucial for simulating natural spoken language. However, the existing technology cannot flexibly insert simulated spoken language sound effects (such as laughter, coughing, etc.) when fusing text and audio, and lacks precision in generating and controlling the intensity of these sound effects, affecting the naturalness of the final generated text and customer experience.

[0005] Moreover, these models often cannot naturally insert symbols that simulate spoken language features (such as laughter, coughing, tone fillers, etc.) during text generation, which makes the generated text appear stiff when simulating natural spoken language. SUMMARY

[0006] The main purpose of the present application is to provide a spoken language text generation method, device, equipment and storage medium, which aims to solve the technical problem that the existing technology lacks deep understanding of context information when processing spoken language text generation, resulting in incoherent and unnatural dialogue text in long text or complex context.

[0007] To achieve the above purpose, the present application provides a spoken language text generation method, comprising:

[0008] segmenting the written text into multiple sentences, and extracting text features of each sentence using a pre-trained language model;

[0009] generating context tokens based on a context attention mechanism and in combination with the text features of each sentence and the oralization instruction;

[0010] projecting the context tokens and the text features of each sentence to a unified feature space through a linear projector to generate content tokens;

[0011] inputting the text features of each sentence into a self-recurrent language model in combination with the content tokens to generate oralized text.

[0012] Further, to achieve the above object, the present application provides a kind of oralized text generation device, comprising:

[0013] feature extraction module, for written text is segmented into multiple sentences, and the text features of each sentence are extracted using a pre-trained language model;

[0014] context attention mechanism module, for generating context tokens based on a context attention mechanism and in combination with the text features of each sentence and the oralization instruction;

[0015] feature projection module, for projecting the context tokens and the text features of each sentence to a unified feature space through a linear projector to generate content tokens;

[0016] oralized text generation module, for inputting the text features of each sentence into a self-recurrent language model in combination with the content tokens to generate oralized text.

[0017] Further, to achieve the above object, the present application also provides a kind of computer equipment, the computer equipment includes memory, processor and stored on the memory and can be run on the processor Oralized text generation program, the oralized text generation program is implemented when the processor executes the steps of oralized text generation method as described above.

[0018] Further, to achieve the above object, the present application also provides a kind of computer readable storage medium, the storage medium is stored with oralized text generation program, the oralized text generation program is implemented when the processor executes the steps of oralized text generation method as described above.

[0019] Beneficial effects: The application relates to the fields of artificial intelligence technology and financial technology, and discloses a spoken text generation method, which comprises the following steps: written text is divided into multiple sentences, and a pre-trained language model is used to extract the text features of each sentence. Based on a context attention mechanism, the text features of each sentence are combined with spoken instructions to generate context tokens. The context tokens and the text features of each sentence are projected to a unified feature space through a linear projector to generate content tokens. The text features of each sentence are input into a self-recurrent language model, and the content tokens are combined to generate spoken text. The application can more effectively combine the association and semantic information between sentences through the context attention mechanism, and the naturalness and fluency of the spoken text are improved. The text features and the context tokens are processed through the linear projector, so that the coherence and consistency of the generated text are ensured. The self-recurrent language model gradually generates text output, so that the text expression is more in line with the natural language expression rules and can process context information in complex dialogue scenarios. The problems of lack of context understanding and incoherent generation in the prior art are effectively solved, the naturalness and accuracy of the generated text are improved, and the method is suitable for multi-scene dialogue generation and voice assistant systems. BRIEF DESCRIPTION OF DRAWINGS

[0020] The application will be further described below in combination with the drawings and embodiments, and the drawings are as follows:

[0021] Figure 1 An application environment diagram of the spoken text generation method in an embodiment of the application;

[0022] Figure 2 A flow diagram of the spoken text generation method in an embodiment of the application;

[0023] Figure 3 A functional module diagram of the preferred embodiment of the spoken text generation device of the application;

[0024] Figure 4 A structure diagram of the computer device in an embodiment of the application;

[0025] Figure 5 Another structure diagram of the computer device in an embodiment of the application. DETAILED DESCRIPTION

[0026] It should be understood that the specific embodiments described herein are only used to explain the application, and are not used to limit the application.

[0027] The spoken text generation method provided by the embodiments of the application can be applied to, for example, Figure 1In an application environment of the present application, a user terminal communicates with a server terminal through a network. The server terminal can segment written text into multiple sentences through the user terminal, and extract text features of each sentence using a pre-trained language model. Based on a context attention mechanism, the text features of each sentence are combined with spoken instructions to generate context tokens. The context tokens and the text features of each sentence are projected to a unified feature space by a linear projector to generate content tokens. The text features of each sentence are input into a self-recurrent language model, and the content tokens are combined to generate spoken text. The present application can more effectively combine the association and semantic information between sentences through the context attention mechanism, improving the naturalness and fluency of the spoken text. The text features and context tokens are processed by the linear projector to ensure the coherence and consistency of the generated text. The self-recurrent language model gradually generates text output, making the text expression more consistent with the natural language expression rules and capable of processing context information in complex dialogue scenarios. The present application effectively solves the problems of lack of context understanding and incoherent generation in the prior art, improves the naturalness and accuracy of the generated text, and is suitable for multi-scene dialogue generation and voice assistant systems. The user terminal can be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers, and portable wearable devices. The server terminal can be implemented by an independent server or a server cluster composed of multiple servers. The present application will be described in detail below through specific embodiments.

[0028] Please refer to Figure 2 , Figure 2 The flowchart of an embodiment of the spoken text generation method provided by the present application is shown in the figure. It should be noted that although the logical order is shown in the flowchart, in some cases, the steps shown or described can be performed in an order different from that shown here.

[0029] As Figure 2 shown, the spoken text generation method provided by the present application includes the following steps:

[0030] S10, segmenting written text into multiple sentences and extracting text features of each sentence using a pre-trained language model;

[0031] In this embodiment, segmenting written text into multiple sentences is a preprocessing step. Text segmentation is usually based on the grammatical rules of the language, especially punctuation marks (such as periods, question marks, exclamation marks, etc.) and line breaks. This step needs to ensure that each sentence in the text can independently extract its semantic features, so the accuracy of segmentation directly affects the subsequent feature extraction.

[0032] Text segmentation can be achieved through regular expression matching of punctuation marks or through automatic segmentation using natural language processing tools such as NLP toolkits. The tools can automatically identify the boundaries of sentences and take into account the complexities of the language, such as certain punctuation marks that may appear inside a sentence, thus avoiding incorrect segmentation.

[0033] The segmented sentences need to be standardized. This includes removing unnecessary white spaces, tabs, and special characters such as line breaks, and normalizing punctuation marks such as different forms of quotation marks into one format. The purpose of preprocessing is to ensure that the data input into the model is clean and standardized, avoiding the model being affected by invalid characters. Usually, it relies on text cleaning tools or custom scripts, which can delete unnecessary characters through regular expressions or other string processing functions to ensure the normalization of text input.

[0034] The preprocessed sentences are input into a pre-trained language model, which extracts the text features of the sentences through context modeling and semantic representation. Pre-trained models such as BERT and GPT have been trained on large datasets and can automatically capture semantic features, grammatical structures, and contextual information in sentences. Each sentence can be converted into a format acceptable to the model, usually through a tokenizer to convert the sentence into a series of word vectors or character vectors. After inputting into the model, the model performs semantic calculation and outputs feature vectors. These feature vectors represent the semantic information of the sentence in high-dimensional space.

[0035] Through the processing of the pre-trained language model, the feature vectors of each sentence are extracted. These feature vectors include semantic, grammatical, and contextual information, providing the basis for subsequent generation of colloquial text. These vectors are comprehensive representations of each sentence, capturing its core semantic features. The model outputs a vector representation of each sentence, usually in the form of a high-dimensional vector. These vectors can be extracted through a specific interface and stored as input for subsequent processing steps.

[0036] For example, through the pre-trained language model, the text features of each sentence are extracted, as follows:

[0037] T i = LLM(S i )

[0038] where LLM represents the pre-trained language model, S i represents the i-th input sentence, and T i represents the text features extracted by the pre-trained language model.

[0039] Example: In a customer service system in the financial field, when the customer inputs a written query text such as "I want to know the interest rate of the loan", the system will first segment the text into multiple sentences according to punctuation and language logic, such as "I want to know" and "the interest rate of the loan". Then, the system will input these sentences into a pre-trained language model (such as the BERT model). The pre-trained language model will extract the text features of each sentence, including the semantics, syntax information, and context association of the sentence.

[0040] For example, after processing the sentence "the interest rate of the loan" through the model, the extracted feature vector can reflect the professional nature of the sentence in the financial field, and combine with the previous sentence "I want to know", so as to help the system more accurately understand the customer's query requirements.

[0041] By segmenting the written text into multiple sentences and using a pre-trained language model to extract the text features of each sentence, the independence of the sentences is ensured, and the semantic features and context association information of the sentences are obtained through the pre-trained model, which improves the naturalness and context coherence of the generated text.

[0042] S20, based on the context attention mechanism, combining the text features of each sentence and the oral instruction, generating context tokens;

[0043] In this embodiment, the context attention mechanism is a technology for processing the association information between sentences in the text. By calculating the relevance of each sentence to other sentences, context association information is generated. In this step, the attention mechanism analyzes the text features of each sentence, combines the context information of the previous and next sentences, and ensures that the generated context tokens accurately reflect the context of the sentence in the entire dialogue. Multi-head attention mechanism can be used to calculate the attention scores between query matrix, key matrix and value matrix. The query matrix is generated from the text features of the current sentence, and the key and value matrices are generated from the text features of other sentences. The context information is obtained by weighted summation of the weights.

[0044] Oral instruction refers to the tone, emotion or style requirements of the system when generating oral text, such as converting text into more natural oral expression. These instructions will be embedded in the context attention mechanism to help the system add tone or semantic features when calculating the association between sentences, so that the generated context tokens not only contain the logical relationship between sentences, but also reflect the oral features. Oral instructions can be encoded into specific vectors and input into the attention mechanism along with text features. The model adjusts the attention weights through these instructions to generate context tokens that conform to the oral style.

[0045] The context token is generated by the context attention mechanism to represent the contextual information of each sentence. It integrates the dependency between sentences and the influence of the colloquial instruction to guide the contextual coherence when generating natural language text. The generation of the context token depends on the calculation results of the attention mechanism. The system will combine the text features and colloquial instructions to generate context tokens through weighted calculation. Each token is a high-dimensional representation of the context and style of the current sentence.

[0046] Example: In a customer service system in the financial field, when the customer inputs a question such as "What is the interest rate of the loan?", the system first analyzes the context of the customer's question by combining the previous dialogue background (such as the previous loan application steps) through the context attention mechanism. Then, the system generates a context token that conforms to the dialogue style according to the text features of the current sentence and the colloquial instruction input by the user.

[0047] For example, the user may also ask "Is the process complicated?" When generating an answer, the system not only needs to combine the current question, but also needs to insert features that simulate colloquial language (such as a relaxed tone, necessary pauses, etc.). The context token helps the system ensure that the generated answer is natural and fluent, such as: "Actually, it's not complicated. You just need to submit some basic information and you can complete the application soon." This dialogue generation not only accurately answers the user's question, but also makes the reply closer to natural dialogue through the context attention mechanism and colloquial instruction.

[0048] For example, the context token is generated by combining the user's instruction using the context attention mechanism. The context token aggregates the timing information of the text and the user's instruction, and the formula is as follows:

[0049] C i = Attention(Q, K, V); where Q, K, and V represent the query, key, and value matrices, respectively, and the context token C is calculated through the attention mechanism i .

[0050] Through the context attention mechanism, the context token is generated by combining the text features of each sentence and the colloquial instruction, so that the system can generate more natural and coherent colloquial text. The generation of the context token not only reflects the semantic association between sentences, but also adjusts the style and tone of the output according to the colloquial instruction, improving the naturalness and accuracy of the dialogue.

[0051] S30, projecting the context token and the text features of each sentence into a unified feature space through a linear projector to generate a content token;

[0052] In this embodiment, the context token represents the contextual information of each sentence, while the text feature contains the semantic and syntactic information of each sentence. In this step, the context token and the text feature are input into a linear projector, aiming to project these two kinds of information into the same feature space for fusion processing. The context token and the text feature can be encoded into high-dimensional vectors respectively and input into the linear projector for projection operation. The linear projector maps these two features into the same feature space by calculating the weight matrix.

[0053] The role of the linear projector is to project feature vectors of different dimensions into a unified feature space, so that information from different sources can be weighted in the same space. In this step, the linear projector projects the context token and the text feature respectively, so that they can be compared and fused in the same dimension. The projector completes the projection process by calculating matrix multiplication. For each input context token and text feature, the linear projector maps them into the same vector space, and the calculation formula is:

[0054] Y = XW; where X is the input vector (context token or text feature), W is the projection matrix, and Y is the projected vector.

[0055] After the context token and the text feature are projected into the same feature space, the system needs to fuse them to generate content tokens that integrate the semantic information and contextual information of the sentence. The fused content token can better capture the contextual information and semantic features of the sentence, which can be used for subsequent text generation. Feature fusion is usually achieved by weighted summation or other feature fusion methods. Different weights can be assigned according to the importance of the context token and the text feature, and the two projected vectors are weighted to generate the final content token.

[0056] The content token is the final output after fusion, which integrates the context information and the text features of the sentence. The content token will be used as input for the subsequent autoregressive generation model to guide the generation of spoken language text. Through linear projection and feature fusion, the content token generated by the system is a high-dimensional vector form, which contains rich semantic and contextual information. These content tokens are used as input for the subsequent steps.

[0057] For example, use the linear projector to project the text feature and the context token into a unified space to generate the content token, as follows: i = W c · C i + W t · T i ; where C i is the context token, T i is the text feature; W c and W tProjection weight matrix of context token and text feature respectively, L i is the content token.

[0058] Example explanation: In a bank customer service scenario, suppose the customer first asks: "What documents are needed for loan application?" The system generates the context token for this sentence through the context attention mechanism, indicating the association of this sentence with the overall dialogue background. At the same time, the text features of this sentence are extracted, capturing the core semantics of "loan application" and "preparation of documents".

[0059] Subsequently, the customer asks: "What is the interest rate?" The system needs to combine the previous context and the current question, and project the context tokens and text features of the two questions into the same feature space through the linear projector. The content token is generated through feature fusion.

[0060] Text features: The feature vector of the sentence "What documents are needed for loan application?" may contain high-dimensional information related to "loan application", "preparation of documents", etc., such as: [0.4, 0.8, -0.5, 1.0], where each value represents the importance of the sentence in a particular semantic dimension.

[0061] Context information: The context token generated by the context attention mechanism may reflect the dependency between the previous and current sentences, such as the semantic association degree between the previous sentence and the current sentence being 0.6. The content of the context token may be: [0.6, -0.2, 0.7, 0.9], indicating the associated information in the context.

[0062] Fusion of content token: The content token is the fusion result of text features and context information. The fused content token may be: [0.5, 0.3, 0.1, 1.5], which indicates that the token integrates the context association of "loan application" and "interest rate", and reflects the overall background of the customer's question through weighted fusion.

[0063] Through this content token, the system generates a more coherent and natural reply. For example, when the customer asks "What is the interest rate?", the system can generate a natural answer: "The current interest rate is 4.5%, you can apply for a loan."

[0064] By projecting the context token and text feature into a unified feature space for fusion processing, the generated content token can better capture the context association information and semantic features of the sentence, improving the coherence and naturalness of text generation.

[0065] S40, input the text features of each sentence into the autoregressive language model, combine the content token, and generate a colloquial text.

[0066] In this embodiment, the text features of each sentence are input into the autoregressive language model. Autoregressive language models are good at generating text step by step by taking the output of the previous step as input for the next step. The core purpose of this step is to use the step-by-step generation mechanism of the autoregressive language model to ensure that the output of the natural language text is coherent and natural. The text features can be high-dimensional vectors extracted from pre-trained models such as BERT and GPT. When input into the autoregressive language model, the model processes these features step by step, generating partial output at each step, which is then used as input for the next step until the complete text is generated.

[0067] During the generation of natural language text, the system not only relies on the input text features, but also combines content tokens. Content tokens contain context information and semantic features. By combining content tokens, the model can adjust the generated text to not only meet the characteristics of the current sentence, but also reflect the information of the entire context, making the text more coherent and consistent with the dialogue logic. Content tokens are additional input vectors that are attached to each generation step. The autoregressive language model combines the context information of the content tokens at each time step to adjust the output text. Content tokens can be embedded into the input layer of the model or weighted using an attention mechanism.

[0068] The generation of natural language text is the output target of the autoregressive language model. Through step-by-step generation, the model combines the text features of each sentence and context information to generate complete natural language text. This process ensures that the generated text is natural and fluent, consistent with the habits of spoken language expression, and can reflect the user's input intent and context requirements. The generation process is autoregressive, with the model generating one word or phrase at a time and adjusting the output based on context and content tokens. The final result is a text sequence that meets the natural language instructions and can be used in dialogue systems, customer service systems, and other scenarios.

[0069] Example: In a bank customer service scenario, when a customer asks "What documents are required for a loan application?", the system generates a natural and fluent answer to the customer's question through the autoregressive language model combined with the content token of the loan background. The system may generate the answer: "Only an ID card and income proof are required for loan application, and the application process is very simple."

[0070] By inputting text features into the autoregressive language model and combining content tokens, natural and fluent text that meets the context information can be generated. Through step-by-step generation, the coherence and naturalness of the text are effectively improved, ensuring that the generated text not only accurately answers the customer's question, but also reflects the habits of natural language expression, making it suitable for financial, customer service, and dialogue generation scenarios.

[0071] The application relates to the technical field of artificial intelligence and the field of financial technology, and discloses a spoken language text generation method, which comprises the following steps: written text is divided into multiple sentences, and a pre-trained language model is used to extract the text features of each sentence; based on a context attention mechanism, the text features of each sentence are combined with spoken language instructions to generate context tokens; the context tokens and the text features of each sentence are projected to a unified feature space by a linear projector to generate content tokens; the text features of each sentence are input into a self-recurrent language model, and the content tokens are combined to generate spoken language text. The context attention mechanism can more effectively combine the correlation and semantic information between sentences, and the naturalness and fluency of the spoken language text are improved. The text features and the context tokens are processed by the linear projector, so that the coherence and consistency of the generated text are ensured. The self-recurrent language model gradually generates text output, so that the text expression is more in line with the natural language expression rules and can process context information in complex dialogue scenarios. The problems of lack of context understanding and incoherent generation in the prior art are effectively solved, the naturalness and accuracy of the generated text are improved, and the method is suitable for multi-scene dialogue generation and voice assistant systems.

[0072] In one embodiment, the above S40 comprises:

[0073] S401, acquiring multi-modal auxiliary data, and extracting multi-modal features from the multi-modal auxiliary data;

[0074] S402, fusing the multi-modal features with the text features of each sentence to generate fused features;

[0075] S403, inputting the fused features into a self-recurrent language model, combining the content tokens, and generating spoken language text.

[0076] In the embodiment, the multi-modal auxiliary data refers to data sources from multiple modalities, such as text, audio, video or image. When generating spoken language text, these multi-modal data can enhance the understanding of the context by the model. By combining text features and multi-modal information (such as emotional features of audio), the system can generate more rich and natural spoken language text. Multi-modal data (such as speech emotion analysis, user emotion recognition, etc.) are obtained from different data sources, and the data are processed. For text generation tasks, the most common multi-modal data include audio features (such as tone, tone) or visual information (such as facial expression). The obtained multi-modal data are standardized to ensure data consistency.

[0077] Multimodal features are important information extracted from multimodal auxiliary data. These features can include emotional changes in audio, facial expressions in images, etc. In this step, the system extracts relevant features from multimodal auxiliary data to assist the text generation task. Multimodal data is processed using specialized feature extraction algorithms, such as convolutional neural networks (CNN) for image feature extraction and Mel spectrum for audio feature extraction. The final generated feature vector can represent core information in multimodal data, such as tone, emotion, or visual feedback, etc.

[0078] By combining the features extracted from multimodal auxiliary data with text features, a fusion feature vector is generated. This fusion feature vector contains semantic information of the text and emotional, visual, etc. features of the multimodal data, which helps to generate more natural and contextually relevant text. Feature fusion is usually done through weighted summation or attention mechanism. The purpose of feature fusion is to combine text features and multimodal features in a unified feature space, making the generated content more coherent and natural. For example, through a multi-head attention mechanism, text features and audio features can be weighted and fused to generate more contextually appropriate content.

[0079] The fused feature vector is input into the autoregressive language model, which gradually generates text. By combining multimodal features and text features, the model can generate conversational text that reflects the actual context, such as inserting tone symbols, laughter, or other non-verbal features in the text generation, enhancing naturalness. The autoregressive language model uses the fused feature vector as input and generates text step by step in each generation step. In each generation step, the context information in the content token is combined to ensure the coherence and natural fluency of the output content.

[0080] For example, text features and visual features are integrated through a multimodal fusion module to generate a comprehensive feature representation, as follows:

[0081] F i = Fusion(T i , V i )

[0082] where T i represents text features, V i represents visual features, and F i represents the fused feature representation; Fusion represents the fusion mechanism, which can be implemented through various methods such as weighted summation, concatenation, or attention mechanism, etc.

[0083] Example: In a bank customer service scenario, a customer may ask a loan-related question through a voice assistant. The system will extract the customer's tone features through voice sentiment analysis and fuse them with the features of the customer's text question. For example, if the customer's tone appears anxious, the system will add more colloquial expressions with empathy and comfort when generating the text, such as: "Please don't worry, applying for a loan is very simple, and we are always here to serve you."

[0084] This embodiment combines multi-modal data, so the system can not only generate text-based colloquial text, but also adjust the generated content according to the user's audio, visual, and other multi-modal information, making the generated text more natural and closer to real-life conversations.

[0085] In one embodiment, the above 20 includes:

[0086] S201, extracting semantic information from each sentence, the semantic information being used to represent the contextual association between sentences;

[0087] S202, generating a query matrix based on the colloquial instructions;

[0088] S203, generating a key matrix and a value matrix based on the semantic information and text features of each sentence, the key matrix being used to represent the degree of association between sentences, and the value matrix being used to store the content information of the sentences;

[0089] S204, analyzing the degree of association between the query matrix and the key matrix through a context attention mechanism, and generating an attention weight for each sentence based on the degree of association;

[0090] S205, generating a context token by combining the value matrix according to the attention weight of each sentence.

[0091] In this embodiment, semantic information is extracted from each sentence, which is used to describe the meaning and role of the sentence in the context. The extracted semantic information can represent the semantic association between sentences and is the basis for generating context tokens. Each sentence is processed by a pre-trained language model (such as BERT or GPT) to extract the semantic information of the sentence. The semantic information of each sentence can be represented as a high-dimensional vector to capture the semantic features and contextual relevance of the sentence.

[0092] The query matrix is a key part of the attention mechanism, which represents the query criteria used when looking for inter-sentence associations. In the context-based attention mechanism-based text generation process, the query matrix is generated based on the text features of each sentence and the colloquial instruction, and is used to represent the query conditions when the target sentence looks for information from other sentences. The query matrix is generated by linear transformation of the semantic information of the sentence and the colloquial instruction. The text features of each sentence can be input as input, and the query matrix is generated by combining with the colloquial instruction.

[0093] The key matrix represents the degree of association between sentences, which is used to measure the relevance of other sentences to the current sentence. The value matrix stores the specific content information of the sentence, which represents the core information of each sentence. In the attention mechanism, the key and value matrices jointly determine the dependency relationship between sentences. The key matrix and the value matrix are generated by the semantic information and the text features of the sentence. The semantic information of the sentence is transformed into the key matrix by linear transformation, which is used to represent the degree of association between sentences. At the same time, the text features of the sentence are mapped into the value matrix, which stores the core content information of each sentence.

[0094] The context attention mechanism determines the relationship between each sentence by calculating the relevance between the query matrix and the key matrix. By calculating the dot product of the two matrices, the degree of association between sentences is obtained. This degree of association is used to generate attention weights, which determine the degree of influence of each sentence on other sentences. The dot product of the query matrix and the key matrix is calculated to obtain the relevance score of each sentence to other sentences. These scores are normalized by the Softmax function to generate attention weights.

[0095] According to the calculated attention weights, the content information of the value matrix is combined to generate the context token. The context token integrates the association information of each sentence with other sentences and combines the features of the colloquial instruction to generate a context representation for subsequent text generation. The context token is generated by weighted sum of the attention weights and the value matrix. The context token can represent the context of the current sentence in the entire dialogue and provide support for subsequent generation steps.

[0096] The embodiment generates the context token through the context attention mechanism, combining the text features of the sentence and the colloquial instruction, so that the system can better understand the association between sentences and ensure that the generated text is coherent and consistent with the dialogue background. The context understanding ability of the dialogue generation is improved, and the generated text is more natural and smooth.

[0097] In one embodiment, the above S30 comprises:

[0098] S301, input the context token and the text features of each sentence to a linear projector;

[0099] S302, apply a context weight matrix to the context tokens and a text weight matrix to the text features, respectively, for weighting processing;

[0100] S303, project and map the weighted context tokens and text features to a unified feature space to generate content tokens.

[0101] In this embodiment, the context tokens represent contextual association information between sentences, while the text features contain semantic and grammatical information of the sentences. In this step, the context tokens and text features are input into a linear projector, which is tasked with mapping them to the same feature space so that the two different types of features can be processed uniformly. The context tokens and text features are input into the linear projector as vectors. The linear projector unifies the context tokens and text features by matrix transformation and projects them into a high-dimensional space. Each vector is linearly transformed when input to adapt to the dimensions of the same feature space.

[0102] To ensure that the context tokens and text features can be processed according to their respective importance when the features are fused, the system applies a context weight matrix to the context tokens and a text weight matrix to the text features. Through weighting processing, the model can dynamically adjust the influence of the two types of information. The context tokens and text features are weighted respectively through the trained context weight matrix and text weight matrix. Weighting processing can be achieved through matrix multiplication, with the goal of assigning different weights to different features during the fusion process, thereby enhancing the influence of important information in the unified feature space.

[0103] The projector maps the weighted context tokens and text features to a unified feature space. In this space, the system can perform fusion processing on features from two different sources. By projecting into the same space, the feature vectors can be processed together to generate content tokens. The linear projector projects the weighted context tokens and text features into a unified high-dimensional feature space through matrix calculation. In this space, the feature vectors can be fused through weighted summation or other means.

[0104] Content tokens are the result of fusing context tokens and text features, integrating the context information and text features of the sentences. Content tokens serve as the core input for generating text in subsequent steps, ensuring that the generated text maintains the coherence of the sentence context while reflecting the semantic features of each sentence. Content tokens are generated by fusing the weighted context tokens and text features. Fusion can be achieved through weighted summation or attention mechanisms, and content tokens are finally output as vectors, containing the comprehensive features of the sentences.

[0105] By projecting the context token and the text feature into a unified feature space, the content token generated by the embodiment can effectively integrate the context information and semantic features of the sentence. This ensures that the system can reflect the independent features of each sentence and maintain the coherence of the text when generating oral text.

[0106] In one embodiment, S10 comprises:

[0107] S101, obtaining written text and dividing the written text into multiple sentences according to punctuation marks, line breaks, or language logic;

[0108] S102, inputting each sentence into a pre-trained language model, which processes the sentence based on context modeling and semantic representation;

[0109] S103, extracting a feature vector for each sentence through the pre-trained language model to obtain a text feature of each sentence, the text feature including semantic, syntactic, and contextual information of the sentence.

[0110] In the embodiment, the written text is divided into multiple sentences. The division can be based on punctuation marks (such as periods, question marks, exclamation marks, etc.), line breaks, or language-based syntactic logic. The purpose of the division is to structure the text so that each sentence can be processed independently, facilitating subsequent feature extraction. The text is divided using natural language processing tools or regular expressions. Regular expressions can identify sentence boundaries by matching punctuation marks, or can be divided according to line breaks or specific language syntax rules. Each sentence after division will be processed as an independent unit.

[0111] The pre-trained language model (such as BERT, GPT, etc.) is a deep learning model that has been trained on a large corpus and can extract semantic and contextual information of a sentence. In this step, each sentence after division is input into the pre-trained language model, which further processes the sentence based on its context modeling and semantic information. First, each sentence is tokenized and converted into a form acceptable to the model (usually word vectors or character vectors). Then these vectors are input into the pre-trained language model, which models the context and captures the semantic, syntactic structure, and contextual dependencies of the sentence.

[0112] The pre-trained language model outputs a feature vector for each sentence by processing the sentence. The feature vector includes semantic information, syntactic structure, and contextual association information of the sentence. The vector can be regarded as a high-dimensional representation of the sentence for further processing or text generation in subsequent steps. After the input sentence is processed by the language model in multiple levels, the generated feature vector is usually a high-dimensional vector (such as a BERT output 768-dimensional vector). These feature vectors reflect the main information of the sentence, including its contextual position in the entire text, syntax, and semantic features. The output feature vector can be stored as the input of the subsequent generation task.

[0113] By segmenting the written text into multiple sentences and extracting text features using a pre-trained language model, the embodiment can ensure the structured processing of the text, while taking advantage of the semantic representation capability of the language model to capture the main semantics, syntax, and context information of each sentence. This enables the system to maintain the coherence of the context and the semantic accuracy when generating natural language text.

[0114] In one embodiment, before inputting the text features of each sentence into the autoregressive language model in S40, the method further comprises:

[0115] S404, obtaining training data containing natural language text and paralanguage symbols, wherein the paralanguage symbols represent the type and intensity of non-verbal features;

[0116] S405, initializing an autoregressive language model and inputting the training data into the autoregressive language model;

[0117] S406, processing the text context and paralanguage symbols in the training data by the autoregressive language model, identifying the paralanguage symbols and corresponding intensities, and processing the non-verbal features corresponding to the paralanguage symbols;

[0118] S407, comparing the output generated by the autoregressive model with the labels in the training data to determine the error;

[0119] S408, adjusting the parameters of the autoregressive language model using the backpropagation method based on the error;

[0120] S409, gradually reducing the error through multiple rounds of training and parameter optimization to obtain a trained autoregressive language model.

[0121] In this embodiment, the training data includes not only natural language text but also paralinguistic symbols. These paralinguistic symbols are used to represent non-verbal features such as laughter, coughing, fillers, etc., which can enhance the model's ability to handle the spoken features in actual conversations. Paralinguistic symbols also come with numerical values representing intensity, which identify the strength of emotions or expressions. During data collection, training data can be manually annotated or automatically added with paralinguistic symbols. Paralinguistic symbols are combined with natural language text to form multi-modal training data. Speech analysis tools can be used to extract emotional features from speech (such as laughter, tone changes, etc.) and add them to the text in symbolic form.

[0122] Autoregressive language models are based on step-by-step generation, with each step generating results dependent on the output of the previous step. During model training, model parameters need to be initialized first, and training data (containing text and paralinguistic symbols) is input into the model. When the autoregressive model is initialized, the model's parameters are randomly initialized, ready to process the input training data. The training data is input into the model in the form of word vectors and symbols, and the model will gradually process these inputs to learn how to generate appropriate outputs.

[0123] Autoregressive language models also process paralinguistic symbols when processing text. The model needs to identify the type of symbol (such as laughter, pauses, etc.) and its intensity (such as strong laughter and slight laughter). These symbols provide non-verbal features in conversations that help generate more natural and realistic spoken language text. Autoregressive models use attention mechanisms or other processing methods to identify and process paralinguistic symbols in text while processing inputs step by step, and generate natural conversations by combining context information. The model can adjust the emotional expression or tone of the generated content based on the type and intensity of the symbol.

[0124] The output generated by the autoregressive model is compared with the pre-set training labels. Training labels are ideal outputs, usually manually annotated correct answers or system-generated reference outputs. By comparing the differences between the output and the label, the performance of the model can be evaluated. The error between the model's output and the training label can be measured by calculating the loss function (such as cross-entropy loss). This process can provide a basis for subsequent model optimization.

[0125] After determining the error, the autoregressive model's parameters are updated using the backpropagation algorithm. The backpropagation algorithm calculates the gradient descent to gradually adjust the model parameters, reducing the error. The model updates the model's weights and biases based on the gradient information of the loss function. The goal of backpropagation is to optimize through multiple iterations so that the output generated by the model is closer and closer to the expected label.

[0126] The training process requires multiple rounds of iteration, with each round optimizing parameters based on new training data and errors. As the number of training rounds increases, the model gradually optimizes and the error gradually decreases until the model can generate outputs that better match the training data. After each round of training, the model adjusts the parameters based on the performance of the training set and the validation set until the error is reduced to the target range. Through this multi-round training and optimization, the final trained autoregressive language model is obtained.

[0127] By obtaining training data containing paralinguistic symbols and training with an autoregressive language model, the system can generate more natural and colloquial text. Paralinguistic symbols allow the model to handle non-verbal features such as laughter, coughing, etc., resulting in more emotionally and interactive text. This can improve the user interaction experience, especially in customer service in the financial field, allowing more accurate expression of service content and reflecting customer care.

[0128] In one embodiment, the above S40 comprises:

[0129] S410, input the text features of each sentence and the content token into the autoregressive language model;

[0130] S411, the autoregressive language model generates text output step by step according to the text features and content tokens;

[0131] S412, in each generation process, combine the output generated in the last step and the context information in the content token to generate new output;

[0132] S413, when the autoregressive language model completes the generation of the entire sequence, output the colloquial text corresponding to the colloquial instruction.

[0133] In this embodiment, the text features of the sentence and the content token are input into the autoregressive language model. The text features are usually the semantic and grammatical information of the sentence, while the content token contains the context information. By combining the two, the model can understand the semantic structure and context association of the sentence, and generate more natural text output. The text features and content tokens are input into the model as high-dimensional vectors, respectively. After the model receives these vectors, it processes them through embedding layers or linear transformations to map them to internal feature spaces for further calculation.

[0134] Autoregressive language models generate text output step by step, with the generation process being autoregressive, meaning that each step of generation depends on the output of the previous step. At each time step, the model combines the current text features and content tokens to generate a partial output and continues generating at the next time step until a complete sentence is generated. Autoregressive models such as GPT generate one word or token at each time step, with the generation based on the current input text features and previously generated words. The model incorporates context information from the content tokens through an attention mechanism to generate a complete sentence step by step.

[0135] At each step of the generation process, the model not only uses the text features and content tokens for the current time step but also incorporates the output generated in the previous step and the context information from before. This ensures that the generated text is coherent and natural in context and avoids semantic disconnection. The model uses an attention mechanism or a recurrent neural network structure to take the previously generated output as input and combine it with the current content tokens and text features to generate new output. This process iterates until a complete sentence or paragraph is generated.

[0136] When the autoregressive language model generates the final output for an entire sentence or sequence, the generation process ends. At this point, the generated text contains all the semantic information of the sentence and remains coherent with the context before. The text also incorporates the spoken instruction, making the generated text conform to the predetermined tone or expression style. The autoregressive language model stops the generation process when it reaches an end token (such as an end symbol) or generates a complete sentence. The final output text is a natural language expression that meets the spoken requirements and ensures the coherence of the context according to the input text features and content tokens.

[0137] This embodiment gradually generates natural spoken text by inputting the text features and content tokens of a sentence into an autoregressive language model, ensuring that the generated text can reflect the independent semantic information of the sentence and maintain coherence with the context. The autoregressive language model generates output step by step and combines the results of the previous step to ensure consistency and naturalness in the text generation process.

[0138] In an embodiment, a spoken text generation device is provided, which corresponds to the spoken text generation method described in the above embodiments. Referring to Figure 3 , Figure 3 The functional module diagram of a preferred embodiment of the spoken text generation device of the present application is shown. The features extraction module 10, the context attention mechanism module 20, the feature projection module 30, and the spoken text generation module 40. The detailed description of each functional module is as follows:

[0139] The feature extraction module 10 is used to divide the written text into multiple sentences and extract the text features of each sentence using a pre-trained language model;

[0140] The context attention mechanism module 20 is configured to generate a context token based on a context attention mechanism and in combination with the text features of each sentence and the spokenization instruction;

[0141] The feature projection module 30 is configured to project the context token and the text features of each sentence to a unified feature space through a linear projector to generate a content token;

[0142] The spokenization text generation module 40 is configured to input the text features of each sentence into a self-recurrent language model and generate spokenization text in combination with the content token.

[0143] In an embodiment, the spokenization text generation module 40 is specifically configured to:

[0144] obtain multi-modal auxiliary data, and extract multi-modal features from the multi-modal auxiliary data;

[0145] fuse the multi-modal features with the text features of each sentence to generate fused features;

[0146] input the fused features into a self-recurrent language model and generate spokenization text in combination with the content token.

[0147] In an embodiment, the context attention mechanism module 20 is specifically configured to:

[0148] extract semantic information from each sentence, the semantic information being used to represent the context association between sentences;

[0149] generate a query matrix based on the spokenization instruction;

[0150] generate a key matrix and a value matrix based on the semantic information and the text features of each sentence, the key matrix being used to represent the association degree between sentences, and the value matrix being used to store the content information of the sentences;

[0151] analyze the association degree between the query matrix and the key matrix through the context attention mechanism, and generate an attention weight of each sentence based on the association degree;

[0152] generate a context token in combination with the value matrix according to the attention weight of each sentence.

[0153] In an embodiment, the feature projection module 30 is specifically configured to:

[0154] input the context token and the text features of each sentence into a linear projector;

[0155] apply a context weight matrix to the context token and a text weight matrix to the text features for weighted processing, respectively;

[0156] The weighted context token and the text feature are projected into a unified feature space to generate a content token.

[0157] In an embodiment, the feature extraction module 10 is specifically configured to:

[0158] Obtain written text, and divide the written text into multiple sentences according to punctuation marks, line breaks, or language logic;

[0159] Input each sentence into a pre-trained language model, which processes the sentence based on context modeling and semantic representation;

[0160] Extract a feature vector for each sentence through the pre-trained language model to obtain a text feature of each sentence, the text feature including semantic, grammatical, and contextual information of the sentence.

[0161] In an embodiment, the spoken text generation module 40 is specifically configured to:

[0162] Obtain training data containing natural language text and paralanguage symbols, the paralanguage symbols representing types and intensities of non-verbal features;

[0163] Initialize an autoregressive language model, and input the training data into the autoregressive language model;

[0164] Process text context and paralanguage symbols in the training data through the autoregressive language model, identify the paralanguage symbols and corresponding intensities, and process non-verbal features corresponding to the paralanguage symbols;

[0165] Compare the output generated by the autoregressive model with labels in the training data to determine an error;

[0166] Based on the error, adjust parameters of the autoregressive language model using a backpropagation method;

[0167] Through multiple rounds of training and parameter optimization, gradually reduce the error, and obtain a trained autoregressive language model.

[0168] In an embodiment, the spoken text generation module 40 is specifically configured to:

[0169] Input the text feature of each sentence and the content token into the autoregressive language model;

[0170] The autoregressive language model gradually generates a text output according to the text feature and the content token;

[0171] In each generation process, combine the output generated in the previous step and the context information in the content token to generate a new output;

[0172] When the autoregressive language model completes the generation of the entire sequence, output the spoken text corresponding to the spoken instruction.

[0173] In one embodiment, a computer device is provided, which can be a server, and an internal structure diagram thereof can be as shown in Figure 4 The computer device includes a processor, a memory, a network interface and a database connected through a system bus. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile and / or volatile storage medium, an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The network interface of the computer device is configured to communicate with an external user terminal through a network connection. The computer program is executed by the processor to implement the functions or steps of the spoken text generation method on the server side.

[0174] In one embodiment, a computer device is provided, which can be a user terminal, and an internal structure diagram thereof can be as shown in Figure 5 The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The network interface of the computer device is configured to communicate with an external server through a network connection. The computer program is executed by the processor to implement the functions or steps of the spoken text generation method on the user terminal side

[0175] In one embodiment, a computer device is provided, which includes a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor implements the following steps when executing the computer program:

[0176] segmenting the written text into multiple sentences, and extracting text features of each sentence using a pre-trained language model;

[0177] generating context tokens based on a context attention mechanism, in combination with the text features of each sentence and the spoken instruction;

[0178] projecting the context tokens and the text features of each sentence to a unified feature space through a linear projector to generate content tokens;

[0179] inputting the text features of each sentence into an autoregressive language model, in combination with the content tokens, to generate spoken text.

[0180] In one embodiment, a computer readable storage medium is provided, having stored thereon a computer program, which when executed by a processor implements the following steps:

[0181] segmenting the written text into multiple sentences, extracting text features of each sentence using a pre-trained language model;

[0182] generating context tokens based on a context attention mechanism, in combination with the text features of each sentence and the spokenization instruction;

[0183] projecting the context tokens and the text features of each sentence to a unified feature space through a linear projector to generate content tokens;

[0184] inputting the text features of each sentence into a self-attention language model, in combination with the content tokens, to generate spokenization text.

[0185] It should be noted that the functions or steps that the above computer readable storage medium or computer device can implement can correspond to the related descriptions of the server side and the user side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0186] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. Any reference to memory, storage, database or other medium used in the embodiments provided in the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0187] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above functional units and modules is taken as an example, and in actual application, the above functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above.

[0188] It should be noted that if non-company software tools or components appear in the embodiments of the present application, they are only used for example introduction and do not represent actual use. The above-described embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A method for generating colloquial text, characterized in that, Includes the following steps: The written text is segmented into multiple sentences, and the text features of each sentence are extracted using a pre-trained language model; Semantic information is extracted from each sentence, and this semantic information is used to represent the contextual relationships between sentences; Generate a query matrix based on colloquial instructions; Based on the semantic information and text features of each sentence, a key matrix and a value matrix are generated. The key matrix is ​​used to represent the degree of correlation between sentences, and the value matrix is ​​used to store the content information of the sentences. By using a contextual attention mechanism, the correlation between the query matrix and the key matrix is ​​analyzed, and attention weights for each sentence are generated based on the correlation. Context tokens are generated based on the attention weights of each sentence and the value matrix. The context token and the text features of each sentence are input into a linear projector; A context weight matrix is ​​applied to the context token, and a text weight matrix is ​​applied to the text features, and weighting is performed on each; The weighted context token and text features are projected and mapped onto a unified feature space to generate a content token. The text features of each sentence are input into an autoregressive language model, and combined with the content token, colloquial text is generated.

2. The colloquial text generation method as described in claim 1, characterized in that, The text features of each sentence are input into an autoregressive language model, and combined with the content token, spoken text is generated, including: Acquire multimodal auxiliary data, and extract multimodal features from the multimodal auxiliary data; The multimodal features are fused with the text features of each sentence to generate fused features; The fused features are input into an autoregressive language model and combined with the content token to generate colloquial text.

3. The colloquial text generation method as described in claim 1, characterized in that, The written text is segmented into multiple sentences, and the text features of each sentence are extracted using a pre-trained language model, including: Obtain written text and segment it into multiple sentences based on punctuation, line breaks, or language logic; Each sentence is input into a pre-trained language model, which processes the sentences based on context modeling and semantic representation; The pre-trained language model extracts feature vectors from each sentence to obtain text features for each sentence, which include semantic, grammatical, and contextual information of the sentence.

4. The colloquial text generation method as described in claim 1, characterized in that, Before inputting the text features of each sentence into the autoregressive language model, the following steps are also included: Acquire training data containing natural language text and paralinguistic symbols, wherein the paralinguistic symbols represent the type and intensity of non-linguistic features; Initialize the autoregressive language model and input the training data into the autoregressive language model; The autoregressive language model processes the text context and paralinguistic symbols in the training data, identifies the paralinguistic symbols and their corresponding intensities, and processes the non-linguistic features corresponding to the paralinguistic symbols. The output generated by the autoregressive model is compared with the labels in the training data to determine the error; Based on the aforementioned error, the parameters of the autoregressive language model are adjusted using the backpropagation method; Through multiple rounds of training and parameter optimization, the error is gradually reduced, resulting in a fully trained autoregressive language model.

5. The colloquial text generation method as described in claim 1, characterized in that, The text features of each sentence are input into an autoregressive language model, and combined with the content token, spoken text is generated, including: The text features of each sentence and the content token are input into the autoregressive language model; The autoregressive language model generates text output step by step based on the text features and content tokens; In each generation process, a new output is generated by combining the output generated in the previous step with the context information in the content token; Once the autoregressive language model has completed generating the entire sequence, it outputs the colloquial text corresponding to the colloquial instructions.

6. A conversational text generation device, characterized in that, The colloquial text generation device includes: The feature extraction module is used to segment written text into multiple sentences and extract text features for each sentence using a pre-trained language model; The context attention mechanism module is used to extract semantic information from each sentence, which represents the contextual association between sentences; generate a query matrix based on colloquial instructions; generate a key matrix and a value matrix based on the semantic information and text features of each sentence, where the key matrix represents the degree of association between sentences and the value matrix stores the content information of sentences; analyze the degree of association between the query matrix and the key matrix through the context attention mechanism, and generate an attention weight for each sentence based on the degree of association; and generate a context token based on the attention weight of each sentence and the value matrix. The feature projection module is used to input the context token and the text features of each sentence into a linear projector; apply a context weight matrix to the context token and a text weight matrix to the text features, and perform weighted processing respectively; project and map the weighted context token and text features to a unified feature space to generate a content token; The colloquial text generation module is used to input the text features of each sentence into an autoregressive language model and combine them with the content token to generate colloquial text.

7. A computer device, characterized in that, The computer device includes a memory, a processor, and a conversational text generation program stored in the memory and executable on the processor, wherein the conversational text generation program, when executed by the processor, implements the steps of the conversational text generation method as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The storage medium stores a colloquial text generation program, which, when executed by a processor, implements the steps of the colloquial text generation method as described in any one of claims 1-5.

Images