Spoken text generation method and device, equipment and storage medium

Abstract

The invention relates to the technical field of artificial intelligence and the field of financial science and technology, and discloses a spoken text generation method, which comprises the following steps of: segmenting a written text into a plurality of sentences, and extracting text features of each sentence by using a pre-training language model; and based on a context attention mechanism, generating a context token in combination with the text feature and the spoken instruction of each sentence. And projecting the context token and the text feature of each sentence to a unified feature space through a linear projector to generate a content token. And inputting the text feature of each sentence into the autoregression language model, and generating a spoken text in combination with the content token. Through a context attention mechanism, association and semantic information between sentences can be more effectively combined, and the naturalness and fluency of the spoken text are improved. Text features and context tokens are processed through a linear projector, so that the coherence and consistency of generated texts are ensured. The problem that text generation is not coherent is solved, and the naturalness and accuracy of the spoken text are improved.

Description

Technical Field

[0001] The present invention relates to the fields of artificial intelligence technology and financial technology, and in particular to a method, device, equipment and storage medium for generating spoken text. Background Art

[0002] With the widespread application of intelligent customer service and dialogue generation technology in the financial field, especially in scenarios such as customer service and risk consulting, oral transcription systems have gradually become a key tool for financial institutions to optimize user experience and improve service efficiency. However, existing oral transcription technology still has many shortcomings in dealing with the complexity and diverse needs of the financial field.

[0003] Existing spoken transcription systems have high computational complexity when processing long text conversations, especially showing obvious deficiencies in context understanding. In financial conversation scenarios, customers usually ask multi-level and multi-context questions, and the system needs to accurately understand the user's intentions based on the context information and generate corresponding responses. However, the existing technology lacks a deep understanding of context information during the generation process, which easily produces context-disconnected and logically incoherent conversation texts, reducing the quality of customer interaction.

[0004] At the same time, existing spoken transcription systems perform poorly when processing multimodal data (such as text and audio), and are unable to fully utilize information such as tone and pauses in audio data to enhance the naturalness and contextual coherence of the text. In the financial field, customer communications are often accompanied by sound features such as emotional expression and tone changes, which are crucial for simulating natural spoken language. However, when fusing text and audio, existing technologies cannot flexibly insert sound effects that simulate spoken language (such as laughter, coughing, etc.), and lack precision in the generation and intensity control of these sound effects, affecting the naturalness of the final generated text and customer experience.

[0005] Moreover, these models are often unable to naturally insert symbols that simulate spoken language features (such as laughter, coughs, intonation fillers, etc.) during the text generation process, which makes the generated text appear stiff when simulating natural spoken language. Summary of the invention

[0006] The main purpose of the present invention is to provide a method, device, equipment and storage medium for generating spoken text, aiming to solve the technical problem that the prior art lacks a deep understanding of contextual information when processing spoken text generation, resulting in the generated dialogue text being not coherent and natural enough in long texts or complex contexts.

[0007] To achieve the above object, the present invention provides a method for generating a colloquial text, comprising:

[0008] Segment the written text into sentences and extract text features of each sentence using a pre-trained language model;

[0009] Based on the contextual attention mechanism, the text features of each sentence and the spoken instructions are combined to generate context tokens;

[0010] Projecting the context tokens and the text features of each sentence into a unified feature space through a linear projector to generate content tokens;

[0011] The text features of each sentence are input into the autoregressive language model and combined with the content tokens to generate colloquial text.

[0012] Furthermore, to achieve the above object, the present invention provides a colloquial text generation device, comprising:

[0013] A feature extraction module is used to segment the written text into multiple sentences and extract the text features of each sentence using a pre-trained language model;

[0014] A context attention mechanism module, which is used to generate context tokens based on the context attention mechanism by combining the text features of each sentence and the spoken instructions;

[0015] A feature projection module, used for projecting the context token and the text features of each sentence into a unified feature space through a linear projector to generate a content token;

[0016] The spoken text generation module is used to input the text features of each sentence into the autoregressive language model and combine them with the content tokens to generate spoken text.

[0017] Furthermore, to achieve the above-mentioned purpose, the present invention also provides a computer device, which includes a memory, a processor, and a spoken text generation program stored in the memory and executable on the processor, and the spoken text generation program, when executed by the processor, implements the steps of the spoken text generation method as described above.

[0018] Furthermore, to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, on which a spoken text generation program is stored, and when the spoken text generation program is executed by a processor, the steps of the spoken text generation method as described above are implemented.

[0019] Beneficial effects: The present invention relates to the fields of artificial intelligence technology and financial technology, and discloses a method for generating spoken text, which divides a written text into multiple sentences and extracts the text features of each sentence using a pre-trained language model. Based on the context attention mechanism, the text features of each sentence are combined with the spoken instructions to generate a context token. The context token and the text features of each sentence are projected to a unified feature space by a linear projector to generate a content token. The text features of each sentence are input into an autoregressive language model, and the spoken text is generated in combination with the content token. The present invention can more effectively combine the association and semantic information between sentences through the context attention mechanism, thereby improving the naturalness and fluency of the spoken text. The text features and the context token are processed by a linear projector to ensure the coherence and consistency of the generated text. The autoregressive language model gradually generates text output, so that the text expression is more in line with the law of natural language expression, and can process the context information in complex dialogue scenes. The problem of lack of context understanding and incoherent generation of spoken text generation in the prior art is effectively solved, the naturalness and accuracy of the generated text are improved, and it is suitable for multi-scenario dialogue generation and voice assistant systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The present invention will be further described below with reference to the accompanying drawings and embodiments, in which:

[0021] Figure 1 A schematic diagram of an application environment of a method for generating a colloquial text in an embodiment of the present invention;

[0022] Figure 2 It is a flow chart of an embodiment of a method for generating a spoken text according to the present invention;

[0023] Figure 3 It is a functional module diagram of a preferred embodiment of the spoken text generation device of the present invention;

[0024] Figure 4 A schematic diagram of the structure of a computer device in one embodiment of the present invention;

[0025] Figure 5 FIG. 4 is another schematic diagram of the structure of a computer device in one embodiment of the present invention. DETAILED DESCRIPTION

[0026] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0027] The method for generating colloquial text provided by the embodiment of the present invention can be applied in the following aspects: Figure 1In an application environment, the user terminal communicates with the server terminal through a network. The server terminal can divide the written text into multiple sentences through the user terminal, and use a pre-trained language model to extract the text features of each sentence. Based on the context attention mechanism, the text features of each sentence and the colloquial instructions are combined to generate context tokens. The context tokens and the text features of each sentence are projected to a unified feature space by a linear projector to generate content tokens. The text features of each sentence are input into an autoregressive language model, and the colloquial text is generated in combination with the content tokens. The present invention can more effectively combine the association and semantic information between sentences through the context attention mechanism, thereby improving the naturalness and fluency of the colloquial text. The text features and the context tokens are processed by the linear projector to ensure the coherence and consistency of the generated text. The autoregressive language model gradually generates text output, so that the text expression is more in line with the law of natural language expression, and can process the context information in complex dialogue scenes. The problem of lack of context understanding and incoherent generation of colloquial text in the prior art is effectively solved, the naturalness and accuracy of the generated text are improved, and it is suitable for multi-scenario dialogue generation and voice assistant systems. The user end may be, but is not limited to, various personal computers, laptop computers, smart phones, tablet computers, and portable wearable devices. The server end may be implemented by an independent server or a server cluster consisting of multiple servers. The present invention is described in detail below through specific embodiments.

[0028] See also Figure 2 , Figure 2 This is a flow chart of an embodiment of a method for generating a colloquial text provided by the present invention. It should be noted that although a logical sequence is shown in the flow chart, in some cases, the steps shown or described may be performed in a different order than that shown here.

[0029] like Figure 2 As shown, the method for generating colloquial text proposed by the present invention comprises the following steps:

[0030] S10, segmenting the written text into multiple sentences and extracting text features of each sentence using a pre-trained language model;

[0031] In this embodiment, segmenting the written text into multiple sentences is a preprocessing step. Text segmentation is usually based on the grammatical rules of the language, especially punctuation marks (such as periods, question marks, exclamation marks, etc.) and line breaks. This step needs to ensure that each sentence in the text can independently extract its semantic features, so the accuracy of segmentation directly affects the subsequent feature extraction.

[0032] Text segmentation can be achieved by matching punctuation marks with regular expressions, or by automatic segmentation using natural language processing tools (such as NLP toolkits). Tools can automatically identify sentence boundaries and take into account the complexity of language (such as the fact that some punctuation marks may appear inside sentences) to avoid incorrect segmentation.

[0033] The segmented sentences need to be standardized. This includes removing extra blank characters, tabs, and special characters (such as line breaks), and normalizing punctuation (such as unifying different forms of quotation marks into one format). The purpose of preprocessing is to ensure that the data input into the model is clean and standardized, and to prevent the model from being affected by invalid characters. Usually, text cleaning tools or custom scripts are used to remove extra characters through regular expressions or other string processing functions to ensure the standardization of text input.

[0034] The preprocessed sentences are input into the pretrained language model, which extracts the text features of the sentences through context modeling and semantic representation. Pretrained models (such as BERT and GPT) have been trained with large-scale data sets and can automatically capture the semantic features, grammatical structure, and contextual information in sentences. Each sentence can be converted into an input format acceptable to the model, usually by converting the sentence into a series of word vectors or character vectors through a tokenizer. After input into the model, the model performs semantic calculations on it and outputs feature vectors. These feature vectors represent the semantic information of the sentence in a high-dimensional space.

[0035] Through the processing of the pre-trained language model, the feature vector of each sentence is extracted. These feature vectors include semantic, grammatical and contextual information, providing the basis for the subsequent generation of spoken text. These vectors are comprehensive representations of each sentence, capable of capturing its core semantic features. The model outputs a vector representation of each sentence, usually in the form of a high-dimensional vector. These vectors can be extracted through a specific interface and stored as input for subsequent processing steps.

[0036] For example, the text features of each sentence are extracted through a pre-trained language model. The formula is as follows:

[0037] T i =LLM(S i )

[0038] Among them, LLM represents the pre-trained language model, S i represents the i-th sentence of the input, T i Represents text features extracted by a pre-trained language model.

[0039] Example: In a customer service system in the financial field, when a customer enters a written query text, such as "I want to know the interest rate of a loan", the system will first segment the text into multiple sentences based on punctuation and language logic, such as "I want to know" and "the interest rate of a loan". Then, the system will input these sentences into a pre-trained language model (such as the BERT model). The pre-trained language model extracts the text features of each sentence, including the semantics, grammatical information, and contextual associations of the sentence.

[0040] For example, after the sentence "what is the interest rate of the loan" is processed by the model, the extracted feature vector can reflect the professionalism of the sentence in the financial field, and combined with the previous sentence "I want to know", it helps the system understand the customer's query needs more accurately.

[0041] By dividing the written text into multiple sentences and using a pre-trained language model to extract the text features of each sentence, the independence of the sentences is ensured. At the same time, the semantic features and contextual information of the sentences are obtained through the pre-trained model, which improves the naturalness and contextual coherence of the generated text.

[0042] S20, based on the contextual attention mechanism, combines the text features of each sentence and the spoken instructions to generate context tokens;

[0043] In this embodiment, the contextual attention mechanism is a technology for processing the correlation information between sentences in the text, and generates contextual correlation information by calculating the correlation between each sentence and other sentences. In this step, the attention mechanism analyzes the text features of each sentence, combines the context information of the previous and next sentences, and ensures that the generated context token can accurately reflect the context of the sentence in the entire conversation. A multi-head attention mechanism can be used to calculate the attention scores between the query matrix, the key matrix, and the value matrix. The query matrix is ​​generated by the text features of the current sentence, and the key and value matrices are generated by the text features of other sentences. The context information is obtained by weighted summation of the weights.

[0044] Colloquial instructions refer to the tone, emotion, or style requirements of the system when generating colloquial text, such as converting text into more natural spoken expressions. These instructions will be embedded in the contextual attention mechanism to help the system add tone or semantic features when calculating the association between sentences, so that the generated context tokens not only contain the logical relationship between sentences, but also reflect the colloquial features. The colloquial instructions can be encoded into specific vectors and input into the attention mechanism together with the text features. Through these instructions, the model adjusts the attention weights to generate context tokens that are more in line with the colloquial style.

[0045] Context tokens are generated through the contextual attention mechanism to represent the contextual information of each sentence. It integrates the dependencies between sentences and the influence of spoken instructions to guide the contextual coherence of subsequent natural language text generation. The generation of context tokens depends on the calculation results of the attention mechanism. The system combines the text features and spoken instructions to generate context tokens through weighted calculation. Each token is a high-dimensional representation of the context and style of the current sentence.

[0046] Example: In a customer service system in the financial field, when a customer enters a question such as "What is the interest rate for a loan?", the system first analyzes the context of the customer's question by combining the previous conversation background (such as the loan application steps mentioned earlier) through the context attention mechanism. Then, the system generates a context token that conforms to the conversation style based on the text features of the current sentence and the spoken instructions entered by the user.

[0047] For example, users may also ask, "Is the application process complicated?" When generating answers, the system not only needs to consider the current question, but also insert features that simulate spoken language (such as a relaxed tone, necessary pauses, etc.). Context tokens help the system ensure that the generated answers are natural and fluent, such as: "It's not complicated, you only need to submit some basic information and you can complete the application quickly." This kind of dialogue generation not only accurately answers user questions, but also makes the response closer to natural dialogue through contextual attention mechanisms and spoken instructions.

[0048] For example, we use the context attention mechanism to generate context tokens in combination with user instructions. The context token aggregates the temporal information of the text and the user instructions, and the formula is as follows:

[0049] C i =Attention(Q,K,V); where Q, K, and V represent query, key, and value matrices respectively, and the context token C is calculated through the attention mechanism. i .

[0050] Through the contextual attention mechanism, the text features of each sentence are combined with the spoken instructions to generate context tokens, making the system more natural and coherent when generating spoken text. The generation of context tokens can not only reflect the semantic association between sentences, but also adjust the output style and tone according to the spoken instructions, improving the naturalness and accuracy of the conversation.

[0051] S30, projecting the context token and the text features of each sentence into a unified feature space through a linear projector to generate a content token;

[0052] In this embodiment, the context token represents the contextual information of each sentence, while the text feature contains the semantic and grammatical information of each sentence. In this step, the context token and the text feature are input into the linear projector, with the purpose of projecting the two types of information into the same feature space for fusion processing. The context token and the text feature can be encoded into high-dimensional vectors respectively and input into the linear projector for projection operation. The linear projector maps the two features into the same feature space by calculating the weight matrix.

[0053] The role of the linear projector is to project feature vectors of different dimensions into a unified feature space so that information from different sources can be weighted in the same space. In this step, the linear projector projects the context tokens and text features separately so that they can be compared and fused in the same dimension. The projector completes the projection process by calculating matrix multiplication. For each input context token and text feature, the linear projector maps them to the same vector space, calculated as:

[0054] Y=XW; where X is the input vector (context token or text feature), W is the projection matrix, and Y is the projected vector.

[0055] After the context tokens and text features are projected into the same feature space, the system needs to fuse them to generate content tokens that integrate the sentence semantic information and context information. The fused content tokens can better capture the context information and semantic features of the sentence for subsequent text generation. Feature fusion is usually achieved through weighted summation or other feature fusion methods. Different weights can be assigned according to the importance of the context tokens and text features, and the two projected vectors can be weighted to generate the final content token.

[0056] The content token is the final output after fusion, integrating the context information and the text features of the sentence. The content token will be used as the input of the subsequent autoregressive generation model to guide the generation of spoken text. Through linear projection and feature fusion, the content token generated by the system is in the form of a high-dimensional vector, which contains rich semantic and contextual information. These content tokens are used as input for subsequent steps.

[0057] For example, a linear projector is used to project text features and context tokens into a unified space to generate content tokens, and the formula is as follows: L i =W c ·C i +W t ·T i Among them, C i is the context token, T i is the text feature; W c and W tare the projection weight matrices for context tokens and text features, respectively, and L i The content token.

[0058] Example description: In a bank customer service scenario, suppose the customer first asks: "What documents do I need to prepare for a loan application?" The system generates a context token for the sentence through the context attention mechanism, indicating the relationship between the sentence and the entire conversation context. At the same time, the text features of the sentence are extracted to capture the two core semantics of "loan application" and "prepare documents".

[0059] The customer then asks, "What is the interest rate?" The system needs to combine the previous context and the current question, and project the context tokens and text features of these two questions into the same feature space through a linear projector. Content tokens are generated through feature fusion.

[0060] Text features: The feature vector of the sentence "What documents are needed for loan application" may contain high-dimensional information related to "loan application" and "prepared documents", for example: [0.4, 0.8, -0.5, 1.0], where each value represents the importance of the sentence in a specific semantic dimension.

[0061] Context information: The context token generated by the context attention mechanism may reflect the dependency between the previous and next sentences. For example, the semantic correlation between the previous sentence and the current sentence is 0.6. The content of the context token may be: [0.6, -0.2, 0.7, 0.9], indicating the correlation information in the context.

[0062] Fusion of content tokens: Content tokens are the fusion result of text features and context information. The fused content token may be: [0.5, 0.3, 0.1, 1.5], which means that the token integrates the contextual association of "loan application" and "interest rate" and reflects the overall background of the customer's question through weighted fusion.

[0063] With this content token, the system generates more coherent and natural responses. For example, when a customer asks "What is the interest rate?", the system combines the contextual content token to generate a natural response: "The current interest rate is 4.5%, you can apply for a loan."

[0064] By projecting context tokens and text features into a unified feature space for fusion processing, the generated content tokens can better capture the contextual information and semantic features of the sentence, thereby improving the coherence and naturalness of text generation.

[0065] S40, inputting the text features of each sentence into an autoregressive language model, combining with the content token, to generate a spoken text.

[0066] In this embodiment, the text features of each sentence are used as input to the autoregressive language model. The autoregressive language model is good at generating text step by step, and the text features of the sentence are gradually processed by taking the output generated in the previous step as the input of the next step. The core purpose of this step is to use the step-by-step generation mechanism of the autoregressive language model to ensure that the output spoken text is coherent and natural. The text features can be high-dimensional vectors extracted from pre-trained models (such as BERT, GPT). When input into the autoregressive language model, the model processes these features step by step. Each step will generate partial output based on the current text features, and then use the output as the input of the next step until the complete text is generated.

[0067] In the process of generating spoken text, the system not only relies on the input text features, but also combines content tokens. Content tokens contain contextual information and semantic features. By combining content tokens, the model can adjust the generated text to not only conform to the characteristics of the current sentence, but also reflect the information of the entire context, making the text more coherent and in line with the logic of the conversation. Content tokens are attached as additional input vectors in each step of the generation process. The autoregressive language model combines the contextual information of the content tokens at each time step to adjust the output text. Content tokens can be embedded in the input layer of the model or weighted through the attention mechanism.

[0068] Generating colloquial text is the output goal of the autoregressive language model. Through a step-by-step generation approach, the model combines the text features and contextual information of each sentence to generate complete colloquial text. This process ensures that the generated text is natural and fluent, conforms to the habits of spoken expression, and can reflect the intention and contextual requirements of the user input. The generation process is autoregressive, and the model generates a word or phrase at each step and adjusts the output based on the context and content tokens. The final result is a text sequence that conforms to colloquial instructions and can be used in scenarios such as dialogue systems and customer service systems.

[0069] Example: In a bank customer service scenario, when a customer asks "What documents are needed for a loan application?", the system uses an autoregressive language model and the content tokens of the loan context to generate a natural, spoken answer that matches the customer's question. The answer the system might generate is: "You only need an ID card and proof of income to apply for a loan, and the application process is very simple."

[0070] By inputting text features into the autoregressive language model and combining it with content tokens, natural spoken text that conforms to the contextual information can be generated. The gradual generation method effectively improves the coherence and naturalness of the text, ensuring that the generated text not only accurately answers customer questions, but also reflects the expression habits of natural language, which is suitable for a variety of scenarios such as finance, customer service, and dialogue generation.

[0071] The present invention relates to the fields of artificial intelligence technology and financial technology, and discloses a method for generating spoken text, which is performed by dividing a written text into a plurality of sentences and extracting the text features of each sentence using a pre-trained language model. Based on a context attention mechanism, a context token is generated by combining the text features of each sentence with a spoken instruction. The context token and the text features of each sentence are projected to a unified feature space by a linear projector to generate a content token. The text features of each sentence are input into an autoregressive language model, and the spoken text is generated by combining the content token. The present invention can more effectively combine the association and semantic information between sentences through the context attention mechanism, thereby improving the naturalness and fluency of the spoken text. The text features and the context token are processed by a linear projector to ensure the coherence and consistency of the generated text. The autoregressive language model gradually generates text output, so that the text expression is more in line with the law of natural language expression, and can process the context information in complex dialogue scenes. The problem of lack of context understanding and incoherent generation of spoken text generation in the prior art is effectively solved, the naturalness and accuracy of the generated text are improved, and the system is suitable for multi-scenario dialogue generation and voice assistant system.

[0072] In one embodiment, the above S40 includes:

[0073] S401, acquiring multimodal auxiliary data, and extracting multimodal features from the multimodal auxiliary data;

[0074] S402, fusing the multimodal feature with the text feature of each sentence to generate a fused feature;

[0075] S403: Input the fusion feature into an autoregressive language model and combine it with the content token to generate a spoken text.

[0076] In this embodiment, multimodal auxiliary data refers to data sources from multiple modalities, such as text, audio, video, or images. When generating spoken text, these multimodal data can enhance the model's understanding of context. By combining text features and multimodal information (such as emotional features of audio), the system can generate richer and more natural spoken text. Multimodal data (such as speech sentiment analysis, user emotion recognition, etc.) is obtained from different data sources and processed. For text generation tasks, the most common multimodal data include audio features (such as pitch, tone) or visual information (such as facial expressions). The acquired multimodal data is standardized to ensure data consistency.

[0077] Multimodal features are important information extracted from multimodal auxiliary data. These features can include emotional changes in audio, facial expressions in images, etc. In this step, the system extracts relevant features from multimodal auxiliary data to assist in text generation tasks. Multimodal data is processed using specialized feature extraction algorithms (such as convolutional neural networks (CNNs) for image feature extraction and Mel spectrum for audio feature extraction). The resulting feature vector can represent the core information in the multimodal data, such as tone, emotion, or visual feedback.

[0078] By combining the features extracted from multimodal auxiliary data with text features, a fused feature vector is generated. This fused feature vector contains the semantic information of the text and the emotional, visual and other features of the multimodal data, which helps to generate text that is more natural and close to the actual context. Feature fusion is usually done through weighted summation or attention mechanism. The purpose of feature fusion is to combine text features with multimodal features in a unified feature space to make the generated content more coherent and natural. For example, text features and audio features can be weightedly fused through a multi-head attention mechanism to generate content that is more in line with the context.

[0079] The fused feature vector is input into the autoregressive language model, and the model generates text step by step. By combining multimodal features and text features, the model can generate colloquial text that reflects the actual context, such as inserting tone symbols, laughter, or other non-linguistic features in text generation to enhance naturalness. The autoregressive language model uses the fused feature vector as input and generates text step by step in each generation step. At each generation step, the contextual information in the content token is combined to ensure the coherence and natural fluency of the output content.

[0080] For example, text features and visual features are integrated through a multimodal fusion module to generate a comprehensive feature representation. The formula is as follows:

[0081] F i =Fusion(T i ,V i )

[0082] Among them, T i represents the text feature, V i represents the visual feature, F i It represents the fused feature representation; Fusion represents the fusion mechanism, which can be implemented in many ways, such as weighted summation, concatenation, or attention mechanism.

[0083] Example: In a banking customer service scenario, a customer may ask loan-related questions through a voice assistant. The system will extract the customer's tone of voice through voice sentiment analysis and merge it with the characteristics of the customer's text question. For example, the customer's tone may sound anxious, and the system will add more friendly and comforting colloquial expressions when generating text, such as: "Don't worry, applying for a loan is very simple, and we are always at your service."

[0084] By combining multimodal data, the system of this embodiment can not only generate text-based spoken text, but also adjust the generated content according to the user's audio, visual and other multimodal information, making the generated text more natural and close to real conversations.

[0085] In one embodiment, the above 20 includes:

[0086] S201, extracting semantic information from each sentence, where the semantic information is used to represent contextual associations between sentences;

[0087] S202, generating a query matrix based on the spoken instruction;

[0088] S203, generating a key matrix and a value matrix based on the semantic information and text features of each sentence, wherein the key matrix is ​​used to represent the association between sentences, and the value matrix is ​​used to store the content information of the sentences;

[0089] S204, analyzing the correlation between the query matrix and the key matrix through a context attention mechanism, and generating an attention weight for each sentence based on the correlation;

[0090] S205, generate context tokens based on the attention weight of each sentence and the value matrix.

[0091] In this embodiment, semantic information is extracted from each sentence, which is used to describe the meaning and role of the sentence in the context. The extracted semantic information can represent the semantic association between sentences and is the basis for generating context tokens. Each sentence is processed by a pre-trained language model (such as BERT or GPT) to extract the semantic information of the sentence. The semantic information of each sentence can be represented as a high-dimensional vector to capture the semantic features and contextual relevance of the sentence.

[0092] The query matrix is ​​a key part of the attention mechanism, which represents the query criteria used when looking for associations between sentences. In the text generation process based on the contextual attention mechanism, the query matrix is ​​generated based on the text features and colloquial instructions of each sentence, and is used to represent the query conditions when the target sentence looks for information about other sentences. The query matrix is ​​generated by linear transformation of the semantic information of the sentence and the colloquial instructions. The text features of each sentence can be used as input and combined with the colloquial instructions to generate the query matrix.

[0093] The key matrix represents the correlation between sentences and is used to measure the relevance of other sentences to the current sentence. The value matrix stores the specific content information of the sentence and represents the core information of each sentence. In the attention mechanism, the key and value matrices work together to determine the dependency between sentences. The key matrix and the value matrix are generated through the semantic information and text features of the sentences. The semantic information of the sentences is transformed into a key matrix through linear transformation to represent the correlation between sentences. At the same time, the text features of the sentences are mapped to the value matrix to store the core content information of each sentence.

[0094] The contextual attention mechanism determines the connection between each sentence by calculating the correlation between the query matrix and the key matrix. By calculating the dot product of the two matrices, the degree of correlation between the sentences is obtained. This correlation is used to generate attention weights, which determine the degree of influence of each sentence on other sentences. The dot product between the query matrix and the key matrix is ​​calculated to obtain the correlation score of each sentence with other sentences. These scores are normalized by the Softmax function to generate attention weights.

[0095] Based on the calculated attention weights, combined with the content information of the value matrix, context tokens are generated. Context tokens integrate the association information of each sentence with other sentences and combine the characteristics of spoken instructions to generate context representations for subsequent text generation. Context tokens are generated by weighted summing the attention weights and the value matrix. Context tokens can represent the context of the current sentence in the entire conversation and provide support for subsequent generation steps.

[0096] This embodiment uses the contextual attention mechanism to combine the text features of sentences and colloquial instructions to generate context tokens, so that the system can better understand the relationship between sentences and ensure that the generated text is coherent and consistent with the conversation context. The contextual understanding ability of dialogue generation is improved, and the generated text is more natural and fluent.

[0097] In one embodiment, the above S30 includes:

[0098] S301, inputting the context token and the text features of each sentence into a linear projector;

[0099] S302, applying a context weight matrix to the context tokens and applying a text weight matrix to the text features, and performing weighted processing respectively;

[0100] S303 , projecting and mapping the weighted context tokens and text features into a unified feature space to generate content tokens.

[0101] In this embodiment, the context tokens represent the contextual association information between sentences, while the text features contain the semantic and grammatical information of the sentences. In this step, the context tokens and text features are input into the linear projector, and the task of the projector is to map them to the same feature space so that these two different types of features can be processed uniformly. The context tokens and text features are input into the linear projector as vectors respectively. The linear projector projects the context tokens and text features uniformly into a high-dimensional space through matrix transformation. Each vector undergoes a linear transformation when input to adapt to the dimension of the same feature space.

[0102] To ensure that context tokens and text features are treated according to their respective importance during feature fusion, the system applies a context weight matrix to context tokens and a text weight matrix to text features. Weighted processing allows the model to dynamically adjust the influence of these two types of information. Context tokens and text features are weighted separately using the trained context weight matrix and text weight matrix. Weighted processing can be achieved through matrix multiplication, with the goal of assigning different weights to different features during the fusion process, thereby enhancing the influence of important information in a unified feature space.

[0103] The projector maps the weighted context tokens and text features to a unified feature space. In this space, the system can fuse features from two different sources. By projecting to the same space, feature vectors can be processed together to generate content tokens. The linear projector projects the weighted context tokens and text features to a unified high-dimensional feature space through matrix calculations. In this space, feature vectors can be fused by weighted summation or other methods.

[0104] Content tokens are the result of fusing context tokens and text features, integrating the context information and text features of a sentence. Content tokens serve as the core input for generating text in subsequent steps, ensuring that the generated text maintains the coherence of the sentence context and reflects the semantic features of each sentence. Content tokens are generated by fusing weighted context tokens and text features. Fusion can be accomplished through weighted summation or attention mechanisms, and content tokens are ultimately output as vectors that contain the comprehensive features of the sentence.

[0105] This embodiment projects context tokens and text features into a unified feature space, and the generated content tokens can effectively integrate the context information and semantic features of the sentence, ensuring that when the system generates colloquial text, it can reflect the independent features of each sentence and maintain the coherence of the text.

[0106] In one embodiment, the above S10 includes:

[0107] S101, obtaining a written text, and dividing the written text into a plurality of sentences according to punctuation marks, line breaks or language logic;

[0108] S102, inputting each sentence into a pre-trained language model, wherein the pre-trained language model processes the sentence based on context modeling and semantic representation;

[0109] S103, extracting a feature vector for each sentence using the pre-trained language model to obtain text features of each sentence, wherein the text features include semantics, grammar, and context information of the sentence.

[0110] In this embodiment, the written text is segmented into multiple sentences. The segmentation basis can be punctuation marks (such as period, question mark, exclamation mark, etc.), line breaks, or division based on the grammatical logic of the language. The purpose of segmentation is to structure the text so that each sentence can be processed independently, which is convenient for subsequent feature extraction. The text is segmented using natural language processing tools or regular expressions. Regular expressions can identify sentence boundaries by matching punctuation marks, or they can be divided according to line breaks or specific grammatical rules of the language. Each sentence after segmentation will be further processed as an independent unit.

[0111] Pre-trained language models (such as BERT, GPT, etc.) are deep learning models that have been trained on large-scale corpora and can extract the semantic and contextual information of sentences. In this step, each segmented sentence is input into the pre-trained language model, and the model performs further processing based on the context modeling and semantic information of the sentence. First, each sentence is segmented and converted into an input form acceptable to the model (usually word vectors or character vectors). These vectors are then input into the pre-trained language model, which performs context modeling on them to capture the semantics, grammatical structure, and contextual dependencies of the sentence.

[0112] The pre-trained language model processes the sentences and outputs a feature vector for each sentence. The feature vector includes the semantic information, grammatical structure, and contextual information of the sentence. This vector can be regarded as a high-dimensional representation of the sentence for further processing or text generation in subsequent steps. After the input sentence is processed at multiple levels by the language model, the generated feature vector is usually a high-dimensional vector (such as BERT outputs a 768-dimensional vector). These feature vectors reflect the main information of the sentence, including its contextual position in the entire text, grammatical and semantic features. The output feature vector can be stored as input for subsequent generation tasks.

[0113] This embodiment can ensure the structured processing of text by segmenting written text into multiple sentences and using the pre-trained language model to extract text features, while using the semantic representation ability of the language model to capture the main semantics, grammar and context information of each sentence. This enables the system to maintain contextual coherence and semantic accuracy when generating natural language text.

[0114] In one embodiment, in the above S40, before inputting the text features of each sentence into the autoregressive language model, the following step is further included:

[0115] S404, obtaining training data including natural language text and paralinguistic symbols, wherein the paralinguistic symbols represent the type and intensity of non-linguistic features;

[0116] S405, initializing an autoregressive language model, and inputting the training data into the autoregressive language model;

[0117] S406, processing the text context and paralinguistic symbols in the training data by using the autoregressive language model, identifying the paralinguistic symbols and the corresponding intensities, and processing the non-linguistic features corresponding to the paralinguistic symbols;

[0118] S407, comparing the output generated by the autoregressive model with the labels in the training data to determine the error;

[0119] S408, adjusting the parameters of the autoregressive language model using a back propagation method based on the error;

[0120] S409, through multiple rounds of training and parameter optimization, the error is gradually reduced to obtain a trained autoregressive language model.

[0121] In this embodiment, the training data includes not only natural language text, but also paralinguistic symbols. These paralinguistic symbols are used to represent non-linguistic features, such as laughter, coughing, intonation fillers, etc., which can enhance the model's ability to process colloquial features in actual conversations. Paralinguistic symbols are also accompanied by numerical values ​​indicating intensity, which are used to identify the strength of emotions or expressions. When collecting data, training data can be added with paralinguistic symbols through manual annotation or automated tools. Paralinguistic symbols are combined with natural language text to form multimodal training data. Emotional features in speech (such as laughter, intonation changes, etc.) can be extracted through speech analysis tools and attached to the text in the form of symbols.

[0122] The autoregressive language model is a model based on step-by-step generation, where the result of each step depends on the output of the previous step. In the model training phase, the model parameters need to be initialized first, and the training data (including text and paralinguistic symbols) are input into the model. When the autoregressive model is initialized, the model parameters are randomly initialized and ready to process the input training data. The training data is input into the model in the form of word vectors and symbols, and the model will gradually process these inputs and learn how to generate appropriate outputs.

[0123] When processing text, the autoregressive language model also processes paralinguistic symbols. The model needs to identify the type of symbols (such as laughter, pauses, etc.) and their intensity (such as strong laughter and light laughter). These symbols provide non-linguistic features in the conversation and help generate more natural and realistic spoken text. When the autoregressive model gradually processes the input, it uses attention mechanisms or other processing methods to identify and process paralinguistic symbols in the text, and combines contextual information to generate natural conversations. The model can adjust the emotional expression or tone of the generated content based on the type and intensity of the symbols.

[0124] The output generated by the autoregressive model is compared with the pre-set training labels. The training labels are ideal outputs, usually the correct answers annotated by humans or reference outputs automatically generated by the system. By comparing the difference between the output and the labels, the performance of the model can be evaluated. The error between the model's output and the training labels can be measured by calculating a loss function (such as cross entropy loss). This process can provide a basis for subsequent model optimization.

[0125] After determining the error, the back propagation algorithm is used to update the parameters of the autoregressive model. The back propagation algorithm gradually adjusts the model parameters by calculating the gradient descent, so that the error gradually decreases. The model updates the weights and biases of the model based on the gradient information of the loss function. The goal of back propagation is to make the output generated by the model closer and closer to the expected label through multiple iterations of optimization.

[0126] The training process requires multiple rounds of iterations, and each round will optimize parameters based on new training data and errors. As the number of training rounds increases, the model is gradually optimized and the error is gradually reduced until the model can generate output that is consistent with the training data. After each round of training, the model will adjust parameters based on the performance of the training set and the validation set until the error is reduced to the target range. Through this multiple rounds of training and optimization, a trained autoregressive language model is finally obtained.

[0127] In this embodiment, by acquiring training data containing paralinguistic symbols and combining it with an autoregressive language model for training, the system can generate more natural colloquial text. Paralinguistic symbols enable the model to process non-linguistic features, such as laughter and coughing, thereby generating more emotional and interactive text. This can enhance the user interaction experience, especially in customer service in the financial field, and can more accurately express service content and reflect customer care.

[0128] In one embodiment, the above S40 includes:

[0129] S410, inputting the text features of each sentence and the content token into the autoregressive language model;

[0130] S411, the autoregressive language model gradually generates text output according to the text features and content tokens;

[0131] S412, in each generation step, combining the output generated in the previous step and the context information in the content token to generate a new output;

[0132] S413, when the autoregressive language model completes the generation of the entire sequence, outputs the spoken text corresponding to the spoken instruction.

[0133] In this embodiment, the text features and content tokens of the sentence are used as inputs to the autoregressive language model. The text features are usually the semantic and grammatical information of the sentence, while the content tokens contain contextual information. By combining the two, the model can understand the semantic structure and contextual association of the sentence, thereby generating a more natural text output. The text features and content tokens are respectively input into the model as high-dimensional vectors. After the model receives these vectors, it processes these input vectors through an embedding layer or a linear transformation, and maps them into the internal feature space for further calculation.

[0134] The autoregressive language model generates text output step by step. The generation process is autoregressive, which means that the generation of each step depends on the output of the previous step. At each time step, the model combines the current text features and content tokens to generate partial output, and continues to generate at the next time step until a complete sentence is generated. Autoregressive models such as GPT generate a word or token at each time step, which is based on the text features of the current input and the previously generated words. The model combines the contextual information in the content token through the attention mechanism to gradually generate complete sentences.

[0135] In each generation process, the model not only uses text features and content tokens of the current time step, but also combines the output generated in the previous step and previous context information. This ensures that the generated text is coherent and natural in context, avoiding semantic disconnection. The model uses the previously generated output as input through an attention mechanism or a recurrent neural network structure, and then combines the current content token and text features to generate new output. This process is iterated until a complete sentence or paragraph is generated.

[0136] The generation process ends when the autoregressive language model generates the final output of the entire sentence or sequence. At this point, the generated text already contains all the semantic information of the sentence and is coherent with the previous context. The text is also combined with colloquial instructions so that the generated text conforms to the predetermined tone or expression style. The autoregressive language model stops the generation process after reaching the end marker (such as the end symbol) or generating a complete sentence. The final output text is a natural language expression that meets the requirements of colloquialism, and the coherence of the context is guaranteed based on the input text features and content tokens.

[0137] This embodiment gradually generates natural colloquial text by inputting the text features and content tokens of the sentence into the autoregressive language model, which can ensure that the generated text can reflect the independent semantic information of the sentence and maintain coherence with the context. The autoregressive language model gradually generates outputs and combines the generation results of the previous step to ensure consistency and naturalness in the text generation process.

[0138] In one embodiment, a colloquial text generation device is provided, and the colloquial text generation device corresponds one-to-one to the colloquial text generation method in the above embodiment. Figure 3 , Figure 3 This is a functional module diagram of a preferred embodiment of the colloquial text generation device of the present invention. Feature extraction module 10, context attention mechanism module 20, feature projection module 30 and colloquial text generation module 40. The functional modules are described in detail as follows:

[0139] A feature extraction module 10, for segmenting the written text into a plurality of sentences and extracting text features of each sentence using a pre-trained language model;

[0140] A context attention mechanism module 20, for generating context tokens based on a context attention mechanism by combining text features and colloquial instructions of each sentence;

[0141] A feature projection module 30, configured to project the context token and the text features of each sentence into a unified feature space through a linear projector to generate a content token;

[0142] The spoken text generation module 40 is used to input the text features of each sentence into the autoregressive language model and combine them with the content tokens to generate spoken text.

[0143] In one embodiment, the spoken text generation module 40 is specifically used to:

[0144] Acquiring multimodal auxiliary data, and extracting multimodal features from the multimodal auxiliary data;

[0145] Fusing the multimodal features with the text features of each sentence to generate fused features;

[0146] The fused features are input into an autoregressive language model and combined with the content tokens to generate spoken text.

[0147] In one embodiment, the context attention mechanism module 20 is specifically used to:

[0148] Extracting semantic information from each sentence, wherein the semantic information is used to represent contextual associations between sentences;

[0149] generating a query matrix based on the spoken instructions;

[0150] Based on the semantic information and text features of each sentence, a key matrix and a value matrix are generated, wherein the key matrix is ​​used to represent the association between sentences, and the value matrix is ​​used to store the content information of the sentences;

[0151] Analyzing the correlation between the query matrix and the key matrix through a context attention mechanism, and generating an attention weight for each sentence based on the correlation;

[0152] Based on the attention weights of each sentence, the context tokens are generated in conjunction with the value matrix.

[0153] In one embodiment, the feature projection module 30 is specifically configured to:

[0154] Inputting the context tokens and the text features of each sentence into a linear projector;

[0155] Apply a context weight matrix to the context tokens and a text weight matrix to the text features for weighted processing respectively;

[0156] The weighted context tokens and text features are projected into a unified feature space to generate content tokens.

[0157] In one embodiment, the feature extraction module 10 is specifically used for:

[0158] Obtaining a written text and segmenting the written text into a plurality of sentences based on punctuation marks, line breaks, or language logic;

[0159] Input each sentence into a pre-trained language model, which processes the sentence based on context modeling and semantic representation;

[0160] The pre-trained language model is used to extract feature vectors from each sentence to obtain text features of each sentence, wherein the text features include semantic, grammatical and contextual information of the sentence.

[0161] In one embodiment, the spoken text generation module 40 is specifically used to:

[0162] Obtaining training data including natural language text and paralinguistic symbols, wherein the paralinguistic symbols represent the type and intensity of non-linguistic features;

[0163] Initializing an autoregressive language model, and inputting the training data into the autoregressive language model;

[0164] Processing text context and paralinguistic symbols in training data through the autoregressive language model, identifying paralinguistic symbols and corresponding intensities, and processing non-linguistic features corresponding to the paralinguistic symbols;

[0165] Compare the output generated by the autoregressive model with the labels in the training data to determine the error;

[0166] Based on the error, adjusting parameters of the autoregressive language model using a back-propagation method;

[0167] Through multiple rounds of training and parameter optimization, the error is gradually reduced to obtain a trained autoregressive language model.

[0168] In one embodiment, the spoken text generation module 40 is specifically used to:

[0169] Inputting the text features of each sentence and the content token into the autoregressive language model;

[0170] The autoregressive language model gradually generates text output according to the text features and content tokens;

[0171] In each generation process, the output generated in the previous step and the context information in the content token are combined to generate a new output;

[0172] When the autoregressive language model completes the generation of the entire sequence, the spoken text corresponding to the spoken instruction is output.

[0173] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 4 As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile and / or volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external user terminal through a network connection. When the computer program is executed by the processor, it realizes the functions or steps of a server side of a spoken text generation method.

[0174] In one embodiment, a computer device is provided. The computer device may be a user terminal, and its internal structure diagram may be as follows: Figure 5 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external server via a network connection. When the computer program is executed by the processor, it realizes the functions or steps of a user-side method for generating a spoken text

[0175] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:

[0176] Segment the written text into sentences and extract text features of each sentence using a pre-trained language model;

[0177] Based on the contextual attention mechanism, the text features of each sentence and the spoken instructions are combined to generate context tokens;

[0178] Projecting the context tokens and the text features of each sentence into a unified feature space through a linear projector to generate content tokens;

[0179] The text features of each sentence are input into the autoregressive language model and combined with the content tokens to generate colloquial text.

[0180] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0181] Segment the written text into sentences and extract text features of each sentence using a pre-trained language model;

[0182] Based on the contextual attention mechanism, the text features of each sentence and the spoken instructions are combined to generate context tokens;

[0183] Projecting the context tokens and the text features of each sentence into a unified feature space through a linear projector to generate content tokens;

[0184] The text features of each sentence are input into the autoregressive language model and combined with the content tokens to generate colloquial text.

[0185] It should be noted that the above functions or steps that can be implemented by the computer-readable storage medium or computer device can refer to the relevant descriptions on the server side and the user side in the aforementioned method embodiment. To avoid repetition, they will not be described one by one here.

[0186] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0187] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0188] It should be noted that if software tools or components other than those of the Company appear in the embodiments of the present application, they are only used for illustration and do not represent actual use. The above-described embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the above-mentioned embodiments, a person of ordinary skill in the art should understand that the technical solutions described in the above-mentioned embodiments can still be modified, or some of the technical features thereof can be replaced by equivalents; and these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A method for generating colloquial text, characterized in that: The following steps are involved: Segment the written text into sentences and extract text features of each sentence using a pre-trained language model; Based on the contextual attention mechanism, the text features of each sentence and the spoken instructions are combined to generate context tokens; Projecting the context tokens and the text features of each sentence into a unified feature space through a linear projector to generate content tokens; The text features of each sentence are input into the autoregressive language model and combined with the content tokens to generate colloquial text.

2. The method for generating a colloquial text according to claim 1, wherein: The text features of each sentence are input into the autoregressive language model and combined with the content tokens to generate colloquial text, including: Acquiring multimodal auxiliary data, and extracting multimodal features from the multimodal auxiliary data; Fusing the multimodal features with the text features of each sentence to generate fused features; The fused features are input into an autoregressive language model and combined with the content tokens to generate spoken text.

3. The method for generating a colloquial text according to claim 1, wherein: Based on the contextual attention mechanism, the text features of each sentence and the spoken instructions are combined to generate contextual tokens, including: Extracting semantic information from each sentence, wherein the semantic information is used to represent contextual associations between sentences; generating a query matrix based on the spoken instructions; Based on the semantic information and text features of each sentence, a key matrix and a value matrix are generated, wherein the key matrix is ​​used to represent the association between sentences, and the value matrix is ​​used to store the content information of the sentences; Analyzing the correlation between the query matrix and the key matrix through a context attention mechanism, and generating an attention weight for each sentence based on the correlation; Based on the attention weights of each sentence, the context tokens are generated in conjunction with the value matrix.

4. The method for generating a colloquial text according to claim 1, wherein: The context tokens and the text features of each sentence are projected into a unified feature space through a linear projector to generate content tokens, including: Inputting the context tokens and the text features of each sentence into a linear projector; Apply a context weight matrix to the context tokens and a text weight matrix to the text features for weighted processing respectively; The weighted context tokens and text features are projected into a unified feature space to generate content tokens.

5. The method for generating a colloquial text according to claim 1, wherein: Segment the written text into sentences and use a pre-trained language model to extract text features for each sentence, including: Obtaining a written text and segmenting the written text into a plurality of sentences based on punctuation marks, line breaks, or language logic; Input each sentence into a pre-trained language model, which processes the sentence based on context modeling and semantic representation; The pre-trained language model is used to extract feature vectors from each sentence to obtain text features of each sentence, wherein the text features include semantic, grammatical and contextual information of the sentence.

6. The method for generating a colloquial text as claimed in claim 1, wherein: Before feeding the text features of each sentence into the autoregressive language model, it also includes: Obtaining training data including natural language text and paralinguistic symbols, wherein the paralinguistic symbols represent the type and intensity of non-linguistic features; Initializing an autoregressive language model, and inputting the training data into the autoregressive language model; Processing text context and paralinguistic symbols in training data through the autoregressive language model, identifying paralinguistic symbols and corresponding intensities, and processing non-linguistic features corresponding to the paralinguistic symbols; Compare the output generated by the autoregressive model with the labels in the training data to determine the error; Based on the error, adjusting parameters of the autoregressive language model using a back-propagation method; Through multiple rounds of training and parameter optimization, the error is gradually reduced to obtain a trained autoregressive language model.

7. The method for generating a colloquial text according to claim 1, wherein: The text features of each sentence are input into the autoregressive language model and combined with the content tokens to generate colloquial text, including: Inputting the text features of each sentence and the content token into the autoregressive language model; The autoregressive language model gradually generates text output according to the text features and content tokens; In each generation process, the output generated in the previous step and the context information in the content token are combined to generate a new output; When the autoregressive language model completes the generation of the entire sequence, the spoken text corresponding to the spoken instruction is output.

8. A device for generating colloquial text, characterized in that: The colloquial text generating device comprises: A feature extraction module is used to segment the written text into multiple sentences and extract the text features of each sentence using a pre-trained language model; A context attention mechanism module, which is used to generate context tokens based on the context attention mechanism by combining the text features of each sentence and the spoken instructions; A feature projection module, used for projecting the context token and the text features of each sentence into a unified feature space through a linear projector to generate a content token; The spoken text generation module is used to input the text features of each sentence into the autoregressive language model and combine them with the content tokens to generate spoken text.

9. A computer device, characterized in that: The computer device includes a memory, a processor, and a colloquial text generation program stored in the memory and executable on the processor. When the colloquial text generation program is executed by the processor, the steps of the colloquial text generation method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The storage medium stores a spoken text generation program, and when the spoken text generation program is executed by the processor, the steps of the spoken text generation method according to any one of claims 1 to 7 are implemented.

Images