Stylized intelligent dialogue method based on retrieval enhancement generation

Abstract

The invention discloses a stylized intelligent dialogue method based on retrieval enhancement generation, which belongs to the field of intelligent dialogues, and comprises the following steps: firstly, receiving text or dialogue data uploaded by a user, analyzing and preprocessing, extracting style features from multiple dimensions such as vocabularies, syntax, emotion and rhetorical by a style definition module, and generating style enhancement cue words; then, a retrieval enhancement module recall related fragments from the knowledge base by using vector retrieval and a graph neural network to ensure content accuracy; the big language model is combined with the cue words and the knowledge fragments to generate answers which conform to specified styles and keep semantic consistency; and finally, through a GPT-SoVITS voice synthesis model, the answer is converted into personalized voice in real time, and the personalized voice is output in a streaming mode. The system supports user-defined styles and low-delay voice interaction, can be deployed on edge equipment, and is suitable for scenes such as education, text travel and customer service.

Description

Technical Field

[0001] This invention belongs to the field of intelligent dialogue, and in particular relates to a stylized intelligent dialogue method based on retrieval enhancement generation. Background Technology

[0002] Currently, intelligent dialogue systems have become widespread across multiple fields thanks to advancements in natural language processing and deep learning technologies. Whether it's chatbots on social media, automated response systems in enterprise customer service, or voice assistants on mobile phones, these applications demonstrate strong practicality. Existing technical solutions typically focus on two core metrics: the accuracy of the responses and the speed of feedback. Engineers implement these functions either through carefully designed dialogue rules or by utilizing intelligent models trained on massive amounts of data.

[0003] However, when we actually use these systems, we often feel a certain disconnect—they always respond with fixed phrases, failing to truly understand each person's unique needs, let alone perceive emotions and provide empathetic responses like humans do. This mechanical, conversational experience makes it difficult to feel a natural and fluent exchange. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides a stylized intelligent dialogue method based on retrieval enhancement generation, comprising:

[0005] Obtain user input text; extract style features based on the user input text using the style definition module; construct style-enhanced prompt words based on the style features;

[0006] Based on the style-enhanced prompts, relevant knowledge fragments are retrieved from the knowledge base using the retrieval enhancement module;

[0007] Based on the relevant knowledge fragments, stylized text is generated through a large language model; based on the stylized text, stylized speech output is generated through a speech synthesis module.

[0008] Optionally, obtaining the user input text includes:

[0009] Receive raw text uploaded by users via web front-end or embedded terminal;

[0010] The original style text is preprocessed by word segmentation, stop word removal, and punctuation standardization to obtain preprocessed text;

[0011] Based on the preprocessed text, a globally unique key-value pair of user ID and style tag is generated, and the key-value pair is stored in the user style association table of the MySQL relational database, with the user ID as the primary key and the style tag index as the foreign key.

[0012] Optionally, the extraction of style features through the style definition module includes:

[0013] Perform part-of-speech tagging on the preprocessed text and statistically analyze the frequency distribution of nouns, verbs, adjectives, adverbs, and interjections;

[0014] Calculate the frequency ratio (CFR) of content words to function words. If the CFR is ≥ 2.8, it is marked as formal writing; if the CFR is < 1.2, it is marked as spoken writing.

[0015] Extract the word length type and syllable type of the top k words by length, and construct a word length feature vector;

[0016] Modal words and idioms are matched using regular expressions. The top m modal words and top n idioms are selected in descending order of TF-IDF and concatenated into a lexical-level style feature vector V_lex.

[0017] Optionally, the extraction of style features through the style definition module further includes:

[0018] The preprocessed text is segmented into sentence sequences using periods, question marks, and exclamation marks as delimiters;

[0019] Count the total number of sentences N, the total number of characters L, and the average sentence length L / N; divide sentences into long and short sentences according to a 20-word threshold, calculate the proportion of short sentences Ps, if Ps>60%, mark it as telegraphic text, if 40%≤Ps≤60%, mark it as mixed text, if Ps<40%, mark it as long sentence text;

[0020] The cnsenti library is used to calculate the sentiment score for each sentence and then sum them up with weights to output the dominant sentiment label.

[0021] Use the finely tuned MacBERT whole-sentence classifier to determine the proportion of whole sentences, Pr.

[0022] Output the main rhetorical label using the finely tuned BERTrhe rhetorical classifier;

[0023] The results are concatenated into a sentence-level style feature vector.

[0024] Optionally, sentence-level analysis of the user input text includes:

[0025] Analyze the ratio of question marks to exclamation marks to determine the emotional tone of the text;

[0026] Use a pre-trained model to determine the sentence structure and main rhetorical devices in a text.

[0027] Optionally, the build style enhancement prompts include:

[0028] Based on the lexical-level style feature vector and the sentence-level style feature vector, a structured style definition prompt is generated;

[0029] The style definition prompt is combined with a destylized parallel corpus to form style-enhanced prompt words.

[0030] Optionally, retrieving relevant knowledge fragments from the knowledge base via the retrieval enhancement module includes:

[0031] Vectorize user input questions;

[0032] Based on the vectorization results, relevant documents are retrieved from the knowledge base through dense retrieval;

[0033] Based on the relevant documents, context-enhanced information is obtained through graph neural network reasoning.

[0034] Optionally, generating stylized text through a large language model includes:

[0035] The style-enhanced prompts are concatenated with the context-enhanced information to form input prompts;

[0036] Based on the input prompts, text content conforming to the target style is generated using a large language model.

[0037] On the other hand, the present invention also provides an electronic device including a memory, a processor, and a computing program stored in the memory and executable on the processor, wherein the processor implements the method when executing the computing program.

[0038] On the other hand, the present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method.

[0039] Compared with the prior art, the present invention has the following advantages and technical effects:

[0040] This invention constructs a comprehensive system integrating style transfer and intelligent dialogue. Through the collaborative work of three core modules, it realizes a complete technical chain of Chinese text style transfer, knowledge-enhanced dialogue, and personalized speech synthesis. The core style transfer module (TSD) adopts a deeply decoupled architecture, achieving high-quality transfer of various literary styles while maintaining the semantic integrity of the original text. Simultaneously, the system innovatively combines the RAG (Retrieval-Enhanced Generation) architecture with the style transfer module. When a user initiates a dialogue, the system first retrieves relevant knowledge fragments from a professional corpus through vector retrieval. Then, the style transfer module performs style adaptation processing on these contents, and finally, a large language model generates a response that is both professional and accurate, conforming to the target style. The speech interaction module uses the latest GPT-SoVITS model as its synthesis engine, enabling personalized customization of speech style. Users only need to provide 30 seconds of reference audio, and the system can extract its timbre, intonation, and rhythm features to generate speech output with a specific style. For performance evaluation of the semantic preservation and style transfer accuracy of the style transfer module, this invention created a parallel corpus based on a large language model and tested semantic preservation and semantic coverage using certain standards, effectively demonstrating the high efficiency of the TSD module in style transfer. Furthermore, this invention designs a user-friendly intelligent dialogue system based on a front-end / back-end + large model deployment architecture. Users can freely upload stylized text and stylized speech to converse with the intelligent robot. For traditional scenarios such as classrooms and scenic spots with more variable scenarios, this invention also designed a voice dialogue robot based on Orange Pie for real-time interaction with users, meeting the stylized intelligent dialogue needs in multimodal and multi-application scenarios. Attached Figure Description

[0041] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0042] Figure 1 This is a schematic diagram of the overall framework of the style definition module in an embodiment of the present invention;

[0043] Figure 2 This is a schematic diagram of the retrieval enhancement module framework according to an embodiment of the present invention;

[0044] Figure 3 This is a schematic diagram of the speech synthesis module framework according to an embodiment of the present invention. Detailed Implementation

[0045] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0046] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0047] Example 1

[0048] This embodiment provides a stylized intelligent dialogue method based on retrieval enhancement generation, including:

[0049] Obtain user input text; extract style features based on the user input text using the style definition module; construct style-enhanced prompt words based on the style features;

[0050] Based on the style-enhanced prompts, relevant knowledge fragments are retrieved from the knowledge base using the retrieval enhancement module;

[0051] Based on the relevant knowledge fragments, stylized text is generated using a large language model; based on the stylized text, stylized speech output is generated using a speech synthesis module, specifically:

[0052] This embodiment innovatively proposes a large-scale model stylization transfer framework based on a dual-module collaborative mechanism. This framework achieves dual assurance of stylization effectiveness and knowledge accuracy through the organic cooperation of a style definition module and a retrieval enhancement module. In the field of natural language processing, how to achieve effective style transfer based on large-scale pre-trained models while ensuring the accuracy of domain knowledge has always been a critical problem that urgently needs to be solved. The solution proposed in this embodiment provides a new approach to this challenge through the collaborative work of two core modules. The style definition module employs a multi-level feature extraction method to model the target style from multiple dimensions, including vocabulary selection, sentence structure, and rhetorical style. The retrieval enhancement module focuses on solving the ambiguity problem of large models in specific domain knowledge. This module constructs a dynamically expandable external knowledge base and uses dense retrieval technology to obtain the most relevant knowledge fragments to the current dialogue in real time.

[0053] The two modules complement each other through a carefully designed collaborative mechanism. During the generation process, the style definition module controls the overall style direction, while the retrieval enhancement module ensures the accuracy of key information. The system employs a dynamic weight allocation strategy, automatically adjusting the influence weight of the two modules based on the user's needs in the current conversation: when the conversation involves professional knowledge, the system enhances the participation of the retrieval module; when the conversation focuses more on style expression, it gives the style module greater freedom in generation. This flexible collaborative approach allows the system to produce text with a distinctive style while ensuring the reliability of professional content.

[0054] The emergence of Large Language Models (LLMs) has fundamentally changed some NLP research paradigms. The limitations of past stylization methods based on semantic structure decomposition and recombination are obvious. Large language models have demonstrated powerful flexibility and adaptability in text-type tasks. Shao et al. suggested using knowledge-based visual question answering heuristics in LLMs. Their work shows that, through carefully designed heuristics, large language models can be effectively guided to generate outputs that meet specific requirements. This hint engineering approach is also applicable to text style transfer tasks—this embodiment can construct a composite prompt containing style descriptions, example text, and transformation rules, enabling the model to accurately capture the feature elements of the target style. Therefore, this embodiment can specify the generation method of LLMs. The comprehensive application of these technologies allows contemporary LLMs to accurately achieve style transfer across various dimensions, from standard texts to classic works and specific figures' arguments, while maintaining the semantic integrity of the original text, thus achieving good style transfer results.

[0055] The style definition module is divided into three parts: a stylized database, a destylated parallel dataset, and a TSD module. The overall framework is as follows: Figure 1 As shown, the first part constructs a stylized corpus based on a certain amount of unstructured text data. These works span from the 1920s to the 1990s, covering the major schools of thought in modern and contemporary Chinese literature. To ensure the diversity and representativeness of the data, high-discrimination style samples are dynamically selected from open-domain texts and supplemented by manual verification. In the second stage, this embodiment performs semantically preservative destylization on the original stylized corpus based on a large language model, generating corresponding neutral style parallel texts. While minimizing the correlation of style features, it ensures that the generated neutral texts are highly consistent with the source texts in terms of information integrity. This parallel corpus is not only used for the style transfer performance evaluation of subsequent models, but also serves as a benchmark dataset for style decoupling tasks. The third part is the style definition TSD module. After processing by the stylized corpus, the style of its sentences and words is summarized and defined by the TSD module. Subsequently, style enhancement prompts are generated to guide LLMs to generate corresponding stylized texts based on the parallel corpus. The model's performance is then evaluated by comparing it with the original stylized corpus, and its semantic preservation is evaluated by comparing it with the parallel corpus.

[0056] In the third part, this embodiment proposes a hierarchical style feature extraction method. By analyzing the statistical distribution of stylized corpora in dimensions such as vocabulary selection, syntactic structure, and rhetorical strategies, a style representation vector is constructed. Given that the text acquisition for the style definition part is relatively random and extensive, this embodiment considers the summarized style features as the overall style features of the article.

[0057] Subsequently, based on the extracted style representations, this embodiment designs style-enhancing cue words according to the style representation vectors generated in the third part, guiding LLMs to generate style-intensified text on a parallel corpus. Finally, the distribution similarity between the generated text and the original styled corpus is compared, as well as the semantic preservation compared with the parallel corpus.

[0058] The TSD module calculates styles for the style definition section at the lexical and sentence levels, generating complete style generation hints for LLMs. Research shows that words form the cornerstone of text construction and play a crucial role in shaping linguistic features. At the lexical level, this embodiment conducts an in-depth analysis of the text's word form from multiple perspectives, including parts of speech, word length and syllables, modal particles, and idioms.

[0059] Part-of-speech distribution feature analysis:

[0060] The frequency and combination patterns of different parts of speech (such as nouns, verbs, adjectives, adverbs, etc.) vary across different text styles. For example, nominalized structures account for a significantly higher proportion in academic texts than in colloquial texts, while the high frequency of interjections is a hallmark of an emotional style. Therefore, the distribution characteristics of parts of speech can be used as one of the dimensions for defining style.

[0061] This module employs an improved jieba2 word segmentation database for part-of-speech tagging and innovatively introduces the content-function dichotomy theory. By defining a content word set (verbs V, nouns N, adjectives A, etc.) and a function word set (prepositions P, conjunctions C, particles U, etc.), and calculating their relative frequency ratio (CFR), it serves as a quantitative indicator of the text's formalization level. Research has found that this indicator effectively distinguishes different writing styles. For example, the CFR value of scientific literature is generally between 2.8 and 3.5, while the CFR value of everyday conversation is below 1.2. Furthermore, the CFR of academic texts is significantly higher than that of spoken texts.

[0062] Word length and syllable word features are inferred in this embodiment by statistically analyzing the length of words in the entire article. The word length type and syllable word type with the specified values ​​are used as type representatives, as shown in formula (1).

[0063] f l =argminK i {I l lenw i ))};

[0064] Where len(w) i f is used to calculate the length of a word. l This represents a function to calculate words of various lengths. This example selects the top K...i The word length type is used as the main word length type. Similarly, the definition method of the syllable word type is the same as above.

[0065] Analysis of the modal expression system: In lexical style analysis, modal features are an important dimension for characterizing the subjectivity and emotional tendency of the text. In this embodiment, based on existing authoritative modal word libraries (such as the Chinese Encyclopedia Semantic Library, HowNet Sentiment Dictionary, etc.), a set of multi-level regular expression rule sets are constructed to accurately match modal expressions in the text (such as "possibly", "should", "ah", "ba"), and the matching results are sorted in descending order of occurrence frequency, and the top Kmodal high-frequency words are extracted to form a modal feature vector:

[0066] fmodal = argTopKmodal{free(Re(w i , Dmodal))};

[0067] For the idiom part, an open-source idiom library (such as idiom-database) is used to match four-character idioms in the text (such as "亡羊补牢", "画龙点睛") through regular expressions. The top Kidiom high-frequency idioms are extracted to form an idiom feature vector:

[0068] fidiom = argTopKidiom{freq(Re(w t , Didiom))};

[0069] Research at the sentence level shows that sentences, as the basic expression units of the text, play a decisive role in constructing language style and conveying the author's intention. At the sentence level, in this embodiment, the syntactic features of the text are systematically analyzed from multiple dimensions such as sentence length distribution and long / short sentence patterns, quantification of emotional color, and modal system expression, and the sentence patterns of different style texts are revealed through computational linguistics methods.

[0070] Total number of sentences (S total ):

[0071] Total number of characters (C total ): C total = ||{c j |c j ∈ [U+4E00,U+9FFF]}||

[0072] Average sentence length

[0073] Existing research has defined methods for classifying sentences by length. Scholars categorize sentences into short or long sentences based on a threshold of 20 words. This embodiment uses this method to statistically analyze the number and proportion of sentences of varying lengths in the text. Next, the sentence style is determined by the proportion of short sentences. For example, a short sentence style (>60%) conforms to the characteristics of a "telegraphic style," producing a brisk rhythm and exhibiting conciseness, clarity, and crispness. A mixed style (40-60%) indicates that the author's writing combines short and long sentences, often used to express complex emotions and creating a sense of rhythm and cadence. A long sentence style (<40%) indicates that the author's writing contains more long sentences, richer in content, more specific in narration, more thorough in reasoning, and more emotionally charged.

[0074] Sentence emotional tone analysis: Punctuation counting is used to define the emotional tone of an article. For example, the percentage of punctuation marks with strong emotional connotations, such as "?" and "!", is counted to analyze the overall emotional composition of the article. A threshold of 0.15 is used. When the usage rate of question marks exceeds this threshold, it indicates that the author's overall grammatical style frequently uses them to guide reader thought or express rhetorical questions, enhancing reader participation and interaction. When the usage rate of exclamation marks exceeds this threshold, it indicates that the author often expresses strong emotions, such as praise or amazement, making the article more emotionally charged. The emotional tone of the article is evaluated using the Emotion function in cnsentilibrary5. This function employs a multi-level mixed analysis framework, outputting a score for each sentence on seven emotions (good, happy, sad, angry, fearful, disgusted, and surprised). A weighted voting mechanism is used to calculate the sentence's emotional score.

[0075]

[0076] To determine the main mood of the article, a deep learning model was introduced for the sections on both complete and fragmented sentences. A Chinese corpus of annotated complete and fragmented sentences was designed, with annotations conforming to linguistic definitions. For example, complete sentences are independent clauses with a subject, verb, and object, such as "He pushed open the door and walked into the room." Fragmented sentences are non-subject-verb structures or dependent clauses, such as "Pushed open the door, quietly." The MacBERT model was fine-tuned to obtain a text binary classifier that can determine whether the article structure is predominantly fragmented or complete. A similar strategy was adopted for the rhetoric section. Sentences with rhetorical devices such as metaphor, personification, hyperbole, quotation, parallelism, irony, rhetorical question, statement, and binaryity were randomly generated based on the large model to form a fine-tuned dataset. Subsequently, this embodiment successfully obtained the Chinese rhetoric multi-classification model BERTrhe. It was then fine-tuned to obtain a rhetoric classifier to summarize the most commonly used rhetorical devices in the article. Combining the above style definitions at the sentence and word levels, this embodiment comprehensively completed the definition of the article style.

[0077] The experimental procedure included: This embodiment selected five groups of literary passages with significant stylistic differences, distinct stylistic attributes, and covering classic Chinese literary works as target style texts. The data was then cleaned and segmented by period to prepare for subsequent analysis of stylization performance and semantic preservation. The study also used a large language model to de-stylize the text. The prompt "Please rewrite the following text in a neutral style, retaining all factual content and basic emotional inclination, but removing the author's unique literary style features. Requirements: 1) Replace distinctive vocabulary with general vocabulary; 2) Simplify rhetorical devices; 3) Standardize sentence structure" yielded a de-stylized text.

[0078] The evaluation criteria are primarily based on two metrics: BLEU and BERTScore. Both metrics can evaluate text similarity. BLEU mainly assesses the surface lexical matching between the generated text and the reference text, with BLEU-1 reflecting lexical similarity and BLEU-2 reflecting phrase combination similarity. BERTScore includes Precision (semantic coverage index between the generated and reference texts, and Recall retention of the reference text in the generated text) and F1score (comprehensive balance score), which can be used to evaluate the deep semantic similarity between the generated and reference texts, suitable for tasks requiring contextual understanding. This embodiment evaluates the stylization performance and semantic retention of the generated text against the original stylized text and the destylized text based on these two metrics. A higher mean evaluation index indicates better performance. This embodiment aims to achieve the highest possible similarity between the generated text and the initial stylized text at the levels of phrase structure and word selection, and to achieve the highest possible similarity between the generated text and the destylized text in semantic expression and other comprehension tasks.

[0079] Furthermore, considering the randomness of text generated by LLMs, this embodiment calculates the final average index result by transferring each type of text ten times.

[0080] Experiments show that the TSD style transfer module achieves excellent results in style transfer and semantic preservation. Regarding the accuracy of style transfer, this experiment, based on the Tongyi Qianwen large language model, after style transfer of the parallel corpus by the style transfer module, compared with the original corpus, found that the lexical accuracy index BURT-1 reached over 40%, while retaining 80% of the semantic restoration (as shown in Table 1). Through in-depth analysis of typical examples, this embodiment also discovered some interesting patterns. In the case of Lu Xun's style transfer, the model not only successfully transformed the iconic progressive sentence structure but also achieved style adaptation through word substitution (such as changing "road" to "rules"), while fully preserving the philosophical connotation of the original text. These successful cases demonstrate that the TSD module truly grasps the unique language habits and expressions of different writers. Table 2 shows the experimental results.

[0081] Table 1

[0082]

[0083] Table 2

[0084]

[0085] The retrieval enhancement module includes: This system has built an intelligent dialogue information retrieval engine for articles with customizable target styles. It adopts the Retrieval-Augmented Generation (RAG) technology framework to achieve an organic combination of knowledge retrieval and generation models.

[0086] The first step in the RAG process is retrieval, which involves retrieving question-related information from a pre-established knowledge base. The purpose of this step is to provide useful contextual information and knowledge support for the subsequent generation process. Augmentation in RAG involves using the retrieved information as contextual input to the generative model (i.e., the Large Language Model) to enhance the model's understanding and ability to answer specific questions. This step aims to integrate external knowledge into the generation process, making the generated text content richer, more accurate, and more user-relevant. Through the augmentation step, the LLM model can fully utilize information from the external knowledge base. This is followed by combining the LLM with the generated answer to meet user needs. The generator uses the retrieved information as contextual input and combines it with the Large Language Model to generate text content, such as... Figure 2 As shown.

[0087] In terms of technical approach, this embodiment, based on an in-depth analysis of the characteristics of dialogue in classical literature, adopts a hybrid architecture of retrieval-enhanced generation (RAG). Because dialogue in classical literature requires handling more complex contextual factors such as language habits of specific historical periods, dynamic changes in character relationships, and rich literary rhetoric, a multi-layered semantic understanding framework was developed that can simultaneously handle classical usage at the word level, rhetorical features at the sentence level, and narrative structure at the text level. In its implementation, the research innovatively transforms traditional literary analysis methods into computable feature representations, such as quantifying character traits as bias terms of attention mechanisms and modeling narrative rhythm as temperature parameters of the generative model.

[0088] The dialogue management module is designed with full consideration of the narrative characteristics of literary works. Since dialogue between literary characters often needs to maintain long-term contextual consistency, a dialogue state tracking mechanism based on memory networks is introduced. This mechanism dynamically maintains contextual information across three dimensions: the current dialogue topic's development trajectory, mentioned literary allusions and metaphors, and dynamic changes in character relationships. This information not only affects the current response generation but also influences subsequent retrieval strategies through latent variables. For example, when the system detects that a dialogue involves the "Havoc in Heaven" plot, it automatically increases the retrieval weight of relevant paragraphs and adjusts the generation style to match the dramatic characteristics of the plot.

[0089] This embodiment also includes a stylized dialogue client platform. Specifically, this embodiment uses a web development technology stack to create a stylized intelligent dialogue client platform that allows users to customize text styles and supports voice dialogue by matching corresponding character voice characteristics. This platform implements microservice communication based on the mainstream backend framework Spring Boot and the frontend framework Theamleaf. A customized style definition interface allows users to upload desired style text or character historical dialogue information through a frontend text box; a stylized intelligent chat interface allows users to freely converse with a stylized robot through message input boxes and buttons.

[0090] Furthermore, to ensure high platform availability and the ability to retain user data, a database and tables were designed for each user's unique identifier (username). These databases store each user's style definition identifier and corresponding style suggestion enhancement prompts, ensuring that the previously defined text style can be invoked each time a user logs in, without the need for re-uploading. The backend service layer uses the HTTP protocol to call the AI ​​model, namely the style definition module and the LLMs large model, to complete the supporting functions for the corresponding operations.

[0091] Spring Boot simplifies the cumbersome XML configuration files used in traditional web development, such as servlets. It also integrates with the Maven package manager, making it much easier to reference external packages in Java development. Simply importing Maven coordinates enables the use and integration of external resources. Package version numbers can be defined easily, and Maven configuration is simplified by modifying the POM file provided by Spring. Spring Boot's rich dependency modules make integrating various common technologies effortless; from database access to security authentication, developers can quickly find ready-made solutions, helping them resolve dependency issues quickly. Furthermore, Spring Boot's rapid development framework, relying on full annotation-based development, effectively solves problems such as code redundancy and cumbersomeness.

[0092] In the platform's implementation, Spring Boot plays a crucial core role. For example, it provides a RESTful API service for front-end and back-end interaction, acting as a bridge between them. Various requests are orchestrated in a unified and standardized manner, allowing the back-end to parse the request type, parameters, and body from the front-end. The back-end, through its controller and service layers, orchestrates and coordinates the various components, while also integrating a database connection pool and underlying database based on Mybatis for database maintenance and data persistence.

[0093] In the front-end design process, the front-end page design was implemented based on the Thymeleaf framework and connected to the back-end control layer components via routing paths. The overall page layout adopted Bootstrap styles, such as font icons, dropdown menus, and navigation styles. Modularization of the front-end page was implemented; for example, the top and side navigation bars were designed as separate modules, achieving "define once, use everywhere" functionality and reducing code duplication. Furthermore, through vivid dynamic page rendering, back-end data was effectively combined with the front-end template to achieve a user-friendly HTML interface.

[0094] In the database design process, this embodiment uses the basic paradigm of relational databases to design corresponding relational data tables for users, passwords, style identifiers, and style definition prompts, ensuring the coupling relationship between primary keys and foreign keys. The platform also follows standard database design principles, including reasonable table structure design, appropriate indexing strategies, and effective data partitioning schemes. In Mybatis-based database calls, this embodiment uses the Druid database connection pool and leverages the synergy of SQL language annotations and configurations to complete user operations for modifying and maintaining the database.

[0095] One of the most innovative features of this platform is its support for users to upload custom-styled text and stylized character dialogue materials. As shown in the image, users can select the "Style Definition" section in the left navigation bar. The main interface on the right will display the existing style definition icons. Hovering the mouse over the icon will display the corresponding style definition prompt. There is a black "Add Style" button at the top. The system will pop up a hover box for adding a style. Users can enter the stylized text content in the text box here and enter the stylization icon, such as "Lu Xun style", "Three-Body style", etc.

[0096] When a user uploads stylized text materials (such as Lu Xun's essays, excerpts of Lin Daiyu's dialogues, etc.), a POST request is sent to the server. The text content, text identifiers, and the user's unique identifier are encapsulated into a JSON data stream, and the system initiates a multi-layered style analysis process. First, text preprocessing is performed, including word segmentation, stop word removal, and punctuation standardization, ensuring the input text is standardized. Then, the TSD module within the style definition module described above extracts style features from the text. These features include surface features such as lexical and sentence structure, as well as deeper style elements such as rhetorical devices and emotional inclinations. This ensures that the final extracted style features accurately reflect the stylistic characteristics of the original text.

[0097] The extracted style features are transformed into structured style definition prompts, which summarize stylistic features at the sentence, vocabulary, and sentiment levels. These prompts include both quantitative indicators and qualitative analysis, collectively forming a complete style profile. The generated style prompts are stored in a MySQL database in a specific format, including fields such as style identifier, creation time, style feature vector, and corresponding user ID, facilitating subsequent retrieval and use. Simultaneously, users can clearly query and view their customized text style summaries on the front-end interface and specify stylization targets in subsequent processes.

[0098] From a technical framework perspective, users access the server address to obtain front-end interface information, initiate a POST request through specific operations, the back-end responds to and processes the user request, and at the same time, based on the network transmission protocol, sends the request carrying the target style text to the style definition module AI server for processing. The Python service runs a deep learning model to perform style analysis, returns a style prompt, and then calls the MySQL database to store the structured style definition.

[0099] Stylized chat is a core feature of this platform. After generating and storing a stylized prompt in the style definition module, users can specify their preferred style in this module. Clicking the dropdown menu above the chat box will retrieve the stylized identifier matching the user's ID from the backend server and display it on the front end, allowing the user to select the appropriate style. After entering the question and clicking the send button, the question content along with the stylized identifier will be transmitted to the backend server. The server will format the stylized prompt and question accordingly, inject it into the context of the large model's question, guide the large model to generate the corresponding stylized text, and return it to the front-end chat box page.

[0100] During the question-and-answer process, the system automatically records and stores the content of each question and answer, so that the historical content can be seen in time every time the page is refreshed. Users can also upload the historical dialogue content as new stylized text to the stylization definition module as a means of collecting stylization enhancement datasets.

[0101] This embodiment implements a real-time intelligent dialogue robot developed based on OrangePie AIpro. By integrating cloud-edge-device services, such as AI model services for speech recognition, stylized large models, and speech synthesis, a complete intelligent dialogue practical application solution is constructed. The system uses WebSocket and HTTP protocols as its core communication framework, achieving low-latency, high-concurrency bidirectional data interaction capabilities. This allows users to engage in smooth role-playing dialogues with the robot using natural speech. For voice acquisition, the system uses a shell script and the audio processing tool SoX to acquire audio through a microphone array, setting a recording stop condition—that is, stopping recording after a certain time interval without sound pickup. This allows for automatic acquisition of audio containing a complete question without manual pausing of recording. To improve recognition accuracy, the system selects a 16kHz sampling frequency, which also meets the format requirements of the speech recognition module. At the initial moment of the polling dialogue, the user sets the dialogue role via voice. The recorded audio is temporarily stored locally and then called to the speech recognition API for text recognition. The recognized role name is then transmitted to the role matching module.

[0102] The role matching module employs fuzzy matching. The system has a pre-set corpus containing popular characters and background knowledge such as Lu Xun, Sun Wukong, and Nezha. Combined with semantic similarity calculation, it can match the user-specified role name. In subsequent polling, the user can ask questions about the role. The system then associates the questions with the current role's characteristics and sends a request to the stylized large language model server. The stylized server module then generates dialogue prompts that match the role's personality and guides the large model to generate the corresponding response text. Since the responses in a real-time dialogue system should not be too long, this embodiment incorporates word count restrictions within the context of the prompts, ensuring concise and accurate responses.

[0103] In the speech synthesis stage, the system selects corresponding voice segments from the speech corpus based on the matched character characteristics. After receiving the text response, the speech synthesis server combines the character's unique intonation, rhythm, and emotional features to generate natural and fluent speech output. To ensure a consistent interactive experience, the server segments the large language model's response by punctuation marks, synthesizes the speech segments, and asynchronously sends them to the development board for reception. The development board system uses WebSocket technology for streaming transmission and leverages the Sounddevice audio library to implement multi-threaded asynchronous audio queue addition and playback. This allows speech synthesis and playback to occur almost in real time.

[0104] The entire system runs on a customized Linux system from OrangePie AIpro, utilizing a multi-threaded architecture to achieve parallel processing of each module. The system also features a robust error handling mechanism, including automatic reconnection after network interruptions and intelligent follow-up questions when voice recognition fails, ensuring stable service across various usage scenarios. Through this collaborative hardware and software design, a natural and fluid role-playing dialogue experience between the user and the robot is ultimately achieved.

[0105] This embodiment is based on the GPT-SoVITS large-scale speech synthesis model. It uses 1 to 2 minutes of personalized audio to fine-tune the GPT-like model and the SoVITS model, and then injects timbre feature information based on the few-shots method to realize a customizable stylized speech generation interface.

[0106] The GPT-SoVITS model uses the Hubert module to obtain phoneme features of the target speech. Hubert, as a self-supervised speech representation learning method, has the advantage of constructing a discrete representation of the speech signal. It uses an autoregressive method (GPT-like model) to predict the phoneme features of the target text and extracts the timbre feature (prompt) of the target speech, injecting it during Mel spectrum decoding. The timbre prompt is injected into the decoder through a cross-attention mechanism. This dynamic injection method maintains timbre stability better than simple feature concatenation, making the speech synthesis effect closer to a real person's timbre and intonation. Figure 3 As shown.

[0107] Meanwhile, this embodiment has created a set of common character audio corpora, such as the voices of common characters in the Four Great Classical Novels and the voices of characters in the recent hit IP Nezha movie. It also uses WebSocket to implement sentence-by-sentence synthesis and asynchronous processing of streaming / receiving data to solve the problem of excessive latency in synthesizing long text audio.

[0108] Initially, it was found that the speech generation module was prone to latency when dealing with long text paragraphs or answers, which negatively impacted the user experience. Therefore, this embodiment designed a WebSocket-based streaming reception architecture. The server segments the long text, synthesizes the speech from each segment, and returns it to the client. The OrangePi client asynchronously receives and plays the speech data using a message queue. This allows the latency of speech generation for each long text to be controlled within the duration of the first sentence, effectively avoiding the latency problem of speech generation for long texts.

[0109] To verify the actual performance and user experience of the system, this embodiment conducted a comprehensive performance evaluation of the real-time role-playing system and demonstrated the system's performance through test data from multiple dimensions.

[0110] In the speech recognition stage, the system demonstrates high accuracy in quiet environments. In the role matching and stylized response generation stages, it accurately identifies the user-specified role and generates natural language responses that match the role's personality. It accurately understands and converts sentences with clear pronunciation or slight accents into text. The system possesses a certain level of contextual understanding; when faced with unclear pronunciation or incomplete sentences, it can intelligently infer based on the dialogue context, significantly improving the success rate of interaction and allowing users to enjoy a good interactive experience without consciously adjusting their speech. Tests show that the system achieves a 96% accuracy rate in matching classic characters such as "Sun Wukong" and "Lin Daiyu." The speech synthesis module performs exceptionally well, with customized speech synthesis based on the GPT-SoVITS model achieving excellent results in terms of timbre fidelity and naturalness.

[0111] After multiple rounds of optimization, the system's response speed has reached a practical level. Seamless integration of various functional components is achieved through modular design and parallel processing technology. The overall system response latency is controlled within 3 seconds, with speech recognition averaging 0.5 seconds, role matching and response generation 1 second, and speech synthesis and streaming 1.5 seconds. Streaming technology ensures that the user's wait time to hear the first reply is no more than 3 seconds, significantly improving the smoothness of the interactive experience. Expressive voice output and appropriate language style combine to create a realistic role-playing dialogue experience, giving users the feeling of truly interacting with virtual characters.

[0112] On the other hand, this embodiment also provides an electronic device, including a memory, a processor, and a computing program stored in the memory and executable on the processor, wherein the processor implements the method when executing the computing program.

[0113] On the other hand, this embodiment also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method.

[0114] The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A stylized intelligent dialogue method based on retrieval enhancement generation, characterized in that, include: Get the user's input text; Based on the user input text, style features are extracted through the style definition module; Based on the aforementioned style characteristics, style-enhanced cue words are constructed; Based on the style-enhanced prompts, relevant knowledge fragments are retrieved from the knowledge base using the retrieval enhancement module; Based on the relevant knowledge fragments, stylized text is generated through a large language model; based on the stylized text, stylized speech output is generated through a speech synthesis module.

2. The method according to claim 1, characterized in that, The acquisition of user input text includes: Receive raw text uploaded by users via web front-end or embedded terminal; The original style text is preprocessed by word segmentation, stop word removal, and punctuation standardization to obtain preprocessed text; Based on the preprocessed text, a globally unique key-value pair of user ID and style tag is generated, and the key-value pair is stored in the user style association table of the MySQL relational database, with the user ID as the primary key and the style tag index as the foreign key.

3. The method according to claim 1, characterized in that, The extraction of style features through the style definition module includes: Perform part-of-speech tagging on the preprocessed text and statistically analyze the frequency distribution of nouns, verbs, adjectives, adverbs, and interjections; Calculate the frequency ratio (CFR) of content words to function words. If the CFR is ≥ 2.8, it is marked as formal writing; if the CFR is < 1.2, it is marked as spoken writing. Extract the word length type and syllable type of the top k words by length, and construct a word length feature vector; Modal words and idioms are matched using regular expressions. The top m modal words and top n idioms are selected in descending order of TF-IDF and concatenated into a lexical-level style feature vector V_lex.

4. The method according to claim 1, characterized in that, The extraction of style features through the style definition module also includes: The preprocessed text is segmented into sentence sequences using periods, question marks, and exclamation marks as delimiters; Count the total number of sentences N, the total number of characters L, and the average sentence length L / N; divide sentences into long and short sentences according to a 20-word threshold, calculate the proportion of short sentences Ps, if Ps>60%, mark it as telegraphic text, if 40%≤Ps≤60%, mark it as mixed text, if Ps<40%, mark it as long sentence text; The cnsenti library is used to calculate the sentiment score for each sentence and then sum them up with weights to output the dominant sentiment label. The proportion of complete sentences was determined using a finely tuned MacBERT complete / incomplete sentence classifier. Output the main rhetorical label using the finely tuned BERTrhe rhetorical classifier; The results are concatenated into a sentence-level style feature vector.

5. The method according to claim 1, characterized in that, Sentence-level analysis of the user input text includes: Analyze the ratio of question marks to exclamation marks to determine the emotional tone of the text; Use a pre-trained model to determine the sentence structure and main rhetorical devices in a text.

6. The method according to claim 1, characterized in that, The build style enhancement prompts include: Based on the lexical-level style feature vector and the sentence-level style feature vector, a structured style definition prompt is generated; The style definition prompt is combined with a destylized parallel corpus to form style-enhanced prompt words.

7. The method according to claim 1, characterized in that, The process of retrieving relevant knowledge fragments from the knowledge base through the retrieval enhancement module includes: Vectorize user input questions; Based on the vectorization results, relevant documents are retrieved from the knowledge base through dense retrieval; Based on the relevant documents, context-enhanced information is obtained through graph neural network reasoning.

8. The method according to claim 1, characterized in that, The generation of stylized text through a large language model includes: The style-enhanced prompts are concatenated with the context-enhanced information to form input prompts; Based on the input prompts, text content conforming to the target style is generated using a large language model.

9. An electronic device comprising a memory, a processor, and a computing program stored in the memory and executable on the processor, characterized in that, When the processor executes the computing program, it implements the method of any one of claims 1-8.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1-8.

Images