Network response retrieval enhancement generation method fused with large language model, electronic equipment and computer readable storage medium

Abstract

A network response retrieval enhancement generation method fused with a large language model, an electronic device and a computer readable storage medium belong to the field of text data processing, are used for automatically generating network responses, and are technically characterized in that an inquiry appeal text and a prompt are input into the large language model, and inquiry appeals and types are output by the large language model; inputting similar cases and prompts of the appeal text into the large language model, and outputting a response normal form to the political appeal of the type in the similar cases by the large language model; a prompt is input into the large language model, the response normal form is generalized through the large language model, and a first framework suitable for appeal response is generated; the method for obtaining the similar cases of the appeal text comprises the following steps: calculating an appeal vector after keyword embedding is carried out on the appeal text; determining a target sub-case library according to a cosine distance between the appeal vector and each sub-case library center based on appeal content clustering; and determining similar cases according to cosine distances between the appeal vectors and the appeal vectors of the cases in the target sub-case library.

Description

Technical Field

[0001] The present invention belongs to the field of text data processing and relates to a network response retrieval enhancement generation method integrating a large language model, an electronic device and a computer-readable storage medium. Background Art

[0002] In the digital age, as a core interactive channel connecting organizations and the public, online responses have become a critical component in improving governance effectiveness, optimizing user experience, and shaping organizational image. With the widespread adoption of internet technology and the intelligent upgrades of various online service platforms (such as government platforms, corporate customer service, social media, and community forums), the public's activeness in providing consultations, feedback, complaints, and suggestions through online channels has increased significantly, and massive amounts of fragmented information have flooded into various platforms. This high level of engagement places unprecedented demands on the timeliness, professionalism, standardization, and communication effectiveness of organizational online responses. Responding to these massive public demands efficiently, accurately, and humanely has become a core challenge in enhancing organizational credibility, service effectiveness, and user satisfaction.

[0003] Online responses are a core component of the entire online interaction process, and their quality directly influences public perception, satisfaction, and trust in organizational services. Existing research and practice indicate that high-quality online responses must meet both task-based and non-task-based values. Task-based values ​​emphasize the practicality and functionality of responses: accuracy is the cornerstone; ensuring the authenticity, reliability, and unambiguity of the content conveyed is fundamental to building trust; targeted responses must closely address the user's core needs and avoid irrelevant answers; and completeness requires providing sufficient information to resolve the user's questions or guide their next steps. Non-task-based values ​​focus on the emotional and communicative dimensions of responses: Understanding and empathizing with the user's situation or emotions (such as expressing concern and apologies), clearly explaining the context of the incident or the rationale for the decision, and candidly describing the organization's efforts or challenges can significantly enhance users' sense of being valued, improve their acceptance of the response, and enhance their overall satisfaction, even if the final outcome fails to fully meet their needs. Furthermore, timeliness of responses (to avoid user anxiety and the accumulation of negative emotions), clarity, and politeness are also important factors influencing user perception. However, current online response practices still rely heavily on human interaction. While manual responses offer flexibility and advantages in handling complex and personalized inquiries, their limitations are becoming increasingly apparent when faced with massive, high-frequency, and repetitive inquiries. On the one hand, the highly concentrated or repetitive nature of many inquiries traps customer service and operations staff in an inefficient cycle of repetitive work, preventing them from freeing up their energy to handle more complex or higher-value matters. On the other hand, the quality of responses is highly dependent on individual staff's knowledge, experience, communication skills, and on-the-spot presence, which can easily lead to inconsistent response standards, inappropriate wording, omissions of key information, and even misunderstandings, damaging the organization's image. Furthermore, faced with a surge in inquiries, manual responses often lag, failing to meet users' expectations for immediate feedback and easily leading to user dissatisfaction. Furthermore, relying on a large human workforce to meet growing demand is costly and difficult to achieve elastic scalability of service capabilities. In recent years, large language models (LLMs) have achieved numerous breakthroughs in general tasks in fields such as natural language processing, primarily in areas such as search enhancement, semantic understanding and reasoning, in-depth case analysis, and machine translation. They are widely used in practical applications such as text creation and generation and intelligent assistants. The successful application of large language models in natural language processing offers new possibilities for addressing these issues. The semantic complexity and multi-domain nature of online responses place even stricter demands on semantic understanding accuracy. The overwhelming advantage of large language models in semantic understanding offers new avenues for automating the generation of responses to online political inquiries. However, the semantic representation and understanding generation capabilities of large models are significantly affected by the training corpus, making their direct application to specific domains susceptible to "hallucination" issues, such as generating false information or overly generalized responses. Therefore, relying solely on large language models to generate responses is unlikely to meet the stringent accuracy and policy compliance requirements of online political inquiries.

[0004] In addition, although existing research has optimized the intermediate processing links of online responses through intelligent technical means and has achieved certain improvements in response efficiency and quality, it has still not been able to build a complete end-to-end processing closed-loop system, and there is still room for further optimization in terms of overall process efficiency and service quality. At the same time, existing research shows that the effectiveness of online responses is closely related to the way of language expression. As scholars such as Song Yingfa pointed out, responders need to pay attention to the artistry of language and reflect characteristics such as popularity, personalization, humanistic care and imagery in their expressions. However, current related research focuses more on the optimization of response generation efficiency and text quality, and lacks in-depth discussion of the correlation mechanism between language artistry and public satisfaction. It is worth noting that artificial intelligence technology can not only improve response efficiency, but also enhance the affinity of response texts through anthropomorphic expressions. Based on this, the present invention believes that building an intelligent response template reference system by integrating historical response cases and information technology means is a feasible path to improve the quality of response services in the digital age. Summary of the Invention

[0005] In order to solve the above problems, in the first aspect, the embodiment of the present application provides a network response retrieval enhancement generation method integrating a large language model, comprising:

[0006] Input the request text and prompts into the large language model, and the large language model outputs the request and type;

[0007] Input similar cases and prompts of the appeal text into the large language model, and the large language model outputs the response paradigm for the appeal of the type in similar cases;

[0008] Input prompts to the large language model, which generalizes the response paradigm and generates a first framework suitable for the appeal response; the method of obtaining similar cases of the appeal text includes:

[0009] Calculate the appeal vector after keyword embedding of the appeal text;

[0010] Determine the target sub-case library based on the cosine distance between the appeal vector and the center of each sub-case library clustered based on the appeal content;

[0011] Similar cases are determined based on the cosine distance between the appeal vector and the appeal vector of the cases in the target sub-case library.

[0012] In a second aspect, an embodiment of the present application further provides an electronic device, comprising: one or more processors, a memory, and one or more programs; wherein the one or more programs are stored in the memory, and the one or more programs include instructions, which, when executed by the electronic device, enable the electronic device to execute the first aspect and any possible technical solution of the first aspect thereof.

[0013] In a third aspect, an embodiment of the present application further provides a computer-readable storage medium, which includes a computer program. When the computer program runs on an electronic device, the electronic device executes the first aspect and any possible technical solution of the first aspect.

[0014] Beneficial effects:

[0015] In the first aspect, the present invention reconstructs a new intelligent question-answering method to effectively solve the problem of processing non-standard texts, provides theoretical and practical references for the automatic generation of network responses, and promotes the intelligent transformation and development of network response services, for example, it can be applied to responding to online political inquiries.

[0016] Secondly, the present invention's similar appeal case retrieval obtains the appeal text and converts it into a vector. A retrieval algorithm is then used to retrieve several similar cases from a historical online case library. Response enhancement generation generates prompts based on a pre-designed prompt template from the retrieved similar cases and appeals, which are then fed into a large model that generates responses to the inquiry. Case library reconstruction utilizes text clustering methods to restructure the case library into multiple sub-case libraries. Incremental clustering methods can also be used to expand the case library, improving the efficiency of similar appeal case retrieval and case library expansion to meet the ever-increasing demand for massive online responses in the information age, such as responses to online inquiries.

[0017] On the third aspect, network response texts, such as online customer service responses, online political inquiries, etc., although the texts have different expression preferences, colloquial expressions, complex and diverse demands, and varying cultural levels, usually have clear problem demands. Therefore, the present invention considers focusing on the theme of the network response text, proposes a vector representation method for integrating text themes, and performs similar demand case retrieval based on the text vector after integrating text themes. The method extracts keywords that can represent the main demands, integrates them into the original demand text, and then vectorizes them to highlight the main demands. The generated fusion vector shows a stronger demand identification ability in the semantic similarity calculation, so that the retrieved similar cases match the concerns more accurately, thereby finding similar cases that are more matched with the demand theme, and then improving the pertinence of the generated response. This text feature is particularly prominent in political inquiry texts. In some specific embodiments, the present invention implements enhanced generation of network political inquiry response retrieval for network political inquiries, achieving good results.

[0018] In the fourth aspect, the present invention reconstructs the entire online political inquiry case library into several sub-case libraries, calculates the distance between the target demand and the center of each sub-case library to determine the target sub-case library, and finally searches only the target sub-case library to realize similar case retrieval. The retrieval calculation amount is reduced from traversing the entire case library to traversing only the target sub-case library, thereby improving the efficiency of similar case retrieval. That is, this method organizes the entire case library according to themes to form multiple sub-case libraries with different themes. When performing similar case retrieval, only the similarity between the target demand and the cases in a certain sub-case library is calculated, which will greatly reduce the number of comparative calculations and effectively improve the retrieval efficiency. In addition, the reconstructed case library can be expanded based on incremental clustering to ensure the efficient dynamic expansion of the reconstructed case library.

[0019] In the fifth aspect, the present invention expects the clustering results to be stable, that is, the category to which the sample belongs remains unchanged. However, as the case base increases, the convergence speed of K-Means++ clustering will slow down. When the clustering is close to convergence, most of the sample categories are stable, and only a few edge sample categories may change, resulting in fine-tuning of the cluster center. Cases at the edge of the cluster usually have relatively unclear themes and are small in number, so changes in a few edge sample categories have little impact on similar case retrieval. The present invention comprehensively considers the efficiency and effectiveness of similar case retrieval, and the convergence condition of case text clustering only needs to meet the stability of the cluster center.

[0020] In a sixth aspect, this invention utilizes a context-based learning-based response framework generation path. Based on the concept of context-based learning, it utilizes similar cases to generate a framework for responding to an appeal, providing a framework for subsequent response text generation. This approach addresses the following issues: 1) Appeals often contain multiple specific questions, requiring separate analysis and responses; 2) Different types of appeals require different approaches to responses.

[0021] In the seventh aspect, after completing the above-mentioned framework generation work, the present invention will next fill in the specific content in the template in a fine-grained manner. First, the main goal of the detailed integration is to clarify: according to the response framework generated in the previous article, the actual policy documents, phone numbers, email addresses and other information are filled in the corresponding positions to ensure that the reply content is more specific, detailed, and in line with the actual situation. Secondly, by integrating this information, the coherence of the response is ensured, making it look more natural and humane, and avoiding traces of human-computer interaction.

[0022] Additional aspects and advantages of the present invention will be set forth in part in the description which follows and, in part, will be obvious from the description which follows, or may be learned by practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is a framework for enhanced generation of online political question and response retrieval methods based on a large language model.

[0024] Figure 2It is a flowchart for embedding keywords into the appeal text.

[0025] Figure 3 A path map for generating a framework for responding to online government inquiries.

[0026] Figure 4 Refine and integrate prompt template design.

[0027] Figure 5 Elbow plot and silhouette coefficient for appeal text clustering. DETAILED DESCRIPTION

[0028] The following describes embodiments of the present application in detail with reference to the accompanying drawings, examples of which are shown in the accompanying drawings. The present application provides a method and an electronic device. The method and device are based on the same technical concept. Since the method and device solve similar problems, the implementation of the device and method can refer to each other, and any repetitions will not be repeated.

[0029] In one embodiment of the present invention, a text keyword extraction and embedding method is provided, comprising:

[0030] For text c i Perform word segmentation and remove stop words;

[0031] The text after word segmentation c i Pass the KeyBERT model to obtain the first q keywords;

[0032] Concatenate q keywords into a string to get text s, separated by spaces;

[0033] Convert text s into vector kv i ;

[0034] The text c i Convert to vector cv i ;

[0035] The vector cv i and vector kv i The vector ckv after keyword embedding of the text is obtained by first summing and then L2 norm normalization. i .

[0036] Among them, the vector cv i and vector kv i The fusion is expressed by the following formula using the method of summing first and then L2 norm normalization:

[0037]

[0038] Where m represents the dimension of the vector.

[0039] Among them, the text s is converted into a vector kv in the stepi ,include

[0040] Perform word segmentation on the text s;

[0041] Pass the segmented text s into the MacBERT model and obtain the output of the last hidden layer of each token;

[0042] The vector after the average pooling of the hidden layer output of all tokens is used as the vector representation kv of the text s i ;

[0043] Among them, in step 2, the text c i Convert to vector cv i ,include

[0044] For text c i Perform word segmentation processing;

[0045] The text after word segmentation c i Pass the MacBERT model and get the output of the last hidden layer of each token;

[0046] The vector after the average pooling of the hidden layer output of all tokens is used as the text c i The vector representation of cv i .

[0047] In one embodiment of the present invention, a method for obtaining similar cases of text is provided, comprising:

[0048] Calculate the vector of the text after keyword embedding;

[0049] Determine the target sub-case library based on the cosine distance between the vector and the center of each sub-case library based on content clustering;

[0050] Similar cases are determined based on the cosine distance between the appeal vector and the appeal vector of the cases in the target sub-case library.

[0051] The vector of the text after keyword embedding is calculated according to the above-mentioned text keyword extraction and embedding method.

[0052] Among them, the construction method of the sub-case library based on content clustering includes:

[0053] S1. Randomly select a sample from the case data set as the first cluster center;

[0054] S2. For each sample in the dataset, calculate the shortest distance between it and the selected cluster center, and calculate the probability of each sample being selected as the next cluster center based on the shortest distance;

[0055] S3. Based on the calculated probability, randomly select a new sample as the next cluster center, and repeat steps S2-S3 until k cluster centers are selected;

[0056] S4. Calculate the distance between the sample and each cluster center, and assign the sample to the cluster center with the shortest distance to it;

[0057] S5. For each cluster center, recalculate the mean similarity between all sample points belonging to it and use the mean as the new cluster center;

[0058] S6. Repeat steps S4-S5 until the convergence condition is reached and the iteration is stopped. All sample points belonging to each cluster center are a sub-case library, and t sub-case libraries are obtained by clustering the case data set based on the content of the appeal.

[0059] The iteration is stopped when the convergence condition is reached, according to the following formula:

[0060]

[0061] Among them, it is assumed that after the tth iteration, the cluster C i The cluster center vector of becomes The change range of its cluster center is shown in formula (2). If the change range of the cluster center of k classes satisfies formula (3), and the sum of the changes of the cluster centers of all categories is less than the preset threshold θ, the cluster center is stable and the iteration stops when the convergence condition is reached.

[0062] In one embodiment of the present invention, a network response retrieval enhancement generation method integrating a large language model is provided. The method can be used for network political response, including:

[0063] Input the request text and prompts into the large language model, and the large language model outputs the request and type;

[0064] Input similar cases and prompts of the appeal text into the large language model, and the large language model outputs the response paradigm for the appeal of the type in similar cases;

[0065] Input prompts to the large language model, which generalizes the response paradigm and generates a first framework suitable for the appeal response; the method of obtaining similar cases of the appeal text includes:

[0066] Calculate the appeal vector after keyword embedding of the appeal text;

[0067] Determine the target sub-case library based on the cosine distance between the appeal vector and the center of each sub-case library clustered based on the appeal content;

[0068] Similar cases are determined based on the cosine distance between the appeal vector and the appeal vector of the cases in the target sub-case library.

[0069] Among them, the network response retrieval enhancement generation method integrating the large language model also includes inputting prompts into the large language model, and the large language model limits the text expression form and tone of the first frame suitable for the appeal response to generate a second frame suitable for the appeal response.

[0070] Among them, the network response retrieval enhancement generation method integrating the large language model also includes inputting filling information for the first frame or the second frame into the large language model, the filling information including policy information and / or contact information; and the large language model outputs the response content including the first frame or the second frame and the filling information.

[0071] Among them, the network response retrieval enhancement generation method that integrates the large language model inputs prompts into the large language model, and the large language model outputs the response content with smooth and coherent content, consistent tone and style.

[0072] Among them, the step of calculating the appeal vector after keyword embedding of the appeal text includes

[0073] Appeal text c i Perform word segmentation and remove stop words;

[0074] The appeal text c after word segmentation i Pass the KeyBERT model to obtain the first q keywords;

[0075] Concatenate q keywords into a string to get text s, separated by spaces;

[0076] Convert text s into vector kv i ;

[0077] The appeal text c i Convert to vector cv i ;

[0078] The vector cv i and vector kv i The method of first summing and then L2 norm normalization is adopted to obtain the appeal vector ckv after keyword embedding of the appeal text. i ;

[0079] Among them, the vector cv i and vector kv i The fusion is expressed by the following formula using the method of summing first and then L2 norm normalization:

[0080]

[0081] Where m represents the dimension of the vector.

[0082] Among them, the text s is converted into a vector kv in the step i ,include

[0083] Perform word segmentation on the text s;

[0084] Pass the segmented text s into the MacBERT model and obtain the output of the last hidden layer of each token;

[0085] The vector after the average pooling of the hidden layer output of all tokens is used as the vector representation kv of the appeal text s i ;

[0086] Among them, the step will appeal to the text c i Convert to vector cv i ,include

[0087] Appeal text c i Perform word segmentation processing;

[0088] The appeal text c after word segmentation i Pass the MacBERT model and get the output of the last hidden layer of each token;

[0089] The vector after the average pooling of the hidden layer output of all tokens is used as the appeal text c i The vector representation of cv i .

[0090] Among them, the construction method of the sub-case library based on the clustering of appeal content includes

[0091] S1. Randomly select a sample from the case data set as the first cluster center;

[0092] S2. For each sample in the dataset, calculate the shortest distance between it and the selected cluster center, and calculate the probability of each sample being selected as the next cluster center based on the shortest distance;

[0093] S3. Based on the calculated probability, randomly select a new sample as the next cluster center, and repeat steps S2-S3 until k cluster centers are selected;

[0094] S4. Calculate the distance between the sample and each cluster center, and assign the sample to the cluster center with the shortest distance to it;

[0095] S5. For each cluster center, recalculate the mean similarity between all sample points belonging to it and use the mean as the new cluster center;

[0096] S6. Repeat steps S4-S5 until the convergence condition is reached and the iteration is stopped. All sample points belonging to each cluster center are a sub-case library, and t sub-case libraries are obtained by clustering the case data set based on the content of the appeal;

[0097]

[0098] Among them, it is assumed that after the tth iteration, cluster C i The cluster center vector of becomes The change range of its cluster center is shown in formula (2). If the change range of the cluster center of k classes satisfies formula (3), and the sum of the changes of the cluster centers of all categories is less than the preset threshold θ, the cluster center is stable and the iteration stops when the convergence condition is reached.

[0099] Among them, in the step, the target sub-case library is determined according to the distance between the appeal vector and the center of each sub-case library clustered based on the appeal content; similar cases are determined according to the cosine distance between the appeal vector and the appeal vector of the case in the target sub-case library, including t sub-case libraries clustered based on the appeal content of the case data set, the cluster center vector ui of each sub-case library, and the appeal vector ckv is calculated. i The cluster center vector u of the t sub-case databases clustered based on the appeal content i The cosine distance of the demand vector ckv i The sub-case library with the smallest cosine distance is used as the target sub-case library; traverse the target sub-case library and calculate the demand vector ckv i The cosine distance between the appeal vectors of all cases in the target sub-case library and the target sub-case library, wherein the case or the first few cases with the smallest cosine distance are similar cases.

[0100] Among them, the network response retrieval enhancement generation method integrating the large language model also includes

[0101] For new cases to be stored, extract the appeal vector ckv of the appeal text in the case i ;

[0102] Calculate the demand vector ckv i The cluster center vector u of the t sub-case databases clustered based on the appeal content i The cosine distance of

[0103] Add the new case to be stored to its cluster center vector and appeal vector ckv i In the sub-case library with the smallest cosine distance, the cluster center vector of the sub-case library is u;

[0104] For the cluster center of the sub-case library, recalculate the mean similarity between all sample points to which it belongs, and use the mean as the new cluster center;

[0105] Calculate the offset between the new cluster center and the original cluster center. If the offset is greater than the set threshold θ1, execute the method of constructing a sub-case library based on appeal content clustering, and reallocate the cluster centers to which the sample points of each case belong in order to reallocate the sub-case library. If the offset is not greater than the set threshold θ1, keep the cluster centers to which the sample points of each case belong.

[0106] Among them, the online response is a response to an online political inquiry, and the response content is the content of the political inquiry response.

[0107] The present invention is described in detail below by taking the online political inquiry response in the online response as an example:

[0108] 1. Terminology:

[0109] Online response: Online response refers to feedback or reaction to specific information, events, behaviors or interactions through the Internet platform.

[0110] Online government inquiries and responses are standardized practices whereby government departments or public authorities publicly respond to, provide feedback on, or engage in interactive communication with the public regarding policy inquiries, livelihood demands, oversight complaints, and other government-related issues raised online. Essentially, this represents a new model of two-way interaction between government and the public in the digital age, aiming to improve transparency, service efficiency, and public engagement.

[0111] 2 Related research

[0112] 2.1 Research on Responses to Online Political Questions

[0113] Online political consultations are a prominent hallmark of the participatory and interactive phase of government informatization and a manifestation of the e-governance model. Online political consultations broaden channels for public expression, promote freedom of speech, deepen the forms and content of citizen political participation, and enhance the effectiveness of online political participation, thereby expanding orderly citizen political participation. Existing research indicates that the quality of government responses is a key factor influencing the effectiveness of online political consultations and government credibility, while public satisfaction with government consultations is a key criterion for measuring the quality of responses. Factors such as response efficiency, response content, response attitude, whether the issue is resolved, incident handling behavior, and government image influence public satisfaction with government consultations. Therefore, considering these influencing factors, constructing responses that satisfy the public is crucial. Regarding improving response efficiency, scholars have used empirical analysis to explore how to mobilize public enthusiasm and participation through establishing consultation assessment indicators, improving consultation models, and refining work mechanisms. Scholars believe that generating responses to online political consultations relies on efficient horizontal coordination within bureaucratic organizations. As public participation and service-oriented digital governance become key issues in my country's pursuit of social governance innovation, scholars have found that government departments can leverage digital technology to develop an automated, institutionalized, and streamlined process for generating responses to government inquiries, thereby achieving a comprehensive online government inquiry handling mechanism. Zhang Xuefeng employed word segmentation and clustering techniques to identify the recurrence of similar issues and demands over time. He then proposed knowledge-based service recommendations, addressing this issue by recommending responses to existing issues based on similarity. Yao Lan constructed an intelligent response system for a government-media integrated inquiry platform. This system uses a topic-based automatic classification model for public messages to accurately categorize them. This system then integrates a classification-based automatic matching model with relevant departments for intelligent connection. Ultimately, a collaborative response framework based on subject and content was employed to generate multi-dimensional, complex response solutions, forming an integrated government inquiry service approach characterized by "intelligent classification, precise matching, and collaborative response." However, while existing research has optimized the intermediate processing steps of inquiry responses through intelligent technologies, achieving some improvements in response efficiency and quality, it has yet to establish a complete, end-to-end closed-loop system for inquiry processing, leaving room for further improvement in overall process efficiency and service quality. At the same time, existing research indicates that the effectiveness of responses to online political inquiries is closely related to the language used. Scholars such as Song Yingfa have pointed out that responders need to prioritize linguistic artistry, embodying characteristics such as accessibility, personalization, humanistic care, and imagery. However, current research focuses on optimizing response efficiency and text quality, lacking in-depth exploration of the correlation between linguistic artistry and public satisfaction. Notably, artificial intelligence technology can not only improve response efficiency but also enhance the relatability of responses through anthropomorphic expression.Based on this, the present invention believes that integrating historical response cases with information technology to build an intelligent response template reference system is a feasible path to improve the quality of government services in the digital age.

[0114] Amid the wave of digital government development and the digital transformation of public services, online government consultations, as a new interactive channel connecting the government and the public, have become a crucial tool for advancing the modernization of social governance. With the widespread adoption of internet technology and the intelligent upgrade of government platforms, visits and inquiries to online government platforms have rapidly increased, significantly boosting public participation in policy discussions and public awareness through online channels. However, this high level of engagement places higher demands on the timeliness, professionalism, and standardization of government responses. Efficiently responding to massive and fragmented public inquiries has become a key challenge in enhancing government credibility and governance effectiveness.

[0115] Responses to online government inquiries are a core component of the entire inquiry process, and their quality directly determines public satisfaction and trust in government services. Existing research indicates that responses to online inquiries must meet both task-based and non-task-based values. Task-based values ​​emphasize the practicality of responses. For example, factual accuracy is the foundation of responses, and ensuring the authenticity and reliability of information conveyed is key to gaining public trust. Non-task-based values ​​focus on the emotional and communicative impact of responses. For example, explaining the incident, expressing concerns, and describing efforts made all contribute to enhancing public acceptance and satisfaction. Furthermore, the timeliness, relevance, and completeness of responses are also important factors influencing response quality. However, current responses to online government inquiries are still primarily manual. While manual responses offer a certain degree of flexibility, their limitations are becoming increasingly apparent. On the one hand, the highly centralized nature of public inquiries leads staff to repeatedly answer similar questions, creating an inefficient cycle. On the other hand, the quality of manual responses relies heavily on individual experience, which carries risks such as inconsistent standards and non-standardized expression.

[0116] 2.2 Research on Response Generation Based on Large Language Model

[0117] Intelligent question answering (QA) is a core subfield of natural language processing, aiming to develop systems that understand and answer natural language questions posed by users. Traditional QA systems typically rely on predefined rules and limited corpora, making them incapable of handling complex, multi-turn conversations. Large language models (LLMs) are a type of deep learning model with a large number of parameters. In the field of natural language processing, they process large amounts of text data to learn language patterns, grammar, and semantics, thereby understanding and generating human language. LLMs not only demonstrate powerful capabilities in text understanding and generation, enabling them to solve complex tasks, but also significantly improve the accuracy and efficiency of QA systems, driving the development of intelligent QA technology. LLMs not only excel in generating grammatically correct and semantically coherent text, but also dynamically adjust generated content based on context, enabling them to solve complex tasks such as long text generation, multi-turn conversations, and style transfer. They also demonstrate considerable reasoning and creativity, opening up new possibilities for the practical application of intelligent QA. When building question-answering systems based on large language models (LLMs), research focuses on prompt learning, knowledge graphs (KGs), and retrieval-augmented generation (RAGs). These methods play a crucial role in building efficient and intelligent question-answering systems.

[0118] 1) Prompt-based learning:

[0119] Prompt learning is a method that enables a model to better understand human questions by constructing prompt information without changing the structure of the pre-trained model. By combining prompt information with the pre-trained model, the performance of the model in intelligent question-answering systems can be improved. Pu Qiumei and other scholars verified the effectiveness and practicality of the prompt learning mechanism in the task of text summarization through empirical research on the construction of artificial prompt templates. Ni Xuanfan and Li Piji proposed a story generation framework that integrates an external common sense library with a prompt learning mechanism, which significantly improved the logical coherence of the generated text. However, the basic prompt strategy still has performance limitations when dealing with complex reasoning tasks. To address this challenge, the academic community proposed a chain-of-thought strategy that enhances the model's logical reasoning ability by guiding the model to explicitly generate intermediate reasoning steps. Denny Zhou and others further proposed a chain-of-thought strategy based on task decomposition, which deconstructs complex problems into a sequence of actionable sub-problems, thereby optimizing the performance of large language models in complex reasoning tasks. Pan Lu et al. proposed a multimodal reasoning method based on Chain of Thought (CoT), which significantly improved the performance of scientific question answering tasks by training language models to generate detailed explanations and interpretations as reasoning steps. Chancharik Mitra et al. proposed the RetLLM-E method, which significantly improved the quality of large language models answering course-related questions in student forums by combining text retrieval and course content-specific prompting strategies.

[0120] 2) Based on knowledge graph:

[0121] Knowledge Graph Question Answering (KGQA) leverages the structured knowledge representation of knowledge graphs and augments LLMs with external knowledge, improving the robustness and intelligence of question-answering systems and enhancing their performance in domain-specific or commonsense reasoning. When faced with domain-specific knowledge, Dong Zhaoan et al. used GraphRAG technology to retrieve relevant entities, relationships, and their attributes from a Traditional Chinese Medicine (TCM) knowledge graph. Combining this retrieved structured knowledge, they enhanced and optimized prompt words, providing background knowledge support for the answers generated by a large language model (LLM), effectively addressing the "hallucination" problem of large models. To enhance reasoning capabilities, Feng Guofeng et al. constructed a knowledge base using knowledge graphs and Neo4j graph database technology. They also implemented user question parsing and intent recognition based on a naive Bayesian model, meeting the practical needs of interactive Q&A in the field of tunnel disease management. However, existing large language model-enhanced knowledge graph Q&A methods still have many limitations. First, knowledge graph construction requires a large amount of data, which is difficult to apply to government affairs. Second, the time-consuming retrieval and reasoning involved in question answering often leads to low efficiency.

[0122] 3) Based on retrieval enhancement:

[0123] Retrieval-Augmented Generation (RAG) technology is an emerging technology in the field of intelligent question answering. It retrieves relevant document fragments from external knowledge bases through semantic similarity calculations, thereby enhancing the capabilities of language models. In 2020, Guu et al. pioneered the introduction of retrieval mechanisms into the pre-training process, constructed an enhanced BERT model based on knowledge retrieval, and verified for the first time the effectiveness of the retrieval-generation architecture in open-domain question answering tasks. Subsequently, the Lewis team systematically proposed a generalized RAG framework, innovatively integrating non-parametric retrievers (such as DPR) and parametric generators (such as BART), and expanding the application of technology to multiple knowledge-intensive scenarios such as open-domain question answering and fact verification. At this point, RAG technology has successfully built a collaborative mechanism for knowledge acquisition and content generation, effectively alleviating the knowledge illusion problem of large models. Huang Bing et al. explored the application of techniques such as Naive Retrieval Enhanced Generation (Naive RAG), Graph Retrieval Enhanced Generation (Graph RAG), and Agent Retrieval Enhanced Generation (Agent RAG) in teaching and demonstrated their feasibility in paleontology. Hai Jiali et al. used the GPT3.5 model as a foundational model, combined with data optimization and RAG techniques, to develop a knowledge question-answering system for traditional Chinese medicine standards with semantic analysis, contextual association, and generation capabilities. These research results fully demonstrate the effectiveness and practicality of RAG techniques in intelligent question-answering tasks in specialized fields.

[0124] However, the current research on the application of intelligent question-answering technology in the field of generating responses to online political inquiries still has obvious deficiencies. Existing academic research mainly focuses on processing standardized long texts, which usually have the characteristics of rigorous logic, clear structural hierarchy, and a high degree of terminology standardization. In contrast, public political texts, as informal language data spontaneously generated by citizens, have typical non-standard characteristics in terms of language characteristics: at the vocabulary level, they have a low usage rate of professional terms and frequent colloquial expressions; at the syntactic level, they have a loose structure and weak logical relevance; at the paragraph level, there are problems such as disordered information organization and scattered semantic focus. There are significant differences between this native text feature and traditional research corpus, which leads to the adaptive bottleneck of existing intelligent question-answering technology in the task of generating responses to political inquiries. In response to this, the present invention has reconstructed a new intelligent question-answering method that effectively solves the problem of processing non-standard texts, provides theoretical and practical references for the automatic generation of responses to online political inquiries, and promotes the intelligent transformation and development of online political inquiry services.

[0125] 3. Retrieval Enhancement Method for Online Political Question Response Based on Large Language Model

[0126] 3.1 Methodological Framework

[0127] The survey found that citizens' political inquiries and demands show a thematic concentration. That is, for similar issues, different members of the public will respond through online political channels, forming multiple similar demands. When government staff handle citizen demands, if they can automatically generate response content for staff based on historical similar demands, the staff's response efficiency can be significantly improved, the professionalism and consistency of the response content can be guaranteed, and public satisfaction can be improved. To solve this problem, the present invention proposes a retrieval-augmented generation method for online political inquiries based on a large language model. This method uses a large language model as the basis for response generation and adopts a retrieval-augmented generation (RAG) framework to improve the generation effect. RAG is a technology that combines information retrieval and generation models, aiming to improve the accuracy, professionalism and timeliness of generated content by dynamically introducing external knowledge bases. RAG technology is mainly divided into three steps: retrieval, enhancement, and generation. Retrieval is the first step in the RAG process, extracting information related to the question from the knowledge base to provide context and knowledge support for subsequent generation. The extracted information is then used as input to the generation model to enhance the model's understanding and answering capabilities of the question, making the generated content richer and more accurate. Finally, the generative model is used to generate answers that meet user needs, ensuring output quality and accuracy.

[0128] like Figure 1The method proposed in the present invention is mainly composed of three modules: similar appeal case retrieval, response enhancement generation, and case library reconstruction and expansion. The similar appeal case retrieval module obtains the citizen appeal text and converts it into a vector, and uses the retrieval algorithm to retrieve several similar cases similar to the appeal from the online political inquiry history case library. The response enhancement generation module generates prompt words based on the designed prompt template for the retrieved similar cases and citizen appeals, and then inputs them into the big model, and the big model generates responses to the political inquiry. The case library reconstruction and expansion module uses the text clustering method to reconstruct the case library into multiple sub-case libraries, and expands the case library through the incremental clustering method to improve the efficiency of similar appeal case retrieval and case library expansion, so as to meet the ever-increasing demand for massive online political inquiries in the information age. The implementation methods of the three modules will be specifically introduced below.

[0129] 3.2 Retrieval of Similar Appeal Cases

[0130] Online government inquiry platforms serve as an important channel for interaction between citizens and the government, allowing them to fully express their demands. However, due to individual preferences, colloquial language, complex and diverse demands, and varying cultural levels, citizen demands, compared to standardized case texts such as legal documents, are characterized by diverse expression, complex logic, and strong emotional overtones. This increases the computational difficulty of retrieving similar cases. While these characteristics are present, citizen inquiry texts typically have clear questions and demands. Therefore, this section focuses on the subject matter of inquiry texts and proposes a method for representing inquiry case vectors that incorporates text themes. This method then uses this text vector to retrieve similar cases. This method extracts keywords that represent citizens' primary demands, incorporates them into the original text of the citizen's demand, and then vectorizes them. This method highlights the citizen's primary demands, thereby identifying similar cases that more closely match the demand themes and improving the targeted nature of the generated responses.

[0131] 3.2.1 Appeal Text Vector Representation Integrating Text Themes

[0132] (1) Citizen Appeal Text Vector Generation Based on MacBERT

[0133] As a numerical representation of text data, text vectors can capture semantic information and significantly improve the accuracy and efficiency of case recommendation systems. Compared with traditional keyword matching, the vector matching mechanism based on semantic similarity effectively reduces the high complexity of original text comparison through low-dimensional space operations, while dynamically adapting to the needs of case library expansion. In recent years, pre-training models represented by BERT have learned general language representations through large-scale corpora and have performed excellently in NLP tasks, but their direct splitting of Chinese characters weakens semantic integrity. To this end, the MacBERT model introduces the LTP word segmentation tool to identify Chinese word boundaries, and uses 5.4 billion word corpora such as Chinese Wikipedia for pre-training. Combined with the improved training strategy of N-Gram masking and similar word replacement, it can more accurately capture Chinese context features. Studies have shown that the model exhibits stronger semantic modeling capabilities in Chinese text understanding tasks, providing a more efficient semantic matching solution for case recommendation systems.

[0134] This paper uses the MacBERT model to convert citizen appeal text into computable vectors, preparing for subsequent calculations. The algorithm flow for calculating the text vectors for a set of n citizen appeal texts, C = {c1, c2, ..., cn}, is shown in Table 1. The text set is looped through the algorithm, and the text vector for each appeal is calculated separately. In each loop, a claim text ci is segmented, and then the segmented text is passed to the MacBERT model to obtain the output of the last hidden layer for each token. Finally, the vector resulting from the average pooling of the hidden layer outputs of all tokens is used as the vector representation of the text ci.

[0135] Table 1. Steps for generating citizen appeal text vectors based on MacBERT

[0136]

[0137] (2) Extraction and embedding of keywords in appeal text

[0138] As a concise expression of the core demands of citizens, the precise extraction and effective embedding of keywords in appeal texts play a key role in improving the quality of similar case retrieval. However, how to select keywords that can fully represent the demands of citizens, how to embed keywords into the question text and convert them into text vectors that can be used for similarity calculations are problems that need to be solved. The present invention proposes a deep keyword extraction and embedding method that integrates KeyBERT. Compared with shallow models such as TF-IDF and TextRank that rely on dictionaries or statistical features, KeyBERT automatically captures deep semantic associations through a self-supervised learning mechanism, which not only avoids the limitations of manually constructed dictionaries, but also generates keyword vectors of fixed dimensions. This method generates sentence-level vectors through the MacBERT pre-training model mentioned above, dynamically calculates the semantic similarity between word vectors and sentence vectors, screens out highly correlated appeal keywords, and fuses their vector representations with the original text vectors to form a hybrid representation with appeal enhancement characteristics. The generated fusion vector exhibits stronger appeal recognition capabilities in semantic similarity calculations, allowing the retrieved similar cases to more accurately match citizen concerns.

[0139] Table 2 shows the steps for the keyword extraction and embedding algorithm for appeal texts. The set of citizen appeal texts, C = {c1, c2, ..., cn}, is input. The algorithm cyclically calculates the keyword-embedded vector for each appeal text. First, the appeal text ci is segmented and stop words are removed to filter out meaningless words. The KeyBERT model is then used to obtain the first q keywords and concatenate them, separating each keyword with a space, to form the keyword representation text ki for text ci. ki is then input into the text vector generation algorithm to obtain the keyword vector kvi. Finally, the sentence vector cvi of ci and the keyword vector kvi are fused by summing them and then L2-norm normalization to obtain the new keyword-embedded vector ckvi for the political appeal text. L2-norm normalization of text vectors can eliminate differences between different text vectors due to different original scales and reduce the impact of text vector outliers on similarity calculations. The most important thing is that after L2 norm normalization, the Euclidean distance of vectors and their cosine similarity are equivalent to a certain extent. This means that no matter which metric is used in the similarity calculation (Euclidean distance or cosine similarity), the L2 norm normalized vectors can give the same similarity score, and it can also reduce the subsequent calculation amount. Assuming that cvi and kvi are both m-dimensional vectors, the fusion formula of the two is as follows:

[0140]

[0141] Table 2. Algorithm steps for keyword extraction and embedding in appeal text

[0142]

[0143] Figure 2This example demonstrates a flowchart for keyword embedding for a citizen appeal text, ci. The algorithm transforms a citizen appeal text into a text vector that can be efficiently compared. This vector is embedded with keyword features that fully represent the citizen appeal. This fused vector effectively describes the citizen appeal, improving recommendation effectiveness in subsequent case development. The following article will further describe a similar case recommendation method based on the fused text vector.

[0144] 3.2.2 Similar Case Retrieval

[0145] In the online political inquiry case database, a historical case that has been responded to often contains information such as the content of the appeal, the time of the appeal, the content of the response, the time of the response, and the responding department. Among them, the content of the appeal is a full expression of the citizen's needs. Therefore, in similar case retrieval, the present invention can search by comparing the similarity between the target appeal and the appeal content in the response case. The present invention adopts a neighbor-based similar case retrieval method, which is highly interpretable and easy to expand the case database without retraining the model. The neighbor-based similar case retrieval method first converts the citizen's appeal into a vector, then compares the cosine similarity with all the case appeal vectors in the case database, and retrieves the most similar cases to provide reference for subsequent response generation. Obviously, the time complexity of the retrieval is directly affected by the size of the case database. With the popularization of Internet technology and the improvement of citizens' awareness of political participation, the number of online political inquiry cases continues to rise, placing higher demands on retrieval efficiency. In this invention, the full online political inquiry case database is reconstructed into several sub-case databases, the distance between the target appeal and the center of each sub-case database is calculated to determine the target sub-case database, and finally, only the target sub-case database is searched to achieve similar case retrieval. The retrieval computational load is reduced from traversing the entire case library to traversing only the target sub-case library, thereby improving the efficiency of similar case retrieval. The case library reconstruction method will be introduced in detail below.

[0146] 3.3 Case Library Reconstruction and Expansion

[0147] To address the immense pressure on retrieval computations imposed by the ever-expanding case database, this section proposes a case database reconstruction method based on content clustering. This method organizes the entire case database by topic, forming multiple sub-case databases with different themes. When searching for similar cases, only the similarity between the target case and cases in a specific sub-case database is calculated, significantly reducing the number of comparisons and effectively improving retrieval efficiency. Furthermore, the reconstructed case database can be expanded using incremental clustering, ensuring efficient and dynamic expansion of the reconstructed case database.

[0148] 3.3.1 Case Library Reconstruction Based on Appeal Content Clustering

[0149] The present invention clusters the appeal content texts of the case library, and reorganizes the case library into multiple sub-case libraries based on the text clustering results. The text clustering of the appeal content of the case library also realizes the information integration of different political issues, which can help relevant departments identify common issues of public concern. In the present invention, the present invention uses the K-Means++ algorithm to cluster the appeal content texts of the case library. The K-Means++ algorithm is a commonly used partition-based clustering algorithm, which aims to solve the problem that the traditional K-Means algorithm is sensitive to the selection of initial cluster centers. It has the advantages of simplicity, efficiency, strong interpretability and applicability to large-scale data sets. The present invention designs the similarity calculation method and convergence conditions according to the needs of online political case retrieval. The calculation process of the K-Means++ algorithm is as follows:

[0150] 1) Randomly select the first cluster center: Randomly select a sample from the set as the first cluster center.

[0151] 2) Calculate distances and probabilities: For each sample in the dataset, calculate the shortest distance between it and the selected cluster center. Based on these distances, calculate the probability of each sample being selected as the next cluster center. The farther the sample is from the selected cluster center, the higher its probability of being selected.

[0152] 3) Select a new cluster center: Based on the calculated probability, randomly select a new sample as the next cluster center. This process is repeated until k initial cluster centers are selected.

[0153] 4) Calculate the class to which all samples belong: Calculate the distance between the sample and each cluster center and assign it to the nearest class.

[0154] 5) Update the cluster center: For each cluster center, recalculate the mean similarity between all the points to which it belongs, and use the mean as the new cluster center.

[0155] 6) Repeat steps 4 and 5 until the convergence condition is reached and clustering is completed.

[0156] The K-Means++ algorithm usually has three convergence conditions:

[0157] 1) The cluster center is stable, that is, the cluster center does not move significantly after two consecutive iterations;

[0158] 2) Cluster membership is stable, that is, the categories to which cluster members belong remain unchanged after two consecutive iterations;

[0159] 3) The preset maximum number of iterations is reached.

[0160] Generally, the present invention expects the clustering results to be stable, that is, the category to which the sample belongs remains unchanged. However, as the case base increases, the convergence speed of K-Means++ clustering will slow down. When the clustering is close to convergence, most of the sample categories are stable, and only a few edge sample categories may change, resulting in fine-tuning of the cluster center. Cases at the edge of the cluster usually have relatively unclear appeal topics and are small in number, so changes in a few edge sample categories have little impact on similar case retrieval. The present invention comprehensively considers the efficiency and effect of similar case retrieval, and the convergence condition of case text clustering only needs to meet the stability of the cluster center. During the iteration process, if the sum of the changes in the cluster centers of all categories is less than a pre-set threshold θ, the cluster center can be considered stable, and the iteration is stopped when the convergence condition is reached. Assume that after the tth iteration of the algorithm, the cluster center of cluster Ci moves to

[0161] Quantity becomes The change in the cluster center is shown in Formula 2. If the change in the cluster centers of the k classes in the algorithm satisfies Formula 3, clustering is complete. After clustering is complete, the case library is organized into t sub-case libraries based on the clustering results, and the cluster center vector ui of each sub-case library is recorded. When searching for similar cases, the cosine distance between the target demand vector and the cluster center vectors of the t sub-case libraries is first calculated, and the sub-case library with the smallest distance is selected as the target sub-case library. Finally, the target sub-case library is traversed, the distance between the target demand vector and all case demand vectors is calculated and sorted, and the most similar cases are found for reference in subsequent response generation.

[0162]

[0163] 3.3.2 Case Library Expansion Based on Incremental Clustering

[0164] With the increasing number of citizen inquiries and online cases, the efficient storage of new cases requires significant attention. Incremental clustering algorithms are capable of handling dynamically updated datasets. This means that as the case database continues to expand and update, the algorithm can gradually adjust the clustering results without having to re-cluster the entire case database. This significantly improves the efficiency of case database updates. Compared to full clustering, incremental clustering reduces unnecessary recalculation by adjusting clusters only for newly added or changed cases, thereby reducing computational costs and resource consumption.

[0165] Integrating the idea of ​​incremental clustering, the present invention designs a case library expansion method based on incremental clustering. For a new online political inquiry case to be stored in the database, the content of the case's appeal is first extracted and vectorized into x*, and then the cosine distance between x* and the cluster center of each sub-case library is calculated respectively, and the cluster center u with the smallest distance is found and the new case is added to this sub-case library. Finally, the cluster center of this sub-case library is updated to u*. Usually, after updating the cluster center, the category to which a small number of samples at the edge of the cluster belong may change. However, considering that a small number of edge cases will not have much impact on the retrieval of similar cases, the original affiliation of the edge cases is maintained when the cluster center offset is small. If the offset of the cluster center is greater than the pre-set threshold θ1, the clustering iteration is triggered until the convergence condition is met. After the case library is reconstructed according to the new clustering results, a round of case library expansion is completed.

[0166] In practical applications, the addition of a single case does not significantly impact case recommendations, so the case library can be continuously expanded through batch updates. Specifically, batch expansion of the case library is initiated when the system has accumulated n new cases. The distance between the appeal content vector and each cluster center is calculated sequentially. After all cases are placed in the cluster with the minimum distance, cluster iterations are triggered until convergence conditions are met. This effectively reduces the overhead of frequent case library updates and improves the efficiency of case library expansion.

[0167] 3.4 Response Enhancement Generation

[0168] This section develops a method for enhancing the generation of responses to online government inquiries based on a large-scale pre-trained language model. Its core mechanism draws inspiration from the cognitive processing of human practical writing. This method comprises two progressive processing stages: In the task analysis and strategy construction stage, semantic features of the request content are first analyzed to clarify the task boundaries. Subsequently, structured solution paradigms are extracted from similar request cases, ultimately integrating them into an actionable response framework. In the content generation and knowledge integration stage, leveraging the established response framework, in-depth analysis of similar request cases enables specialized expansion of response content elements. This two-stage processing model retains the planning characteristics of human writing while improving the language model's adaptability to text generation in government contexts through a historical case-driven mechanism.

[0169] Studies have shown that for pre-trained language models based on dialogue interaction architectures (typical representatives such as ChatGPT), the use of a phased task planning strategy has significant advantages over single long-range instruction input. Specifically, deconstructing the overall task into a sequence of logically related sub-tasks and implementing progressive instruction input through a multi-round dialogue mechanism can effectively improve the model's depth of understanding and execution accuracy of complex tasks. This iterative guidance method reduces the single cognitive load, allowing the model to focus more on sub-goal optimization, thereby achieving overall performance improvement in the dynamic process of task decomposition and integration. Therefore, based on the cognitive processing process of human writing and the theory and technology of prompt engineering, this section achieves accurate generation of responses to political questions through precise analysis of political questions and prompt optimization. The response enhancement generation method will be elaborated in detail below.

[0170] 3.4.1 Response Framework Generation Path Based on Contextual Learning

[0171] In-Context Learning (ICL) is a new machine learning paradigm in the field of large language models. Its core feature is that knowledge transfer and task adaptation can be achieved through a small number of demonstration samples without adjusting model parameters. The present invention applies this method to the field of generating responses to online government inquiries. The large model generated based on the template can learn new rules from a small number of case samples and effectively generalize to new government consultation texts. There are significant differences in the response strategies for different types of government consultation content (such as "problem reflection type" and "consultation type"): the former needs to focus on emotional comfort, while the latter needs to enhance the credibility of information. Therefore, before generating a response, each government consultation text needs to dynamically construct an adaptive response framework in three dimensions: word order expression, emotional association, and response structure.

[0172] Specifically, the generated political inquiry response framework should solve the following problems in steps: 1) Citizens' demands often contain multiple specific issues, which need to be analyzed and responded to separately. 2) There are differences in the response ideas for different types of demands. For example, complaint and suggestion texts should first investigate the complaint issues, clarify whether they are directed at the policy itself or the policy implementation issues, and then determine the existing solutions, and finally put forward improvement suggestions and clarify the attitudes of government staff. 3) The language expression of the government response must meet the official format specifications and language style requirements. To this end, the present invention is based on the idea of ​​context learning and uses similar cases to generate a demand response framework to provide a framework support for the subsequent response text generation. Figure 3This paper presents the response framework path of our invention, which consists of two key components: prompt engineering as the main architectural element of template design, and the thought process output generated by the DeepSeek model under the paradigm of our invention. In particular, the thought process output of the model not only demonstrates the reasoning process, but also provides an interpretable basis for the design intent of our method.

[0173] The specific response frame generation method is as follows:

[0174] (1) Classification and decomposition of demands. By inputting the demand text and structured prompts, prompts such as Figure 3 As shown, the large language model is guided to complete two core tasks: first, classifying appeals and clarifying their core response requirements. Second, problem deconstruction, identifying and breaking down specific issues within the appeals to be addressed. During this process, the invention uses role definition to clarify the model's task positioning, namely, as a government employee, they need to provide professional responses to citizens' political inquiries and demands, and details the specific task objectives to be completed at this stage.

[0175] (2) Response paradigm extraction. By inputting a set of similar cases and corresponding prompts, prompts such as Figure 3 As shown, the model is guided to perform the following core tasks case by case: First, it summarizes response patterns for similar cases. Second, it extracts essential content, precisely identifying the key elements that must be included in a response. During this process, the large language model accurately identifies and extracts the core response elements from each case.

[0176] (3) Standardize content elements. Through certain prompt information, prompts such as Figure 3 As shown, the large language model is guided to adaptively generalize the summarized response paradigm and apply it to the current political inquiries to be processed.

[0177] (4) Standardize the expression elements. Tips: Figure 3 As shown in the figure, by emphasizing key information such as text expression, tone, and intonation, the large language model is prompted to summarize the emotional expression and text expression that should be followed in the response based on similar cases.

[0178] 3.4.2 Refinement and Integration of Responses to Political Questions

[0179] After completing the above framework generation work, the next step is to fill in the specific content in the template in a fine-grained manner. First, clarify the main goal of detailed integration: according to the response framework generated in the previous article, fill in the actual policy documents, phone numbers, email addresses and other information into the corresponding positions to ensure that the response content is more specific, detailed, and in line with the actual situation. Secondly, by integrating this information, ensure the coherence of the response, make it look more natural and humane, and avoid obvious traces of human-computer interaction. The specific detailed integration process is as follows: Figure 4As shown in the figure, its core consists of two key components: the hint engineering as the main architectural element of the hint template in the refinement and integration process, and the refinement and integration of the DeepSeek model compared to the framework in this process. This part can intuitively demonstrate the design concept of this process.

[0180] (1) Information filling and refinement. At this stage, it is necessary to fill in the corresponding positions in the framework with relevant policy clauses, contact information and other specific details based on the content of each specific inquiry and the actual situation. For example, if the inquiry involves the implementation of a specific policy, the response should clearly quote the number or specific clause of the policy document; if the inquiry involves contacting relevant departments or personnel, accurate contact information such as telephone numbers and email addresses should be provided. The key to this process is to fill in the information accurately to ensure the pertinence and practicality of the response.

[0181] (2) Information integration and optimization. After filling in the information, the next step is to integrate the content. Figure 4 As shown, this process requires reorganizing individual pieces of information into a logical sequence, creating a smooth, coherent response. During this process, particular attention should be paid to the cohesion and transitions between sentences to avoid appearing too fragmented or jumpy. Furthermore, consistency in tone and style should be ensured, minimizing overly mechanical expressions to create a more natural, communicative response. It is particularly important to note that while optimizing content, efforts should be made to eliminate the traces of human interaction inherent in the enumerated responses within the template, thereby increasing semantic coherence and textual affinity.

[0182] 4 Experimental design and result analysis

[0183] 4.1 Experimental design

[0184] This experiment uses a Python program to crawl the online government inquiry platform in Luzhou, Sichuan Province, China, from 2020 to 2023, including three fields: inquiry content, responding unit, and response content. First, data cleaning is performed to remove duplicates, meaningless characters, and meaningless content from the original data, resulting in a total of 45,805 online government inquiry data. 1,500 cases are randomly selected from the dataset to generate 1,500 inquiry cases as the test set, and the remaining 44,305 cases are used as the original case library. Based on this data, the following three experiments are designed:

[0185] (1) Similar case retrieval quality experiment, analyzing the retrieval quality of the similar claim case retrieval method proposed in the text.

[0186] (2) Similar case retrieval efficiency experiment, analyzing the retrieval efficiency of the case library reconstruction method proposed in the text.

[0187] (3) Experiment on the quality of response generation to political questions, analyzing the quality of response enhancement generated text.

[0188] 4.2 Similar Case Retrieval Quality Experiment

[0189] In this experiment, the similar appeal case retrieval method proposed in the present invention was used to retrieve 4 similar cases for each of the 1,500 test set case appeal samples. This task is an unsupervised task, and the experimental data is real political inquiry data crawled from the web, so it is difficult to evaluate the retrieval results from the perspectives of accuracy, recall, etc. Taking into account that the public's political inquiries are ultimately responded to and fed back by relevant government departments, it is possible to consider using the matching degree of the departments corresponding to similar cases to evaluate the retrieval quality. In addition, the degree of topic relevance can also measure the reliability of the retrieval results from one aspect. Therefore, the present invention designs similar case retrieval quality indicators based on the response department matching degree and keyword matching degree. RC__num represents the number of similar cases, and RC__simd represents the number of cases where the processing department of the retrieved similar cases is the same as the experimental sample Ci. The response department matching degree dept__sim can be used CK__num is used to represent the number of keywords in the sample appeal, and RCK__num is used to represent the number of keywords that overlap between similar cases and experimental samples. To express the keyword matching degree between a similar case and the experimental sample. The average keyword matching degree of RC__num similar cases It can represent the keyword matching degree kw__sim for experimental sample C. The retrieval quality rec__qual for experimental sample C can be represented by the average of the response department matching degree and the keyword matching degree. Its calculation formula is shown in Formula 4.

[0190]

[0191] The advantage of the similar case retrieval method proposed in this paper lies in embedding citizens' appeal themes within text vectors to highlight their primary appeals, thereby improving the effectiveness of similar case retrieval. To verify the effectiveness of topic embedding on similar case retrieval, the present invention used experimental results based on BERT and MacBERT vectors without appeal theme embedding as two control groups. The BERT pre-trained model selected was bert-base-chinese, and the MacBERT pre-trained model was chinese-macbert-base. The key parameters of the two models were the same, as shown in Table 3.

[0192] Table 3 Pre-training model parameters

[0193]

[0194] The experimental results of similar case retrieval quality are shown in Table 4. It can be seen that the online political inquiry similar case retrieval method proposed in the present invention performs best in both response department matching and keyword matching, and has the highest retrieval quality index. The comparative advantages of MacBERT and BERT show that the pre-trained model trained with Chinese corpus has obvious advantages in text similarity calculation. Taking advantage of MacBERT's advantages in Chinese vector representation, after incorporating text topic keywords into the appeal text, the response department matching degree increased by 0.04, the keyword matching degree increased by 0.09, and the retrieval quality index increased by 0.06. This shows that, as analyzed above, the integration of text topics can effectively highlight the main demands of citizens, thereby improving the quality of retrieval of similar cases.

[0195] Table 4 Comparison of the quality results of retrieval of similar cases

[0196]

[0197] In order to more intuitively demonstrate the effect of the retrieval method proposed in the present invention, the present invention selects and displays the first three similar cases of three typical cases with significant characteristics: provident fund withdrawal (the length of the inquiry text is relatively long and the semantics are relatively complex), airport route planning (containing a large number of place names), and freight qualification certificate (the length of the inquiry text is relatively short).

[0198] 1) Provident Fund Withdrawal

[0199] The inquiry text reads, "Hello, I work at Luzhou Tuzhu School, but my registered residence is in Chongqing, and I also purchased a house there. I last contacted the Municipal Provident Fund Center, and they said I can withdraw my provident fund in Lu County if I purchase a house in my registered residence. I've been paying my mortgage for a year and have been paying into the provident fund for over a year. What procedures are required for my first withdrawal?" The inquiry department is the Municipal Provident Fund Center. This inquiry text is long and semantically complex. The top three similar cases and their similarity levels are shown in Table 5. This shows that for long and semantically complex provident fund withdrawal inquiries, the recommendation algorithm can provide high-similarity and high-quality inquiries for reference by the Municipal Provident Fund Center staff.

[0200] Table 5 Recommended results of similar cases of provident fund withdrawal claims

[0201]

[0202] 2) Airport route planning

[0203] The inquiry text reads, "I would like to inquire about when Luzhou Airport will be able to introduce new airlines, or which airlines it plans to introduce, whether new routes will be added this season, and when are flights to Xichang, Nanjing, Dali, Changchun, Shenyang, Xinyang, and Ordos expected to resume?" The inquiry was made by Luzhou Airport (Group) Co., Ltd. This inquiry text is long, has a clear topic, and contains numerous place names. The top three similar cases and their similarity scores are shown in Table 6. It can be seen that for this long inquiry text about airport route planning that contains numerous place name keywords, the recommendation algorithm can provide high-similarity and high-quality inquiry cases for reference by Luzhou Airport Co., Ltd. staff. The route plans presented in the recommended cases are highly consistent with those in the inquiry text.

[0204] 3) Freight qualification certificate

[0205] The question text is "Does the freight practitioner qualification certificate need to continue studying and taking exams every two years?" The questioning department is the Municipal Transportation Bureau. The question text is short in length and has clear question keywords. The top three similar cases and their similarities are shown in Table 7. It can be seen that for the freight practitioner qualification certificate question text with a short length and clear question theme, the recommendation algorithm can provide high-similarity and high-quality question cases for reference by the staff of the Municipal Transportation Bureau. The content of the recommended case is highly consistent with the question text. Combining the objective and subjective evaluations of the recommendation quality, it can be concluded that the keyword-embedded case vector representation has a positive impact on case recommendation and can effectively

[0206] Improve the quality of case recommendations.

[0207] Table 6 Recommendation results of similar cases for airport route planning

[0208]

[0209] Table 7 Recommended results of similar cases for freight practitioner qualification certificate claims

[0210]

[0211] 4.3 Similar Case Retrieval Efficiency Experiment

[0212] When reconstructing the online political inquiry case library, the number of clusters will directly affect the efficiency of similar case retrieval. The more clusters there are, the smaller the scale of the reorganized sub-case library will be, and the smaller the amount of similarity calculation performed in the sub-case library will be. However, in terms of retrieval effect, the more clusters there are, the smaller the sub-case library recommended for screening each time, and the worse the recommendation effect will be. Therefore, when reconstructing the case library, the number of clusters must be determined first to ensure a balance between efficiency and effect. From the perspective of similar case retrieval applications, if the reconstructed sub-case libraries each represent a clear category of demands, that is, the boundaries between classes are clear, this will simultaneously guarantee retrieval efficiency and effect. Clear cluster boundaries are consistent with the pursuit of the K-Means++ algorithm. Therefore, the present invention will select a suitable number of clusters for case library reconstruction based on the relevant performance indicators of cluster quality assessment.

[0213] The method for selecting the number of clusters depends on the distance measurement method between text vectors. When Euclidean distance is used, methods such as the Kalinsky-Harabas index (CH), the sum of squared errors (SSE), the Davis-Boulding index (DBI) and the interval statistic are commonly used; if other distance metrics are used, the optimal number of clusters can be determined by the silhouette coefficient (Silhouette Coefficient). The citizen appeal text vectors processed by the present invention have all been standardized (L2 norm normalized to unit vectors). Since the squared Euclidean distance and the cosine distance are equivalent in cluster determination, the present invention comprehensively adopts the sum of squared errors within the group and the silhouette coefficient to select the number of clusters. The sum of squared errors within the group reflects the degree of tightness within the cluster, and the silhouette coefficient comprehensively measures the cohesion within the cluster and the separation between clusters. The combination of the two can effectively determine the optimal number of clusters. By drawing curves of the two as the number of clusters changes, the clustering effect can be intuitively observed.

[0214] like Figure 5 As shown in the figure, the silhouette coefficient reaches its peak when the number of clusters is 7, indicating that the similarity of samples within the cluster is high, the differences between clusters are obvious, and the cluster structure is reasonable. The SSE of the elbow diagram tends to be flat after the number of clusters is 10, indicating that increasing the number of clusters no longer significantly improves the compactness within the cluster. When the number of clusters is 7, the elbow diagram has a more obvious decline, the silhouette coefficient has a maximum value, and the value drops significantly after 7. Therefore, considering all factors, the number of clusters in this experimental data is determined to be 7. In order to further observe the effect of clustering, the present invention calculates the TF-IDF score for the first 50 keywords of each cluster, and uses the first 10 keywords of the cluster as the theme representation of the cluster. The clustering results and keywords are shown in Table 8, and the TF-IDF score is in brackets. The case library is divided into seven sub-case libraries: urban construction, urban transportation and public order, urban planning, housing, social security and provident fund, public service, professional qualification and education examination. Each keyword shows a clear theme feature.

[0215] To verify the impact of appeal content clustering on case recommendation efficiency, we designed two comparative experiments based on the full case library and the reconstructed case library. We retrieved 10 similar cases from the 1,500 cases in the test set and calculated the retrieval time. The results are shown in Table 9. As can be seen, the retrieval efficiency of the reconstructed case library is 5.39 times higher than that of the full case library, demonstrating that similar cases based on appeal content clustering significantly improve recommendation efficiency.

[0216] 4.4 Experiment on the Quality of Political Response Generation

[0217] In current research, there is still a significant gap in the evaluation indicators of automatic response generation systems in the field of political inquiry. Existing general retrieval-enhanced generation evaluation systems (such as RAGAs) have two limitations when applied to the evaluation of political inquiry texts: First, in terms of technology, existing methods have the problem of insufficient structural adaptation. The core indicators designed by frameworks such as RAGAs (such as answer fidelity and contextual relevance) are mainly aimed at segment-level semantic alignment, and their evaluation mechanism has two flaws: 1) There is a lack of quantitative standards for the global coherence of long texts (such as logical connection across paragraphs and thematic consistency); 2) The semantic parsing ability of unstructured political inquiry content (such as multi-level administrative term nesting and policy clause citation) is insufficient; secondly, in terms of domain characteristics, general indicators fail to cover the core requirements of political inquiry responses. Online political inquiry texts have distinct government attributes, and their quality assessment requires comprehensive consideration: 1) Conformity with administrative regulations: including official document format standards (such as the "three elements" structure) and accuracy of policy expressions; 2) Emotional appropriateness: the need to balance professionalism and accessibility, avoiding bureaucracy or excessive colloquialism; 3) Logical traceability: requiring a closed-loop argumentation of policy basis, treatment plan, and responsible departments; therefore, the text constructs corresponding indicators based on the characteristics of online political inquiry texts.

[0218] Based on the evaluation framework of the RAGAs index system, this paper constructs a fine-grained hierarchical evaluation method targeting the multi-question characteristics of political inquiry texts.

[0219] (1) Factual relevance

[0220] Factual Correlation, a key metric for evaluating the quality of responses to online government inquiries, focuses on measuring the authenticity and reliability of generated content, aiming to ensure that government responses withstand factual verification and logical scrutiny. The core evaluation logic of this metric deeply examines potential factual errors and logical contradictions within the response text, focusing on examining whether there are inconsistencies, causal inversions, and other issues within the response text. This metric is calculated by comparing the response information with the underlying facts and the text of the appeal. The answer value is scaled to a range of (0, 1), with higher values ​​indicating better factual correlation.

[0221] If the generated response information is consistent with the basic facts and conforms to common sense, and there are no logical loopholes in the text context, then the response information is considered to be factually relevant. Specifically, the factual relevance evaluation system adopts a three-level evaluation framework:

[0222] Text deconstruction layer: Considering the common complex question characteristics of political inquiry texts (i.e., a single political inquiry may contain multiple interrelated sub-questions), this paper uses a large language model to perform semantic segmentation on the political inquiry texts;

[0223] Statement Assessment: A multi-dimensional scoring system is set up for each deconstructed statement. The scoring mechanism has the following characteristics: 1) Scoring is based on a continuous value range of 0-1, with four decimal places of accuracy; 2) Each statement is evaluated independently to avoid interference between scores of other questions;

[0224] Comprehensive Calculation Layer: After obtaining the scores for all statements, the average is taken to calculate the overall score. This tiered evaluation method effectively addresses two major challenges that traditional evaluation systems face when dealing with complex political inquiries: 1) scoring bias caused by the coupling of multiple questions; and 2) standardization difficulties caused by a variable number of questions.

[0225] Let R be the set of response information generated by the test set; G be the original response information in the test set; S be the appeal text in the test set; R i is the response information corresponding to each case in each test set; G i is the original response information corresponding to each case in the test set; n represents

[0226] Number of test set cases; S i is the appeal text corresponding to each case in the test set; φI represents the calculation of each statement using the large language model.

[0227] Real correlation score; is a connection symbol. The specific calculation method is as shown in Formula 5:

[0228]

[0229] (2) Answer relevance

[0230] Answer relevance is designed to assess the pertinence and completeness of the response information, determine whether the response information is relevant to the question text, whether there are any missing questions or whether it contains redundant information or irrelevant answers. The evaluation of answer relevance continues the three-level evaluation framework of factual consistency. The formula for this indicator is defined as the average semantic similarity between the question text and a series of artificial questions generated based on basic facts. The specific calculation method is as shown in Formula 6:

[0231]

[0232] Among them, R is the response information set generated by the test set; S is the appeal text in the test set; R i is the response information corresponding to each case in each test set; n represents the number of test set cases; S i is the appeal text corresponding to each case in the test set; φr represents the relevance score of each statement fact calculated using the large language model; is a connection symbol. The fact relevance score can be calculated as follows.

[0233] (3) Attitude suitability

[0234] Attitudinal Appropriateness evaluates whether the response attitude is appropriate. Yao Lan's research indicates that in government-media converged platforms for political inquiry, the phenomenon of "mutual buck-passing" is a significant factor affecting the quality of responses to political inquiries and should be considered a key dimension for measuring response attitudes. Furthermore, some scholars believe that the use of empathetic language is widely considered a key indicator for measuring information service quality. Based on the above research, this article defines "attitude appropriateness" as: evaluating whether there is buck-passing in response attitudes, whether the use of empathetic language enhances emotional resonance, and measuring whether the text effectively conveys concern, empathy, and a sense of responsibility. Specifically, it includes two aspects: 1) No buck-passing: whether the conflict is not diverted for issues that are truly within the scope of responsibility, and whether the subsequent path for handling the incident is clear; 2) Empathetic language: whether caring language (such as "attach great importance" and "deeply apologize") is used.

[0235] Attitude appropriateness is scored using a large language model in two areas: "no evasion" and "empathy." This paper uses an equally spaced scoring method for evaluation.

[0236] Table 10 Attitude suitability scoring criteria

[0237]

[0238] Based on the aforementioned evaluation metrics, we used a large language model to evaluate the generated responses. Although the values ​​generated by the large language model are subject to fluctuation, repeated evaluations can, to a certain extent, reflect the true performance of the RAG application.

[0239] The choice of generative model depends on its performance in the evaluation indicators. This paper uses three large models, DeepSeek, Qwen and Baichuan2, for comparison, and uses the method proposed in this paper to generate responses. Qwen (Tongyi Qianwen) is a large language model developed by Alibaba Cloud. It aims to provide an efficient, open source, and commercially available large language model that supports tasks such as text, code, and mathematical reasoning, and performs well in multiple benchmarks. Baichuan2 is a new generation of open source large language model launched by Baichuan Intelligence. It achieves the best results of the same size on authoritative Chinese and English benchmarks. Not only that, it has demonstrated excellent performance in tasks in specific fields (such as medicine and law). This provides strong support for applications in specific fields. In addition, in order to comprehensively evaluate the effectiveness of the question-and-answer response generation method, the present invention also sets up a baseline model for comparative experiments. The baseline model uses the retrieval enhancement generation method proposed in this paper, and only uses the ChatGLM model to directly generate responses according to the given response requirements. This paper uses the test set to evaluate response generation, and the results are shown in Table 11:

[0240] Table 11 Model response quality comparison experiment

[0241]

[0242] The experimental results of the present invention show that the DeepSeek model exhibits optimal performance in all three evaluation indicators. This advantage mainly stems from the following two factors: First, DeepSeek demonstrates stronger semantic understanding and analogical reasoning capabilities in logical reasoning tasks, can more accurately parse similar case information, and conduct more logical reasoning analysis under the guidance of structured prompt templates; secondly, from the perspective of model architecture, the DeepSeek model used in this experiment has a parameter volume of 70 billion, while the Baichuan2 and Qwen models used as the control group have a parameter volume of 7 billion. Existing studies have shown that the number of model parameters is positively correlated with reasoning performance, so a larger-scale model architecture provides DeepSeek with a stronger reasoning capability foundation, which has also been verified in the experimental results. Therefore, the present invention selects DeepSeek as the question-and-response generation model.

[0243] At the same time, large language models, after applying the retrieval-augmented generation method developed in this paper, significantly improved their performance in policy response generation compared to the baseline model, ChatGLM. Baichuan2, Qwen, and DeepSeek achieved 12.44%, 10.29%, and 19.37% improvements in factual relevance, respectively, after using the response generation method; they also achieved 6.43%, 5.13%, and 16.35% improvements in answer completeness; and Baichuan2 and DeepSeek achieved 3.41% and 12.7% improvements in attitude appropriateness, respectively. Experimental results show that while the Qwen model did not improve attitude appropriateness, it showed significant improvements in factual relevance and answer completeness. Overall, the proposed method can effectively leverage similar cases to generate responses to online political inquiries under the guidance of prompt templates, achieving significant improvements in content authenticity, information completeness, and emotional expression.

[0244] To determine whether the retrieval and prompt modules effectively improve the quality of model response generation, we evaluated the proposed retrieval-enhanced generation method using various metrics based on a constructed test set. Table 12 reports the results of the ablation experiment conducted on the test set.

[0245] Table 12 Comparison of the quality of responses to government inquiries

[0246]

[0247] Experimental results show that compared to the unenhanced basic large model, this hybrid approach demonstrates significant advantages in key evaluation metrics: an 8.12% improvement in factual relevance, a 3.8% improvement in answer completeness, and a 3.61% improvement in attitude appropriateness. In particular, compared to simply incorporating prompt templates or learning from similar cases, this approach also achieves improvements on all three metrics. This performance improvement is primarily due to the following reasons: relevant clauses and cases ensure the policy-based and factual accuracy of the answers; at the same time, the optimized prompt templates, through structured guidance, effectively integrate similar case responses into the generation process, standardizing the format, content organization, and presentation of the responses. The combined application of these two approaches not only strengthens the factual basis of the generated content, but also improves the degree of alignment between the answers and the political demands, thereby improving the overall response quality of the system.

[0248] In particular, the results of this study show that the prompt module and the retrieval module both play a key role in the response generation process: (1) The effectiveness of the prompt template. Compared with the unoptimized basic large model, the responses to the political questions generated using the prompt template in this study have improved in key evaluation indicators: factual relevance increased by 0.19%, answer completeness increased by 0.34%, and attitude suitability increased by 0.48%. This improvement is mainly attributed to the two core optimization mechanisms of the prompt template: First, by systematically analyzing the key information elements that the political question response should contain, the logic of the large model generation is enhanced, and the emotional expression of the generated political question response is improved, so that the template effectively guides the large model to achieve comprehensive improvement in format standardization, content completeness, expression accuracy and emotional suitability. Second, the gradual refinement strategy from "coarse" to "fine" is adopted to effectively improve the common problem of generalization in the responses of the large model. However, the effect of improving factual relevance is not very obvious, mainly because the prompt template does not improve the "hallucination" problem of the large model. (2) The effectiveness of the retrieval model. Responses generated based on learning similar examples show a 7.14% improvement in factual relevance, a 3.54% improvement in answer completeness, and a 3.58% improvement in attitude appropriateness compared to the large-scale model alone. This improvement stems from the fact that similar examples learned from the retrieval model provide a reliable "factual anchor" for the generation process, effectively constraining the generation space of the large language model.

[0249] 4.5 Discussion of experimental results

[0250] (1) Similar case retrieval quality experiment: Integrating text topics to improve retrieval accuracy

[0251] Experimental results demonstrate that the proposed method for retrieving similar cases from online political inquiries achieves optimal performance in both responding department and keyword matching, with retrieval quality indicators significantly exceeding those of traditional methods. This conclusion validates the theoretical analysis presented above: by integrating text topic features, the model can more accurately capture the core semantics of citizen appeals, effectively highlighting key information within them. This topic fusion mechanism not only improves the comprehensiveness of case matching but also, through proximity calculations in semantic vector space, achieves a deeper understanding of the essence of appeals, providing more accurate case references for subsequent response generation.

[0252] (2) Similar Case Retrieval Efficiency Experiment: Case Library Reconstruction Achieves Efficiency Improvement

[0253] Case database reconstruction experiments have shown that the sub-case database constructed using text clustering methods increases the retrieval speed of similar cases by 5.93 times. This efficiency improvement is mainly due to the division of the full case database into N thematic sub-databases according to themes. During retrieval, only the target sub-database needs to be traversed instead of the entire database, significantly reducing computational complexity. In addition, the incremental clustering method dynamically updates the sub-database structure, accurately locating the most relevant sub-database for insertion when a new case is added, avoiding the exponential computational cost of re-clustering the entire database and showing good scalability.

[0254] (3) Experiment on the quality of response generation: RAG technology enhances response reliability

[0255] Experimental results on response generation show that responses generated using Retrieval-Augmented Generation (RAG) technology improve factual relevance by 8.12%, answer completeness by 3.8%, and attitude appropriateness by 3.61% compared to the pure large-scale model. This improvement stems from the dual advantages of the RAG framework: first, the retrieval module extracts similar cases from the case library, providing reliable factual anchors for the generation process, effectively constraining the generation space of the large-scale model; second, by designing prompt templates that semantically integrate citizen appeals with similar cases, the generation model acquires richer contextual information when understanding the context.

[0256] (4) General discussion and future prospects

[0257] Through three experiments, the effectiveness of the network response retrieval enhancement generation method proposed in this invention that integrates a large language model was verified. However, there is still room for improvement. Future attention should be paid to the following: Dynamic knowledge update mechanism: Establishing a real-time synchronization channel between policy texts and case libraries, automatically triggering and fine-tuning the generation model when new policies are released; Multimodal information fusion: Explore incorporating structured data such as netizen sentiment analysis and regional characteristics into the retrieval dimension to construct a more three-dimensional appeal representation space; Optimizing the stability of generation results: Introducing reinforcement learning to impose strategic constraints on the generation process, and guiding the model output to maintain a higher degree of consistency with the official caliber through a reward mechanism.

[0258] Based on the above embodiments, an embodiment of the present application further provides a computer program, which, when executed on a computer, enables the computer to execute the method provided in the above embodiments.

[0259] Based on the above embodiments, an embodiment of the present application further provides a computer storage medium, in which a computer program is stored. When the computer program is executed by a computer, the computer executes the method provided in the above embodiments.

[0260] The storage medium may be any available medium that can be accessed by a computer. By way of example and not limitation, computer-readable media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage media or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer.

[0261] Based on the above embodiments, an embodiment of the present application further provides a chip, which is used to read a computer program stored in a memory to implement the method provided in the above embodiments.

[0262] Based on the above embodiments, an embodiment of the present application provides a computer program product, which implements the methods provided in the above embodiments when the computer program product is run on an electronic device.

[0263] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0264] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0265] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0266] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0267] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. A network response retrieval enhancement generation method integrating a large language model, characterized in that: include Input the political demand text and prompts into the large language model, and the large language model will output the political demand and type; Input similar cases and prompts of the appeal text into the large language model, and the large language model outputs the response paradigm for the appeal of the type in similar cases; Input prompts to the large language model, which generalizes the response paradigm and generates a first framework suitable for responding to the appeal; Among them, the method of obtaining similar cases of the appeal text includes: Calculate the appeal vector after keyword embedding of the appeal text; Determine the target sub-case library based on the cosine distance between the appeal vector and the center of each sub-case library clustered based on the appeal content; Similar cases are determined based on the cosine distance between the appeal vector and the appeal vector of the cases in the target sub-case library.

2. The network response retrieval enhancement generation method integrating a large language model according to claim 1 is characterized in that: It also includes inputting prompts into the large language model, and the large language model limits the text expression and tone of the first frame suitable for responding to the appeal, and generates a second frame suitable for responding to the appeal.

3. The network response retrieval enhancement generation method integrating a large language model according to claim 1 is characterized in that: It also includes inputting filling information for the first frame or the second frame into the large language model, where the filling information includes policy information and / or contact information; and the large language model outputs response content including the first frame or the second frame and the filling information.

4. The network response retrieval enhancement generation method integrating a large language model according to claim 3 is characterized in that: Prompts are input to the large language model, which then outputs responses that are fluent, coherent, consistent in tone and style.

5. The network response retrieval enhancement generation method integrating a large language model according to claim 1 is characterized in that: The step calculates the appeal vector after keyword embedding of the appeal text, including Appeal text c i Perform word segmentation and remove stop words; The appeal text c after word segmentation i Pass the KeyBERT model to obtain the first q keywords; Concatenate q keywords into a string to get text s, separated by spaces; Convert text s into vector kv i ; The appeal text c i Convert to vector cv i ; The vector cv i and vector kv i The method of first summing and then L2 norm normalization is adopted to obtain the appeal vector ckv after keyword embedding of the appeal text. i ; Among them, the vector cv i and vector kv i The fusion is expressed by the following formula using the method of summing first and then L2 norm normalization: Where m represents the dimension of the vector.

6. The network response retrieval enhancement generation method integrating a large language model according to claim 5 is characterized in that: in, In the step, the text s is converted into a vector kv i ,include Perform word segmentation on the text s; Pass the segmented text s into the MacBERT model and obtain the output of the last hidden layer of each token; The vector after the average pooling of the hidden layer output of all tokens is used as the vector representation kv of the appeal text s i ; Among them, the step will appeal to the text c i Convert to vector cv i ,include Appeal text c i Perform word segmentation processing; The appeal text c after word segmentation i Pass the MacBERT model and get the output of the last hidden layer of each token; The vector after the average pooling of the hidden layer output of all tokens is used as the appeal text c i The vector representation of cv i .

7. The network response retrieval enhancement generation method integrating a large language model according to claim 1 is characterized in that: in, The construction method of the sub-case library based on appeal content clustering includes: S1. Randomly select a sample from the case data set as the first cluster center; S2. For each sample in the dataset, calculate the shortest distance between it and the selected cluster center, and calculate the probability of each sample being selected as the next cluster center based on the shortest distance; S3. Based on the calculated probability, randomly select a new sample as the next cluster center, and repeat steps S2-S3 until k cluster centers are selected; S4. Calculate the distance between the sample and each cluster center, and assign the sample to the cluster center with the shortest distance to it; S5. For each cluster center, recalculate the mean similarity between all sample points belonging to it and use the mean as the new cluster center; S6. Repeat steps S4-S5 until the convergence condition is reached and the iteration is stopped. All sample points belonging to each cluster center are a sub-case library, and t sub-case libraries are obtained by clustering the case data set based on the content of the appeal; Among them, it is assumed that after the tth iteration, the cluster C i The cluster center vector of becomes The change range of its cluster center is shown in formula (2). If the change range of the cluster center of k classes satisfies formula (3), and the sum of the changes of the cluster centers of all categories is less than the preset threshold θ, the cluster center is stable and the iteration stops when the convergence condition is reached.

8. The network response retrieval enhancement generation method integrating a large language model according to claim 1 or 7 is characterized in that: In the step, the target sub-case library is determined according to the distance between the appeal vector and the center of each sub-case library clustered based on the appeal content; similar cases are determined according to the cosine distance between the appeal vector and the appeal vector of the case in the target sub-case library; include The case data set is clustered into t sub-case libraries based on the appeal content, and the cluster center vector ui of each sub-case library is used to calculate the appeal vector ckv i The cluster center vector u of the t sub-case databases clustered based on the appeal content i The cosine distance of the demand vector ckv i The sub-case library with the smallest cosine distance is used as the target sub-case library; Traverse the target sub-case library and calculate the demand vector ckv i The cosine distance between the appeal vectors of all cases in the target sub-case library and the target sub-case library, wherein the case or the first few cases with the smallest cosine distance are similar cases.

9. The network response retrieval enhancement generation method integrating a large language model according to claim 7 is characterized in that: Also includes For new cases to be stored, extract the appeal vector ckv of the appeal text in the case i ; Calculate the demand vector ckv i The cluster center vector u of the t sub-case databases clustered based on the appeal content i The cosine distance of Add the new case to be stored to its cluster center vector and appeal vector ckv i In the sub-case library with the smallest cosine distance, the cluster center vector of the sub-case library is u; For the cluster center of the sub-case library, recalculate the mean similarity between all sample points to which it belongs, and use the mean as the new cluster center; Calculate the offset between the new cluster center and the original cluster center. If the offset is greater than the set threshold θ1, execute the method of constructing a sub-case library based on the clustering of the appeal content, and reallocate the cluster center to which the sample points of each case belong in order to reallocate the sub-case library. If the offset is not greater than the set threshold θ1, the cluster center to which the sample points of each current case belong is maintained.

10. The network response retrieval enhancement generation method integrating a large language model according to claim 1 is characterized in that: The online response is a response to an online political inquiry, and the response content is the content of the political inquiry response.

11. An electronic device, comprising: One or more processors, a memory, and one or more programs; wherein the one or more programs are stored in the memory, and the one or more programs include instructions that, when executed by the electronic device, cause the electronic device to perform the method of any one of claims 1-10. 12 . A computer-readable storage medium comprising a computer program, wherein when the computer program is executed on an electronic device, the electronic device is caused to execute the method according to claim 1 .

Images