Private domain-oriented large language model personalized question and answer method and related products

Abstract

The application discloses a large language model individualization question and answer method for a private domain and related products, relates to the technical field of large language model question and answer, and comprises the following steps: matching a rewritten question with data in an external database to obtain a relevant context set; clustering the relevant context set to obtain a plurality of category clusters; inputting each category cluster into a large language model to obtain summary data of each category cluster; and inputting the summary data and an original question into the large language model to obtain an answer to the question. The rewritten question in the application is a question obtained by using parameter knowledge of the large language model to complete and adjust the original question, so that the quality of the user question is effectively improved; the relevant context set is clustered to obtain a plurality of category clusters; and each category cluster is inputted into the large language model to obtain summary data of each category cluster, so that the input of low-quality search results can be avoided.

Description

Technical Field

[0001] This application relates to the field of large language model question answering technology, and in particular to a personalized question answering method and related products for large language models in the private domain. Background Technology

[0002] In past research and applications, traditional Large Language Models (LLMs) question answering techniques have demonstrated excellent general knowledge capabilities, directly answering user questions by utilizing the model's internal parameter knowledge. However, in applications with limited use cases, high levels of specialization, and scarce relevant data in private knowledge domains, problems such as illusions, information loss, and incomplete answers arise. Currently, many studies have developed a popular and reliable solution—Retrieval Augmentation Generation (RAG) technology—by injecting relevant knowledge from private domains into external databases as augmenting data to large language models.

[0003] For the private knowledge domain, past research has shown that retrieval augmentation techniques encounter several challenges. First, because the quality of user questions cannot be guaranteed, retrieval augmentation techniques struggle to retrieve all relevant blocks during the retrieval phase, risking the loss of key information and resulting in inconsistent search results. Second, if low-quality search results are input into a large language model during the augmentation phase, it can mislead the model, causing it to misperform and reducing the quality and reliability of the responses. Summary of the Invention

[0004] The purpose of this application is to provide a personalized question-answering method and related products for large language models in private domains, which can improve the quality of user questions, avoid inputting low-quality search results, and improve the quality and reliability of answers.

[0005] To achieve the above objectives, this application provides the following solution:

[0006] Firstly, this application provides a personalized question-answering method for large language models in a private domain, the personalized question-answering method for large language models in a private domain including:

[0007] The user's original question is input into the large language model to obtain the rewritten question; the rewritten question is the question after the large language model uses its own parameter knowledge to complete and professionally adjust the original question;

[0008] The rewriting problem is matched with data in an external database to obtain a relevant context set;

[0009] Clustering the relevant context set yields several category clusters;

[0010] Each of the aforementioned category clusters is input into the large language model to obtain summary data for each of the aforementioned category clusters;

[0011] The summarized data and the original question are input into a large language model to obtain the answer to the question.

[0012] Optionally, the relevant context set includes several documents; clustering the relevant context set yields several category clusters, specifically including:

[0013] Calculate the similarity between each document in the relevant context set and the original question;

[0014] Sort all documents in the relevant context set by similarity to obtain a sorted set;

[0015] Obtain the top k documents from the sorted set to get the similarity set;

[0016] The similarity sets are clustered to obtain several category clusters.

[0017] Optionally, the relevant context set is clustered to obtain several category clusters, specifically including:

[0018] The relevant context set is vectorized to obtain a vectorized set;

[0019] KMeans clustering is performed on the vectorized set to obtain several category clusters.

[0020] Optionally, the rewriting problem is matched with data in an external database to obtain a relevant context set, specifically including:

[0021] The rewriting problem and the original problem are input into an external database for matching and retrieval to obtain a relevant context set.

[0022] Optionally, the formula for obtaining the rewriting problem is:

[0023] qSet = LLM(q, n);

[0024] Where n is a hyperparameter, n=4, q is the original problem, qSet is the rewriting problem, and LLM is the large language model.

[0025] Optionally, the method for constructing the external database specifically includes:

[0026] The text data is segmented to obtain text data blocks;

[0027] The text data block is encoded using a vectorization model to obtain an external database.

[0028] Secondly, this application provides a personalized question-answering device for a large language model in a private domain, the personalized question-answering device for a large language model in a private domain includes:

[0029] The user input question module is used to receive the original question input by the user;

[0030] The private domain knowledge expansion module based on question rewriting is used to input the user's original question into a large language model and obtain a rewritten question; the rewritten question is the question after the large language model uses its own parameter knowledge to complete and professionally adjust the original question;

[0031] The private domain knowledge base retrieval module is used to match the rewriting problem with data in an external database to obtain a relevant context set;

[0032] The context clustering module is used to cluster the relevant context set to obtain several category clusters;

[0033] The clustering knowledge simplification module is used to input each of the aforementioned category clusters into the large language model to obtain summary data for each of the aforementioned category clusters;

[0034] The answer generation module is used to input the summarized data and the original question into the large language model to obtain the answer to the question.

[0035] Thirdly, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the private domain-oriented large language model personalized question answering method described above.

[0036] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the private domain-oriented large language model personalized question-answering method described above.

[0037] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the private domain-oriented large language model personalized question-answering method described above.

[0038] According to the specific embodiments provided in this application, the following technical effects are disclosed:

[0039] This application provides a personalized question-answering method and related products for private domains using a large language model. It involves inputting the user's original question into a large language model to obtain a rewritten question. The rewritten question is a question that the large language model completes and professionally adjusts using its own parameter knowledge, thus solving the problem of inconsistent user question quality and effectively improving the quality of user questions. The method further involves clustering the relevant context set to obtain several category clusters. Each category cluster is then input into the large language model to obtain summary data for each cluster, thus avoiding the input of low-quality search results. This application improves the quality of user questions while avoiding the input of low-quality search results, thereby enhancing the quality and reliability of the answers. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 An application environment diagram for a personalized question-answering method for a large language model oriented to private domains provided in this application embodiment;

[0042] Figure 2 A flowchart illustrating a personalized question-answering method for a large language model oriented towards private domains, provided in Embodiment 1 of this application;

[0043] Figure 3 This is a schematic diagram of the private domain knowledge expansion scheme based on problem rewriting provided in Embodiment 1 of this application;

[0044] Figure 4 A schematic diagram of a context simplification scheme based on clustering and summarization provided in Embodiment 1 of this application;

[0045] Figure 5 This is a schematic diagram of a supplementary solution to user problems provided in Embodiment 1 of this application;

[0046] Figure 6 This is a schematic diagram of a personalized question-answering device for a large language model oriented to the private domain, provided in Embodiment 2 of this application.

[0047] Figure 7 This is a schematic diagram of a personalized question-answering method for a large language model oriented towards private domains, provided in Embodiment 3 of this application.

[0048] Figure 8This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0049] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0050] In past research and applications, traditional Large Language Models (LLMs) question answering techniques have demonstrated excellent general knowledge capabilities, directly answering user questions by utilizing the model's internal parameter knowledge. However, in applications with limited use cases, high levels of specialization, and scarce relevant data in private knowledge domains, problems such as illusions, information loss, and incomplete answers arise. Currently, many studies have developed a popular and reliable solution—Retrieval Augmentation Generation (RAG) technology—by injecting relevant knowledge from private domains into external databases as augmenting data to large language models.

[0051] To address the aforementioned issues in traditional large language model question answering techniques, retrieval-enhanced generation techniques can be primarily divided into three stages. The first stage, the indexing stage, aims to store the original documents in a vector database, providing data for subsequent retrieval and enhancement generation stages. This stage first segments the original text into smaller, more easily understood blocks to meet the context length limitations of the language model. Then, it encodes these blocks using a vectorization model and stores them in an external vector database. The second stage, the retrieval stage, focuses on finding document blocks from the database that can answer the user's question. After receiving a user query, the retrieval-enhanced generation technique encodes the user's question using the same vectorization model as in the first stage. It then calculates the similarity score between the question vector and the documents in the database, returning the k documents that best match the user's question. The third stage, the enhancement generation stage, combines the user's question and the retrieved documents into a coherent prompt, which is then input into the large language model. With the enhancement of supplementary documents, the large model returns more accurate and professional results.

[0052] For question answering technology based on large language models targeting private domains, since generalized large language models often lack relevant knowledge, it is necessary to supplement and enhance the knowledge of the large model through external databases. In addition, since the cost of supplementing external databases is relatively lower than the cost of fine-tuning the large model, people can supplement the large language model with the latest knowledge in the private knowledge domain in real time, making it timely.

[0053] However, for the private knowledge domain, past research has shown that retrieval augmentation techniques encounter several challenges. First, because the quality of user questions cannot be guaranteed, retrieval augmentation techniques struggle to retrieve all relevant blocks during the retrieval phase, risking the loss of key information and resulting in inconsistent search results. Second, if low-quality search results are input into a large language model during the augmentation phase, it can mislead the model, causing it to "hang in the air" and reducing the quality and reliability of the responses. Furthermore, incoherent search results may lead to logically flawed responses from the large model; while repetitive and redundant search results may result in duplicate responses from the large model.

[0054] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0055] The personalized question-answering method for large language models oriented to private domains provided in this application can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be set up independently, integrated into server 104, or placed in the cloud or on another server. Terminal 102 can send the original question from the user to server 104. After receiving the original question, server 104 inputs it into a large language model to obtain a rewritten question. The rewritten question is a question after the large language model uses its own parameter knowledge to complete and professionally adjust the original question. The rewritten question is matched with data in an external database to obtain a relevant context set. The relevant context set is clustered to obtain several category clusters. Each category cluster is input into the large language model to obtain summary data for each category cluster. The summary data and the original question are input into the large language model to obtain the answer. Server 104 can then provide feedback to terminal 102 regarding the obtained original question from the user. In addition, in some embodiments, the personalized question-answering method for large language models in the private domain can also be implemented by the server 104 or the terminal 102 separately. For example, the terminal 102 can directly process the original question of the user to be processed, or the server 104 can obtain the original question of the user to be processed from the data storage system and process it.

[0056] The terminal 102 can be, but is not limited to, various desktop computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, and smart in-vehicle devices. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted devices. The server 104 can be implemented using a standalone server or a server cluster composed of multiple servers, or it can be a cloud server.

[0057] Example 1:

[0058] In one exemplary embodiment, such as Figure 2 As shown, a personalized question-answering method for large language models in private domains is provided. This method is executed by a computer device, specifically a terminal or server, or both. In this embodiment, the method is applied to... Figure 1 Taking server 104 as an example, the explanation includes the following steps S1 to S5.

[0059] in:

[0060] S1. Input the user's original question into the large language model to obtain the rewritten question; the rewritten question is the question after the large language model uses its own parameter knowledge to complete and professionally adjust the original question.

[0061] Assuming the user inputs the original question q, this embodiment inputs it into a large language model. Utilizing the model's own parameter knowledge, it rewrites the question into n more comprehensive and specialized related questions, resulting in a rewritten question set qSet = {q1, q2, ..., q...}. n The specific formula for rewriting the problem is: qSet = LLM(q, n). Here, n is a hyperparameter, and experimental results show that the best results are achieved when n = 4.

[0062] S2. Match the rewriting problem with data in an external database to obtain a relevant context set.

[0063] This embodiment matches the rewritten questions with data in an external database to obtain a relevant context set. The external database is constructed in the initial stage of the system to provide a data source for subsequent retrieval and generation stages. Specifically, this embodiment first segments the original text data into smaller, more easily understood blocks to meet the context length limitations of the language model; then, it encodes the data using a vectorization model and stores it in an external vector database. After question rewriting, for each rewritten question q in qSet... i In this embodiment, it is vectorized to obtain Where i∈n; then, this vector is used to retrieve data from the external database db, and the similarity between this vector and the vectors of the context documents in the external database is calculated. The m most similar context documents are selected to obtain the set dSet. i ={d i1 ,d i2 ,...,d im The formula for calculating the context document set is: Thus, this step yields a set dSet = {d ...} containing n × m documents. 11 ,d 12 ,...,d nm Next, in this embodiment, all documents in the document set dSet are sorted by similarity, and the k documents most relevant to the user's original question are obtained. The calculation formula is: contexts = semantic_search(q, k, dSet). In this embodiment, k is 5. Finally, the matched documents can be used as augmented knowledge and input together with the user's initial question into the large language model to obtain a more accurate and comprehensive answer.

[0064] See also Figure 3 , Figure 3 This is a schematic diagram of the private domain knowledge expansion scheme based on question rewriting in this embodiment. This scheme aims to supplement the information in the user's original question, add relevant knowledge from the private domain, and increase the diversity of user questions, thereby improving the accuracy and comprehensiveness of the search results.

[0065] In this private domain knowledge augmentation scheme based on question rewriting, this embodiment retrieves all questions from the question set qSet to obtain several matching documents dSet, which are then input into a large language model. Since these documents are generated from the same initial question through expansion and similarity searching, they often have some redundancy. Directly inputting them into the large language model during the generation stage would lead to logical duplication in the final generated results, reducing the quality of the answers. Furthermore, due to the performance limitations of the large language model, it may be unable to receive all relevant documents during the generation stage, resulting in the loss of augmented knowledge and impacting the overall system performance, thus degrading the user experience. Performing similarity matching again on these documents could also lead to information loss, having a counterproductive effect on the knowledge augmentation technology.

[0066] To reduce redundancy in augmented information and avoid semantic repetition in the context input generation stage of a large language model, this embodiment introduces a context simplification scheme based on clustering and summarization. This scheme clusters relevant contexts into several categories, and then summarizes each category. (See also...) Figure 4 , Figure 4This is a schematic diagram of the context simplification scheme based on clustering and summarization in this embodiment.

[0067] S3. Cluster the relevant context set to obtain several category clusters.

[0068] First, for each question-document pair {q} obtained from the private domain knowledge augmentation scheme based on question rewriting. i ,d ij In this embodiment, it is vectorized to obtain e. ij The calculation formula is:

[0069] e ij =Embedding(q i +d ij ),q i ∈qSet,d ij ∈dSet i ;

[0070] Secondly, this embodiment will vectorize all e in the question-document pair set. ij KMeans clustering was performed, resulting in t sets. Here, t is a hyperparameter. Experimental results show that the performance is better when t=4. Furthermore, to optimize the clustering objective function... To achieve better results, this embodiment performs 10 initializations during each clustering process, selecting the optimal result. After this step, each question-document pair corresponds to a unique category cluster c. i , where i∈[1,t].

[0071] S4. Input each category cluster into the large language model to obtain summary data for each category cluster.

[0072] S5. Input the summarized data and the original question into the large language model to obtain the answer to the question.

[0073] To ensure that the context in clustering is not truncated by the large language model during the generation stage, thus preventing information loss, this embodiment summarizes the content of each category cluster. For each category cluster c, let it include u question-document pairs, represented as: {q c1 ,d c1 ;q c2 ,d c2 ;...;q cu ,d cu This embodiment uses a large language model to summarize and refine these u question-document pairs, obtaining a summary of knowledge for each category cluster. c The calculation formula is: summary c =LLM(q c1 ,dc1 ,q c2 ,d c2 ,...,q cu ,d cu ).

[0074] This embodiment can thus use the summarized knowledge from different types of clusters to enhance the large language model and obtain the final answer.

[0075] The private domain knowledge expansion scheme based on question rewriting makes the final generated answer more logical while maintaining the diversity and comprehensiveness of the private domain knowledge expansion scheme based on question rewriting.

[0076] Through the two schemes described above, the user's original question undergoes multiple processing steps. However, the enhanced knowledge obtained by the aforementioned private domain-oriented knowledge augmentation-simplification scheme carries the risk that the clustered and summarized answers may deviate from the focus of the user's question. To make the large model in the generation stage more relevant to the user's question, this embodiment introduces a user question supplementation technique, which increases the weight of the user's initial question in two aspects. Please refer to [link / reference]. Figure 5 , Figure 5 This is a schematic diagram illustrating a supplementary solution for user issues in this embodiment.

[0077] Firstly, in the knowledge expansion stage, this embodiment inputs both the original question and the rewritten question into an external database for similarity matching, resulting in a more relevant set of documents during the retrieval stage. The implementation formula for this scheme is as follows:

[0078] qSet = LLM(q, n)

[0079] qSet = {q} ∪ qSet

[0080]

[0081] Secondly, in the answer generation stage, this invention repeats the user's initial question several times. This project assembles the prompts from the input large model according to the following formula to obtain the final answer:

[0082]

[0083] The number of times the user's initial question is rewritten, r, is a hyperparameter. According to experimental results, when r = 5, the generated answer is neither off-topic nor lacks comprehensiveness.

[0084] This embodiment provides a personalized question-answering method for private domain large language models. It involves inputting the user's original question into a large language model to obtain a rewritten question. The rewritten question is a question that the large language model completes and professionally adjusts using its own parameter knowledge, thus solving the problem of inconsistent user question quality and effectively improving the quality of user questions. The method further involves clustering the relevant context set to obtain several category clusters. Each category cluster is then input into the large language model to obtain summary data for each cluster, thus avoiding the input of low-quality search results. This application improves the quality of user questions and avoids inputting low-quality search results, thereby enhancing the quality and reliability of the answers.

[0085] Example 2:

[0086] Based on the same inventive concept, this application also provides a device for implementing the aforementioned private domain-oriented large language model personalized question answering method. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more embodiments provided below can be found above, and will not be repeated here.

[0087] In one exemplary embodiment, such as Figure 6 As shown, a personalized question-answering device for large language models in private domains is provided, including:

[0088] The user input question module is used to receive the original question input by the user.

[0089] The private domain knowledge expansion module based on question rewriting is used to input the user's original question into a large language model and obtain a rewritten question; the rewritten question is the question after the large language model uses its own parameter knowledge to complete and professionally adjust the original question.

[0090] The private domain knowledge base retrieval module is used to match the rewriting problem with data in an external database to obtain a relevant context set.

[0091] The context clustering module is used to cluster the relevant context set to obtain several category clusters.

[0092] The clustering knowledge simplification module is used to input each of the aforementioned category clusters into the large language model to obtain summary data for each of the aforementioned category clusters.

[0093] The answer generation module is used to input the summarized data and the original question into the large language model to obtain the answer to the question.

[0094] When using this device, users can directly operate on the personalized question-and-answer system based on the large language model for private domains, tailoring it to their specific needs. This not only enhances the system's usability but also improves its flexibility and efficiency in different domains.

[0095] The specific execution flow of a private domain-oriented large language model personalized question-answering device that uses all modules is as follows: Starting with the user inputting a question, the user question is passed to the private domain knowledge expansion module based on question rewriting; after organizing the expanded questions, they are input to the private domain knowledge base retrieval module for relevant question matching; then, the matching results are passed to the context clustering module and the clustering knowledge simplification module in sequence for knowledge simplification; finally, the user question supplementation module passes the simplified enhanced knowledge along with the user's initial question to the answer generation module to obtain a professional medical domain answer.

[0096] Example 3:

[0097] In one exemplary embodiment, a personalized question-answering method for a large language model in a private domain is provided. Please refer to [link to relevant documentation]. Figure 7 To address the issues of traditional retrieval enhancement generation techniques failing to fully understand user query intent, high redundancy of enhanced knowledge injected into large models, and deviation from the focus of generated answers, this embodiment improves the performance of large language models in the private domain by first expanding and then simplifying the knowledge. First, in the retrieval phase, this embodiment expands and queries the user query using a private domain knowledge expansion scheme based on question rewriting, obtaining relevant context from multiple perspectives. Then, a context simplification scheme based on clustering and summarization simplifies relevant knowledge according to categories, eliminating redundant data and obtaining higher-quality relevant knowledge. Finally, this embodiment increases the weight of the initial user query through a user query supplementation scheme, improving the fidelity of the generated answer to the initial query and preventing the answer from deviating from the focus of the user query.

[0098] To achieve the above objectives, this embodiment optimizes the retrieval enhancement generation process and provides a knowledge expansion and simplification scheme for the private domain. To address the issue of low retrieval result accuracy, this embodiment rewrites the initial user question, expands relevant knowledge, and supplements incomplete user queries. To address the issue of redundant database information, this embodiment clusters and summarizes the retrieved context to prevent redundant enhancement knowledge. To address the issue of the model's answer deviating from its intended focus, this embodiment supplements the initial user question by increasing its weight, ensuring that the generated answer does not deviate from the user's query intent. The knowledge expansion and simplification scheme for the private domain consists of three sub-schemes, among which the private domain knowledge expansion based on question rewriting is the most critical. This embodiment, through the synergistic assistance of the context simplification scheme based on clustering and summarization and the user question supplementation scheme, ultimately improves the quality of the final answer generated by the large language model question answering system. The specific implementation scheme is as follows:

[0099] The private domain knowledge expansion based on question rewriting is the first implementation scheme. This scheme mainly addresses the problem of low retrieval accuracy. Since the input questions are generated by human users, the way the questions are expressed and their level of expertise are closely related to the user of the retrieval enhancement generation technology. Traditional retrieval enhancement generation technology directly inputs the user's input question into a vector database for similarity retrieval, and the quality of the retrieval results directly depends on the quality of the original question input by the user. In some fields with limited relevant knowledge, few application scenarios, and high professional knowledge thresholds, users without relevant background may not be able to accurately and comprehensively express their questions, causing traditional retrieval enhancement generation technology to fail to fully utilize relevant knowledge to make subsequent answers. To address the above problems, this invention improves the accuracy of retrieval results through a private domain knowledge expansion scheme. The implementation steps of this scheme are as follows:

[0100] The first step in this embodiment is to input the user's question into the general knowledge large language model, guide the large language model to generate several questions related to the input question, and use the parameter knowledge inside the model to complete the user's input question.

[0101] In the second step, this embodiment uses the relevant questions generated in the first step as input for the retrieval stage in the retrieval enhancement generation, and matches documents in the vector database to obtain a richer set of relevant contexts.

[0102] Third, in this embodiment, the relevant context obtained in the second step is reordered to obtain several context documents most similar to the user's initial question. These are then used as augmented knowledge and input into the large language model along with the user's original question to generate the final answer.

[0103] This solution not only understands the user's questioning intent from multiple perspectives and improves the diversity of the original questions, but also transforms the user's colloquial questions into professional expressions, thereby obtaining a more accurate and richer relevant context.

[0104] The context simplification based on clustering and summarization is the second execution scheme. This scheme mainly addresses the problem of high redundancy in retrieval results. Due to the abundance of data in the database, there are often multiple related documents for the same information, and these documents may contain semantic redundancy. Directly inputting them into a subsequent large language model would lead to logical repetition in the generated answers, reducing the quality of the generated results. The main execution steps of this scheme are as follows:

[0105] The first step in this embodiment is to perform cluster analysis on the relevant contexts obtained by the private domain knowledge augmentation scheme based on problem rewriting, and divide them into several categories.

[0106] In the second step, for each category obtained in the first step, this embodiment inputs the relevant context into the large language model to obtain the summarized knowledge of that category, which serves as the enhanced knowledge of the large language model in the subsequent generation stage. This knowledge is then input into the large language model along with the user's initial question to obtain the final answer.

[0107] The context simplification scheme based on clustering and summarization enables the relevant context in the retrieval stage to avoid information redundancy while ensuring comprehensive knowledge coverage, making the retrieval results more concise and thus improving the logical consistency of the generated answers.

[0108] The user question supplement is the third execution scheme. This scheme mainly addresses the problem of low fidelity in the generated answer. To supplement the private domain knowledge of the large language model, traditional retrieval-enhanced generation techniques typically retrieve as many relevant documents as possible. However, when too many external documents are retrieved, irrelevant content may be mixed in, causing the generated answer to deviate from the user's intent and significantly reducing its accuracy. Because this embodiment first expands the user question, then simplifies the relevant context, and finally inputs it into the large language model to generate the answer, each stage involves related knowledge processing, which may cause the relevant knowledge input into the model to deviate from the focus of the user's question.

[0109] To ensure the final answer aligns with the user's key questions, this implementation supplements the user's questions in the following two stages, thereby guaranteeing the relevance of the generated answer:

[0110] Firstly, in the knowledge expansion stage, this embodiment inputs the user's initial question and related questions obtained from the private domain knowledge expansion scheme based on question rewriting into the vector database for retrieval, so that the relevant context obtained in the retrieval stage is closer to the user's initial question.

[0111] Secondly, in the answer generation stage, this embodiment repeats the user's initial question several times when constructing the prompts for the generation stage, and then connects and assembles it with the enhanced knowledge, directly improving the relevance of the generated answer to the user's initial question.

[0112] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 8 As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media to run. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements a personalized question-answering method based on a large language model oriented towards a private domain.

[0113] Those skilled in the art will understand that Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0114] In one exemplary embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0115] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0116] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0117] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0118] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0119] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0120] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0121] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A large language model personalization question and answer method for a private domain, characterized in that, The private domain-oriented large language model personalized question and answer method comprises: inputting an original question of a user into a large language model to obtain a rewritten question; the rewritten question is a question after the original question is completed and professionally adjusted by the large language model using its own parameter knowledge; matching the rewritten question with data in an external database to obtain a relevant context set; specifically, for each rewritten question, after vectorization, the question is retrieved from the external database, the similarity between the vector of the question and the vectors of context documents in the external database is calculated, and the m context documents with the highest similarity are taken as the relevant context set; clustering the relevant context set to obtain a plurality of category clusters; inputting each category cluster into the large language model to obtain summary data of each category cluster; for each category cluster, if the category cluster includes u question-document pairs, the u question-document pairs are summarized and refined by the large language model to obtain summary knowledge of each category cluster; inputting the summary data and the original question into the large language model to obtain an answer to the question; in the answer generation stage, the original question of the user is repeated five times to obtain a final answer; clustering the relevant context set to obtain a plurality of category clusters, specifically comprising: vectorizing the relevant context set to obtain a vectorized set; specifically, each question-document pair is vectorized to obtain the vectorized set; performing KMeans clustering on the vectorized set to obtain a plurality of category clusters.

2. The private-domain-oriented large language model personalized question and answer method of claim 1, wherein, The relevant context set includes a plurality of documents; clustering the relevant context set to obtain a plurality of category clusters specifically comprises: calculating the similarity between each document in the relevant context set and the original question; sorting all the documents in the relevant context set according to the similarity to obtain a sorted set; obtaining the first k documents in the sorted set to obtain a similar set; clustering the similar set to obtain a plurality of category clusters.

3. The private-domain-oriented large language model personalized question answering method according to claim 1, characterized in that, The relevant context set is obtained by matching the rewritten question with data in an external database, specifically comprising: inputting the rewritten question and the original question into the external database for matching retrieval to obtain the relevant context set.

4. The private-domain-oriented large language model personalized question answering method according to claim 1, characterized in that, The formula for obtaining the rewritten question is: qSet=LLM(q,n); wherein n is a hyperparameter, n=4, q is the original question, qSet is the rewritten question, and LLM is the large language model.

5. The private-domain-oriented large language model personalized question answering method according to claim 1, characterized in that, The construction method of the external database specifically comprises: segmenting text materials to obtain text material blocks; encoding the text material blocks by using a vectorization model to obtain the external database.

6. A private domain-oriented large language model personalized question and answer device, characterized in that, The private domain-oriented large language model personalized question and answer device comprises: a user input question module configured to receive an original question input by a user; a private domain knowledge expansion module based on question rewriting, configured to input the original question of the user into a large language model to obtain a rewritten question; the rewritten question is a question after the original question is completed and professionally adjusted by the large language model using its own parameter knowledge; a category cluster input module, configured to input each category cluster into the large language model to obtain summary data of each category cluster; for each category cluster, if the category cluster includes u question-document pairs, the u question-document pairs are summarized and refined by the large language model to obtain summary knowledge of each category cluster; The private domain knowledge base retrieval module is configured to match the rewritten question with data in an external database to obtain a relevant context set. Specifically, for each rewritten question, the question is vectorized and then retrieved from the external database. The similarity between the vector of the question and the vectors of context documents in the external database is calculated. The m context documents with the highest similarity are selected to obtain the relevant context set. The context clustering module is configured to cluster the relevant context set to obtain a plurality of category clusters. The context clustering module is configured to cluster the relevant context set to obtain a plurality of category clusters, and specifically includes the following steps: The context clustering module is configured to cluster the relevant context set to obtain a plurality of category clusters, and specifically includes the following steps: The context clustering module is configured to cluster the relevant context set to obtain a plurality of category clusters, and specifically includes the following steps: The clustering knowledge refinement module is configured to input each category cluster into a large language model to obtain summary data of each category cluster. For each category cluster, if the category cluster includes u question-document pairs, the u question-document pairs are summarized and refined using the large language model to obtain summary knowledge of each category cluster. The answer generation module is configured to input the summary data and the original question into a large language model to obtain a question answer. In the answer generation stage, the original question of the user is repeated 5 times to obtain a final answer.

7. A computer device comprising: The memory, the processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the private domain-oriented large language model personalized question answering method of any one of claims 1-5.

8. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the private domain-oriented large language model personalized question answering method of any one of claims 1-5.

9. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the private domain-oriented large language model personalized question answering method of any one of claims 1-5.

Images