Private domain-oriented large language model personalized question and answer method and related products

Abstract

The invention discloses a private domain-oriented large language model personalized question and answer method and related products, and relates to the technical field of large language model question and answer, the method comprises the following steps: matching a rewritten question with data in an external database to obtain a related context set; clustering the related context set to obtain a plurality of category clusters; respectively inputting each category cluster into a large language model to obtain summary data of each category cluster; and inputting the summarized data and the original questions into a large language model to obtain answers to the questions. The rewriting problem in the application is a problem obtained after the large language model performs complementation and professional adjustment on the original problem by using own parameter knowledge, so that the quality of the user problem is effectively improved; clustering the related context sets to obtain a plurality of category clusters; and respectively inputting each category cluster into the large language model to obtain summary data of each category cluster, so that a retrieval result with low quality can be prevented from being input.

Description

Technical Field

[0001] The present application relates to the technical field of large language model question answering, and in particular to a large language model personalized question answering method for private domains and related products. Background Art

[0002] In past research and applications, traditional large language models (LLMs) question-answering technology has demonstrated excellent general knowledge capabilities. This technology uses the parameter knowledge inside the model to directly answer user questions. However, in applications in private knowledge fields with few usage scenarios, high professionalism, and little relevant information, there are problems such as hallucinations, information loss, and incomplete answers. Currently, many studies have obtained a more popular and reliable solution by injecting relevant knowledge in the private field into external databases as enhanced data to supplement large language models - retrieval augmentation generation (RAG) technology.

[0003] For the private knowledge domain, past research results show that retrieval-augmented generation technology encounters the following challenges. First, since the quality of user questions cannot be guaranteed, retrieval-augmented generation technology is difficult to fully retrieve all relevant blocks during the retrieval stage, and there is a risk of losing key information, resulting in uneven quality of retrieval results. Second, during the enhanced generation stage, if retrieval-augmented generation technology inputs low-quality retrieval results into a large language model, it will mislead the large model, causing the model to hallucinate and reduce the quality and reliability of the answer. Summary of the invention

[0004] The purpose of this application is to provide a large language model personalized question answering method and related products for private domains, which can improve the quality of user questions, avoid inputting low-quality retrieval results, and improve the quality and reliability of answers.

[0005] To achieve the above objectives, this application provides the following solutions:

[0006] In a first aspect, the present application provides a large language model personalized question answering method for a private domain, and the large language model personalized question answering method for a private domain includes:

[0007] Input the user's original question into the large language model to obtain a rewritten question; the rewritten question is the question after the large language model uses its own parameter knowledge to complete and professionally adjust the original question;

[0008] Matching the rewriting problem with data in an external database to obtain a relevant context set;

[0009] Clustering the relevant context set to obtain a plurality of category clusters;

[0010] Input each of the category clusters into a large language model to obtain summary data of each of the category clusters;

[0011] The summary data and the original question are input into a large language model to obtain an answer to the question.

[0012] Optionally, the relevant context set includes a plurality of documents; clustering the relevant context set to obtain a plurality of category clusters specifically includes:

[0013] Calculate the similarity between each document in the relevant context set and the original question;

[0014] Sort all documents in the relevant context set by similarity to obtain a sorted set;

[0015] Obtain the first k documents in the sorted set to obtain a similar set;

[0016] The similarity sets are clustered to obtain several category clusters.

[0017] Optionally, clustering is performed on the relevant context set to obtain several category clusters, specifically including:

[0018] Vectorizing the relevant context set to obtain a vectorized set;

[0019] KMeans clustering is performed on the vectorized set to obtain several category clusters.

[0020] Optionally, the rewriting problem is matched with data in an external database to obtain a relevant context set, specifically including:

[0021] The rewritten question and the original question are input into an external database for matching and retrieval to obtain a relevant context set.

[0022] Optionally, the formula for obtaining the rewriting problem is:

[0023] qSet=LLM(q,n);

[0024] Among them, n is a hyperparameter, n=4, q is the original problem, qSet is the rewriting problem, and LLM is the large language model.

[0025] Optionally, the method for constructing the external database specifically includes:

[0026] Segment the text data to obtain text data blocks;

[0027] The text data block is encoded using a vectorization model to obtain an external database.

[0028] In a second aspect, the present application provides a large language model personalized question-answering device for a private domain, the large language model personalized question-answering device for a private domain comprising:

[0029] A user input question module, used to receive the original question input by the user;

[0030] The private domain knowledge expansion module based on question rewriting is used to input the user's original question into the large language model to obtain a rewritten question; the rewritten question is the question after the large language model uses its own parameter knowledge to complete and professionally adjust the original question;

[0031] A private domain knowledge base retrieval module, used to match the rewriting problem with data in an external database to obtain a relevant context set;

[0032] A context clustering module, used for clustering the relevant context set to obtain a plurality of category clusters;

[0033] A clustering knowledge simplification module, used for inputting each of the category clusters into a large language model to obtain summary data of each of the category clusters;

[0034] The answer generation module is used to input the summary data and the original question into the large language model to obtain the answer to the question.

[0035] In a third aspect, the present application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement any of the above-mentioned large language model personalized question-answering methods for private domains.

[0036] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-mentioned large language model personalized question-answering methods for private domains.

[0037] In a fifth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements any of the above-mentioned large language model personalized question-answering methods for private domains.

[0038] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0039] The present application provides a large language model personalized question-answering method and related products for private domains, which obtains a rewritten question by inputting the user's original question into the large language model; the rewritten question is the question after the large language model uses its own parameter knowledge to complete and professionally adjust the original question, which solves the problem that the quality of user questions cannot be guaranteed and effectively improves the quality of user questions; clustering the relevant context set through steps to obtain several category clusters; each of the category clusters is input into the large language model respectively to obtain summary data of each category cluster, which solves the problem of inputting low-quality retrieval results into the large language model and can avoid inputting low-quality retrieval results; the present application can improve the quality of user questions, and at the same time can avoid inputting low-quality retrieval results, and improve the quality and reliability of answers. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0041] Figure 1 An application environment diagram of a large language model personalized question-answering method for a private domain provided in an embodiment of the present application;

[0042] Figure 2 A flowchart of a large language model personalized question-answering method for a private domain provided in Example 1 of the present application;

[0043] Figure 3 A schematic diagram of a private domain knowledge expansion solution based on question rewriting provided in Example 1 of the present application;

[0044] Figure 4 A schematic diagram of a context simplification solution based on clustering and summarization provided in Example 1 of the present application;

[0045] Figure 5 A schematic diagram of a supplementary solution to a user problem provided in Example 1 of the present application;

[0046] Figure 6 A schematic diagram of a large language model personalized question-answering device for private domains provided in Example 2 of the present application;

[0047] Figure 7 A schematic diagram of a large language model personalized question-answering method for a private domain provided in Example 3 of the present application;

[0048] Figure 8A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0049] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0050] In past research and applications, traditional large language models (LLMs) question-answering technology has demonstrated excellent general knowledge capabilities. This technology uses the parameter knowledge inside the model to directly answer user questions. However, in applications in private knowledge fields with few usage scenarios, high professionalism, and little relevant information, there are problems such as hallucinations, information loss, and incomplete answers. Currently, many studies have obtained a more popular and reliable solution by injecting relevant knowledge in the private field into external databases as enhanced data to supplement large language models - retrieval augmentation generation (RAG) technology.

[0051] In response to the above problems in traditional large language model question answering technology, the retrieval enhancement generation technology can be mainly divided into three stages. The first stage is the indexing stage, which aims to store the original documents in the vector database to provide a data source for the subsequent retrieval and enhancement generation stages. The retrieval enhancement generation technology first divides the original text into smaller, easy-to-understand blocks to meet the context length limit of the language model. Then, it is encoded using a vectorization model and stored in an external vector database. The second stage is the retrieval stage, which is dedicated to finding document blocks that can answer user questions from the database. After the retrieval enhancement generation technology receives the user query, it uses the same vectorization model as the first stage to encode the user question, then calculates the similarity score between the question vector and the documents in the database, and returns the k documents that best match the user question. The third stage is the enhancement generation stage. The retrieval enhancement generation technology synthesizes the user's question and the retrieved documents into a coherent prompt and inputs it into the large language model, so that the large model can return more accurate and professional results with the enhancement of supplementary documents.

[0052] For private domain-oriented large language model question answering technology, since general large language models often lack relevant knowledge, it is necessary to supplement and enhance the knowledge of large models through external databases. In addition, since the cost of supplementing external databases is relatively lower than the cost of fine-tuning large models, people can supplement the large language model with the latest knowledge in the private knowledge field in real time, making it timely.

[0053] However, for the private knowledge domain, past research results have shown that retrieval enhancement generation technology encounters the following challenges. First, since the quality of user questions cannot be guaranteed, retrieval enhancement generation technology is difficult to fully retrieve all relevant blocks in the retrieval stage, and there is a risk of losing key information, resulting in uneven quality of retrieval results. Secondly, in the enhancement generation stage, if retrieval enhancement generation technology inputs low-quality retrieval results into a large language model, it will mislead the large model, causing hallucinations in the model and reducing the quality and reliability of the answer. In addition, incoherent retrieval results may cause the results returned by the large model to have poor logic; and repeated and redundant retrieval results may cause repeated responses in the results returned by the large model.

[0054] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0055] The large language model personalized question answering method for private domains provided in the embodiments of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be set up separately, integrated on the server 104, or placed on the cloud or other servers. The terminal 102 can send the original question of the user to be processed to the server 104. After the server 104 receives the original question of the user to be processed, the user's original question is input into the large language model to obtain a rewritten question; the rewritten question is the question after the large language model uses its own parameter knowledge to complete and professionally adjust the original question; the rewritten question is matched with the data in the external database to obtain a related context set; the related context set is clustered to obtain several category clusters; each of the category clusters is input into the large language model respectively to obtain summary data of each category cluster; the summary data and the original question are input into the large language model to obtain the answer to the question. The server 104 can feedback the obtained original question of the user to the terminal 102. In addition, in some embodiments, the large language model personalized question-answering method for private domains can also be implemented independently by the server 104 or the terminal 102. For example, the terminal 102 can directly process the original question of the user to be processed, or the server 104 can obtain the original question of the user to be processed from the data storage system and process it.

[0056] The terminal 102 may be, but is not limited to, various desktop computers, laptop computers, smart phones, tablet computers, IoT devices, and portable wearable devices. The IoT devices may be smart speakers, smart TVs, smart air conditioners, smart vehicle-mounted devices, etc. The portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. The server 104 may be implemented as an independent server or a server cluster consisting of multiple servers, or may be a cloud server.

[0057] Embodiment 1:

[0058] In an exemplary embodiment, Figure 2 As shown, a large language model personalized question answering method for private domain is provided. The method is executed by a computer device, and can be executed by a computer device such as a terminal or a server alone, or can be executed by a terminal and a server together. In the embodiment of the present application, the method is applied to Figure 1 The server 104 in the example is used for explanation, and the steps include the following steps S1 to S5.

[0059] in:

[0060] S1. Input the user's original question into the large language model to obtain a rewritten question; the rewritten question is the question after the large language model uses its own parameter knowledge to complete and professionally adjust the original question.

[0061] Assuming that the original question input by the user is q, this embodiment inputs it into the large language model and rewrites it into n more comprehensive and professional related questions using the model's own parameter knowledge, obtaining the rewritten question set qSet = {q1, q2, ..., q n The specific rewriting problem formula is: qSet = LLM(q,n). Among them, n is a hyperparameter. According to experimental results, when n = 4, the best effect can be achieved.

[0062] S2. Match the rewriting problem with data in an external database to obtain a relevant context set.

[0063] This embodiment matches the rewritten question with the data in the external database to obtain a set of relevant contexts. The external database is constructed in the initial stage of the system to provide a data source for the subsequent retrieval and generation stages. Specifically, this embodiment first divides the original text data into smaller, easier-to-understand blocks to meet the context length limit of the language model; then, it is encoded using a vectorized model and stored in an external vector database. After the question is rewritten, for each rewritten question q in qSet i , this embodiment vectorizes it and obtains Where i∈n; then, use this vector to retrieve from the external database db, calculate the similarity between it and the vector of the context document in the external database, and take the m relevant context documents with the highest similarity to obtain the set dSet i ={d i1 ,d i2 ,...,d im}, the calculation formula for the context document set is: Thus, this step obtains a set dSet containing n×m documents = {d 11 ,d 12 ,...,d nm}. Next, this embodiment sorts all documents in the document set dSet by similarity, and obtains the k documents most relevant to the user's original question. The calculation formula is: contexts = semantic_search (q, k, dSet). In this embodiment, k is 5. Finally, the matched documents can be used as enhanced knowledge and input into the large language model together with the user's initial question to obtain a more accurate and comprehensive answer.

[0064] See also Figure 3 , Figure 3 Schematic diagram of the private domain knowledge expansion solution based on question rewriting in this embodiment. This solution aims to complete the information of the user's original question, supplement the relevant knowledge in the private domain, and improve the diversity of user questions, thereby improving the accuracy and comprehensiveness of the search results.

[0065] In the private domain knowledge expansion scheme based on question rewriting, this embodiment retrieves all questions in the question set qSet, obtains several matching documents dSet, and inputs them into the large language model. Since these documents are generated by expanding and similarity searching the same initial question, there is often a certain degree of duplication and redundancy between them. If they are directly input into the large language model in the generation stage, the final generated result will be logically repeated, reducing the quality of the answer. At the same time, due to the performance limitations of the large language model, the large model in the generation stage may not be able to receive all relevant documents, resulting in the loss of enhanced knowledge, and will also affect the time performance of the entire system and reduce the user experience. If they are matched for similarity again, it may also cause information loss, which has a counter-effect on the knowledge expansion technology.

[0066] In order to reduce the redundancy of enhanced information and avoid the large language model in the context input generation stage with semantic repetition, this embodiment introduces a context simplification scheme based on clustering and summarization. This scheme clusters the relevant contexts to obtain several categories, and then summarizes each category separately. Figure 4 , Figure 4Schematic diagram of the context simplification solution based on clustering and summarization in this embodiment.

[0067] S3. Clustering the relevant context set to obtain several category clusters.

[0068] First, for each question-document pair {q i ,d ij}, this embodiment vectorizes it to obtain e ij , the calculation formula is:

[0069] e ij =Embedding(q i +d ij ),q i ∈qSet,d ij ∈dSet i ;

[0070] Second, this embodiment converts all e in the vectorized set of question-document pairs into ij Perform KMeans clustering to obtain t sets. Among them, t is a hyperparameter. Experimental results show that when t = 4, the effect is better. In addition, in order to make the objective function of clustering To achieve better results, this embodiment performs 10 initializations at each clustering to obtain the best result. After this step, each question-document pair corresponds to a unique category cluster c i , where i∈[1,t].

[0071] S4. Input each of the category clusters into the large language model to obtain summary data of each of the category clusters.

[0072] S5. Input the summary data and the original question into a large language model to obtain an answer to the question.

[0073] In order to ensure that the context in the cluster is not truncated by the large language model in the generation phase, resulting in information loss, this embodiment summarizes the content in each category cluster. For each category cluster c, let it include u question-document pairs, represented as: {q c1 ,d c1 ;q c2 ,d c2 ;...;q cu ,d cu This embodiment uses a large language model to summarize and refine these u question-document pairs to obtain the summary knowledge summary of each category cluster. c , the calculation formula is: summary c =LLM(q c1 ,dc1 ,q c2 ,d c2 ,...,q cu ,d cu ).

[0074] This embodiment can thus use the summarized knowledge of different category clusters to enhance the large language model and obtain the final answer.

[0075] The private domain knowledge expansion scheme based on question rewriting makes the final generated answer more logical while maintaining the diversity and comprehensiveness of the private domain knowledge expansion scheme based on question rewriting.

[0076] Through the above two solutions, the user's original question has been processed in multiple steps. However, the enhanced knowledge obtained by the private domain-oriented knowledge expansion and simplification solution has the risk that the clustered summary answer deviates from the focus of the user's question. In order to make the large model in the generation stage more targeted at answering the user's question focus, this embodiment introduces user question supplementation technology to increase the weight of the user's initial question in two aspects. Please refer to Figure 5 , Figure 5 This is a schematic diagram of a supplementary solution to a user problem in this embodiment.

[0077] First, in the knowledge expansion stage, this embodiment inputs the original question and the rewritten question into the external database for similarity matching, and obtains a more relevant set of related documents in the retrieval stage. The implementation formula of this solution is as follows:

[0078] qSet=LLM(q,n)

[0079] qSet={q}∪qSet

[0080]

[0081] Second, in the answer generation stage, the present invention repeats the user's initial question several times. This subject assembles the prompts input into the large model according to the following formula to obtain the final answer answer:

[0082]

[0083] The number of times r that the user's initial question is rewritten is a hyperparameter. Experimental results show that when r = 5, the generated answer does not deviate from the topic while ensuring comprehensiveness.

[0084] This embodiment provides a large language model personalized question-answering method for private domains, which obtains a rewritten question by inputting the user's original question into the large language model; the rewritten question is the question after the large language model uses its own parameter knowledge to complete and professionally adjust the original question, which solves the problem that the quality of user questions cannot be guaranteed, and effectively improves the quality of user questions; clustering the relevant context set through steps to obtain several category clusters; each of the category clusters is input into the large language model respectively to obtain summary data of each category cluster, which solves the problem of inputting low-quality retrieval results into the large language model, and can avoid inputting low-quality retrieval results; this application can improve the quality of user questions, and at the same time can avoid inputting low-quality retrieval results, and improve the quality and reliability of answers.

[0085] Embodiment 2:

[0086] Based on the same inventive concept, the embodiment of the present application also provides a large language model personalized question-answering device for private domains for implementing the large language model personalized question-answering method for private domains involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more embodiments provided below can refer to the above limitations and will not be repeated here.

[0087] In an exemplary embodiment, Figure 6 As shown, a large language model personalized question-answering device for private domains is provided, including:

[0088] The user input question module is used to receive the original question input by the user.

[0089] The private domain knowledge expansion module based on question rewriting is used to input the user's original question into the large language model to obtain a rewritten question; the rewritten question is the question after the large language model uses its own parameter knowledge to complete and professionally adjust the original question.

[0090] The private domain knowledge base retrieval module is used to match the rewriting problem with the data in the external database to obtain a relevant context set.

[0091] The context clustering module is used to cluster the relevant context set to obtain a plurality of category clusters.

[0092] The clustering knowledge simplification module is used to input each of the category clusters into the large language model to obtain summary data of each of the category clusters.

[0093] The answer generation module is used to input the summary data and the original question into the large language model to obtain the answer to the question.

[0094] When using the device, users can directly operate on the large language model personalized question-answering system for private domains according to the special needs of different private domains, thereby customizing a personalized system that meets their special needs. This operation method not only enhances the ease of use of the system, but also improves its flexibility and efficiency in different fields.

[0095] The specific execution process of a large language model personalized question-answering device for private domains that uses all modules is as follows: starting from the user inputting the question, the user question is passed into the private domain knowledge expansion module based on question rewriting; after the expanded question is sorted out, it is input into the private domain knowledge base retrieval module for matching related questions; then, the matching results are passed into the context clustering module and the clustering knowledge simplification module in turn to simplify the knowledge; finally, the user question supplement module is executed, and the simplified enhanced knowledge is passed into the answer generation module together with the user's initial question to obtain a professional medical field answer.

[0096] Embodiment three:

[0097] In an exemplary embodiment, a large language model personalized question answering method for private domain is provided. Figure 7 In order to solve the problems that traditional search enhancement generation technology cannot fully understand the user's question intention, the enhanced knowledge injected into the large model has high redundancy, and the generated answers deviate from the focus, this embodiment improves the performance of the large language model in the private domain by expanding first and then simplifying. First, in the retrieval stage, this embodiment expands and queries the user's questions through a private domain knowledge expansion scheme based on question rewriting to obtain relevant contexts from multiple angles; then, through a context simplification scheme based on clustering and summarization, the relevant knowledge is simplified according to categories, redundant data is eliminated, and higher-quality relevant knowledge is obtained; finally, this embodiment uses a user question supplement scheme to increase the weight of the initial user question, improve the loyalty of the generated answer to the initial question, and avoid the answer from deviating from the focus of the user's question.

[0098] To achieve the above objectives, the present embodiment optimizes the retrieval enhancement generation process and provides a knowledge expansion and simplification scheme for private domains. To address the problem of low accuracy of retrieval results, the present embodiment rewrites the user's initial question, expands relevant knowledge, and completes incomplete user questions. To address the problem of duplicate and redundant database information, the present embodiment clusters and summarizes the context obtained from the retrieval to prevent duplicate and redundant enhanced knowledge. To address the problem of deviation from the focus of the model answer, the present embodiment supplements the user's initial question and ensures that the generated answer does not deviate from the user's question intention by increasing the weight of the user's initial question. The knowledge expansion and simplification scheme for private domains consists of three sub-schemes, among which the private domain knowledge expansion based on question rewriting is the most critical scheme. The present embodiment improves the quality of the answer finally generated by the large language model question and answer system through the synergistic auxiliary effect of the context simplification scheme based on clustering and summarization and the user question supplement scheme. The specific implementation scheme is as follows:

[0099] The private domain knowledge expansion based on question rewriting is the first implementation scheme. This scheme mainly solves the problem of low precision of retrieval results. Since the input questions are generated by human users, the expression method and professional level of the questions are closely related to the users of the retrieval enhancement generation technology. However, the traditional retrieval enhancement generation technology directly inputs the questions input by the user into the vector database for similarity retrieval, and the quality of the retrieval results directly depends on the quality of the original questions input by the user. In some fields with little relevant knowledge, few usage scenarios, and high professional knowledge thresholds, users without relevant backgrounds may not be able to express their questions accurately and comprehensively, resulting in the inability of traditional retrieval enhancement generation technology to make full use of relevant knowledge and make subsequent answers. In response to the above problems, the present invention improves the precision of retrieval results through a private domain knowledge expansion scheme. The execution steps of this scheme are as follows:

[0100] In the first step, this embodiment inputs the user input question into the general language model, guides the large language model to generate several questions related to the input question, and uses the parameter knowledge inside the model to complete the question input by the user.

[0101] In the second step, this embodiment uses the related questions generated in the first step as input to the retrieval stage in the retrieval enhancement generation, matches the documents in the vector database, and obtains a richer set of related contexts.

[0102] In the third step, this embodiment reorders the relevant context obtained in the second step to obtain several context documents that are most similar to the user's initial question, and uses them as enhanced knowledge and inputs them into the large language model together with the user's original question to generate the final answer.

[0103] This solution not only understands the user's question intention from multiple perspectives and improves the diversity of the original questions, but also converts the user's colloquial questions into professional expressions, thereby obtaining more accurate and rich relevant context;

[0104] The context simplification based on clustering and summarization is the second implementation scheme. This scheme mainly solves the problem of high redundancy of retrieval results. Due to the rich information in the database, there are usually multiple related documents for the same information, and these documents may have semantic duplication and redundancy. If they are directly input into the subsequent large language model, it will lead to logical duplication in the generated answers, reducing the quality of the generated results. The main implementation steps of this scheme are as follows:

[0105] In the first step, this embodiment performs cluster analysis on the relevant contexts retrieved through the private domain knowledge expansion solution based on question rewriting, and divides them into several categories.

[0106] In the second step, for each category obtained in the first step, this embodiment inputs the relevant context into the large language model to obtain the summary knowledge of the category, which is used as the enhanced knowledge of the large language model in the subsequent generation stage, and is input into the large language model together with the user's initial question to obtain the final answer.

[0107] The context simplification scheme based on clustering and summarization makes the relevant context in the retrieval stage avoid information redundancy while covering comprehensive knowledge, making the retrieval results more refined, thereby improving the logic of generated answers.

[0108] The user question is supplemented as the third execution scheme. This scheme mainly solves the problem of low fidelity of generated answers. In order to complete the private domain knowledge of the large language model, traditional retrieval enhancement generation technology usually retrieves as many relevant documents as possible. When there are too many external documents retrieved, content that is irrelevant to the original question may be mixed in, causing the answer in the generation stage to deviate from the user's question intention, greatly reducing the correctness of the generated answer. Since this embodiment first expands the user question, then simplifies the relevant context, and finally inputs the large language model to generate the answer, each stage performs relevant processing on the knowledge, which may cause the relevant knowledge of the input model to deviate from the focus of the user's question.

[0109] In order to make the final answer closer to the key points of the user's question, this embodiment supplements the user's question in the following two stages to ensure the relevance of the generated answer:

[0110] First, in the knowledge expansion stage, this embodiment inputs the user's initial question and the related questions obtained by the private domain knowledge expansion solution based on question rewriting into the vector database for retrieval, so that the relevant context obtained in the retrieval stage is closer to the user's initial question.

[0111] Second, in the answer generation stage, when constructing the prompts of the generation stage, this embodiment repeats the user's initial question several times, and then connects and assembles it with the enhanced knowledge, directly improving the relevance of the generated answer to the user's initial question.

[0112] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 8 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a large language model personalized question and answer method for a private domain is implemented.

[0113] Those skilled in the art will understand that Figure 8 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0114] In an exemplary embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0115] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0116] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0117] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0118] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0119] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0120] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0121] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A large language model personalized question answering method for private domains, characterized in that: The large language model personalized question answering method for private domains includes: Input the user's original question into the large language model to obtain a rewritten question; the rewritten question is the question after the large language model uses its own parameter knowledge to complete and professionally adjust the original question; Matching the rewriting problem with data in an external database to obtain a relevant context set; Clustering the relevant context set to obtain a plurality of category clusters; Input each of the category clusters into a large language model to obtain summary data of each of the category clusters; The summary data and the original question are input into a large language model to obtain an answer to the question.

2. The method for personalized question answering based on a large language model for private domain according to claim 1, characterized in that: The relevant context set includes a plurality of documents; the relevant context set is clustered to obtain a plurality of category clusters, specifically including: Calculate the similarity between each document in the relevant context set and the original question; Sort all documents in the relevant context set by similarity to obtain a sorted set; Obtain the first k documents in the sorted set to obtain a similar set; The similarity sets are clustered to obtain several category clusters.

3. The method for personalized question answering based on a large language model for private domain according to claim 1, characterized in that: The relevant context set is clustered to obtain several category clusters, including: Vectorizing the relevant context set to obtain a vectorized set; KMeans clustering is performed on the vectorized set to obtain several category clusters.

4. The method for personalized question answering based on a large language model for private domain according to claim 1, characterized in that: The rewriting problem is matched with data in an external database to obtain a relevant context set, specifically including: The rewritten question and the original question are input into an external database for matching and retrieval to obtain a relevant context set.

5. The method for personalized question answering based on a large language model for private domain according to claim 1, characterized in that: The formula for obtaining the rewriting problem is: qSet=LLM(q,n); Among them, n is a hyperparameter, n=4, q is the original problem, qSet is the rewriting problem, and LLM is the large language model.

6. The method for personalized question answering based on a large language model for private domain according to claim 1, characterized in that: The method for constructing the external database specifically includes: Segment the text data to obtain text data blocks; The text data block is encoded using a vectorization model to obtain an external database.

7. A large language model personalized question-answering device for private domains, characterized in that: The large language model personalized question-answering device for private domains includes: A user input question module, used to receive the original question input by the user; The private domain knowledge expansion module based on question rewriting is used to input the user's original question into the large language model to obtain a rewritten question; the rewritten question is the question after the large language model uses its own parameter knowledge to complete and professionally adjust the original question; A private domain knowledge base retrieval module, used to match the rewriting problem with data in an external database to obtain a relevant context set; A context clustering module, used for clustering the relevant context set to obtain a plurality of category clusters; A clustering knowledge simplification module, used for inputting each of the category clusters into a large language model to obtain summary data of each of the category clusters; The answer generation module is used to input the summary data and the original question into the large language model to obtain the answer to the question.

8. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the large language model personalized question-answering method for a private domain as described in any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the large language model personalized question-answering method for private domains described in any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the large language model personalized question-answering method for private domains described in any one of claims 1 to 6 is implemented.

Images