Large model retrieval enhancement generation method and device, equipment, storage medium and product

Abstract

The invention discloses a large model retrieval enhancement generation method and device, equipment, a storage medium and a product, and relates to the technical field of question and answer large models.The method comprises the steps that questions input by a user are vectorized, retrieval is conducted in a preset vector database according to the vectorized questions, and a result set meeting preset conditions is obtained; and combining the vectorized question and result set into context information according to the prompt project template, and inputting the context information into a large language model for processing to generate a question answer. According to the method, retrieval is carried out according to the index tree of the vectorized question in the vector database, then the large model generates the answer according to the result set and the question, information in the long document is effectively integrated on different levels, different abstract levels are covered, the question can be understood and answered on different levels, and the user experience is improved. And the capability of processing complex queries is effectively improved.

Description

Technical Field

[0001] The present application relates to the technical field of question-answering big models, and in particular to a big model retrieval enhancement generation method, device, equipment, storage medium and product. Background Art

[0002] The wave of large language models (LLM) has swept almost all industries, but when it comes to professional scenarios or industry segments, general large models will face the problem of insufficient professional knowledge. The knowledge of general large models comes entirely from their training data, while the training sets of existing mainstream large models are basically built on public data on the Internet. Some real-time, non-public or offline data cannot be obtained, so this part of knowledge is not available. The knowledge of large models is static and comes from the data at the time of training, that is, large models cannot directly provide the latest information. In addition, large models also have the problem of hallucination. The underlying principles of all models are based on mathematical probability, and their model output is essentially a series of numerical operations. Large models are no exception, so there will be wrong answers, especially in scenarios where the large model itself does not have knowledge in a certain aspect or is not good at it. It is difficult to distinguish this kind of hallucination problem because it requires the user to have knowledge in the corresponding field.

[0003] It is in this context that Retrieval-Augmented Generation (RAG) technology came into being and became a major trend in the era of large models. RAG greatly improves the accuracy and relevance of content by retrieving relevant information from a wide range of professional document databases before the language model generates answers, and then using this professional information to guide the results generated by the large model. RAG effectively alleviates the hallucination problem, increases the speed of knowledge updating (large model + search engine), and enhances the traceability of content generation, making large language models more practical and credible in practical applications. However, existing retrieval-augmented generation technology solutions usually rely on the retrieved short text blocks to generate answers for the large model. However, when processing documents with long contexts, simply cutting the document or only processing its context is obviously not enough, especially for non-continuous documents, cross-document topics, and documents with scattered subject content. The question-answering effect is not good. Summary of the invention

[0004] The main purpose of this application is to provide a large-model retrieval enhancement generation method, device, equipment, storage medium and product, aiming to solve the technical problem that the existing retrieval enhancement generation scheme has poor question-answering effect when processing documents with long contexts.

[0005] To achieve the above objectives, the present application proposes a large model retrieval enhancement generation method, the method comprising:

[0006] Vectorize the questions input by the user to obtain the vectorized questions;

[0007] Searching the index tree in the preset vector database according to the vectorized question to obtain a result set that meets the preset conditions;

[0008] combining the vectorized question and the result set into context information according to a prompt engineering template;

[0009] The context information is input into a large language model for processing to generate an answer corresponding to the question input by the user.

[0010] In one embodiment, the step of searching the index tree in the preset vector database according to the vectorized question to obtain a result set that meets the preset conditions includes:

[0011] Collapse the index tree in the preset vector database into a single layer to obtain a collapsed set including all nodes in the index tree;

[0012] Determining a cosine similarity score between a query embedding corresponding to the vectorized question and an embedding of each node in the collapsed set;

[0013] Filtering the first target node in the folded set whose cosine similarity score is greater than a preset similarity threshold;

[0014] Iteratively select a preset number of second target nodes with the highest cosine similarity scores of the first target node, and add the second target nodes to the result set until a preset maximum number of tags is reached.

[0015] In one embodiment, before the step of vectorizing the question input by the user to obtain the vectorized question, the step further includes:

[0016] Obtaining a file uploaded by a user, and parsing the file to obtain a parsed file, wherein the parsed file includes text information, non-text information, and layout structure information;

[0017] Building an index tree according to the parsed file;

[0018] The index tree is stored in a preset vector database.

[0019] In one embodiment, the step of constructing an index tree according to the parsed file includes:

[0020] Splitting the parsed file into text blocks, vectorizing the text blocks to obtain vectorized text blocks, and using the text blocks and the vectorized text blocks as leaf nodes of an index tree;

[0021] Clustering the vectorized text blocks according to a clustering algorithm to obtain clustered groups;

[0022] Generating a summary based on the text blocks in the clustered groups;

[0023] The abstract summary is used as the text block, and the step of vectorizing the text block to obtain the vectorized text block is executed cyclically until the clustering termination condition is reached to obtain an index tree.

[0024] In one embodiment, the step of dividing the parsed file into text blocks includes:

[0025] Splitting the parsed file into short continuous text blocks with a length of preset characters;

[0026] When there is a target sentence in the parsed file whose length exceeds a preset character length, generating a sentence text block according to the target sentence;

[0027] The short continuous text block and the sentence text block are concatenated to obtain a text block.

[0028] In one embodiment, the step of clustering the vectorized text blocks according to a clustering algorithm to obtain clustered groups includes:

[0029] Determining nearest neighbor parameters in a uniform manifold approximation and projection method based on the dimensions of the vectorized text block;

[0030] Determining a hierarchical clustering structure by transforming the nearest neighbor parameter, wherein the hierarchical clustering structure includes global clustering and local clustering;

[0031] The number of clusters of the Gaussian mixture model is selected according to the Bayesian information criterion to obtain the optimal number of clusters;

[0032] The vectorized text blocks are clustered according to the hierarchical clustering structure and the optimal number of clusters by using the Gaussian mixture model to obtain clustered groups.

[0033] In addition, to achieve the above purpose, the present application also proposes a large model retrieval enhancement generation device, the large model retrieval enhancement generation device comprising:

[0034] A question vectorization module is used to vectorize the questions input by the user to obtain the vectorized questions;

[0035] A result set retrieval module, used to search the index tree in the preset vector database according to the vectorized question to obtain a result set that meets the preset conditions;

[0036] A context combination module, used for combining the vectorized question and the result set into context information according to a prompt engineering template;

[0037] The answer generation module is used to input the context information into the large language model for processing and generate an answer corresponding to the question input by the user.

[0038] In addition, to achieve the above-mentioned purpose, the present application also proposes a large model retrieval enhancement generation device, which includes: a memory, a processor, and a computer program stored on the memory and executable on the processor, and the computer program is configured to implement the steps of the large model retrieval enhancement generation method as described above.

[0039] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by the processor, the steps of the large model retrieval enhancement generation method described above are implemented.

[0040] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the steps of the large model retrieval enhancement generation method as described above.

[0041] The present application provides a large model retrieval enhancement generation method, which vectorizes the questions input by the user, searches in a preset vector database according to the vectorized questions, and obtains a result set that meets the preset conditions; the vectorized questions and result sets are combined into context information according to the prompt engineering template, and input into the large language model for processing to generate answers to the questions. The present application searches in the index tree in the vector database according to the vectorized questions, and then the large model generates answers based on the result set and the questions, effectively integrating the information in the long document at different levels, covering different levels of abstraction, and being able to understand and answer questions at different levels, effectively improving the ability to handle complex inquiries. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0043] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0044] Figure 1 A flowchart diagram of the first embodiment of the large model retrieval enhancement generation method of the present application;

[0045] Figure 2 A flow chart of the second embodiment of the large model retrieval enhancement generation method provided in this application;

[0046] Figure 3 This is a schematic diagram of the process of folding the search index tree in the second embodiment of the present application;

[0047] Figure 4 A flowchart diagram of the third embodiment of the large model retrieval enhancement generation method of the present application;

[0048] Figure 5 This is a schematic diagram of the overall scenario of the large model retrieval enhancement generation method of this application;

[0049] Figure 6 This is a schematic diagram of the overall process of tree construction and tree retrieval for this application;

[0050] Figure 7 This is a schematic diagram of the module structure of a large model retrieval enhancement generation device according to an embodiment of the present application;

[0051] Figure 8 This is a schematic diagram of the device structure of the hardware operating environment involved in the large model retrieval enhancement generation method in the embodiment of the present application.

[0052] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0053] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0054] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0055] The main solution of the embodiment of the present application is: vectorize the question input by the user to obtain the vectorized question; search the index tree in the preset vector database according to the vectorized question to obtain a result set that meets the preset conditions; combine the vectorized question and the result set into context information according to the prompt engineering template; input the context information into the large language model for processing to generate an answer corresponding to the question input by the user.

[0056] Existing retrieval enhancement generation technology solutions usually rely on the retrieved short text blocks to generate answers for large models. However, when dealing with documents with long contexts, simply cutting the document or only processing its context is obviously not enough, especially for non-continuous documents, cross-document topics, and documents with scattered topic content, the question answering effect is not good.

[0057] This application provides a solution, which vectorizes the questions input by the user, searches the preset vector database according to the vectorized questions, and obtains the result set that meets the preset conditions; the vectorized questions and the result set are combined into context information according to the prompt engineering template, and input into the large language model for processing to generate the answer to the question. This application searches the index tree in the vector database according to the vectorized questions, and then the large model generates answers based on the result set and the questions, which effectively integrates the information in the long document at different levels, covers different levels of abstraction, and can understand and answer questions at different levels, effectively improving the ability to handle complex inquiries.

[0058] It should be noted that the execution subject of the method of this embodiment can be a computing service device with large model retrieval enhancement generation, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc.; it can also be a large model retrieval enhancement generation device with the same or similar functions. This embodiment and the following embodiments will be described by taking the large model retrieval enhancement generation device as an example.

[0059] Based on this, the embodiment of the present application provides a large model retrieval enhancement generation method, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the large model retrieval enhancement generation method of the present application.

[0060] In this embodiment, the large model retrieval enhancement generation method includes steps S10 to S40:

[0061] Step S10, vectorizing the question input by the user to obtain the vectorized question.

[0062] It is understandable that the question input by the user can be vectorized through the sentence embedding model (Sentence-BERT, SBERT) to obtain the vectorized question.

[0063] Step S20, searching the index tree in the preset vector database according to the vectorized question to obtain a result set that meets the preset conditions.

[0064] It should be understood that the preset vector database pre-stores an index tree, which is an index tree generated based on the long context document uploaded by the user. The index tree effectively integrates the information in the long document at different levels and covers different abstract levels. Therefore, by searching in the index tree according to the vectorized question, a result set that better matches the user's question can be retrieved more accurately.

[0065] Step S30: combining the vectorized question and the result set into context information according to the prompt engineering template.

[0066] It is understandable that the vectorized questions and result sets can be combined into contextual information according to the prompt engineering template. Among them, the prompt engineering template is a structured text format that provides clear instructions and contextual information to the large language model to guide it to generate specific types of outputs. Prompt engineering templates are a technology for designing, experimenting, and optimizing input prompt words to guide the large language model to generate high-quality, accurate, and targeted outputs. These templates help the large language model better understand the task requirements and generate expected answers through carefully designed prompt words.

[0067] Step S40: input the context information into a large language model for processing to generate an answer corresponding to the question input by the user.

[0068] It should be understood that the generated context information can be input into a large language model for processing to generate an answer corresponding to the question input by the user.

[0069] This embodiment provides a large model retrieval enhancement generation method, which vectorizes the questions input by the user, searches in a preset vector database according to the vectorized questions, and obtains a result set that meets the preset conditions; the vectorized questions and result sets are combined into context information according to the prompt engineering template, and input into the large language model for processing to generate answers to the questions. This application searches the index tree in the vector database according to the vectorized questions, and then the large model generates answers based on the result set and the questions, effectively integrating the information in the long document at different levels, covering different levels of abstraction, and being able to understand and answer questions at different levels, effectively improving the ability to handle complex inquiries.

[0070] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can refer to the above introduction, and will not be repeated later. Figure 2 , step S20, the large model retrieval enhancement generation method further includes steps S201 to S204:

[0071] Step S201: fold an index tree in a preset vector database into a single layer to obtain a folded set including all nodes in the index tree.

[0072] It should be noted that you can refer to Figure 3 The retrieval method process of the folded tree is explained. Figure 3 This is a flow chart of folding the search index tree in the second embodiment of the present application. In this embodiment, all nodes in the index tree are considered at the same time. This method is not layer by layer, but folds the multi-layer tree into a single layer, essentially placing all nodes at the same level for comparison. First, the entire tree is folded into a single layer. This new node set, denoted as C, contains the nodes in each layer of the original index tree.

[0073] Step S202, determining a cosine similarity score between the query embedding corresponding to the vectorized question and the embedding of each node in the folded set.

[0074] It can be understood that SBERT can be used to vectorize user queries, obtain vectorized questions, and calculate the cosine similarity score between the query embedding corresponding to the vectorized question and the embeddings of all nodes in the collapsed set.

[0075] Step S203 , screening the first target node in the folded set whose cosine similarity score is greater than a preset similarity threshold.

[0076] It should be understood that a similarity threshold value may be preset, and then nodes in the collapsed set whose chord similarity scores are greater than the preset similarity threshold value may be preliminarily screened to obtain the preliminarily screened first target node.

[0077] Step S204, iteratively selecting a preset number of second target nodes having the highest cosine similarity scores of the first target node, and adding the second target nodes to a result set until a preset maximum number of tags is reached.

[0078] It should be understood that, under the condition that the threshold is met, the top-k second target nodes with the highest query cosine similarity scores are selected from the first target nodes. The second target nodes are added to the result set until the predefined maximum number of tags is reached to ensure that the input limit of the model is not exceeded.

[0079] In this embodiment, an index tree in a preset vector database is collapsed into a single layer to obtain a collapsed set containing all nodes in the index tree; the cosine similarity score between the query embedding corresponding to the vectorized question and the embedding of each node in the collapsed set is determined; the first target node in the collapsed set whose cosine similarity score is greater than a preset similarity threshold is screened; the first preset number of second target nodes with the highest cosine similarity score of the first target node are iteratively selected, and the second target nodes are added to the result set until the preset maximum number of tags is reached, so that when retrieving information in the index tree, fragments representing texts at different levels can be loaded to achieve integration across different abstract layers of text.

[0080] Based on the first embodiment of the present application, in the third embodiment of the present application, the same or similar contents as those in the first embodiment can be referred to the above description, and will not be described in detail later. Figure 4 Before step S10, the large model retrieval enhancement generation method further includes steps S01 to S03:

[0081] Step S01, obtaining a file uploaded by a user, and parsing the file to obtain a parsed file, wherein the parsed file includes text information, non-text information and layout structure information.

[0082] It is understandable that professional knowledge files in word, pdf, ppt, excel, html and other formats uploaded by users are obtained. File parsing is performed on files uploaded by users. Using machine learning algorithms, document content is automatically recognized, understood and processed. File parsing not only includes the recognition of text content, but also involves the parsing of non-text elements such as images, charts and tables. The Yolov8 (You Only Look Once version8) model is used for optical character recognition (OCR) / layout recognition / table structure recognition (TSR) to identify the layout structure of the document, such as titles, paragraphs, tables, images, etc.

[0083] Step S02: construct an index tree according to the parsed file.

[0084] It can be understood that the construction of the index tree for the parsed file information includes text segmentation, vectorization, clustering, summarization, etc. of the file until the index tree construction is completed.

[0085] In a feasible implementation, step S02 may include steps S021 to S024:

[0086] Step S021, dividing the parsed file into text blocks, vectorizing the text blocks to obtain vectorized text blocks, and using the text blocks and the vectorized text blocks as leaf nodes of an index tree.

[0087] It should be understood that the retrieval corpus (i.e., the parsed file information) can be divided into multiple text blocks based on existing retrieval enhancement technology. These texts are then vectorized using FlagOpen's bge-m3 model. These blocks and their corresponding vectors form the leaf nodes of a tree structure. Among them, FlagOpen is an open source system of large model technology jointly built by multiple research parties. The bge-m3 model is a general vector model designed to support multiple languages, long texts, and multiple retrieval methods.

[0088] In a feasible implementation, step S021 may include steps S0211 to S0213:

[0089] Step S0211, dividing the parsed file into short continuous text blocks with a length of preset characters.

[0090] It is understandable that the parsed file can be segmented into short continuous text blocks with a preset character length (for example, 128 word tokens).

[0091] Step S0212, when there is a target sentence whose length exceeds a preset character length in the parsed file, a sentence text block is generated according to the target sentence.

[0092] It should be understood that if a sentence exceeds the limit of 128 tokens, it is segmented at the granularity of "sentences", and the entire sentence is moved to the next chunk to generate sentence text chunks instead of truncating the sentence in the middle. This preserves the context and semantic coherence of the text within each chunk.

[0093] Step S0213, concatenating the short continuous text block and the sentence text block to obtain a text block.

[0094] It can be understood that a text block can be obtained by concatenating short continuous text blocks and sentence text blocks.

[0095] Step S022: clustering the vectorized text blocks according to a clustering algorithm to obtain clustered groups.

[0096] It is understandable that the vectorized text blocks can be clustered according to a clustering algorithm. The clustering algorithm plays a key role in the process of building a tree, and it can effectively organize text segments into groups with intrinsic connections. This step clusters related content together, which helps the subsequent retrieval process. The soft clustering algorithm is used to allow nodes to span multiple clusters instead of being limited to a fixed number of clusters. Because a single text paragraph often contains information related to multiple topics, it needs to be included in multiple different summaries.

[0097] In a feasible implementation, step S022 may include steps S0221 to S0224:

[0098] Step S0221, determining the nearest neighbor parameters in the uniform manifold approximation and projection method according to the dimension of the vectorized text block.

[0099] It is understandable that Gaussian Mixture Models (GMMs) are an implementation of soft clustering. Clustering can be performed using GMMs. However, the high dimensionality of vector embeddings poses a challenge to traditional GMMs because distance metrics may not perform well in high-dimensional spaces. To address this problem, Uniform Manifold Approximation and Projection (UMAP), a manifold learning technique for dimensionality reduction, can be used. The nearest neighbor parameter n_neighbors in UMAP determines the balance between preserving local structure and global structure. The nearest neighbor parameter n_neighbors takes the value of: n_neighbors = length(d)*0.8. Among them, length(d) is the dimension of the text vector corresponding to the vectorized text block.

[0100] Step S0222: determining a hierarchical clustering structure by transforming the nearest neighbor parameter, wherein the hierarchical clustering structure includes global clustering and local clustering.

[0101] It can be understood that a hierarchical clustering structure can be created by varying n_neighbors: first identifying global clusters, and then performing local clustering within these global clusters. This two-step clustering process captures the broad relationships between text data, from broad topics to specific details.

[0102] Step S0223, selecting the number of clusters of the Gaussian mixture model according to the Bayesian Information Criterion to obtain the optimal number of clusters.

[0103] It should be noted that if the combined context of a local cluster exceeds the labeling threshold of the summary model, the algorithm will recursively apply clustering within that cluster to ensure that the context remains within the labeling threshold. To determine the optimal number of clusters, the Bayesian Information Criterion (BIC) is used for model selection. BIC not only penalizes model complexity, but also rewards goodness of fit.

[0104] It is worth noting that for a given GMMs, the BIC is:

[0105]

[0106] Where N is the number of text segments (or data points), k is the number of model parameters, is the maximum value of the model likelihood function. In the context of GMMs, the number of parameters k is a function of the dimension of the input vector and the number of clusters. After obtaining the optimal number of clusters determined by BIC, the expectation-maximization algorithm (EM) is used to estimate the GMMs parameters, namely the mean, covariance, and mixing weights.

[0107] Step S0224, clustering the vectorized text blocks according to the hierarchical clustering structure and the optimal number of clusters using the Gaussian mixture model to obtain clustered groups.

[0108] It is understandable that the vectorized text blocks can be clustered according to the hierarchical clustering structure based on the Gaussian mixture model corresponding to the determined optimal number of clusters. GMMs assumes that data points are generated from a mixture of several Gaussian distributions. Given a set of N text segments (i.e., vectorized text blocks), each text segment is represented as a d-dimensional dense vector embedding, assuming that the probability of a text vector x belonging to the kth Gaussian distribution is expressed as:

[0109] p(x|k)=N(x;μ k ,∑k),

[0110] The weighted sum is calculated using the following formula to calculate the overall probability distribution:

[0111]

[0112] Among them, π k represents the mixture weight of the k-th Gaussian distribution.

[0113] Step S023: generating a summary based on the text blocks in the clustered groups.

[0114] It should be understood that after clustering the nodes using the clustering algorithm, the nodes in each cluster are sent to the large language model for summarization. Based on the large language model qwen-2.5-14B, similar text blocks are used to generate a concise and coherent summary of the selected node. The prompt words (Prompt) of the large model summary can be designed as shown in the following table.

[0115]

[0116] Step S024, taking the abstract summary as the text block, and looping the step of vectorizing the text block to obtain the vectorized text block, until the clustering termination condition is reached, and obtaining the index tree.

[0117] As can be understood, the text of these summaries is then re-vectorized, and the cycle of vectorization, clustering, and summarization continues until further clustering becomes infeasible, resulting in a structured, multi-level tree representation of the original document.

[0118] Step S03: storing the index tree in a preset vector database.

[0119] In this embodiment, a tree structure containing different levels of summaries is constructed from the bottom up by recursively vectorizing, clustering and summarizing the abstract text. This facilitates the effective integration of information in long documents at different levels during retrieval, covers different levels of abstraction, can understand and answer questions at different levels, and effectively improves the ability to handle complex inquiries.

[0120] For example, to help understand the implementation scenario of the large model retrieval enhancement generation method obtained by combining this embodiment with the above-mentioned embodiment 1, please refer to Figure 5 , Figure 5 This is a schematic diagram of the overall scenario of the large model retrieval enhancement generation method of this application, specifically:

[0121] This application consists of two parts, online and offline. Among them, the offline part is mainly the process of vectorizing private domain data such as professional knowledge files, building an index tree and storing it in the database. First, the professional knowledge files in word, pdf, ppt, excel, html and other formats uploaded by users are parsed. Then the index tree is constructed for the parsed file information, including text segmentation, vectorization, clustering, and summarization of the file until the index tree is built, and then the index tree is stored in the vector database. In addition, the online part is mainly combined with the prompt engineering template to form a complete context information for the user-entered questions and the top-k related paragraphs obtained based on the tree retrieval scheme to generate answers for the large model. First, the user-entered question query is vectorized, and then the top-k nodes with the highest cosine similarity score that meets the preset score threshold are retrieved in the vector database based on the tree retrieval method proposed in this application. Finally, combined with the prompt engineering template, the user-entered questions and the top-k nodes obtained by the tree retrieval form context information and are given to the large model to generate answers.

[0122] It is understandable that, combined with Figure 6 The overall process of tree construction (S1-S4) and tree retrieval (S5-S8) is explained. First, the process of tree construction is explained. S1: Start by splitting the parsed document information into short continuous texts of 128 tokens in length, similar to traditional retrieval enhancement techniques. If a sentence exceeds the 128 token limit, the entire sentence is moved to the next block instead of cutting it in the middle of the sentence. This preserves the context and semantic coherence of the text within each block. These texts are then vectorized using FlagOpen's bge-m3 model. These blocks and their corresponding vectors form the leaf nodes of the tree structure. S2: The nodes are clustered and grouped using a Gaussian mixture model. S3: After clustering, the nodes in each cluster are sent to a large language model for summarization. A summary is generated based on the grouped text using the large language model gpt-3.5-turbo. S4: These summarized texts are then re-vectorized, and the cycle of vectorization, clustering, and summary continues until further clustering becomes infeasible, resulting in a structured, multi-level tree representation of the original document.

[0123] Secondly, the process of tree retrieval is explained. S5: First, the entire tree is collapsed into a single layer. This new node set, denoted as, contains the nodes in each layer of the original tree. S6: Next, use SBERT to vectorize the user query, obtain the vectorized question, and calculate the cosine similarity between the query embedding corresponding to the vectorized question and the embedding of all nodes in the collapsed set. S7: Filter out the nodes that exceed the similarity threshold, and select the top-k nodes with the highest cosine similarity score with the query under the condition that the threshold is met. S8: Finally, iteratively add nodes to the result set until the predefined maximum number of tags is reached to ensure that the input limit of the model is not exceeded.

[0124] This application uses a recursive clustering algorithm and text summarization technology to create a hierarchical tree structure that can synthesize information between various parts of the search corpus. At the query stage, by combining the original text and high-level summaries for retrieval, it can effectively respond to complex topic queries to detail-oriented questions. Relying on this tree structure enables more efficient retrieval.

[0125] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the large model retrieval enhancement generation method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0126] This application also provides a large model retrieval enhancement generation device, please refer to Figure 7 , the large model retrieval enhancement generation device comprises:

[0127] A question vectorization module 10 is used to vectorize the question input by the user to obtain the vectorized question;

[0128] A result set retrieval module 20, used to search the index tree in the preset vector database according to the vectorized question to obtain a result set that meets the preset conditions;

[0129] A context combining module 30, for combining the vectorized question and the result set into context information according to a prompt engineering template;

[0130] The answer generation module 40 is used to input the context information into the large language model for processing, and generate an answer corresponding to the question input by the user.

[0131] The large model retrieval enhancement generation device provided by the present application adopts the large model retrieval enhancement generation method in the above embodiment to solve the technical problem. Compared with the prior art, the beneficial effects of the large model retrieval enhancement generation device provided by the present application are the same as the beneficial effects of the large model retrieval enhancement generation method provided by the above embodiment, and the other technical features in the large model retrieval enhancement generation device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0132] The present application provides a large model retrieval enhancement generation device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the large model retrieval enhancement generation method in the above-mentioned embodiment one.

[0133] Reference below Figure 8 , which shows a schematic diagram of the structure of a large model retrieval enhancement generation device suitable for implementing the embodiment of the present application. The large model retrieval enhancement generation device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 8 The large model retrieval enhancement generation device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0134] like Figure 8As shown, the large model retrieval enhancement generation device may include a processing device 1001 (such as a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. Various programs and data required for the operation of the large model retrieval enhancement generation device are also stored in RAM1004. The processing device 1001, ROM1002 and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 1003 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 1009. The communication device 1009 can allow the large model retrieval enhancement generation device to communicate with other devices wirelessly or by wire to exchange data. Although the large model retrieval enhancement generation device with various systems is shown in the figure, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems may be implemented or have instead.

[0135] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0136] The large model retrieval enhancement generation device provided by the present application adopts the large model retrieval enhancement generation method in the above embodiment, which can solve the technical problem of large model retrieval enhancement generation. Compared with the prior art, the beneficial effects of the large model retrieval enhancement generation device provided by the present application are the same as the beneficial effects of the large model retrieval enhancement generation method provided by the above embodiment, and the other technical features in the large model retrieval enhancement generation device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0137] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0138] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0139] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the large model retrieval enhancement generation method in the above-mentioned embodiment.

[0140] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0141] The computer-readable storage medium may be included in the large model retrieval enhancement generation device; or may exist independently without being assembled into the large model retrieval enhancement generation device.

[0142] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the large model retrieval enhancement generation device, the large model retrieval enhancement generation device: vectorizes the question input by the user to obtain the vectorized question; searches the index tree in the preset vector database according to the vectorized question to obtain a result set that meets the preset conditions; combines the vectorized question and the result set into context information according to the prompt engineering template; inputs the context information into the large language model for processing to generate an answer corresponding to the question input by the user.

[0143] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0144] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0145] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0146] The readable storage medium provided in this application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned large model retrieval enhancement generation method, and can solve the technical problem. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as the beneficial effects of the large model retrieval enhancement generation method provided in the above-mentioned embodiment, and will not be repeated here.

[0147] The present application also provides a computer program product, including a computer program, which implements the steps of the large model retrieval enhancement generation method as described above when executed by a processor.

[0148] The computer program product provided by this application can solve the technical problem. Compared with the prior art, the beneficial effects of the computer program product provided by this application are the same as the beneficial effects of the large model retrieval enhancement generation method provided by the above embodiment, which will not be repeated here.

[0149] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A large model retrieval enhancement generation method, characterized in that: The method includes: Vectorize the questions input by the user to obtain the vectorized questions; Searching the index tree in the preset vector database according to the vectorized question to obtain a result set that meets the preset conditions; combining the vectorized question and the result set into context information according to a prompt engineering template; The context information is input into a large language model for processing to generate an answer corresponding to the question input by the user.

2. The method according to claim 1, characterized in that The step of searching the index tree in the preset vector database according to the vectorized question to obtain a result set that meets the preset conditions includes: Collapse the index tree in the preset vector database into a single layer to obtain a collapsed set including all nodes in the index tree; Determining a cosine similarity score between a query embedding corresponding to the vectorized question and an embedding of each node in the collapsed set; Filtering the first target node in the folded set whose cosine similarity score is greater than a preset similarity threshold; Iteratively select a preset number of second target nodes with the highest cosine similarity scores of the first target node, and add the second target nodes to the result set until a preset maximum number of tags is reached.

3. The method according to claim 1, characterized in that Before the step of vectorizing the question input by the user to obtain the vectorized question, the method further includes: Obtaining a file uploaded by a user, and parsing the file to obtain a parsed file, wherein the parsed file includes text information, non-text information, and layout structure information; Building an index tree according to the parsed file; The index tree is stored in a preset vector database.

4. The method according to claim 3, characterized in that The step of constructing an index tree according to the parsed file comprises: Splitting the parsed file into text blocks, vectorizing the text blocks to obtain vectorized text blocks, and using the text blocks and the vectorized text blocks as leaf nodes of an index tree; Clustering the vectorized text blocks according to a clustering algorithm to obtain clustered groups; Generating a summary based on the text blocks in the clustered groups; The abstract summary is used as the text block, and the step of vectorizing the text block to obtain the vectorized text block is executed cyclically until the clustering termination condition is reached to obtain an index tree.

5. The method according to claim 4, characterized in that The step of dividing the parsed file into text blocks comprises: Splitting the parsed file into short continuous text blocks with a length of preset characters; When there is a target sentence in the parsed file whose length exceeds a preset character length, generating a sentence text block according to the target sentence; The short continuous text block and the sentence text block are concatenated to obtain a text block.

6. The method according to claim 4, characterized in that The step of clustering the vectorized text blocks according to a clustering algorithm to obtain clustered groups includes: Determining nearest neighbor parameters in a uniform manifold approximation and projection method based on the dimensions of the vectorized text block; Determining a hierarchical clustering structure by transforming the nearest neighbor parameter, wherein the hierarchical clustering structure includes global clustering and local clustering; The number of clusters of the Gaussian mixture model is selected according to the Bayesian information criterion to obtain the optimal number of clusters; The vectorized text blocks are clustered according to the hierarchical clustering structure and the optimal number of clusters by using the Gaussian mixture model to obtain clustered groups.

7. A large model retrieval enhancement generation device, characterized in that: The large model retrieval enhancement generation device comprises: A question vectorization module is used to vectorize the questions input by the user to obtain the vectorized questions; A result set retrieval module, used to search the index tree in the preset vector database according to the vectorized question to obtain a result set that meets the preset conditions; A context combination module, used for combining the vectorized question and the result set into context information according to a prompt engineering template; The answer generation module is used to input the context information into the large language model for processing and generate an answer corresponding to the question input by the user.

8. A large model retrieval enhancement generation device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the large model retrieval enhancement generation method according to any one of claims 1 to 6.

9. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the large model retrieval enhancement generation method according to any one of claims 1 to 6 are implemented.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the large model retrieval enhancement generation method according to any one of claims 1 to 6 are implemented.

Images