Retrieval enhancement generation method and device of large language model, equipment and medium

Abstract

The invention provides a retrieval enhancement generation method and device for a large language model, equipment and a medium, and relates to the technical field of computers. The method comprises the following steps: converting a query statement of a user into a query vector; according to the similarity between the query vector and the index vector, a plurality of target information blocks are retrieved in a vector database, the target information blocks are data fragments related to the query statement, and vectors of knowledge data are stored in the vector database; the method comprises the following steps: analyzing a query statement of a user to determine a query context of the query statement; calling an expansion strategy engine to determine an expansion strategy of the plurality of target information blocks according to the query context of the user to expand the plurality of target information blocks to obtain a plurality of associated information blocks associated with each target information block; and determining a reply statement of the query statement based on the plurality of target information blocks and the plurality of associated information blocks. According to the embodiment of the invention, the retrieval process is more accurate, and the accuracy and comprehensiveness of the reply content can be improved.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a retrieval enhancement generation method, device, equipment and medium for a large language model. Background Art

[0002] Large Language Models (LLMs) are deep learning models trained with large amounts of text data that have the ability to generate natural language text, understand context, and perform various language tasks. Large language models perform well in a wide range of natural language processing (NLP) tasks, such as machine translation, question-answering systems, and text summarization.

[0003] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0004] The present disclosure provides a retrieval enhancement generation method, device, equipment and medium for a large language model, which at least to a certain extent improves the problem of insufficient accuracy and comprehensiveness of response content in large language models in related technologies.

[0005] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0006] According to one aspect of the present disclosure, a retrieval enhancement generation method of a large language model is provided, comprising:

[0007] Convert the user's query statement into a query vector;

[0008] According to the similarity between the query vector and the index vector, multiple target information blocks are retrieved from the vector database, where the target information blocks are data fragments related to the query statement, and the vector database stores vectors of knowledge data;

[0009] By analyzing the user's query statement, the query context of the query statement is determined;

[0010] Calling an extension strategy engine to determine extension strategies for multiple target information blocks according to the user's query context, and extending the multiple target information blocks based on the extension strategies to obtain multiple associated information blocks related to each target information block;

[0011] Based on the multiple target information blocks and the multiple associated information blocks, a reply sentence of the query sentence is determined.

[0012] In one embodiment of the present disclosure, before calling the block strategy engine to retrieve multiple target information blocks from the vector database according to the similarity between the query vector and the index vector, the method further includes:

[0013] Acquire knowledge data from multiple data sources;

[0014] Analyze the characteristics of the acquired knowledge data, and determine a data segmentation strategy based on the characteristics of the knowledge data, wherein the characteristics of the knowledge data include one or more of data length, content complexity, and structured degree;

[0015] Based on the data block strategy, the knowledge data is divided into multiple information blocks, each of which retains context information;

[0016] Based on the context information retained by each information block, similarity analysis is performed on multiple information blocks to construct an inter-block relationship graph;

[0017] The embedding model is called to vectorize multiple information blocks and store them in the vector database.

[0018] In one embodiment of the present disclosure, determining a reply statement to a query statement based on a plurality of target information blocks and a plurality of associated information blocks includes:

[0019] sorting all the associated information blocks based on the magnitude of the correlation value between each associated information block and the corresponding target information block;

[0020] According to the sorting, k related information blocks are selected from all the related information blocks;

[0021] Based on the multiple target information blocks and the k associated information blocks, a reply statement of the query statement is determined.

[0022] In one embodiment of the present disclosure, before selecting k associated information blocks with the largest correlation values ​​from all associated information blocks according to the sorting, the method further includes:

[0023] The value of k is calculated based on the corpus length of the user's query context, the scenario to which the query statement belongs, the size of the information block, the maximum context length of the large language model, and the semantic similarity of the information block.

[0024] In one embodiment of the present disclosure, the relationship between the corpus length of the user's query context, the maximum context length of the large language model and the semantic similarity of the information block and the value of k is positively correlated;

[0025] The relationship between the information block size and the value of k is negatively correlated.

[0026] In one embodiment of the present disclosure, according to the similarity between the query vector and the index vector, multiple target information blocks are retrieved from the vector database, including:

[0027] According to the similarity between the query vector and the index vector, multiple information blocks are retrieved from the vector database;

[0028] Sorting all information blocks based on the similarity value between the index vector of each information block and the query vector;

[0029] According to the sorting, multiple target information blocks are selected from all the information blocks.

[0030] In one embodiment of the present disclosure, the multiple target information blocks are expanded based on the expansion strategy to obtain multiple associated information blocks related to each target information block, including one or more of the following methods:

[0031] Searching for the parent-child relationship, finding the parent information block of the target information block, and using the parent information block as the associated information block of the target information block;

[0032] and / or, searching for a context information block, and taking an information block adjacent to the target information block as an associated information block of the target information block;

[0033] And / or, query the associated information blocks of the target information block through the inter-block relationship graph.

[0034] According to another aspect of the present disclosure, a retrieval enhancement generation device for a large language model is provided, comprising a query vector conversion module, a target information retrieval module, a context determination module, an information expansion module and a reply determination module.

[0035] A query vector conversion module, used to convert the user's query statement into a query vector;

[0036] A target information retrieval module is used to retrieve multiple target information blocks from a vector database according to the similarity between the query vector and the index vector. The target information blocks are data fragments related to the query statement. The vector database stores vectors of knowledge data.

[0037] A context determination module, used to determine the query context of the query statement by analyzing the user's query statement;

[0038] The information extension module is used to call the extension strategy engine to determine the extension strategy of multiple target information blocks according to the user's query context, and to extend the multiple target information blocks based on the extension strategy to obtain multiple associated information blocks related to each target information block;

[0039] The reply determination module is used to determine a reply statement to the query statement based on multiple target information blocks and multiple associated information blocks.

[0040] According to another aspect of the present disclosure, an electronic device is provided, including: a memory for storing instructions; and a processor for calling the instructions stored in the memory to implement the above-mentioned retrieval enhancement generation method of a large language model.

[0041] According to another aspect of the present disclosure, a computer-readable storage medium is provided, on which computer instructions are stored. When the computer instructions are executed by a processor, the above-mentioned retrieval enhancement generation method of the large language model is implemented.

[0042] According to another aspect of the present disclosure, a computer program product is provided. The computer program product stores instructions, and when the instructions are executed by a computer, the computer implements the above-mentioned retrieval enhancement generation method of a large language model.

[0043] According to yet another aspect of the present disclosure, there is provided a chip, comprising at least one processor and an interface;

[0044] An interface for providing program instructions or data to at least one processor;

[0045] At least one processor is used to execute program instructions to implement the above-mentioned retrieval enhancement generation method of a large language model.

[0046] The retrieval enhancement generation method, device, equipment and medium of the large language model provided by the embodiments of the present disclosure can quickly locate the most relevant multiple target information blocks by converting the user's query statement into a query vector and performing a similarity search in a vector database, thereby ensuring the high relevance of the retrieval results and improving the accuracy of the answer; based on the user's query context, multiple associated information blocks related to each target information block are further searched, and these additional information blocks provide richer background and supporting details, thereby enhancing the content depth and logical coherence of the answer; in addition, when querying the multiple associated information blocks related to each target information block, the user's query context is referenced, and thus in application scenarios involving multiple rounds of dialogue, the dialogue state can be effectively maintained, the context changes can be understood, and consistent and coherent answers can be provided.

[0047] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0049] Obviously, the drawings described below are only some embodiments of the present disclosure, and a person skilled in the art can obtain other drawings based on these drawings without any creative work.

[0050] Figure 1 A flow chart of a retrieval enhancement generation method of a large language model in an embodiment of the present disclosure is shown;

[0051] Figure 2 A flowchart of constructing a vector database in an embodiment of the present disclosure is shown;

[0052] Figure 3 A flowchart of retrieving a target information block in an embodiment of the present disclosure is shown;

[0053] Figure 4 A flow chart of determining a reply statement for a query statement in an embodiment of the present disclosure is shown;

[0054] Figure 5 A schematic diagram of a retrieval enhancement generation system of a large language model in an embodiment of the present disclosure is shown;

[0055] Figure 6 A schematic diagram of a retrieval enhancement generation device for a large language model in an embodiment of the present disclosure is shown;

[0056] Figure 7 A structural block diagram of an electronic device in an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical scheme and advantages of the embodiments of the present disclosure clearer, the technical scheme in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The components of the embodiments of the present disclosure generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the drawings is not intended to limit the scope of the present disclosure for protection, but merely represents the selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present disclosure.

[0058] For ease of understanding, the following first explains the relevant concepts involved in the present disclosure as follows:

[0059] Large Language Model (LLM): refers to a deep learning model trained with a large amount of text data that can generate natural language text or understand the meaning of language text. Large language models can handle a variety of natural language tasks, such as text classification, question answering, dialogue, etc., and are an important path to artificial intelligence.

[0060] Retrieval-Augmented Generation (RAG): A deep learning technology that combines retrieval and generation capabilities, belonging to large model technology, aiming to enhance the model's generation ability by introducing external knowledge bases or datasets, enabling it to generate more accurate, contextually relevant, and factually consistent text content.

[0061] It can be understood that the data involved in this technical solution (including but not limited to the data itself, data acquisition, or use) should comply with the requirements of corresponding laws, regulations, and related provisions.

[0062] The following will explain this exemplary embodiment in detail with reference to the accompanying drawings and embodiments.

[0063] First, an embodiment of the present disclosure provides a retrieval-augmented generation method for a large language model, which can be executed by any electronic device with computing and processing capabilities.

[0064] Figure 1 A flowchart showing a retrieval-augmented generation method for a large language model in an embodiment of the present disclosure is as Figure 1 As shown, the retrieval-augmented generation method for a large language model provided in an embodiment of the present disclosure includes S101 - S105.

[0065] In S101, the user's query statement is converted into a query vector.

[0066] In some embodiments, before converting the user's query statement into a query vector, the query statement can be preprocessed first. The preprocessing can include text cleaning, tokenization, stop word removal, lemmatization, and stemming, etc. Among them, text cleaning is used to remove unnecessary characters, punctuation marks, etc., to ensure that the input text is clean and tidy. Tokenization is used to split the sentence into words or sub-word units. For example, tools such as NLTK and spaCy are used for tokenization. Stop word removal is used to remove common meaningless words (such as "de", "shi", etc.) to reduce noise and focus on meaningful content. Lemmatization / Stemming is used to restore words to their basic forms to better capture semantic information.

[0067] Converting the user's query statement into a query vector can include selecting an embedding model suitable for the task requirements, and according to the selected embedding model, converting the preprocessed query statement into a vector representation. Among them, the specific method depends on the type of model used.

[0068] In S102, according to the similarity between the query vector and the index vector, a plurality of target information blocks are retrieved from a vector database, where the target information blocks are data segments related to the query statement, and the vector database stores vectors of knowledge data.

[0069] A vector database is a database for storing knowledge data, wherein the knowledge data is stored in the database in the form of vectors. An "information block" can also be called a data block or a data slice, and a target information block can be a text data in a vector database.

[0070] In S103, the query context of the query statement is determined by analyzing the query statement of the user.

[0071] The query context is the specific background information of the question or request raised by the user (including the query statement above), as well as the previous interaction content related to the query.

[0072] In S104, the extension policy engine is called to determine the extension policies of the multiple target information blocks according to the user's query context, and the multiple target information blocks are extended based on the extension policies to obtain multiple associated information blocks related to each target information block.

[0073] The associated information block is an information block related to the target information block, wherein the correlation may be content correlation, including the superordinate concept knowledge or subordinate concept knowledge of the knowledge point involved in the target information block, and the context in the source data of the target information block, etc.

[0074] In S105, a reply sentence of the query sentence is determined based on the plurality of target information blocks and the plurality of associated information blocks.

[0075] Multiple related information blocks can be used to supplement the knowledge of the corresponding target information block and expand it, such as providing richer background and supporting details, which can enhance the content depth and logical coherence of the answer.

[0076] Before retrieving multiple target information blocks from the vector database based on the similarity between the query vector and the index vector, a vector database can also be constructed. Figure 2 As shown, constructing a vector database may include S201-S205.

[0077] In S201 , knowledge data is acquired from multiple data sources.

[0078] Acquiring knowledge data from multiple data sources includes acquiring data from various text data sources, such as Wikipedia, news articles, books, code libraries, etc. The data formats include word, pdf, csv, etc., and converting the data into a unified text format, such as JSON, etc. Finally, the data is cleaned, such as removing noise data, erroneous data, duplicate data, etc., to ensure data quality.

[0079] In S202, the characteristics of the acquired knowledge data are analyzed, and a block strategy engine is called to determine a data block strategy based on the characteristics of the knowledge data, wherein the characteristics of the knowledge data include one or more of data length, content complexity, and structured degree.

[0080] Determining the data chunking strategy based on the characteristics of the knowledge data may be calling a chunking strategy engine to select a data chunking strategy from a chunking strategy library based on the characteristics of the knowledge data.

[0081] Commonly used strategies in the block strategy library include but are not limited to the following:

[0082] Fixed-length segmentation: directly divide the text into multiple blocks according to a fixed number of characters or words. The block size k is determined by the segmentation strategy engine based on the characteristics of the data.

[0083] Sentence-based segmentation: In this method, Chunks are segmented based on sentence boundaries, that is, natural language processing tools are used to identify the end of sentences, such as punctuation marks such as periods and question marks, and then the text is divided into multiple parts based on sentence boundaries.

[0084] Sliding Window: In this approach, the creation of chunks involves an overlapping sliding window. For example, the window size can be set to 500 words and the step size can be set to 100 words. The window size and step size are generated and adjusted in real time by the chunking strategy engine.

[0085] Topic or paragraph-based segmentation: This advanced segmentation strategy relies on the intrinsic structure of the text, such as paragraphs or topic paragraphs. Chunks are segmented by identifying the topic change points in paragraphs or texts, which can maintain a high degree of semantic coherence and is suitable for well-structured texts, such as academic papers or formal reports.

[0086] Semantic similarity-based segmentation: Use machine learning models to evaluate the semantic similarity between sentences or paragraphs and segment them when the similarity drops below a certain threshold. This method attempts to optimize retrieval while maintaining a high degree of semantic coherence.

[0087] In S203, based on the data segmentation strategy, the knowledge data is segmented into multiple information blocks, each of which retains context information.

[0088] Dividing knowledge data into multiple information blocks, each of which retains contextual information, means that when processing large-scale text data, it is not only necessary to divide it into smaller and more manageable fragments (i.e., information blocks or data blocks), but also to ensure that these fragments contain sufficient background information to maintain their meaning and coherence.

[0089] In addition, when knowledge data is divided into multiple information blocks, each information block (data block) needs to be numbered or marked for subsequent processing and indexing. Special cases such as text paragraphs and chapters should be handled to ensure that they are not split at logical boundaries.

[0090] In S204, based on the context information retained by each information block, similarity analysis is performed on multiple information blocks to construct an inter-block relationship graph.

[0091] Similarity analysis of multiple information blocks can be performed to identify content relevance or topic similarity, build a relationship graph between blocks, use natural language processing technology (such as text embedding, similarity calculation) to evaluate the relationship between blocks, and generate a relationship matrix or graph structure to facilitate relationship search during subsequent retrieval.

[0092] In S205, the embedding model is called to vectorize the multiple information blocks and store them in the vector database.

[0093] It can be understood that the data generated by the above steps can be locally persisted for subsequent retrieval and reading, wherein the vector data is stored in the vector database, and the text data and slice relationships (i.e., the relationships between information blocks) are stored in the corresponding database.

[0094] In some embodiments, Figure 3 As shown, the above S102 retrieves multiple target information blocks from the vector database according to the similarity between the query vector and the index vector, including S301-S303.

[0095] In S301, multiple information blocks are retrieved from a vector database according to the similarity between the query vector and the index vector;

[0096] In S302, all information blocks are sorted based on the similarity value between the index vector of each information block and the query vector;

[0097] In S303, multiple target information blocks are selected from all the information blocks according to the ranking.

[0098] In some embodiments, the ranking algorithm includes BM25, TF-IDF, etc. Sorting can also be performed based on a reranking model: such as bge-reranker-base and bge-reranker-large, which are deep learning models trained based on machine learning algorithms and learn how to rerank search results through training data.

[0099] In some embodiments, the above S104 expands the multiple target information blocks based on the expansion strategy to obtain the multiple associated information blocks for each target information block, which may include one or more of the following methods:

[0100] Searching for the parent-child relationship, finding the parent information block of the target information block, and using the parent information block as the associated information block of the target information block;

[0101] and / or, searching for a context information block, and taking an information block adjacent to the target information block as an associated information block of the target information block;

[0102] And / or, query the associated information blocks of the target information block through the inter-block relationship graph.

[0103] In some embodiments, Figure 4 As shown, the above S105 determines the reply statement of the query statement based on multiple target information blocks and multiple associated information blocks, including S401-S403.

[0104] In S401, all associated information blocks are sorted based on the magnitude of the correlation value between each associated information block and the corresponding target information block;

[0105] In S402, k related information blocks are selected from all related information blocks according to the ranking;

[0106] In S403, a reply statement to the query statement is determined based on the multiple target information blocks and the k associated information blocks.

[0107] The inventors found that in the above embodiments, the determination of the k value is very critical. How to choose an appropriate k value directly affects the quality and relevance of the text generated by the model and the response speed of the system. If the k value is too small, the model may not be able to obtain enough context information; if the k value is too large, it will exceed the maximum context length limit of the model, or introduce too much irrelevant information, reducing the generation efficiency.

[0108] In some embodiments, based on the sorting, before selecting the k associated information blocks with the largest relevance values ​​from all the associated information blocks, the value of k can also be calculated based on the corpus length of the user's query context, the scenario to which the query statement belongs, the information block size, the maximum context length of the large language model and the semantic similarity of the information blocks.

[0109] In some embodiments, the relationship between the corpus length of the user's query context, the maximum context length of the large language model and the semantic similarity of the information block and the value of k is positively correlated; the relationship between the information block size and the value of k is negatively correlated.

[0110] The inventors found that factors affecting the k value include: the corpus length of the user's query context, the scenario to which the query statement belongs, the information block size (slice size), the maximum context length of the large language model, and the semantic similarity of the information block (slice semantic similarity).

[0111] If the corpus is short, the number of extended slices k can be smaller. If the corpus is long, increase k appropriately.

[0112] For scenarios with high accuracy requirements (such as medical and legal documents), more contextual information support is required, so k can be set larger.

[0113] The larger the slice (information block), the higher the information density, and k can be reduced accordingly; otherwise, k can be appropriately increased.

[0114] If the context length is shorter, k needs to be reduced to avoid exceeding the processing capacity of the model.

[0115] If the retrieved fragment (information block) has a high degree of similarity with the previous and next fragments, k can be increased; otherwise, it can be decreased.

[0116] As an example, the k value calculation model is as follows:

[0117]

[0118] Among them, L is the expected context corpus length, s is the average size of the slice, C is the maximum context length of the model, and f(R,S) is the scene adjustment factor, which is a weighted function based on the semantic similarity R and the scene S. The function is defined and assigned based on the specific application scenario and corpus.

[0119] Figure 5 A retrieval enhancement generation system of a large language model in an embodiment of the present disclosure is shown. Figure 5 As shown, the retrieval enhancement generation system of the large language model includes five functional modules: strategy engine, data segmentation processing, retrieval information expansion, strategy library and data storage module.

[0120] The decision engine is the core of the system, which consists of two sub-modules: the chunking strategy engine and the expansion strategy engine. They are responsible for deciding how to chunk data and how to expand the retrieval context information.

[0121] Chunking Strategy Engine: Large-scale data sets usually need to be divided into smaller chunks for easier processing and retrieval. This module is responsible for determining how to chunk the data, that is, performing real-time intelligent analysis of the data, capturing the structure and characteristics of the data, and dynamically selecting the appropriate chunking scheme from the strategy library. Specific considerations include: data type, data coherence, data adaptation task type, improving retrieval efficiency or reducing resource consumption, etc.

[0122] The Expansion Strategy Engine determines how to expand context information during the retrieval process. The Expansion Strategy Engine analyzes the input query or data and selects the most appropriate way to obtain relevant context information, thereby improving retrieval accuracy.

[0123] The Data Chunking Processing module is responsible for dividing large data sets into small chunks, while ensuring that the logical relationships between these chunks are maintained. This module consists of two parts: data chunking and building relationships between chunks.

[0124] Data Chunking divides the data set into small blocks suitable for retrieval and processing according to the instructions of the chunking strategy engine. Chunking can effectively reduce the processing burden of large models on ultra-large-scale data.

[0125] Inter-Chunk Relationship Construction After data segmentation is completed, it ensures that the relationship between different data blocks can be reasonably expressed and maintained. This is crucial for large models to understand contextual information, perform cross-chunk retrieval, and integrate retrieval results.

[0126] The Retrieval Context Expansion module is used to enhance the model's understanding and output effects through context information during the retrieval process. It consists of two parts: extended information extraction and extended information merging.

[0127] Expansion Information Extraction extracts contextual information related to the current retrieval task from raw data, external knowledge bases, or other information sources. By expanding relevant information, the large model can understand the query context more comprehensively, thereby generating more accurate retrieval results.

[0128] Expansion Information Merging merges the extracted extended information with the original data to enhance the understanding ability of the large model during the retrieval process. This step ensures that the extended information is effectively used without introducing information redundancy or noise. Specific considerations: semantically fuse the extended information with the core data in the retrieval task to ensure that the contextual information can be seamlessly integrated into the query. By merging the extended information, the relevance and accuracy of the retrieval results are improved, and the interference of information irrelevant to the query is reduced.

[0129] The Policy Library stores the policy sets used by the system in different tasks and situations. These policies are used to guide the decision-making process of the block policy engine and the extended policy engine. The policy library not only contains static rules, but can also be dynamically adjusted based on historical search results and feedback.

[0130] The Data Storage Module stores and manages all data used in the system, including original data, segmented data, expanded information, and related metadata. The Data Storage Module ensures data security, accessibility, and efficient retrieval.

[0131] Combine the following Figure 5 The retrieval enhancement generation system of the large language model shown in the figure describes in detail the retrieval enhancement generation method of the large language model provided by the embodiment of the present disclosure, including steps 1 to 14.

[0132] Step 1, data reading: connect to the data source to complete the acquisition of raw data. The data source includes obtaining data from various text data sources, such as Wikipedia, news articles, books, code libraries, etc. The data format includes word, pdf, csv, etc., and convert the data into a unified text format, such as JSON, etc. Finally, clean the data, such as removing noise data, erroneous data, duplicate data, etc., to ensure data quality.

[0133] Step 2, generate chunking strategy: Analyze data characteristics, such as length, content complexity, and structured degree, and generate chunking strategies and define chunking standards based on the chunking strategies already accumulated in the strategy library. Consider the balance between retrieval performance, memory usage, and computational efficiency when formulating strategies.

[0134] Commonly used strategies in the strategy library include but are not limited to the following:

[0135] Fixed-length segmentation: directly divide the text into multiple blocks according to a fixed number of characters or words. The block size k is determined by the segmentation strategy engine based on the characteristics of the data.

[0136] Sentence-based segmentation: In this method, Chunks are segmented based on sentence boundaries, that is, natural language processing tools are used to identify the end of sentences, such as punctuation marks such as periods and question marks, and then the text is divided into multiple parts based on sentence boundaries.

[0137] Sliding Window: In this approach, the creation of chunks involves an overlapping sliding window. For example, the window size can be set to 500 words and the step size can be set to 100 words. The window size and step size are generated and adjusted in real time by the chunking strategy engine.

[0138] Topic or paragraph-based segmentation: This advanced segmentation strategy relies on the intrinsic structure of the text, such as paragraphs or topic paragraphs. Chunks are segmented by identifying the topic change points in paragraphs or texts, which can maintain a high degree of semantic coherence and is suitable for well-structured texts, such as academic papers or formal reports.

[0139] Semantic similarity-based segmentation: Use machine learning models to evaluate the semantic similarity between sentences or paragraphs and segment them when the similarity drops below a certain threshold. This method attempts to optimize retrieval while maintaining a high degree of semantic coherence.

[0140] Step 3, data segmentation: Divide the original data into multiple data blocks according to the segmentation strategy, ensuring that each block maintains a certain context. Number or mark each data block for subsequent processing and indexing. Handle special cases such as text paragraphs, chapters, etc., and ensure that they are not segmented at logical boundaries.

[0141] Step 4, block relationship generation: Perform similarity analysis on data blocks, identify content relevance or topic similarity, and build a block relationship graph. Use natural language processing techniques (such as text embedding and similarity calculation) to evaluate the relationship between blocks. Generate a relationship matrix or graph structure to facilitate relationship search during subsequent retrieval.

[0142] Step 5, index construction: Select the Embedding model and call the model to vectorize the slice data generated in step 3.

[0143] Step 6, data persistence: The data generated in the above steps is locally persisted for subsequent retrieval and reading, where vector data is stored in a vector database, and text data and slice relationships are stored in the corresponding database.

[0144] Step 7, retrieval: Convert the user's query statement into a vector representation, and retrieve the most relevant text fragments in the vector database based on the similarity between the query vector and the index vector. Retrieval strategies include: nearest neighbor search, range search, etc. The nearest neighbor search is to find the k vectors closest to the query vector; the range search is to find all vectors within a certain range of distance from the query vector.

[0145] Step 8, reranking: Sort the search results according to their similarity scores, with the most relevant results at the front. Sorting algorithms include BM25, TF-IDF, etc. Sorting can also be based on reranking models: such as bge-reranker-base and bge-reranker-large, which are deep learning models trained by machine learning algorithms and learn how to rerank search results through training data.

[0146] Step 9, expansion strategy generation: Based on the user's query context and retrieval results, the expansion strategy engine performs calculations to formulate information expansion strategies.

[0147] The extended strategy engine queries the database to find information blocks that are closely related to the target information block. It first searches for the parent-child relationship to find the parent information block of the target information block and expands it based on the content of the parent information block. It then searches for contextual information blocks and expands them based on the information blocks adjacent to the target information block. Finally, it queries the information blocks associated with the target information block through the knowledge graph.

[0148] After obtaining the set of associated information blocks, sort all the information blocks by relevance, and select the first k information blocks for information expansion. The determination of the k value is very critical. How to choose a suitable k value directly affects the quality, relevance and response speed of the model-generated text. If the k value is too small, the model may not be able to obtain enough context information; if the k value is too large, it will exceed the maximum context length limit of the model, or introduce too much irrelevant information, reducing the generation efficiency. For the specific k value selection algorithm, see Algorithm 1. Generative model with variable number of associated information blocks k.

[0149] Step 10, information expansion: Apply expansion strategies to obtain supplementary information from external resources to enhance the search results. Screen and process the expanded information to ensure its relevance and accuracy. Combine the expanded information with the original search results to generate enhanced information.

[0150] Step 11, extended information merging: Merge the extended information with the original data block to form comprehensive context information. Process the redundancy of information, remove duplicate or irrelevant information, and ensure the high quality of the merged result. Format the merged information to ensure that it is easy to read and understand.

[0151] Step 12, outputting context extension information: outputting the finally generated context extension information for generating large model prompt words.

[0152] Step 13, evaluate output quality: evaluate the output result of step 10.

[0153] Step 14, updating the policy library: updating the relevant policies of the policy library according to the evaluation result of step 11.

[0154] The disclosed embodiment introduces the optimization iteration of the policy library and the block policy engine, so that the policy can be adjusted based on the real-time analysis of data characteristics, and has wide adaptability. The disclosed embodiment focuses on proposing a generation model of a variable number of associated information blocks k, and provides adaptive adjustment and optimization for factors such as different context corpus lengths, scenarios, slice sizes, large model maximum context lengths, slice semantic similarities, etc., and has strong adaptability and resilience.

[0155] RAG (Retrieval Augmented Generation): Retrieval enhanced generation is a technology that combines information retrieval and generative models. It allows a large language model to enhance the accuracy and comprehensiveness of its answers by retrieving relevant information. The disclosed embodiment is one of the methods for implementing large model RAG information retrieval. It expands the retrieved contextual information based on adaptive data segmentation and information expansion strategies, achieves good large model question and answer effects, and has broad application scenarios in intelligent customer service, content creation, personalized recommendations, knowledge questions and answers, education and training, etc.

[0156] The disclosed embodiments can solve the problem that in the large-model RAG retrieval process, the data segmentation and information expansion methods are fixed and single, and cannot be flexibly adjusted and adaptively configured according to different application scenarios and data characteristics. By solving this problem, the large-model answering effect is greatly improved.

[0157] In the embodiments of the present disclosure, the terms “first”, “second” and “third” are used for descriptive purposes only and should not be understood as indicating or implying relative importance.

[0158] The term "and / or" in this disclosure is only a description of the association relationship of associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0159] Furthermore, although the steps of the method in the present disclosure are described in a particular order in the drawings, this does not require or imply that the steps must be performed in this particular order or that all the steps shown must be performed to achieve the desired results.

[0160] In some embodiments, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0161] Based on the same inventive concept, the disclosed embodiment also provides a retrieval enhancement generation device for a large language model, as described in the following embodiment. Since the principle of solving the problem in the device embodiment is similar to that in the above method embodiment, the implementation of the device embodiment can refer to the implementation of the above method embodiment, and the repeated parts will not be repeated.

[0162] Figure 6 A schematic diagram of a retrieval enhancement generation device for a large language model in an embodiment of the present disclosure is shown. Figure 6 As shown, the retrieval enhancement generation device of the large language model includes a query vector conversion module 601, a target information retrieval module 602, a context determination module 603, an information expansion module 604 and a reply determination module 605.

[0163] A query vector conversion module 601, used to convert a user's query statement into a query vector;

[0164] The target information retrieval module 602 is used to retrieve multiple target information blocks from the vector database according to the similarity between the query vector and the index vector. The target information blocks are data fragments related to the query statement. The vector database stores vectors of knowledge data.

[0165] A context determination module 603 is used to determine the query context of the query statement by analyzing the user's query statement;

[0166] The information expansion module 604 is used to call the expansion strategy engine to determine the expansion strategy of multiple target information blocks according to the user's query context, and expand the multiple target information blocks based on the expansion strategy to obtain multiple associated information blocks related to each target information block;

[0167] The reply determination module 605 is used to determine a reply statement to the query statement based on multiple target information blocks and multiple associated information blocks.

[0168] In some embodiments, the retrieval enhancement generation device of the large language model further includes a database construction module.

[0169] A database construction module is used to obtain knowledge data from multiple data sources before retrieving multiple target information blocks in a vector database based on the similarity between a query vector and an index vector; analyze the characteristics of the acquired knowledge data, and call a block strategy engine to determine a data block strategy based on the characteristics of the knowledge data, wherein the characteristics of the knowledge data include one or more of data length, content complexity, and degree of structure; based on the data block strategy, divide the knowledge data into multiple information blocks, each of which retains context information; based on the context information retained by each information block, perform similarity analysis on multiple information blocks and construct a block relationship graph; call an embedding model to vectorize multiple information blocks and store them in a vector database.

[0170] In some embodiments, the reply determination module 605 is used to sort all associated information blocks based on the size of the correlation value between each associated information block and the corresponding target information block; select k associated information blocks from all associated information blocks based on the sorting; and determine the reply statement of the query statement based on multiple target information blocks and k associated information blocks.

[0171] In some embodiments, before selecting k associated information blocks with the largest relevance values ​​from all associated information blocks based on the sorting, the method also includes: calculating the value of k based on the corpus length of the user's query context, the scenario to which the query statement belongs, the information block size, the maximum context length of the large language model and the semantic similarity of the information blocks.

[0172] In some embodiments, the relationship between the corpus length of the user's query context, the maximum context length of the large language model, and the semantic similarity of the information block and the value of k is positively correlated;

[0173] The relationship between the information block size and the value of k is negatively correlated.

[0174] In some embodiments, the target information retrieval module 602 is used to retrieve multiple information blocks from the vector database based on the similarity between the query vector and the index vector; sort all information blocks based on the similarity value between the index vector and the query vector of each information block; and select multiple target information blocks from all information blocks based on the sorting.

[0175] In some embodiments, the information expansion module 604 expands multiple target information blocks based on the expansion strategy to obtain multiple associated information blocks related to each target information block, including one or more of the following methods:

[0176] Searching for the parent-child relationship, finding the parent information block of the target information block, and using the parent information block as the associated information block of the target information block;

[0177] and / or, searching for a context information block, and taking an information block adjacent to the target information block as an associated information block of the target information block;

[0178] And / or, query the associated information blocks of the target information block through the inter-block relationship graph.

[0179] The concepts of “first”, “second”, etc. mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0180] Regarding the retrieval enhancement generation device for a large language model in the above-mentioned embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the retrieval enhancement generation method for the large language model, and will not be elaborated here.

[0181] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, such division is not mandatory.

[0182] In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0183] Some of the blocks shown in the accompanying drawings are functional entities that do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0184] Refer to the following Figure 7 To describe the electronic device provided by the embodiment of the present disclosure. Figure 7 The electronic device 700 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0185] Figure 7 FIG. 1 is a schematic diagram showing the architecture of an electronic device 700 provided by an embodiment of the present disclosure. Figure 7 As shown, the electronic device 700 includes but is not limited to: at least one processor 710 and at least one memory 720.

[0186] The memory 720 is used to store instructions.

[0187] In some embodiments, the memory 720 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 7201 and / or a cache memory unit 7202 , and may further include a read-only memory unit (ROM) 7203 .

[0188] In some embodiments, the memory 720 may also include a program / utility 7204 having a set (at least one) of program modules 7205, such program modules 7205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0189] In some embodiments, the memory 720 may store an operating system, which may be a real-time operating system (RTX), LINUX, UNIX, WINDOWS, or OS X.

[0190] In some embodiments, data may also be stored in the memory 720 .

[0191] As an example, the processor 710 may read data stored in the memory 720 , where the data may be stored at the same storage address as the instruction, or the data may be stored at a different storage address than the instruction.

[0192] The processor 710 is used to call the instructions stored in the memory 720 to implement the steps of various exemplary embodiments of the present disclosure described in the above “Exemplary Method” section of this specification. For example, the processor 710 can execute each step of the above method embodiment.

[0193] It should be noted that the processor 710 may be a general-purpose processor or a special-purpose processor. The processor 710 may include one or more processing cores, and the processor 710 executes various functional applications and data processing by running instructions.

[0194] In some embodiments, the processor 710 may include a central processing unit (CPU) and / or a baseband processor.

[0195] In some embodiments, the processor 710 may determine an instruction according to the priority identifier and / or function category information carried in each control instruction.

[0196] In the present disclosure, the processor 710 and the memory 720 may be provided separately or integrated together.

[0197] As an example, the processor 710 and the memory 720 may be integrated on a single board or a system on chip (SOC).

[0198] like Figure 7 As shown, the electronic device 700 is in the form of a general-purpose computing device. The electronic device 700 may further include a bus 730 .

[0199] Bus 730 may be a bus representing one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus architectures.

[0200] The electronic device 700 may also communicate with one or more external devices 740 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), one or more devices that enable a user to interact with the electronic device 700, and / or any device that enables the electronic device 700 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed through an input / output (I / O) interface 750.

[0201] Furthermore, the electronic device 700 can also communicate with one or more networks (eg, a local area network (LAN), a wide area network (WAN) and / or a public network, such as the Internet) through the network adapter 760 .

[0202] like Figure 7 As shown, the network adapter 760 communicates with other modules of the electronic device 700 via the bus 730 .

[0203] It should be understood that although not shown in the figures, other hardware and / or software modules may be used in conjunction with the electronic device 700, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0204] It is to be understood that the structure shown in the embodiment of the present disclosure does not constitute a specific limitation on the electronic device 700. In other embodiments of the present disclosure, the electronic device 700 may include: Figure 7 More or fewer components may be shown, or some components may be combined or separated, or the components may be arranged differently. Figure 7 The components shown may be implemented in hardware, software or a combination of software and hardware.

[0205] The present disclosure also provides a computer-readable storage medium having computer instructions stored thereon. When the computer instructions are executed by a processor, the retrieval enhancement generation method of the large language model described in the above method embodiment is implemented.

[0206] The computer-readable storage medium in the embodiments of the present disclosure is a computer instruction that can be sent, propagated or transmitted for use by or in conjunction with an instruction execution system, apparatus or device.

[0207] As one example, computer readable storage media are non-volatile storage media.

[0208] In some embodiments, more specific examples of computer-readable storage media in the present disclosure may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, a USB flash drive, a mobile hard disk, or any suitable combination of the foregoing.

[0209] In the embodiments of the present disclosure, the computer-readable storage medium may include a data signal propagated in a baseband or as a part of a carrier wave, in which computer instructions (readable program codes) are carried.

[0210] Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing.

[0211] In some examples, computing instructions contained on a computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0212] The embodiments of the present disclosure also provide a computer program product, which stores instructions. When the instructions are executed by a computer, the computer implements the retrieval enhancement generation method of the large language model described in the above method embodiment.

[0213] The above instructions may be program codes. In specific implementation, the program codes may be written in any combination of one or more programming languages.

[0214] The programming language includes object-oriented programming languages, such as Java, C++, etc., and also includes conventional procedural programming languages, such as "C" language or similar programming languages.

[0215] The program code may execute entirely on the user's computing device, partly on the user's computing device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device or entirely on the remote computing device or server.

[0216] Where a remote computing device is involved, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0217] The embodiment of the present disclosure also provides a chip, including at least one processor and an interface;

[0218] An interface for providing program instructions or data to at least one processor;

[0219] At least one processor is used to execute program instructions to implement the retrieval enhancement generation method of the large language model described in the above method embodiment.

[0220] In some embodiments, the chip may further include a memory, which is used to store program instructions and data, and the memory is located inside or outside the processor.

[0221] Those skilled in the art will appreciate that all or part of the steps for implementing the above embodiments may be implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, which may be collectively referred to herein as "circuits", "modules" or "systems".

[0222] Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein.

[0223] The present disclosure is intended to cover any variation, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary technical means in the technical field not disclosed in the present disclosure. The description and examples are to be regarded as exemplary only, and the true scope and spirit of the present disclosure are indicated by the appended claims.

Claims

1. A retrieval enhancement generation method for a large language model, characterized in that: include: Convert the user's query statement into a query vector; According to the similarity between the query vector and the index vector, a plurality of target information blocks are retrieved from a vector database, wherein the target information blocks are data fragments related to the query statement, and the vector database stores vectors of knowledge data; By analyzing the query statement of the user, determining the query context of the query statement; Calling an extension strategy engine to determine an extension strategy for the multiple target information blocks according to the query context of the user, and extending the multiple target information blocks based on the extension strategy to obtain multiple associated information blocks related to each of the target information blocks; Based on the multiple target information blocks and the multiple associated information blocks, a reply sentence of the query sentence is determined.

2. The method according to claim 1, characterized in that Before retrieving a plurality of target information blocks from a vector database according to the similarity between the query vector and the index vector, the method further comprises: Acquire knowledge data from multiple data sources; Analyze the characteristics of the acquired knowledge data, and call a block strategy engine to determine a data block strategy based on the characteristics of the knowledge data, wherein the characteristics of the knowledge data include one or more of data length, content complexity, and structured degree; Based on the data segmentation strategy, the knowledge data is divided into a plurality of information blocks, each of which retains context information; Based on the context information retained by each of the information blocks, similarity analysis is performed on the multiple information blocks to construct an inter-block relationship graph; The embedding model is called to vectorize the multiple information blocks and store them in the vector database.

3. The method according to claim 1, characterized in that The step of determining a reply statement to the query statement based on the multiple target information blocks and the multiple associated information blocks includes: sorting all the associated information blocks based on the magnitude of the correlation value between each associated information block and the corresponding target information block; According to the ranking, k related information blocks are selected from all the related information blocks; Based on the multiple target information blocks and the k associated information blocks, a reply statement for the query statement is determined.

4. The method according to claim 3, characterized in that Before selecting k associated information blocks with the largest correlation values ​​from all associated information blocks according to the sorting, the method further includes: The value of k is calculated based on the corpus length of the user's query context, the scenario to which the query statement belongs, the size of the information block, the maximum context length of the large language model, and the semantic similarity of the information block.

5. The method according to claim 4, characterized in that The relationship between the corpus length of the user's query context, the maximum context length of the large language model and the semantic similarity of the information block and the value of k is positively correlated; The relationship between the information block size and the value of k is negatively correlated.

6. The method according to claim 1, characterized in that The retrieving a plurality of target information blocks from a vector database according to the similarity between the query vector and the index vector comprises: Retrieving a plurality of information blocks from a vector database according to the similarity between the query vector and the index vector; Sorting all information blocks based on the similarity value between the index vector of each information block and the query vector; According to the ranking, multiple target information blocks are selected from all the information blocks.

7. The method according to claim 1, characterized in that The step of expanding the multiple target information blocks based on the expansion strategy to obtain multiple associated information blocks related to each of the target information blocks includes one or more of the following methods: Searching for the parent-child relationship, finding the parent information block of the target information block, and using the parent information block as the associated information block of the target information block; and / or, searching for a context information block, and taking an information block adjacent to the target information block as an associated information block of the target information block; And / or, query the associated information blocks of the target information block through the inter-block relationship graph.

8. A retrieval enhancement generation device for a large language model, characterized in that: include: A query vector conversion module, used to convert the user's query statement into a query vector; A target information retrieval module, used to retrieve a plurality of target information blocks from a vector database according to the similarity between the query vector and the index vector, wherein the target information blocks are data fragments related to the query statement, and the vector database stores vectors of knowledge data; A context determination module, configured to determine a query context of the query statement by analyzing the query statement of the user; An information expansion module, configured to call an expansion strategy engine to determine expansion strategies for the plurality of target information blocks according to the query context of the user, and to expand the plurality of target information blocks based on the expansion strategy to obtain a plurality of associated information blocks related to each of the target information blocks; The reply determination module is used to determine a reply statement of the query statement based on the multiple target information blocks and the multiple associated information blocks.

9. An electronic device, characterized in that: include: A memory for storing instructions; A processor is used to call the instructions stored in the memory to implement the retrieval enhancement generation method of the large language model as described in any one of claims 1 to 7.

10. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the computer instructions are executed by a processor, the retrieval enhancement generation method of a large language model described in any one of claims 1 to 7 is implemented.

Images