RAG knowledge base construction method based on layout analysis and query generation

Abstract

The invention provides an RAG knowledge base construction method based on layout analysis and query generation. The method comprises the following steps: receiving a plurality of query documents; for each query document, performing layout analysis on the query document by utilizing a layout analysis tool; segmenting and merging the text content according to the layout analysis result to obtain a plurality of text segments; generating a title for the query document by using a large language model, and respectively generating a preset number of queries for each text segment; for each text segment, generating a first vector based on the title and the text segment, and querying and generating a second vector based on a preset number of the text segments; and merging and storing each text segment with the first vector and the second vector of the text segment to construct an RAG knowledge base. Therefore, the text segments are obtained by performing effective layout analysis on the query document, and the query is generated for each text segment to expand semantics, so that a more comprehensive RAG knowledge base is constructed, and the retrieval enhancement generation effect of the system can be improved.

Description

Technical Field

[0001] The present application relates to the field of information retrieval technology, and in particular to a method for constructing a RAG knowledge base based on layout analysis and query generation. Background Art

[0002] Retrieval-augmented generation (RAG) is an artificial intelligence technology that combines information retrieval and language generation models. The RAG technical workflow consists of two core stages: the retrieval stage and the generation stage. The retrieval stage is responsible for retrieving documents or information related to the query from the RAG knowledge base, while the generation stage uses this information to generate answers or content.

[0003] The RAG knowledge base stores a large amount of documents and information, which are used to support the retrieval phase of the RAG system. In the process of building the RAG knowledge base, the traditional processing is to parse the document, segment the parsed text by fixed length or punctuation, and then store the text segments and vector information in a vector database (such as FAISS, Milvus) or a search engine (such as Elasticsearch). Obtaining text segments in this way may cause semantically related texts to be separated.

[0004] In addition, in traditional word and vector-based retrieval, similarity is usually used to express relevance, and the similarity score is mathematically defined as the dot product or cosine value of the encoding vector of the user query and the document. However, sometimes this similarity score may not accurately reflect the semantic relevance. For example, user query 1 "What is the formula for the area of ​​a circle" and user query 2 "How to calculate the area of ​​a circle" are very close in similarity score, so the query result of user query 1 may be the same as the query result of user query 2. However, the target query result of user query 1 is more inclined to the document or content "The formula for calculating the area of ​​a circle is XXX" or "The definition and formula of a circle in geometry is XXX".

[0005] Therefore, more refined text segment acquisition methods and more effective semantic matching methods are very important for improving the retrieval enhancement generation effect. Summary of the invention

[0006] In order to solve the above problems, the present application proposes a RAG knowledge base construction method based on layout analysis and query generation, a computer-readable storage medium and an electronic device, which can improve the retrieval enhancement generation effect of the system.

[0007] In a first aspect, the present application provides a method for constructing a RAG knowledge base based on layout analysis and query generation, the method comprising: receiving a number of query documents; for each query document, performing layout analysis on the query document using a layout analysis tool to obtain a layout analysis result of the query document; the layout analysis result of the query document comprises a plurality of sections, and section analysis results of the plurality of sections, the section analysis results comprising text content, location information and section category; the layout analysis tool comprises at least three layout analysis models respectively trained using annotation data of manuals, papers and legal documents; segmenting and merging text content according to the layout analysis result of the query document to obtain a plurality of text sections; generating a title for the query document using a large language model, and generating a preset number of queries for each text section; for each text section, generating a first vector based on the title and the text section, and generating a second vector based on a preset number of queries for the text section; merging and storing each text section with its first vector and second vector respectively to construct a RAG knowledge base; the RAG knowledge base is used for processing user query operations by a retrieval enhancement generation system.

[0008] Therefore, by performing effective layout analysis on the query document to obtain text segments and generating queries for each text segment to expand the semantics, a more comprehensive RAG knowledge base can be constructed, which can improve the retrieval enhancement generation effect of the system.

[0009] In a possible implementation, the method also includes: setting document attributes for each query document; the document attributes include the document author, document creation date, and document modification date of the query document; using a machine learning model to generate keywords for each text segment; merging and storing each text segment with its first vector and second vector respectively to construct a RAG knowledge base, including: for each text segment, merging and storing it with the document attributes of the corresponding query document, the keywords of the text segment, the first vector and the second vector to construct a RAG knowledge base according to the storage units corresponding to each of the multiple text segments of the query document.

[0010] In one possible implementation, generating a first vector based on the title and the text segment includes: inputting the title and the text segment into a text embedding model respectively to generate a title vector and a text segment vector of preset dimensions; combining the title vector and the text segment vector to generate the first vector; and querying the preset quantity of the text segment to generate a second vector, including: inputting the preset quantity queries corresponding to the text segment into the text embedding model respectively to generate query vectors of preset dimensions corresponding to each of the preset quantity queries; and combining the query vectors of preset dimensions corresponding to the preset quantity queries to generate the second vector.

[0011] In a possible implementation, the calculation formula of the first vector includes:

[0012] V1=weight titie *Vtitle +(1-weight titie )*V chunk , where V1 is the first vector, V title and V chunk are the title vector and the text segment vector respectively, weiqht titie is the weight of the title vector in the first vector, 0≤weight titie ≤1, weight titie Preset.

[0013] In a possible implementation, the calculation formula of the second vector includes: Among them, V2 is the second vector, V i (1≤i≤n, i∈Z) is the i-th query vector of the text segment, and n is a preset number.

[0014] In a possible implementation, performing layout analysis on the query document using a layout analysis tool includes: selecting a corresponding layout analysis model from the layout analysis tool according to the document type of the query document; and performing layout analysis on the query document using the corresponding layout analysis model.

[0015] In one possible implementation, text content is segmented and merged based on the results of layout analysis of the query document, including: determining a corresponding segmentation strategy based on the text content and section category of each section, and segmenting the text content according to the segmentation strategy; based on the position information of multiple sections, merging text contents that are adjacent in position and logically belong to the same content.

[0016] In a possible implementation, before using the layout analysis tool to perform layout analysis on the query document, the method further includes: preprocessing the query document.

[0017] In a second aspect, the present application provides a retrieval enhancement generation processing method for user queries, the method comprising: receiving a user query; calling a RAG knowledge base as described in the first aspect or any possible implementation of the first aspect to perform data retrieval according to the user query, and obtaining a sorting result of storage units corresponding to multiple text segments; the data retrieval comprises comparing a query vector corresponding to the user query with a first vector and a second vector of each text segment; re-sorting the sorting result using a sorting model or a manually set sorting rule; and inputting the text segments and the user query in one or more storage units with the highest sorting into a large language model, so that the large language model outputs the query result of the user query under the guidance of a preset prompt word.

[0018] In a third aspect, the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed in a computer, the computer is caused to execute the first aspect or any possible implementation of the first aspect, or the method described in the second aspect.

[0019] In a fourth aspect, the present application provides an electronic device comprising: at least one memory for storing programs; and at least one processor for executing the programs stored in the memory; wherein, when the program stored in the memory is executed, the processor is used to execute the first aspect or any possible implementation of the first aspect, or the method described in the second aspect.

[0020] It can be understood that the beneficial effects of the second to fourth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0022] Figure 1 It is a schematic diagram of a RAG knowledge base construction process based on layout analysis and query generation provided in an embodiment of the present application;

[0023] Figure 2 It is a flow chart of a method for constructing a RAG knowledge base based on layout analysis and query generation provided in an embodiment of the present application;

[0024] Figure 3 It is a flow chart of a retrieval enhancement generation processing method for user queries provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0026] In the description of the embodiments of the present application, words such as "exemplary", "for example" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary", "for example" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary", "for example" or "for example" is intended to present related concepts in a concrete way.

[0027] In the description of the embodiments of the present application, the term "and / or" is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, B exists alone, and A and B exist at the same time. In addition, unless otherwise specified, the term "multiple" means two or more. For example, multiple systems refers to two or more systems, and multiple screen terminals refers to two or more screen terminals.

[0028] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. The terms "include", "comprises", "has" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.

[0029] In the process of building the RAG knowledge base, the traditional processing is to parse the document, segment the parsed text by fixed length or punctuation, and then store the text segments and vector information in a vector database (such as FAISS, Milvus) or a search engine (such as Elasticsearch). Obtaining text segments in this way may cause semantically related texts to be separated.

[0030] In addition, in traditional word and vector-based retrieval, similarity is usually used to express relevance. However, sometimes this similarity score may not accurately reflect the semantic relevance.

[0031] In view of this, the embodiment of the present application performs effective layout analysis on the query document through the trained layout analysis model, obtains multiple text segments based on the layout analysis results, and uses a large language model to generate queries for each text segment to expand the semantics. Vectors are generated based on each text segment and its corresponding query, thereby building a more comprehensive RAG knowledge base, which can improve the system's retrieval enhancement generation effect.

[0032] For example, Figure 1 A schematic diagram of a RAG knowledge base construction process based on layout analysis and query generation provided by an embodiment of the present application is shown. The RAG knowledge base construction process can be implemented by any device, apparatus, platform, server, device cluster, etc. with computing and processing capabilities.

[0033] like Figure 1 As shown in the figure, the construction process of the RAG knowledge base is as follows:

[0034] First, a set of query documents to be processed is received. The query documents can be in PDF, PPT, image and other formats.

[0035] Secondly, the query document is subjected to text recognition, including using the trained layout analysis tool to perform layout analysis on the query document to obtain the text blocks, images, tables and other sections in the document, as well as the text content, location information and section category of the sections. Then, based on the results of the layout analysis, the text content in the sections is segmented or merged to obtain meaningful text segments, such as text segment 1, text segment 2, text segment 3, etc.

[0036] Next, the large language model is used for semantic expansion, including generating titles for the query documents and generating a preset number of queries for each text segment. For each text segment, the generated title is combined with the text segment to create a first vector for the text segment; and the preset data queries for the text segment are combined to generate a second vector for the text segment. In addition, other representation information is expanded for each text segment, such as keywords of the text segment.

[0037] Finally, each text segment is merged and stored with its first vector, second vector and other representation information to construct the RAG knowledge base.

[0038] Therefore, by effectively analyzing the layout of the query document, multiple text segments can be obtained based on the layout analysis results. And by using the large language model to generate queries for each text segment, the semantic expression types of the text segment can be expanded and the semantic expression knowledge can be enriched to achieve more effective similarity calculation and semantic matching. This will ultimately improve the performance of the retrieval enhancement generation system, enabling the system to understand and respond to user queries more accurately.

[0039] Figure 2 The flowchart of the RAG knowledge base construction method based on layout analysis and query generation provided by the embodiment of the present application is shown. Figure 2 As shown, the method comprises the following steps:

[0040] Step S201, receiving a number of query documents.

[0041] In one embodiment, academic literature, research reports, legal documents, etc. are obtained as query documents by accessing professional databases related to the target field. The target field is the field in which the user query processed by the RAG system is located.

[0042] These query documents include electronic documents in various formats, such as editable formats such as PDF, Word, Excel, PPT, etc., or image formats such as JPEG, PNG, etc.

[0043] Exemplarily, the query document is preprocessed after being received.

[0044] Preprocessing includes denoising, contrast adjustment, rotation correction, cropping, resizing, and binarization, etc. Preprocessing the acquired query documents can improve the accuracy of subsequent processing steps.

[0045] Step S202: for each query document, a layout analysis tool is used to perform layout analysis on the query document to obtain a layout analysis result of the query document. The layout analysis result of the query document includes multiple sections and section analysis results of multiple sections, and the section analysis result includes text content, location information and section category. The layout analysis tool includes at least three layout analysis models trained using annotation data of manuals, papers and legal documents respectively.

[0046] In one embodiment, for each query document, the layout analysis tool uses the output of the preprocessing in step S201 as part of the input to further understand the structure and layout of the query document. This includes identifying and classifying different types of sections on the document page, such as text blocks, titles, tables, pictures, lists, etc. The layout analysis tool also outputs the text content of each section, as well as position information.

[0047] For example, after analyzing the query document through the layout analysis tool, the output is multiple data in the following format, each of which corresponds to a section on the document page and can be used as the section analysis result of the section. The section analysis result can be exemplarily expressed as:

[0048]

[0049] Among them, "x0", "x1", "top", and "bottom" are combined to represent the location information of the section, "text" represents the text content of the section, and "layout_type" represents the category of the section. "page_number" represents the document page information.

[0050] "layout_type" is an enumeration data structure that can include the following enumeration values:

[0051]

[0052]

[0053] For example, "Table" represents the table category, "Table caption" represents the table title category, etc., which will not be repeated here.

[0054] In one implementation, three layout analysis models trained using annotated data of manuals, papers, and legal documents are used to perform layout analysis on query documents.

[0055] The layout analysis model is trained with reference to the training method provided in PaddleOCR. During the training process, the layout characteristics of manuals, papers, and legal documents are used for data annotation, and the annotated data of each file is used to train three different layout analysis models. These layout analysis models are optimized for manuals, papers, and legal documents, respectively, and can recognize and understand the specific structure and terminology of documents in their respective fields.

[0056] For example, a legal model that is optimized through special training based on the characteristics of legal documents can identify and extract key information in legal texts, such as legal clauses, case citations, etc.

[0057] Next, a corresponding layout analysis model is selected from the layout analysis tool according to the document type of the query document, and the layout analysis of the query document is performed using the corresponding layout analysis model.

[0058] For example, if the document type of the query document is a legal document, a legal model is selected from the layout analysis tool to perform layout analysis on the query document to obtain a layout analysis result of the query document.

[0059] Step S203, segmenting and merging the text content according to the layout analysis result of the query document to obtain multiple text segments.

[0060] In one implementation, for each section, a corresponding segmentation strategy can be formulated for each section based on the text content and section category included in the layout analysis results. The text content of the section is segmented according to the segmentation strategy. The segmentation strategy includes treating the sections of the above-mentioned "Title" category as independent paragraphs; segmenting the sections of the above-mentioned "Text" category according to sentence structure or paragraph structure; treating each row or column of the above-mentioned "Table" category as a separate text segment, etc.

[0061] Based on the determined segmentation strategy, natural language processing (NLP) tools such as sentence segmentation models can be used to identify sentence boundaries or to segment the text content of a section based on punctuation marks (such as periods, commas) and text structure.

[0062] For each section, visually adjacent and logically related sections can also be identified based on the position information of multiple sections. For these adjacent sections, if they belong to the same content or context, their text content is merged into a larger text segment. The merged objects may include: paragraphs across pages, long texts that are incorrectly divided, adjacent list items or bullet points, etc.

[0063] Through this process, the text content in the query document can be organized into structured text segments, which can be used to build a RAG knowledge base, thereby improving the results of retrieval and generation tasks.

[0064] Step S204: Generate a title for the query document using the large language model, and generate a preset number of queries for each text segment.

[0065] In one implementation, the large language model may include an open source pre-trained model, such as GPT-3, LLaMA, Bloom, etc.

[0066] The query document can be input into the large language model, which can parse the content of the query document under the guidance of the prompt word Prompt, generate a title for the query document, or optimize and adjust the original title to obtain a new title. The title is a text that can directly express the content of the query document.

[0067] Special prompts can also be designed based on historical user searches, behavior patterns, contextual information or current popular query trends in the target field, so that the large language model can predict questions that users may want to ask or keywords that they may search for based on the text segments input separately, that is, a preset number of queries for each text segment.

[0068] Therefore, based on the text recognition of the query document to obtain multiple text segments, the large language model is used to generate titles for the query document and queries for each text segment, which can expand the semantic expression type of each text segment and enrich the semantic expression knowledge.

[0069] Step S205 : for each text segment, a first vector is generated based on the title and the text segment, and a second vector is generated based on a preset number query of the text segment.

[0070] In one implementation, for each text segment, the title and the text segment are respectively input into the text embedding model to generate a title vector and a text segment vector of preset dimensions. These vectors are used to represent the sentiment, semantics, theme and other characteristics of the model input text. The preset dimension is a pre-set value that represents the dimension of the output vector of the text embedding model, such as 512, 1024, etc.

[0071] Text embedding models include but are not limited to OpenAI's text embedding (openai-text embedding), bilingual general embedding (BGE), etc. These embedding models are trained based on a large amount of text data and can generate context-sensitive embedding representations to capture complex semantic relationships and language patterns.

[0072] Next, the title vector and the text segment vector are combined to generate the first vector. The calculation formula of the first vector includes:

[0073] V1=weight titie *V title +(1-weight titie )*V chunk (1)

[0074] Among them, V1 is the first vector, V title and V chunk are the title vector and the text segment vector, respectively, weight titie (0≤weight titie ≤1) is the weight of the title vector in the first vector, weight titie Preset.

[0075] Further, for each text segment, the corresponding preset number of queries are respectively input into the above text embedding model to generate the corresponding query vectors of the preset dimensions. The query vectors of the preset dimensions corresponding to the preset number of queries are combined to generate a second vector. The calculation formula of the second vector includes:

[0076]

[0077] Among them, V2 is the second vector, V i (1≤i≤n, i∈Z) is the i-th query vector of the text segment, and n is a preset number. The preset number is also preset.

[0078] Step S206: Each text segment is merged and stored with its first vector and second vector respectively to construct a RAG knowledge base. The RAG knowledge base is used by the retrieval enhancement generation system to process the user query operation.

[0079] In one implementation, document attributes are set for each query document, including the document author, document creation date, and document modification date of the query document.

[0080] Optionally, keywords are generated for each text segment. Keyword extraction involves automatically identifying words or phrases from the text that best represent the text's theme and content. This task can be achieved through traditional TF-IDF or TextRank deep learning methods, or through pre-trained language models based on Transformer structures (such as BERT, GPT, etc.).

[0081] Exemplarily, for each text segment, it is merged and stored with the document attribute of the corresponding query document, the keyword of the text segment, the first vector and the second vector to construct a RAG knowledge base according to the storage units corresponding to the multiple text segments of the query document.

[0082] Therefore, based on the text recognition of the query document to obtain multiple text segments, the present application uses a large language model to generate multiple predictive queries for each text segment, and generates a title for the query document. This can expand the semantic expression types of the query document and enrich the semantic expression knowledge to achieve more effective similarity calculation and semantic matching, thereby improving the retrieval efficiency and retrieval performance of the RAG system for user queries.

[0083] Based on the method in the above embodiment, illustratively, Figure 3 A flowchart of a method for generating enhanced retrieval of a user query provided by an embodiment of the present application is shown. Figure 3 As shown, the method comprises the following steps:

[0084] Step S301, receiving a user query.

[0085] Exemplarily, the user query is an input to the RAG system, which involves query matching based on it to provide relevant information or data. The RAG system can receive the user query of the user through the user interface, and the user query can be a text input type, such as a simple keyword, a complex sentence, or a specific inquiry.

[0086] Optionally, in order to improve the accuracy of retrieval, the RAG system may expand the user's original query before searching based on the user's query, such as enriching the query content by adding synonyms, related words or contextual information based on parsing the user's query intent.

[0087] Step S302: Call the following according to the user query: Figure 2 The RAG knowledge base performs data retrieval to obtain the sorting results of storage units corresponding to multiple text segments.

[0088] Exemplarily, the text embedding model in step S204 described above is used to generate a corresponding user query vector for the user query.

[0089] based on Figure 2 The text segment information stored in the RAG knowledge base is retrieved using a hybrid query method of word-based query and vector-based nearest neighbor algorithm (k-nearest neighbors, kNN) query. The hybrid query method can combine the advantages of word-based retrieval and vector-based retrieval to improve the accuracy and recall of retrieval. For example, you can use word-based retrieval to quickly narrow the search scope, and then use vector-based retrieval to accurately find the most relevant text segment.

[0090] The vector-based retrieval process includes comparing the query vector corresponding to the user query with the vectors in the RAG knowledge base. For example, by calculating the distance between the user query vector and the vectors of each text segment (including the first vector and the second vector), their similarity can be evaluated. Common distance metrics include Euclidean distance, cosine similarity, and Manhattan distance. According to the calculated distance, multiple text segments in the RAG knowledge base are sorted, and the vector with the closest distance (i.e., the most similar text segment) will be ranked first.

[0091] Ultimately, the hybrid query outputs a ranked list of text segments that can be used to generate answers, provide information retrieval services, or support other downstream applications.

[0092] Step S303: re-sort the sorting results using a sorting model or manually set sorting rules.

[0093] Exemplarily, the ranking results are re-ranked using a pre-trained ranking model or manually set rules to ensure that the most relevant text segments are ranked at the top. These top text segments are considered to be most likely to contain answers or relevant information to the user's query.

[0094] Step S304: input the text segments and the user query in the top-ranked one or several storage units into the large language model, so that the large language model outputs the query result of the user query under the guidance of the preset prompt words.

[0095] Exemplarily, the text segments and user queries in one or more storage units with the highest ranking are input into a large language model, which usually involves constructing an input context including the user query and related text segments, and using preset prompt words to guide the generation process of the model. These prompt words can help the model better understand the nature of the task and the user's intention.

[0096] The large language model generates query results based on the input context and prompt words. This may involve generating an answer, a summary, a continuation of text, or any other form of output, depending on the design of the RAG system and the task requirements.

[0097] Furthermore, the output generated by the large language model can be post-processed to improve the quality of the results. Post-processing includes grammar checking, answer formatting, and removal of irrelevant information.

[0098] Finally, the processed query results will be presented to the user as the RAG system's response to the user's query.

[0099] It should be noted that although the operations of the method of the embodiment of the present application are described in a specific order in the above embodiment, this does not require or imply that these operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired result. On the contrary, the steps depicted in the flow chart can change the order of execution. Additionally or alternatively, some steps can be omitted, multiple steps can be combined into one step for execution, and / or one step can be decomposed into multiple steps for execution.

[0100] Based on the method in the above embodiment, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a processor, the processor executes the method in the above embodiment.

[0101] Based on the method in the above embodiment, an embodiment of the present application provides an electronic device. The electronic device may include: at least one memory for storing programs; and at least one processor for executing the programs stored in the memory. When the programs stored in the memory are executed, the processor is used to execute the method described in the above embodiment.

[0102] Exemplarily, the electronic device may be a mobile phone, a tablet computer, a desktop computer, a laptop computer, a handheld computer, a notebook computer, a server, an ultra-mobile personal computer (UMPC), a netbook, a cellular phone, a personal digital assistant (PDA), an augmented reality (AR) device, a virtual reality (VR) device, an artificial intelligence (AI) device, a wearable device, an in-vehicle device, a smart home device and / or a smart city device. The embodiments of the present application do not impose any special restrictions on the specific category of the electronic device.

[0103] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state disk (SSD)), etc.

[0104] It is understood that the various numerical numbers involved in the embodiments of the present application are only for the convenience of description and are not used to limit the scope of the embodiments of the present application. It should be understood that in the embodiments of the present application, the size of the sequence number of the above-mentioned processes does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0105] The specific implementation methods described above further illustrate the purpose, technical solutions and beneficial effects of the present application in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present application. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solutions of the present application should be included in the scope of protection of the present application.

Claims

1. A method for constructing a RAG knowledge base based on layout analysis and query generation, characterized in that: The method comprises: receiving a number of query documents; For each query document, a layout analysis tool is used to perform layout analysis on the query document to obtain a layout analysis result of the query document; the layout analysis result of the query document includes multiple sections and section analysis results of the multiple sections, and the section analysis result includes text content, location information and section category; the layout analysis tool includes at least three layout analysis models trained using annotation data of manuals, papers and legal documents respectively; Segmenting and merging the text content according to the layout analysis result of the query document to obtain multiple text segments; Generate titles for query documents using a large language model, and generate a preset number of queries for each text segment; For each text segment, a first vector is generated based on the title and the text segment, and a second vector is generated based on a preset number query of the text segment; Each text segment is merged and stored with its first vector and second vector respectively to construct the RAG knowledge base; the RAG knowledge base is used for the retrieval enhancement generation system to process the user query operation.

2. The method according to claim 1, characterized in that The method further comprises: Setting document attributes for each query document; the document attributes include the document author, document creation date, and document modification date of the query document; Use machine learning models to generate keywords for each text segment; The step of merging and storing each text segment with its first vector and second vector to construct the RAG knowledge base includes: For each text segment, it is combined and stored with the document attribute of the corresponding query document, the keyword of the text segment, the first vector and the second vector, so as to construct the RAG knowledge base according to the storage units corresponding to the multiple text segments of the query document.

3. The method according to claim 1, characterized in that The step of generating a first vector based on the title and the text segment includes: Inputting the title and the text segment into a text embedding model respectively to generate a title vector and a text segment vector of a preset dimension; Combining the title vector with the text segment vector to generate a first vector; The generating a second vector based on a preset number of queries for the text segment includes: Inputting the preset number of queries corresponding to the text segment into the text embedding model respectively to generate the query vectors of the preset dimensions corresponding to each of them; The query vectors of the preset dimension corresponding to each of the preset number of queries are combined to generate a second vector.

4. The method according to claim 3, characterized in that The calculation formula of the first vector is include: V1=weight titie *V title +(1-weight titie )*V chunk , Among them, V1 is the first vector, V title and V chunk are the title vector and the text segment vector, respectively, weight titie is the weight of the title vector in the first vector, 0≤weight titie ≤1, weight titie Preset.

5. The method according to claim 3, characterized in that: The calculation formula of the second vector includes: Among them, V2 is the second vector, V i (1≤i≤n, i∈Z) is the i-th query vector of the text segment, and n is the preset number.

6. The method according to claim 1, characterized in that The use of a layout analysis tool to perform layout analysis on the query document includes: Selecting a corresponding layout analysis model from the layout analysis tool according to the document type of the query document; The corresponding layout analysis model is used to perform layout analysis on the query document.

7. The method according to claim 1, characterized in that The segmenting and merging of the text content according to the layout analysis result of the query document includes: Determine the corresponding segmentation strategy according to the text content and the category of each section, and segment the text content according to the segmentation strategy: Based on the location information of multiple sections, text contents that are adjacent in location and logically belong to the same content are merged.

8. The method according to claim 1, characterized in that Before using the layout analysis tool to analyze the query document, the method further includes: The query document is preprocessed.

9. A method for generating and processing retrieval enhancement for user queries, characterized in that: The method comprises: Receive user inquiries; According to the user query, the RAG knowledge base according to any one of claims 1 to 8 is called to perform data retrieval to obtain a sorting result of storage units corresponding to multiple text segments; the data retrieval includes comparing the query vector corresponding to the user query with the first vector and the second vector of each text segment; Reorder the sorting results using a sorting model or manually set sorting rules; The text segments in one or more storage units with the highest ranking and the user query are input into the large language model, so that the large language model outputs the query result of the user query under the guidance of the preset prompt word.

10. An electronic device, characterized in that: include: at least one memory for storing a program; At least one processor, configured to execute the program stored in the memory; wherein, when the program stored in the memory is executed, the processor is configured to execute the method according to any one of claims 1 to 9.

Images