Document knowledge intelligent retrieval method and system based on large model and RAG technology

Abstract

The invention discloses an intelligent document knowledge retrieval method and system based on a large model and an RAG technology, and relates to the technical field of artificial intelligence. The method comprises the steps that a document knowledge base is constructed, and documents are converted into high-dimensional vectors through a vectorization technology and stored; training a large model in the field based on document knowledge, and improving the specialty of the model by adopting prompt learning and fine tuning technologies; receiving a user query, analyzing a user intention through a natural language processing technology and extracting key elements; in combination with the RAG technology, vector retrieval and keyword retrieval are fused, relevant information is retrieved from a document knowledge base, and accurate and coherent answers are generated. According to the method, the problems that a traditional retrieval method is insufficient in semantic understanding and low in retrieval result correlation are effectively solved, and efficient and accurate retrieval and intelligent generation of massive documents are achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence, and in particular to a document knowledge intelligent retrieval method and system based on a large model and RAG technology. BACKGROUND

[0002] With the rapid development of information technology, the number of documents accumulated by various organizations grows exponentially, covering enterprise reports, technical documents, laws and regulations, academic papers and other types. Traditional document retrieval methods are mainly based on keyword matching, such as using search engines such as ElasticSearch to implement full-text retrieval. Although this method has certain efficiency in specific scenarios, its retrieval effect is seriously dependent on the accuracy and integrity of the keywords, and it is difficult to understand the deep semantic intent of user queries, resulting in insufficient relevance and accuracy of the retrieval results. Especially in a large number of unstructured documents, traditional methods are prone to miss important information and cannot meet the user's demand for efficient and accurate retrieval.

[0003] In recent years, large language models (LLM) have shown strong text understanding and generation capabilities in the field of natural language processing, and can to some extent realize semantic-level question answering and reasoning. However, large models themselves have problems such as knowledge lag, hallucination generation, and insufficient understanding of domain professional knowledge, and when directly applied to document knowledge retrieval, it is often difficult to guarantee the accuracy and professionalism of the information.

[0004] To make up for this deficiency, retrieval-augmented generation (RAG) technology is proposed, which combines external knowledge bases with large model generation capabilities to improve the reliability of the question answering system. Existing RAG-based methods usually use vectorization retrieval methods to convert documents into high-dimensional vectors and perform similarity matching. However, this method still has the following problems in practical application: first, a single vector retrieval model (such as Dense Passage Retrieval) may miss important documents in the preliminary recall stage; second, there is a lack of effective re-ranking mechanism, resulting in low relevance and accuracy of the retrieval results; third, it fails to fully combine the advantages of traditional keyword retrieval and semantic retrieval, making it difficult to achieve multi-dimensional and deep-level document understanding and knowledge mining.

[0005] Therefore, the existing technology still faces problems such as slow retrieval speed, low accuracy, and shallow semantic understanding when dealing with massive, multi-source, and heterogeneous documents, and there is an urgent need for a document knowledge intelligent retrieval method that can integrate multiple advanced artificial intelligence technologies and achieve efficient, accurate, and interpretable retrieval. SUMMARY

[0006] To this end, the embodiment of the present application provides a document knowledge intelligent retrieval method and system based on a large model and RAG technology, to solve the problems of low relevance, insufficient accuracy, lack of deep semantic understanding of retrieval results in the prior art, and ineffective fusion of traditional keyword retrieval and vector retrieval.

[0007] To solve the above technical problems, the embodiment of the present application provides a document knowledge intelligent retrieval method based on a large model and RAG technology, which comprises the following steps: Step S1: constructing a document knowledge base, including collecting, cleaning, preprocessing and vectorizing the document knowledge, generating vector representation and storing it into a vector database; Step S2: constructing a document knowledge domain large model, automatically generating text corpus, training the large language model by using prompt learning and fine-tuning technology, and forming a professional model suitable for the document knowledge domain; Step S3: receiving a document knowledge query request of a user, analyzing user demand by natural language processing technology, and extracting key elements; Step S4: based on the key elements, using the document knowledge domain large model and RAG technology to retrieve relevant information from the document knowledge base, and generating an answer using the retrieved relevant information.

[0008] Preferably, the method of constructing a document knowledge base specifically comprises: Step S11: collecting and cleaning document knowledge, removing noise and standardizing format; Step S12: preprocessing the cleaned document knowledge, including text segmentation, part-of-speech tagging and syntax analysis; Step S13: using vectorization technology to convert the preprocessed data into high-dimensional vector representation to capture text semantic relationship; Step S14: constructing a document knowledge base based on the vector representation, and storing the vector data into a vector database.

[0009] Preferably, the vectorization technology uses a word embedding model, including Word2Vec or Doc2Vec, and the vector database includes Milvus or Chroma.

[0010] Preferably, the method of constructing a document knowledge domain large model specifically comprises: Step S21: automatically generating text corpus based on document knowledge as training samples; Step S22: using prompt learning technology, designing targeted prompts to guide the large language model to understand the semantics and context of the document knowledge; Step S23: through fine-tuning technology, iteratively training on labeled document knowledge to optimize model parameters; Step S24: periodically evaluate the model to form a specialized document knowledge field large model.

[0011] Preferably, the method of receiving a user's document knowledge query request, analyzing user needs through natural language processing technology, and extracting key elements specifically includes: Step S31: performing word segmentation, part-of-speech tagging, and syntax analysis on the user's query request through natural language processing technology; Step S32: extracting key elements in the query request, including query topic, time range, and keywords; Step S33: capturing user query intent to support retrieval and generation tasks.

[0012] Preferably, the step S4 of retrieving relevant information includes: Vectorizing the user query and performing similarity retrieval in the vector database to obtain a first candidate document set; Performing keyword retrieval in the search engine to obtain a second candidate document set; De-duplicating and cross-merging the first candidate document set and the second candidate document set to form a fusion document set.

[0013] Preferably, the step S4 of generating an answer includes: Combining the fusion document set with the user query into a prompt text; Inputting the prompt text into the document knowledge field large model to generate an initial answer; Performing text summarization and sentence merging post-processing on the initial answer to improve readability and accuracy.

[0014] The embodiment of the application also provides a document knowledge intelligent retrieval system based on a large model and RAG technology, which is used to implement the above-mentioned document knowledge intelligent retrieval method based on a large model and RAG technology, and specifically includes: A document knowledge base construction module for constructing a document knowledge base, including collecting, cleaning, preprocessing, and vectorizing document knowledge, generating vector representations and storing them in a vector database; A domain large model training module for constructing a document knowledge domain large model, automatically generating text corpus, training a large language model using prompt learning and fine-tuning techniques, and forming a professional model adapted to the document knowledge domain; A user query understanding module for receiving a user's document knowledge query request, analyzing user needs through natural language processing technology, and extracting key elements; An intelligent retrieval and generation module for retrieving relevant information from the document knowledge base based on the key elements using the document knowledge domain large model and RAG technology, and generating an answer using the retrieved relevant information.

[0015] The embodiment of the present application also provides an electronic device, which comprises a processor, a memory and a bus system, the processor and the memory are connected through the bus system, the memory is used for storing instructions, and the processor is used for executing the instructions stored in the memory to realize the document knowledge intelligent retrieval method based on the large model and the RAG technology.

[0016] The embodiment of the present application also provides a computer storage medium, which stores a computer software product, the computer software product comprises a plurality of instructions, and is used for enabling a computer device to execute the document knowledge intelligent retrieval method based on the large model and the RAG technology.

[0017] From the above technical solutions, the present application has the following beneficial effects: (1) The present application effectively overcomes the limitations of traditional keyword retrieval by fusing the dual-path recall mechanism of vector retrieval and keyword retrieval, and combining the deep semantic understanding ability of the field large model, and significantly improves the retrieval accuracy and semantic matching degree under complex queries.

[0018] (2) With the help of the field-adaptive large model and the RAG technology, the system can generate professional and accurate answers based on the retrieved structured knowledge, effectively alleviate the illusion problem and knowledge lag of the large model itself, and improve the credibility of the output content.

[0019] (3) By constructing an updatable vectorized document knowledge base and integrating natural language understanding and intelligent generation processes, the system supports efficient organization and retrieval of massive multi-source documents, and provides interactive experience with context awareness, greatly improving the efficiency and convenience of users to obtain knowledge. BRIEF DESCRIPTION OF DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly described below, and the features and advantages of the present application will be more clearly understood by referring to the drawings. The drawings are schematic and should not be understood as any limitation on the present application. For those skilled in the art, other drawings can be obtained without creative labor on the basis of these drawings. Among them: Figure 1 is a flowchart of a document knowledge intelligent retrieval method based on a large model and a RAG technology provided by the present application; Figure 2 is a flowchart of a document knowledge base construction method provided by the present application; Figure 3 is a flowchart of a method for constructing a document knowledge field large model in the present application; Figure 4 is a method flow diagram for receiving a user's document knowledge query request, analyzing user needs, and extracting key elements in the application; Figure 5 is a block diagram of a document knowledge intelligent retrieval system based on a large model and RAG technology provided by the application. DETAILED DESCRIPTION

[0021] To make the purpose, technical solution and advantages of the embodiments of the application clearer, the technical solutions in the embodiments of the application will be described clearly and completely below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are part of the embodiments of the application, not all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of the application.

[0022] To solve the problems of low relevance, insufficient accuracy, lack of deep semantic understanding of retrieval results, and ineffective fusion of traditional keyword retrieval and vector retrieval in the prior art, as shown in Figure 1 , the application proposes a document knowledge intelligent retrieval method based on a large model and RAG technology, which includes the following steps: Step S1: Constructing a document knowledge base, including collecting, cleaning, preprocessing and vectorizing the document knowledge, generating vector representation and storing it into a vector database; Step S2: Constructing a document knowledge domain large model, automatically generating text corpus, training the large language model by using prompt learning and fine-tuning technology, and forming a professional model suitable for the document knowledge domain; Step S3: Receiving a user's document knowledge query request, analyzing user needs by natural language processing technology, and extracting key elements; Step S4: Based on the key elements, using the document knowledge domain large model and RAG technology to retrieve relevant information from the document knowledge base, and generating answers using the retrieved relevant information.

[0023] From the above technical solution, the present application proposes a document knowledge intelligent retrieval method based on large model and RAG technology, which realizes efficient and accurate retrieval of massive document knowledge through four core steps of constructing document knowledge base, training domain large model, understanding user query and intelligent retrieval generation. Among them, step S1 constructs a professional document knowledge base through vectorization technology, laying a foundation for efficient retrieval; step S2 adopts prompt learning and fine-tuning technology to train the domain large model, which significantly improves the understanding and generation ability of the model in the professional field; step S3 deeply analyzes the user query intention through natural language processing technology to ensure the accuracy of the retrieval; step S4 innovatively combines vector retrieval and keyword retrieval, and generates accurate and coherent answers combined with RAG technology, effectively solving the problems of insufficient semantic understanding and low relevance of retrieval results in traditional retrieval methods.

[0024] In step S1, as shown in Figure 2 The construction method of the document knowledge base specifically includes the following steps: Step S11: Collect and clean the document knowledge, remove noise and standardize the format. Specifically, first, collect electronic document or scanned document knowledge from multiple reliable sources such as historical documents, government documents, academic materials, etc. Then, use advanced data cleaning techniques such as regular expression matching and deduplication algorithm to remove noise data such as format errors, duplicates, irrelevant information, etc. Finally, standardize the data format to ensure that all data comply with a unified coding and structure.

[0025] Step S12: Preprocess the cleaned document knowledge, including text segmentation, part-of-speech tagging and syntax analysis. Specifically, the cleaned document knowledge is preprocessed by natural language processing (NLP), including text segmentation, part-of-speech tagging and syntax analysis. Chinese word segmentation technology is used to cut continuous text into meaningful word or phrase units; then part-of-speech tagging is performed to assign each word a grammatical role such as noun or verb; in addition, syntax analysis is used to analyze sentence structure and identify subject, predicate, object and other components. These preprocessing steps help the machine to deeply understand the text content and provide necessary information for subsequent vectorization processing.

[0026] Step S13: Use vectorization technology to convert preprocessed data into high-dimensional vector representation to capture semantic relationships between texts. Specifically, vectorization technology is used to convert preprocessed text data into vectors in high-dimensional space to accurately capture semantic relationships between texts. Based on the preprocessed document knowledge data, advanced word embedding models such as Word2Vec and Doc2Vec are selected to learn the semantic representation of words by training a large amount of text data, converting high-dimensional text data into low-dimensional continuous vectors, reducing data dimensionality and improving computational efficiency.

[0027] Step S14: Construct a document knowledge base based on the vector representation and store the vector data into the vector database. Specifically, based on the vectorized representation of the document knowledge, a comprehensive and professional document knowledge base is constructed to support efficient retrieval and generation tasks. In the vector database (such as Milvus or Chroma), a document knowledge index is established according to the vector features, and the vectorized data is stored in the vector database. Users can quickly retrieve the required information through keywords, topics, etc., providing strong support for scientific research, historical research, etc.

[0028] In step S2, as shown in Figure 3 The method for constructing a document knowledge field large model includes the following steps: Step S21: Automatically generate text corpus based on document knowledge as training samples. Specifically, a large amount of text corpus is automatically generated as training samples based on document knowledge; the text corpus automatically generated based on document knowledge is divided into training set, validation set and test set for training, validation and testing of the large language model. At the same time, in order to improve the diversity and richness of the corpus, data enhancement techniques such as text replacement and sentence reorganization are used to make the corpus have the ability to deeply understand the text in the document knowledge field.

[0029] Among them, the training set is used to guide the model to learn the mapping relationship between input and output, and through continuous optimization of model parameters, the prediction accuracy and generalization ability of the model on unknown data are improved; the test set is used to evaluate the final performance of the model; the validation set is used to optimize the model hyperparameters, and then evaluate the model performance.

[0030] Step S22: Use prompt learning technology to design targeted prompts to guide the large language model to understand the semantics and context of document knowledge. Specifically, using prompt learning technology, design targeted prompts (Prompt) such as in the form of questions or fill-in-the-blank, guide the large language model to deeply understand the semantics and context of document knowledge. The selected corpus segment is used as Prompt, and the automatic Prompt optimization method of P-tuningv2 is used to train the language large model based on the large model training data set, optimize the trainable continuous Prompt, and obtain the best prompt vector.

[0031] It should be noted that the prompt of the large language model is the input text provided by the user to the model, which is used to generate the output text of the model; in the natural language generation task, the prompt is usually a question, a topic or a partial sentence, and the model will generate the corresponding answer or complete the sentence based on this.

[0032] Further, in the embodiment of the present application, the step S22 specifically comprises: generating an optimal prompt vector according to the corpus segment by using an artificial design method or an automatic prompt optimization method of P-tuningv2. The artificial design method generates an optimal prompt vector according to the corpus segment and requirements. The automatic prompt optimization method (P-tuningv2) performs preference learning training on the language large model based on a large model training data set by using the P-tuningv2 technology and taking the corpus segment as a prompt, and optimizes the trainable discrete and continuous prompts, thereby obtaining the optimal prompt vector. The P-tuningv2 is improved on the basis of P-tuning, and is not only operated at the input layer, but also adds a prompt token at each layer of the language model, thereby providing more trainable learning parameters for the preference training of the prompt fine-tuning-based large model and bringing more direct and significant effect feedback.

[0033] Step S23: iteratively training on the marked document knowledge by using a fine-tuning technology to optimize the model parameters. Specifically, the model parameters are iteratively trained on the marked document knowledge by using the fine-tuning technology to optimize the model parameters, so as to adapt to the specific requirements of the document knowledge field. After the automatic P-tuningv2 method is used to pre-warn the large model, the trainable continuous prompt is optimized, and the optimal prompt vector is obtained, the LoRA (Low-Rank Adaptation, low-rank adaptation) fine-tuning method is used to fine-tune the large language model, the model parameters are added to the large language model, and finally the document knowledge field large model meeting the user requirements is obtained. LoRA is a low-rank adaptation efficient fine-tuning method, and the purpose is to improve the fine-tuning efficiency of the large language model.

[0034] Further, in the embodiment of the present application, the step S23 specifically comprises: first fine-tuning the optimal prompt by using the P-tuning method, and then performing model parameter fine-tuning by using LoRA. After adding the prompt and performing efficient LoRA fine-tuning on the model, the large model does not need to perform full parameter optimization on the document knowledge retrieval in different task scenarios, but only needs to adjust the intrinsic dimension of the large model, so as to realize the rapid improvement of the model performance under the condition of limited resources; at the same time, the document information can be more intelligently and individually provided in the actual application scene, and the user decision is assisted.

[0035] Step S24: Periodically evaluate the model to form a specialized document knowledge field large model. Specifically, periodically evaluate the model to ensure that a highly specialized document knowledge field large model is ultimately formed, which has the professional knowledge and skills in the field. The evaluation of the large language model can be achieved through common sense reasoning, closed-book question answering, reading comprehension, mathematical reasoning, and code generation; in addition, artificial review will be carried out regularly, and the generated text will be evaluated by artificial reviewers to obtain more accurate feedback and improvement direction, and then the document knowledge field large model is formed and continuously optimized.

[0036] In step S3, as shown in Figure 4 the document knowledge query request of the user is received, the user demand is analyzed by natural language processing technology, and the key elements are extracted. The method includes the following steps: Step S31: Perform word segmentation, part-of-speech tagging, and syntax analysis on the user's query request through natural language processing technology. Specifically, use natural language processing (NLP) technology to process the user's query request, including word segmentation, part-of-speech tagging, and syntax analysis (the processing logic is consistent with the document processing in the document knowledge base). Specifically, use NLP technology to perform word segmentation on the user's input Query, cutting continuous text into meaningful words or phrase units; then perform part-of-speech tagging to assign each word a grammatical role such as noun or verb; in addition, syntax analysis helps the machine parse sentence structure and identify subject, predicate, object, and other components to help the machine understand the text content in depth and provide necessary information for subsequent vectorization processing.

[0037] Step S32: Extract the key elements in the query request, including the query topic, time range, and keywords. Specifically, extract the key elements in the query request, such as the query topic, time range, and keywords. Use keyword extraction algorithms, topic models, and other techniques to extract key elements related to document content from the query request; at the same time, use vector models to convert user input questions into a form that computers can understand to obtain higher-level semantic information.

[0038] Step S33: Capture the user's query intent to support retrieval and generation tasks. Specifically, capture the user's query intent to provide accurate information support for subsequent retrieval and generation tasks. After deeply analyzing the user's query request, focus on capturing and deeply understanding the user's query intent, which not only covers the user's surface search demand for document knowledge, but also includes their potential information needs, research purposes, or points of interest. Specifically, use advanced natural language understanding technology, combined with context analysis, user historical query data, and document knowledge field professional knowledge, to accurately capture the deep intent behind the user's query.

[0039] In step S4, based on the key elements, relevant information is retrieved from the document knowledge base using the document knowledge domain large model and RAG technology, and an answer is generated using the retrieved relevant information.

[0040] This step integrates RAG technology with document knowledge domain large model, realizes multi-dimensional retrieval and high-quality answer generation, and incorporates recall optimization and data post-processing links, as follows: Step S41: Multi-source retrieval and candidate document set acquisition Vector similarity retrieval: input the user query vector QE1 into the Milvus or Chroma vector database, use IVF_FLAT, HNSW, etc. index algorithm, calculate the similarity of QE1 and all vectors in the database through Euclidean distance, cosine similarity or Jaccard similarity, etc. metric method, retrieve the K most similar text passages, and form the first candidate document set PK1.

[0041] Keyword retrieval: input the user query keywords into the ElasticSearch search engine, and get the K text passages with the highest matching degree through the full-text retrieval algorithm, and form the second candidate document set PK2.

[0042] Recall optimization: the first round of recall is performed through the preliminary vector recall model (Dense Passage Retrieval), and the second round of sorting is performed on the candidate documents using the rerank model to improve the relevance of the retrieval results.

[0043] Step S42: Candidate document set fusion processing Use the deduplication algorithm to remove duplicate passages in PK1 and PK2, integrate the two document sets through cross-merge strategy, form the fusion document set Mix-PK covering semantic matching and keyword matching, and ensure the comprehensiveness and uniqueness of the retrieval results.

[0044] Step S43: Answer generation and optimization Prompt text construction: use the fusion document set Mix-PK as the reference knowledge base, and combine it with the user's original query to form the prompt text Prompt1, ensuring that the prompt text contains both user requirements and sufficient knowledge support.

[0045] Initial answer generation: input Prompt1 into the document knowledge domain large model, and the model generates a coherent initial answer based on the reference knowledge base and domain professional knowledge. For complex queries, the model is decomposed into multiple sub-questions, and the RAG technology is used to answer them in parallel and then integrate the results.

[0046] Post-processing optimization: Text summarization algorithms are used to extract the core information of the initial answer, sentence merging technology is used to optimize the expression logic, remove redundant content, and improve the readability and accuracy of the answer.

[0047] Furthermore, this invention utilizes a large-scale document knowledge domain model and RAG technology to retrieve relevant information from massive search results based on user needs, and then generates document-based answers. The specific process is as follows: After the user inputs question Q1, a vector QE1 is first generated using a vector embedding model. Vector QE1 is then input into a vector database to retrieve the K semantically most similar text segment groups PK1. Simultaneously, question Q1 is input into an Elasticsearch search engine to retrieve the K highest-scoring text segment groups PK2. PK1 and PK2 are deduplicated and cross-merged to form a new text segment group Mix-PK (e.g., PK1-1, PK2-1, PK1-2, PK2-2… where PK1-1 represents the first text segment of group PK1, PK2-1 represents the first text segment of group PK2, and so on). Mix-PK is used as a reference knowledge base and, together with the user-input question Q1, forms the prompt text Prompt1. Finally, Prompt1 is input into a large language model to obtain the corresponding answer.

[0048] Furthermore, this invention incorporates a recall model training and comparative learning method throughout the entire retrieval process. Iterative training optimizes the recall model parameters, reducing the probability of missing important documents. Simultaneously, it performs tag generation, cluster analysis, and inverted index construction on the recalled data, further improving retrieval accuracy and response speed.

[0049] Furthermore, this invention performs secondary verification on the fused document data, categorizes similar documents through cluster analysis, and uses an inverted index to quickly locate key information, ensuring that the final search results accurately match the user's query intent and providing reliable support for answer generation.

[0050] Example 2: Figure 5 As shown, this invention provides a document knowledge intelligent retrieval system based on large model and RAG technology. This system is used to implement the document knowledge intelligent retrieval method based on large model and RAG technology described in Embodiment 1 above, specifically including: The document knowledge base construction module 100 is used to build a document knowledge base, including collecting, cleaning, preprocessing and vectorizing document knowledge, generating vector representations and storing them in a vector database; The Domain Large Model Training Module 200 is used to construct a large model for the document knowledge domain. It trains the large language model by automatically generating text corpora and using prompting learning and fine-tuning techniques to form a professional model adapted to the document knowledge domain. The user query understanding module 300 is configured to receive a document knowledge query request of a user, analyze the user demand through a natural language processing technology, and extract key elements. The intelligent retrieval and generation module 400 is configured to retrieve relevant information from a document knowledge base based on the key elements by using a document knowledge field large model and a RAG technology, and generate an answer by using the retrieved relevant information.

[0051] The document knowledge intelligent retrieval system based on the large model and the RAG technology according to the embodiment is used to implement the document knowledge intelligent retrieval method based on the large model and the RAG technology, and thus the specific embodiments of the document knowledge intelligent retrieval system based on the large model and the RAG technology can be found in the embodiment part of the document knowledge intelligent retrieval method based on the large model and the RAG technology, for example, the document knowledge base construction module 100, the field large model training module 200, the user query understanding module 300, and the intelligent retrieval and generation module 400 are respectively used to implement steps S1, S2, S3, and S4 in the document knowledge intelligent retrieval method based on the large model and the RAG technology, and thus the specific embodiments can refer to the description of the respective embodiment parts, and details are not described herein again to avoid redundancy.

[0052] Embodiment three: the embodiment provides an electronic device, the electronic device includes a processor, a memory, and a bus system, the processor and the memory are connected through the bus system, the memory is used to store instructions, and the processor is used to execute the instructions stored in the memory to implement the document knowledge intelligent retrieval method based on the large model and the RAG technology.

[0053] Embodiment four: the embodiment provides a computer storage medium, the computer storage medium stores a computer software product, the computer software product includes a plurality of instructions, and the instructions are used to make a computer device execute the document knowledge intelligent retrieval method based on the large model and the RAG technology.

[0054] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can adopt a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can adopt a computer program product in the form of one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program codes.

[0055] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flow or blocks Figure 1 one or more flow or blocks

[0056] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flow or blocks Figure 1 one or more flow or blocks Figure 1 The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flow or blocks

[0057] Obviously, the above-mentioned embodiments are only examples for clearly illustrating the present application and are not intended to limit the implementation modes. Based on the above description, other different forms of changes or variations can be made by those skilled in the art. Here, it is not necessary and also impossible to enumerate all the implementation modes. The obvious changes or variations derived therefrom are still within the protection scope of the present application.

Claims

1. A document knowledge intelligent retrieval method based on large model and RAG technology, characterized in that, Includes the following steps: Step S1: Construct a document knowledge base, including collecting, cleaning, preprocessing and vectorizing document knowledge, generating vector representations and storing them in a vector database; Step S2: Construct a large-scale model for the document knowledge domain. By automatically generating text corpora, the large language model is trained using prompting learning and fine-tuning techniques to form a professional model adapted to the document knowledge domain. Step S3: Receive the user's document knowledge query request, analyze the user's needs through natural language processing technology, and extract key elements; Step S4: Based on the key elements, using the document knowledge domain big model and RAG technology, retrieve relevant information from the document knowledge base, and generate an answer using the retrieved relevant information.

2. The document knowledge intelligent retrieval method based on large model and RAG technology according to claim 1, characterized in that, The method for constructing a document knowledge base specifically includes: Step S11: Collect and clean the document knowledge, remove noise and standardize the format; Step S12: Preprocess the cleaned document knowledge, including text segmentation, part-of-speech tagging, and syntactic analysis; Step S13: Use vectorization techniques to transform the preprocessed data into a high-dimensional vector representation in order to capture the semantic relationships in the text; Step S14: Construct a document knowledge base based on the vector representation and store the vector data in a vector database.

3. The document knowledge intelligent retrieval method based on large model and RAG technology according to claim 2, characterized in that, The vectorization technique employs a word embedding model, including Word2Vec or Doc2Vec, and the vector database includes Milvus or Chroma.

4. The document knowledge intelligent retrieval method based on large model and RAG technology according to claim 1, characterized in that, The method for constructing a large model of document knowledge domain specifically includes: Step S21: Automatically generate text corpus based on document knowledge as training samples; Step S22: Employ prompting learning techniques to design targeted prompts and guide the large language model to understand the semantics and context of document knowledge; Step S23: Optimize model parameters by iteratively training on the labeled document knowledge using fine-tuning techniques; Step S24: Regularly evaluate the model to form a professional document knowledge domain model.

5. The document knowledge intelligent retrieval method based on large model and RAG technology according to claim 1, characterized in that, The method for receiving a user's document knowledge query request, analyzing the user's needs through natural language processing technology, and extracting key elements specifically includes: Step S31: Perform word segmentation, part-of-speech tagging, and syntactic analysis on the user's query request using natural language processing technology; Step S32: Extract key elements from the query request, including the query topic, time range, and keywords; Step S33: Capture user query intent to support retrieval and generation tasks.

6. The document knowledge intelligent retrieval method based on large model and RAG technology according to claim 1, characterized in that, The relevant information retrieval in step S4 includes: The user query is vectorized and similarity is retrieved in the vector database to obtain the first candidate document set. A keyword search is performed in the search engine to obtain a second set of candidate documents; The first and second candidate document sets are deduplicated and cross-merged to form a merged document set.

7. The document knowledge intelligent retrieval method based on large model and RAG technology according to claim 6, characterized in that, The step S4 of generating the answer includes: Combine the document set with the user query into a prompt text; Input the prompt text into the document's knowledge domain model to generate an initial answer; The initial responses are processed by text summarization and sentence merging to improve readability and accuracy.

8. A document knowledge intelligent retrieval system based on large model and RAG technology, characterized in that, The system is used to implement the document knowledge intelligent retrieval method based on large model and RAG technology as described in any one of claims 1 to 7, specifically including: The document knowledge base construction module is used to build a document knowledge base, including collecting, cleaning, preprocessing and vectorizing document knowledge, generating vector representations and storing them in a vector database; The domain-specific large model training module is used to build a large model for the document knowledge domain. It trains the large language model by automatically generating text corpora and using prompting learning and fine-tuning techniques to form a professional model adapted to the document knowledge domain. The user query understanding module is used to receive users' document knowledge query requests, analyze user needs through natural language processing technology, and extract key elements. The intelligent retrieval and generation module is used to retrieve relevant information from the document knowledge base based on the key elements, using the document knowledge domain big model and RAG technology, and to generate answers using the retrieved relevant information.

9. An electronic device, characterized in that, The electronic device includes a processor, a memory, and a bus system. The processor and the memory are connected through the bus system. The memory is used to store instructions, and the processor is used to execute the instructions stored in the memory to implement the document knowledge intelligent retrieval method based on large model and RAG technology as described in any one of claims 1 to 7.

10. A computer storage medium, characterized in that, The computer storage medium stores a computer software product, which includes several instructions to cause a computer device to execute the document knowledge intelligent retrieval method based on large model and RAG technology as described in any one of claims 1 to 7.

Images