A method, apparatus, medium, and equipment for intelligent planning of large-scale RAG recall strategy.

Abstract

This invention relates to the field of intelligent question answering technology, and in particular to an intelligent planning method, apparatus, medium, and device for a large-scale model RAG recall strategy. By introducing prompt words, the initial question is transformed into a target question. The target question undergoes multi-dimensional processing of key information and vectors to obtain reference recall data based on a preset large-scale model. The initial scenario category, preset recall path, target question, reference recall data, and preset prompt words are input into the preset large-scale model to obtain the target recall path. All data recall steps in the target recall path are executed sequentially to obtain the target recall data. By providing multi-dimensional reference information to the large-scale model, it assists the model in obtaining the target recall path composed of multiple sequentially arranged data recall steps, making the data recall process more efficient and adaptable to various complex and changing application scenarios and question types, thus improving the accuracy and efficiency of the recall method.

Description

Technical Field

[0001] This invention relates to the field of intelligent question answering technology, and in particular to an intelligent planning method, apparatus, medium and equipment for a large-scale model RAG recall strategy. Background Technology

[0002] With the rapid development of information technology, users' demand for information retrieval and question-answering systems is increasing. Traditional information retrieval methods are often based on simple keyword matching, such as retrieval methods based on inverted indexes. These methods can only mechanically match keywords and are difficult to understand the semantics and context of the question. When faced with complex natural language question-answering tasks, they are unable to accurately understand the user's intent and provide comprehensive and accurate answers, resulting in a significant reduction in the accuracy and relevance of the search results.

[0003] With the development of artificial intelligence technology, large models, after being pre-trained on a large amount of text data, are able to understand and generate natural language to a certain extent, and have gradually become the focus of the field of natural language processing. However, the knowledge reserves of large models themselves have limitations. When faced with some questions that require specific domain knowledge or the latest information, it is difficult to provide a comprehensive and accurate answer by relying solely on the parameterized knowledge inside the model.

[0004] To address these issues, existing retrieval-augmented generation (RAG)-based recall tasks attempt to combine large models with external knowledge sources. However, these approaches still face numerous challenges in practical applications. For instance, the initial user-input question lacks effective analysis and transformation paths during the question processing phase, failing to fully uncover multiple potential expressions and related sub-questions, potentially leading to the omission of important information in subsequent searches. Furthermore, the recall path lacks refined and personalized recall plans for different question types, resulting in weak targeting of the recalled document data. The processing of recall results is also rather crude, lacking effective filtering, fusion, and rearrangement mechanisms, potentially leading to information redundancy and logical inconsistencies in the final answers presented to users, failing to meet their demands for high-quality and efficient information acquisition.

[0005] Therefore, how to intelligently plan RAG recall strategies to improve the accuracy and efficiency of recall results in various scenarios has become an urgent problem to be solved. Summary of the Invention

[0006] To address the aforementioned technical problems, the present invention employs a large-scale RAG recall strategy intelligent planning method, which includes the following steps:

[0007] S1: Obtain the user-inputted initial question, preset large model, preset database, initial scenario category, and preset recall path corresponding to each initial scenario category. The preset database stores initial document data, and each preset recall path includes several ordered data recall steps.

[0008] S2, input the initial question and the first preset prompt words into the preset large model to obtain several target question data. The first preset prompt words include question decomposition prompt words and question expansion prompt words. The target question data includes several target questions and the question identifier corresponding to each target question.

[0009] S3 extracts key information and transforms vectors for each target problem to obtain a list of key information and a target problem vector for each target problem.

[0010] S4. Input the target key information list, target question vector, initial document data and second preset prompt words corresponding to each target question into the preset large model to obtain the reference recall data corresponding to each target question. The reference recall data includes several documents and the recall priority of each document.

[0011] S5. Input each initial scenario category, the preset recall path corresponding to each initial scenario category, each target question, the reference recall data corresponding to each target question, and the third preset prompt word into the preset large model to obtain the target recall path corresponding to the initial question. The target recall path includes several ordered data recall steps.

[0012] S6, execute all data recall steps in the target recall path in sequence to obtain the target recall data corresponding to the initial question. The target recall data includes several document fragments.

[0013] The present invention also provides an intelligent planning device for a large-scale RAG recall strategy, which includes:

[0014] The data acquisition module is used to acquire the user's input initial question, preset large model, preset database, initial scenario category, and preset recall path corresponding to each initial scenario category. The preset database stores the initial document data, and each preset recall path includes several ordered data recall steps.

[0015] The target question acquisition module is used to input the initial question and the first preset prompt words into the preset large model to obtain several target question data. The first preset prompt words include question decomposition prompt words and question expansion prompt words. The target question data includes several target questions and the question identifier corresponding to each target question.

[0016] The target problem processing module is used to extract key information and transform vectors for each target problem, and obtain the target key information list and target problem vector corresponding to each target problem.

[0017] The first data recall module is used to input the target key information list, target question vector, initial document data and second preset prompt words corresponding to each target question into the preset large model to obtain the reference recall data corresponding to each target question. The reference recall data includes several documents and the recall priority of each document.

[0018] The target recall path acquisition module is used to input each initial scenario category, the preset recall path corresponding to each initial scenario category, each target question, the reference recall data corresponding to each target question, and the third preset prompt word into the preset large model to obtain the target recall path corresponding to the initial question. The target recall path includes several ordered data recall steps.

[0019] The second data recall module is used to execute all data recall steps in the target recall path in sequence to obtain the target recall data corresponding to the initial question. The target recall data includes several document fragments.

[0020] The present invention also provides a non-transitory computer-readable storage medium storing at least one instruction or at least one program, wherein the at least one instruction or at least one program is loaded and executed by a processor to implement the above-described intelligent planning method for large-scale RAG recall strategy.

[0021] The present invention also provides an electronic device, including a processor and the aforementioned non-transitory computer-readable storage medium.

[0022] This invention has at least the following beneficial effects: By introducing problem decomposition prompts and problem expansion prompts, the original initial problem is transformed into several more targeted and detailed target problems, improving the comprehensiveness of the large model's analysis of the initial problem. For each target problem, multi-dimensional processing of key information and vectors is performed. The target key information list, target problem vector, and second preset prompts corresponding to each target problem are input into the preset large model to obtain reference recall data corresponding to each target problem. Furthermore, each initial scenario category, the preset recall path corresponding to each initial scenario category, each target problem, the reference recall data corresponding to each target problem, and the third preset prompts are input into the preset large model to obtain the initial problem... The target recall path corresponding to the question executes all data recall stages in the target recall path sequentially to obtain the target recall data corresponding to the initial question. Based on multi-dimensional input information, it provides multi-dimensional reference information for the large model, giving full play to the natural language processing capabilities of the large model and the guiding role of different prompt words. This helps the large model obtain the target recall path composed of multiple sequentially arranged data recall stages, making the data recall process more efficient. Moreover, the target recall path can be adjusted according to the specific target question, enabling the large model RAG recall strategy intelligent planning method to adapt to various complex and changing application scenarios and question types, thus improving the accuracy and efficiency of the large model RAG recall strategy intelligent planning method. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 A flowchart of an intelligent planning method for a large-scale RAG recall strategy provided in Embodiment 1 of the present invention;

[0025] Figure 2 This is a schematic diagram of the structure of a large-model RAG recall strategy intelligent planning device provided in Embodiment 2 of the present invention. Detailed Implementation

[0026] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It is understood that, where appropriate, the terms used to distinguish similar objects can be interchanged so that the invention can also be implemented in other embodiments besides the illustrated or described embodiments. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products, or devices.

[0028] Example 1

[0029] This embodiment provides an intelligent planning method for a large-scale RAG recall strategy. This method includes the following steps: Figure 1 As shown:

[0030] S1: Obtain the user-inputted initial question, preset large model, preset database, initial scenario category, and preset recall path corresponding to each initial scenario category. The preset database stores initial document data, and each preset recall path includes several ordered data recall steps.

[0031] Among them, the pre-defined large model possesses powerful natural language processing capabilities, enabling it to understand text meaning, perform semantic analysis, and generate text. For example, the pre-defined large model can be a specific large language model trained based on the Transformer architecture.

[0032] The initial document data stored in the pre-defined database provides a knowledge source for the entire retrieval process, covering various fields and types, such as academic papers, news reports, product manuals, encyclopedias, etc. It includes various document-related data such as document fragments, basic document information, document paragraph information, document table information, document image information, document link information, document recommendation Q&A information, as well as initial key information data and initial vector data, so as to increase the comprehensiveness and accuracy of answering questions through rich document data.

[0033] The document fragments can be segments obtained by dividing the entire document according to semantics, word count, or paragraph segmentation methods. Basic document information can include the word count, number of chapters, abstract, key information, vectorization method, and entities contained in the title for each document. Document paragraph information can include the word count, summary, abstract, and key information for each paragraph. Document table information can include the tables contained in each document, the page number of each table, the context of the table, and the table description. Document image information can include the images contained in each document, the page number of each image, the context of the image, the image description, and the image source. Document link information can include the links contained in each document, the URL of each link, title content, body content, publication date, publishing institution, and publishing site. Document recommended Q&A information can include recommended questions and answers for each document.

[0034] This embodiment is applicable to problem analysis and document retrieval under various scenario categories. The initial scenario category can be set by the implementer according to the actual situation. Problems under different scenario categories have different processing methods and expected retrieval results. Therefore, in order to improve the efficiency and accuracy of retrieval, different retrieval paths are formulated for different initial scenario categories.

[0035] In one specific implementation, the initial scenario category may include scenarios such as summary scenario, recommendation question and answer scenario, original text recall scenario, and abstract recall scenario. The preset data recall process includes data recall method and data processing method. The data recall method includes at least one of text ranking recall, vector recall, abstract recall, question and answer recall, chart recall, full text recall, summary recall, and external data recall. The data processing method includes at least one of no processing, data rearrangement, data compression, and data fusion.

[0036] The preset recall path corresponding to each initial scenario category can be obtained by testing and analyzing the recall paths and recall results of a large number of questions under each initial scenario category. This can help the large model to execute the recall operation efficiently and accurately according to the predetermined path, providing users with high-quality recall services and accurate recall results.

[0037] The implementation methods for text ranking recall include: First, text ranking recall can be based on certain text relevance evaluation criteria, ranking the text data corresponding to documents and prioritizing document fragments with high relevance to the query. Second, vector recall can be implemented by first converting text data into vector representations, then calculating the similarity between the question vector and the text vector to measure the degree of association between the data, and recalling document fragments with high relevance. Third, summary recall can be implemented by selecting document fragments whose summaries match the query content from a large number of documents. Fourth, question-and-answer recall can be implemented by targeting document recommendation question-and-answer information stored in question-and-answer format, matching the user's question against existing document recommendation question-and-answer information, and finding the question and answer that is most similar to and best answers the user's question. Fifth, chart recall can be implemented by classifying, labeling, and searching for images and tables in document tables and image information based on user needs (such as finding images and tables related to a specific topic, specific types of images and tables, etc.), and retrieving and matching based on text descriptions related to images and tables, recalling document fragments containing images and tables that meet the requirements. Full-text retrieval can be achieved through broad retrieval methods such as simple keyword matching. As long as a document fragment contains content relevant to the query, regardless of its relevance or location within the document, the entire fragment will be retrieved. Summary retrieval can be achieved by extracting key information from numerous documents, summarizing it to form a relatively complete and concise summary answer, and then retrieving document fragments related to that summary answer. External data retrieval can be achieved when the data within the database cannot meet the requirements. This involves establishing connections with external data sources and retrieving relevant data from external databases, network resources, etc., according to specific interface protocols and retrieval rules.

[0038] The implementation method without processing can be to directly present the recalled document fragments to the user or proceed with subsequent processes unchanged, without any additional operations. This is suitable when the recalled document fragments already meet the requirements, or when subsequent steps do not have special requirements regarding data format. The implementation method of data rearrangement can be to rearrange the recalled document fragments according to certain rules, such as chronological order, relevance, or importance. The implementation method of data compression can be to simplify the recalled document fragments, reducing the data volume, typically by removing redundant document fragments and extracting key document fragments. The implementation method of data fusion can be to integrate document fragments from different sources, in different formats, or with certain correlations, making them a whole to better meet user needs or facilitate subsequent analysis and processing.

[0039] In the abstract recall scenario, the query is usually about the abstract of a specific document, such as "extract the abstract content of Chapter 4, Section 2 of document A". This type of question requires accurately retrieving relevant document fragments about the abstract of Chapter 4, Section 2 of document A from the database. The corresponding preset recall path can be abstract recall or no processing.

[0040] The key to the recommended question-and-answer scenario is to filter out suitable recommendations from the database based on user needs. The database stores document recommended question-and-answer information. It can perform similarity analysis between the user's input question and the recommended questions corresponding to the document, and recall recommended questions and answers based on the similarity. Therefore, the corresponding preset recall path can be question-and-answer recall.

[0041] Summarizing scenarios requires identifying key information and core viewpoints from a large number of documents. This involves extracting key information from numerous economic news articles and reports, summarizing it, and forming a relatively complete and concise summary answer. Taking "Summarize the main trends in global economic development this year" as an example, the corresponding recall path should be able to filter relevant content on key trends such as economic growth, inflation, and exchange rate changes from numerous economic documents. To recall more data for a comprehensive analysis and answer to the initial question, the corresponding preset recall path could be sequential text sorting recall, vector recall, chart recall, data fusion, data rearrangement, and data compression. The input data volume of the large model is dynamically calculated based on the maximum tokens requirement of the preset large model and the output data volume of the initial question. The amount of data to be recalled is determined based on the input data volume of the large model and the input data volume of the initial question. Simultaneously, data filtering, data fusion, and data rearrangement operations are performed on the recalled data. When the total word count of the recalled document data is less than the maximum tokens requirement of the large model, the original document text can be directly recalled to improve the richness and completeness of the data. Tokens are the basic units for language models to process text. Since the computing resources and memory of large models are limited, in order to ensure the performance and efficiency of the model, the number of tokens contained in the text is limited during the input text or text generation process, i.e., a maximum token requirement is set.

[0042] Original text retrieval scenarios often require quickly providing accurate original text answers. Since the database stores documents, it is possible to search and analyze the user's input question and document content to quickly locate the most directly relevant original text. The corresponding preset retrieval paths can be vector retrieval, data rearrangement, and data compression.

[0043] The above-mentioned acquisition of the user's initial question, the preset large model, the initial scenario category, and the preset recall path corresponding to each initial scenario category lays the foundation for accurate and efficient processing of user questions and recall of appropriate document data.

[0044] S2, input the initial question and the first preset prompt words into the preset large model to obtain several target question data. The first preset prompt words include question decomposition prompt words and question expansion prompt words. The target question data includes several target questions and the question identifier corresponding to each target question.

[0045] In one specific implementation, the problem identifier includes a first problem identifier and a second problem identifier.

[0046] The preset large model is used to determine the preset problem category corresponding to the initial problem. The preset problem categories include related problems and unrelated problems.

[0047] The pre-set large model is also used to obtain several decomposed problems and the first problem identifier corresponding to each decomposed problem when the pre-set problem category corresponding to the initial problem is determined to be a related problem. It also obtains several target problems corresponding to each decomposed problem and the first problem identifier corresponding to each target problem based on the initial problem and problem decomposition prompt words. The first problem identifier includes the main problem and the sub-problems.

[0048] The pre-set large model is also used to obtain several target questions corresponding to the initial question and a second question identifier corresponding to each target question when it is determined that the pre-set question category corresponding to the initial question is an unrelated question, based on the initial question and question expansion prompts. The second question identifier includes questions without hierarchy.

[0049] The preset question categories include related questions and unrelated questions. Related questions indicate that the initial question has a close intrinsic connection with a specific subject, theme, or context, requiring in-depth analysis of various related elements. Unrelated questions are relatively more independent and general, and do not rely on in-depth correlation analysis of a specific thing.

[0050] When the initial question is determined to belong to a predefined problem category, the predefined large model, based on the initial question and problem decomposition prompts, breaks down the relatively complex, comprehensive, and broad initial related question into several more easily analyzed and processed, more specific decomposition questions. It also obtains a first problem identifier for each decomposition question, indicating whether each decomposition question is a main problem or a sub-problem, thus clarifying the problem hierarchy and assisting the large model in planning an efficient problem-solving path. The large model then generates more detailed and diverse target questions and their corresponding first problem identifiers based on each decomposition question, its corresponding problem identifier, and problem expansion prompts. This allows for the gradual expansion of a single initial related question into multiple more targeted target questions that can be explored in depth from different angles, facilitating a more comprehensive retrieval of document data related to the initial question.

[0051] When the initial question is determined to be a non-related question in the preset question category, the large model refines and expands the general initial question based on the initial question and question expansion prompts, obtaining several target questions corresponding to the initial question and a second question identifier for each target question. This helps to more accurately retrieve document data related to the target questions from the preset database. For example, if the initial question is "What are the common types of heat dissipation technologies for electronic products?", after adding question expansion prompts, the large model may generate target questions such as "In the field of laptops, what are the common types of heat dissipation technologies for electronic products?", "From the perspective of heat dissipation principles, what are the common types of heat dissipation technologies for electronic products?", and "What are the commonly used heat dissipation technologies for electronic products in different price ranges?", thus broadening the diversity and scope of the questions.

[0052] In one specific implementation, the correlation problem includes parallel problems and sequential dependency problems.

[0053] Parallel questions involve information queries from multiple entities on the same dimension. When decomposing parallel questions, they can be broken down by entity, and the first question identifier corresponding to each decomposed question can be obtained. Then, each decomposed question can be expanded to obtain the corresponding target question. For example, "What were the operating revenues of Company C and Company D in 2022?", where Company C and Company D are parallel entities. In a real document retrieval scenario, it is necessary to search the pre-set database for documents that can provide relevant information on the operating revenues of Company C and Company D in 2022. The corresponding decomposed questions could be "What were the operating revenues of Company C in 2022?" and "What were the operating revenues of Company D in 2022?", with the first question identifier being the main question. The expanded target questions could be "What were the operating revenues of Company C in each quarter of 2022?", "How does Company C rank in terms of operating revenue in the same industry in 2022?", "What were the operating revenues of Company D in each quarter of 2022?", and "How does Company D rank in terms of operating revenue in the same industry in 2022?", with the first question identifier being the main question.

[0054] The characteristic of a dependency problem is that the parts of the problem have a logical sequence and dependency relationship. When decomposing a dependency problem, it is necessary to follow its inherent logical order. For example, it can be decomposed into a basic information problem and a related cause problem, or into a stage-based problem. For example, in the question "What was the change in Company E's revenue in 2022 compared to 2021? What were the specific reasons?", the specific reasons and the revenue change are dependent. In a real document retrieval scenario, it is necessary to find a document that can provide both data comparison and cause analysis. The corresponding decomposition questions could be "What was the change in Company E's revenue in 2022 compared to 2021?", with the first question identified as the main question, and "What were the specific reasons for the change in Company E's revenue in 2022 compared to 2021?", with the first question identified as the sub-question. The expanded target questions could be: "What were the revenue changes for each business segment of Company E in 2022 and 2021 respectively?", with the first question marked as the main question; "How did Company E's revenue change in different market regions in 2022 and 2021?", with the first question marked as the main question; "From a market competition perspective, what are the reasons for the change in Company E's revenue in 2022 compared to 2021?", with the first question marked as a sub-question; and "From a product roadmap perspective, what are the reasons for the change in Company E's revenue in 2022 compared to 2021?", with the first question marked as a sub-question.

[0055] As described above, by introducing problem decomposition prompts and problem expansion prompts, the original initial problem is transformed into several more targeted and detailed target problems and a corresponding problem identifier for each target problem. This can improve the comprehensiveness of the large model's analysis of the initial problem and provide a data foundation for more accurately retrieving relevant document data from the preset database.

[0056] S3 extracts key information and transforms vectors for each target problem to obtain a list of key information and a target problem vector for each target problem.

[0057] This process involves extracting key words or phrases from each target question to accurately reflect its core content, facilitating rapid location of relevant content within document data. Converting the target questions into vector form allows for better measurement of the similarity between the target questions and document vectors in a pre-defined database within a vector space. In a high-dimensional vector space, semantically similar texts have closer vector distances, enabling precise retrieval of documents most semantically closest to the target question by calculating vector similarity when searching for related document data.

[0058] In one specific embodiment, S3 includes the following steps:

[0059] S31, extract keywords for each target question to obtain a list of target keywords corresponding to each target question.

[0060] S32, extract entities for each target problem to obtain a list of target entities corresponding to each target problem.

[0061] S33. Based on the list of target keywords and the list of target entities corresponding to each target question, obtain the list of target key information corresponding to each target question.

[0062] The specific keyword extraction method can be set by the implementer according to the actual situation. For example, lexical analysis and part-of-speech tagging techniques from natural language processing can be used to identify nouns, verbs, adjectives, and other key words as keywords. Alternatively, predefined domain vocabularies can be used to further filter and determine important keywords, ensuring that the extracted keywords represent the key information of the target problem to the greatest extent possible.

[0063] Entity extraction identifies and extracts meaningful entity information from a textual target question, organizing these entities into a target entity list for further analysis, processing, or use in other applications. Specific entity extraction methods can be set by the implementer based on the actual situation; for example, rule-based methods, statistical machine learning-based methods, and deep learning-based methods.

[0064] The specific vector transformation technique can be set by the implementer according to the actual situation. For example, word embedding technology (such as Word2Vec, BERT and other pre-trained models) can be used to first convert each word in the target question into its corresponding vector representation, and then the word vectors can be integrated into the overall vector representation of the target question through averaging, weighted averaging or a combination method based on neural networks. Those skilled in the art will know that any keyword extraction method, any entity extraction method, and any vector transformation technique in the prior art falls within the protection scope of this invention, and will not be elaborated further here.

[0065] The above-mentioned multi-dimensional processing of key information and vectors for each target problem lays the foundation for the next step of retrieving document data related to the problem from the database, thereby providing a data foundation for more accurate and efficient data retrieval from the preset database.

[0066] S4. Input the target key information list, target question vector and second preset prompt words corresponding to each target question into the preset large model to obtain the reference recall data corresponding to each target question. The reference recall data includes several documents and the recall priority of each document.

[0067] The second preset prompt word serves to guide and standardize the large model in retrieving reference recall data from the database. The specific content of the second preset prompt word can be set by the implementer according to the actual situation. For example, the third preset prompt word could be "Based on the above content, output the corresponding document from the database as reference recall data".

[0068] In one specific implementation, the reference recall data includes first reference recall data, second reference recall data, and third reference recall data, and the initial document data includes several documents, initial key information data corresponding to the documents, and initial vector data.

[0069] The pre-set large model is used to match key information between the target key information list and the initial key information data, and to filter out the first reference recall data from the initial document data based on the key information matching results. The number of documents in the first reference recall data meets the first preset number.

[0070] The pre-set large model is also used to perform similarity matching between the target question vector and the initial vector data, and to filter out second reference recall data from the initial document data based on the similarity matching results. The number of documents in the second reference recall data meets the second pre-set number.

[0071] The preset large model is also used to filter out third reference recall data from the initial document data that was not selected as first reference recall data and second reference recall data, wherein the number of documents in the third reference recall data meets the third preset number.

[0072] Among them, the first preset quantity is greater than the second preset quantity, and the second preset quantity is greater than the third preset quantity.

[0073] Due to the large number of documents in the initial document data, some potentially useful document fragments may still remain undetected even with the two filtering methods of key information matching and similarity matching. The third reference recall data involves further filtering from these undetected documents, aiming to uncover documents with unique perspectives or slightly weaker relevance but still possessing value, thus improving the comprehensiveness of the recalled documents. Furthermore, as a supplement to the key information matching and similarity matching data, the third preset number is set to be smaller than the first and second preset numbers, forming a hierarchical and focused document recall system, thereby improving the accuracy of the entire RAG recall method.

[0074] In one specific implementation, the initial key information data includes an initial key information set corresponding to each document, and the key information matching result includes the number of key information duplicates between the initial key information set corresponding to each document and the target key information list. First reference recall data is filtered out based on the key information matching result, including:

[0075] Documents that meet a first preset number are selected based on the number of repetitions of key information, from most to least, and the first reference recall data is formed based on the selected documents that meet the first preset number.

[0076] Among them, when there is a lot of duplicate key information between the target key information list and the initial key information set corresponding to the document, that is, there are many duplicate keywords and duplicate entities, the document is likely to be closely related to the target question in terms of content theme, and can provide more direct and clear information for answering the target question. Therefore, the first reference recall data is given the largest data volume quota to ensure that as many highly relevant documents as possible are recalled, so as to provide sufficient material basis for subsequent analysis from the key information level on the surface of the text.

[0077] Specifically, documents that meet the first preset number are selected according to the number of repetitions of key information, in descending order of quantity. This ensures that the selected first reference recall data is strongly correlated with the initial question, thereby improving the accuracy of the entire RAG recall method.

[0078] In one specific implementation, the initial vector data includes an initial vector set corresponding to each document, and the similarity matching result includes the vector similarity between the initial vector set corresponding to each document and the target question vector. Second reference recall data is selected based on the similarity matching result, including:

[0079] Documents that meet the second preset number of similarity criteria are selected in descending order of similarity, and the selected documents that meet the second preset number of similarity criteria are used to form the second reference recall data.

[0080] In this approach, some documents may not contain a large amount of duplicated key information as the target question, but their overall semantic expression closely matches the target question. Therefore, this embodiment, based on key information matching, further recalls documents that are semantically similar to the target question by calculating the similarity between the target question vector and the initial vector set corresponding to the documents. This supplements the document information from a deeper semantic understanding perspective, providing richer and more semantically relevant reference content for answering the question. Furthermore, as a supplementary data element for key information matching, a second preset number is set smaller than the first preset number to form a hierarchical and focused document recall system, thereby improving the accuracy of the entire RAG recall method.

[0081] Specifically, documents that meet the second preset number are selected according to the order of similarity from high to low to ensure that the selected second reference recall data has a high degree of relevance to the initial question, thereby improving the accuracy and reliability of the entire RAG recall method.

[0082] The above approach first retrieves a large number of highly relevant documents based on key information matching, then supplements them by filtering documents with high semantic fit through semantic vector similarity, and finally uses a fallback method to filter out and select third-party reference recall data. This ensures that the recalled documents comprehensively cover content directly related to the target question, while also taking into account deeper semantic relevance and some potentially valuable information. This provides a solid and rich material foundation for generating accurate final results that meet the user's initial question requirements, and improves the accuracy of the entire large-scale RAG recall strategy intelligent planning method.

[0083] S5. Input each initial scenario category, the preset recall path corresponding to each initial scenario category, each target question, the reference recall data corresponding to each target question, and the third preset prompt word into the preset large model to obtain the target recall path corresponding to the initial question. The target recall path includes several ordered data recall steps.

[0084] The third preset prompt word serves to guide and standardize the model in generating the target recall path. The specific content of the third preset prompt word can be set by the implementer according to the actual situation. For example, the third preset prompt word could be "output the corresponding target recall path based on the above content".

[0085] The initial scenario category clarifies the overall framework and background of the problem. Different initial scenario categories have different task priorities and data processing logics, corresponding to different preset recall paths. The preset recall path is a relatively standardized recall process obtained through a lot of practice, testing and analysis in the early stage. Each initial scenario category and its corresponding preset recall path provide a basic reference framework for the initial problem to be dealt with. This allows the large model to be adjusted and optimized based on the characteristics of the initial problem, ensuring the scientific nature and effectiveness of the target recall path.

[0086] The target question clearly indicates the specific question or task to be solved, enabling the large model to accurately plan the recall path around the target question, ensuring that the final recalled data is highly relevant to the initial question corresponding to solving the target question.

[0087] The reference recall data for each target question consists of documents that are related to that target question. This helps the large model better understand the data range, features, and association patterns involved in the target question, and helps the large model construct the target recall path more reasonably.

[0088] In one specific embodiment, S5 further includes the following steps:

[0089] S51, obtain M historical similar questions corresponding to the initial question, and the target recall path corresponding to each historical similar question.

[0090] S52, input each target question, each initial scenario category, the preset recall path corresponding to each initial scenario category, the reference recall data corresponding to each target question, M historical similar questions, the target recall path corresponding to each historical similar question, and the third preset prompt word into the preset large model to obtain the target recall path corresponding to the initial question.

[0091] Among them, historical similar problems are those that have been processed in the past and are similar to the current initial problem in terms of problem nature, theme, structure, etc. The target recall paths determined for similar problems in the past contain experience and effective strategies that can be learned from. Therefore, historical similar problems and their corresponding target recall paths are input into the preset large model to obtain the target recall path corresponding to the initial problem, so as to improve the efficiency and accuracy of the target recall path generation for the current initial problem.

[0092] In one specific embodiment, S5 further includes the following steps:

[0093] S53, obtain the description text corresponding to each preset data retrieval step.

[0094] S54. Input each target question, each initial scenario category, the preset recall path corresponding to each initial scenario category, the reference recall data corresponding to each target question, the description text corresponding to each preset data recall stage, and the third preset prompt word into the preset large model to obtain the target recall path corresponding to the initial question.

[0095] The descriptive text details the working principles, operational methods, applicable scenarios, and related characteristics of each data retrieval stage. For example, the descriptive text for the text ranking retrieval stage explains how to score and rank documents based on factors such as keyword distribution and semantic relevance. The descriptive text for the vector retrieval stage explains how to convert text in documents into vector representations and how to perform data retrieval by calculating the similarity between vectors. By incorporating the descriptive text of each data retrieval stage into the input of the large model, the model can gain a deeper understanding of the details of each retrieval stage, thereby improving the efficiency and accuracy of generating the initial target retrieval path.

[0096] In one specific embodiment, S5 further includes the following steps:

[0097] S55, obtain the M historical similar questions corresponding to the initial question, the target recall path corresponding to each historical similar question, and the description text corresponding to each preset data recall step.

[0098] S56. Input each target question, each initial scenario category, the preset recall path corresponding to each initial scenario category, the reference recall data corresponding to each target question, M historical similar questions, the target recall path corresponding to each historical similar question, the description text corresponding to each preset data recall stage, and the third preset prompt word into the preset large model to obtain the target recall path corresponding to the initial question.

[0099] As described above, by leveraging a pre-set large model and providing multi-dimensional reference information based on various input information, the large model is assisted in obtaining a target recall path consisting of multiple sequentially arranged data recall stages. The orderly nature of the data recall stages makes the data recall process more efficient, and the target recall path can be adjusted according to the specific target problem. This enables the entire large model RAG recall strategy intelligent planning method to adapt to various complex and ever-changing application scenarios and problem types, ensuring the quality of the final target recall data.

[0100] S6, execute all data recall steps in the target recall path in sequence to obtain the target recall data corresponding to the initial question. The target recall data includes several document fragments.

[0101] The target recall data is presented in the form of several document fragments. These document fragments can be further organized, analyzed, and summarized to provide strong support for related decision-making, research, or other applications.

[0102] As described above, by introducing problem decomposition and problem expansion prompts, the original initial problem is transformed into several more targeted and detailed target problems, improving the comprehensiveness of the large model's analysis of the initial problem. For each target problem, multi-dimensional processing of key information and vectors is performed. The target key information list, target problem vector, and second preset prompts for each target problem are input into the preset large model to obtain reference recall data for each target problem. Furthermore, each initial scenario category, the preset recall path corresponding to each initial scenario category, each target problem, the reference recall data for each target problem, and the third preset prompts are input into the preset large model to obtain the target recall for the initial problem. The path executes all data recall stages in the target recall path sequentially to obtain the target recall data corresponding to the initial question. Based on multi-dimensional input information, it provides multi-dimensional reference information to the large model, giving full play to the natural language processing capabilities of the large model and the guiding role of different prompt words. This helps the large model obtain the target recall path composed of multiple sequentially arranged data recall stages, making the data recall process more efficient. Moreover, the target recall path can be adjusted according to the specific target question, enabling the entire large model RAG recall strategy intelligent planning method to adapt to various complex and changing application scenarios and question types, thus improving the accuracy and efficiency of the entire large model RAG recall strategy intelligent planning method.

[0103] Example 2

[0104] This second embodiment provides an intelligent planning device for a large-scale RAG recall strategy. This device includes, for example: Figure 2 As shown:

[0105] The data acquisition module 21 is used to acquire the user's input initial question, preset large model, preset database, initial scenario category and preset recall path corresponding to each initial scenario category. The preset database stores the initial document data, and each preset recall path includes several ordered data recall steps.

[0106] The target question acquisition module 22 is used to input the initial question and the first preset prompt words into the preset large model to obtain several target question data. The first preset prompt words include question decomposition prompt words and question expansion prompt words. The target question data includes several target questions and the question identifier corresponding to each target question.

[0107] The target problem processing module 23 is used to extract key information and transform vectors for each target problem, and obtain the target key information list and target problem vector corresponding to each target problem.

[0108] The first data recall module 24 is used to input the target key information list, target question vector, initial document data and second preset prompt words corresponding to each target question into the preset large model to obtain the reference recall data corresponding to each target question. The reference recall data includes several documents and the recall priority corresponding to each document.

[0109] The target recall path acquisition module 25 is used to input each initial scenario category, the preset recall path corresponding to each initial scenario category, each target question, the reference recall data corresponding to each target question, and the third preset prompt word into the preset large model to obtain the target recall path corresponding to the initial question. The target recall path includes several ordered data recall steps.

[0110] The second data recall module 26 is used to execute all data recall steps in the target recall path in sequence to obtain the target recall data corresponding to the initial question. The target recall data includes several document fragments.

[0111] In one specific implementation, in the target problem acquisition module 22, the problem identifier includes a first problem identifier and a second problem identifier.

[0112] The preset large model is used to determine the preset problem category corresponding to the initial problem. The preset problem categories include related problems and unrelated problems.

[0113] The pre-set large model is also used to obtain several decomposed problems and the problem identifier corresponding to each decomposed problem when the pre-set problem category corresponding to the initial problem is determined to be a related problem. It also obtains several target problems corresponding to each decomposed problem and the first problem identifier corresponding to each target problem based on each decomposed problem, the first problem identifier corresponding to each decomposed problem, and the problem expansion prompt. The first problem identifier includes the main problem and the sub-problems.

[0114] The pre-set large model is also used to obtain several target questions corresponding to the initial question and a second question identifier corresponding to each target question when it is determined that the pre-set question category corresponding to the initial question is an unrelated question, based on the initial question and question expansion prompts. The second question identifier includes questions without hierarchy.

[0115] In one specific embodiment, the target problem processing module 23 includes:

[0116] The keyword extraction submodule is used to extract keywords for each target question and obtain a list of target keywords corresponding to each target question.

[0117] The entity extraction submodule is used to extract entities for each target question and obtain a list of target entities corresponding to each target question.

[0118] The key information list retrieval submodule is used to retrieve the target key information list corresponding to each target question based on the target keyword list and target entity list corresponding to each target question.

[0119] In one specific implementation, in the target recall path acquisition module 25, the reference recall data includes first reference recall data, second reference recall data and third reference recall data, and the initial document data includes several documents, initial key information data corresponding to the documents and initial vector data.

[0120] The pre-set large model is used to match key information between the target key information list and the initial key information data, and to filter out the first reference recall data from the initial document data based on the key information matching results. The number of documents in the first reference recall data meets the first preset number.

[0121] The pre-set large model is also used to perform similarity matching between the target question vector and the initial vector data, and to filter out second reference recall data from the initial document data based on the similarity matching results. The number of documents in the second reference recall data meets the second pre-set number.

[0122] The preset large model is also used to filter out third reference recall data from the initial document data that was not selected as first reference recall data and second reference recall data, wherein the number of documents in the third reference recall data meets the third preset number.

[0123] In one specific embodiment, the target recall path acquisition module 25 further includes:

[0124] The historical data acquisition submodule is used to acquire M historical similar questions corresponding to the initial question, as well as the target recall path corresponding to each historical similar question.

[0125] The first target recall path acquisition submodule is used to input each target question, each initial scenario category, the preset recall path corresponding to each initial scenario category, the reference recall data corresponding to each target question, M historical similar questions, the target recall path corresponding to each historical similar question, and the third preset prompt word into the preset large model to obtain the target recall path corresponding to the initial question.

[0126] In one specific embodiment, the target recall path acquisition module 25 further includes:

[0127] The description text acquisition submodule is used to acquire the description text corresponding to each preset data retrieval stage.

[0128] The second target recall path acquisition submodule is used to input each target question, each initial scenario category, the preset recall path corresponding to each initial scenario category, the reference recall data corresponding to each target question, the description text corresponding to each preset data recall stage, and the third preset prompt word into the preset large model to obtain the target recall path corresponding to the initial question.

[0129] In one specific implementation, the initial scenario categories in the data acquisition module 21 include summary scenario, recommendation question and answer scenario, original text recall scenario, and summary recall scenario.

[0130] The pre-defined data retrieval process includes data retrieval methods and data processing methods. Data retrieval methods include at least one of the following: text ranking retrieval, vector retrieval, summary retrieval, question-and-answer retrieval, chart retrieval, full-text retrieval, summary retrieval, and external data retrieval. Data processing methods include at least one of the following: no processing, data rearrangement, data compression, and data fusion.

[0131] It should be noted that the information interaction and execution process between the above modules are based on the same concept as the method embodiments of the present invention. For details on their specific functions and technical effects, please refer to the method embodiments section, which will not be repeated here.

[0132] Example 3

[0133] Embodiment 3 of the present invention provides a non-transitory computer-readable storage medium, which stores at least one instruction or at least one program segment, wherein the at least one instruction or at least one program segment is loaded and executed by a processor to implement the following steps:

[0134] S1: Obtain the user-inputted initial question, preset large model, preset database, initial scenario category, and preset recall path corresponding to each initial scenario category. The preset database stores initial document data, and each preset recall path includes several ordered data recall steps.

[0135] S2, input the initial question and the first preset prompt words into the preset large model to obtain several target question data. The first preset prompt words include question decomposition prompt words and question expansion prompt words. The target question data includes several target questions and the question identifier corresponding to each target question.

[0136] S3 extracts key information and transforms vectors for each target problem to obtain a list of key information and a target problem vector for each target problem.

[0137] S4. Input the target key information list, target question vector, initial document data and second preset prompt words corresponding to each target question into the preset large model to obtain the reference recall data corresponding to each target question. The reference recall data includes several documents and the recall priority of each document.

[0138] S5. Input each initial scenario category, the preset recall path corresponding to each initial scenario category, each target question, the reference recall data corresponding to each target question, and the third preset prompt word into the preset large model to obtain the target recall path corresponding to the initial question. The target recall path includes several ordered data recall steps.

[0139] S6, execute all data recall steps in the target recall path in sequence to obtain the target recall data corresponding to the initial question. The target recall data includes several document fragments.

[0140] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0141] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0142] Example 4

[0143] Embodiment 4 of the present invention provides an electronic device, which includes a processor and a non-transitory computer-readable storage medium as described in Embodiment 3 of the present invention.

[0144] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A method for intelligent planning of large-scale model RAG recall strategy, characterized in that, The intelligent planning method for the large-scale RAG recall strategy includes the following steps: S1, obtain the user-input initial question, preset large model, preset database, initial scene category and preset recall path corresponding to each initial scene category, wherein the preset database stores initial document data, and each preset recall path includes several ordered data recall steps; S2, input the initial question and the first preset prompt word into the preset large model to obtain several target question data, wherein the first preset prompt word includes question decomposition prompt word and question expansion prompt word, and the target question data includes several target questions and a question identifier corresponding to each target question; S3, extract key information and transform vectors for each target problem to obtain a list of key information and a target problem vector for each target problem; S4, input the target key information list, target question vector, initial document data and second preset prompt words corresponding to each target question into the preset large model to obtain the reference recall data corresponding to each target question, wherein the reference recall data includes several documents and the recall priority corresponding to each document; S5, input each initial scene category, the preset recall path corresponding to each initial scene category, each target question, the reference recall data corresponding to each target question, and the third preset prompt word into the preset large model to obtain the target recall path corresponding to the initial question, wherein the target recall path includes several ordered data recall steps; S6, execute all data recall steps in the target recall path in sequence to obtain the target recall data corresponding to the initial question, wherein the target recall data includes several document fragments.

2. The intelligent planning method for large-scale RAG recall strategy according to claim 1, characterized in that, The problem identifier includes a first problem identifier and a second problem identifier; The preset large model is used to determine the preset question category corresponding to the initial question, wherein the preset question category includes related questions and unrelated questions; The preset large model is also used to, when determining that the preset problem category corresponding to the initial problem is a related problem, obtain several decomposed problems and a problem identifier corresponding to each decomposed problem based on the initial problem and the problem decomposition prompt words, and obtain several target problems corresponding to each decomposed problem and a first problem identifier corresponding to each target problem based on each decomposed problem, the first problem identifier corresponding to each decomposed problem and the problem expansion prompt words, wherein the first problem identifier includes a main problem and a sub-problem; The preset large model is also used to obtain, when it is determined that the preset question category corresponding to the initial question is an unrelated question, a number of target questions corresponding to the initial question and a second question identifier corresponding to each target question based on the initial question and the question expansion prompt words, wherein the second question identifier includes unlevel questions.

3. The intelligent planning method for large-scale RAG recall strategy according to claim 1, characterized in that, S3 includes the following steps: S31, Extract keywords for each target question to obtain a list of target keywords for each target question; S32, extract entities for each target problem to obtain a list of target entities corresponding to each target problem; S33. Based on the list of target keywords and the list of target entities corresponding to each target question, obtain the list of target key information corresponding to each target question.

4. The intelligent planning method for large-scale RAG recall strategy according to claim 1, characterized in that, The reference recall data includes first reference recall data, second reference recall data and third reference recall data, and the initial document data includes several documents, initial key information data corresponding to the documents and initial vector data; The preset large model is used to perform key information matching between the target key information list and the initial key information data, and to filter out the first reference recall data from the initial document data based on the key information matching results, wherein the number of documents in the first reference recall data meets the first preset number. The preset large model is also used to perform similarity matching between the target question vector and the initial vector data, and to filter out second reference recall data from the initial document data based on the similarity matching results, wherein the number of documents in the second reference recall data meets the second preset number; The preset large model is also used to filter out third reference recall data from the initial document data that was not selected as first reference recall data and second reference recall data, wherein the number of documents in the third reference recall data meets the third preset number.

5. The intelligent planning method for large-scale RAG recall strategy according to claim 1, characterized in that, S5 also includes the following steps: S51, obtain M historical similar questions corresponding to the initial question, and the target recall path corresponding to each historical similar question; S52, input each target question, each initial scenario category, the preset recall path corresponding to each initial scenario category, the reference recall data corresponding to each target question, M historical similar questions, the target recall path corresponding to each historical similar question, and the third preset prompt word into the preset large model to obtain the target recall path corresponding to the initial question.

6. The intelligent planning method for large-scale RAG recall strategy according to claim 1, characterized in that, S5 also includes the following steps: S53, obtain the description text corresponding to each preset data retrieval step; S54, input each target question, each initial scenario category, the preset recall path corresponding to each initial scenario category, the reference recall data corresponding to each target question, the description text corresponding to each preset data recall stage, and the third preset prompt word into the preset large model to obtain the target recall path corresponding to the initial question.

7. The intelligent planning method for large-scale RAG recall strategy according to claim 1, characterized in that, The initial scenario categories include summary scenarios, recommendation and question-and-answer scenarios, original text recall scenarios, and abstract recall scenarios; The preset data retrieval process includes a data retrieval method and a data processing method. The data retrieval method includes at least one of text sorting retrieval, vector retrieval, summary retrieval, question-and-answer retrieval, chart retrieval, full-text retrieval, summary retrieval, and external data retrieval. The data processing method includes at least one of no processing, data rearrangement, data compression, and data fusion.

8. A large-scale model RAG recall strategy intelligent planning device, characterized in that, The intelligent planning device for the large-scale RAG recall strategy includes: The data acquisition module is used to acquire the user's input initial question, preset large model, preset database, initial scene category and preset recall path corresponding to each initial scene category. The preset database stores initial document data, and each preset recall path includes several ordered data recall steps. The target question acquisition module is used to input the initial question and the first preset prompt words into the preset large model to obtain a number of target question data. The first preset prompt words include question decomposition prompt words and question expansion prompt words. The target question data includes a number of target questions and a question identifier corresponding to each target question. The target problem processing module is used to extract key information and transform vectors for each target problem, and obtain the target key information list and target problem vector corresponding to each target problem; The first data recall module is used to input the target key information list, target question vector, initial document data and second preset prompt words corresponding to each target question into the preset large model to obtain reference recall data corresponding to each target question. The reference recall data includes several documents and the recall priority corresponding to each document. The target recall path acquisition module is used to input each initial scene category, the preset recall path corresponding to each initial scene category, each target question, the reference recall data corresponding to each target question, and the third preset prompt word into the preset large model to obtain the target recall path corresponding to the initial question. The target recall path includes several ordered data recall steps. The second data retrieval module is used to execute all data retrieval steps in the target retrieval path in sequence to obtain the target retrieval data corresponding to the initial question, wherein the target retrieval data includes several document fragments.

9. A non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores at least one instruction or at least one program, characterized in that, The at least one instruction or the at least one program segment is loaded and executed by the processor to implement the intelligent planning method for large-scale RAG recall strategy as described in any one of claims 1-7.

10. An electronic device, characterized in that, Includes a processor and the non-transitory computer-readable storage medium as described in claim 9.

Images