Adaptive classification retrieval-augmented generation model system and method

Abstract

An adaptive classification retrieval-augmented generation model system and a method, relating to the technical field of artificial intelligence. The system comprises: a query splitting module, a classifier module, a processing module, a user interaction module, and an evaluation and optimization module. The user interaction module is used to receive an original query of a user, and display a generated answer. The query splitting module is used to split the original query received by the user interaction module into multiple equivalent queries. By means of integrating a classifier module, the complexities of different tasks are effectively identified and, corresponding processing methods therefor are determined, so that the understanding and application efficiency of the augmented generation technology is optimized. Advanced RAG technologies and specially adjusted classification algorithms are combined, so that not only is the accuracy of question answering improved, but processing speed is also improved, and consumption of running resources is significantly reduced.

Description

An adaptive classification retrieval enhanced generation model system and method

[0001] The present application claims priority to the Chinese patent application No. 202410953917.7, filed on July 17, 2024, and entitled "An adaptive classification retrieval enhanced generation model system and method", the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] The present application belongs to the field of artificial intelligence technology, and specifically relates to an adaptive classification retrieval enhanced generation model system and method. BACKGROUND

[0003] In the field of intelligent technology and language processing, enhanced language models that integrate non-parametric knowledge have become key tools for information retrieval and knowledge management. These advanced tools focus on parsing complex queries from users and providing accurate and relevant answers. Thanks to advanced retrieval enhancement algorithms such as single-step retrieval and multi-step retrieval, these systems exhibit excellent performance in handling open-domain problems. They can generate coherent and rich answers by combining the inherent parametric memory of the model and external dynamic knowledge.

[0004] Although large language models perform well in handling high-complexity question-answering tasks, they still face many challenges when dealing with diverse user requests in reality. Actual documents often contain tasks of different complexity levels, covering both complex and simple information in user requests. In the past, enhanced language models have difficulty choosing appropriate enhancement algorithms when facing such problems due to the lack of clear guidance strategies. This leads to the model's inability to fully understand the structure and context of the document, affecting its generation efficiency and consuming excessive computing resources. Therefore, traditional models still have deficiencies in terms of accuracy and efficiency of answers. SUMMARY

[0005] The present application adopts the technical solution as follows to solve the problems in the prior art:

[0006] In a first aspect, the present application provides an adaptive classification retrieval enhanced generation model system, characterized in that it comprises a query splitting module, a classifier module, a processing module, a user interaction module, and an evaluation and optimization module.

[0007] The user interaction module is configured to receive the original query from the user and display the generated answer.

[0008] The query splitting module is configured to split the original query received by the user interaction module into multiple equivalent queries.

[0009] The classifier module is used to classify the split equivalent queries, determine their different processing methods, and select the path containing the most equivalent queries as the processing method of the original query based on the judgment result of the classifier;

[0010] The processing module is used to gradually obtain relevant information according to the selected processing method, and finally generate a complete answer;

[0011] The evaluation and optimization module is responsible for collecting and analyzing user feedback and system performance data, evaluating the accuracy and user experience of the system, and making targeted system optimization.

[0012] Further, the user interaction module includes a user interface design unit, a query interpretation and feedback unit, a personalized customization unit, and a multi-terminal adaptation unit;

[0013] The user interface design unit is used to access and operate the system;

[0014] The query interpretation and feedback unit is used to display query results, improvement suggestions, and error prompt information;

[0015] The personalized customization unit is used to customize interface settings and search preferences;

[0016] The multi-terminal adaptation unit is used to enable the system to be used on different terminal devices, and to optimize the interface layout and interaction method according to different screen sizes and operation habits.

[0017] Further, the query splitting module includes a query analysis unit, a synonym and syntax transformation processing unit, a query generation unit, and a query effect evaluation unit;

[0018] The query analysis unit is used to conduct in-depth analysis of the original query of the user, identify the keywords, intent, and context in the query using the hidden Markov model and conditional random field lexical analysis model;

[0019] The synonym and syntax transformation processing unit uses a rich synonym library and syntax transformation rules to rewrite the original query;

[0020] The query generation unit is responsible for generating multiple new query expressions based on the replaced and transformed data;

[0021] The query effect evaluation unit evaluates the effect of the generated query variants.

[0022] Further, the classifier module includes a query complexity analysis unit, a classification processing method unit, and a processing method selection unit;

[0023] The query complexity analysis unit is used to evaluate the content and intent of each query variant and determine its complexity level;

[0024] The classification processing method unit is used to classify the query into four different processing methods including non-retrieval method, single-step retrieval method, multi-step retrieval method and knowledge graph method according to the result of complexity analysis;

[0025] The processing method selection unit selects the method containing the most equivalent queries as the optimal strategy for processing the user's original query based on the judgment result of the classification processing method, and counts the number of query variants matching the user query under each processing method.

[0026] Further, the processing module includes an information acquisition and data retrieval unit, an answer generation unit;

[0027] The information acquisition and data unit is used to extract the original inquiry related documents or knowledge graph from the external database or knowledge base system, to ensure that accurate and relevant information is obtained;

[0028] The answer generation unit uses advanced natural language processing technology to generate specific answers to user queries based on the collected data.

[0029] Further, the evaluation and optimization module includes a performance evaluation unit, an algorithm and process optimization unit, a resource management and scheduling unit, and an automatic optimization strategy unit;

[0030] The performance evaluation unit is responsible for monitoring the performance indicators of the system during operation, such as response time, user experience, and resource utilization;

[0031] The algorithm and process optimization unit accelerates the knowledge integration and reasoning process by improving the data processing flow and algorithm parallel means;

[0032] The resource management and scheduling unit is used to manage the computing resources and storage space required by the system to avoid resource waste;

[0033] The automatic optimization unit enables the system to develop automatic optimization strategies and mechanisms, so as to automatically adjust parameters and configurations according to different scenarios and requirements.

[0034] In a second aspect, the present application provides a working method of the adaptive classification retrieval enhancement generation model system as described in the first aspect, comprising:

[0035] Receiving the user's original query through the user interaction module and displaying the generated answer;

[0036] Splitting the original query accepted by the user interaction module into multiple equivalent queries through the query splitting module;

[0037] The split equivalent queries are classified by the classifier module to determine their different processing methods, and based on the judgment results of the classifier, the path containing the most equivalent queries is selected as the processing method of the original query of the user;

[0038] Through the processing module, relevant information is gradually obtained according to the selected processing method, and a complete answer is finally generated;

[0039] Through the evaluation and optimization module, user feedback and system performance data are collected and analyzed to evaluate the accuracy of the system and user experience, and the system is optimized accordingly.

[0040] Compared with the prior art, the advantages and positive effects of the present application are:

[0041] The present application proposes an adaptive classification retrieval augmented generation system (AC-RAG), which effectively identifies the complexity of different tasks and determines their corresponding processing methods by integrating a classifier module. This method optimizes the understanding and application efficiency of augmented generation technology. The AC-RAG system combines advanced RAG technology and specially adjusted classification algorithms, not only improving the accuracy of question answering, but also improving the processing speed and significantly reducing the consumption of running resources. Therefore, the AC-RAG system provides a more efficient and accurate solution for complex queries and answers in various practical application scenarios. BRIEF DESCRIPTION OF DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced below. Obviously, the drawings in the following description are some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0043] Fig. 1 is a structural schematic diagram of the adaptive classification retrieval augmented generation model system of the present application;

[0044] Fig. 2 is a structural schematic diagram of the user interaction module of the present application;

[0045] Fig. 3 is a structural schematic diagram of the query splitting module of the present application;

[0046] Fig. 4 is a structural schematic diagram of the classifier module of the present application;

[0047] Fig. 5 is a structural schematic diagram of the processing module of the present application;

[0048] Fig. 6 is a structural schematic diagram of the evaluation and optimization module of the present application. DETAILED DESCRIPTION

[0049] In order to enable a more clear understanding of the above-mentioned purposes, features and advantages of the present application, the present application will be further described below in conjunction with the accompanying drawings and examples. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0050] In the following description, many specific details are set forth in order to provide a thorough understanding of the present application, however, the present application can be practiced in different ways from those described herein, and therefore, the present application is not limited to the specific embodiments disclosed in the following description.

[0051] Embodiment 1, as shown in FIGS. 1-6, the present application proposes a system of adaptive classification retrieval augmented generation model (AC-RAG), aiming to optimize the efficiency and accuracy of the question answering system. The core of the AC-RAG system is to combine the retrieval augmented generation (RAG) technology with the fine-tuned large language model (LLM), while adopting a query splitting module to generate multiple similar queries, and introducing a classifier module to determine different query processing methods.

[0052] The AC-RAG system is composed of the following key modules:

[0053] Query splitting module: responsible for generating several similar or equivalent expressions according to the user's original query. This module uses a variety of techniques, including synonym replacement, syntactic transformation, etc., to split the original query into multiple similar queries.

[0054] Classifier module: used to classify the split equivalent queries and determine their different processing methods, including non-retrieval method, single-step retrieval method, multi-step retrieval method and knowledge graph method. Based on the judgment result of the classifier, the path containing the most equivalent queries is selected as the processing method of the user's original query.

[0055] Processing module: according to the selected processing method, relevant information is gradually obtained, and finally a complete answer is generated.

[0056] User interaction module: used to receive the user's original query and display the generated answer.

[0057] Evaluation and optimization module: responsible for collecting user feedback and system performance data. By analyzing these data, the accuracy and user experience of the system are evaluated, and the system is optimized accordingly.

[0058] The main steps are as follows:

[0059] The user inputs an original query into the system, and the query splitting module receives the user's query and generates several equivalent or similar queries using synonym transformation or syntax transformation, etc. These query variants are then sent to the classifier module for classification to determine the different processing methods to which each query belongs. Based on the judgment results of the classifier, the system selects the method containing the most equivalent queries as the processing method of the user's original query. Subsequently, the processing module gradually obtains relevant information according to the selected method to generate the final answer. The user interaction module is the starting and ending point of the process, responsible for receiving the user's original query and presenting the final answer generated by the system. Users can also interact with the system through this module, such as providing feedback or making further queries. The evaluation and optimization module is responsible for collecting user feedback and system performance data, and making optimization suggestions for the system based on the evaluation results. These suggestions will be fed back to all other modules.

[0060] The present application mainly includes different module functions as follows:

[0061] The user interaction module consists of four main parts: user interface design unit, query interpretation and feedback unit, personalized customization unit and multi-terminal adaptation unit. The user interface design unit enables users to easily access and operate the system through the design of an intuitive interface. The query interpretation and feedback unit is responsible for displaying information such as query results, improvement suggestions and error prompts, helping users understand the system's processing method for the query, and allowing users to provide satisfaction feedback on the results. The personalized customization unit supports users to customize interface settings and search preferences according to their own preferences and needs. The multi-terminal adaptation unit ensures that users can conveniently use the system on different terminal devices, and optimizes the interface layout and interaction method according to different screen sizes and operation habits, improving the user experience. See Figure 2.

[0062] The query splitting module is composed of several parts: query analysis unit, synonym and syntax transformation processing unit, query generation unit and query effect evaluation unit. The core task of this module is to generate a series of similar or equivalent query expressions from the user's original query. The query analysis unit conducts in-depth analysis on the user's original query, uses lexical analysis models such as Hidden Markov Model (HMM) and Conditional Random Field (CRF) to identify keywords, intent and context in the query, and provides a basis for subsequent transformation. The synonym and syntax transformation unit processes use a rich synonym library and syntax transformation rules to rewrite the original query, for example, the original query is "How to improve English writing skills?", the keyword "writing skills" can be replaced with "writing ability" using the synonym library, and the syntax "How to improve …?" can be replaced with "What is the method to improve …?" using syntax transformation. The query generation unit is responsible for generating multiple new query expressions based on the data after replacement and transformation, for example, the two new query expressions generated from the above query are "How to improve English writing ability?" and "What is the method to improve English writing skills?" These queries aim to cover as many possible interpretations of the original query as possible, increasing the accuracy of the system's understanding and response to user queries. The query effect evaluation unit finally evaluates the effect of the generated query variants. By calculating the semantic similarity between the variants and the original query and judging by the large model, to ensure that each newly generated query has sufficient relevance to the original query. See Figure 3.

[0063] The classifier module is composed of several key parts: query complexity analysis, classification processing method and processing method selection. This module is mainly responsible for receiving the query variants generated by the query splitting module, and using the class operation model, a small model specially trained, to conduct in-depth analysis and classification. Query complexity analysis assesses the content and intent of each query variant to determine its complexity level. This process involves evaluating the query context, difficulty level and possible processing strategies. The classification processing method section is the core of the module, which classifies queries into four different processing methods including non-retrieval method, single-step retrieval method, multi-step retrieval method and knowledge graph method based on the results of complexity analysis. This model uses advanced machine learning techniques to ensure accurate and efficient classification. The processing method selection section selects the method that contains the most equivalent queries as the optimal strategy for processing the user's original query based on the judgment results of the classification processing method, for example, an original query has ten variants, six of which are classified as single-step retrieval methods, two as multi-step retrieval methods, and two as knowledge graph methods, then select the single-step retrieval method as the optimal strategy for the original query. See Figure 4.

[0064] The processing module is part of the system core, which mainly includes information acquisition and data retrieval unit, answer generation unit and information integration unit. This module starts after the classifier module determines the non-retrieval method, single-step retrieval method, multi-step retrieval method or knowledge graph method as the processing strategy. In the information acquisition and data retrieval unit, the original inquiry related documents or knowledge graph are extracted from the external database or knowledge base system to ensure that accurate and relevant information is obtained. At the same time, the access to multiple data sources is monitored and optimized to improve the efficiency and accuracy of information retrieval. The answer generation unit uses advanced natural language processing technology to generate specific answers to user queries based on the collected data combined with the prompt engineering through large language models. The information integration unit integrates and stores the query, retrieval information and answer, which can be used as historical dialogue. As shown in Figure 5.

[0065] The evaluation and optimization module consists of four main parts: performance evaluation unit, algorithm and process optimization unit, resource management and scheduling unit, and automated optimization strategy unit. The performance evaluation unit is responsible for monitoring the performance indicators of the system during operation, such as response time, user experience, resource utilization, etc. Through real-time analysis, potential performance bottlenecks and problems are identified to provide data support for system optimization. The algorithm and process optimization unit speeds up the knowledge integration and reasoning process by improving data processing processes and algorithm parallelism. The resource management and scheduling unit manages the computing resources and storage space required by the system to avoid resource waste. The automated optimization unit integrates multiple data processing and model training methods to generate multiple modules, allowing the system to develop automated optimization strategies and mechanisms through module combination, thereby automatically adjusting parameters and configurations according to different scenarios and requirements. As shown in Figure 6.

[0066] To better understand the adaptive classification retrieval enhancement generation model (AC-RAG) system of the present application, we take an automated customer service as an example to illustrate its implementation process.

[0067] Query splitting module: receives the initial query proposed by the user and applies a lexical analysis model to identify keywords, intent and context in the query, generating multiple similar or equivalent query versions. For example, the original query "I want to query the order status" is converted into "I want to query the order", "Please help me retrieve the order information" and other variants.

[0068] Classifier module: all generated query variants are sent to the classifier module for classification. The classifier determines the complexity level of the query based on its content and intent, and classifies it into different processing methods, including:

[0069] Non-retrieval method: directly answer the user's query without retrieving information from the database or knowledge base.

[0070] Single-step retrieval method: the answer can be obtained through one retrieval.

[0071] Multi-step retrieval method: multiple retrievals are needed to obtain a complete answer.

[0072] Knowledge graph method: processing method based on knowledge graph.

[0073] Based on the judgment result of the classifier, the system counts the number of query variants matching the user query under each processing method, and selects the method containing the most equivalent queries as the processing method of the user's original query. For example, if the original query "I want to query the order status" is determined to be suitable for the "single-step retrieval method" by classifying the query variants, this method is transmitted to the processing module, and the system proceeds to the next step.

[0074] Processing module: after selecting the processing method, the processing module of the system starts to gradually obtain relevant information. For example, in the case of the user asking about the order status, the system will process the original query according to the "single-step retrieval method", such as querying the order database and returning detailed order information.

[0075] User interaction module: the user submits the query through the user interface, and the system displays the generated answer. The user can provide feedback on the accuracy of the system-generated answer or ask further questions. These feedbacks can be collected through the user interaction module and used to optimize system performance.

[0076] Evaluation and optimization module: the evaluation and optimization module is responsible for collecting user feedback and system performance data, and making optimization suggestions for the system based on the evaluation results. These optimization suggestions may include improving query splitting algorithms, optimizing classifier performance, or updating knowledge base content.

[0077] Through the coordinated work of the above modules, the AC-RAG system can realize automated customer service, improve response speed and problem-solving accuracy, and reduce the burden on the artificial customer service team.

[0078] The above is only the preferred embodiment of the present application, and does not limit the form of the present application. Any skilled person in the art can use the disclosed technical content to make changes or modifications as equivalent embodiments applied to other fields, but any simple modification, equivalent change and modification made according to the technical essence of the present application to the above embodiments shall fall within the protection scope of the present application.

Claims

1. A self-adapting category search enhanced generative model system, characterized by, The application relates to a system and method for processing user queries, comprising: a query splitting module, a classifier module, a processing module, a user interaction module and an evaluation and optimization module; the user interaction module is used for receiving a user's original query and displaying a generated answer; the query splitting module is used for splitting the original query accepted by the user interaction module into multiple equivalent queries; the classifier module is used for classifying the split equivalent queries, determining different processing methods thereof, and selecting a path containing the most equivalent queries as a processing method of the user's original query based on a judgment result of the classifier; the processing module is used for gradually obtaining relevant information according to the selected processing method, and finally generating a complete answer; the evaluation and optimization module is responsible for collecting and analyzing user feedback and system performance data, evaluating the accuracy and user experience of the system, and performing targeted system optimization.

2. The adaptive categorical retrieval boosted generation model system of claim 1, wherein, The user interaction module comprises a user interface design unit, a query interpretation and feedback unit, a personalized customization unit and a multi-terminal adaptation unit; the user interface design unit is used for accessing and operating the system; the query interpretation and feedback unit is used for displaying query results, improvement suggestions and error prompt information; the personalized customization unit is used for customizing interface settings and search preferences; the multi-terminal adaptation unit is used for enabling the system to be used on different terminal devices, and optimizing the interface layout and interaction mode according to different screen sizes and operation habits.

3. The adaptive categorical retrieval boosted generation model system of claim 1, wherein, The query splitting module comprises a query analysis unit, a synonym and syntax transformation processing unit, a query generation unit and a query effect evaluation unit; the query analysis unit is used for deeply analyzing the user's original query, identifying keywords, intentions and contexts in the query by using a hidden Markov model and a conditional random field lexical analysis model; the synonym and syntax transformation processing unit uses a rich synonym library and syntax transformation rules to rewrite the original query; the query generation unit is responsible for generating multiple new query expressions based on the replaced and transformed data; the query effect evaluation unit evaluates the effects of the generated query variants.

4. The adaptive categorical retrieval boosted generation model system of claim 1, wherein, The classifier module comprises a query complexity analysis unit, a classification processing method unit and a processing method selection unit; the query complexity analysis unit is used for evaluating the content and intention of each query variant and determining the complexity level thereof; the classification processing method unit is used for classifying the query into four different processing methods, including a non-retrieval method, a single-step retrieval method, a multi-step retrieval method and a knowledge graph method, according to a result of complexity analysis; the processing method selection unit selects a method containing the most equivalent queries as an optimal strategy for processing the user's original query based on a judgment result of the classification processing method.

5. The adaptive categorical retrieval boosted generative model system of claim 1, wherein, The processing module comprises an information acquisition and data retrieval unit and an answer generation unit; the information acquisition and data unit is used for extracting original query related documents or knowledge graphs from external databases or knowledge base systems, and ensuring that accurate and relevant information is obtained; the answer generation unit generates specific answers to the user's query by using advanced natural language processing technology according to the collected data.

6. The adaptive categorical retrieval boosted generative model system of claim 1, wherein, The evaluation and optimization module includes a performance evaluation unit, an algorithm and process optimization unit, a resource management and scheduling unit, and an automated optimization strategy unit. The performance evaluation unit is responsible for monitoring the performance indicators of the system during operation, such as response time, user experience, and resource utilization. The algorithm and process optimization unit improves data processing flow and algorithm parallelism to accelerate knowledge integration and reasoning process. The resource management and scheduling unit is used to manage the computing resources and storage space required by the system, avoiding resource waste. The automated optimization unit enables the system to develop automated optimization strategies and mechanisms, automatically adjusting parameters and configurations according to different scenarios and requirements.

7. A method of operating a self-adapting categorization search enhanced generative model system according to any of claims 1-6, characterized by, It includes: Through the user interaction module, the original query of the user is received, and the generated answer is displayed. Through the query splitting module, the original query accepted by the user interaction module is split into multiple equivalent queries. Through the classifier module, the split equivalent queries are classified to determine their different processing methods, and based on the judgment results of the classifier, the path containing the most equivalent queries is selected as the processing method of the user's original query. Through the processing module, relevant information is gradually obtained according to the selected processing method, and finally a complete answer is generated. Through the evaluation and optimization module, user feedback and system performance data are collected and analyzed to evaluate the accuracy and user experience of the system, and targeted system optimization is carried out.

Images