Multi-source heterogeneous data multi-mode hybrid retrieval method and system based on large model reasoning

Abstract

The present invention belongs to the field of data processing technology. A multi-source heterogeneous data multi-mode hybrid retrieval method and system based on large model reasoning is proposed. In each step of the reasoning process, the large language model determines the sub-query questions and corresponding sub-query answers of different retrieval modes according to the initial query question; according to the sub-query questions and sub-query answers corresponding to the query mode of each step of the reasoning process, the optimal retrieval path is adaptively determined with the maximum reward function as the goal, and the final answer corresponding to the initial query question is determined according to the optimal retrieval path; wherein, the optimal retrieval path includes: the optimal answer of each retrieval mode and the retrieval order of each retrieval mode, and the retrieval mode includes database retrieval, semantic retrieval and graph retrieval; while ensuring retrieval efficiency, the present invention supports multi-hop reasoning of complex queries, realizes the dynamic fusion of multiple types of retrieval modes, and greatly improves retrieval efficiency and retrieval accuracy.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a multi-source heterogeneous data multi-mode hybrid retrieval method and system based on large model reasoning. Background Art

[0002] The statements in this section merely provide background art related to the present invention and do not necessarily constitute prior art.

[0003] As digital transformation deepens across industries, the generation and accumulation of domain data is exploding. This data is not only massive in scale, but also comes from diverse sources and structures, resulting in large-scale, multi-source, heterogeneous data. Effectively retrieving, analyzing, and utilizing this multi-source, heterogeneous data presents a challenge for all industries.

[0004] Existing technologies face multiple difficulties when processing multi-source heterogeneous data: the lack of a unified processing mechanism for different data formats makes it impossible to effectively support queries for complex problems; the huge amount of data and complex relationships result in poor accuracy and low efficiency in multi-source heterogeneous data retrieval; for example, although the information retrieval method based on RAG (Retrieval-Augmented Generation) can effectively obtain real and relevant content in the knowledge base and promptly feedback data changes and new content, when processing multi-source heterogeneous data, its computing and storage resources are consumed enormously. In addition, due to the large amount of multi-source heterogeneous data and its dispersed sources, it is easy to lead to low RAG retrieval efficiency and poor accuracy; for another example, the information retrieval method based on knowledge graph enhances the reasoning ability of large language models in multi-hop question answering tasks, but still faces challenges when processing multi-source heterogeneous data. It lacks an effective fusion mechanism for data types other than unstructured text (such as relational databases, time series sensor data, semi-structured logs, etc.). Summary of the Invention

[0005] In order to address the shortcomings of the existing technology, the present invention provides a multi-source heterogeneous data multi-mode hybrid retrieval method and system based on large model reasoning. By designing a hybrid retrieval architecture driven by large language model reasoning, while ensuring retrieval efficiency, it supports multi-hop reasoning of complex queries, realizes the dynamic fusion of multiple types of retrieval modes, and greatly improves retrieval efficiency and retrieval accuracy.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] In a first aspect, the present invention provides a multi-source heterogeneous data multi-mode hybrid retrieval method based on large model reasoning.

[0008] A multi-source heterogeneous data multi-mode hybrid retrieval method based on large model reasoning includes the following steps:

[0009] Obtain the user's initial query. During each step of the reasoning process, the large language model determines subqueries and corresponding subquery answers based on the initial query.

[0010] Based on the sub-query questions and sub-query answers corresponding to the query patterns in each step of the reasoning process, the optimal retrieval path is adaptively determined with the goal of maximizing the reward function, and the final answer corresponding to the initial query question is determined based on the optimal retrieval path; the optimal retrieval path includes: the optimal answer of each retrieval pattern and the retrieval order of each retrieval pattern. The retrieval patterns include database retrieval, semantic retrieval, and graph retrieval.

[0011] In a second aspect, the present invention provides a multi-source heterogeneous data multi-mode hybrid retrieval system based on large model reasoning.

[0012] A multi-source heterogeneous data multi-mode hybrid retrieval system based on large model reasoning, including:

[0013] The pattern retrieval unit is configured to: obtain the user's initial query question, and in each step of the reasoning process, the large language model determines sub-query questions and corresponding sub-query answers for different retrieval patterns based on the initial query question;

[0014] The path optimization unit is configured to adaptively determine the optimal search path based on the sub-query questions and sub-query answers corresponding to the query patterns of each step of the reasoning process, with the goal of maximizing the reward function, and determine the final answer corresponding to the initial query question based on the optimal search path; wherein the optimal search path includes: the optimal answer of each search pattern and the search order of each search pattern, and the search patterns include database search, semantic search, and graph search.

[0015] In a third aspect, the present invention provides a computer device comprising: a processor and a computer-readable storage medium;

[0016] a processor adapted to execute a computer program;

[0017] Computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the multi-source heterogeneous data multi-mode hybrid retrieval method based on large model reasoning as described in the first aspect of the present invention is implemented.

[0018] In a fourth aspect, the present invention provides a computer-readable storage medium storing a computer program, wherein the computer program is suitable for being loaded by a processor and executing a multi-source heterogeneous data multi-mode hybrid retrieval method based on large model reasoning as in the first aspect of the present invention.

[0019] Compared with the prior art, the present invention has the following beneficial effects:

[0020] 1. The present invention innovatively proposes a multi-source heterogeneous data multi-mode hybrid retrieval method based on large-model reasoning. According to the sub-query questions and sub-query answers corresponding to the query patterns in each step of the reasoning process, the optimal retrieval path is adaptively determined with the goal of maximizing the reward function. The final answer corresponding to the initial query question is determined based on the optimal retrieval path. By designing a hybrid retrieval architecture driven by large language model reasoning, while ensuring retrieval efficiency, it supports multi-hop reasoning of complex queries, realizes the dynamic fusion of multiple types of retrieval modes, and greatly improves retrieval efficiency and retrieval accuracy.

[0021] 2. The present invention innovatively proposes a multi-source heterogeneous data multi-mode hybrid retrieval method based on large-scale model reasoning. The reward function is designed by integrating cumulative rewards, call times and historical status. The parameter c is dynamically adjusted to balance the exploration and utilization of multiple types of retrieval modes. The optimal retrieval path is determined by taking the maximum reward function as the goal. It can adaptively allocate the order and answer weight of each retrieval mode in database retrieval, semantic retrieval and graph retrieval modes according to actual conditions, thereby more accurately determining the final answer corresponding to the initial query question, effectively improving the accuracy and efficiency of the retrieval, and providing users with better retrieval services.

[0022] 3. The present invention innovatively proposes a multi-source heterogeneous data multi-mode hybrid retrieval method based on large-scale model reasoning. The state space clearly defines the various states of the reasoning process, which is convenient for tracking and managing the retrieval process; the action space clarifies four types of reasoning actions, so that the retrieval operation has a clear direction, such as database retrieval answers, which can accurately obtain different types of data; the state transition rules ensure the consistency and logic of the retrieval process, and can reasonably update to the next state after executing the action based on the current state; combined with the reward function, it can adaptively screen out the optimal retrieval path from many possible retrieval paths with the goal of maximizing the reward, determine the optimal answer and order of each retrieval mode, thereby efficiently integrating multi-source heterogeneous data, providing users with accurate and comprehensive final answers, and improving retrieval quality and user experience.

[0023] 4. The present invention innovatively proposes a multi-source heterogeneous data multi-mode hybrid retrieval method based on large model reasoning, and adopts targeted processing methods for different retrieval modes. When answering database retrieval, it generates query statements and summarizes and integrates the content with the help of a large language model, ensuring the accuracy and efficiency of database data retrieval; when answering semantic retrieval, it accesses cross-modal semantic information and uses semantic vector retrieval to enhance the depth and breadth of semantic understanding and improve the quality of semantic retrieval; when answering graph retrieval, it uses a subgraph clustering algorithm to cluster the knowledge graph, which can accurately retrieve information related to the semantics of the subquery question; the final answer is generated by a large language model combined with historical reasoning information, making full use of the information of the entire reasoning process to ensure the comprehensiveness and accuracy of the answer. The present invention can adaptively determine the optimal retrieval path, effectively integrate multi-source heterogeneous data, and provide users with more accurate and comprehensive retrieval results, greatly improving the retrieval experience.

[0024] 5. The present invention innovatively proposes a multi-source heterogeneous data multi-mode hybrid retrieval method based on large model reasoning. It uses reinforcement learning algorithm to train and optimize the large language model, making it more intelligent and accurate in generating sub-query questions, sub-query results and final answers. It can dynamically adjust the retrieval strategy according to different query scenarios and needs, better adapt to the complex and changing data environment, and provide users with final answers that are more in line with their needs, more comprehensive and accurate, greatly improving the user experience and the practicality of the retrieval system.

[0025] 6. This invention innovatively proposes a multi-mode hybrid retrieval method for multi-source heterogeneous data based on large-model reasoning. When determining the optimal retrieval path with the goal of maximizing the reward function, preference pairs can assist the large language model in more accurately weighing the pros and cons of various retrieval modes among numerous possible retrieval paths, rationally determining the retrieval order, and selecting the optimal answer. This enables the retrieval system to better adapt to complex multi-source heterogeneous data environments, fully leveraging the characteristics of different retrieval modes to provide users with more tailored and accurate final answers, significantly improving retrieval quality and efficiency.

[0026] Advantages of additional aspects of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The accompanying drawings, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0028] Figure 1 A schematic diagram of an inference-based hybrid retrieval architecture provided by an exemplary embodiment of the present invention;

[0029] Figure 2A schematic diagram of a multi-source heterogeneous data multi-mode hybrid retrieval system based on large model reasoning provided by an exemplary embodiment of the present invention;

[0030] Figure 3 A schematic diagram of a computer device is provided for an exemplary embodiment of the present invention. DETAILED DESCRIPTION

[0031] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0032] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present invention belongs.

[0033] In response to the current problems of insufficient cross-modal data semantic fusion and low retrieval efficiency in multi-source heterogeneous data retrieval, namely the fragmented retrieval methods of graph databases (knowledge graphs), vector databases (natural language embedding representations), and relational databases (structured data), a multi-structure collaborative retrieval mechanism needs to be established. In view of this, this implementation method proposes a multi-source heterogeneous data multi-modal hybrid retrieval method based on large model reasoning. By designing a hybrid retrieval architecture driven by large language model reasoning, it supports multi-hop reasoning for complex queries while ensuring retrieval efficiency, breaking through the technical difficulty of dynamic fusion of multi-type retrieval modes that needs to be solved urgently. The multi-type retrieval mode hybrid retrieval architecture based on large model reasoning uses Monte Carlo tree search and reinforcement learning technology to deeply integrate database retrieval, semantic retrieval, and graph retrieval modes, providing technical support for the in-depth utilization of multi-source heterogeneous data.

[0034] More specifically, we deeply integrate multi-type retrieval modes into the information retrieval and utilization system based on the large language model, and build an inference-based hybrid retrieval architecture, such as Figure 1As shown, orange represents database retrieval, yellow represents semantic retrieval, green represents graph retrieval, and blue represents the final answer, realizing a closed-loop retrieval architecture that includes full-scenario business query data set preparation, retrieval path search, and reinforcement learning reasoning path planning. It is mainly composed of database retrieval, semantic retrieval, graph retrieval, and reasoning hybrid retrieval architecture. Through path planning based on Monte Carlo tree search, it comprehensively explores multi-type information hybrid retrieval paths for user natural language queries, and realizes coverage of the needs of business full scenarios and user query multi-type information retrieval and utilization. The present invention solves the problem of retrieval method separation of knowledge graphs, vector libraries, and structured databases by designing a multi-type retrieval mode hybrid retrieval architecture based on large model reasoning, supports the coordinated optimization of multi-hop reasoning and semantic similarity calculation under logical rule constraints, realizes accurate matching of complex queries, and improves the efficiency and explainability of heterogeneous knowledge services.

[0035] In this implementation, a multi-type retrieval mode is constructed, structured query statements are generated based on the NL2SQL model, cross-modal semantic retrieval is realized using RAG technology, and the interpretability of graph retrieval is improved by combining a keyword-driven subgraph clustering algorithm. Through a retrieval path planning strategy based on Monte Carlo Tree Search (MCTS), multi-type retrieval modes and final answers are used as search actions, and the optimal retrieval path is explored through a "simulation-expansion-backward" process. In addition, a reinforcement learning algorithm is used to train and optimize the model's inference retrieval capabilities, realizing efficient retrieval and intelligent reasoning of multi-source heterogeneous data.

[0036] In terms of multi-type retrieval mode, the retrieval difficulties covered by the constructed multi-level knowledge base system are handled from the perspective of different types of data retrieval, including: database retrieval, semantic retrieval and graph retrieval; in the database retrieval mode, the NL2SQL model is used to generate database query statements, and the query content is summarized and integrated; in the semantic retrieval mode, multi-source heterogeneous data cross-modal semantic information is accessed, such as document relations, graph types, text corpus and chart descriptions, and advanced methods based on RAG technology are used to realize retrieval and analysis of cross-modal semantics; in the graph retrieval mode, a subgraph clustering algorithm combined with keyword retrieval is proposed based on the constructed multi-source heterogeneous data spatiotemporal knowledge graph to retrieve knowledge graph information related to the query semantics, thereby improving the integrity and interpretability of the retrieval content.

[0037] In terms of inferential hybrid retrieval architectures, efficiently and accurately integrating multi-type retrieval modes and large-model answer generation for complex multi-source heterogeneous data retrieval tasks is key to improving overall retrieval effectiveness. To this end, this paper deeply integrates multi-type retrieval modes into an information retrieval and utilization system based on a large language model, and proposes an inferential path optimization strategy for multi-source heterogeneous data retrieval, enabling intelligent information retrieval for full-scenario business queries.

[0038] The present invention defines the multi-source heterogeneous data retrieval process based on a large language model as a Markov decision process, which consists of a state space (S), an action space (A), a state transition (P) and a reward function (R).

[0039] Represents the state space, each state:

[0040] (1);

[0041] The above formula represents the reasoning process based on the large language model, where For the initial query, Respectively represent Step 1: Subquery questions and subquery answers under the specified type of retrieval mode generated by the large model reasoning; They represent the subquery question and the subquery answer under the specified type of retrieval mode generated by the large model inference in step 1 respectively; They represent the subquery question and the subquery answer under the specified type of retrieval mode generated by the large model inference in step 2 respectively; Respectively represent The subquery questions and subquery answers under the specified type of retrieval mode are generated by the large model reasoning.

[0042] A represents the action space. The present invention integrates multiple types of retrieval modes to define four types of reasoning actions, including: database retrieval answer, semantic retrieval answer, graph retrieval answer and final answer.

[0043] Indicates state transition, based on Step Status Execute an action After the operation, the environment will update the status:

[0044] (2);

[0045] in, Representing the The sub-query questions and sub-query answers under the specified type of retrieval mode are generated by the large model reasoning. Representative Step status.

[0046] when When searching for answers in the database, it is in database search mode. The NL2SQL model is called to generate database query statements, and the large language model summarizes and integrates the query content to generate Answer;

[0047] when When answering for semantic retrieval, it is a semantic retrieval mode, which accesses cross-modal semantic information of multi-source heterogeneous data such as document relations, graph types, text corpus and graph descriptions, and uses RAG-based semantic vector retrieval technology to realize retrieval analysis of cross-modal semantics, and generates a large language model Answer;

[0048] when When answering graph retrieval, it is the graph retrieval mode. The subgraph clustering algorithm combined with keyword retrieval is used to perform cluster analysis on the constructed multi-source heterogeneous data spatiotemporal knowledge graph, retrieve the knowledge graph information related to the subquery semantics, and generate the large language model. Answer;

[0049] when For the final answer, The final answer is generated by a large language model combined with all the information generated during the historical reasoning process.

[0050] Represents the reward function, which has the following form:

[0051] (3);

[0052] in, Indicates the cumulative rewards of the large language model under different types of retrieval modes. Indicates the number of calls of different types of retrieval modes, Indicates the number of calls to the previous state retrieval mode; is a constant that controls the trade-off between exploration and exploitation in retrieval mode.

[0053] Finally, the large language model repeatedly simulates the decision paths of different types of retrieval modes. By dynamically focusing on retrieval path branches with high reward returns, it can adaptively determine when retrieval is needed and which retrieval model to adopt based on user query questions, reducing unnecessary retrieval processes and improving information retrieval performance.

[0054] In this implementation, a multi-source heterogeneous data retrieval dataset is constructed based on specific business scenarios, and the training dataset, validation dataset, and test dataset are divided into a ratio of 7:1:2. The above process search is performed on the training dataset to obtain an adaptive retrieval mode selection path generated by large model reasoning for a specific multi-source heterogeneous data retrieval problem. This path contains the optimal answer strategy for different types of retrieval modes, determining whether retrieval is required and what type of retrieval is required. Based on the reasoning retrieval process data, the present invention constructs a preference pair for each sub-query problem to indicate the optimal action selection. Through a preference learning process based on reinforcement learning, the reasoning retrieval capability of the large language model integrating multiple types of retrieval modes is improved.

[0055] It can be seen that through the above implementation method, the present invention constructs three retrieval modes covering database retrieval, semantic retrieval and graph retrieval to cope with the complex retrieval needs in multi-source heterogeneous data scenarios; designs the NL2SQL dynamic query module to convert natural language queries into precise structured database query statements to realize database retrieval automation; uses RAG technology to realize cross-modal semantic retrieval, integrating multi-source heterogeneous data semantic information such as document relations, graph types, text corpus and chart descriptions; designs a subgraph clustering algorithm based on keyword retrieval to extract highly interpretable subgraph structures and calculate semantic relevance, and accurately retrieves knowledge graph information related to the query semantics; by analyzing the complexity and challenges of multi-source heterogeneous data retrieval tasks, it is converted into a Markov decision process, and the four core elements of state space, action space, state transition and reward function are clarified; starting from the user's initial query question, the subquery questions generated by the large model at each step of reasoning and their answers under the corresponding retrieval mode are incorporated into the state space, each state node records the current retrieval progress, and provides a key reference for subsequent reasoning directions.

[0056] The present invention defines the transition logic from the current state to the next state. The database retrieval action triggers the NL2SQL model to generate and execute SQL statements. The results are semantically integrated by the large language model and the state is updated. The semantic retrieval action activates the vector retrieval process of multi-source heterogeneous data and generates candidate answers through semantic similarity matching. The graph retrieval action uses the structured characteristics of the knowledge graph to locate relevant knowledge nodes through path search and cluster analysis. The present invention integrates the reward function formula of cumulative rewards, number of calls and historical states, and dynamically adjusts the parameters. Balance the exploration and utilization of multi-type retrieval modes; use reinforcement learning algorithms to train and optimize the model's reasoning retrieval capabilities to ensure that large language models generate reliable sub-queries, intermediate results, and final answers.

[0057] In this implementation, a Monte Carlo tree is constructed with different search states as nodes and search actions (such as selecting a certain search mode) as edges. During the search process, operations are performed according to the process of selection, expansion, simulation, and backtracking to explore different search paths, aiming to maximize the reward function and obtain the optimal search path.

[0058] It is understandable that in some other implementations, the similarity between the subquery and the query result can also be introduced into the reward function, specifically including:

[0059] (4);

[0060] in, is the updated reward function, Representative subqueries, Representative subquery results, represents the cross-modal semantic similarity weight, Representative Subqueries With Representative No. Subquery results similarity.

[0061] By incorporating cross-modal semantic matching into the reward function of reinforcement learning, the reward not only focuses on the balance of retrieval mode calls, but also on the "semantic adaptation quality of retrieval results and queries", guiding the model to prioritize retrieval paths with high semantic matching. By associating rewards with semantic quality, the model is guided to select high-matching retrieval paths, thereby enhancing the adaptability of multimodal retrieval to complex semantics.

[0062] Optionally, in some other implementations, when reinforcement learning selects a retrieval mode, the mode call threshold is dynamically adjusted based on query complexity and historical modality performance to make the mode selection more suitable for the retrieval scenario and avoid the poor adaptability of a fixed threshold. Specifically, the following methods are used:

[0063] (5);

[0064] (6);

[0065] in, represents the query at step t, Representative Step 1 Dynamic call threshold for each retrieval mode, Representative The basic threshold of the retrieval mode, represents the threshold adjustment coefficient, Represents the current query complexity, Represents the historical maximum query complexity, Representative The historical accuracy of the retrieval mode, Representation status The next Search Mode Action value, hour, This means selecting Through real-time adjustment of dynamic thresholds, the retrieval mode can be more closely aligned with the query characteristics and modal performance, ensuring multi-modal collaboration capabilities in complex scenarios while improving retrieval efficiency in simple scenarios. Represents other situations, i.e. Less than situation.

[0066] Optionally, other implementations can quantify the information gain and path derivation clarity of each modality in multi-hop reasoning, transforming the reasoning process from a "black box" to an "explainable" one, thereby enhancing the business trust and review value of heterogeneous knowledge services. Specifically, these include:

[0067] (7);

[0068] (8);

[0069] in, Representative Step 1 of the search The results of the search mode are The information gain contribution of represents the explainability score of the reasoning path, Representative Subquery and query results The correlation, Represents the number of retrieval steps, used to measure the Step 1 of the search Search mode results For the final query The information gain of

[0070] When a certain reasoning path When it is greater than the set threshold, it is considered to be a highly interpretable path. The path clarity can be quantified, making the "contribution of each step" of reasoning traceable and enhancing the credibility of the results when the business is implemented.

[0071] Optionally, in some other implementations, a comprehensive evaluation model integrating accuracy, coverage, and timeliness is constructed to quantitatively evaluate the quality of hybrid search results. When the evaluation does not meet the standards, a feedback search is triggered. The accuracy of the final result is improved through iterative optimization, forming a closed loop of "search-evaluation-feedback". Specifically, the evaluation score is:

[0072] (9);

[0073] in, represents the comprehensive evaluation score of the mixed retrieval results in step t, Represents the accuracy of the result, represents the result coverage, Represents the search time. 、 and are weight coefficients respectively.

[0074] When the calculated evaluation score is lower than the set threshold, feedback adjustment is required to improve the final score. For example, when the accuracy is insufficient, dynamic threshold control is performed and the weight of the cross-modal semantic reward function is adjusted; when the coverage is insufficient, feedback adjustment is performed. Locate subqueries with "information gain contribution of 0" (for example, a subquery does not retrieve valid results), and supplement the modal retrieval of the subquery in a targeted manner (such as calling graph retrieval to supplement related information); when it takes too long, optimize the call of the retrieval mode by dynamically adjusting the call threshold and explainability score.

[0075] Figure 2 A multi-source heterogeneous data multi-mode hybrid retrieval system based on large model reasoning is shown, including:

[0076] The pattern retrieval unit 201 is configured to: obtain the user's initial query question, and in each step of the reasoning process, the large language model determines sub-query questions and corresponding sub-query answers of different retrieval modes based on the initial query question;

[0077] The path optimization unit 202 is configured to adaptively determine an optimal search path based on the sub-query questions and sub-query answers corresponding to the query patterns of each step of the reasoning process, with the goal of maximizing the reward function, and determine the final answer corresponding to the initial query question based on the optimal search path; wherein the optimal search path includes: the optimal answer of each search pattern and the search order of each search pattern, and the search patterns include database search, semantic search, and graph search.

[0078] The present invention innovatively constructs a multi-source heterogeneous data multi-mode hybrid retrieval architecture based on large-scale model reasoning, deeply integrates multi-type data retrieval modes, and proposes a dynamic path planning strategy based on Monte Carlo tree search and reinforcement learning. It solves the problem of the separation of retrieval methods of knowledge graphs, vector libraries and structured databases, supports the collaborative optimization of multi-hop reasoning and semantic similarity calculation under logical rule constraints, realizes accurate matching of complex queries, enhances the interpretability of heterogeneous knowledge services, and significantly improves the retrieval efficiency and multi-step reasoning capabilities in complex query scenarios.

[0079] It is understandable that each of the above-mentioned units can be separately or completely combined into one or several other units to form a unit, or one (or some) of the units can be further divided into multiple functionally smaller units to form a unit, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present application. The above-mentioned units are divided based on logical functions. In actual applications, the functions of one unit can also be implemented by multiple units, or the functions of multiple units can be implemented by one unit. In other embodiments of the present application, the system may also include other units. In actual applications, these functions can also be implemented with the assistance of other units and can be implemented by the collaboration of multiple units.

[0080] According to another embodiment of the present application, the system described in this embodiment can be constructed by running a computer program (including program code) capable of executing the steps involved in the corresponding method of the present invention on a general-purpose computing device such as a computer, which includes processing elements and storage elements such as a central processing unit (CPU), random access memory (RAM), and read-only memory (ROM). The computer program can be recorded on, for example, a computer-readable recording medium, and loaded into the above-mentioned computing device through the computer-readable recording medium and run therein.

[0081] Figure 3 A computer device is shown, which includes a processor 301, a communication interface 302, and a computer-readable storage medium 303. The processor 301, the communication interface 302, and the computer-readable storage medium 303 may be connected via a bus or other means.

[0082] Among them, the communication interface 302 is used to receive and send data, the computer-readable storage medium 303 can be stored in the memory of the electronic device, the computer-readable storage medium 303 is used to store computer programs, the computer programs include program instructions, and the processor 301 is used to execute the program instructions stored in the computer-readable storage medium 303.

[0083] The processor 301 is the computing core and control core of the electronic device, which is suitable for implementing one or more instructions, and is specifically suitable for loading and executing one or more instructions to implement corresponding method processes or corresponding functions.

[0084] The processor 301 is configured to perform the following process:

[0085] Obtain the user's initial query question. During each step of the reasoning process, the large language model determines subqueries and corresponding subquery answers for different retrieval modes based on the initial query question.

[0086] Based on the sub-query questions and sub-query answers corresponding to the query patterns of each step of the reasoning process, the optimal search path is adaptively determined with the goal of maximizing the reward function, and the final answer corresponding to the initial query question is determined based on the optimal search path; wherein, the optimal search path includes: the optimal answer of each search pattern and the search order of each search pattern, and the search modes include database search, semantic search and graph search.

[0087] The present invention also provides a computer-readable storage medium, which is a memory device in an electronic device for storing programs and data. It is understood that the computer-readable storage medium herein may include both built-in storage media in the electronic device and, of course, extended storage media supported by the electronic device. The computer-readable storage medium provides storage space that stores the processing system of the electronic device.

[0088] Furthermore, the storage space also stores one or more instructions suitable for being loaded and executed by the processor. These instructions may be one or more computer programs (including program code). It should be noted that the computer-readable storage medium herein may be a high-speed RAM memory or a non-volatile memory, such as at least one disk storage device; alternatively, it may be at least one computer-readable storage medium located remotely from the processor.

[0089] In one embodiment, the computer-readable storage medium stores one or more instructions; the processor loads and executes the one or more instructions stored in the computer-readable storage medium to implement the following process:

[0090] Obtain the user's initial query question. During each step of the reasoning process, the large language model determines subqueries and corresponding subquery answers for different retrieval modes based on the initial query question.

[0091] Based on the sub-query questions and sub-query answers corresponding to the query patterns of each step of the reasoning process, the optimal search path is adaptively determined with the goal of maximizing the reward function, and the final answer corresponding to the initial query question is determined based on the optimal search path; wherein, the optimal search path includes: the optimal answer of each search pattern and the search order of each search pattern, and the search modes include database search, semantic search and graph search.

[0092] The present invention also provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the electronic device to perform the following process:

[0093] Obtain the user's initial query question. During each step of the reasoning process, the large language model determines subqueries and corresponding subquery answers for different retrieval modes based on the initial query question.

[0094] Based on the sub-query questions and sub-query answers corresponding to the query patterns of each step of the reasoning process, the optimal search path is adaptively determined with the goal of maximizing the reward function, and the final answer corresponding to the initial query question is determined based on the optimal search path; wherein, the optimal search path includes: the optimal answer of each search pattern and the search order of each search pattern, and the search modes include database search, semantic search and graph search.

[0095] Those skilled in the art will appreciate that the units and algorithmic steps of each example described in conjunction with the embodiments disclosed in this application can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional technical personnel may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0096] The above embodiments can be implemented in whole or in part using software, hardware, firmware, or any combination thereof. When implemented using software, they can be implemented in whole or in part in the form of a computer program product. A computer program product comprises one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions according to the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted via a computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, optical fiber, digital line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that can be accessed by a computer, or a data processing device such as a server or data center that integrates one or more available media. Available media can include magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives).

[0097] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A multi-source heterogeneous data multi-mode hybrid retrieval method based on large model reasoning, characterized by: The following processes are included: Obtain the user's initial query question. During each step of the reasoning process, the large language model determines subqueries and corresponding subquery answers for different retrieval modes based on the initial query question. Based on the subquery questions and subquery answers corresponding to the query patterns in each step of the reasoning process, an optimal search path is adaptively determined with the goal of maximizing the reward function, and a final answer corresponding to the initial query question is determined based on the optimal search path; wherein the optimal search path includes: the optimal answer of each search pattern and the search order of each search pattern, and the search patterns include database search, semantic search, and graph search; When selecting a retrieval mode in reinforcement learning, the mode call threshold is dynamically adjusted based on query complexity and historical modality performance. Specifically: ； ； in, represents the query at step t, Representative Step 1 Dynamic call threshold for each retrieval mode, Representative The basic threshold of the retrieval mode, represents the threshold adjustment coefficient, Represents the current query complexity, Represents the historical maximum query complexity, Representative The historical accuracy of the retrieval mode, Representation status The next Search Mode action value; Quantify the information gain and path inference clarity of each modal result in multi-hop reasoning, specifically: ； ； in, Representative Step 1 of the search The results of the search mode are The information gain contribution of represents the explainability score of the reasoning path, Representative Subquery and query results The correlation, Represents the number of retrieval steps, used to measure the Step 1 of the search Search mode results For the final query The information gain of 2. The multi-source heterogeneous data multi-mode hybrid retrieval method based on large model reasoning according to claim 1 is characterized in that: The reward function is: ,in, Represents the cumulative rewards of the large language model when selecting different types of retrieval modes; Represents the number of calls for different types of retrieval modes; Represents the number of calls to the previous state retrieval mode; Represents a constant.

3. The multi-source heterogeneous data multi-mode hybrid retrieval method based on large model reasoning according to claim 1 is characterized in that: assumed Represents the state space, each state Represents the reasoning process based on a large language model, For the initial query, Respectively represent Step 1: Subquery questions and subquery answers under the specified type of retrieval mode generated by the large model reasoning; assumed Represents the action space, which includes four types of reasoning actions: database retrieval answer, semantic retrieval answer, graph retrieval answer and final answer; assumed Indicates state transition, based on state Execute an action After the operation, the status is updated to ,in, Representing the The subquery questions and subquery answers under the specified type of retrieval mode are generated by the large model inference step.

4. The multi-source heterogeneous data multi-mode hybrid retrieval method based on large model reasoning according to claim 3 is characterized in that: when When searching for answers in the database, a database query statement is generated, and the query content is summarized and integrated through the large language model to generate Answer; when When answering semantic retrieval, we access the cross-modal semantic information of multi-source heterogeneous data, use semantic vector retrieval to perform cross-modal semantic retrieval, and generate Answer; when When answering graph retrieval, we use the subgraph clustering algorithm combined with keyword retrieval to cluster the constructed multi-source heterogeneous data spatiotemporal knowledge graph, retrieve the knowledge graph information related to the semantics of the subquery question, and generate the Answer.

5. The multi-source heterogeneous data multi-mode hybrid retrieval method based on large model reasoning according to claim 3 or 4 is characterized in that: when For the final answer, The final answer is generated by a large language model combined with all the information generated during the historical reasoning process.

6. The multi-source heterogeneous data multi-mode hybrid retrieval method based on large model reasoning according to claim 3 or 4 is characterized in that: A reinforcement learning algorithm is used to train and optimize the large language model so that the large language model can generate sub-query questions, sub-query results and final answers.

7. The multi-source heterogeneous data multi-mode hybrid retrieval method based on large model reasoning according to claim 6 is characterized in that: Each subquery problem has a corresponding preference pair to indicate the optimal action selection.

8. A multi-source heterogeneous data multi-mode hybrid retrieval system based on large model reasoning, characterized by: The multi-source heterogeneous data multi-mode hybrid retrieval method based on large model reasoning as described in any one of claims 1 to 7 is adopted, comprising: The pattern retrieval unit is configured to: obtain the user's initial query question, and in each step of the reasoning process, the large language model determines sub-query questions and corresponding sub-query answers of different retrieval modes based on the initial query question; The path optimization unit is configured to: adaptively determine the optimal search path based on the sub-query questions and sub-query answers corresponding to the query patterns of each step of the reasoning process, with the goal of maximizing the reward function, and determine the final answer corresponding to the initial query question based on the optimal search path; wherein the optimal search path includes: the optimal answer of each search pattern and the search order of each search pattern, and the search patterns include database search, semantic search and graph search.

9. A computer device, characterized in that: include: a processor and a computer-readable storage medium; a processor adapted to execute a computer program; A computer-readable storage medium having a computer program stored therein, wherein the computer program, when executed by the processor, implements the multi-source heterogeneous data multi-mode hybrid retrieval method based on large model reasoning as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which is suitable for being loaded by a processor and executing the multi-source heterogeneous data multi-mode hybrid retrieval method based on large model reasoning as described in any one of claims 1 to 7.

Images