A high-performance knowledge base system based on multi-modal hybrid retrieval

Abstract

The application relates to the technical field of artificial intelligence, knowledge representation and knowledge retrieval, in particular to a high-performance knowledge base system based on multi-modal hybrid retrieval, which comprises a vector calculation module used for converting unstructured text into vector data represented by feature vectors; a knowledge graph module used for converting structured text into knowledge graph data; a hybrid retrieval module used for retrieval according to a query request; and a hybrid retrieval fusion module used for retrieval of a storage module according to the query request; the hybrid retrieval comprises vector retrieval and graph retrieval; and the fusion retrieval result is generated based on the vector retrieval and the graph retrieval. The unstructured and structured data are processed by the vector calculation module and the knowledge graph module respectively, the hybrid retrieval fusion module performs vector retrieval and graph retrieval respectively based on the query request, and the retrieval results are fused, so that the semantic matching precision and the relationship reasoning capability are greatly improved.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence, knowledge representation and knowledge retrieval, and specifically to a high-performance knowledge base system based on multimodal hybrid retrieval. Background Technology

[0002] In today's information explosion era, massive amounts of data are rapidly increasing in various forms, including structured and unstructured data. Knowledge base systems, as important tools for data storage, management, and retrieval, have been widely used in various fields. Traditional knowledge base systems typically use relational databases to store and manage structured data, and use structured query languages ​​such as SQL to retrieve data. However, this approach has certain limitations when dealing with complex semantic relationships and knowledge reasoning, and it is difficult to meet the needs of in-depth knowledge mining.

[0003] Meanwhile, when faced with large amounts of unstructured data such as text and documents, traditional systems often employ simple retrieval methods like keyword matching. This approach fails to effectively understand the semantic information of the text, resulting in poor accuracy and relevance of the retrieval results. Although vector-based retrieval technologies have emerged in recent years, capable of converting unstructured text into vector data and achieving semantic retrieval by calculating the similarity between vectors, vector retrieval alone cannot fully utilize the knowledge connections inherent in structured data. Summary of the Invention

[0004] (a) Purpose of the invention

[0005] The purpose of this invention is to provide a high-performance knowledge base system based on multimodal hybrid retrieval, which improves the accuracy of retrieval by using a hybrid retrieval module to support vector retrieval and graph retrieval in parallel.

[0006] (II) Technical Solution

[0007] To address the above problems, this invention provides a high-performance knowledge base system based on multimodal hybrid retrieval, comprising: a vector calculation module, a knowledge graph module, a storage module, and a hybrid retrieval fusion module;

[0008] The vector calculation module is used to convert unstructured text into vector data represented by feature vectors;

[0009] The knowledge graph module is used to convert structured text into knowledge graph data;

[0010] The storage module is connected to the vector calculation module and the knowledge graph module, and is used to store the vector data and the knowledge graph data.

[0011] The hybrid retrieval fusion module is connected to the storage module and is used to retrieve data from the storage module according to a query request. The hybrid retrieval includes vector retrieval and graph retrieval, and a fusion retrieval result is generated based on the vector retrieval and graph retrieval.

[0012] In another aspect of the present invention, preferably, the vector calculation module is based on a quantum neural network model;

[0013] The vector calculation module includes: a quantum data encoding unit and a quantum text feature extraction unit;

[0014] The quantum data encoding unit is used to encode unstructured text into quantum states;

[0015] The quantum text feature extraction unit evolves the quantum state through parameterized quantum circuits and collapses the output quantum state into a feature vector based on quantum measurement, forming vector data represented by the feature vector.

[0016] The dimension of the feature vector output by the quantum text feature extraction unit is determined by the number of qubits and the measurement method.

[0017] In another aspect of the present invention, preferably, the vector calculation module further includes a quantum verification unit;

[0018] The quantum verification unit re-encodes the feature vector into a first quantum state and encodes the feature vector of the reference text into a second quantum state.

[0019] The fidelity between the first and second quantum states is calculated using a quantum test circuit.

[0020] When the fidelity is greater than or equal to the fidelity threshold, the feature vector transformation is deemed valid.

[0021] In another aspect of the present invention, preferably, the knowledge graph module is used to convert structured text into knowledge graph data, including:

[0022] Parse structured text and identify the entities, attributes, and relationships within the structured text;

[0023] The recognition results are mapped to graph structure data, where entities are nodes and relationships are edges, forming knowledge graph data.

[0024] In another aspect of the present invention, preferably, the storage module includes a PostgreSQL unit and a FAISS vector index unit;

[0025] The PostgreSQL unit includes the knowledge_bases table, documents table, chunks table, knowledge_graphs table, and permissions table;

[0026] The knowledge_bases table is used to store basic information about the knowledge base, the documents table is used to store raw data, the chunks table is used to store unstructured text and its vector data, the knowledge_graphs table is used to store knowledge graph data, and the permissions table is used to manage user access permissions to the knowledge base.

[0027] The FAISS vector indexing unit is used to create a vector index for the vector data.

[0028] In another aspect of the present invention, preferably, the hybrid retrieval module includes a map retrieval unit;

[0029] The graph retrieval unit receives user query instructions and extracts a list of key entities and potential relationship keywords from the user query instructions.

[0030] Using the list of key entities and potential relationship keywords as input, the knowledge graph data in the storage module is retrieved;

[0031] The retrieval includes: locating the entity nodes of the key entities in the key entity list in the knowledge graph, expanding along the relation edges of potential relation keywords starting from the entity nodes, extracting associated entities and relation paths, and obtaining a structured subgraph.

[0032] In another aspect of the present invention, preferably, the hybrid retrieval module includes a vector retrieval unit;

[0033] The vector retrieval unit converts the structured subgraph into vector retrieval constraints and semantic enhancement vectors;

[0034] The user query instruction is converted into a raw vector, and the raw vector is fused with the semantic enhancement vector to obtain a fused vector;

[0035] Based on the constraints of the fused vector and vector retrieval, the vector data in the storage module is retrieved to generate fused retrieval results.

[0036] In another aspect of the present invention, preferably, the constraints of the vector retrieval include: extracting unique identifier information contained in each entity node from the structured subgraph to form an entity identifier set;

[0037] The entity identifier set serves as a filter tag, which is used in the vector retrieval process to limit the retrieval range of candidate vectors;

[0038] The generation of the semantic enhancement vector includes:

[0039] Extract the text description information associated with each entity node in the structured subgraph, and concatenate the text description information to form semantic input text;

[0040] The semantic input text is vectorized and encoded to generate semantically enhanced vectors.

[0041] In another aspect of the present invention, preferably, the hybrid retrieval and fusion module further includes an iteration unit.

[0042] The iterative unit is used to mine entity relationships in the fused retrieval results and identify new entities and new relationships that are not included in the current key entity list and potential relationship keyword set.

[0043] Based on the newly added information, an expanded list of key entities and a set of relational keywords are constructed.

[0044] The expanded list of key entities and the set of relational keywords will be used as input for a new round of graph retrieval and vector retrieval.

[0045] In another aspect of the present invention, preferably, the hybrid retrieval module further includes a convergence determination unit, which is used to determine whether the retrieval convergence condition is met during the execution of multi-round fusion retrieval, so as to terminate the iteration;

[0046] The convergence condition includes any one of the following:

[0047] The overlap between the expanded structured subgraph and the entities calculated in the previous round exceeds a preset threshold.

[0048] The number of newly added entities and newly added relationships is lower than a preset threshold.

[0049] (III) Beneficial Effects

[0050] The above-described technical solution of the present invention has the following beneficial technical effects:

[0051] This invention processes unstructured and structured data respectively through a vector computation module and a knowledge graph module. A hybrid retrieval and fusion module performs vector retrieval and knowledge graph retrieval based on the query request, and then fuses the retrieval results, significantly improving semantic matching accuracy and relational reasoning capabilities. Compared to traditional single-modal retrieval methods, this invention can more comprehensively capture the semantic information and structural relationships in user queries, enabling multi-hop reasoning and accurate responses to complex problems, providing key technical support for building intelligent knowledge service systems. Attached Figure Description

[0052] Figure 1 This is a schematic diagram of the overall structure of one embodiment of the present invention. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments and the accompanying drawings. It should be understood that these descriptions are merely exemplary and not intended to limit the scope of the invention. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concept of the invention.

[0054] The accompanying drawings show structural schematic diagrams according to embodiments of the present invention. These drawings are not drawn to scale, and some details have been enlarged for clarity, and some details may have been omitted. The shapes of the various regions and layers shown in the drawings, as well as their relative sizes and positional relationships, are merely exemplary and may deviate from reality due to manufacturing tolerances or technical limitations. Furthermore, those skilled in the art can design regions / layers with different shapes, sizes, and relative positions as needed.

[0055] Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0056] Furthermore, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0057] The invention will now be described in more detail with reference to the accompanying drawings. In the various drawings, the same elements are indicated by similar reference numerals. For clarity, the various parts in the drawings are not drawn to scale.

[0058] Example 1

[0059] A high-performance knowledge base system based on multimodal hybrid retrieval. Figure 1 A schematic diagram of the overall structure of an embodiment of the present invention is shown, as follows. Figure 1 As shown, it includes: a vector calculation module, a knowledge graph module, a storage module, and a hybrid retrieval and fusion module;

[0060] The vector computation module is used to convert unstructured text into vector data represented by feature vectors. This module can be based on natural language processing techniques, such as pre-trained language models, to perform deep semantic understanding of the unstructured text. First, through basic processing such as word segmentation and part-of-speech tagging, the text is decomposed into computable units. Then, a deep learning model is used to map the text into feature vectors in a high-dimensional space, where each dimension represents different semantic features of the text. In this embodiment, the vector computation module is based on a quantum neural network model.

[0061] The vector calculation module includes: a quantum data encoding unit and a quantum text feature extraction unit;

[0062] The quantum data encoding unit is used to encode unstructured text into quantum states. Amplitude encoding is employed to map the unstructured text to quantum states. For text strings, they are first converted into numerical vectors using classical word embeddings, and then converted into quantum state amplitudes through normalization. In quantum computing, a quantum state can be represented by a complex vector, where the square of the modulus of each component represents the probability of measuring the quantum state as being in its corresponding ground state. The purpose of normalization is to ensure that the sum of these probabilities is 1. In this way, the original unstructured text is encoded into a quantum state, enabling subsequent processing in the quantum computing system.

[0063] The quantum text feature extraction unit evolves the quantum state through a parameterized quantum circuit and collapses the output quantum state into a feature vector based on quantum measurement, forming vector data represented by the feature vector. The parameterized quantum circuit is composed of a series of quantum gates, and the operating parameters of the quantum gates can be adjusted according to training data. Quantum gates are the basic operating units in quantum computing, similar to logic gates in classical computing. Different combinations of quantum gates can realize various transformations of the quantum state. In the quantum text feature extraction process, the parameterized quantum circuit performs a series of quantum gate operations on the qubits according to the input quantum state to achieve feature extraction and transformation of unstructured text. Utilizing the superposition and entanglement properties of quantum computing, multiple text features can be processed simultaneously, greatly improving computational efficiency. After evolution by the parameterized quantum circuit, the output quantum state is in a superposition state, containing multiple possible state information. Through quantum measurement operations, the quantum state is collapsed into a definite state, and the result is the feature vector. The dimension of the feature vector output by the quantum text feature extraction unit is determined by the number of qubits and the measurement method. The more qubits, the larger the state space that can be represented, and the richer the extracted features. Different measurement methods can also affect the dimension and specific form of the eigenvector. For example, different measurement methods such as projective measurement and weak measurement will yield different measurement results, thus affecting the final eigenvector. The number of qubits and the measurement method can be set according to the actual application scenario.

[0064] Furthermore, in this embodiment, the vector calculation module also includes a quantum verification unit;

[0065] The quantum verification unit re-encodes the feature vector into a first quantum state and encodes the feature vector of the reference text into a second quantum state; the encoding process is similar to that of the quantum data encoding unit, both converting information into a quantum state representation. The feature vector of the reference text can be a series of texts.

[0066] The fidelity between the first and second quantum states is calculated using a quantum testing circuit; fidelity measures the similarity between two quantum states. In quantum computing, higher fidelity indicates greater similarity between the two quantum states. When the fidelity is greater than or equal to a fidelity threshold, the feature vector transformation is considered valid. In the case of a series of texts, only one fidelity greater than or equal to the fidelity threshold is required. The fidelity threshold is preset based on the specific application scenario and requirements. If the fidelity is lower than the threshold, it indicates that the feature vector may contain errors or be inaccurate, requiring adjustment of the parameters of the quantum text feature extraction unit or a re-extraction process to ensure that the final feature vector accurately represents the features of the text data. This improves the accuracy and effectiveness of the feature vectors output by the quantum text feature extraction unit.

[0067] The knowledge graph module is used to convert structured text into knowledge graph data, including:

[0068] Parse structured text and identify the entities, attributes, and relationships within the structured text;

[0069] The recognition results are mapped to graph-structured data, where entities are nodes and relationships are edges, forming knowledge graph data. The knowledge graph module converts structured text into knowledge graph data, and can use rule-driven, machine learning, and other methods to parse the text, accurately identify entities, attributes, and relationships, determine attributes through dependency parsing, and extract relationships based on pattern matching, etc. The recognition results are mapped to graph-structured data, with entities as nodes and relationships as edges. Furthermore, nodes can be classified and metadata can be added to edges.

[0070] The storage module is connected to the vector calculation module and the knowledge graph module, and is used to store the vector data and knowledge graph data. In this embodiment, the storage module includes a PostgreSQL unit and a FAISS vector index unit. The PostgreSQL unit includes the knowledge_bases table, documents table, chunks table, knowledge_graphs table, and permissions table. The knowledge_bases table is used to store basic information of the knowledge base, the documents table is used to store raw data, the chunks table is used to store unstructured text and its vector data, the knowledge_graphs table is used to store knowledge graph data, and the permissions table is used to manage user access permissions to the knowledge base.

[0071] The FAISS vector indexing unit is used to create a vector index for the vector data.

[0072] The `knowledge_bases` table uniformly records metadata such as knowledge base name, creation time, and responsible person, providing a global perspective for managing multiple knowledge bases. The `documents` table stores raw data, preserving the original form of documents, reports, and other file formats. The `chunks` table segments unstructured text and synchronously stores corresponding vector data, laying the foundation for subsequent semantic retrieval. The `knowledge_graphs` table specifically carries node, edge, and relationship data generated by the knowledge graph module, ensuring the integrity of the graph structure information. The `permissions` table uses permission control policies to achieve fine-grained management of user access to the knowledge base, ensuring data security. The FAISS vector index unit processes vector data, using algorithms such as hierarchical clustering and hash mapping to create an efficient index structure for text vectors in the `chunks` table, greatly improving similarity retrieval efficiency and enabling rapid matching and querying of massive amounts of vector data.

[0073] Furthermore, in this embodiment, a synchronization mechanism is set between the PostgreSQL unit and the FAISS vector index unit;

[0074] The synchronization mechanism is used to perform timed full synchronization operations between the PostgreSQL unit and the FAISS vector index unit. Timed full synchronization performs a comprehensive comparison and update of the data in both units according to a preset period, such as every morning or every weekend. During full synchronization, the system rescans all unstructured text and its vector data in the PostgreSQL unit's chunks table, synchronizing the latest data status to the FAISS vector index unit, rebuilding the vector index structure, and preventing the index from becoming disconnected from the actual data due to long-term data accumulation, thus ensuring the accuracy of the index in large-scale data environments.

[0075] It tracks data changes in PostgreSQL cells using timestamps and synchronizes FAISS vector index cells based on the tracking results. Timestamp-based data change tracking and synchronization enables real-time response. Whenever data in a PostgreSQL cell is added, modified, or deleted, a timestamp of the corresponding data row is automatically recorded. The synchronization mechanism continuously monitors changes to these timestamps, and once a data update is detected, it performs the appropriate operation based on the change type. This ensures that relational data and vector data remain consistent during storage and updates, laying a solid foundation for the efficient operation of hybrid retrieval.

[0076] The hybrid retrieval fusion module is connected to the storage module and is used to retrieve data from the storage module according to a query request. The hybrid retrieval includes vector retrieval and graph retrieval, and a fused retrieval result is generated based on the vector retrieval and graph retrieval. The graph retrieval and vector retrieval units combine structured information from the knowledge graph with text semantic representation to achieve high-precision semantically enhanced retrieval of user natural language queries. In this embodiment, the hybrid retrieval module includes a graph retrieval unit.

[0077] The graph retrieval unit receives user query instructions and extracts a list of key entities and potential relationship keywords from the user query instructions. The user query instructions are natural language query instructions. In this embodiment, a lightweight natural language processing model, such as BERT or ERNIE, is called to analyze and extract key entities from the user query instructions, such as names of people, names of organizations, and technical terms. Potential relationship keywords are also extracted, such as influence, inclusion, and cooperation.

[0078] Using the list of key entities and potential relationship keywords as input, the knowledge graph data in the storage module is retrieved;

[0079] The retrieval process includes: locating the entity nodes of key entities in the key entity list within the knowledge graph; starting from each entity node, expanding along the relational edges of potential relational keywords to extract associated entities and relational paths, thus obtaining a structured subgraph. Based on the extracted key entity list, the corresponding entity nodes are searched in the knowledge graph, and expanded along the relational edges of potential relational keywords (one or two edges can be expanded) to obtain a structured subgraph related to the entity. The structured subgraph includes: entity nodes, relational paths between nodes, node attribute values, and their contextual text descriptions. To avoid graph explosion, the following settings can be configured: each node can expand up to M edges; hop count limit: a maximum of 1-2 hops.

[0080] Furthermore, in this embodiment, the hybrid retrieval module includes a vector retrieval unit;

[0081] The vector retrieval unit converts the structured subgraph into constraints and semantic enhancement vectors for vector retrieval; it then uses graph neural networks or graph embedding algorithms to embed and encode the structured subgraph. The semantic enhancement vector of the structured subgraph is output.

[0082] The user query instruction is converted into a raw vector, and semantic features are extracted using a pre-trained language model to vectorize the user intent, resulting in the raw query vector. The raw vector is then fused with the semantically enhanced vector to obtain a fused vector; a weighted method is used for fusion, and the weights can be set according to the type of question.

[0083] Furthermore, in this embodiment, the constraints of the vector retrieval include: extracting unique identifier information contained in each entity node from the structured subgraph to form an entity identifier set; in the vector retrieval process, the setting of constraints can effectively improve the accuracy and efficiency of the retrieval.

[0084] The entity identifier set serves as a filter tag, applied during vector retrieval to limit the search scope of candidate vectors. The principle behind using the entity identifier set as a filter tag in vector retrieval is to limit the search scope. In real-world retrieval scenarios, massive amounts of vector data are like a vast ocean; without constraints, retrieval would be like finding a needle in a haystack. The entity identifier set, acting as a filter tag, selects candidate vectors that meet specific conditions.

[0085] The generation of the semantic enhancement vector includes:

[0086] The text description information associated with each entity node in the structured subgraph is extracted and concatenated to form semantic input text. The generation of semantic enhancement vectors aims to better express the semantic information of entities, thereby improving retrieval accuracy. The extracted text description information provides detailed information about entity features, attributes, and functions, serving as a crucial carrier of entity semantics. The extracted text description information is then concatenated to form semantic input text. This integrates scattered and fragmented text information into a complete semantic unit for subsequent vectorization encoding. The concatenation process can utilize appropriate delimiters or formats to connect different types of text information in an orderly manner, based on actual needs.

[0087] The semantic input text is vectorized to generate semantically enhanced vectors. Vectorization techniques, such as word vector models in deep learning or architecture-based models, are used to process the semantic input text and convert it into semantically enhanced vectors. By capturing semantic relationships and contextual information in the text, the generated semantically enhanced vectors not only contain the textual information of entities but also contain deeper semantic features, providing richer semantic basis for subsequent retrieval.

[0088] Based on the constraints of the fused vector and vector retrieval, the vector data in the storage module is retrieved to generate fused retrieval results. The fused vector combines information from multiple dimensions, providing a more comprehensive description of entity characteristics. The constraints of vector retrieval clarify the scope and direction of the retrieval. When retrieving vector data from the storage module, candidate vectors are initially screened using entity identifier set filtering tags based on the constraints of vector retrieval, narrowing the retrieval scope. For example, in an intelligent customer service system, when a user asks a question, the system converts the question into a semantically enhanced vector. Simultaneously, based on information such as the business domain and product category involved in the question, constraints for vector retrieval are set, retrieving the most relevant knowledge vector data from the storage module to quickly and accurately provide answers to the user, greatly improving user experience and service efficiency.

[0089] Furthermore, in this embodiment, the hybrid retrieval fusion module further includes an iteration unit, which includes: when mining entity relationships in the fusion retrieval results, the iteration unit performs in-depth analysis of the text and graph structure in the retrieval results using natural language processing and graph analysis techniques.

[0090] The iterative unit is used to mine entity relationships in the fused retrieval results and identify new entities and new relationships that are not included in the current key entity list and potential relationship keyword set.

[0091] Based on the newly added information, an expanded list of key entities and a set of relational keywords are constructed. The iterative unit then constructs the expanded list of key entities and the set of relational keywords based on the mined new information. This process is not simply a piling up of information, but rather integrates the new entities and relationships into the existing system through logical organization and semantic association.

[0092] The expanded list of key entities and the set of relational keywords will serve as input for the next round of graph retrieval and vector retrieval. This expanded list of key entities and the set of relational keywords will guide the retrieval system to explore a broader and deeper knowledge space, obtaining more comprehensive and accurate information.

[0093] The hybrid retrieval module also includes a convergence determination unit, which is used to determine whether the retrieval convergence condition is met during the multi-round fusion retrieval process in order to terminate the iteration. By setting reasonable convergence conditions, the retrieval process is prevented from endlessly looping meaninglessly, ensuring that the iteration is terminated at the appropriate time and outputting stable and reliable retrieval results.

[0094] The convergence condition includes any one of the following:

[0095] The overlap between the expanded structured subgraph and the entities calculated in the previous round exceeds a preset threshold. In actual retrieval, as the iteration proceeds, the structured subgraph continues to expand. If most entities in the newly generated subgraph are the same as those in the previous round, it indicates that the retrieval has approached "saturation" and it is difficult to obtain valuable new information.

[0096] The number of newly added entities and relationships is below a preset threshold. When the number of newly added entities and relationships mined in each iteration gradually decreases, even falling below a certain set value, it means that the retrieval system has fully explored the relevant knowledge domain, and further iterations are unlikely to bring significant information gains. The iterative unit and the convergence determination unit work together to give the hybrid retrieval fusion module strong adaptive capabilities.

[0097] This invention processes unstructured and structured data respectively through a vector computation module and a knowledge graph module. A hybrid retrieval and fusion module performs vector retrieval and knowledge graph retrieval based on the query request, and then fuses the retrieval results, significantly improving semantic matching accuracy and relational reasoning capabilities. Compared to traditional single-modal retrieval methods, this invention can more comprehensively capture the semantic information and structural relationships in user queries, enabling multi-hop reasoning and accurate responses to complex problems, providing key technical support for building intelligent knowledge service systems.

[0098] It should be understood that the specific embodiments described above are merely illustrative or explanatory of the principles of the invention and do not constitute a limitation thereof. Therefore, any modifications, equivalent substitutions, improvements, etc., made without departing from the spirit and scope of the invention should be included within the protection scope of the invention. Furthermore, the appended claims are intended to cover all variations and modifications falling within the scope and boundaries of the appended claims, or equivalent forms of such scope and boundaries.

[0099] The present invention has been described above with reference to embodiments thereof. However, these embodiments are merely illustrative and not intended to limit the scope of the invention. The scope of the invention is defined by the appended claims and their equivalents. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of the invention, and all such substitutions and modifications should fall within the scope of the invention.

[0100] Although embodiments of the present invention have been described in detail, it should be understood that various changes, substitutions, and modifications can be made to the embodiments of the present invention without departing from the spirit and scope of the invention.

[0101] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A high-performance knowledge base system based on multimodal hybrid retrieval, characterized in that, include: Vector computation module, knowledge graph module, storage module, and hybrid retrieval and fusion module; The vector calculation module is used to convert unstructured text into vector data represented by feature vectors; The knowledge graph module is used to convert structured text into knowledge graph data; The storage module is connected to the vector calculation module and the knowledge graph module, and is used to store the vector data and the knowledge graph data. The hybrid retrieval and fusion module is connected to the storage module and is used to retrieve data from the storage module according to a query request. The hybrid retrieval includes vector retrieval and graph retrieval, and a fusion retrieval result is generated based on the vector retrieval and graph retrieval. The hybrid retrieval and fusion module includes a map retrieval unit and a vector retrieval unit; The graph retrieval unit receives a user query command, extracts a list of key entities and potential relationship keywords from the user query command, and uses the list of key entities and potential relationship keywords as input to retrieve the knowledge graph data in the storage module to obtain a structured subgraph. The vector retrieval unit converts the structured subgraph into vector retrieval constraints and semantic enhancement vectors; it converts the user query command into a raw vector, merges the raw vector with the semantic enhancement vector to obtain a fused vector; and based on the fused vector and vector retrieval constraints, it retrieves the vector data in the storage module to generate fused retrieval results. The constraints of the vector retrieval include: extracting unique identifier information contained in each entity node from the structured subgraph to form an entity identifier set; the entity identifier set is used as a filter label to limit the retrieval range of candidate vectors during the vector retrieval process; The generation of the semantic enhancement vector includes: extracting the text description information associated with each entity node in the structured subgraph, concatenating the text description information to form semantic input text, and vectorizing the semantic input text to generate a semantic enhancement vector.

2. The high-performance knowledge base system based on multimodal hybrid retrieval according to claim 1, characterized in that, The vector calculation module is based on a quantum neural network model; The vector calculation module includes: a quantum data encoding unit and a quantum text feature extraction unit; The quantum data encoding unit is used to encode unstructured text into quantum states; The quantum text feature extraction unit evolves the quantum state through parameterized quantum circuits and collapses the output quantum state into a feature vector based on quantum measurement, forming vector data represented by the feature vector. The dimension of the feature vector output by the quantum text feature extraction unit is determined by the number of qubits and the measurement method.

3. The high-performance knowledge base system based on multimodal hybrid retrieval according to claim 2, characterized in that, The vector calculation module also includes a quantum verification unit; The quantum verification unit re-encodes the feature vector into a first quantum state and encodes the feature vector of the reference text into a second quantum state. The fidelity between the first and second quantum states is calculated using a quantum test circuit. When the fidelity is greater than or equal to the fidelity threshold, the feature vector transformation is deemed valid.

4. The high-performance knowledge base system based on multimodal hybrid retrieval according to claim 1, characterized in that, The knowledge graph module is used to convert structured text into knowledge graph data, including: Parse structured text and identify the entities, attributes, and relationships within the structured text; The recognition results are mapped to graph structure data, where entities are nodes and relationships are edges, forming knowledge graph data.

5. The high-performance knowledge base system based on multimodal hybrid retrieval according to claim 1, characterized in that, The storage module includes PostgreSQL units and FAISS vector index units; The PostgreSQL unit includes the knowledge_bases table, documents table, chunks table, knowledge_graphs table, and permissions table; The knowledge_bases table is used to store basic information about the knowledge base, the documents table is used to store raw data, the chunks table is used to store unstructured text and its vector data, the knowledge_graphs table is used to store knowledge graph data, and the permissions table is used to manage user access permissions to the knowledge base. The FAISS vector indexing unit is used to create a vector index for the vector data.

6. The high-performance knowledge base system based on multimodal hybrid retrieval according to claim 1, characterized in that, The retrieval includes: locating the entity nodes of the key entities in the key entity list in the knowledge graph, expanding along the relation edges of potential relation keywords starting from the entity nodes, extracting associated entities and relation paths, and obtaining a structured subgraph.

7. The high-performance knowledge base system based on multimodal hybrid retrieval according to claim 1, characterized in that, The hybrid retrieval and fusion module also includes an iteration unit. The iterative unit is used to mine entity relationships in the fused retrieval results and identify new entities and new relationships that are not included in the current key entity list and potential relationship keyword set. Based on the newly added entities and relationships, an expanded list of key entities and a set of relationship keywords are constructed. The expanded list of key entities and the set of relational keywords will be used as input for a new round of graph retrieval and vector retrieval.

8. The high-performance knowledge base system based on multimodal hybrid retrieval according to claim 7, characterized in that, The hybrid retrieval and fusion module also includes a convergence determination unit, which is used to determine whether the retrieval convergence condition is met during the execution of multiple rounds of fusion retrieval, so as to terminate the iteration; The convergence condition includes any one of the following: The overlap between the expanded structured subgraph and the entities calculated in the previous round exceeds a preset threshold. The number of newly added entities and newly added relationships is lower than a preset threshold.

Images