Native vector graph database and data structure construction method

Abstract

The application relates to the technical field of database construction, and discloses a native vector graph database and a data structure construction method, which comprises the following steps: acquiring node data and edge data in traditional graph data; performing vectorization on the node data and the edge data to obtain node vectors and edge vectors; constructing a first data structure and a second data structure according to at least one group of the node vectors and the node data and the edge vectors and the edge data; and constructing a vector graph database based on the first data structure and the second data structure. The first data structure and the second data structure are constructed according to the node vectors and the edge vectors obtained by vectorizing the node data and the edge data, so that the vector graph database is constructed, the problem that a traditional vector graph database cannot realize reasoning of different levels of a graph atlas is solved, and the fusion of graph database technology and vector database technology and reasoning of different levels of a graph are realized.

Description

Technical Field

[0001] This invention relates to the field of database construction technology, and in particular to a native vector graph database and a data structure construction method. Background Technology

[0002] A knowledge graph is a graphical structure used to organize and represent knowledge, showcasing the semantic relationships between entities and connections through the connections of nodes and edges. For example, a knowledge graph might contain information about a specific company, including its location, products, employees, and the relationships between these entities. A graph database is a database specifically designed for storing and managing graph-structured data; its data model is based on graph theory. In a graph database, data is stored as vertices (nodes) and edges to represent entities and relationships between them; vertices and edges can be appended with attributes. Commonly used graph databases include Neo4j, OrientDB, ArangoDB, JanusGraph, Microsoft Graph Engine, and Amazon Neptune, each with its own advantages and suitability for different needs and scenarios. Graph databases provide the infrastructure for storing, querying, and managing knowledge graphs, while knowledge graphs bring rich application scenarios for knowledge rendering and expression to graph databases. The two complement each other, jointly driving the development of graph data processing technology.

[0003] Vector databases are a special type of database that stores data differently from traditional relational databases. In vector databases, data is stored as vectors, which can be representations of any modality, such as image, audio, or text embeddings. In practical applications, vector databases can be used in various scenarios, such as search engines, recommendation systems, and social network analysis. In large language model question-answering systems, vector databases can be combined with large language models to build more powerful and efficient question-answering systems. When a user asks a question, the question-answering system can use the large language model to understand the semantics of the question and transform it into vector form. Vector databases can quickly retrieve the answer vector most similar to the question vector, thereby improving the efficiency of question-answering. There are various vector databases available on the market: commercial databases such as Chroma, LanceDB, Marqo, Milvus / Zilliz, Pinecone, Qdrant, Vald, Vespa, and Weaviate, as well as open-source vector libraries such as Faiss, Annoy, and Hnswlib.

[0004] With the development of vectorization technology, traditional graph databases have successively launched versions with vectorization capabilities. NoSQL databases that support vectors include MongoDB, Cassandra / DataStax Astra, CosmosDB, and Rockset. There are also key-value databases like Redis and other special-purpose databases such as Neo4j (a graph database). SQL databases that support vectors include SingleStoreDB, PostgreSQL, Clickhouse, and Kinetica's pgvector / Supabase Vector. However, current graph databases with vector embedding technology primarily rely on word embedding algorithms from natural language processing, such as Word2Vec, GloVe, FastText, and ELMo, and sentence embedding algorithms, such as Doc2Vec and SentenceBERT. These algorithms can only extract semantic information from a single node and are mainly used for searching similar nodes in a global graph. This type of graph database is called a traditional vector graph database. Vector embedding algorithms based on knowledge graph topology and the embedded vectors they generate have rich semantic information at various levels, including node level, relation level, subgraph level, and full graph level. Currently, there is no graph database based on vector embedding algorithms that integrates knowledge graph topology information for reasoning at different levels of knowledge graphs. Summary of the Invention

[0005] The main objective of this invention is to provide a native vector graph database and a data structure construction method, which aims to solve the technical problem that existing traditional vector graph databases cannot realize reasoning at different levels of the graph.

[0006] To achieve the above objectives, this invention provides a native vector graph database and a data structure construction method, the method comprising the following steps:

[0007] Retrieve node and edge data from traditional graph data;

[0008] The node data and the edge data are vectorized to obtain node vectors and edge vectors;

[0009] Construct a first data structure and a second data structure based on at least one of the node vectors and node data, and the edge vectors and edge data;

[0010] A vector graph database is constructed based on the first data structure and the second data structure.

[0011] Optionally, the step of vectorizing the node data and the edge data to obtain node vectors and edge vectors includes:

[0012] The node data is vectorized based on the node vector embedding model to generate node vectors;

[0013] The edge data is vectorized based on the edge vector embedding model to generate edge vectors.

[0014] Optionally, the step of vectorizing the node data based on the node vector embedding model to generate node vectors includes:

[0015] Generate first sub-embedding vectors with different semantic dimensions based on the node vector embedding model;

[0016] The first sub-embedding vectors of different semantic dimensions are concatenated to obtain the node embedding vector;

[0017] The node embedding vector is combined with the node data to generate graph nodes;

[0018] Node vectors are determined based on the graph nodes.

[0019] Optionally, the step of vectorizing the edge data based on the edge vector embedding model to generate edge vectors includes:

[0020] Generate second embedding vectors with different semantic dimensions based on the edge vector embedding model;

[0021] The second embedding vectors with different semantic dimensions are concatenated to obtain the edge embedding vector;

[0022] The edge embedding vector is combined with the edge data to generate graph edges;

[0023] The edge vectors are determined based on the graph edges.

[0024] Optionally, constructing the first data structure and the second data structure based on at least one set of the node vector and the node data, and the edge vector and the edge data, includes:

[0025] Generate a first key-value pair based on the node vector and the node data;

[0026] Construct a first data structure based on the first key-value pair;

[0027] Generate a second key-value pair based on the edge vector and the edge data;

[0028] The second data structure is constructed based on the second key-value pair.

[0029] Optionally, constructing a vector graph database based on the first data structure and the second data structure includes:

[0030] Generate a vector knowledge graph based on the first data structure and the second data structure;

[0031] Construct a vector index based on the node vectors and edge vectors;

[0032] A vector graph database is constructed based on the vector index and the vector knowledge graph.

[0033] Optionally, before obtaining node data and edge data from traditional graph data, the method further includes:

[0034] Obtain business requirement information and construct a knowledge graph ontology structure based on the business requirement information;

[0035] Acquire business knowledge information and generate structured data based on the business knowledge information;

[0036] Establish mapping relationships between the structured data and nodes and edges respectively;

[0037] A business knowledge graph is generated based on the mapping relationship and the knowledge graph ontology structure.

[0038] A traditional graph database is constructed based on the business knowledge graph, and traditional graph data is generated based on the traditional graph database.

[0039] Furthermore, to achieve the above objectives, the present invention also proposes a native vector graph database and data structure construction device, the native vector graph database and data structure construction device comprising:

[0040] The acquisition module is used to acquire node data and edge data from traditional graph data;

[0041] A vectorization module is used to vectorize the node data and the edge data to obtain node vectors and edge vectors;

[0042] The construction module is used to construct a first data structure and a second data structure based on at least one of the node vectors and the node data, and the edge vectors and the edge data;

[0043] The construction module is also used to construct a vector graph database based on the first data structure and the second data structure.

[0044] Furthermore, to achieve the above objectives, the present invention also proposes a native vector graph database and data structure construction device, the native vector graph database and data structure construction device comprising: a memory, a processor, and a native vector graph database and data structure construction program stored on the memory and executable on the processor, the native vector graph database and data structure construction program being configured to implement the steps of the native vector graph database and data structure construction method described above.

[0045] Furthermore, to achieve the above objectives, the present invention also proposes a storage medium storing a native vector graph database and a data structure construction program, wherein when the native vector graph database and data structure construction program is executed by a processor, the steps of the native vector graph database and data structure construction method described above are implemented.

[0046] This invention acquires node and edge data from traditional graph data; vectorizes the node and edge data to obtain node vectors and edge vectors; constructs a first data structure and a second data structure based on at least one pair of node vectors and node data, and edge vectors and edge data; and constructs a vector graph database based on the first and second data structures. Through this method, by constructing the first and second data structures from the node and edge vectors obtained from the vectorization of node and edge data, a vector graph database is built. This solves the problem that traditional vector graph databases cannot achieve reasoning at different levels of the graph, realizing the integration of graph database technology and vector database technology, and enabling reasoning at different levels of the graph. Attached Figure Description

[0047] Figure 1 This is a schematic diagram of the structure of the native vector graph database and data structure construction device of the hardware operating environment involved in the embodiments of the present invention;

[0048] Figure 2 This is a flowchart illustrating the first embodiment of the native vector graph database and data structure construction method of the present invention;

[0049] Figure 3 This is a technical principle diagram of an embodiment of the native vector graph database and data structure construction method of the present invention;

[0050] Figure 4 This is an application architecture diagram of the native vector graph database, which is an embodiment of the native vector graph database and data structure construction method of the present invention.

[0051] Figure 5 This is a schematic diagram illustrating the training principle of a vector embedding model based on a twin network structure, according to an embodiment of the native vector graph database and data structure construction method of the present invention.

[0052] Figure 6 This is a technical relationship diagram of an embodiment of the native vector graph database and data structure construction method of the present invention;

[0053] Figure 7 This is a flowchart illustrating the second embodiment of the native vector graph database and data structure construction method of the present invention;

[0054] Figure 8 This is a structural block diagram of the first embodiment of the native vector graph database and data structure construction device of the present invention.

[0055] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0056] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0057] Reference Figure 1 , Figure 1 This is a schematic diagram of the native vector graph database and data structure construction device of the hardware operating environment involved in the embodiments of the present invention.

[0058] like Figure 1 As shown, the native vector graph database and data structure construction device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to implement communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wireless-Fidelity (Wi-Fi) interface). The memory 1005 may be high-speed random access memory (RAM) or stable non-volatile memory (NVM), such as a disk storage device. The memory 1005 may also optionally be a storage device independent of the aforementioned processor 1001.

[0059] Those skilled in the art will understand that Figure 1 The structure shown does not constitute a limitation on native vector graph databases and data structure construction devices, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0060] like Figure 1 As shown, the memory 1005, which serves as a storage medium, may include an operating system, a network communication module, a user interface module, and a native vector graph database and data structure construction program.

[0061] exist Figure 1In the native vector graph database and data structure construction device shown, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and memory 1005 in the native vector graph database and data structure construction device of the present invention can be set in the native vector graph database and data structure construction device. The native vector graph database and data structure construction device calls the native vector graph database and data structure construction program stored in the memory 1005 through the processor 1001 and executes the native vector graph database and data structure construction method provided in the embodiment of the present invention.

[0062] This invention provides a native vector graph database and a data structure construction method, referring to... Figure 2 , Figure 2 This is a flowchart illustrating the first embodiment of the native vector graph database and data structure construction method of the present invention.

[0063] In this embodiment, the native vector graph database and data structure construction method include the following steps:

[0064] Step S10: Obtain node data and edge data from traditional graph data.

[0065] It should be noted that the execution entity in this embodiment is the native vector graph database and data structure construction device. It can also be other devices that implement the same or similar functions. This embodiment does not impose specific restrictions on this. This embodiment uses the native vector graph database and data structure construction device as an example for explanation.

[0066] As is understandable, graph data is the managed object of a graph database, and knowledge graphs are stored in the form of graph data in a graph database. Graph data consists of nodes and edges, both of which can contain properties. Nodes typically represent entities, such as people, places, organizations, or concepts. Edges represent relationships between nodes, such as friendships, family relationships, or work relationships. Each node and edge can have multiple properties, which are used to describe the characteristics or state of the node or edge. For example, a person node might have properties such as name, age, and gender, while an edge might have properties such as relationship type, weight, timestamp, and membership.

[0067] In its implementation, the native vector graph database consists of four parts: graph rendering layer, graph data layer, graph algorithm layer, and physical layer. The graph data layer includes node data, edge data, and attribute data of the graph data. The graph data layer is the logical organization layer of the knowledge graph nodes and edge relationships.

[0068] Furthermore, before acquiring node data and edge data from traditional graph data, the method further includes: acquiring business requirement information and constructing a knowledge graph ontology structure based on the business requirement information; acquiring business knowledge information and generating structured data based on the business knowledge information; establishing mapping relationships between the structured data and nodes and edges respectively; generating a business knowledge graph based on the mapping relationships and the knowledge graph ontology structure; constructing a traditional graph database based on the business knowledge graph; and generating traditional graph data based on the traditional graph database.

[0069] It should be noted that, based on the business needs of different fields, the knowledge graph ontology structure is designed, business knowledge is summarized to form structured data, and mapping relationships with nodes and edges are established. Based on the constraint relationships of the ontology structure, instance nodes and edges are attached to the ontology concept to form a business knowledge graph.

[0070] Understandably, business knowledge graph data is transmitted to the corresponding initial database in the backend via JSON, XML, or other format files, and stored in the form of graph data to form the graph data layer of the native vector graph database.

[0071] Step S20: Vectorize the node data and the edge data to obtain node vectors and edge vectors.

[0072] It should be noted that the graph rendering layer, or knowledge graph, is a graphical structure used to organize and represent knowledge, showcasing the semantic relationships between entities and relations through the connections of nodes and edges. Traditional knowledge graph node and edge data are augmented with embedded vector information. Vectors exist as attributes of nodes or edges. All embedded vectors are formed by concatenating sub-embedded vectors with different semantic dimensions generated by different vector embedding models. Node-level vector embedding algorithms include Node2Vec, DeepWalk, and Line; edge-level vector embedding algorithms include Deep GraphKernels; subgraph-level vector embedding algorithms include SubGraph2Vec, Sub2Vec, and SimGNN; and full-graph-level vector embedding algorithms include Graph2Vec. This embodiment does not impose specific limitations on these algorithms.

[0073] Understandably, the graph algorithm layer includes classic graph algorithms and vector embedding algorithms. Vector embedding algorithms can be used to vectorize node and edge data. The embedding vectors generated by vector embedding algorithms are required to reflect the semantic features of nodes, edges, subgraphs, and the entire graph at multiple dimensions.

[0074] Step S30: Construct a first data structure and a second data structure based on at least one of the node vectors and node data, and the edge vectors and edge data.

[0075] It should be noted that the first data structure can be a data structure with node vectors and edge vectors as elements, such as a graph, network, or matrix. The first data structure is used to store the relationship between node vectors and edge vectors to support subsequent vector index construction and graph database reasoning.

[0076] Understandably, the second data structure can be a data structure with node data and edge data as elements, such as a table, dictionary, or array. The second data structure is used to store the original information of the node and edge data to support subsequent attribute queries and retrieval of nodes and edges.

[0077] Further, the step of constructing a first data structure and a second data structure based on at least one of the node vector and the node data, and the edge vector and the edge data, includes: generating a first key-value pair based on the node vector and the node data; constructing a first data structure based on the first key-value pair; generating a second key-value pair based on the edge vector and the edge data; and constructing a second data structure based on the second key-value pair.

[0078] It should be noted that the first data structure refers to (node ​​vector + node) data, which is formed by constructing key-value pairs from node vectors and node data, and the second data structure refers to (edge ​​vector + edge) data, which is formed by constructing key-value pairs from edge vectors and edge data.

[0079] It is understandable that the vectorized embedding algorithm is used to generate corresponding embedding vectors for the node and edge data in the knowledge graph, and form node and edge data in the format of {Key:Value} with the original node and edge data, where the Key corresponds to the embedding vector and the Value corresponds to the node or edge data.

[0080] Step S40: Construct a vector graph database based on the first data structure and the second data structure.

[0081] It should be noted that by integrating the first and second data structures, a vector graph database can be constructed. This database contains both the vector information of nodes and edges and retains their original data, thus achieving a fusion of vector database technology and graph database technology. After the vector graph database is constructed, it needs to be indexed to generate a vector index. The vector index is a core component of the vector database, used to support efficient vector retrieval and similarity matching. Common vector indexing methods include tree-based indexes (such as KD-trees and ball trees), hash-based indexes (such as locality-sensitive hashing), and quantization-based indexes. Through vector indexes, different levels of graph reasoning can be achieved. Specifically, the relationships and attributes between a target node or edge and other nodes or edges can be inferred by calculating their vector similarity. This reasoning can be based on local neighbor information or global graph structure information, thus realizing the different levels of reasoning capabilities of the graph database.

[0082] It is understandable that native vector graph databases are graph databases built based on key-value pair vector data, where the key of a key-value pair is a vector, and the value is node or edge data. By building native vector graph databases, it is possible to support efficient vector-based query and reasoning operations, thereby improving the query and reasoning performance of knowledge graphs.

[0083] In its implementation, the native vector graph database consists of four parts: a graph rendering layer, a graph data layer, a graph algorithm layer, and a physical layer. The overall working principle is as follows: The embedding algorithm in the graph algorithm layer is used to vectorize the node, edge, and attribute data in the graph data layer. This vectorized data is then combined with the original data to form key-value pairs, creating a new vector knowledge graph. This graph is displayed in the graph rendering layer and stored in a newly created physical space specifically for vectors in the physical layer. By adding multi-dimensional and hierarchical indexes for nodes, edges, subgraphs, and the entire graph to the index in the graph data layer, a native vector graph database is formed that possesses both the relational expression and fast search capabilities of traditional graph databases, as well as the vectorization, similarity search, recommendation, and other reasoning functions of vector databases.

[0084] like Figure 3 As shown, Figure 3 This is a technical principle diagram. Based on the graph neural network in vectorization technology, graph data is vectorized to generate node vectors and edge vectors. An index is built based on the node vectors and edge vectors. (Node vector + node) data and (edge ​​vector + edge) data are generated based on the node vectors, edge vectors, and node data and edge data in traditional graph data. A graph database is built based on the (node ​​vector + node) data and (edge ​​vector + edge) data. A vector graph database is built based on the graph database and the index.

[0085] It is worth noting that the physical layer refers to the physical medium used to store graph data, which can be one or a combination of media such as hard drives, flash memory, or physical memory used to store the actual data. In this embodiment, the primary technical approach to implementing the native vector graph database is to construct a native vector graph database and set up dedicated vector storage locations in the database's physical and data layers. These storage locations can be triples or tuples composed of key-value pairs such as {(node ​​vector: node)-(relation vector: relation)-(node ​​vector: node)} or {(node: node vector)-(relation: relation vector)-(node: vector)}. Specific physical storage can be achieved through adjoining matrices, adjoining lists, cross-linked lists, adjoining multi-tables, and edge set arrays, etc. This embodiment does not impose specific limitations on these methods.

[0086] like Figure 4 As shown, Figure 4 This is an application architecture diagram for a native vector graph database, including an application layer, a data layer, an algorithm layer, and a physical layer. The application layer includes knowledge graph applications and question-answering system applications; the data layer includes graph data and indexes, with graph data including vector nodes and vector edges, and vector indexes based on vector nodes and vector edges, as well as traditional graph data indexes; the algorithm layer includes vector embedding models, such as node2vec, deepwalk, Line, subgraph2vec, graph2vec, and traditional graph algorithms; the physical layer includes adjoining matrices, adjoining lists, cross-linked lists, and adjoining multi-tables.

[0087] Furthermore, the step of constructing a vector graph database based on the first data structure and the second data structure includes: generating a vector knowledge graph based on the first data structure and the second data structure; constructing a vector index based on the node vectors and edge vectors; and constructing a vector graph database based on the vector index and the vector knowledge graph.

[0088] It's important to note that a vector index layer has been reconstructed based on the embedded vectors of nodes and edges in the knowledge graph, placing it alongside traditional graph database indexes. The core of vector indexing is calculating the distances between vectors, such as Euclidean distance and cosine similarity. To improve computational efficiency, optimization algorithms such as KD-trees and BK-trees are typically used, and hash methods can also be employed for index construction. To ensure the efficiency of the vector index, specific index update and optimization strategies need to be implemented. For example, when inserting or deleting vectors, the vector database needs to update the index to maintain query efficiency. Common update strategies include incremental updates and full updates. Incremental updates only update the affected index entries, while full updates rebuild the entire index. Common optimization techniques include data sharding, caching strategies, and batch processing.

[0089] In the specific implementation, a native vector graph database with the function of generating embedding vectors for graph nodes and edges is constructed. Based on domain business requirements, a knowledge graph ontology structure is designed, business knowledge is summarized to form structured data, and mapping relationships with nodes and edges are established. Instance nodes and edges are attached to ontology concepts based on the constraints of the ontology structure, forming a business knowledge graph. The business knowledge graph data is transmitted to the corresponding vector graph database in the backend via JSON or other formats. It is parsed by a node relationship parsing engine and stored in the database as triplet structures such as {node-edge-node}, {node-attribute-attribute value}, and {node-vector-vector value}. An index is generated using traditional graph database indexing techniques. Training samples are constructed according to specific business domain requirements to train the vector embedding algorithm model. The trained vector embedding algorithm model generates corresponding embedding vectors based on the node and edge data in the knowledge graph, forming {Key:Value} key-value pairs of node and edge data. Based on the node and edge embedding vectors, a vector index that meets business requirements is constructed using similarity algorithms, clustering algorithms, etc. Develop and apply business systems that include business knowledge graphs and large language question answering systems based on native vector graph databases. Classic business scenarios include question answering systems, recommendation systems, and knowledge bases.

[0090] It's worth noting that training vector embedding algorithm models can involve choosing Continuous Bag-of-Words (CBOW) and Skip-Gram models as language models, building a base network based on Graph Neural Networks (GNNs) such as GCN and GAN, or training the neural network using a Siamese neural network model. A Siamese neural network is a special neural network architecture that shares weights between two or more sub-networks with identical structures. Its working principle is to determine the similarity between input samples by comparing the outputs of two sub-networks. During training, the network attempts to minimize the distance between the outputs of two networks. If two samples are similar, their outputs should be very close; if two samples are dissimilar, their outputs should have significant differences.

[0091] like Figure 5 As shown, Figure 5This diagram illustrates the training principle of a vector embedding model based on a Siamese network structure. The Siamese network structure includes input layers Input-A and Input-B, graph neural networks GNN-A and GNN-B, and a similarity calculation layer. GNN-A and GNN-B share weights. Knowledge graphs KG-A and KG-B are input into GNN-A and GNN-B through Input-A and Input-B, respectively, outputting vectors Vector-A and Vector-B. Vector-A and Vector-B are then input into the similarity calculation layer to calculate the similarity value.

[0092] Understandably, native vector graph databases possess both vector database and graph database functionalities. When performing knowledge retrieval, they can acquire prior business knowledge of local graph relation links, as well as business knowledge of relationships at different levels (node ​​level, edge level, subgraph level, and full graph level) of graphs similar to the retrieval object. By utilizing vector embedding technology based on deep graph neural networks, traditional graph databases based on node-level similarity search are extended to native vector graph databases with different levels of reasoning capabilities, achieving a deep integration of graph database technology and vector database technology.

[0093] like Figure 6 As shown, Figure 6 This diagram illustrates the technical relationships of the native vector graph database and data structure construction method implemented in this paper. Knowledge graphs are retrieved and stored in the graph database as graph data, which is the managed object of the graph database. ChatGPT-type question-answering systems rely on large language models. These models retrieve and store information in the vector database, which in turn relies on vectorization techniques. Vectorization techniques rely on deep neural networks. Graph neural networks, like other deep neural networks, are subclasses of deep neural networks, and their processing objects are graph data.

[0094] This embodiment acquires node and edge data from traditional graph data; vectorizes the node and edge data to obtain node vectors and edge vectors; constructs a first data structure and a second data structure based on at least one pair of node vectors and node data, and edge vectors and edge data; and constructs a vector graph database based on the first and second data structures. By constructing the first and second data structures from the node and edge vectors obtained from the vectorization of node and edge data, a vector graph database is built. This solves the problem that traditional vector graph databases cannot achieve reasoning at different levels of the graph, realizing the integration of graph database technology and vector database technology, and enabling reasoning at different levels of the graph.

[0095] refer to Figure 7 , Figure 7This is a flowchart illustrating the second embodiment of the native vector graph database and data structure construction method of the present invention.

[0096] Based on the first embodiment described above, the native vector graph database and data structure construction method of this embodiment includes the following in step S20:

[0097] Step S201: Vectorize the node data based on the node vector embedding model to generate node vectors.

[0098] It should be noted that vector embedding algorithms can be used to vectorize node and edge data. The embedded vectors generated by these algorithms must reflect multi-dimensional semantic features of nodes, edges, subgraphs, and the entire graph. Vector embedding algorithms are based on the premise that connected nodes in the graph are close in vector space, and that nodes with the same content, attributes, or properties in the graph are also close in vector space.

[0099] It is understood that node vector embedding models can include algorithms such as node2vec, deepwalk, and line, and this embodiment does not impose specific restrictions on them.

[0100] Furthermore, the step of vectorizing the node data based on the node vector embedding model to generate node vectors includes: generating first sub-embedding vectors with different semantic dimensions based on the node vector embedding model; concatenating the first sub-embedding vectors with different semantic dimensions to obtain node embedding vectors; combining the node embedding vectors with the node data to generate graph nodes; and determining node vectors based on the graph nodes.

[0101] It's important to note that generating first sub-embedded vectors with different semantic dimensions based on node vector embedding models refers to characterizing the semantic features of nodes from multiple angles or levels through different embedding algorithms or parameter settings. For example, node2vec can capture local and global information of nodes by adjusting the parameters of the random walk, thereby generating sub-embedded vectors with different semantic dimensions. Concatenating these first sub-embedded vectors with different semantic dimensions yields a richer node embedding vector that integrates semantic information from multiple dimensions. The concatenation process can be as simple as linking the individual sub-embedded vectors end-to-end to form a longer vector. Then, this node embedding vector is fused with the original node data to generate the final node vector. The fusion can be done through simple concatenation or by combining the two using a function or model. The goal is to preserve the information of the original node data while introducing the semantic information of the vector embedding, so that the generated node vector reflects both the original attributes of the node and its position and structural relationships in the graph.

[0102] It is understandable that by vectorizing node data, node vectors can be generated that contain both the original node data information and rich semantic features.

[0103] It is worth noting that the graph nodes contain node data and node vectors generated from the node data, and the node vectors can be extracted based on the graph nodes.

[0104] Step S202: Vectorize the edge data based on the edge vector embedding model to generate edge vectors.

[0105] It should be noted that the edge vector embedding model can have algorithm models such as TransE, TransH, TransR, RotatE, etc., and this embodiment does not impose specific restrictions on them.

[0106] Furthermore, the step of vectorizing the edge data based on the edge vector embedding model to generate edge vectors includes: generating second embedding vectors with different semantic dimensions based on the edge vector embedding model; concatenating the second embedding vectors with different semantic dimensions to obtain edge embedding vectors; combining the edge embedding vectors with the edge data to generate graph edges; and determining edge vectors based on the graph edges.

[0107] It should be noted that a similar method to that used for generating node vectors can be employed when generating edge vectors. First, a second sub-embedding vector with different semantic dimensions is generated using an edge vector embedding model. Then, these sub-embedding vectors are concatenated to obtain the edge embedding vector. Finally, the edge embedding vector is fused with the edge data to generate the edge vector. This ensures that the edge vector contains both information from the original edge data and rich semantic features.

[0108] It is understandable that by vectorizing the edge data, we can generate edge vectors that contain both the original edge data information and rich semantic features.

[0109] It is worth noting that the graph edges contain edge data and edge vectors generated from the edge data, and the edge vectors can be extracted based on the graph edges.

[0110] This embodiment vectorizes the node data using a node vector embedding model to generate node vectors, and vectorizes the edge data using an edge vector embedding model to generate edge vectors. By vectorizing the node and edge data using the corresponding vector embedding models, the semantic features of the data can be enriched.

[0111] Reference Figure 8 , Figure 8 This is a structural block diagram of the first embodiment of the native vector graph database and data structure construction device of the present invention.

[0112] like Figure 8As shown, the native vector graph database and data structure construction device proposed in this embodiment of the invention include:

[0113] The acquisition module 10 is used to acquire node data and edge data from traditional graph data.

[0114] The vectorization module 20 is used to vectorize the node data and the edge data to obtain node vectors and edge vectors.

[0115] The construction module 30 is used to construct a first data structure and a second data structure based on at least one of the node vectors and the node data, and the edge vectors and the edge data.

[0116] The construction module 30 is also used to construct a vector graph database based on the first data structure and the second data structure.

[0117] This embodiment acquires node and edge data from traditional graph data; vectorizes the node and edge data to obtain node vectors and edge vectors; constructs a first data structure and a second data structure based on at least one pair of node vectors and node data, and edge vectors and edge data; and constructs a vector graph database based on the first and second data structures. By constructing the first and second data structures from the node and edge vectors obtained from the vectorization of node and edge data, a vector graph database is built. This solves the problem that traditional vector graph databases cannot achieve reasoning at different levels of the graph, realizing the integration of graph database technology and vector database technology, and enabling reasoning at different levels of the graph.

[0118] In one embodiment, the vectorization module 20 is further configured to vectorize the node data based on a node vector embedding model to generate node vectors; and to vectorize the edge data based on an edge vector embedding model to generate edge vectors.

[0119] In one embodiment, the vectorization module 20 is further configured to generate first sub-embedding vectors with different semantic dimensions based on the node vector embedding model; concatenate the first sub-embedding vectors with different semantic dimensions to obtain node embedding vectors; combine the node embedding vectors with the node data to generate graph nodes; and determine node vectors based on the graph nodes.

[0120] In one embodiment, the vectorization module 20 is further configured to generate second embedding vectors with different semantic dimensions based on the edge vector embedding model; concatenate the second embedding vectors with different semantic dimensions to obtain edge embedding vectors; combine the edge embedding vectors with the edge data to generate graph edges; and determine edge vectors based on the graph edges.

[0121] In one embodiment, the construction module 30 is further configured to generate a first key-value pair based on the node vector and the node data; construct a first data structure based on the first key-value pair; generate a second key-value pair based on the edge vector and the edge data; and construct a second data structure based on the second key-value pair.

[0122] In one embodiment, the construction module 30 is further configured to generate a vector knowledge graph based on the first data structure and the second data structure; construct a vector index based on the node vectors and edge vectors; and construct a vector graph database based on the vector index and the vector knowledge graph.

[0123] In one embodiment, the acquisition module 10 is configured to acquire business requirement information, construct a knowledge graph ontology structure based on the business requirement information; acquire business knowledge information and generate structured data based on the business knowledge information; establish mapping relationships between the structured data and nodes and edges respectively; generate a business knowledge graph based on the mapping relationships and the knowledge graph ontology structure; construct a traditional graph database based on the business knowledge graph, and generate traditional graph data based on the traditional graph database.

[0124] Furthermore, to achieve the above objectives, the present invention also proposes a native vector graph database and data structure construction device, the native vector graph database and data structure construction device comprising: a memory, a processor, and a native vector graph database and data structure construction program stored on the memory and executable on the processor, the native vector graph database and data structure construction program being configured to implement the steps of the native vector graph database and data structure construction method described above.

[0125] Since this native vector graph database and data structure construction device adopts all the technical solutions of all the above embodiments, it has at least all the beneficial effects brought about by the technical solutions of the above embodiments, which will not be repeated here.

[0126] Furthermore, this embodiment of the invention also proposes a storage medium storing a native vector graph database and a data structure construction program. When the native vector graph database and data structure construction program is executed by a processor, it implements the steps of the native vector graph database and data structure construction method described above.

[0127] Since this storage medium adopts all the technical solutions of all the above embodiments, it has at least all the beneficial effects brought about by the technical solutions of the above embodiments, which will not be repeated here.

[0128] It should be understood that the above are merely illustrative examples and do not constitute any limitation on the technical solutions of the present invention. In specific applications, those skilled in the art can make settings as needed, and the present invention does not impose any restrictions on this.

[0129] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of this invention. In practical applications, those skilled in the art can select some or all of the workflow to achieve the purpose of this embodiment according to actual needs, and no restrictions are imposed here.

[0130] In addition, for technical details not described in detail in this embodiment, please refer to the native vector graph database and data structure construction method provided in any embodiment of the present invention, which will not be repeated here.

[0131] Furthermore, it should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0132] It should be understood that although the steps in the flowcharts of this application's embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least a portion of the sub-steps or stages of other steps.

[0133] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0134] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as read-only memory (ROM) / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0135] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A native vector graph database and data structure construction method, characterized in that, The method includes: Retrieve node and edge data from traditional graph data; The node data and the edge data are vectorized to obtain node vectors and edge vectors; Construct a first data structure and a second data structure based on at least one of the node vectors and node data, and the edge vectors and edge data; Construct a vector graph database based on the first data structure and the second data structure; The step of vectorizing the node data and the edge data to obtain node vectors and edge vectors includes: vectorizing the node data based on a node vector embedding model to generate node vectors; and vectorizing the edge data based on an edge vector embedding model to generate edge vectors. The step of constructing a first data structure and a second data structure based on at least one of the node vector and the node data, and the edge vector and the edge data, includes: generating a first key-value pair based on the node vector and the node data; constructing a first data structure based on the first key-value pair; generating a second key-value pair based on the edge vector and the edge data; and constructing a second data structure based on the second key-value pair. The step of constructing a vector graph database based on the first data structure and the second data structure includes: generating a vector knowledge graph based on the first data structure and the second data structure; constructing a vector index based on the node vectors and edge vectors; and constructing a vector graph database based on the vector index and the vector knowledge graph. The vector graph database is a native vector graph database, which includes a graph rendering layer, a graph data layer, a graph algorithm layer, and a physical layer. The graph algorithm layer includes the node vector embedding model and the edge vector embedding model; The graph rendering layer is used to display the vector knowledge graph; The vector knowledge graph is stored in the vector storage physical space of the physical layer; The graph data layer is used to store the node data and the edge data, and is provided with a graph data layer index; the graph data layer index includes index relationships at the node level, edge level, subgraph level, and full graph level.

2. The method as described in claim 1, characterized in that, The step of vectorizing the node data based on the node vector embedding model to generate node vectors includes: Generate first sub-embedding vectors with different semantic dimensions based on the node vector embedding model; The first sub-embedding vectors of different semantic dimensions are concatenated to obtain the node embedding vector; The node embedding vector is combined with the node data to generate graph nodes; Node vectors are determined based on the graph nodes.

3. The method as described in claim 2, characterized in that, The step of vectorizing the edge data based on the edge vector embedding model to generate edge vectors includes: Generate second embedding vectors with different semantic dimensions based on the edge vector embedding model; The second embedding vectors with different semantic dimensions are concatenated to obtain the edge embedding vector; The edge embedding vector is combined with the edge data to generate graph edges; The edge vectors are determined based on the graph edges.

4. The method as described in claim 1, characterized in that, Before obtaining node and edge data from traditional graph data, the process also includes: Obtain business requirement information and construct a knowledge graph ontology structure based on the business requirement information; Acquire business knowledge information and generate structured data based on the business knowledge information; Establish mapping relationships between the structured data and nodes and edges respectively; A business knowledge graph is generated based on the mapping relationship and the knowledge graph ontology structure. A traditional graph database is constructed based on the business knowledge graph, and traditional graph data is generated based on the traditional graph database.

5. A native vector graph database and data structure construction device, characterized in that, The native vector graph database and data structure construction device include: The acquisition module is used to acquire node data and edge data from traditional graph data; A vectorization module is used to vectorize the node data and the edge data to obtain node vectors and edge vectors; The construction module is used to construct a first data structure and a second data structure based on at least one of the node vectors and the node data, and the edge vectors and the edge data; The construction module is also used to construct a vector graph database based on the first data structure and the second data structure; Specifically, the vectorization module is used to vectorize the node data based on the node vector embedding model to generate node vectors; and to vectorize the edge data based on the edge vector embedding model to generate edge vectors. Specifically, the construction module is used to generate a first key-value pair based on the node vector and the node data; construct a first data structure based on the first key-value pair; generate a second key-value pair based on the edge vector and the edge data; construct a second data structure based on the second key-value pair; generate a vector knowledge graph based on the first data structure and the second data structure; construct a vector index based on the node vector and the edge vector; and construct a vector graph database based on the vector index and the vector knowledge graph. The vector graph database is a native vector graph database, which includes a graph rendering layer, a graph data layer, a graph algorithm layer, and a physical layer. The graph algorithm layer includes the node vector embedding model and the edge vector embedding model. The graph rendering layer is used to display the vector knowledge graph. The vector knowledge graph is stored in the vector storage physical space of the physical layer. The graph data layer is used to store the node data and the edge data, and is equipped with a graph data layer index. The graph data layer index includes node-level, edge-level, subgraph-level, and full-graph-level index relationships.

6. A native vector graph database and data structure construction device, characterized in that, The native vector graph database and data structure construction device includes: a memory, a processor, and a native vector graph database and data structure construction program stored on the memory and executable on the processor, wherein the native vector graph database and data structure construction program is configured to implement the native vector graph database and data structure construction method as described in any one of claims 1 to 4.

7. A storage medium, characterized in that, The storage medium stores a native vector graph database and a data structure construction program, which, when executed by a processor, implements the native vector graph database and data structure construction method as described in any one of claims 1 to 4.

Images