Data operation method and device, storage medium and computer program product

Abstract

The invention provides a data operation method and device, a storage medium and a computer program product, and relates to the technical field of computers. According to the method and the device, the graph data are stored through the tensor data structure, and the related operation of the graph data is executed by using the execution function formed by the tensor operators. Due to the fact that the tensor operator is optimally designed for various types of processors during design, various processors such as a CPU, a GPU and an NPU can be generally supported, various processing resources can be well utilized by the method, and the efficiency of graph data related operation is improved.

Description

Technical Field

[0001] This disclosure relates to the field of database technology, and in particular to a method, apparatus, storage medium, and computer program product for data manipulation. Background Technology

[0002] With the advent of the big data era, the use of graph data is becoming increasingly widespread. Graph databases, as databases specifically designed for storing, managing, and querying graph data, have also gained widespread use. However, many operations in general graph databases are quite complex, leading to relatively low processing efficiency. Summary of the Invention

[0003] This disclosure provides a method, apparatus, storage medium, and computer program product for data manipulation that can improve the efficiency of graph database operations.

[0004] Firstly, a data manipulation method is provided. This method can be applied to a graph database. In this graph database, graph data is stored, including vertices and edges connecting the vertices. The edges connecting the vertices include various edge types. Edge connection information for each edge type is stored using a tensor data structure. This edge connection information records the connection relationship between the source and target vertices connected by each edge of the corresponding edge type. The tensor data structure may include at least two tensors, where one tensor records the degree of a vertex at one end of each edge of the corresponding edge type, and the other tensor records the vertex at the other end of each edge of the corresponding edge type. The degree of a vertex indicates the number of edges connected to that vertex.

[0005] The processing steps of this method include: first, obtaining operation instructions for the graph database; then, determining the corresponding target execution function based on the operation instructions, wherein the target execution function includes one or more tensor operators; and finally, running the target execution function to process the edge connection information stored in the graph database through the tensor data structure mentioned above using the corresponding tensor operators, thereby obtaining the operation result of the target operation instructions.

[0006] In this way, graph data is stored using tensor data structures, and execution functions composed of tensor operators are used to perform related operations on the graph data. Because tensor operators are designed to be optimized for various types of processors, they can generally support various processors such as central processing units (CPUs), graphics processing units (GPUs), and neural processing units (NPUs). Therefore, this method can make good use of various processing resources to improve the efficiency of graph data-related operations.

[0007] In one possible implementation, the tensor data structure storing the corresponding edge connection information includes a first tensor data structure, which can consist of three one-dimensional tensors: a first tensor, a second tensor, and a third tensor. In the first tensor data structure, the first tensor records the source vertices; the second tensor records the out-degree of each source vertex under the corresponding edge type, following the order of the source vertices in the first tensor, where the out-degree indicates the number of edges originating from the corresponding source vertex under the corresponding edge type; and the third tensor records the target vertices connected to each edge of the corresponding edge type, following the order of the corresponding source vertices in the first tensor.

[0008] In this way, for the common scenario of high overall out-degree in graph databases, compared to the traditional table-based data structure (which records the two vertices of all edges), the amount of data can be significantly reduced, saving storage space and reducing the amount of data processing, thus improving processing efficiency. Moreover, for cases with a large number of vertices with an out-degree of 0, the reduction in data volume, storage space, and processing efficiency can be even more effective.

[0009] In one possible implementation, the tensor data structure storing the corresponding edge connection information includes a second tensor data structure, which can consist of three one-dimensional tensors: a fourth tensor, a fifth tensor, and a sixth tensor. In the second tensor data structure, the fourth tensor records the target vertices; the fifth tensor records the in-degree of each target vertex under the corresponding edge type, following the order of the target vertices in the fourth tensor, where the in-degree indicates the number of edges under the corresponding edge type that terminate at the corresponding source vertex; and the sixth tensor records the source vertices connected to each edge of the corresponding edge type, following the order of the corresponding target vertices in the fourth tensor.

[0010] In this way, for the common scenario of a high overall in-degree in graph databases, compared to the traditional table-based data structure (which records the two vertices of all edges), the data volume can be significantly reduced, saving storage space and reducing the amount of data processing, thus improving processing efficiency. Moreover, for cases with a large number of vertices with an in-degree of 0, the data volume can be reduced even more effectively, saving storage space and improving processing efficiency.

[0011] In one possible implementation, the tensor data structure storing edge connection information includes a third tensor data structure, which can consist of two one-dimensional tensors, namely the seventh tensor and the eighth tensor. In the third tensor data structure, the seventh tensor is used to record the out-degree of each source vertex under the corresponding edge type, taking all vertices in the graph database as source vertices, according to the first specified order of each source vertex in the graph database; the eighth tensor is used to record the target vertices connected to each edge of the corresponding edge type, according to the first specified order of the corresponding source vertices.

[0012] In this way, for the common case of high overall out-degree in graph databases, compared with the traditional table-based data structure (which records the two vertices of all edges), the amount of data can be significantly reduced to save storage space and reduce the amount of data processing to improve processing efficiency.

[0013] In one possible implementation, the tensor data structure storing edge connection information includes a fourth tensor data structure, which can consist of two one-dimensional tensors, namely the ninth tensor and the tenth tensor. In the fourth tensor data structure, the ninth tensor is used to record the in-degree of each target vertex under the corresponding edge type, according to a second specified order in the graph database, with all vertices in the graph database as target vertices; the tenth tensor is used to record the source vertices connected to each edge of the corresponding edge type, according to the second specified order of the corresponding target vertices.

[0014] In this way, for common cases of high overall in-degree in graph databases, compared with the traditional table-based data structure (which records the two vertices of all edges), the amount of data can be significantly reduced, thus saving storage space and reducing the amount of data processing, thereby improving processing efficiency.

[0015] In one possible implementation, within the graph database, edge connection information for each edge type can be stored using its respective tensor data structure. The corresponding rule could be: if the edge connection information meets a specified condition, it is stored using a first specified tensor data structure; otherwise, it is stored using a second specified tensor data structure.

[0016] This improves the flexibility of graph database data storage and allows for the adoption of more suitable tensor data structures based on the characteristics of edge connection information for different edge types.

[0017] In one possible implementation, the specified condition could be: in the graph database, the proportion of vertices with an out-degree greater than 0 under the corresponding edge type is less than half. The first specified tensor data structure could be the first tensor data structure described above, and the second specified tensor data structure could be the third tensor data structure described above. The first tensor could be used to record all source vertices with an out-degree greater than 0 under the corresponding edge type.

[0018] Alternatively, the specified condition could be: in the graph database, the proportion of vertices with an in-degree greater than 0 under the corresponding edge type is less than half. The first specified tensor data structure could be the second tensor data structure described above, and the second specified tensor data structure could be the fourth tensor data structure described above. The fourth tensor can be used to record all source vertices with an in-degree greater than 0 under the corresponding edge type.

[0019] Thus, when less than half of the vertices have an out-degree greater than 0, the first tensor data structure requires less data and correspondingly less processing compared to the third tensor data structure. In this case, using the first tensor data structure can better save storage and processing resources and improve processing efficiency, and vice versa. Similarly, when less than half of the vertices have an in-degree greater than 0, the second tensor data structure requires less data and correspondingly less processing compared to the fourth tensor data structure. In this case, using the second tensor data structure can better save storage and processing resources and improve processing efficiency, and vice versa.

[0020] In one possible implementation, the process of determining the target execution function based on the operation instruction can be as follows: First, determine the target edge type corresponding to the operation instruction and obtain the initial execution function corresponding to the operation instruction; further, if the edge connection information corresponding to the target edge type is stored through a first specified tensor data structure, then the initial execution function is determined as the target execution function; if the edge connection information corresponding to the target edge type is stored through a second specified tensor data structure, then the specified statement is deleted from the initial execution function, and the initial execution function after deleting the statement is determined as the target execution function.

[0021] In this way, for the same operation instruction, only one initial execution function needs to be stored for different tensor data structures, instead of storing multiple execution functions, which can reduce the consumption of storage resources.

[0022] In one possible implementation, the process of determining the target execution function based on the operation instruction can be as follows: First, determine the target edge type corresponding to the operation instruction; then, determine the type of the target tensor data structure that stores the edge connection information corresponding to the target edge type; and finally, based on the correspondence between the operation instruction, the type of the tensor data structure, and the execution function, determine the target execution function that corresponds to both the current operation instruction and the type of the target tensor data structure.

[0023] In this way, appropriate execution functions can be set for different types of tensor data structures for the same operation instruction, which has high flexibility and can better take advantage of the priority of different tensor data structures to optimize the performance of graph database.

[0024] In one possible implementation, a traditional graph database can be transformed into a graph database with tensor data structures. The corresponding process can be as follows: First, in the initial graph database, obtain the edge connection information corresponding to each edge type that is stored through other data structures, and convert the corresponding edge connection information from other data structures into tensor data structures; then, based on the converted edge connection information corresponding to each edge type, obtain the graph database with tensor data structures.

[0025] This allows for more flexible use of the solution. For traditional graph databases, they can be loaded as graph databases with tensor data structures during the loading process, thereby improving the processing efficiency of graph databases.

[0026] Secondly, a data query apparatus is provided, the apparatus comprising at least one module for implementing the method provided in the first aspect and its possible implementations.

[0027] Thirdly, a computing device cluster is provided, including at least one computing device, each computing device including a processor and a memory; the processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device, such that the computing device cluster performs the methods provided by the first aspect and its possible implementations described above.

[0028] Fourthly, a computer-readable storage medium is provided, comprising computer program instructions that, when executed by a cluster of computing devices, perform the methods provided in the first aspect and its possible implementations.

[0029] Fifthly, a computer program product containing instructions is provided, which, when run by a cluster of computing devices, causes the cluster of computing devices to perform the method provided in the first aspect and its possible implementations. Attached Figure Description

[0030] Figure 1 This is a schematic diagram of graph data provided in an embodiment of this disclosure;

[0031] Figure 2 This is a schematic diagram of a first tensor data structure provided in an embodiment of this disclosure;

[0032] Figure 3This is a schematic diagram of a second tensor data structure provided in an embodiment of this disclosure;

[0033] Figure 4 This is a schematic diagram of a third tensor data structure provided in an embodiment of this disclosure;

[0034] Figure 5 This is a schematic diagram of a fourth tensor data structure provided in an embodiment of this disclosure;

[0035] Figure 6 This is a schematic diagram of a fifth tensor data structure provided in an embodiment of this disclosure;

[0036] Figure 7 This is a schematic diagram of a sixth tensor data structure provided in an embodiment of this disclosure;

[0037] Figure 8 This is a schematic diagram of a data manipulation method processing flow provided in an embodiment of this disclosure;

[0038] Figure 9 This is a schematic diagram of the execution flow of an execution function provided in an embodiment of this disclosure;

[0039] Figure 10 This is a schematic diagram of graph data provided in an embodiment of this disclosure;

[0040] Figure 11 This is a schematic diagram of a first tensor data structure provided in an embodiment of this disclosure;

[0041] Figure 12 This is a schematic diagram illustrating a method for processing execution functions provided in an embodiment of this disclosure;

[0042] Figure 13 This is a schematic diagram illustrating a method for processing execution functions provided in an embodiment of this disclosure;

[0043] Figure 14 This is a schematic diagram illustrating a method for processing execution functions provided in an embodiment of this disclosure;

[0044] Figure 15 This is a schematic diagram illustrating a method for processing execution functions provided in an embodiment of this disclosure;

[0045] Figure 16 This is a schematic diagram illustrating a method for processing execution functions provided in an embodiment of this disclosure;

[0046] Figure 17 This is a schematic diagram illustrating a method for processing execution functions provided in an embodiment of this disclosure;

[0047] Figure 18 This is a schematic diagram illustrating a method for processing execution functions provided in an embodiment of this disclosure;

[0048] Figure 19 This is a schematic diagram of the execution flow of an execution function provided in an embodiment of this disclosure;

[0049] Figure 20 This is a schematic diagram illustrating a method for processing execution functions provided in an embodiment of this disclosure;

[0050] Figure 21 This is a schematic diagram illustrating a method for processing execution functions provided in an embodiment of this disclosure;

[0051] Figure 22 This is a schematic diagram illustrating a method for processing execution functions provided in an embodiment of this disclosure;

[0052] Figure 23 This is a schematic diagram illustrating a method for processing execution functions provided in an embodiment of this disclosure;

[0053] Figure 24 This is a schematic diagram illustrating a method for processing execution functions provided in an embodiment of this disclosure;

[0054] Figure 25 This is a schematic diagram illustrating a method for processing execution functions provided in an embodiment of this disclosure;

[0055] Figure 26 This is a schematic diagram illustrating a method for processing execution functions provided in an embodiment of this disclosure;

[0056] Figure 27 This is a schematic diagram of the execution flow of an execution function provided in an embodiment of this disclosure;

[0057] Figure 28 This is a schematic diagram of the execution flow of an execution function provided in an embodiment of this disclosure;

[0058] Figure 29 This is a schematic diagram of the execution flow of an execution function provided in an embodiment of this disclosure;

[0059] Figure 30 This is a schematic diagram of the execution flow of an execution function provided in an embodiment of this disclosure;

[0060] Figure 31 This is a schematic diagram of the structure of a data manipulation apparatus provided in an embodiment of this disclosure;

[0061] Figure 32 This is a schematic diagram of a computing device provided in an embodiment of this disclosure;

[0062] Figure 33 This is a schematic diagram of a computing device cluster provided in an embodiment of this disclosure;

[0063] Figure 34 This is a schematic diagram of a computing device cluster provided in an embodiment of this disclosure. Detailed Implementation

[0064] To make the objectives, technical solutions, and advantages of this disclosure clearer, the embodiments of this disclosure will be described in further detail below with reference to the accompanying drawings.

[0065] The concepts involved in this disclosure are explained below:

[0066] Image data

[0067] Graph data consists of a large number of vertices and edges.

[0068] Vertices can be used to represent people, objects, knowledge, etc.

[0069] In graph data, edges connect two vertices, representing the relationship between them. For example, edges can represent friendships or colleague relationships between accounts, or connections between devices, and so on. Edges can be divided into undirected edges and directed edges. For instance, the friendship between account A and account B can be represented as an undirected edge between A and B, and account A following account B can be represented as a directed edge from A to B. In data processing, an undirected edge can be considered a combination of two directed edges for appropriate processing.

[0070] In this way, graph data can be used to represent relationships between people, between people and things, between things, and between pieces of knowledge. Figure 1 An example of graph data is given. For instance, graph data can be used to represent social networks, communication network topologies, and knowledge graphs, etc.

[0071] Source vertex and destination vertex

[0072] The starting vertex of a directed edge can be called the source vertex, and the ending vertex can be called the destination vertex. The two vertices connected by an undirected edge are each other's source and destination vertices.

[0073] Neighbor

[0074] An edge (directed or undirected) connects two vertices that are neighbors. In the case of directed edges, the target vertex can also be called the out-neighbor of the source vertex, and the source vertex can also be called the in-neighbor of the target vertex.

[0075] Graph database

[0076] Graph databases are specialized databases used for storing, managing, and querying graph-related data. When storing graph data, graph databases can store information such as edge connectivity, edge attributes, and vertex attributes.

[0077] Edge connection information

[0078] Edge connection information, also known as neighbor relationship information, is used to record the connection relationship between the source vertex and the target vertex connected by each edge. In related technologies, Figure 1 The edge connection information of the graph data shown can be recorded in Table 1 below.

[0079] Table 1

[0080] 0 2 0 5 2 0 2 5 3 0 4 1 4 2 4 5 5 3

[0081] Graph databases can include multiple edge types. For example, friendship is one edge type, colleague relationships are another, and family relationships are yet another. Correspondingly, edge connection information can be set separately for each edge type. That is, each edge connection information is only used to record the connection relationship between the source vertex and the target vertex connected by each edge of that edge type.

[0082] Edge attribute information

[0083] Edge attribute information can be set according to the needs of the actual application scenario. For example, edge attribute information can be the time when the friendship was established, the level of intimacy, etc.

[0084] Vertex attribute information

[0085] Vertex attribute information can be set according to the needs of the actual application scenario. For example, vertex attribute information can be account level, device hardware information, etc.

[0086] Spend

[0087] Degree is used to indicate the number of edges connected to a given vertex. Degree can be divided into outdegree and indegree.

[0088] Out-degree: For a given vertex, the number of edges originating from that vertex is its out-degree. In other words, the out-degree indicates the number of edges originating from the corresponding source vertex. For example, Figure 1 In the example, vertex 0 has an out-degree of 2, and vertex 4 has an out-degree of 3.

[0089] When there are multiple edge types in the graph database, the out-degree of any vertex under different edge types can be calculated separately.

[0090] In-degree, for a given vertex, is the number of edges that have that vertex as their destination vertex. In other words, in-degree indicates the number of edges that terminate at the corresponding source vertex. For example, Figure 1 In the example, vertex 3 has an in-degree of 1, and vertex 5 has an in-degree of 3.

[0091] When there are multiple edge types in the graph database, the in-degree of any vertex under different edge types can be calculated separately.

[0092] tensor

[0093] In mathematics, a tensor is a one-dimensional or multi-dimensional array. In computer science, tensors generally have a specific data format.

[0094] Tensor data structure

[0095] A tensor data structure is a data structure composed of one or more tensors. This disclosure provides various tensor data structures, which will be described in detail later.

[0096] tensor operators

[0097] Tensor operators are fundamental functions used to perform tensor computations. Various tensor computation runtime (TCR) frameworks, such as PyTorch and TVM, provide a large number of tensor operators.

[0098] Tensor operators are characterized by performing a large number of repetitive simple operations and generally support running on various processors, such as central processing units (CPUs), graphics processing units (GPUs), and neural processing units (NPUs).

[0099] With the widespread use of graph data, graph databases, as databases specifically designed for storing, managing, and querying graph data, have also gained widespread adoption. Graph databases involve a large number of operations such as adding, deleting, modifying, and querying, placing very high demands on the hardware of the devices. In particular, query operations involve complex processing and a large number of queries, consuming significant device resources.

[0100] In related technologies, graph databases use ordinary array data structures to store graph data, including edge connection information, edge attribute information, and vertex attribute information. However, operators designed for ordinary array data structures can only run on the CPU and cannot make good use of computing resources such as GPUs and NPUs. This leads to relatively low operational efficiency.

[0101] This disclosure provides a data manipulation method that stores graph data using a tensor data structure and uses execution functions composed of tensor operators to perform related operations on the graph data. Because tensor operators generally support various processors such as CPUs, GPUs, and NPUs, this method can effectively utilize various processing resources to improve the efficiency of graph data operations.

[0102] This method can be executed by a computer device, which can be a server or a terminal device. A database master program can run on the computer device to implement the method. This disclosure uses a server providing cloud services as an example to illustrate the solution; other cases are similar and will not be described in detail.

[0103] The tensor data structure provided in the embodiments of this disclosure is described below:

[0104] 1. Tensor data structure for storing edge connection information

[0105] This tensor structure can include at least two tensors. One of the tensors records the degree (out-degree or in-degree) of a vertex at one end of each edge of the corresponding edge type, and the other tensor records the vertex at the other end of each edge of the corresponding edge type. Vertex records in the tensor data structure can be in the form of vertex identifiers. Compared to recording all vertices at both ends of each edge, this tensor data structure reduces the amount of data that needs to be stored, and correspondingly, also reduces the amount of processing involved in related operations.

[0106] The following content provides further explanation of this tensor data structure.

[0107] (1) First Tensor Data Structure

[0108] The first tensor data structure includes a first tensor, a second tensor, and a third tensor.

[0109] The first tensor is used to record the source vertex.

[0110] The second tensor is used to record the out-degree of each source vertex under the corresponding edge type, in the order of the source vertices in the first tensor.

[0111] The third tensor is used to record the target vertices connected to each edge of the corresponding edge type in the order of the corresponding source vertices in the first tensor.

[0112] The second tensor is one of the aforementioned at least two tensors (the tensor recording the degree), and the third tensor is the other of the aforementioned at least two tensors (the tensor recording the other end vertex). The first tensor records all source vertices with an out-degree greater than 0 under the corresponding edge type, but does not record vertices with an out-degree equal to 0 under that edge type, and the recorded vertices are not duplicated. Based on this characteristic, the first tensor data structure can be called a compressed uniquesource (CUS) data structure.

[0113] like Figure 2 As shown, is Figure 1 The graph data in the image corresponds to the CUS data structure with edge connection information. It can be seen that in the first tensor, the source vertex is unique and non-repeating, and in the second tensor, all out-degrees are greater than 0. Figure 2 In the first tensor, the first source vertex is vertex 0. The first out-degree 2 recorded in the second tensor is the out-degree of vertex 0. Based on this out-degree 2, the first two target vertices, namely vertex 2 and vertex 5, are determined in the third tensor, which are the neighbors of vertex 0.

[0114] (2) Second Tensor Data Structure

[0115] The second tensor data structure includes the fourth, fifth, and sixth tensors.

[0116] The fourth tensor is used to record the target vertex.

[0117] The fifth tensor is used to record the in-degree of each target vertex under the corresponding edge type, according to the order of the target vertices in the fourth tensor.

[0118] The sixth tensor is used to record the source vertices connected to each edge of the corresponding edge type in the order of the corresponding target vertices in the fourth tensor.

[0119] This fifth tensor is one of the aforementioned at least two tensors (the tensor recording the degree), and this sixth tensor is the other of the aforementioned at least two tensors (the tensor recording the other end vertex). The fourth tensor records all target vertices with an in-degree greater than 0 under the corresponding edge type, but does not record vertices with an in-degree equal to 0 under that edge type, and the recorded vertices are unique. Based on this characteristic, the fourth tensor data structure can be called a compressed unique target (CUT) data structure.

[0120] like Figure 3 As shown, is Figure 1 The graph data in the image contains edge connection information corresponding to the CUT data structure. It can be seen that in the fourth tensor, the target vertex is unique and non-repeating, and in the fifth tensor, all vertices have an in-degree greater than 0.

[0121] (3) Third Tensor Data Structure

[0122] The third tensor data structure includes the seventh and eighth tensors.

[0123] The seventh tensor is used to record the out-degree of each source vertex under the corresponding edge type, according to the first specified order of each source vertex in the graph database, taking all vertices in the graph database as source vertices.

[0124] The eighth tensor is used to record the target vertices connected to each edge of the corresponding edge type in the first specified order of the corresponding source vertices.

[0125] This seventh tensor is one of the aforementioned at least two tensors (the tensor recording the degree), and this eighth tensor is the other of the aforementioned at least two tensors (the tensor recording the other vertex). The first specified order can be sorted by vertex identifier. It can be seen that the seventh tensor records the out-degrees of all vertices, and these out-degrees are arranged in the order of vertex identifiers. In this way, it is not necessary to set a tensor to record the source vertex to determine the out-degree of each vertex, thus saving one tensor. The third tensor data structure can be called the CUS (with virtual source-vertexidentification, vsid) data structure.

[0126] like Figure 4 As shown, is Figure 1 The graph data in the diagram uses the CUS(vsid) data structure to represent the edge connection information. The source vertex tensor shown by the dashed line does not actually exist; it is only used to indicate the vertex corresponding to each out-degree in the seventh vector. It can be seen that vertex 0 has an out-degree of 2, and vertex 1 has an out-degree of 0, meaning that vertex 1 is not the source vertex for this edge type. Other vertices are not described individually.

[0127] (4) Fourth Tensor Data Structure

[0128] The fourth tensor data structure includes the ninth and tenth tensors.

[0129] The ninth tensor is used to record the in-degree of each target vertex under the corresponding edge type, according to the second specified order of each target vertex in the graph database, with all vertices in the graph database as target vertices.

[0130] The tenth tensor is used to record the source vertices connected to each edge of the corresponding edge type in the second specified order of the corresponding target vertices.

[0131] This ninth tensor is one of the aforementioned at least two tensors (the tensor recording the degree), and this tenth tensor is the other of the aforementioned at least two tensors (the tensor recording the other vertex). The second specified order can be sorted by vertex identifier. It can be seen that the ninth tensor records the in-degree of all vertices, and these in-degrees are arranged in the order of vertex identifiers. In this way, it is not necessary to set a tensor to record the target vertex, and the in-degree of each vertex can be determined, thus saving one tensor. The fourth tensor data structure can be called the CUT(vsid) data structure.

[0132] like Figure 5 As shown, is Figure 1 The graph data in the image contains the CUT(vsid) data structure corresponding to the edge connection information. The target vertex tensor shown by the dashed line does not actually exist; it is only used to indicate the vertex corresponding to each in-degree in the ninth vector.

[0133] 2. Tensor data structure for storing edge attribute information

[0134] In graph databases, edge attribute information for each edge type is stored using tensor data structures. These tensor data structures include a fifth tensor data structure, such as... Figure 6 As shown, the fifth tensor data structure includes at least one eleventh tensor, each of which records one edge attribute information for each edge of the edge type.

[0135] For an eleventh tensor, each value corresponds to an edge attribute. These edge attributes are arranged in a specified order (e.g., identifier order) of the source vertices of the corresponding edge, or they can be arranged in a specified order (e.g., identifier order) of the target vertices of the corresponding edge. In a graph database, two fifth tensor data structures can be created to store the edge attribute information. In one fifth tensor data structure, the edge attribute information is arranged in a specified order of the source vertices of the corresponding edge, and in the other fifth tensor data structure, the edge attribute information is arranged in a specified order of the target vertices of the corresponding edge.

[0136] When there are multiple edge attribute information, multiple eleventh-level quantities can be used to record them separately. For example, one eleventh-level quantity can represent the time when the friendship was established, another eleventh-level quantity can represent the level of intimacy, and so on.

[0137] 3. Tensor data structure for storing vertex attribute information

[0138] In a graph database, the vertex attribute information corresponding to each vertex is stored using a tensor data structure. This tensor data structure includes a sixth tensor data structure, such as... Figure 7 As shown, the sixth tensor data structure includes at least one twelfth tensor, each twelfth tensor being used to record one vertex attribute information for each vertex.

[0139] For a twelfth tensor, each value corresponds to a vertex attribute, and these vertex attributes are arranged in a specified order (such as the identifier order) of the corresponding vertices.

[0140] When there are multiple vertex attribute information, multiple twelfth tensors can be used to record them separately. For example, one twelfth tensor represents the remaining storage space of the device, another twelfth tensor represents the device's processor utilization, and so on.

[0141] In practical applications, there are multiple possible ways to form a graph database with the above tensor data structure, which will be introduced in several cases below.

[0142] Scenario 1: The existing graph database (which can be called the initial graph database) uses other data structures (i.e., non-tensor data structures, or other tensor data structures besides the tensor data structures mentioned above). The data structure conversion can be performed during the loading process (loading the graph database from the hard disk into memory) before the initial graph database runs.

[0143] The corresponding processing could be as follows: In the initial database, obtain the edge connection information corresponding to each edge type stored in other data structures, convert the edge connection information corresponding to each edge type from other data structures into tensor data structures; and obtain the graph database based on the converted edge connection information corresponding to each edge type.

[0144] When performing data structure transformation, a selection mechanism can be set. For edge connection information corresponding to any edge type, if the edge connection information meets the specified conditions, the edge connection information is transformed from other data structures into the first specified tensor data structure; if the edge connection information does not meet the specified conditions, the edge connection information is transformed from other data structures into the second specified tensor data structure.

[0145] The first specified tensor data structure, the second specified tensor data structure, and the specified conditions can be set based on actual needs. In one possible scheme, the first specified tensor data structure may include the first tensor data structure and / or the second tensor data structure mentioned above, the second specified tensor data structure may include the third tensor data structure and / or the fourth tensor data structure mentioned above, and the corresponding specified conditions may be that the proportion of vertices with an out-degree greater than 0 under the corresponding edge type is less than half, or that the proportion of vertices with an in-degree greater than 0 under the corresponding edge type is less than half.

[0146] For example, in the case of undirected edges, if the proportion of vertices with an out-degree (or in-degree, which are equal) greater than 0 is less than half, the edge connection information can be converted from other data structures to both the first and second tensor data structures, that is, storing the edge connection information of both tensor data structures. If the proportion of vertices with an out-degree greater than 0 is not less than half, the edge connection information can be converted from other data structures to both the third and fourth tensor data structures, that is, storing the edge connection information of both tensor data structures.

[0147] Thus, when the proportion of vertices with an out-degree greater than 0 is less than half, the first and second tensor data structures have less data volume and correspondingly less processing volume compared to the third and fourth tensor data structures. In this case, using the first and second tensor data structures can better save storage and processing resources and improve processing efficiency, and vice versa.

[0148] For example, in the case of directed edges, if the proportion of vertices with an out-degree greater than 0 is less than half, the edge connection information can be converted from other data structures to the first tensor data structure. If the proportion of vertices with an out-degree greater than 0 is not less than half, the edge connection information can be converted from other data structures to the third tensor data structure.

[0149] Since both the first and third tensor data structures involve out-degree, out-degree is used as the selection criterion.

[0150] Thus, when the proportion of vertices with an out-degree greater than 0 is less than half, the first tensor data structure has less data volume and correspondingly less processing volume compared to the third tensor data structure. In this case, using the first tensor data structure can better save storage and processing resources and improve processing efficiency, and vice versa.

[0151] For example, in the case of directed edges, if the proportion of vertices with an in-degree greater than 0 is less than half, the edge connection information can be converted from other data structures to a second tensor data structure. If the proportion of vertices with an in-degree greater than 0 is not less than half, the edge connection information can be converted from other data structures to a fourth tensor data structure.

[0152] Since both the second and fourth tensor data structures involve in-degree, in-degree is used as the selection criterion.

[0153] Thus, when the proportion of vertices with an in-degree greater than 0 is less than half, the second tensor data structure has less data volume and correspondingly less processing volume compared to the fourth tensor data structure. In this case, using the second tensor data structure can better save storage and processing resources and improve processing efficiency, and vice versa.

[0154] In addition to transforming the data structure of the edge connection information in the initial database, the data structure of the edge attribute information in the initial database can also be transformed. That is, the edge attribute information corresponding to each edge type in the initial database can be converted from other data structures to tensor data structures, specifically the fifth tensor data structure mentioned above. Moreover, the data structure of the vertex attribute information in the initial database can also be transformed. That is, the vertex attribute information in the initial database can be converted from other data structures to tensor data structures, specifically the sixth tensor data structure mentioned above.

[0155] Scenario 2: Existing graph databases use other data structures, and the data structure of graph databases stored on hard drives can be converted.

[0156] The handling of Case 2 is similar to that of Case 1, with the only difference being that Case 1 involves structural conversion when reading data from the hard drive into memory, while Case 2 involves structural conversion of the data on the hard drive itself. The specific processing logic can be the same or similar; please refer to the description of Case 1, which will not be repeated here.

[0157] Scenario 3: In the newly established graph database, the tensor data structure described above is used directly for graph data storage.

[0158] Technicians can pre-define the corresponding tensor data structure for the edge connection information of each edge type. During this process, they can estimate the approximate size of the edge connection information for each edge type after the graph database reaches a certain scale, and then select a tensor data structure for each edge type based on this estimate. The selection process can still utilize the specified conditions from Case 1, and the corresponding processing is similar to Case 1, so it will not be elaborated further here.

[0159] During the use of the graph database, the tensor data structure used to store the edge connection information corresponding to each edge type can be periodically readjusted. The corresponding processing can be similar to that in Case 1. The triggering condition for readjustment can be periodic, manual, or data size-based (triggered once every time the data volume of the graph database reaches a certain data size threshold). In this way, the graph database is dynamically maintained in the following state: for the edge connection information corresponding to each edge type, if the edge connection information meets the specified conditions, the edge connection information is stored through the first specified tensor data structure; if the edge connection information does not meet the specified conditions, the edge connection information is stored through the second specified tensor data structure.

[0160] For graph databases with the aforementioned tensor data structure, the following data manipulation methods can be used to execute operation instructions, and the corresponding processing can be as follows: Figure 8 As shown, it includes the following steps:

[0161] 801, retrieves operation commands for the graph database.

[0162] The operation instruction can be an instruction for various basic operations in the graph database, such as query, add, delete, and modify operations. Query-related operations involve many subdivided basic operations, such as neighbor query and subgraph query.

[0163] In practical applications, the database main program can run on a cloud server to provide database services to users through cloud services. Alternatively, the database main program can run on a server built by the user themselves. In these cases, the user can input a single operation command or a program consisting of multiple operation commands via a terminal, which is then sent to the server for execution. Alternatively, the database main program can run directly on the terminal, where the user inputs a single operation command or a program consisting of multiple operation commands, which is then executed directly by the terminal. Step 801 could involve obtaining a single operation command directly input by the user.

[0164] Additionally, during the execution of user-inputted operation instructions by the server or terminal running the database main program, the user-inputted operation instructions may be translated into other operation instructions before being executed. Alternatively, an operation instruction may trigger the execution of a program that may contain multiple operation instructions, which can then be executed one by one. Step 801 may involve obtaining the translated operation instructions or obtaining a specific operation instruction from that program segment.

[0165] 802, Based on the operation instructions, determine the target execution function.

[0166] The operation instruction can be a function call instruction, used to trigger the invocation of the target execution function. The target execution function includes at least one tensor operator and may contain one or more statements, some or all of which can be execution statements of tensor operators, while some statements may not involve tensor operators. An acceleration library can be built within the database main program, where various execution functions and tensor operators can be stored and invoked for execution.

[0167] When performing step 802, the target execution function can be directly called from the acceleration library using operation instructions. Alternatively, the target execution function to be used can be determined through a series of processes. Two feasible processing methods are given below:

[0168] Method 1 (for situations where edge connection information for certain edge types is stored in a graph database using a first specified tensor data structure, and edge connection information for other edge types is stored using a second specified tensor data structure)

[0169] Determine the target edge type corresponding to the operation instruction and obtain the initial execution function corresponding to the operation instruction. If the edge connection information corresponding to the target edge type is stored through a first specified tensor data structure, then the initial execution function is determined as the target execution function. If the edge connection information corresponding to the target edge type is stored through a second specified tensor data structure, then delete the specified statement in the initial execution function and determine the initial execution function after deleting the statement as the target execution function.

[0170] Wherein, if the first specified tensor data structure is the first tensor data structure, the second specified tensor data structure can be the third tensor data structure. If the first specified tensor data structure is the second tensor data structure, the second specified tensor data structure can be the fourth tensor data structure.

[0171] When designing execution functions, based on the characteristics of the first and third tensor data structures, engineers can design execution functions for the first and third tensor data structures to share some common statements. The execution function for the first tensor data structure will have several additional statements compared to the one for the third tensor data structure. When storing the execution functions, the function for the first tensor data structure can be stored as the initial execution function for the operation instruction. Deleting the specified statements from the initial execution function will yield the execution function for the third tensor data structure.

[0172] In this way, if the edge connection information of the target edge type corresponding to the currently executed operation instruction is stored in the first tensor data structure, the initial execution function can be directly called as the target execution function for subsequent execution. If the edge connection information of the target edge type corresponding to the currently executed operation instruction is stored in the third tensor data structure, the initial execution function can be called, and specified statements can be deleted in the initial execution function. The deleted initial execution function can then be used as the target execution function for subsequent execution.

[0173] The processing of the second and fourth tensor data structures is the same as that of the first and third tensor data structures described above, and will not be repeated in this embodiment.

[0174] Method 2 (for situations where edge connection information for different edge types is stored in a graph database using their respective tensor data structures, which can be the same as the situation addressed by Method 1)

[0175] Determine the target edge type corresponding to the operation instruction. Determine the type of the target tensor data structure that stores the edge connection information corresponding to the target edge type. Based on the correspondence between the operation instruction, the type of the target tensor data structure, and the execution function, determine the target execution function that corresponds to both the obtained operation instruction and the type of the target tensor data structure.

[0176] Technicians can pre-establish a table showing the correspondence between operation instructions, tensor data structure types, and execution functions. Table 2 below provides an example.

[0177] Table 2

[0178] Operation instructions Types of tensor data structures Execution function Instruction A First Tensor Data Structure Function 1 Instruction A Second Tensor Data Structure Function 2 Instruction A Third Tensor Data Structure Function 3 Instruction A Fourth Tensor Data Structure Function 4 Instruction B First Tensor Data Structure Function 5 Instruction B Second Tensor Data Structure Function 6 …… …… ……

[0179] Upon obtaining the currently executing operation instruction, the target edge type corresponding to the operation instruction can be determined, thereby determining the type of the target tensor data structure used to store the edge connection information corresponding to the target edge type. Then, in the aforementioned correspondence table, using the operation instruction and the type, the execution function that they both correspond to, i.e., the target execution function, can be found.

[0180] 803, Run the target execution function to process the edge connection information stored in the graph database through tensor data structures using tensor operators, and obtain the operation result of the target operation instruction.

[0181] During the execution of the target function, the processing of edge attribute information and / or vertex attribute information may also be involved. If the edge attribute information and / or vertex attribute information are stored in a tensor data structure, the corresponding tensor operators can also be used to process them.

[0182] In this embodiment, graph data is stored using a tensor data structure, and execution functions composed of tensor operators are used to perform related operations on the graph data. Because tensor operators are designed to be optimized for various types of processors and can generally support various processors such as CPUs, GPUs, and NPUs, this method can make good use of various processing resources to improve the efficiency of graph data-related operations.

[0183] To improve the efficiency of graph database operations, technicians can design execution functions composed of tensor operators for common graph database operations. The following describes the operation process of several execution functions, including each step of the execution function, focusing on the execution functions related to query operations.

[0184] Execution function one

[0185] The corresponding operation instruction one is used to query the target vertex that has at least one first vertex as the source vertex under the target edge type (i.e., neighbor lookup).

[0186] This execution function can perform query processing using the edge connection information stored in the first tensor data structure. The corresponding function execution process can be as follows: Figure 9 As shown, it includes the following steps:

[0187] 901. Initialize the first output tensor, wherein the length of the first output tensor is the same as the length of the input tensor, and all values ​​included in the first output tensor are 0.

[0188] 902. Match the input tensor corresponding to the query instruction with the first tensor in the first tensor data structure corresponding to the target edge type to obtain a first index tensor and a second index tensor. The input tensor records the at least one first vertex, which is arranged in the input tensor in the order of input. The first index tensor records the position index of the first vertex that was successfully matched in the input tensor. The second index tensor records the position index of the source vertex that was successfully matched in the first tensor. The position indices in the first index tensor and the second index tensor correspond to each other according to the matching relationship.

[0189] 903, determine the reordering tensor required to reorder the first index tensor into a third index tensor, wherein the third index tensor is a tensor obtained by arranging the position indices of the first index tensor according to their order of position in the input tensor, and the reordering tensor is used to record the position index of each value in the third index tensor in the first index tensor.

[0190] 904. Based on the reordering tensor, the second index tensor is reordered to obtain the fourth index tensor.

[0191] 905. In the third tensor of the first tensor data structure corresponding to the target edge type, determine the position index of the target vertex corresponding to the fourth index tensor to form the fifth index tensor.

[0192] 906. For each position index in the fourth index tensor, in the second tensor of the first tensor data structure corresponding to the target edge type, the out-degree corresponding to the position index is determined. In the third index tensor, the target position index corresponding to the position index is determined. Based on the target position index, the out-degree is recorded in the first output tensor.

[0193] 907. In the third tensor, the target vertex corresponding to the fifth index tensor is determined to form the second output tensor.

[0194] 908, output the first output tensor and the second output tensor.

[0195] The following is about the above. Figure 9 The process is given as an example, assuming the target edge type in the graph database corresponds to the graph data as follows: Figure 10 As shown, the first tensor data structure (CUS data structure) that records the graph data is as follows: Figure 11 As shown, the first tensor src = [0, 2, 3, 5], the second tensor deg = [3, 2, 1, 2], and the third tensor dst = [2, 3, 5, 0, 5, 0, 0, 2].

[0196] The input tensor is inputIds = [4, 5, 3], and the length of the tensor is 3, which is the number of elements in the tensor.

[0197] For the 901 mentioned above, see [link / reference]. Figure 12 Create a first output tensor neighborCounts = [0, 0, 0] with the same length as the input tensor inputIds and all elements having a value of 0.

[0198] For the 902 mentioned above, see [link / reference]. Figure 13 The input tensor `inputIds` and the first tensor `src` are matched. The first element 0 in `src` has no match in `inputIds`, so it is not recorded. The second element 2 in `src` has no match in `inputIds`, so it is not recorded. The third element 3 in `src` has a matching element 3 in `inputIds`, so the match is successful. The index 2 of element 3 in `inputIds` is recorded in the first index tensor `leftOutIdx`, and the index 2 of element 3 in `src` is recorded in the second index tensor `rightOutIdx`. The fourth element 5 in `src` has a matching element 5 in `inputIds`, so the match is successful. The index 1 of element 5 in `inputIds` is recorded in the first index tensor `leftOutIdx`, and the index 3 of element 5 in `src` is recorded in the second index tensor `rightOutIdx`. This results in the first index tensor `leftOutIdx` = [2, 1] and the second index tensor `rightOutIdx` = [2, 3]. The above description describes multiple elements separately. However, in actual processing, tensor operators are used to perform batch parallel operations on multiple elements. Similar content in the following text belongs to the same case and will not be explained one by one.

[0199] For the 903 mentioned above, see [link / reference]. Figure 14The first index tensor leftOutIds is sorted using the tensor operator sort, resulting in the first index tensor leftOutIdx = [1, 2] (which is the third index tensor) arranged in ascending order. The position index of 1 in the third index tensor is 1 in the first index tensor, and the position index of 2 in the third index tensor is 0 in the first index tensor. This gives the sorted tensor indices = [1, 0].

[0200] For the 904 error mentioned above, see [link / reference]. Figure 15 The second index tensor `rightOutIdx` is sorted using the reordering tensor `indices`. The first element in the reordering tensor `indices` is 1. The element 3, with index 1 in the second index tensor `rightOutIdx`, is placed in the first position of the fourth index tensor. The second element in the reordering tensor `indices` is 0. The element 3, with index 0 in the second index tensor `rightOutIdx`, is placed in the second position of the fourth index tensor. This results in the fourth index tensor `[3, 2]`, which corresponds to the order of the third index tensor.

[0201] For the 905 mentioned above, see [link / reference]. Figure 16 Based on the second tensor `deg` and the fourth index tensor, the indices of the unique source vertex tensor are expanded to the indices of the target vertex tensor. Specifically, the first element of the fourth index tensor is 3, corresponding to element 2 at index 3 in the second tensor `deg`, resulting in the first two elements [6, 7] of the fifth index tensor `dstIdx`. [6, 7] is obtained by summing the elements [3, 2, 1] at indices 0 to 2 in the second tensor `deg` and adding 1. The second element of the fourth index tensor is 2, corresponding to element 1 at index 2 in the second tensor `deg`, resulting in the third element 5 of the fifth index tensor `dstIdx`. 5 is obtained by summing the elements [3, 2] at indices 0 to 1 in the second tensor `deg`. Therefore, the fifth index tensor `dstIdx` = [6, 7, 5].

[0202] For the 906 mentioned above, see [link / reference]. Figure 17Based on the element values ​​in the third index tensor, the element values ​​in the first output tensor `neighborCounts` are modified using the out-degree values ​​obtained from the second tensor `deg = [3, 2, 1, 2]` via the fourth index tensor. The first element in the third index tensor is 1, and the first element in the fourth index tensor is 3, so the element value at index 1 in the first output tensor `neighborCounts` is changed to the element 2 at index 3 obtained from the second tensor; the second element in the third index tensor is 2, and the second element in the fourth index tensor is 2, so the element value at index 2 in the first output tensor `neighborCounts` is changed to the element 1 at index 2 obtained from the second tensor. This results in the first output tensor `neighborCounts = [0, 2, 1]`.

[0203] For the 907 mentioned above, see [link / reference]. Figure 18 The neighboring vertices are obtained from the third tensor dst based on the fifth index tensor dstIdx. The first element of the fifth index tensor dstIdx is 6, so the element 0 with index 6 is obtained from the third tensor dst and filled into the first position of the second output tensor neighbors. The second element of the fifth index tensor dstIdx is 7, so the element 2 with index 7 is obtained from the third tensor dst and filled into the second position of the second output tensor neighbors. The third element of the fifth index tensor dstIdx is 5, so the element 0 with index 5 is obtained from the third tensor dst and filled into the third position of the second output tensor neighbors. The second output tensor neighbors = [0, 2, 0].

[0204] Corresponding to 908 above, the first output tensor neighborCounts = [0, 2, 1] and the second output tensor neighbors = [0, 2, 0].

[0205] Execution function two

[0206] The corresponding operation instruction two is used to query in batches whether the input edges (which can be represented by the vertices at both ends) exist.

[0207] This execution function can perform query processing using the edge connection information stored in the first tensor data structure. The input data of this execution function can include an input source vertex tensor and an input target vertex tensor, where the corresponding source and target vertices represent an edge. The corresponding function execution process can be as follows: Figure 19 As shown, it includes the following steps:

[0208] 1901, Get the length L of the input source vertex tensor.

[0209] 1902. For each source vertex in the first tensor, copy the source vertex according to the corresponding out-degree in the second tensor, and arrange all the copied source vertices in the order in the first tensor to obtain a new first tensor.

[0210] 1903. The input source vertex tensor and the new first tensor are concatenated together to obtain the concatenated source vertex tensor.

[0211] 1904. Concatenate the input target vertex tensor and the third tensor together to obtain the concatenated target vertex tensor.

[0212] In 1905, the Get Agent Tensor (an operator name) was used (this operator can be considered an execution function, and its execution can also be performed using...). Figure 8 The processing flow shown will be explained in the following content (see Execution Function 5 below). It calculates the weighted sum of the elements with the same index in the source vertex tensor and the target vertex tensor to obtain the proxy tensor.

[0213] 1906. Obtain the first L elements from the surrogate tensor to get the input surrogate tensor.

[0214] 1907. Obtain all elements except the first L elements from the proxy tensor to obtain the edge proxy tensor.

[0215] 1908. Check if any element in the proxy tensor exists in the edge proxy tensor to obtain the result tensor. If any element in the result tensor takes the first value (e.g., 1), it means that the element at the corresponding position in the input proxy tensor exists in the edge proxy tensor. If any element in the result tensor takes the second value (e.g., 0), it means that the element at the corresponding position in the input proxy tensor does not exist in the edge proxy tensor.

[0216] The following is about the above. Figure 19 The process is given as an example, assuming the target edge type in the graph database corresponds to the graph data as follows: Figure 10 As shown, the first tensor data structure (CUS data structure) that records the graph data is as follows: Figure 11 As shown, the first tensor src = [0, 2, 3, 5], the second tensor deg = [3, 2, 1, 2], and the third tensor dst = [2, 3, 5, 0, 5, 0, 0, 2].

[0217] The input source vertex tensor is inputSrcIds = [4, 5, 3], and the input target vertex tensor is inputDstIds = [1, 2, 0].

[0218] For the reference to 1901 above, see [link / reference]. Figure 20 Get the length of the input source vertex tensor inputSrcIds, denoted as inputLength = 3.

[0219] For the reference to 1902 above, see [link / reference]. Figure 21 The first element of the first tensor `src` is 0, which corresponds to the first element of the second tensor `deg` being 3. Therefore, the number of times the element 0 is copied is 3, and they are arranged in order. The second element of the first tensor `src` is 2, which corresponds to the second element of the second tensor `deg` being 2. Therefore, the number of times the element 2 is copied is 2, and they are arranged in order. The third element of the first tensor `src` is 3, which corresponds to the third element of the second tensor `deg` being 1. Therefore, the number of times the element 3 is copied is 1, and they are arranged in order. The fourth element of the first tensor `src` is 5, which corresponds to the fourth element of the second tensor `deg` being 2. Therefore, the number of times the element 5 is copied is 2, and they are arranged in order. This results in a new first tensor `newSrc` = [0, 0, 0, 2, 2, 3, 5, 5].

[0220] For the reference to 1903 above, see [link / reference]. Figure 22 The input source vertex tensor inputSrcIds and the new first tensor newSrc are concatenated together in order to obtain the concatenated source vertex tensor catSrc = [4, 5, 3, 0, 0, 0, 2, 2, 3, 5, 5].

[0221] For the 1904 mentioned above, see [link / reference]. Figure 23 The input target vertex tensor inputDstIds and the third tensor dst are concatenated in sequence to obtain the concatenated target vertex tensor catDst = [1, 2, 0, 2, 3, 5, 0, 5, 0, 0, 2].

[0222] For the 1905 mentioned above, see [link / reference]. Figure 24 Using the Get Agent Tensor operator, the elements with the same index in catSrc and catDst are summed with weights (the origin of the corresponding weights will be explained in detail below) to obtain the agent tensor agent = [25, 32, 18, 2, 3, 5, 12, 17, 18, 30, 32].

[0223] For the reference to 1906 above, see [link / reference]. Figure 25 Given the input length inputLength = 3, extract the first 3 elements of the agent tensor agent to obtain the input agent tensor inputAgent = [25, 32, 18].

[0224] For the 1907 mentioned above, see [link / reference]. Figure 25 Given the input length inputLength = 3, extract all elements except the first 3 elements from the agent tensor agent to obtain the edge agent tensor edgeAgent = [2, 3, 5, 12, 17, 18, 30, 32].

[0225] For the 1908 mentioned above, see [link / reference]. Figure 26 Check if the element in the proxy tensor inputAgent exists in the edge proxy tensor edgeAgent, and obtain the result tensor resultMap = [0, 1, 1], where 1 indicates that the element at the corresponding position in the input proxy tensor inputAgent exists in the edge proxy tensor edgeAgent, and 0 indicates that the element at the corresponding position in the input proxy tensor inputAgent does not exist in the edge proxy tensor edgeAgent.

[0226] resultMap = [0, 1, 1] indicates that the edge between vertices 4 and 1 in inputSrcIds = [4, 5, 3] and inputDstIds = [1, 2, 0] does not exist, the edge between vertices 5 and 2 exists, and the edge between vertices 1 and 3 exists.

[0227] Execution function three

[0228] The corresponding operation instruction is number three, which is used for subgraph matching.

[0229] This execution function can perform query processing using edge connection information stored in the first tensor data structure. The input to this execution function can include two two-dimensional tensors: a first input tensor and a second input tensor. The first input tensor initially contains only the first column of elements, which records at least one vertex. In subsequent processing, the second column records the neighboring vertices of each vertex, the third column records the neighboring vertices of those neighboring vertices, and so on. The corresponding second input tensor records the out-degree of each vertex in the first input tensor; its initial state can be empty. The corresponding function execution process can be as follows: Figure 27 As shown, it includes the following steps:

[0230] 2701. Based on the input column index, read the elements of the first column from the first input tensor to form the input vertex tensor.

[0231] 2702. In the first tensor data structure, take all elements of the first tensor to get the source vertex tensor, take all elements of the second tensor to get the out-degree tensor, and take all elements of the third tensor to get the target vertex tensor.

[0232] 2703, based on the input vertex tensor, source vertex tensor, out-degree tensor, and target vertex tensor, the traverse_neighbors operator (which can be considered an execution function, and its call execution can also be performed using...) is used to... Figure 8The processing flow shown above (see Execution Function 1 above) yields a neighbor count tensor (i.e., the first output tensor of Execution Function 1) and a neighbor tensor (i.e., the second output tensor of Execution Function 1).

[0233] 2704. Add the neighbor tensor as the second column to the first input tensor to obtain the modified first input tensor.

[0234] 2705. Add the neighbor count tensor as the first column to the second input tensor to obtain the modified second input tensor.

[0235] The modified first input tensor and the modified second output tensor constitute the query result. This query result can be used directly to determine the final output, or it can be used as an intermediate result as input to execution function three for the next round of queries. Execution function three can repeatedly execute multiple rounds of queries to obtain multi-level neighbors.

[0236] The following is about the above. Figure 27 The process is given as an example, assuming the target edge type in the graph database corresponds to the graph data as follows: Figure 10 As shown, the first tensor in the first tensor data structure (CUS data structure) that records the graph data is CUSEdges.src = [0, 2, 3, 5], the second tensor is CUSEdges.deg = [3, 2, 1, 2], and the third tensor is CUSEdges.dst = [2, 3, 5, 0, 5, 0, 0, 2].

[0237] The input data for this execution function may include a first input tensor intermediateResult.columns = [[4, 5, 3]] (which is a two-dimensional tensor), a second input tensor intermediateResult.rptList = [] (which is a two-dimensional tensor), and the input column index colIdx = 0.

[0238] Corresponding to 2701 above, based on the input column index colIdx, the input vertex tensor inputIds = [4, 5, 3] with index 0 is retrieved from the first input tensor intermediateResult.columns.

[0239] Corresponding to 2702 above, take all elements of the first tensor CUSEdges.src to obtain the source vertex tensor src = [0, 2, 3, 5], take all elements of the second tensor CUSEdges.deg to obtain the out-degree tensor deg = [3, 2, 1, 2], and take all elements of the third tensor CUSEdges.dst to obtain the target vertex tensor dst = [2, 3, 5, 0, 5, 0, 0, 2].

[0240] Corresponding to 2703 above, based on the input vertex tensor inputIds, the source vertex tensor src, the out-degree tensor deg, and the target vertex tensor dst, the traverse_neighbors operator (which can be considered an execution function, and its call execution can also be performed using...) is used to... Figure 8 The processing flow shown above (see Execution Function 1 above) yields neighborCounts = [0, 2, 1] and neighbors = [0, 2, 0].

[0241] Corresponding to 2704 above, add neighbors to the first input tensor intermediateResult.columns to obtain the modified first input tensor intermediateResult.columns = [[4, 5, 3], [0, 2, 0]].

[0242] Corresponding to 2705 above, add neighborCounts to the second input tensor intermediateResult.rptList to obtain the modified second input tensor intermediateResult.rptList = [[0, 2, 1]].

[0243] Execution function four

[0244] The corresponding operation instruction four is used to execute the fifth step of the above-mentioned execution function one (i.e., step 905).

[0245] The tensors input to this execution function can be called the initial index tensor and the repeating tensor. The repeating tensor can be the second tensor in the first tensor data structure, and the initial index tensor can be the fourth index tensor mentioned during the execution of the first execution function. The corresponding function execution process can be as follows: Figure 28 As shown, it includes the following steps:

[0246] 2801. The tensor operator cumsum (a type of tensor operator) is used to accumulate repeated tensors, resulting in an accumulated repeated tensor. The value of the nth element of the accumulated repeated tensor is equal to the sum of the first n elements of the repeated tensor.

[0247] 2802, using the initial index tensor, extract the corresponding output duplicate tensor from the duplicate tensor.

[0248] 2803, use the initial index tensor to extract the corresponding element from the accumulated repeating tensor to obtain the output accumulated repeating tensor.

[0249] 2804, subtract the output cumulative repeating tensor from the output repeating tensor to obtain the starting index.

[0250] 2805. The tensor operator cumsum is used to accumulate the output repeating tensor to obtain the accumulated output repeating tensor.

[0251] 2806, subtract the accumulated output repeat tensor and the output repeat tensor by their positions to obtain the new starting index tensor.

[0252] 2807. For each element in the starting index tensor, copy it using the element at the corresponding position in the output repeating tensor as the number of copies, and arrange all the copied elements in the order they are in the starting index tensor to obtain the modified starting index tensor.

[0253] 2808. For each element in the new starting index tensor, copy it using the element at the corresponding position in the output repeating tensor as the number of copies, and arrange all the copied elements in the order in the new starting index tensor to obtain the modified new starting index tensor.

[0254] 2809. Based on the length L of the modified starting index tensor, generate a continuous integer sequence of length L starting from 0 to form the adjacent group index offset tensor.

[0255] 2810. Subtract the adjacent group index offset tensor from the modified new starting index tensor to obtain the index offset tensor.

[0256] 2811. The modified starting index tensor and index offset tensor are added bit by bit to obtain the target vertex index tensor.

[0257] 2812, returns the target vertex index tensor.

[0258] For function one, the fifth index tensor during its execution is the target vertex index tensor.

[0259] The following is about the above. Figure 28 The process is illustrated with an example, assuming the initial index tensor indices = [3, 2] and the repeating tensor rpt = [3, 2, 1, 2]. Here, the initial index tensor indices corresponds to the fourth index tensor in Algorithm 1, and the repeating tensor rpt corresponds to the second tensor deg in Algorithm 1.

[0260] Corresponding to 2801 above, the tensor operator cumsum (a type of tensor operator) is used to accumulate the repeated tensor rpt, resulting in the accumulated repeated tensor cumRpt = [3, 5, 6, 8]. Here, the first element of the repeated tensor rpt corresponds to the first element of the accumulated repeated tensor cumRpt; the sum of the first and second elements of the repeated tensor rpt corresponds to the second element of the accumulated repeated tensor cumRpt; the sum of the first three elements of the repeated tensor rpt corresponds to the third element of the accumulated repeated tensor cumRpt; and the sum of the first four elements of the repeated tensor rpt corresponds to the fourth element of the accumulated repeated tensor cumRpt.

[0261] Corresponding to 2802 above, the initial index tensor indices is used to extract the corresponding output repeated tensor outRpt = [2, 1] from the repeated tensor rpt.

[0262] Corresponding to 2803 above, the initial index tensor indices is used to extract the corresponding element from the cumulative repeating tensor cumRpt to obtain the output cumulative repeating tensor OutcumRpt = [8, 6].

[0263] Corresponding to 2804 above, by subtracting the output repeating tensor outRpt from the output accumulating repeating tensor OutcumRpt, we obtain the starting index startldx = [6, 5].

[0264] Corresponding to 2805 above, the tensor operator cumsum is used to accumulate the output repeating tensor outRpt, resulting in the accumulated output repeating tensor cumOutRpt = [2, 3]. Here, the first element of the output repeating tensor outRpt corresponds to the first element of the accumulated output repeating tensor cumOutRpt; the sum of the first and second elements of the output repeating tensor outRpt corresponds to the second element of the accumulated output repeating tensor cumOutRpt.

[0265] Corresponding to 2806 above, the new starting index tensor startldxNew = [0, 2] is obtained by subtracting the output repeat tensor outRpt from the accumulated output repeat tensor cumOutRpt.

[0266] Corresponding to 2807 above, the starting index tensor startldx is repeatedly expanded using the output repeating tensor outRpt. The first element in the output repeating tensor outRpt has a value of 2, so the first element in the starting index tensor startldx is repeated twice, resulting in [6, 6]. The second element in the output repeating tensor outRpt has a value of 1, so the second element in the starting index tensor startldx is repeated once, resulting in [5]. The results are concatenated to obtain the modified starting index tensor startldx = [6, 6, 5].

[0267] Corresponding to 2808 above, the new starting index tensor startldxNew is repeatedly expanded using the output repeating tensor outRpt. The first element of the output repeating tensor outRpt is 2. The first element of the new starting index tensor startldxNew is repeated twice to obtain [0, 0]. The second element of the output repeating tensor outRpt is 1. The second element of the new starting index tensor startldxNew is repeated once to obtain [2]. The results are concatenated to obtain the modified new starting index tensor startldxNew = [0, 0, 2].

[0268] Corresponding to 2809 above, based on the length of the starting index tensor startldx of 3, the tensor operator arange (a name of a tensor operator) is used to generate a tensor starting from 0, increasing by 1 at each position, up to 3 (excluding 3), to obtain the adjacent group index offset tensor rng = [0, 1, 2].

[0269] Corresponding to 2810 above, the index offset tensor is obtained by subtracting the new starting index tensor startldxNew from the adjacent group index offset tensor rng.

[0270] Corresponding to 2811 above, by adding the index offset tensor = [0, 1, 0] to the starting index tensor startldx, we obtain the target vertex index tensor dstIndices = [6, 7, 5].

[0271] Corresponding to 2812 above, the target vertex index tensor dstIndices = [6, 7, 5] is returned.

[0272] Execution function five

[0273] The corresponding operation instruction five is used to perform weighted summation processing. The fifth step of the above-mentioned execution function two (i.e., step 1905) uses this execution function five.

[0274] The input to this execution function can include a two-dimensional tensor and a one-dimensional list. The two-dimensional tensor is called the input tensor, and the one-dimensional list is called the sequence list. The input tensor contains at least one one-dimensional tensor to be processed, used to record at least one vertex. The sequence list indicates whether each one-dimensional tensor to be processed is processed sequentially or in reverse order. The sequence list is a list of Boolean values ​​(which can be True or False) with the same number of columns as the input tensor. The corresponding function execution process can be as follows: Figure 29 As shown, it includes the following steps:

[0275] 2901. Initialize the surrogate tensor, where the length of the surrogate tensor is the same as the length of the one-dimensional tensor contained in the input tensor, and all values ​​in the surrogate tensor are 0.

[0276] 2902, Create the first index, set the value of the first index to the length of the input tensor minus 1, and perform the following loop:

[0277] 29021. Based on the value of the first index, retrieve the corresponding column tensor from the input tensor and denote it as the target tensor.

[0278] 29022. Based on the value of the first index, take the Boolean value at the corresponding position in the sequential list. If the Boolean value is true, subtract the smallest element in the target tensor from each element in the target tensor to obtain the modified target tensor. If the Boolean value is false, subtract the largest element from each element in the target tensor to obtain the modified target tensor.

[0279] 29023. Check if adding 1 to the largest element in the surrogate tensor exceeds the upper limit of the numerical value. If it does, map all elements in the surrogate tensor in ascending order to a sequence of consecutive integers (or integer sequences) starting from 0, obtaining an index sequence to construct a new surrogate tensor. This upper limit of the numerical value can be the upper limit of integer data, such as 2 to the power of 31. Each element in the index sequence represents the position index of the element in the integer sequence mapped to the corresponding element in the surrogate tensor.

[0280] 29024. Check if adding 1 to the largest element in the target tensor exceeds the above numerical upper limit. If it does, map all elements in the target tensor to a sequence of consecutive integers (which can be called an integer sequence) starting from 0, in ascending order, to obtain an index sequence, and construct a new target tensor. Each element in the index sequence represents the position index of the element in the integer sequence mapped to the corresponding element in the target tensor.

[0281] 29025, take the largest element in the current proxy tensor, and add 1 to the element and record the value as the weight.

[0282] 29026. Multiply each element in the current target tensor by its weight, and then add it to the element at the corresponding position in the current proxy tensor to obtain a new proxy tensor.

[0283] 29027. After each loop, the value of the first index is decremented by 1, and a judgment is made. If the value of the first index is greater than or equal to 0, the loop is skipped to step 29021 to continue the loop. If the value of the first index is less than 0, the loop ends and the subsequent step 2903 is performed.

[0284] 2903, Output the current proxy tensor.

[0285] The following is about the above. Figure 29 The process is illustrated with an example. Assume the input is a two-dimensional tensor `tensorList = [[4, 5, 3, 0, 0, 0, 2, 2, 3, 5, 5], [1, 2, 0, 2, 3, 5, 0, 5, 0, 0, 2]]` and an `ascendingList = [True, True]`, where `ascendingList` is a list of Boolean values ​​(True or False) with the same number of columns as `tensorList`. This list indicates whether each tensor should be processed in ascending (smallest to largest) or descending (largest to smallest) order. The two tensors in `tensorList` correspond to the source vertex tensor `catSrc` and the target vertex tensor `catDst` in Algorithm 2, respectively.

[0286] Corresponding to 2901 above, initialize a proxy tensor agent = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] with the same length as each tensor in the two-dimensional tensor tensorList.

[0287] Corresponding to 2902 above, for the first index idx from 1 to 0 (where 1 is obtained by subtracting 1 from the length 2 of the two-dimensional tensorList), perform the following loop twice:

[0288] Corresponding to 29021 above, take the element with the first index idx = 1 in the two-dimensional tensor tensorList, that is, concatenate the target vertex tensor catDst = [1, 2, 0, 2, 3, 5, 0, 5, 0, 0, 2], and denote it as the target tensor curTensor.

[0289] Corresponding to 29022 above, if the first index idx position in ascendingList is True, then for each element in the target tensor curTensor, subtract the smallest element of the target tensor curTensor. In this loop, the smallest element of the target tensor curTensor is 0, resulting in a new curTensor = [1, 2, 0, 2, 3, 5, 0, 5, 0, 0, 2]. If the first index idx position in ascendingList is False, then for each element in the target tensor curTensor, subtract the largest element of the target tensor curTensor (which is 5 in this loop) to obtain a new element value.

[0290] Corresponding to 29023 above, check if adding 1 to the largest element in the surrogate tensor exceeds the numerical upper limit. If it does, map all elements in the surrogate tensor to a sequence of consecutive integers (which can be called an integer sequence) starting from 0, in ascending order, to obtain an index sequence, and construct a new surrogate tensor. This numerical upper limit can be the upper limit of integer data, such as 2 to the power of 31. Each element in the index sequence represents the position index of the element in the integer sequence mapped to the corresponding element in the surrogate tensor.

[0291] Corresponding to 29024 above, check if adding 1 to the largest element in the target tensor exceeds the aforementioned numerical upper limit. If it does, map all elements in the target tensor to a sequence of consecutive integers (which can be called an integer sequence) starting from 0, in ascending order, to obtain an index sequence, and construct a new target tensor. Each element in the index sequence represents the position index of the element in the integer sequence mapped to the corresponding element in the target tensor.

[0292] Corresponding to 29025 above, take the maximum value in the agent tensor and add 1, which is denoted as weight. In the first loop, the maximum value of the agent tensor is 0, so add 1 to get weight = 1.

[0293] Corresponding to 29026 above, each element in the target tensor curTensor is multiplied by the weight, and then added to the element at the corresponding position in the agent to obtain the new agent tensor agent = [1, 2, 0, 2, 3, 5, 0, 5, 0, 0, 2].

[0294] Corresponding to step 29027 above, the loop ends (the above example only shows the result of one loop; if the loop end condition is not met, the index idx is decremented by one, and the loop jumps back to step 29021 above to continue).

[0295] Corresponding to 2903 above, the agent tensor is returned.

[0296] Execution function six

[0297] The corresponding operation instruction six is ​​used to expand the compressed neighbor relationship information into the complete neighbor relationship information (for each edge, the corresponding two vertices are completely recorded without any compression).

[0298] The input to this execution function is generally the intermediate query results obtained from other query processing, which may include the first input tensor and repeated tensors (see the execution process of execution function three). The corresponding function execution process can be as follows: Figure 30 As shown, it includes the following steps:

[0299] 3001, Create an empty first output tensor.

[0300] 3002, Create the first index, initialize the value of the first index to 0, and repeat the following process until the value of the first index becomes the length of the first input tensor minus 1. After each execution of step 30024, increment the first index by 1.

[0301] 30021, in the first input tensor, read the element of the column corresponding to the first index to obtain the current column tensor.

[0302] 30022, Determine the level corresponding to the current column element. This level can be related to the order in which the column elements in the first input tensor were generated. For example, the level represents how many neighbor visits the initial column tensor was used to obtain this column tensor. In the first input tensor, the first column (input vertex) corresponds to a level of 0, the second column (first-level neighbor vertex) corresponds to a level of 1, the second column (second-level neighbor vertex) corresponds to a level of 2, and so on.

[0303] 30023, Create a second index. Initialize the value of the second index to the level corresponding to the current column element. Repeat the following process until the value of the second index becomes the length of the repeating tensor minus 1. After each execution of step 300231, increment the second index by 1.

[0304] 300231. For each source vertex in the current column tensor, copy the source vertex according to the corresponding element in the reference tensor, and arrange all the copied source vertices in the order in the current column tensor to obtain a new current column tensor.

[0305] The reference tensor is the column tensor corresponding to the second index in the repeated tensor.

[0306] 30024, add the current column tensor as a new column to the first output tensor.

[0307] 3003, output the first output tensor.

[0308] The following is about the above. Figure 30 The process is given as an example. Assume the input is the intermediate query result intermediateResult, which contains the first input tensor intermediateResult.columns = [[4, 5, 3], [0, 2, 0]] and the repeating tensor intermediateResult.rptList = [[0, 2, 1]].

[0309] Corresponding to 3001 above, create an empty first output tensor unfoldedResult.columns = [].

[0310] Corresponding to 3002 above, create the first index idx1, initialize the value of the first index to 0, and repeat the following process until the value of the first index becomes the length of the first input tensor intermediateResult.columns minus 1.

[0311] Corresponding to 30021 above, in the first input tensor intermediateResult.columns, the element of the column corresponding to the first index idx1 is read to obtain the current column tensor. In the first loop, idx1 = 0, curCol = [4, 5, 3].

[0312] Corresponding to 30022 above, determine the level corresponding to the current column element. The level represents how many neighbor access operations the column tensor was obtained from the first column. The level of the first column is 0.

[0313] Corresponding to 30023 above, create a second index idx2, initialize the value of the second index idx2 to the current value of level, and repeat the following process until the value of the second index becomes the length of the repeating tensor minus 1.

[0314] Corresponding to 300231 above, the current column tensor curCol is repeatedly expanded using the column tensor corresponding to the second index idx2=0 in the repeated tensor intermediateResult.rptList. The repeating process is similar to the process corresponding to 1902 in the execution function example. The new current column tensor curCol=[5, 5, 3] is obtained.

[0315] The second loop ends (the above example only shows the result of one loop. If the loop termination condition is not met, the second index idx2 is incremented by 1, and the loop jumps back to the step corresponding to 300231 above to continue the loop).

[0316] Corresponding to 30024 above, the current column tensor curCol = [5, 5, 3] is added to the first output tensor unfoldedResult.columns, resulting in unfoldedResult.columns = [[5, 5, 3]].

[0317] The first loop ends (the above example only shows the result of one loop. If the loop termination condition is not met, the first index idx is incremented by 1, and the loop jumps back to the corresponding step 30021 to continue the loop).

[0318] Corresponding to 3003 above, the first output tensor is unfoldedResult.columns = [[5, 5, 3], [0, 2, 0]].

[0319] Based on the same technical concept, this disclosure provides a data manipulation apparatus applied to a graph database. The graph database stores graph data including vertices and edges connecting the vertices. The edges connecting the vertices include multiple edge types, and edge connection information corresponding to each edge type is stored in a tensor data structure. The edge connection information records the connection relationship between the source vertex and the target vertex connected by each edge of the corresponding edge type. The tensor data structure includes at least two tensors. One of the tensors records the degree of a vertex at one end of each edge of the corresponding edge type, and the other tensor records the vertex at the other end of each edge of the corresponding edge type. The degree of a vertex indicates the number of edges connected to that vertex. Figure 31 As shown, the device includes:

[0320] The acquisition module 3110 is used to acquire operation instructions for the graph database. Specifically, it can implement the processing function of step 801 above, as well as other implicit steps.

[0321] The determining module 3120 is used to determine the target execution function based on the operation instruction, wherein the target execution function includes at least one tensor operator. Specifically, it can implement the processing function of step 802 above, as well as other implicit steps.

[0322] The execution module 3130 is used to run the target execution function to process the edge connection information stored in the graph database through the tensor data structure using the tensor operator, and obtain the operation result of the target operation instruction. Specifically, it can implement the processing function of step 803 above, as well as other implicit steps.

[0323] In one possible implementation, the tensor data structure storing the edge connection information includes a first tensor data structure, which includes a first tensor, a second tensor, and a third tensor.

[0324] The first tensor is used to record the source vertex;

[0325] The second tensor is used to record the out-degree of each source vertex under the edge type in the order of the source vertices in the first tensor, wherein the out-degree is used to indicate the number of edges under the edge type that originate from the corresponding source vertex;

[0326] The third tensor is used to record the target vertices connected to each edge of the edge type in the order of the corresponding source vertices in the first tensor.

[0327] In one possible implementation, the tensor data structure storing the edge connection information includes a second tensor data structure, which includes a fourth tensor, a fifth tensor, and a sixth tensor.

[0328] The fourth tensor is used to record the target vertex;

[0329] The fifth tensor is used to record the in-degree of each target vertex under the edge type according to the order of the target vertices in the fourth tensor, wherein the in-degree is used to indicate the number of edges under the edge type that terminate at the corresponding source vertex.

[0330] The sixth tensor is used to record the source vertices connected to each edge of the edge type in the order of the corresponding target vertices in the fourth tensor.

[0331] In one possible implementation, the tensor data structure storing the edge connection information includes a third tensor data structure, which includes a seventh tensor and an eighth tensor.

[0332] The seventh tensor is used to record the out-degree of each source vertex under the edge type, taking all vertices in the graph database as source vertices, according to the first specified order of each source vertex in the graph database, wherein the out-degree is used to indicate the number of edges under the edge type that start from the corresponding source vertex.

[0333] The eighth tensor is used to record the target vertices connected to each edge of the edge type in the first specified order of the corresponding source vertices.

[0334] In one possible implementation, the tensor data structure storing the edge connection information includes a fourth tensor data structure, which includes a ninth tensor and a tenth tensor.

[0335] The ninth tensor is used to record the in-degree of each target vertex under the edge type, with all vertices in the graph database as target vertices, according to the second specified order of each target vertex in the graph database, wherein the in-degree is used to indicate the number of edges under the edge type that end at the corresponding source vertex;

[0336] The tenth tensor is used to record the source vertices connected to each edge of the edge type in the second specified order of the corresponding target vertices.

[0337] In one possible implementation, in the graph database, for edge connection information corresponding to each edge type, if the edge connection information meets a specified condition, the edge connection information is stored through a first specified tensor data structure; if the edge connection information does not meet the specified condition, the edge connection information is stored through a second specified tensor data structure.

[0338] In one possible implementation, the specified condition is: in the graph database, the proportion of vertices with an out-degree greater than 0 under the edge type is less than half;

[0339] The first specified tensor data structure includes a first tensor, a second tensor, and a third tensor; the first tensor is used to record all source vertices with an out-degree greater than 0 under the edge type; the second tensor is used to record the out-degree of each source vertex in the first tensor under the edge type according to the order of the source vertices in the first tensor, wherein the out-degree is used to indicate the number of edges under the edge type originating from the corresponding source vertex; the third tensor is used to record the target vertices connected to each edge of the edge type according to the order of the corresponding source vertices in the first tensor.

[0340] The second specified tensor data structure includes a seventh tensor and an eighth tensor; the seventh tensor is used to record the out-degree of each source vertex under the edge type in the graph database according to the first specified order of each source vertex in the graph database, taking all vertices in the graph database as source vertices; the eighth tensor is used to record the target vertices connected by each edge of the edge type in the first specified order of the corresponding source vertices.

[0341] In one possible implementation, the determining module 3120 is configured to:

[0342] Determine the target edge type corresponding to the operation instruction, and obtain the initial execution function corresponding to the operation instruction;

[0343] If the edge connection information corresponding to the target edge type is stored through the first specified tensor data structure, then the initial execution function is determined as the target execution function. If the edge connection information corresponding to the target edge type is stored through the second specified tensor data structure, then the specified statement is deleted from the initial execution function, and the initial execution function after deleting the statement is determined as the target execution function.

[0344] In one possible implementation, the determining module 3120 is configured to:

[0345] Determine the target edge type corresponding to the operation instruction;

[0346] Determine the type of the target tensor data structure that stores the edge connection information corresponding to the target edge type;

[0347] Based on the correspondence between the operation instruction, the type of the target tensor data structure, and the execution function, a target execution function corresponding to both the operation instruction and the type of the target tensor data structure is determined.

[0348] In one possible implementation, the device further includes a conversion module for:

[0349] In the initial graph database, obtain the edge connection information corresponding to each edge type stored in other data structures, and convert the edge connection information corresponding to each edge type from the other data structures into the tensor data structure;

[0350] The graph database is obtained based on the transformed edge connection information corresponding to each edge type.

[0351] In this embodiment, graph data is stored using a tensor data structure, and execution functions composed of tensor operators are used to perform related operations on the graph data. Because tensor operators are designed to be optimized for various types of processors and can generally support various processors such as CPUs, GPUs, and NPUs, this method can make good use of various processing resources to improve the efficiency of graph data-related operations.

[0352] The acquisition module 3110, the determination module 3120, and the execution module 3130 can all be implemented in software or in hardware. For example, the implementation of the acquisition module 3110 will be described below. Similarly, the implementation of the determination module 3120 and the execution module 3130 can refer to the implementation of the acquisition module 3110.

[0353] As an example of a software functional unit, module 3110 may include code running on a computing instance. The computing instance may include at least one of a physical host (computing device), a virtual machine, or a container. Further, the aforementioned computing instance may be one or more. For example, module 3110 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed in the same region or in different regions. Further, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including one or more geographically proximate data centers. Typically, a region may include multiple AZs.

[0354] Similarly, multiple hosts / virtual machines / containers used to run this code can be distributed within the same Virtual Private Cloud (VPC) or across multiple VPCs. Typically, a VPC is set up within a region. Communication between two VPCs within the same region, as well as between VPCs in different regions, requires a communication gateway to be set up within each VPC to enable interconnection between VPCs.

[0355] As an example of a hardware functional unit, the acquisition module 3110 may include at least one computing device, such as a server. Alternatively, the acquisition module 3110 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be implemented using a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), generic array logic (GAL), or any combination thereof.

[0356] The multiple computing devices included in the acquisition module 3110 can be distributed in the same region or in different regions. Similarly, the multiple computing devices included in the acquisition module 3110 can be distributed in the same Availability Zone (AZ) or in different AZs. Likewise, the multiple computing devices included in the acquisition module 3110 can be distributed in the same Virtual Private Cloud (VPC) or in multiple VPCs. These multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0357] It should be noted that, in other embodiments, the acquisition module 3110, the determination module 3120, and the running module 3130 can be used for any step in the data operation method. The steps implemented by the acquisition module 3110, the determination module 3120, and the running module 3130 can be specified as needed. By implementing different steps in the data operation method through the acquisition module 3110, the determination module 3120, and the running module 3130, the device for data operation can realize all functions of the data operation.

[0358] This disclosure also provides a computing device 100. For example... Figure 32 As shown, the computing device 100 includes a bus 102, a processor 104, a memory 106, and a communication interface 108. The processor 104, the memory 106, and the communication interface 108 communicate with each other via the bus 102. The computing device 100 may be a server or a terminal device. It should be understood that this disclosure does not limit the number of processors and memories in the computing device 100.

[0359] Bus 102 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, Figure 33 The bus 102 may be represented by a single line, but this does not mean that there is only one bus or one type of bus. The bus 102 may include a path for transmitting information between various components of the computing device 100 (e.g., memory 106, processor 104, communication interface 108).

[0360] The processor 104 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0361] Memory 106 may include volatile memory, such as random access memory (RAM). Memory 106 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0362] The memory 106 stores executable program code, and the processor 104 executes the executable program code to implement the functions of the aforementioned acquisition module 3110, determination module 3120, and execution module 3130, thereby realizing the data operation method. That is, the memory 106 stores instructions for the data operation method.

[0363] Alternatively, the memory 106 stores executable code, which the processor 104 executes to implement the functions of the aforementioned acquisition module 3110, determination module 3120, and execution module 3130, thereby realizing the data operation method. That is, the memory 106 stores instructions for the data operation method.

[0364] The communication interface 108 uses transceiver modules such as, but not limited to, network interface cards and transceivers to enable communication between the computing device 100 and other devices or communication networks.

[0365] This disclosure also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smartphone.

[0366] like Figure 33 As shown, the computing device cluster includes at least one computing device 100. The memory 106 of one or more computing devices 100 in the computing device cluster may store the same instructions for methods of data manipulation.

[0367] In some possible implementations, the memory 106 of one or more computing devices 100 in the computing device cluster may also store partial instructions for methods of data manipulation. In other words, a combination of one or more computing devices 100 can jointly execute the instructions for methods of data manipulation.

[0368] It should be noted that the memory 106 in different computing devices 100 within the computing device cluster can store different instructions, each used for a portion of the functions of the data operation device. That is, the instructions stored in the memory 106 of different computing devices 100 can implement the functions of one or more of the aforementioned acquisition module 3110, determination module 3120, and execution module 3130.

[0369] In some possible implementations, one or more computing devices in a computing device cluster can be connected via a network. This network can be a wide area network (WAN) or a local area network (LAN), and can be a Transmission Control Protocol (TCP) network or a Remote Direct Memory Access (RDMA) network, etc. Figure 34 One possible implementation is shown. For example... Figure 34 As shown, the two computing devices 100A and 100B are connected via a network. Specifically, they are connected to the network through communication interfaces in each computing device. In this possible implementation, the memory 106 in computing device 100A stores instructions for executing the functions of the acquisition module 3110 and the determination module 3120. Simultaneously, the memory 106 in computing device 100B stores instructions for executing the functions of the operation module 3130.

[0370] Figure 34 The connection method between the computing device clusters shown may be that, considering the data operation method provided in this disclosure requires a large amount of data storage, the functions implemented by the running module 3130 are delegated to the computing device 100B for execution.

[0371] It should be understood that Figure 34 The functions of the computing device 100A shown can also be performed by multiple computing devices 100. Similarly, the functions of the computing device 100B can also be performed by multiple computing devices 100.

[0372] This disclosure also provides another computing device cluster. The connection relationships between the computing devices in this computing device cluster can be similarly referred to... Figure 33 and Figure 34The connection method of the computing device cluster. The difference is that the memory 106 of one or more computing devices 100 in the computing device cluster can store the same instructions for data operation methods.

[0373] In some possible implementations, the memory 106 of one or more computing devices 100 in the computing device cluster may also store partial instructions for methods of data manipulation. In other words, a combination of one or more computing devices 100 can jointly execute the instructions for methods of data manipulation.

[0374] This disclosure also provides a computer program product containing instructions. The computer program product may be a software or program product containing instructions capable of running on a computing device or stored on any usable medium. When the computer program product is run on at least one computing device, it provides a method for causing data manipulation on at least one computing device.

[0375] This disclosure also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium capable of being stored by a computing device, or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., digital video disk (DVD)), or a semiconductor medium (e.g., solid-state drive). The computer-readable storage medium includes instructions that instruct a method for data manipulation by a computing device, or instruct a computing device to perform a data manipulation.

[0376] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this disclosure, and are not intended to limit them. Although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of this disclosure.

Claims

1. A method for data manipulation, characterized in that, The method is applied to a graph database. The graph database stores graph data including vertices and edges connecting the vertices. The edges connecting the vertices include multiple edge types. Edge connection information corresponding to each edge type is stored through a tensor data structure. The edge connection information records the connection relationship between the source vertex and the target vertex connected by each edge of the corresponding edge type. The tensor data structure includes at least two tensors. One of the tensors records the degree of a vertex at one end of each edge of the corresponding edge type, and the other tensor records the vertex at the other end of each edge of the corresponding edge type. The degree of a vertex indicates the number of edges connected to that vertex. The method includes: Obtain operation instructions for the graph database; Based on the operation instructions, a target execution function is determined, wherein the target execution function includes at least one tensor operator; The target execution function is run to process the edge connection information stored in the graph database through the tensor data structure using the tensor operator, so as to obtain the operation result of the target operation instruction.

2. The method according to claim 1, characterized in that, The tensor data structure storing the edge connection information includes a first tensor data structure, which includes a first tensor, a second tensor, and a third tensor. The first tensor is used to record the source vertex; The second tensor is used to record the out-degree of each source vertex under the edge type in the order of the source vertices in the first tensor, wherein the out-degree is used to indicate the number of edges under the edge type that originate from the corresponding source vertex; The third tensor is used to record the target vertices connected to each edge of the edge type in the order of the corresponding source vertices in the first tensor.

3. The method according to claim 1 or 2, characterized in that, The tensor data structure storing the edge connection information includes a second tensor data structure, which includes a fourth tensor, a fifth tensor, and a sixth tensor. The fourth tensor is used to record the target vertex; The fifth tensor is used to record the in-degree of each target vertex under the edge type according to the order of the target vertices in the fourth tensor, wherein the in-degree is used to indicate the number of edges under the edge type that terminate at the corresponding source vertex. The sixth tensor is used to record the source vertices connected to each edge of the edge type in the order of the corresponding target vertices in the fourth tensor.

4. The method according to any one of claims 1-3, characterized in that, The tensor data structure storing the edge connection information includes a third tensor data structure, which includes a seventh tensor and an eighth tensor. The seventh tensor is used to record the out-degree of each source vertex under the edge type, taking all vertices in the graph database as source vertices, according to the first specified order of each source vertex in the graph database, wherein the out-degree is used to indicate the number of edges under the edge type that start from the corresponding source vertex. The eighth tensor is used to record the target vertices connected to each edge of the edge type in the first specified order of the corresponding source vertices.

5. The method according to any one of claims 1-4, characterized in that, The tensor data structure storing the edge connection information includes a fourth tensor data structure, which includes a ninth tensor and a tenth tensor. The ninth tensor is used to record the in-degree of each target vertex under the edge type, with all vertices in the graph database as target vertices, according to the second specified order of each target vertex in the graph database, wherein the in-degree is used to indicate the number of edges under the edge type that end at the corresponding source vertex; The tenth tensor is used to record the source vertices connected to each edge of the edge type in the second specified order of the corresponding target vertices.

6. The method according to any one of claims 1-5, characterized in that, In the graph database, for each edge type, if the edge connection information meets the specified conditions, the edge connection information is stored through a first specified tensor data structure; if the edge connection information does not meet the specified conditions, the edge connection information is stored through a second specified tensor data structure.

7. The method according to claim 6, characterized in that, The specified condition is: in the graph database, the proportion of vertices with an out-degree greater than 0 under the edge type is less than half; The first specified tensor data structure includes a first tensor, a second tensor, and a third tensor; the first tensor is used to record all source vertices with an out-degree greater than 0 under the edge type; the second tensor is used to record the out-degree of each source vertex in the first tensor under the edge type according to the order of the source vertices in the first tensor, wherein the out-degree is used to indicate the number of edges under the edge type originating from the corresponding source vertex; the third tensor is used to record the target vertices connected to each edge of the edge type according to the order of the corresponding source vertices in the first tensor. The second specified tensor data structure includes a seventh tensor and an eighth tensor; the seventh tensor is used to record the out-degree of each source vertex under the edge type in the graph database according to the first specified order of each source vertex in the graph database, taking all vertices in the graph database as source vertices; the eighth tensor is used to record the target vertices connected by each edge of the edge type in the first specified order of the corresponding source vertices.

8. The method according to claim 6 or 7, characterized in that, The step of determining the target execution function based on the operation instruction includes: Determine the target edge type corresponding to the operation instruction, and obtain the initial execution function corresponding to the operation instruction; If the edge connection information corresponding to the target edge type is stored through the first specified tensor data structure, then the initial execution function is determined as the target execution function. If the edge connection information corresponding to the target edge type is stored through the second specified tensor data structure, then the specified statement is deleted from the initial execution function, and the initial execution function after deleting the statement is determined as the target execution function.

9. The method according to claim 6 or 7, characterized in that, The step of determining the target execution function based on the operation instruction includes: Determine the target edge type corresponding to the operation instruction; Determine the type of the target tensor data structure that stores the edge connection information corresponding to the target edge type; Based on the correspondence between the operation instruction, the type of the target tensor data structure, and the execution function, a target execution function corresponding to both the operation instruction and the type of the target tensor data structure is determined.

10. The method according to any one of claims 1-9, characterized in that, Before obtaining the operation instructions for the graph database, the method further includes: In the initial graph database, obtain the edge connection information corresponding to each edge type stored in other data structures, and convert the edge connection information corresponding to each edge type from the other data structures into the tensor data structure; The graph database is obtained based on the transformed edge connection information corresponding to each edge type.

11. A data manipulation apparatus, characterized in that, The device is applied to a graph database. The graph database stores graph data including vertices and edges connecting the vertices. The edges connecting the vertices include multiple edge types. Edge connection information corresponding to each edge type is stored through a tensor data structure. The edge connection information records the connection relationship between the source vertex and the target vertex connected by each edge of the corresponding edge type. The tensor data structure includes at least two tensors. One of the tensors records the degree of a vertex at one end of each edge of the corresponding edge type, and the other tensor records the vertex at the other end of each edge of the corresponding edge type. The degree of a vertex indicates the number of edges connected to that vertex. The device includes: The acquisition module is used to acquire operation instructions for the graph database; A determination module is configured to determine a target execution function based on the operation instructions, wherein the target execution function includes at least one tensor operator; The execution module is used to run the target execution function to process the edge connection information stored in the graph database through the tensor data structure using the tensor operator, so as to obtain the operation result of the target operation instruction.

12. The apparatus according to claim 11, characterized in that, The tensor data structure storing the edge connection information includes a first tensor data structure, which includes a first tensor, a second tensor, and a third tensor. The first tensor is used to record the source vertex; The second tensor is used to record the out-degree of each source vertex under the edge type in the order of the source vertices in the first tensor, wherein the out-degree is used to indicate the number of edges under the edge type that originate from the corresponding source vertex; The third tensor is used to record the target vertices connected to each edge of the edge type in the order of the corresponding source vertices in the first tensor.

13. The apparatus according to claim 11 or 12, characterized in that, The tensor data structure storing the edge connection information includes a second tensor data structure, which includes a fourth tensor, a fifth tensor, and a sixth tensor. The fourth tensor is used to record the target vertex; The fifth tensor is used to record the in-degree of each target vertex under the edge type according to the order of the target vertices in the fourth tensor, wherein the in-degree is used to indicate the number of edges under the edge type that terminate at the corresponding source vertex. The sixth tensor is used to record the source vertices connected to each edge of the edge type in the order of the corresponding target vertices in the fourth tensor.

14. The apparatus according to any one of claims 11-13, characterized in that, The tensor data structure storing the edge connection information includes a third tensor data structure, which includes a seventh tensor and an eighth tensor. The seventh tensor is used to record the out-degree of each source vertex under the edge type, taking all vertices in the graph database as source vertices, according to the first specified order of each source vertex in the graph database, wherein the out-degree is used to indicate the number of edges under the edge type that start from the corresponding source vertex. The eighth tensor is used to record the target vertices connected to each edge of the edge type in the first specified order of the corresponding source vertices.

15. The apparatus according to any one of claims 11-14, characterized in that, The tensor data structure storing the edge connection information includes a fourth tensor data structure, which includes a ninth tensor and a tenth tensor. The ninth tensor is used to record the in-degree of each target vertex under the edge type, with all vertices in the graph database as target vertices, according to the second specified order of each target vertex in the graph database, wherein the in-degree is used to indicate the number of edges under the edge type that end at the corresponding source vertex; The tenth tensor is used to record the source vertices connected to each edge of the edge type in the second specified order of the corresponding target vertices.

16. The apparatus according to any one of claims 11-15, characterized in that, In the graph database, for each edge type, if the edge connection information meets the specified conditions, the edge connection information is stored through a first specified tensor data structure; if the edge connection information does not meet the specified conditions, the edge connection information is stored through a second specified tensor data structure.

17. The apparatus according to claim 16, characterized in that, The specified condition is: in the graph database, the proportion of vertices with an out-degree greater than 0 under the edge type is less than half; The first specified tensor data structure includes a first tensor, a second tensor, and a third tensor; the first tensor is used to record all source vertices with an out-degree greater than 0 under the edge type; the second tensor is used to record the out-degree of each source vertex in the first tensor under the edge type according to the order of the source vertices in the first tensor, wherein the out-degree is used to indicate the number of edges under the edge type originating from the corresponding source vertex; the third tensor is used to record the target vertices connected to each edge of the edge type according to the order of the corresponding source vertices in the first tensor. The second specified tensor data structure includes a seventh tensor and an eighth tensor; the seventh tensor is used to record the out-degree of each source vertex under the edge type in the graph database according to the first specified order of each source vertex in the graph database, taking all vertices in the graph database as source vertices; the eighth tensor is used to record the target vertices connected by each edge of the edge type in the first specified order of the corresponding source vertices.

18. The apparatus according to claim 16 or 17, characterized in that, The determining module is used for: Determine the target edge type corresponding to the operation instruction, and obtain the initial execution function corresponding to the operation instruction; If the edge connection information corresponding to the target edge type is stored through the first specified tensor data structure, then the initial execution function is determined as the target execution function. If the edge connection information corresponding to the target edge type is stored through the second specified tensor data structure, then the specified statement is deleted from the initial execution function, and the initial execution function after deleting the statement is determined as the target execution function.

19. The apparatus according to claim 16 or 17, characterized in that, The determining module is used for: Determine the target edge type corresponding to the operation instruction; Determine the type of the target tensor data structure that stores the edge connection information corresponding to the target edge type; Based on the correspondence between the operation instruction, the type of the target tensor data structure, and the execution function, a target execution function corresponding to both the operation instruction and the type of the target tensor data structure is determined.

20. The apparatus according to any one of claims 11-19, characterized in that, The device further includes a conversion module for: In the initial graph database, obtain the edge connection information corresponding to each edge type stored in other data structures, and convert the edge connection information corresponding to each edge type from the other data structures into the tensor data structure; The graph database is obtained based on the transformed edge connection information corresponding to each edge type.

21. A computing device cluster, characterized in that, It includes at least one computing device, each computing device including a processor and memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device to cause the cluster of computing devices to perform the method as described in any one of claims 1-10.

22. A computer-readable storage medium, characterized in that, Includes computer program instructions, which, when executed by a cluster of computing devices, perform the method as described in any one of claims 1-10.

23. A computer program product containing instructions, characterized in that, When the instruction is executed by the computing device cluster, the computing device cluster causes the computing device cluster to perform the method as described in any one of claims 1-10.

Images