Method, device and equipment for optimizing expression performance in graph database

Abstract

The present invention discloses a method, device, and equipment for optimizing the performance of expressions in a graph database, which relates to the field of graph database optimization and is used for optimizing expressions in a Nebula graph database. The method is characterized in that it includes the following steps: obtaining a user's query request and passing the query request to a query processing module in the graph database; the query processing module performs lexical and syntactic analysis on the query request through a parser module, and converts the query request input by the user into an untyped expression; the untyped expression is passed to a validator module for type deduction and conversion into a typed expression. Main technical solutions and effects: By improving the lifecycle management of expressions, a type deduction and runtime calculation mechanism for expressions are designed, computing efficiency is improved, graph metadata is parsed and used for expression form conversion, strongly typed expressions are more efficient, and the expression lifecycle and code implementation framework are clearly defined.

Description

Technical Field

[0001] The present invention relates to the field of graph database optimization, and in particular to a method, device and equipment for optimizing expression performance in a graph database. Background Art

[0002] Expressions in graph databases are primarily used to describe data computations. Common expressions include constant expressions, attribute expressions, scalar function expressions, aggregation expressions, and graph function expressions (such as pattern functions, existence graph functions, and metadata functions). Processing these expressions, especially graph function expressions, presents performance bottlenecks due to their high computational complexity, especially in distributed graph database architectures.

[0003] Traditional relational databases already have mature solutions for expression implementation, but the unique data model and query language (GQL) of graph databases lead to new problems in expression implementation.

[0004] Specific defects include:

[0005] The query expressions of graph databases need to be adapted according to the topological structure of the graph data, but the existing graph database expression implementations cannot effectively handle this adaptation problem, resulting in poor performance.

[0006] The design of the expression lifecycle in graph databases is relatively complex, especially in the distributed architecture of graph databases, which lacks effective optimization mechanisms. Summary of the Invention

[0007] Purpose of the invention: The purpose of the present invention is to solve the defects in the prior art and provide a method, device and equipment for optimizing the performance of expressions in a graph database.

[0008] Technical solution:

[0009] In a first aspect, the present application proposes a method for optimizing expression performance in a graph database, which is used for optimizing expressions in a graph database, comprising the following steps:

[0010] Obtain the user's query request and pass it to the query processing module in the graph database;

[0011] The query processing module performs lexical and grammatical analysis on the query request through the parser module, and converts the query request input by the user into a typeless expression;

[0012] Pass the untyped expression to the validator module for type inference, converting it into a typed expression. The goal of the inference is to attach the correct data type to each node of the expression.

[0013] Inputting a typed expression into an executor module so that a runtime calculation address of the expression is bound and stored to generate a runtime expression, wherein the expression includes specific execution information;

[0014] The runtime expression is passed to the executor module for execution. Based on the query request, the executor module returns the query result by calculating and searching the graph data.

[0015] In the validator or executor phase, a metadata synchronization mechanism is set up, including broadcasting graph metadata from the metadata management module to each cluster node through the remote procedure call heartbeat mechanism. Each node saves a local metadata snapshot for reference in expression calculation in the query.

[0016] The remote procedure call heartbeat mechanism is used to control the broadcast and publication of metadata. After the node receives the confirmation signal in the publishing phase, the metadata snapshot is used for type inference and calculation of expressions.

[0017] Preferably, the metadata management module is also responsible for the creation, broadcasting and publishing of graph metadata, and synchronizing metadata updates to distributed cluster nodes through a regular remote procedure call heartbeat mechanism, including: users defining entity types, edge types, node labels, and attribute type information of the graph database by executing graph metadata definition language requests;

[0018] After executing the data definition language request, the graph metadata is persisted to the metadata management module and broadcast to other nodes in the cluster;

[0019] Each node confirms whether it has received the new metadata based on the heartbeat signal and returns a confirmation signal.

[0020] Preferably, for each data partition of a storage node in each cluster node, the storage node receives and saves a snapshot of the metadata;

[0021] For each query processing node in the cluster, a snapshot of the graph metadata is stored in the local service process.

[0022] Preferably, the runtime expression is passed to the executor module for execution. Based on the query request, the executor module calculates and searches the graph data and returns the query results, including:

[0023] The expression execution process is divided into operator layer and expression layer;

[0024] Among them, the operator layer is responsible for managing the input table and output table, and the data table contains multiple batches;

[0025] The expression layer is responsible for receiving input data from the operator layer, accessing the data through views, performing expression calculations, and outputting the results as data blocks back to the operator layer.

[0026] Preferably, at the expression layer, after the calculation is completed, the output data will be optimized for writing through the writer and summarized into data blocks and written into the output table of the operator layer;

[0027] The management of data tables includes the division of batches;

[0028] Each data table is managed by the operator layer and divided into multiple batches to process data in blocks.

[0029] Preferably, during expression calculation, data access is abstracted through views, and the expression layer operates on data through views;

[0030] When writing output data, the data is optimized by the corresponding writer.

[0031] Preferably, the method includes: a pattern function, an existence graph function;

[0032] Optimize the pattern function and existence graph function, including:

[0033] When the graph data is first pulled from the storage node, the corresponding topology subgraph is pre-fetched and saved in the cache of the local query processing node. The degree n of the pre-fetched topology subgraph can be configured by the user. During the pre-fetching process, for each node, the topology data is requested from the storage node, and the relevant topology subgraph nodes and edges are gradually loaded into the topology subgraph of the query processing node.

[0034] When performing graph function calculations, topology subgraph data is obtained from the local topology subgraph for iterative calculations first;

[0035] When the cached topology subgraph is hit, the topology calculation is performed directly in the local cache; otherwise, a remote procedure call request is sent to the storage node to pull the required graph data and update the local topology subgraph;

[0036] Preferably, the graph database further includes a graph function expression, including: a metadata graph function;

[0037] Use the graph metadata synchronization mechanism to ensure that each node in the cluster locally stores a consistent version of the metadata snapshot;

[0038] During the metadata function expression calculation process, the untyped expression undergoes type inference in the validator module to generate a typed expression with a clear type and corresponding catalog version, including:

[0039] The typed expression after type inference will contain the return data type and the corresponding directory version, and will be converted into a runtime expression by the executor module during the execution phase so that the correct version is used for accessing the metadata during calculation. The runtime expression can generate the corresponding metadata accessor, specify the required entity type and attribute name through the attribute accessor, and use the local directory snapshot to retrieve and return the metadata results requested by the user.

[0040] In a second aspect, an embodiment of the present invention provides a device for optimizing expression performance in a graph database, including the method described in any one of the above embodiments, including:

[0041] An acquisition and transmission unit is used to obtain the user's query request and transmit the query request to the query processing module in the graph database;

[0042] The analysis and conversion unit is used for the query processing module to perform lexical and grammatical analysis on the query request through the parser module, and convert the query request input by the user into a typeless expression;

[0043] The deduction conversion unit is used to pass the untyped expression to the validator module for type deduction and convert it into a typed expression, wherein the deduction goal is to attach the correct data type to each node of the expression;

[0044] The execution unit is used to input the typed expression into the executor module, bind and store the runtime calculation address of the expression, and generate a runtime expression, wherein the expression contains specific execution information; the runtime expression is passed to the executor module for execution. According to the query request, the executor module calculates and searches the graph data and returns the query result; wherein, in the validator or executor stage, a metadata synchronization mechanism is set up, including broadcasting the graph metadata from the metadata management module to each cluster node through the remote procedure call heartbeat mechanism. Each node saves a local metadata snapshot for reference in the expression calculation in the query;

[0045] The remote procedure call heartbeat mechanism is used to control the broadcast and publication of metadata. After the node receives the confirmation signal in the publishing phase, the metadata snapshot is used for type inference and calculation of expressions.

[0046] In a third aspect, an embodiment of the present invention provides an electronic device comprising a processor and a memory. The memory is configured to store one or more computer programs; when the processor executes the one or more computer programs stored in the memory, the electronic device is capable of implementing any of the possible design methods of the first aspect.

[0047] In a fourth aspect, the present invention provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the method as described in any one of the above embodiments is implemented.

[0048] In a fifth aspect, an embodiment of the present invention further provides a computer program product, which, when running on an electronic device, enables the electronic device to execute any possible design method of any of the above aspects.

[0049] Beneficial effects:

[0050] Efficient implementation of graph database expressions. By improving expression lifecycle management, designing expression type inference and runtime calculation mechanisms, computing efficiency is improved. Graph metadata is parsed and used for expression form conversion, strongly typed expressions are more efficient, and the expression lifecycle and code implementation framework are clearly defined.

[0051] Optimize graph function expressions. For pattern functions and existence graph functions, a pre-fetch mechanism of subgraph topology cache is adopted, which reduces the dependence on the storage layer during each calculation, significantly reduces the remote procedure call overhead, and improves the calculation speed.

[0052] A metadata synchronization mechanism was designed based on the remote procedure call heartbeat mechanism to ensure metadata consistency across nodes in the distributed cluster, thereby optimizing the computing performance of metadata functions.

[0053] The implementation path for performance optimization successfully improves the execution efficiency of complex expressions in graph databases, especially in large-scale distributed environments, through specialized optimization of graph functions and localization of graph metadata.

[0054] It greatly improves the expression calculation capabilities of graph databases, reduces network communication overhead, and optimizes overall performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 Provide a schematic flow chart of the method of the present invention;

[0056] Figure 2 Provides a schematic diagram of the expression life cycle for the present invention;

[0057] Figure 3 This is a schematic diagram of the data structure of the untyped expression of the present invention;

[0058] Figure 4 Schematic diagram of metadata synchronization mechanism of the present invention;

[0059] Figure 5 Design diagram for when expression of the present invention is run;

[0060] Figure 6 Optimizing the design graph for the pattern function or the existence graph function of the present invention;

[0061] Figure 7 This is a schematic diagram showing the implementation principle of the metadata function expression of the present invention;

[0062] Figure 8 Provide a schematic diagram of a device for the present invention;

[0063] Figure 9 A schematic diagram of an apparatus is provided for the present invention. DETAILED DESCRIPTION

[0064] In order to make the technical solution of the present invention clearer, the present invention is further described in detail below with reference to the specific embodiments of the drawings.

[0065] Example 1

[0066] In order to make the purpose, technical solutions and advantages of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. Unless otherwise defined, the technical terms or scientific terms used herein should be the common meanings understood by people with ordinary skills in the field to which the present invention belongs. The words "including" and similar words used in this article mean that the elements or objects appearing before the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects.

[0067] In view of the problems existing in the existing technology, such as Figure 1-2 As shown, the present invention provides a method for optimizing expression performance in a graph database, which is used for optimizing expressions in a graph database, including the steps of:

[0068] S101. Obtain the user's query request and pass it to the query processing module in the graph database (graphd, query processing is one of the core modules in the Nebula graph database, mainly used to respond to user query requests and execute query operations);

[0069] The user sends a query request through the graph database interface (such as the GQL query language), and the query request first enters the query processing module of the graph database.

[0070] S102, the query processing module performs lexical and grammatical analysis on the query request through the parser module (Parser), and converts the query request input by the user into a typeless expression;

[0071] After receiving a query request, the parser module first performs lexical and grammatical analysis on it. The goal of parsing is to convert the user's query request into a form that the graph database can understand.

[0072] After analysis, the query request will be converted into an untyped expression (UntypedExpr), which is an intermediate expression without a specific data type and usually has a tree structure.

[0073] S103: Pass the untyped expression to a validator module for type deduction to convert it into a typed expression. The deduction goal is to attach the correct data type to each node of the expression.

[0074] Untyped expressions are passed to the validator module for type inference. The validator's role is to infer the correct data type for each node in the expression based on the graph database metadata (such as node type, edge type, attribute type, etc.);

[0075] After deduction, each node of the expression will be attached with the correct data type to generate a typed expression (TypedExpr).

[0076] S104: Input the typed expression into the executor module (Executor), bind and store the runtime computation address of the expression, and generate a runtime expression (RuntimeExpr, which represents the actual operation of the expression when it is executed in the computation engine in the graph database. It carries all the information required during the execution process and ensures that the expression can be correctly calculated and optimized). The expression contains specific execution information.

[0077] Typed expressions enter the executor module, which is responsible for converting the type information of the expression into a computational form that can be executed at runtime.

[0078] In the executor phase, the computation address of the expression is bound and stored, ultimately generating a runtime expression (RuntimeExpr). The runtime expression contains specific execution information, including the functions and data types required for the computation.

[0079] The runtime expression is passed to the executor module for execution. Based on the query request, the executor module returns the query result by calculating and searching the graph data.

[0080] The runtime expression is passed to the executor module for further execution. The executor will return the query results through graph data calculation and search based on the query request. The query results will be calculated and output by the executor module and finally returned to the user.

[0081] During the validator or executor phase, a metadata synchronization mechanism is set up. This involves broadcasting graph metadata from the metadata management module (the metad module, a key component of the Nebula graph database responsible for metadata persistence and management) to each cluster node through the Remote Procedure Call Heartbeat Mechanism (RPC), a mechanism used to ensure metadata consistency across nodes in a distributed graph database system. Each node stores a local snapshot of the metadata for reference in query expression calculations.

[0082] The remote procedure call heartbeat mechanism is used to control the broadcast and publication of metadata. After the node receives the confirmation signal in the publishing phase, the metadata snapshot is used for type inference and calculation of expressions.

[0083] During the validator or executor phase, the system uses a metadata synchronization mechanism to ensure metadata consistency across different cluster nodes. This mechanism is implemented through the remote procedure call heartbeat mechanism.

[0084] Remote procedure call heartbeat mechanism: The metadata management module broadcasts the graph metadata to each cluster node. Each node receives the heartbeat message and updates the local metadata snapshot (Catalog Snapshot) for use in queries.

[0085] Only after receiving the confirmation signal (ACK) in the metadata publishing phase will the node use the updated graph metadata to perform expression type inference and calculation execution. This process ensures that in a distributed environment, all nodes can access consistent metadata, thereby ensuring the correctness of query calculations.

[0086] In some specific embodiments, combined Figure 3 ;

[0087] Top level: count expression

[0088] The outermost layer is a count expression, which is usually used in queries to count the number of elements that meet the conditions.

[0089] Nested: case expression (conditional expression)

[0090] Below the count expression, a case expression is nested. This is a conditional expression that returns different values depending on whether the condition is true or false.

[0091] In this diagram, the conditional checks whether n.score (the score of entity n) is greater than 60.

[0092] Condition (case expression)

[0093] The conditional part checks whether n.score is greater than 60.

[0094] If the condition is true (i.e. the score is greater than 60), then return 1, indicating that the condition is met (perhaps by counting the items that meet the condition).

[0095] If the condition is false (the score is not greater than 60), null is returned, indicating that the condition is not met;

[0096] The attribute expression represents access to the score attribute of entity n; this attribute expression is used in the conditional part of caseexpression to compare whether n.score is greater than 60.

[0097] It is analyzed that it has five leaf nodes, each of which represents an element in the expression. The following is a detailed analysis of their meaning and type deduction process:

[0098] 1. Leaf node: null type: empty type (null). In SQL or graph database queries, null usually represents a missing value or an empty value. It has no data type itself.

[0099] 2. Leaf node: 1 Type: Integer, 1 is a constant, representing the value returned when n.score > 60. Its type is integer, representing a numeric value;

[0100] 3. Leaf node: 60 Type: Integer. 60 is also a constant, representing a comparison value in the case statement, used for comparison with n.score. Its type is also an integer.

[0101] 4. Leaf node: n, type: node type. n represents an entity in the graph database, usually a node. In this example, n represents the student node. The type of n is node type, which means it is an entity node in the graph database and contains certain attributes and labels.

[0102] 5. Leaf node: score, type: string. Score is an attribute that typically represents a specific property of a node. Here, the score attribute is used to store the student's score. Its type is string, representing the attribute name.

[0103] The type deduction process of the above example is as follows:

[0104] Catalog Snapshot: Catalog snapshots contain the definitions, relationships, and properties of all entity types in a graph database. For example, the node types Teacher and Student in a graph database, as well as the edge type between them (such as Teach), are all defined in the catalog snapshot.

[0105] Graph Pattern: In a query, n is generated by a graph pattern (e.g., (v:Teacher)-[:Teach]->(n:Student)). The graph pattern here describes the relationships and nodes in the graph, and n represents the Student node.

[0106] Accessing graph metadata: Given a label in the graph schema (e.g., Student), the query accesses the graph metadata to infer all attribute types for the node type Student. The graph metadata lists all attributes and their types for the node. For example, the score attribute might be of type Integer or Float.

[0107] Inferring attribute types: In this example, n is derived from the graph schema:

[0108] Generated by (v:Teacher)-[:Teach]->(n:Student), by accessing the metadata, the query can infer the data type of the attribute score of n, and then determine the type of the score attribute used in the case when n.score > 60.

[0109] In some specific embodiments, combined Figure 4 The metadata management module is also responsible for the creation (CREATE), broadcast (BROADCAST) and publication (PUBLIC) of graph metadata, and synchronizes metadata updates to distributed cluster nodes through a regular remote procedure call heartbeat mechanism, including: users define the entity type, edge type, node label, and attribute type information of the graph database by executing graph metadata definition language requests;

[0110] After executing the data definition language request (DLL request), the graph metadata (Part 1, Part 2) is persisted to the metadata management module, and the persisted graph metadata is broadcast to other nodes in the cluster;

[0111] Each node confirms whether it has received the new metadata based on the heartbeat signal and returns a confirmation signal.

[0112] In some specific embodiments, for each data partition of a storage node in each cluster node (storaged, a storage node in the cluster, responsible for storing graph data and metadata and providing data to the query processing module), the storage node receives and saves a catalog snapshot of the graph metadata.

[0113] For the query processing node in each cluster node (graph query module, query module of the graph database, responsible for processing user query requests (Query Request) and executing graph queries according to query conditions. It will parse and optimize queries based on metadata) a snapshot of the graph metadata is retained in the local service process.

[0114] Specifically, users define or modify information such as entity types (such as node types, edge types), node labels, attribute types, etc. in the graph database by executing DDL (Data Definition Language) requests. These requests are first processed by the metadata management module, which is responsible for creating and persistently storing graph metadata and broadcasting this graph metadata to other nodes in the cluster. After executing the data definition language request, the graph metadata will be persisted in Metad and pushed to other nodes through the broadcast mechanism. Each node in the cluster (such as storage nodes and query processing) will receive the broadcasted graph metadata and confirm whether it has received the updated metadata. The node will return an acknowledgment signal (ACK) message to the metadata management module, indicating that they have received and processed the updated metadata. Once the graph metadata synchronization is completed, each node can use the updated metadata to process query requests, such as Expression Pushdown (an optimization technology that pushes part of the computing tasks in the query to the storage layer for execution, thereby reducing data transmission and improving query efficiency) and Expression Evaluation (performing query expression calculation operations, query processing and calculation based on the graph metadata, and returning the final query results) to execute specific query tasks; the heartbeat synchronization mechanism is used to ensure the consistency of metadata of each distributed node and metadata localization to optimize expression implementation performance.

[0115] In some specific embodiments, combined Figure 5 , passing the runtime expression to the executor module for execution. Based on the query request, the executor module calculates and searches the graph data and returns the query results, including:

[0116] The expression execution process is divided into operator layer and expression layer;

[0117] Among them, the operator layer is responsible for managing the input table and output table, and the data table contains multiple batches;

[0118] The expression layer is responsible for receiving input data from the operator layer, accessing the data through views, performing expression calculations, and outputting the results as data blocks back to the operator layer.

[0119] In some specific embodiments, at the expression layer, after the calculation is completed, the output data will be optimized for writing through the writer and summarized into data blocks and written into the output table of the operator layer;

[0120] The management of data tables includes the division of batches;

[0121] Each data table is managed by the operator layer and divided into multiple batches to process data in blocks.

[0122] In some specific embodiments, during expression calculation, data access is abstracted through views, and the expression layer operates on data through views;

[0123] When writing output data, the data is optimized by the corresponding writer.

[0124] Specifically, the operator layer:

[0125] This layer manages and processes tables. Each table contains multiple batches. During query execution, input data is organized into batches, called Input Tables. Output results are also stored in Output Tables. This layer is primarily responsible for data transmission and organization.

[0126] Expression Layer:

[0127] This layer is responsible for performing expression calculations. It receives data from the operator layer and accesses the data through views (such as NodeView).

[0128] During expression evaluation, the execution function (such as Expr::BatchEval) processes each batch of data, performs necessary calculations, and generates results. The calculation results are returned in the form of data chunks and written to the output table of the operator layer.

[0129] The writing process uses the corresponding type of writer (such as node writer) to optimize data writing and ensure that the calculation results are written efficiently.

[0130] Data is input into the expression calculation module in batches. Batch processing can effectively reduce the number of data processing times and resource consumption, improving the efficiency of query processing. By passing data to the expression calculation module in batches, memory and computing resources can be better managed, avoiding performance bottlenecks when processing large amounts of data at a time.

[0131] During the computation process, data is accessed through views and output is optimized through a write mechanism (writer). This mechanism ensures efficient data reading and writing, especially when processing large-scale data. It can avoid frequent disk I / O operations and improve the efficiency of data transmission and storage. Each data type has a corresponding view and write optimizer (such as a node writer), making data computation and writing more efficient.

[0132] In some specific embodiments, combined Figure 6 , including: pattern function, exists function;

[0133] Optimize the pattern function and existence graph function, including:

[0134] When the graph data is first pulled from the storage node, the corresponding topology subgraph is pre-fetched and saved in the cache of the local query processing node. The degree n of the pre-fetched topology subgraph can be configured by the user. During the pre-fetching process, for each node, the topology data is requested from the storage node, and the relevant topology subgraph nodes and edges are gradually loaded into the topology subgraph of the query processing node.

[0135] When performing graph function calculations, topology subgraph data is obtained from the local topology subgraph for iterative calculations first;

[0136] When the cached topology subgraph is hit, the topology calculation is performed directly in the local cache; otherwise, a remote procedure call request is sent to the storage node to pull the required graph data and update the local topology subgraph;

[0137] Specifically, the relationship between the storage layer and the graph query layer is as follows: Figure 6 The left side of the figure is the storage node layer, which stores node and edge data. The stored data is stored on disk via disk storage. The right side of the figure is the graph query node layer, which contains the subgraph cache, which is used to store graph data pre-fetched from the storage layer. The red dots are the graph data designed for the last query. The yellow dots and dotted edges in the figure are pre-fetched from the storage through remote procedure calls to the subgraph cache for query processing. The black dots and corresponding edges in the figure are not pre-fetched.

[0138] When executing graph queries, especially for functions such as pattern functions or graph functions that require graph topology calculations, the graph query layer requests node and edge data from the storage layer through remote procedure calls, and pre-fetches the n-degree topology subgraph to the local subgraph cache.

[0139] When executing a query, the graph query layer first searches the local cache for the required topological data. If a cache hit is found, the calculation is performed directly in memory without sending a remote procedure call request to the storage layer. By prefetching commonly used topological subgraphs into the cache, it is possible to avoid pulling data from the storage layer for each query, thereby reducing the remote procedure call communication overhead with the storage layer and optimizing query performance. If a cache hit is found, the execution speed of the pattern function expression is greatly improved because the calculation can be performed directly in memory, solving the problem of optimizing the performance of pattern function expressions in the existing technology: Pattern functions are a type of graph query expression unique to graph databases. Due to the need for frequent topological calculations, they usually result in poor performance. By using a subgraph cache mechanism (Subgraph Cache) to prefetch topological subgraphs and reduce communication with the storage layer, the execution efficiency of pattern functions is significantly improved.

[0140] In some specific embodiments, combined Figure 7 ,The graph database also includes graph function expressions, which include: metadata graph function (metadata function);

[0141] Use the graph metadata synchronization mechanism to ensure that each node in the cluster locally stores a consistent version of the metadata snapshot;

[0142] During the metadata function expression calculation process, the untyped expression undergoes type inference in the validator module to generate a typed expression with a clear type and corresponding catalog version (Catalog Version), including:

[0143] The typed expression after type inference will contain the return data type and the corresponding directory version, and will be converted into a runtime expression by the executor module during the execution phase so that the correct version is used for accessing the metadata during calculation. The runtime expression can generate the corresponding metadata accessor, specify the required entity type and attribute name through the attribute accessor, and use the local directory snapshot to retrieve and return the metadata results requested by the user.

[0144] Specifically, metadata functions: Untyped expressions: Metadata function expressions (such as hasProperty(v, "prop")) request graph metadata (such as the property prop of node v) in a query. The initial expression is an untyped expression (UntypedExpr), indicating that the expression has not yet been typed.

[0145] Metadata function: TypedExpr:

[0146] In the validator module, untyped expressions are converted into typed expressions (TypedExpr) through type inference. During the type inference phase, explicit type information is added to the expression and the associated metadata version number (catalog version) is included.

[0147] For example, the return value type BOOL and the entity type Teacher are deduced here.

[0148] Metadata function: RuntimeExpr:

[0149] In the executor module, typed expressions are converted into runtime expressions (RuntimeExpr) in preparation for actual calculation.

[0150] Runtime expressions generate accessors related to metadata access.

[0151] (such as PropertyGetter), and obtain the corresponding metadata through the local metadata snapshot.

[0152] The accessor specifies the desired entity type (such as Teacher) and property name (such as prop), retrieves and returns the final result from the local metadata snapshot.

[0153] Metadata function expressions are often used to access and query graph database metadata. Because they need to frequently access graph metadata sources (such as metad) to obtain information such as node attributes, performance is often poor. This solution optimizes them through expression lifecycle design: during the type inference phase, the type of the expression is inferred in advance and the appropriate metadata accessor is constructed; queries are executed through local metadata snapshots, reducing frequent remote procedure call communications with metad, thereby reducing latency and resource consumption.

[0154] In some embodiments, the present application proposes a device for optimizing expression performance in a graph database, combining Figure 8 , including the method as described in the above embodiment, including:

[0155] The acquisition and transmission unit 301 is used to obtain the user's query request and transmit the query request to the query processing module in the graph database;

[0156] The analysis and conversion unit 302 is used for the query processing module to perform lexical and grammatical analysis on the query request through the parser module, and convert the query request input by the user into a typeless expression;

[0157] The derivation conversion unit 303 is used to pass the untyped expression to the validator module for type derivation to convert it into a typed expression, wherein the derivation goal is to attach the correct data type to each node of the expression;

[0158] The execution unit 304 is used to input the typed expression into the executor module, bind and store the runtime calculation address of the expression, and generate a runtime expression, wherein the expression contains specific execution information. The runtime expression is passed to the executor module for execution. According to the query request, the executor module calculates and searches the graph data and returns the query result. In the validator or executor stage, a metadata synchronization mechanism is set up, including broadcasting the graph metadata from the metadata management module to each cluster node through the remote procedure call heartbeat mechanism. Each node saves a local metadata snapshot for reference in the expression calculation in the query.

[0159] The remote procedure call heartbeat mechanism is used to control the broadcast and publication of metadata. After the node receives the confirmation signal in the publishing phase, the metadata snapshot is used for type inference and calculation of expressions.

[0160] All relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module and will not be repeated here.

[0161] In other embodiments of the present invention, the present invention discloses an electronic device, such as Figure 9 As shown, the electronic device may include: one or more processors 401; a memory 402; a display 403; one or more applications (not shown); and one or more computer programs 404. The above components may be connected via one or more communication buses 405. The one or more computer programs 404 are stored in the memory 402 and configured to be executed by the one or more processors 401. The one or more computer programs 404 include instructions, which may be used to execute the following: Figures 1 to 7 and each step in the corresponding embodiment.

[0162] Through the description of the above embodiments, those skilled in the art will clearly understand that for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working processes of the above-described systems, devices, and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0163] Each functional unit in each embodiment of the present invention may be integrated into a processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The above-mentioned integrated units may be implemented in the form of hardware or software functional units.

[0164] If the integrated unit is implemented as a software functional unit and sold or used as a standalone product, it can be stored on a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of the present invention, or the portion that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product, stored on a storage medium, includes instructions for causing a computer device (such as a personal computer, server, or network device) or processor to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as flash memory, mobile hard disks, read-only memory, random access memory, magnetic disks, or optical disks.

[0165] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present invention should be covered by the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A method for optimizing expression performance in a graph database, used for expression optimization in Nebula graph database, characterized in that: Including steps: Obtain the user's query request and pass it to the query processing module in the graph database; The query processing module performs lexical and grammatical analysis on the query request through the parser module, and converts the query request input by the user into a typeless expression; Pass the untyped expression to the validator module for type inference, converting it into a typed expression. The goal of the inference is to attach the correct data type to each node of the expression. Inputting a typed expression into an executor module so that a runtime calculation address of the expression is bound and stored to generate a runtime expression, wherein the expression includes specific execution information; The runtime expression is passed to the executor module for execution. Based on the query request, the executor module calculates and searches the graph data and returns the query results. This includes: The expression execution process is divided into operator layer and expression layer; Among them, the operator layer is responsible for managing the input table and output table, and the data table contains multiple batches; The expression layer is responsible for receiving input data from the operator layer, accessing the data through views, performing expression calculations, and outputting the results as data blocks back to the operator layer. At the expression layer, after the calculation is completed, the output data will be optimized by the writer and summarized into data blocks and written to the output table of the operator layer; The management of data tables includes the division of batches; Each data table is managed by the operator layer and divided into multiple batches to process data in blocks; During expression calculation, data access is abstracted through views, and the expression layer operates on data through views; When writing output data, the data is optimized through the corresponding writer; In the validator or executor phase, a metadata synchronization mechanism is set up, including broadcasting graph metadata from the metadata management module to each cluster node through the remote procedure call heartbeat mechanism. Each node saves a local metadata snapshot for reference in expression calculation in the query. The remote procedure call heartbeat mechanism is used to control the broadcast and publication of metadata. After the node receives the confirmation signal in the publishing phase, the metadata snapshot is used for type inference and calculation of expressions.

2. The method according to claim 1, characterized in that The metadata management module is also responsible for the creation, broadcasting, and publishing of graph metadata, and synchronizes metadata updates to distributed cluster nodes through a regular remote procedure call heartbeat mechanism. This includes: users defining the entity type, edge type, node label, and attribute type information of the graph database by executing graph metadata definition language requests; After executing the data definition language request, the graph metadata is persisted to the metadata management module and broadcast to other nodes in the cluster; Each node confirms whether it has received the new metadata based on the heartbeat signal and returns a confirmation signal.

3. The method according to claim 1, characterized in that For each data partition of the storage node in each cluster node, the storage node will receive and save a snapshot of the metadata; For each query processing node in the cluster, a snapshot of the graph metadata is stored in the local service process.

4. The method according to claim 1, wherein The graph database also includes graph function expressions, including: pattern functions, existence graph functions; Optimize the pattern function and existence graph function, including: When pulling graph data from the storage node for the first time, the corresponding topology subgraph is pre-fetched and saved in the cache of the local query processing node; The degree n of the prefetched topology subgraph can be configured by the user; When performing graph function calculations, topology subgraph data is obtained from the local topology subgraph for iterative calculations first; When the cached topology subgraph is hit, the topology calculation is performed directly in the local cache; otherwise, a remote procedure call request is sent to the storage node to pull the required graph data and update the local topology subgraph.

5. The method according to claim 1, wherein Graph databases also include graph function expressions, which include: metadata graph functions; Use the graph metadata synchronization mechanism to ensure that each node in the cluster locally stores a consistent version of the metadata snapshot; During the metadata function expression calculation process, the untyped expression undergoes type inference in the validator module to generate a typed expression with a clear type and corresponding catalog version, including: After type inference, the typed expression will contain the return data type and the corresponding directory version. During the execution phase, the executor module will convert it into a runtime expression so that the correct version is used when accessing the preserved data. Among them, the runtime expression can generate the corresponding metadata accessor, specify the required entity type and attribute name through the attribute accessor, and use the local directory snapshot to retrieve and return the metadata results requested by the user.

6. A device for optimizing expression performance in a graph database, comprising the method according to any one of claims 1 to 5, characterized in that: include: An acquisition and transmission unit is used to obtain the user's query request and transmit the query request to the query processing module in the graph database; The analysis and conversion unit is used for the query processing module to perform lexical and grammatical analysis on the query request through the parser module, and convert the query request input by the user into a typeless expression; The deduction conversion unit is used to pass the untyped expression to the validator module for type deduction and convert it into a typed expression, wherein the deduction goal is to attach the correct data type to each node of the expression; The execution unit is used to input the typed expression into the executor module, bind and store the runtime calculation address of the expression, and generate a runtime expression, wherein the expression contains specific execution information; the runtime expression is passed to the executor module for execution. According to the query request, the executor module calculates and searches the graph data and returns the query result; wherein, in the validator or executor stage, a metadata synchronization mechanism is set up, including broadcasting the graph metadata from the metadata management module to each cluster node through the remote procedure call heartbeat mechanism. Each node saves a local metadata snapshot for reference in the expression calculation in the query; The remote procedure call heartbeat mechanism is used to control the broadcast and publication of metadata. After the node receives the confirmation signal in the publishing phase, the metadata snapshot is used for type inference and calculation of expressions.

7. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the computer program is executed by the processor, the processor implements the method according to any one of claims 1 to 5.

Images