Query method and system for large-scale hypergraphs

Abstract

The application provides a query method and system for a large-scale hypergraph, comprising a data loading step: hypergraph data is loaded and divided by a distributed method, and the hypergraph data loaded into the memory is stored in a key-value pair form; a thread starting step: one or more agent threads and working threads are started; an analysis step: the agent thread receives a hypergraph query request from a user, analyzes the query request of the user, and dispatches a hypergraph query task obtained after the analysis to the working thread; a query step: after the working thread receives the hypergraph query task, the hypergraph pattern in the hypergraph query task is gradually executed to obtain a query result, and the query result is returned to the agent thread; a feedback step: the agent thread returns the query result to the user. The application can quickly calculate the position of the data required by the current query, can better reuse the modules of the original system, and improves the query performance of the hypergraph data.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and more specifically, to a method and system for querying large-scale hypergraphs. Background Technology

[0002] In the internet age, with the widespread adoption of the internet, the data generated by social networks has exploded. This data has several notable characteristics: enormous volume, extremely rapid updates, and a complex, interconnected nature, much like a network. Against this backdrop, to adapt to these new scenarios, a number of graph databases based on graph models have emerged. In graph models, vertices and edges are used to represent entities and relationships within the data. Compared to traditional relational models, graph models are easier to represent, extend, query, and analyze, making them better suited to network data.

[0003] Knowledge graphs are a data description model based on graph models, and represent the most closely related and widely used application of graph databases. RDF (Relational Data Framework) is a data model defined by the W3C, using graph models to represent data relationships on the Web. SPARQL, also defined by the W3C, is a query language specifically designed for querying data stored using the RDF model. SPARQL statements typically consist of several <subject, verb, object> triple patterns, and graph query systems find the target data by matching these triple patterns.

[0004] However, as online data becomes increasingly massive and complex, the limitations of traditional graph models have become apparent. Some articles point out that groups are a fundamental component of many social interactions, from participating in activities together to teamwork; multi-entity relationships, or "group" relationships, are sometimes even more common than binary relationships. Traditional RDF graphs only contain entities and binary relationships between them, making it difficult to represent this concept of "groups."

[0005] The emergence of hypergraphs addresses the shortcomings of traditional graph models in representing "group" relationships between entities. Hypergraphs are an extension of graph models; unlike traditional graph models that abstract entities and their binary relationships as nodes and edges respectively, hypergraphs abstract multi-faceted relationships between entities as hyperedges. The concept of hypergraphs can be applied to many real-world scenarios; for example, common shareholders of two companies or all co-authors of a paper can be represented using hyperedge relationships. Therefore, applying the concept of hypergraphs to traditional graph query systems to express multi-faceted relationships between entities becomes a solution to the problem mentioned above.

[0006] Currently, there is a wealth of research on RDF graph query systems both domestically and internationally, but research papers specifically on hypergraph query systems are extremely rare. Some related papers delve into the partitioning of hypergraph data in distributed systems, including vertex and edge partitioning algorithms. Other papers on hypergraph query systems mostly explore subgraph queries, that is, finding graphs within or contained within the query subgraph from graph data (multiple graphs). Very few hypergraph query systems address graph structure matching queries, which involve matching a specific graph pattern within graph data (a large graph) based on the query information. Furthermore, hypergraph query systems that address graph structure matching queries either lack specific performance test results or fail to simultaneously meet the following two metrics:

[0007] (1) Supports storage of billions of vertices and superedges.

[0008] (2) Complete basic hypergraph queries within a few milliseconds.

[0009] Therefore, how to design and optimize the storage structure and query method for hypergraph models so that both metrics can be satisfied simultaneously is a major problem that researchers in this field urgently need to solve. Summary of the Invention

[0010] In view of the deficiencies in the prior art, the purpose of this invention is to provide a method and system for querying large-scale hypergraphs.

[0011] A method for querying large-scale hypergraphs according to the present invention includes:

[0012] Data loading steps: The hypergraph data is loaded and partitioned using a distributed method, and the hypergraph data loaded into memory is stored in key-value pairs.

[0013] Thread startup steps: Start one or more proxy threads and worker threads;

[0014] Parsing steps: The proxy thread receives the hypergraph query request from the user, parses the user's query request, and dispatches the obtained hypergraph query task to the worker thread.

[0015] Query steps: After receiving the hypergraph query task, the worker thread executes the hypergraph mode in the hypergraph query task step by step to obtain the query results, and returns the query results to the agent thread.

[0016] Feedback Steps: The proxy thread returns the query results to the user.

[0017] Preferably, the hypergraph data is persistently stored in one or more text files. Each text file consists of data rows in the form of variable-length tuples. Each tuple includes three parts, separated by punctuation. The first part includes two elements: the name of the hyperedge in the hypergraph data and the ID obtained after converting the hyperedge type. The second part is a variable-length tuple, which is a list of IDs obtained after converting the list of vertices contained in the hyperedge. The last part is a list of hyperedge attributes, containing two elements: a timestamp representing the start and end times of the hyperedge's validity period.

[0018] Preferably, the data loading step includes:

[0019] In the edge-to-vertex key-value pair, the key is the hyperedge ID, and the value is a list of IDs of the vertices contained in the hyperedge. This key-value pair is used to represent all the adjacent vertices of a hyperedge.

[0020] The key in a vertex-to-edge key-value pair is a tuple consisting of the vertex ID and the hyperedge type ID, and the value is a list of related hyperedge IDs. This key-value pair is used to represent all hyperedges of a preset type connected to a preset vertex.

[0021] Preferably, the thread initiation step includes:

[0022] The number of proxy threads and worker threads is configured before system startup. Proxy threads parse query tasks and perform load balancing on the dispatch of query tasks, while worker threads are responsible for executing query tasks. The load on proxy threads is less than the load on worker threads, and the number of worker threads is greater than the number of proxy threads.

[0023] Preferably, the query step uses SPARQL-H query commands. The SPARQL-H query commands include: each query pattern following the WHERE keyword consists of three parts: an input list, a query type parameter, and an output. The input and output positions are constants or variables. There are five query types: GE / GV, E2V, V2E, E2E, and V2V, which are used to obtain a list of hyperedges / vertices based on the type, obtain vertices based on hyperedges, obtain hyperedges based on vertices, obtain hyperedges based on hyperedges, and obtain vertices based on vertices, respectively.

[0024] Preferably, the hypergraph query task includes one or more hypergraph query steps;

[0025] The worker threads execute the query modes sequentially according to the order given in the Hypergraph query task. Each query step starts from the input variables and queries the output variables. The input variables must be known, and the output variables can be known or unknown. When the output variable is unknown, the query step needs to start from the known input to obtain the unknown variable and record it in the intermediate result. When the output variable is known, it only needs to check whether the input and output variables meet the relationship specified by the query mode. Data rows that do not meet the requirements will be removed from the intermediate result.

[0026] A query system for large-scale hypergraphs provided by the present invention includes:

[0027] Data loading module: It uses a distributed approach to load and partition hypergraph data, and stores the hypergraph data loaded into memory in key-value pairs;

[0028] Thread startup module: Starts one or more proxy threads and worker threads;

[0029] Parsing module: The proxy thread receives hypergraph query requests from users, parses the user's query requests, and dispatches the resulting hypergraph query tasks to worker threads;

[0030] Query module: After receiving a hypergraph query task, the worker thread executes the hypergraph mode in the hypergraph query task step by step to obtain the query results, and returns the query results to the agent thread;

[0031] Feedback module: The agent thread returns the query results to the user.

[0032] Preferably, the hypergraph data is persistently stored in one or more text files. Each text file consists of data rows in the form of variable-length tuples. Each tuple includes three parts, separated by punctuation. The first part includes two elements: the name of the hyperedge in the hypergraph data and the ID obtained after converting the hyperedge type. The second part is a variable-length tuple, which is a list of IDs obtained after converting the list of vertices contained in the hyperedge. The last part is a list of hyperedge attributes, containing two elements: a timestamp representing the start and end times of the hyperedge's validity period.

[0033] Preferably, the data loading module includes:

[0034] In the edge-to-vertex key-value pair, the key is the hyperedge ID, and the value is a list of IDs of the vertices contained in the hyperedge. This key-value pair is used to represent all the adjacent vertices of a hyperedge.

[0035] The key in a vertex-to-edge key-value pair is a tuple consisting of the vertex ID and the hyperedge type ID, and the value is a list of related hyperedge IDs. This key-value pair is used to represent all hyperedges of a preset type connected to a preset vertex.

[0036] Preferably, the thread startup module includes:

[0037] The number of proxy threads and worker threads is configured before system startup. Proxy threads parse query tasks and perform load balancing on the dispatch of query tasks, while worker threads are responsible for executing query tasks. The load on proxy threads is less than the load on worker threads, and the number of worker threads is greater than the number of proxy threads.

[0038] Compared with the prior art, the present invention has the following beneficial effects:

[0039] 1. This invention uses edge partitioning to divide hypergraph data across multiple machines, enabling rapid calculation of the location of the data required for the current query during query execution; storing the hypergraph's adjacency list in key-value pair format allows for better reuse of modules from the original system and improves the query performance of hypergraph data.

[0040] 2. This invention utilizes RDMA technology, which can greatly improve the efficiency of data communication between different machines and reduce the overhead of data partitioning in distributed scenarios.

[0041] 3. Based on demand research, this invention designs a hypergraph query syntax SPARQL-H on the basis of the SPARQL language standard formulated by W3C. It can meet most practical needs by retrieving hyperedges / vertices by type, retrieving vertices by hyperedges, retrieving hyperedges by vertices, retrieving hyperedges by hyperedges, and retrieving vertices by vertices.

[0042] 4. While realizing the storage of hypergraphs and the parsing and execution of SPARQL-H statements, this invention retains some SPARQL syntax such as SELECT and WHERE keywords, making it more user-friendly for those familiar with SPARQL syntax.

[0043] 5. Based on the characteristics of the SPARQL-H hypergraph query syntax, this invention optimizes the execution of various hypergraph query modes and designs and implements various intermediate result storage structures to improve the traversal efficiency of intermediate results, thereby improving the overall query execution efficiency. Attached Figure Description

[0044] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0045] Figure 1 This is a flowchart illustrating the design of a query method for large-scale hypergraph data used in this invention.

[0046] Figure 2 The SPARQL-H hypergraph query statement is used in this example. Detailed Implementation

[0047] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0048] Example 1

[0049] This invention provides a query method and system for large-scale hypergraph data, such as... Figure 1-2 As shown, it includes:

[0050] Step S1: When the system starts, a distributed method is used to load and partition the hypergraph data, and the hypergraph data loaded into memory is stored in key-value pairs.

[0051] Thread startup steps: Start several agent threads and several worker threads on each machine;

[0052] Parsing steps: The proxy thread receives the hypergraph query request from the user, parses the user's query request, and dispatches the obtained hypergraph query task to the worker thread.

[0053] Query steps: After receiving the hypergraph query task, the worker thread executes the hypergraph patterns in the query step by step to obtain the final query result, and returns the query result to the agent thread;

[0054] Feedback Steps: The proxy thread returns the query results to the user.

[0055] Specifically, the hypergraph data is persistently stored in several text files. Each text file consists of data rows in the form of variable-length tuples. Each tuple is divided into three parts, separated by punctuation. The first part includes two elements: the name of the hyperedge and the ID obtained after the hyperedge type conversion. The second part is a variable-length tuple, which is a list of IDs obtained after the conversion of the list of vertices contained in the hyperedge. The last part is a list of hyperedge attributes, which generally contains two elements: a timestamp indicating the start and end times of the edge's validity period.

[0056] Specifically, the data loading step uses the following: The key in the edge-to-vertex key-value pair is the hyperedge ID, and the value is a list of IDs of the vertices contained in the hyperedge. This key-value pair is used to represent all adjacent vertices of a hyperedge. The key in the vertex-to-edge key-value pair is a tuple composed of vertex ID and hyperedge type ID, and the value is a list of related hyperedge IDs. For example, the hyperedge corresponding to the key composed of vertex A and hyperedge type B is the hyperedge connected to vertex A and of type B. This key-value pair is used to represent all hyperedges of a specific type connected to a specific vertex.

[0057] Specifically, the thread startup steps are as follows: the number of proxy threads and worker threads is configured before system startup; the main task of the proxy threads is to parse the queries and perform load balancing on the dispatch of query tasks, with a relatively small load, while the worker threads are responsible for executing query tasks, with a relatively large load. Therefore, it is stipulated that the number of worker threads must be higher than the number of proxy threads, and the former is often more than ten times the latter.

[0058] Specifically, the user's query request uses SPARQL-H, a self-designed SPARQL-like hypergraph query command; one of the parameters of the hypergraph query command is the path name of a text file, the content of which is a SPARQL-H statement containing several hypergraph query patterns.

[0059] SPARQL-H query commands include: Each query pattern following the WHERE keyword consists of three parts: an input list, query type parameters, and output. The input and output positions can be constants or variables. There are five query types: GE / GV, E2V, V2E, E2E, and V2V, which are used to obtain a list of hyperedges / vertices based on the type, obtain vertices based on hyperedges, obtain hyperedges based on vertices, obtain hyperedges based on hyperedges, and obtain vertices based on vertices, respectively.

[0060] More specifically, the GE / GV query keywords are "etype" and "vtype", used to retrieve relevant hyperedges based on hyperedge type and relevant vertices based on vertex type, respectively; the E2V query keyword is "vertices", used to retrieve all vertices connected to a given input edge, and when there are multiple input edges, it retrieves the intersection of their adjacent vertices; the V2E query keyword is "edges", used to retrieve all hyperedges connected to a given input vertex, and when there are multiple input vertices, it retrieves the intersection of their adjacent hyperedges; the E2E query keywords are "intersectEdges", "inEdges", and "containEdges", used to retrieve all hyperedges that intersect, contain, or are contained with a given input hyperedge, respectively. This type of query mode generally includes parameters to specify the output hyperedge type and the degree of intersection; the V2V query keyword is "intersectVertices", used to retrieve all vertices that intersect with a given input vertex's adjacent hyperedges. This type of query mode generally includes parameters to specify the type of intersecting hyperedges.

[0061] Specifically, the parsing step involves sending the parsed query request to a worker thread on a preset machine via a TCP or RDMA network.

[0062] Specifically, the query task includes several hypergraph query steps;

[0063] The query process employs the following steps: worker threads execute queries sequentially according to the order given in the query task. Since Hypergraph data is typically distributed across various machines, it is often necessary to obtain complete data by publishing subqueries to other machines during the execution process. As the query is executed, the intermediate results are continuously updated, and the final result is obtained when all query steps have been completed.

[0064] Each query step starts from the input variables and queries the output variables. The input variables must be known, while the output variables can be known or unknown. When the output variable is unknown, the query step needs to start from the known input to obtain the unknown variable and record it in the intermediate results. When the output variable is known, it only needs to check whether the input and output variables conform to the relationship specified by the query mode. Data rows that do not conform will be removed from the intermediate results.

[0065] To support the storage and retrieval of large-scale hypergraph data, this invention employs a distributed architecture, using multiple machines to store the graph data. Each machine is responsible for storing a portion of the graph, and all machines can collaboratively execute a SPARQL-H query to accelerate query speed. Communication between the machines uses TCP or RDMA networks. If RDMA-enabled hardware is available, it should be used for communication, as this significantly reduces query latency. Of course, using TCP communication can also achieve correct queries.

[0066] This invention redesigns the underlying data structure to support the storage of hypergraph data based on the existing native knowledge graph query system. It also designs a hypergraph query syntax, SPARQL-H, based on the SPARQL standard syntax. Furthermore, it modifies the query syntax parsing module and query execution module in the original system to enable them to support the hypergraph query syntax SPARQL-H, ultimately achieving efficient querying of large-scale hypergraph data.

[0067] Example 2

[0068] The present invention also provides a query system for large-scale hypergraph data. The query system for large-scale hypergraph data can be implemented by executing the process steps of the query method for large-scale hypergraph data. That is, those skilled in the art can understand the query method for large-scale hypergraph data as a preferred embodiment of the query system for large-scale hypergraph data.

[0069] This invention provides a query system for large-scale hypergraph data, comprising:

[0070] Data loading module: When the system starts up, it uses a distributed method to load and partition the hypergraph data, and stores the hypergraph data loaded into memory in key-value pairs;

[0071] Thread startup module: Starts several agent threads and several worker threads on each machine;

[0072] Parsing module: The proxy thread receives hypergraph query requests from users, parses the user's query requests, and dispatches the resulting hypergraph query tasks to worker threads;

[0073] Query module: After receiving a hypergraph query task, the worker thread executes the hypergraph patterns in the query step by step to obtain the final query result, and returns the query result to the agent thread;

[0074] Feedback module: The agent thread returns the query results to the user.

[0075] Specifically, the hypergraph data is persistently stored in several text files. Each text file consists of data rows in the form of variable-length tuples. Each tuple is divided into three parts, separated by punctuation. The first part includes two elements: the name of the hyperedge and the ID obtained after the hyperedge type conversion. The second part is a variable-length tuple, which is a list of IDs obtained after the conversion of the list of vertices contained in the hyperedge. The last part is a list of hyperedge attributes, which generally contains two elements: a timestamp indicating the start and end times of the edge's validity period.

[0076] Specifically, the data loading module uses the following: In edge-to-vertex key-value pairs, the key is the hyperedge ID, and the value is a list of IDs of the vertices contained in the hyperedge. This key-value pair is used to represent all adjacent vertices of a hyperedge. In vertex-to-edge key-value pairs, the key is a tuple composed of vertex ID and hyperedge type ID, and the value is a list of related hyperedge IDs. For example, the hyperedge corresponding to the key composed of vertex A and hyperedge type B is the hyperedge connected to vertex A and of type B. This key-value pair is used to represent all hyperedges of a specific type connected to a specific vertex.

[0077] Specifically, the thread startup module adopts the following approach: the number of proxy threads and worker threads is configured before system startup; the main task of the proxy threads is to parse the queries and perform load balancing on the dispatch of query tasks, which has a relatively small load, while the worker threads are responsible for executing query tasks, which has a relatively large load. Therefore, it is stipulated that the number of worker threads must be higher than the number of proxy threads, and the former is often more than ten times the latter.

[0078] Specifically, the user's query request uses SPARQL-H, a self-designed SPARQL-like hypergraph query command; one of the parameters of the hypergraph query command is the path name of a text file, the content of which is a SPARQL-H statement containing several hypergraph query patterns.

[0079] SPARQL-H query commands include: Each query pattern following the WHERE keyword consists of three parts: an input list, query type parameters, and output. The input and output positions can be constants or variables. There are five query types: GE / GV, E2V, V2E, E2E, and V2V, which are used to obtain a list of hyperedges / vertices based on the type, obtain vertices based on hyperedges, obtain hyperedges based on vertices, obtain hyperedges based on hyperedges, and obtain vertices based on vertices, respectively.

[0080] More specifically, the GE / GV query keywords are "etype" and "vtype", used to retrieve relevant hyperedges based on hyperedge type and relevant vertices based on vertex type, respectively; the E2V query keyword is "vertices", used to retrieve all vertices connected to a given input edge, and when there are multiple input edges, it retrieves the intersection of their adjacent vertices; the V2E query keyword is "edges", used to retrieve all hyperedges connected to a given input vertex, and when there are multiple input vertices, it retrieves the intersection of their adjacent hyperedges; the E2E query keywords are "intersectEdges", "inEdges", and "containEdges", used to retrieve all hyperedges that intersect, contain, or are contained with a given input hyperedge, respectively. This type of query mode generally includes parameters to specify the output hyperedge type and the degree of intersection; the V2V query keyword is "intersectVertices", used to retrieve all vertices that intersect with a given input vertex's adjacent hyperedges. This type of query mode generally includes parameters to specify the type of intersecting hyperedges.

[0081] Specifically, the parsing module sends the parsed query request to a worker thread on a preset machine via a TCP or RDMA network.

[0082] Specifically, the query task includes several hypergraph query steps;

[0083] The query module adopts a mode where worker threads execute queries sequentially according to the order given in the query task. Since HyperGraph data is usually distributed across various machines, it is often necessary to obtain complete data by publishing subqueries to other machines during the execution process. As the query is executed, the intermediate results are continuously updated, and the final result is obtained when all query steps have been executed.

[0084] Each query step starts from the input variables and queries the output variables. The input variables must be known, while the output variables can be known or unknown. When the output variable is unknown, the query step needs to start from the known input to obtain the unknown variable and record it in the intermediate results. When the output variable is known, it only needs to check whether the input and output variables conform to the relationship specified by the query mode. Data rows that do not conform will be removed from the intermediate results.

[0085] To support the storage and retrieval of large-scale time-series data, this invention employs a distributed architecture, using multiple machines to store graph data. Each machine is responsible for storing a portion of the graph, and all machines can collaboratively execute a SPARQL-H query to accelerate query speed. Communication between the machines uses TCP or RDMA networks. If RDMA-enabled hardware is available, it should be used for communication, as this significantly reduces query latency. Of course, correct queries can also be achieved using TCP.

[0086] This invention redesigns the underlying data structure to support the storage of hypergraph data based on the existing native knowledge graph query system. It also designs a hypergraph query syntax, SPARQL-H, based on the SPARQL standard syntax. Furthermore, it modifies the query syntax parsing module and query execution module in the original system to enable them to support the hypergraph query syntax SPARQL-H, ultimately achieving efficient querying of large-scale hypergraph data.

[0087] Example 3

[0088] Example 3 is a preferred example of Example 1.

[0089] This invention provides a query method and system for large-scale hypergraph data, enabling queries on such data. The following example uses eight machines. Figure 1 The following steps are described in detail:

[0090] In step 1, the system starts simultaneously on multiple machines, each loading hypergraph data from a user-specified directory. Hypergraph data typically consists of two text files storing the mapping between strings and IDs, and several graph data files. The latter usually comprises tens of thousands of rows of variable-length tuples. The first part of each tuple contains the hyperedge name and its type ID; the second part is a list of vertex IDs contained in the hyperedge; and the third part is a list of hyperedge attributes. These three parts are separated by the "|" symbol. Finally, the system distributes and evenly stores the data across the memory of the eight machines.

[0091] In step 2, the system creates one agent thread and 16 worker threads on each machine;

[0092] In step 3, the user sends a SPARQL-H query request to agent thread 0 on machine 0. This agent thread receives the query from the user (the specific content of the query is as follows...). Figure 2 After the request is made, the parsing module in the system is called to convert the query request from string form into a data structure. In other words, the useful content in the query request is extracted and organized into a data structure that the worker thread can understand. Then the proxy thread selects worker thread 8 on machine 15 as the target worker thread and forwards the query task to the worker thread through TCP or RDMA network.

[0093] Step 4: After receiving the parsed query request, worker thread 8 of machine 15 begins to execute the query task. In this embodiment, the query task consists of 5 query steps. During execution, the data involved in some query steps is distributed across various machines, requiring the acquisition of complete query data through single or multiple subtask releases. As the query is executed, the intermediate results are continuously updated. When all 5 query steps are completed, the final result is obtained. After obtaining the result, worker thread 8 of machine 15 sends the query result to the proxy thread (i.e., proxy thread 0 of machine 0) that was designated to execute the query via TCP or RDMA network.

[0094] Step 5: Agent thread 0 of machine 0 returns the query result to the user, and then returns to step 3.

[0095] Those skilled in the art will understand that, besides implementing the system and its various devices, modules, and units provided by this invention in the form of purely computer-readable program code, the same functions can be achieved entirely through logical programming of the method steps, making the system and its various devices, modules, and units of this invention function in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, the system and its various devices, modules, and units provided by this invention can be considered as a hardware component, and the devices, modules, and units included therein for implementing various functions can also be considered as structures within the hardware component; alternatively, the devices, modules, and units for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0096] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. A method for querying large-scale hypergraphs, characterized in that, include: Data loading steps: The hypergraph data is loaded and partitioned using a distributed method, and the hypergraph data loaded into memory is stored in key-value pairs. Thread startup steps: Start one or more proxy threads and worker threads; Parsing steps: The proxy thread receives the hypergraph query request from the user, parses the user's query request, and dispatches the obtained hypergraph query task to the worker thread. Query steps: After receiving the hypergraph query task, the worker thread executes the hypergraph mode in the hypergraph query task step by step to obtain the query results, and returns the query results to the agent thread. Feedback steps: The proxy thread returns the query results to the user; The query steps employ SPARQL-H query commands. Each query pattern following the WHERE keyword comprises three parts: an input list, a query type parameter, and an output. The input and output positions contain constants or variables. There are five query types: GE / GV, E2V, V2E, E2E, and V2V, which are used to obtain a list of hyperedges / vertices based on the type, obtain vertices based on hyperedges, obtain hyperedges based on vertices, obtain hyperedges based on hyperedges, and obtain vertices based on vertices, respectively.

2. The query method for large-scale hypergraphs according to claim 1, characterized in that, The hypergraph data is persistently stored in one or more text files. Each text file consists of data rows in the form of variable-length tuples. Each tuple includes three parts, separated by punctuation. The first part includes two elements: the name of the hyperedge in the hypergraph data and the ID obtained after converting the hyperedge type. The second part is a variable-length tuple, which is a list of IDs obtained after converting the list of vertices contained in the hyperedge. The last part is a list of hyperedge attributes, containing two elements: a timestamp representing the start and end times of the hyperedge's validity period.

3. The query method for large-scale hypergraphs according to claim 2, characterized in that, The data loading steps include: In the edge-to-vertex key-value pair, the key is the hyperedge ID, and the value is a list of IDs of the vertices contained in the hyperedge. This key-value pair is used to represent all the adjacent vertices of a hyperedge. The key in a vertex-to-edge key-value pair is a tuple consisting of the vertex ID and the hyperedge type ID, and the value is a list of related hyperedge IDs. This key-value pair is used to represent all hyperedges of a preset type connected to a preset vertex.

4. The method for querying large-scale hypergraphs according to claim 1, characterized in that, The thread initiation steps include: The number of proxy threads and worker threads is configured before system startup. Proxy threads parse query tasks and perform load balancing on the dispatch of query tasks, while worker threads are responsible for executing query tasks. The load on proxy threads is less than the load on worker threads, and the number of worker threads is greater than the number of proxy threads.

5. The method for querying large-scale hypergraphs according to claim 1, characterized in that, The hypergraph query task includes one or more hypergraph query steps; The worker threads execute the query mode sequentially according to the order given in the hypergraph query task. Each query step starts from the input variables and queries the output variables. The input variables must be known, and the output variables can be known or unknown. When the output variable is unknown, the query step needs to start from the known input to obtain the unknown variable and record it in the intermediate result. When the output variable is known, it is only necessary to check whether the input and output variables conform to the relationship specified by the query pattern. Data rows that do not conform will be removed from the intermediate results.

6. A query system for large-scale hypergraphs, characterized in that, include: Data loading module: It uses a distributed approach to load and partition hypergraph data, and stores the hypergraph data loaded into memory in key-value pairs; Thread startup module: Starts one or more proxy threads and worker threads; Parsing module: The proxy thread receives hypergraph query requests from users, parses the user's query requests, and dispatches the resulting hypergraph query tasks to worker threads; Query module: After receiving a hypergraph query task, the worker thread executes the hypergraph mode in the hypergraph query task step by step to obtain the query results, and returns the query results to the agent thread; Feedback module: The agent thread returns the query results to the user; The query module uses SPARQL-H query commands. Each query mode following the WHERE keyword consists of three parts: an input list, a query type parameter, and an output. The input and output positions contain constants or variables. There are five query types: GE / GV, E2V, V2E, E2E, and V2V, which are used to obtain a list of hyperedges / vertices based on the type, obtain vertices based on hyperedges, obtain hyperedges based on vertices, obtain hyperedges based on hyperedges, and obtain vertices based on vertices, respectively.

7. The query system for large-scale hypergraphs according to claim 6, characterized in that, The hypergraph data is persistently stored in one or more text files. Each text file consists of data rows in the form of variable-length tuples. Each tuple includes three parts, separated by punctuation. The first part includes two elements: the name of the hyperedge in the hypergraph data and the ID obtained after converting the hyperedge type. The second part is a variable-length tuple, which is a list of IDs obtained after converting the list of vertices contained in the hyperedge. The last part is a list of hyperedge attributes, containing two elements: a timestamp representing the start and end times of the hyperedge's validity period.

8. The query system for large-scale hypergraphs according to claim 7, characterized in that, The data loading module includes: In the edge-to-vertex key-value pair, the key is the hyperedge ID, and the value is a list of IDs of the vertices contained in the hyperedge. This key-value pair is used to represent all the adjacent vertices of a hyperedge. The key in a vertex-to-edge key-value pair is a tuple consisting of the vertex ID and the hyperedge type ID, and the value is a list of related hyperedge IDs. This key-value pair is used to represent all hyperedges of a preset type connected to a preset vertex.

9. The query system for large-scale hypergraphs according to claim 6, characterized in that, The thread startup module includes: The number of proxy threads and worker threads is configured before system startup. Proxy threads parse query tasks and perform load balancing on the dispatch of query tasks, while worker threads are responsible for executing query tasks. The load on proxy threads is less than the load on worker threads, and the number of worker threads is greater than the number of proxy threads.

Images