Distributed RDF data query method, device and computer equipment
By using a consistent hash ring in the distributed RDF data query system to divide the RDF data set and optimize the sending of SPARQL query statements, the problems of data partition overhead and uneven load are solved, and the efficiency of data query is significantly improved.
Patent Information
- Application Number
- CN202111644837.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-29
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2041-12-29
AI Technical Summary
The existing distributed RDF data query system has problems such as excessive data partition overhead and uneven load when improving parallel processing, which affects the efficiency of data query.
Use a consistent hash ring to divide the RDF data set into the corresponding child nodes, and send the subquery statements in the SPARQL query statement to the corresponding child nodes according to the query data processing priority according to the mapping relationship, perform intersection operations to obtain the target Query results.
It effectively solves the problems of excessive data partition overhead and uneven load, and improves the efficiency and speed of data query.
Smart Images

Figure CN114297260B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of graph data management technology, and in particular to a distributed RDF data query method, apparatus, computer equipment, storage medium and computer program product. Background Art
[0002] RDF (Resource Description Framework), or resource management framework, was developed by W3C. The document defines an abstract syntax (data model) for linking all RDF-based languages and specifications. While RDF data management has received widespread attention, the existence of the standard query language SPARQL defined by W3C has promoted work focused on efficiently executing SPARQL queries on large RDF datasets. As the amount of RDF data grows, the computational complexity of indexing and querying large datasets becomes challenging. Therefore, research on the optimization of SPARQL queries on large-scale RDF datasets is of great significance.
[0003] Most current SPARQL query systems are based on distributed systems, which use shared-nothing computing clusters, are built on distributed data processing frameworks, or implement proprietary distributed computing methods. RDF graphs are partitioned across multiple machines to process large data sets, and query execution is parallelized to reduce running time. Answering queries usually involves processing local data on each machine and requires data exchange between multiple machines.
[0004] However, while the above-mentioned existing distributed RDF data query processing system improves parallelism to reduce the running time, it also has problems such as excessive data partitioning overhead and uneven load balancing, which directly affect the efficiency of data query and reduce the efficiency of data query. Summary of the invention
[0005] Based on this, it is necessary to provide a distributed RDF data query method, apparatus, computer equipment, computer-readable storage medium and computer program product that can improve data query efficiency in response to the above technical problems.
[0006] In a first aspect, the present application provides a distributed RDF data query method. The method comprises:
[0007] Get RDF datasets and SPARQL query statements;
[0008] According to the constructed consistent hash ring, the RDF data set is divided into corresponding child nodes. The consistent hash ring includes the mapping relationship between the hash value of each RDF data subject value and the virtual node position of each child node;
[0009] According to the mapping relationship, the sub-query statements in the SPARQL query statement are sent to the corresponding child nodes according to the query data processing priority;
[0010] Receive the query results fed back by each child node, and perform an intersection operation on the query results to obtain the target query result.
[0011] In one embodiment, before dividing the RDF data set into corresponding child nodes according to the constructed consistent hash ring, the method further includes:
[0012] Encode the RDF data set to obtain an encoded RDF data set and a bidirectional dictionary;
[0013] According to the constructed consistent hash ring, the RDF dataset is divided into corresponding sub-nodes including:
[0014] According to the constructed consistent hash ring, the encoded RDF dataset is divided into corresponding child nodes.
[0015] In one embodiment, before sending the sub-query statements in the SPARQL query statement to the corresponding child nodes respectively according to the mapping relationship and the preset query data processing priority, the method further includes:
[0016] According to the bidirectional dictionary, the SPARQL query statement is encoded to obtain an encoded SPARQL query statement;
[0017] According to the mapping relationship, the sub-query statements in the SPARQL query statement are sent to the corresponding child nodes according to the query data processing priority, including:
[0018] According to the mapping relationship, the sub-query statements in the encoded SPARQL query statement are sent to the corresponding child nodes according to the query data processing priority.
[0019] In one embodiment, dividing the RDF data set into corresponding child nodes according to the constructed consistent hash ring includes:
[0020] Perform hash calculation on the subject value of each RDF data in the RDF data set to obtain the hash value of the subject value;
[0021] Find the virtual node position corresponding to the hash value of the subject value on the consistent hashing ring;
[0022] Each RDF data is sent to the child node corresponding to the virtual node position.
[0023] In one embodiment, according to the mapping relationship, sending the sub-query statements in the SPARQL query statement to the corresponding child nodes according to the query data processing priority respectively includes:
[0024] Sorting the SPARQL query statements according to a preset query data processing priority to obtain sorted SPARQL query statements;
[0025] According to the mapping relationship, the sorted SPARQL query statements are sent to the corresponding child nodes.
[0026] In one embodiment, the consistent hashing ring is constructed in the following manner:
[0027] Allocate a preset number of virtual nodes to each child node;
[0028] Use the hash algorithm to obtain the location of each virtual node and obtain a consistent hash ring.
[0029] In a second aspect, the present application also provides a distributed RDF data query device. The device comprises:
[0030] Data acquisition module, used to obtain RDF data sets and SPARQL query statements;
[0031] A data partitioning module, used to partition the RDF data set into corresponding child nodes according to a constructed consistent hash ring, wherein the consistent hash ring includes a mapping relationship between a hash value of each RDF data subject value and a virtual node position of each child node;
[0032] A query statement sending module, used for sending sub-query statements in the SPARQL query statement to corresponding child nodes according to the query data processing priority according to the mapping relationship;
[0033] The query result processing module is used to receive the query results fed back by each sub-node and perform an intersection operation on the query results to obtain a target query result.
[0034] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:
[0035] Get RDF datasets and SPARQL query statements;
[0036] According to the constructed consistent hash ring, the RDF data set is divided into corresponding child nodes. The consistent hash ring includes the mapping relationship between the hash value of each RDF data subject value and the virtual node position of each child node;
[0037] According to the mapping relationship, the sub-query statements in the SPARQL query statement are sent to the corresponding child nodes according to the query data processing priority;
[0038] Receive the query results fed back by each child node, and perform an intersection operation on the query results to obtain the target query result.
[0039] In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:
[0040] Get RDF datasets and SPARQL query statements;
[0041] According to the constructed consistent hash ring, the RDF data set is divided into corresponding child nodes. The consistent hash ring includes the mapping relationship between the hash value of each RDF data subject value and the virtual node position of each child node;
[0042] According to the mapping relationship, the sub-query statements in the SPARQL query statement are sent to the corresponding child nodes according to the query data processing priority;
[0043] Receive the query results fed back by each child node, and perform an intersection operation on the query results to obtain the target query result.
[0044] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the following steps are implemented:
[0045] Get RDF datasets and SPARQL query statements;
[0046] According to the constructed consistent hash ring, the RDF data set is divided into corresponding child nodes. The consistent hash ring includes the mapping relationship between the hash value of each RDF data subject value and the virtual node position of each child node;
[0047] According to the mapping relationship, the sub-query statements in the SPARQL query statement are sent to the corresponding child nodes according to the query data processing priority;
[0048] Receive the query results fed back by each child node, and perform an intersection operation on the query results to obtain the target query result.
[0049] The above-mentioned distributed RDF data query method, apparatus, computer equipment, storage medium and computer program product divide the RDF data set into corresponding child nodes according to the consistent hash ring including the mapping relationship between the hash value of each RDF data subject value and the virtual node position of each child node, and can divide the RDF data set with the same subject value into the same node, so that the data division of each node is more uniform, which can effectively solve the problems of excessive data partitioning overhead and uneven load. Furthermore, according to the mapping relationship, the sub-query statements in the SPARQL query statement are sent to the corresponding child nodes according to the query data processing priority, which can optimize the SPARQL query and distribute the query data to each node for distributed execution, which effectively improves the speed of data query. In summary, the adoption of this solution can effectively improve the efficiency of data query. BRIEF DESCRIPTION OF THE DRAWINGS
[0050] Figure 1 An application environment diagram of a distributed RDF data query method in one embodiment;
[0051] Figure 2 A schematic diagram of a process of a distributed RDF data query method in one embodiment;
[0052] Figure 3 is a flow chart of a distributed RDF data query method in another embodiment;
[0053] Figure 4 A data flow diagram of a distributed RDF data query method in an embodiment;
[0054] Figure 5 is a structural block diagram of a distributed RDF data query device in one embodiment;
[0055] Figure 6 is a structural block diagram of a distributed RDF data query device in one embodiment;
[0056] Figure 7 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION
[0057] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0058] The distributed RDF data query method provided in the embodiment of the present application can be applied to Figure 1In the application environment shown, the master node 102 communicates with the child node (also referred to as the working node) 104 through the network. The data storage system can store the data that the master node 102 needs to process. The data storage system can be integrated on the master node 102, or it can be placed on the cloud or other network servers. Specifically, the staff may input the RDF data set and the SPARQL query statement into the main node 102, and send a data processing instruction to the main node 102. The main node 102 responds to the instruction, obtains the RDF data set and the SPARQL query statement, and divides the RDF data set into corresponding child nodes according to the constructed consistent hash ring, wherein the consistent hash ring includes a mapping relationship between the hash value of each RDF data subject value and the virtual node position of each child node. Then, according to the mapping relationship, the sub-query statements in the SPARQL query statement are sent to the corresponding child nodes according to the query data processing priority, and the query results fed back by each child node are received. The query results are obtained by each child node by querying the divided RDF data according to the received sub-query statements. Among them, the main node 102 and the child node 104 can be, but are not limited to, various personal computers, laptops, smart phones, tablet computers, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc.
[0059] In one embodiment, if Figure 2 As shown, a distributed RDF data query method is provided, which is applied to Figure 1 Taking the master node 102 in the example as an example, the following steps are included:
[0060] Step 202: Obtain an RDF dataset and a SPARQL query statement.
[0061] An RDF dataset is a set of multiple RDF triples, each of which includes a subject value, an attribute value, and an object value. A SPARQL query statement is a data set including multiple sub-SPARQL query statements, wherein each SPARQL query statement also exists in the form of a triple.
[0062] like Figure 3 As shown, in one embodiment, before step 204, the method further includes: step 203, encoding the RDF data set to obtain an encoded RDF data set and a bidirectional dictionary.
[0063] In practical applications, due to the large amount of data in the RDF data set and the large data length of a single RDF data, both data storage and data transmission require a lot of overhead. In order to solve this problem, in this embodiment, the edge data of the vertex of each RDF triple can be encoded, and the encoding starts from 0. Each encoding corresponds to one data, and finally the corresponding vertex dictionary and edge dictionary, i.e., a two-way dictionary, and the encoded RDF data set are obtained. Among them, it can be for the attribute value of the triple. A two-way dictionary is constructed separately, and a two-way dictionary is constructed for the subject value and object value of the triple. In this embodiment, by encoding the RDF data set, long strings can be encoded into numbers, which can effectively reduce storage overhead.
[0064] Step 204, divide the RDF data set into corresponding child nodes according to the constructed consistent hash ring, and the consistent hash ring includes a mapping relationship between the hash value of each RDF data subject value and the virtual node position of each child node.
[0065] The consistent hash ring contains a mapping relationship between the hash value of each RDF data (RDF triple) subject value and the position of the virtual node assigned to each child node, wherein the hash value of the subject value of each RDF triple corresponds to the position of a virtual node. Specifically, the consistent hash ring constructed can be used to divide the RDF data set using a consistent hash algorithm, and the RDF data in the RDF data set can be divided into several child nodes. Specifically, each RDF data can be sent to the corresponding child node through MPI (Multi Point Interface).
[0066] In another embodiment, step 204 includes: step 224, dividing the encoded RDF data set into corresponding child nodes according to the constructed consistent hash ring.
[0067] In specific implementation, the encoded RDF data set may be divided into corresponding child nodes according to the constructed consistent hash ring. Similarly, each encoded RDF data may be sent to the corresponding child node through MPI. In this embodiment, sending the encoded RDF data to the corresponding child node can significantly reduce the data transmission overhead.
[0068] like Figure 3 As shown, in one embodiment, before step 206, the method further includes: step 205, encoding the SPARQL query statement according to the bidirectional dictionary to obtain an encoded SPARQL query statement.
[0069] In a specific implementation, the character string contained in the SPARQL query statement can find the corresponding character string and the corresponding code of the character string in the bidirectional dictionary. Therefore, in order to further save the data transmission overhead of the SPARQL query statement, the SPARQL query statement can be encoded according to the bidirectional dictionary to obtain the encoded SPARQL query statement. Specifically, the subject value, attribute value and object value in the SPARQL query statement can be traversed, and then the bidirectional dictionary is searched to see whether there is data identical to the subject value, attribute value and object value in the SPARQL query statement. If so, the subject value, attribute value and / or object value in the SPARQL query statement is encoded into the corresponding digital code according to the correspondence between the character string and the digital code in the bidirectional dictionary. In this way, the lengthy SPARQL query statement can be converted into a short query statement, greatly reducing the data transmission overhead.
[0070] Step 206: According to the mapping relationship, the sub-query statements in the SPARQL query statement are sent to the corresponding child nodes respectively according to the query data processing priority.
[0071] In this embodiment, the query data processing priority may be to execute the subquery statement with known subject value first, and then execute the subquery statement with known object value. If both the subject and the object are unknown, find the subject variable with the highest frequency in the subquery statement, and then execute the subquery with this subject variable as the object, and finally execute the remaining subquery statements. Specifically, the subquery statements of the SPARQL query statement may be sent to the corresponding child nodes respectively according to the mapping relationship between the hash value and the virtual node position, so that the child node executes the subquery statement and feeds back the corresponding query result.
[0072] In another embodiment, step 206 includes: step 226, according to the mapping relationship, sending the sub-query statements in the encoded SPARQL query statement to the corresponding child nodes according to the query data processing priority.
[0073] In this embodiment, similarly, the sub-query statements of the encoded SPARQL query statement can be sent to corresponding child nodes respectively according to the mapping relationship between the hash value and the virtual node position, so that the child node executes the sub-query statement and feeds back the corresponding query result, which can greatly save the data transmission overhead.
[0074] Step 208: receiving the query results fed back by each child node, and performing an intersection operation on the query results to obtain a target query result.
[0075] In actual applications, after receiving the corresponding sub-query statement, the child node executes the query task, performs a query in the RDF data obtained by the division according to the received sub-query statement, obtains the corresponding query result, and then feeds the query result back to the main node. Since the number of query results fed back by the child node is large and complex, and there may be repeated query results, in this embodiment, after receiving the query result fed back by the child node, each sub-query statement is used as a reference to perform a connection operation on the corresponding query result to obtain the target query result of the sub-query statement, and then the target query results of each sub-query statement are aggregated to obtain the final target query result.
[0076] In the above-mentioned distributed RDF data query method, the RDF data set is divided into corresponding child nodes according to the consistent hash ring including the mapping relationship between the hash value of each RDF data subject value and the virtual node position of each child node, and the RDF data with the same subject value can be divided into the same node, so that the data division of each node is more uniform, which can effectively solve the problems of excessive data partitioning overhead and uneven load. Furthermore, according to the mapping relationship, the sub-query statements in the SPARQL query statement are sent to the corresponding child nodes according to the query data processing priority, so that the SPARQL query can be optimized, and the query data can be distributed to each node for distributed execution, which effectively improves the speed of data query. In summary, the adoption of this scheme can effectively improve the efficiency of data query.
[0077] In one embodiment, step 204 includes: performing hash calculation on the subject value of each RDF data in the RDF data set to obtain the hash value of the subject value, searching for the virtual node position corresponding to the hash value of the subject value on the consistent hash ring, and sending each RDF data to the child node corresponding to the virtual node position.
[0078] In specific implementation, the subject value of each RDF triple in the RDF data set can be hashed to obtain the hash value of the subject value, and then the virtual node position corresponding to the hash value of the subject value is searched clockwise on the consistent hash ring, and then each RDF data is sent to the child node corresponding to the virtual node position. In this way, the triples with the same subject value can be divided into the same child node, so that the RDF data is evenly distributed to each child node, solving the problem of unbalanced node load. It can be understood that in other embodiments, the virtual node position can also be searched counterclockwise, which can be determined according to the actual situation and is not limited here.
[0079] In one of the embodiments, the consistent hash ring is constructed based on the following method: a preset number of virtual nodes are allocated to each child node, and a hash algorithm is used to obtain the position of each virtual node to obtain a consistent hash ring.
[0080] In practical applications, building a consistent hash ring can be to assign 100 virtual nodes to each physical subnode, and then use the crc32 algorithm to calculate the position of each virtual node to build a consistent hash ring. Specifically, the node name of the virtual node can be used as a keyword to hash and obtain the position of each virtual node on the hash ring. It is understandable that in other embodiments, 200 virtual nodes or other numbers of virtual nodes can be assigned to each physical subnode. This embodiment is only an illustrative example and is not limited here.
[0081] In one of the embodiments, according to the mapping relationship, sending the sub-query statements in the SPARQL query statement to the corresponding child nodes according to the query data processing priority includes: sorting the SPARQL query statements according to the preset query data processing priority to obtain sorted SPARQL query statements, and sending the sorted SPARQL query statements to the corresponding child nodes according to the mapping relationship.
[0082] In specific implementation, the SPARQL query statements may be sorted and optimized first according to the preset query data processing priority, and then the sorted and optimized SPARQL query statements may be sent to the corresponding child nodes according to the mapping relationship. Specifically, the sorted and optimized SPARQL query statements may be sent to the corresponding child nodes as follows:
[0083] Step (1-1) First, select a subquery statement with a known subject value (hereinafter referred to as subquery), hash the subject value of the subquery statement to obtain a hash value, and then find the position of the virtual node corresponding to the hash value according to the mapping relationship contained in the consistent hash ring, and then send the subquery statement to the child node corresponding to the position of the found virtual node for execution, without sending it to all nodes for execution, thereby reducing communication overhead;
[0084] Step (1-2) Secondly, select a subquery with a known object value, and send such a subquery to all nodes for execution. At this time, the value of the subject variable of the subquery fed back by each child node (i.e., the subject value) will be obtained. After obtaining the subject value, return to step (1-1);
[0085] In step (1-3), if both the subject value and the object value are unknown, find the subject variable that appears most frequently in the subquery, and then send the subquery with this subject variable as the object to all child nodes. At this time, the subject value of this type of subquery will be obtained. After obtaining the value of the subject variable, return to step (1-1);
[0086] In step (1-4), after being screened by steps (1-1) and (1-3), the remaining subqueries are sent to all nodes for query.
[0087] In this embodiment, the sorted and optimized SPARQL query statements are sent to corresponding child nodes in the above manner, which is different from the prior art of sending the query statements to all nodes, thereby reducing communication overhead and improving query speed.
[0088] In order to clearly illustrate the distributed RDF data query method provided by the present application, the following is a specific embodiment and Figure 4 To explain:
[0089] The master node responds to the data processing instruction, obtains the RDF data set and SPARQL query statement, encodes the edge data of the vertex of each RDF triple, and the encoding starts from 0. Each encoding corresponds to one data. Finally, the corresponding vertex dictionary and edge dictionary, i.e., the bidirectional dictionary, and the encoded RDF data set are obtained. According to the constructed consistent hash ring, the encoded RDF data set is divided into corresponding child nodes. Among them, the consistent hash ring includes the mapping relationship between the hash value of each RDF data subject value and the virtual node position of each child node. The construction of the consistent hash ring can be to allocate 100 virtual nodes to each physical child node, and then use the crc32 algorithm to calculate the position of each virtual node to obtain the consistent hash ring. Then, according to the bidirectional dictionary, the SPARQL query statement is encoded and processed to obtain the encoded SPARQL query statement. According to the query data processing priority, the sub-query statements in the encoded SPARQL query statement are sorted, and then the sorted SPARQL query statements are sent to the corresponding child nodes according to the mapping relationship, so that the child nodes can execute the query task, query the divided RDF data according to the received sub-query statement, obtain the corresponding query result, and then feed the query result back to the main node.
[0090] Specifically, it can be:
[0091] Step (1-1) First, select a subquery statement with a known subject value (hereinafter referred to as subquery), hash the subject value of the subquery statement to obtain a hash value, and then find the position of the virtual node corresponding to the hash value according to the mapping relationship contained in the consistent hash ring, and then send the subquery statement to the child node corresponding to the position of the found virtual node for execution, without sending it to all nodes for execution, thereby reducing communication overhead;
[0092] Step (1-2) Secondly, select a subquery with a known object value, and send such a subquery to all nodes for execution. At this time, the value of the subject variable of the subquery fed back by each child node (i.e., the subject value) will be obtained. After obtaining the subject value, return to step (1-1);
[0093] In step (1-3), if both the subject value and the object value are unknown, find the subject variable that appears most frequently in the subquery, and then send the subquery with this subject variable as the object to all child nodes. At this time, the subject value of this type of subquery will be obtained. After obtaining the value of the subject variable, return to step (1-1);
[0094] In step (1-4), after being screened by steps (1-1) and (1-3), the remaining subqueries are sent to all nodes for query.
[0095] The main node receives the query results fed back by the child nodes. Since the query results fed back by the child nodes are large in number and relatively complex, and there may be repeated query results, in this embodiment, after receiving the query results fed back by the child nodes, each sub-query statement is used as a benchmark to perform a connection operation on the corresponding query results to obtain the target query result of the sub-query statement, and then, the target query results of each sub-query statement are aggregated to obtain the final target query result.
[0096] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.
[0097] Based on the same inventive concept, the embodiment of the present application also provides a distributed RDF data query device for implementing the above-mentioned distributed RDF data query method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above-mentioned method, so the specific limitations in the one or more distributed RDF data query device embodiments provided below can refer to the above-mentioned limitations on the distributed RDF data query method, and will not be repeated here.
[0098] In one embodiment, Figure 5 As shown, a distributed RDF data query device is provided, including: a data acquisition module 510, a data partitioning module 520, a query statement sending module 530 and a query result processing module 540, wherein:
[0099] A data acquisition module 510 is used to acquire an RDF data set and a SPARQL query statement;
[0100] A data partitioning module 520 is used to partition the RDF data set into corresponding child nodes according to the constructed consistent hash ring, where the consistent hash ring includes a mapping relationship between the hash value of each RDF data subject value and the virtual node position of each child node;
[0101] The query statement sending module 530 is used to send the sub-query statements in the SPARQL query statement to the corresponding child nodes according to the query data processing priority according to the mapping relationship;
[0102] The query result processing module 540 is used to receive the query results fed back by each child node and perform an intersection operation on the query results to obtain a target query result.
[0103] The above-mentioned distributed RDF data query device divides the RDF data set into corresponding child nodes according to the consistent hash ring including the mapping relationship between the hash value of each RDF data subject value and the virtual node position of each child node, and can divide the RDF data with the same subject value into the same node, so that the data division of each node is more uniform, which can effectively solve the problems of excessive data partitioning overhead and uneven load. Furthermore, according to the mapping relationship, the sub-query statements in the SPARQL query statement are sent to the corresponding child nodes according to the query data processing priority, which can optimize the SPARQL query and distribute the query data to each node for distributed execution, which effectively improves the speed of data query. In summary, the use of this device can effectively improve the efficiency of data query.
[0104] like Figure 6 As shown, in one embodiment, the device also includes an RDF data encoding processing module 550, which is used to encode the RDF data set to obtain an encoded RDF data set and a bidirectional dictionary; the data partitioning module 520 is also used to partition the encoded RDF data set into corresponding child nodes according to the constructed consistent hash ring.
[0105] like Figure 6 As shown, in one embodiment, the device also includes a query data encoding processing module 560, which encodes the SPARQL query statement according to the bidirectional dictionary to obtain an encoded SPARQL query statement; the data sending module 530 is also used to send the sub-query statements in the encoded SPARQL query statement to the corresponding child nodes according to the query data processing priority according to the mapping relationship.
[0106] In one of the embodiments, the data partitioning module 520 is also used to perform hash calculation on the subject value of each RDF data in the RDF data set to obtain the hash value of the subject value, find the virtual node position corresponding to the hash value of the subject value on the consistent hash ring, and send each RDF data to the child node corresponding to the virtual node position.
[0107] In one embodiment, the data sending module 530 is further used to sort the SPARQL query statements according to a preset query data processing priority to obtain sorted SPARQL query statements, and send the sorted SPARQL query statements to corresponding child nodes according to the mapping relationship.
[0108] In one of the embodiments, the device further includes a hash ring construction module 570 for allocating a preset number of virtual nodes to each child node, using a hash algorithm to obtain the position of each virtual node, and obtaining a consistent hash ring.
[0109] Each module in the above-mentioned distributed RDF data query device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each module above.
[0110] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 7 As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data such as RDF data sets, SPARQL query statements and consistent hash rings. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a distributed RDF data query method is implemented.
[0111] Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0112] In one embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above-mentioned distributed RDF data query method when executing the computer program.
[0113] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned distributed RDF data query method are implemented.
[0114] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above-mentioned distributed RDF data query method when executed by a processor.
[0115] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.
[0116] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.
[0117] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0118] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.
Claims
1. A distributed RDF data query method, characterized in that: The method comprises: Get RDF datasets and SPARQL query statements; Divide the RDF data set into corresponding child nodes according to the constructed consistent hash ring, wherein the consistent hash ring includes a mapping relationship between the hash value of each RDF data subject value and the virtual node position of each child node; According to the mapping relationship, the sub-query statements in the SPARQL query statement are sent to the corresponding child nodes respectively according to the query data processing priority; Receive the query results fed back by each child node, and perform an intersection operation on the query results to obtain the target query result.
2. The distributed RDF data query method according to claim 1, characterized in that: Before dividing the RDF data set into corresponding child nodes according to the constructed consistent hash ring, the method further includes: Encoding the RDF data set to obtain an encoded RDF data set and a bidirectional dictionary; The step of dividing the RDF data set into corresponding child nodes according to the constructed consistent hash ring includes: According to the constructed consistent hash ring, the encoded RDF dataset is divided into corresponding child nodes.
3. The distributed RDF data query method according to claim 2, characterized in that: Before sending the sub-query statements in the SPARQL query statement to the corresponding child nodes respectively according to the mapping relationship and the preset query data processing priority, the method further includes: According to the bidirectional dictionary, the SPARQL query statement is encoded to obtain an encoded SPARQL query statement; The step of sending the sub-query statements in the SPARQL query statement to corresponding child nodes according to the query data processing priority according to the mapping relationship comprises: According to the mapping relationship, the sub-query statements in the encoded SPARQL query statement are sent to corresponding child nodes respectively according to the query data processing priority.
4. The distributed RDF data query method according to any one of claims 1 to 3, characterized in that: According to the constructed consistent hash ring, dividing the RDF data set into corresponding child nodes includes: Performing hash calculation on the subject value of each RDF data in the RDF data set to obtain a hash value of the subject value; Finding a virtual node position on the consistent hash ring that corresponds to the hash value of the subject value; Each RDF data is sent to a child node corresponding to the position of the virtual node.
5. The distributed RDF data query method according to any one of claims 1 to 3, characterized in that: According to the mapping relationship, sending the sub-query statements in the SPARQL query statement to the corresponding child nodes respectively according to the query data processing priority includes: Sorting the SPARQL query statements according to the preset query data processing priority to obtain sorted SPARQL query statements; According to the mapping relationship, the sorted SPARQL query statements are sent to corresponding child nodes in sequence.
6. The distributed RDF data query method according to any one of claims 1 to 3, characterized in that: The consistent hashing ring is constructed in the following way: Allocate a preset number of virtual nodes to each child node; Use the hash algorithm to obtain the location of each virtual node and obtain a consistent hash ring.
7. A distributed RDF data query device, characterized in that: The device comprises: Data acquisition module, used to obtain RDF data sets and SPARQL query statements; A data partitioning module, used to partition the RDF data set into corresponding child nodes according to a constructed consistent hash ring, wherein the consistent hash ring includes a mapping relationship between a hash value of each RDF data subject value and a virtual node position of each child node; A query statement sending module, used for sending sub-query statements in the SPARQL query statement to corresponding child nodes according to the query data processing priority according to the mapping relationship; The query result processing module is used to receive the query results fed back by each sub-node and perform an intersection operation on the query results to obtain a target query result.
8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.
10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.
