RDF (Resource Description Framework) data query optimization method and system for distributed system

Abstract

The invention discloses an RDF (Resource Description Framework) data query optimization method and system for a distributed system, a storage medium and computer equipment, the method comprises the following steps: in response to a query instruction input by a user, analyzing the query instruction, and according to an analysis result and a data storage condition corresponding to each slave node combination in the distributed system, determining a data storage condition corresponding to each slave node combination in the distributed system; all sub-query instructions corresponding to the query instruction are generated, the query instruction is generated based on SPARQL, and each slave node combination comprises an RDF database and a relational database; sending each sub-query instruction to a corresponding slave node combination, so that the slave node combination obtains a resource relationship query result from the RDF database and / or obtains a resource attribute query result from a relationship database based on the sub-query instruction; and receiving a query result returned by each slave node combination, and integrating the query results to obtain a final query result corresponding to the query instruction.

Description

Technical Field

[0001] The present application relates to the field of database technology, and in particular to an RDF data query optimization method and system, storage medium, and computer equipment for distributed systems. Background Art

[0002] With the rapid development of information technology, especially the rise of the Internet and big data technology, the storage, management and query requirements of data are becoming increasingly complex. As an efficient data processing architecture, distributed systems are widely used in the field of large-scale data processing because they can use multiple computers or servers to complete tasks together, realize efficient storage and fast access to data. In distributed systems, data is usually stored in multiple nodes in a dispersed manner to improve the scalability and fault tolerance of the system. Among the data processed by distributed systems, RDF (Resource Description Framework) data has been widely used in knowledge graphs, semantic webs and other fields due to its unique semantic expression ability and flexibility. RDF data represents resources and their relationships in the form of triples (subject-predicate-object), which is very suitable for describing complex data structures. However, with the rapid growth of RDF data, how to efficiently query this data in distributed systems has become an urgent problem to be solved.

[0003] Traditional RDF data query methods, such as those based on SPARQL (SPARQL Protocol and RDF Query Language), can provide powerful query functions, but in a distributed system environment, these methods often face challenges such as low query efficiency and high resource consumption. This is because RDF data is usually stored in an RDF database or a relational database. In order to implement SPARQL queries on RDF data, a connection method is usually used. Specifically, this method involves parsing the SPARQL query and distributing it to each node. Each node performs a subgraph matching calculation, and then merges the matching results of each node. Since a large amount of data is stored in different data tables, this method usually requires a large number of data table connection queries when performing SPARQL queries. When the amount of data is large and there are many machine nodes, the connection cost is high and the query delay is significantly increased. At the same time, the connection operation of the data table causes a large amount of data exchange between system nodes, which increases the complexity of data query. Summary of the invention

[0004] In view of this, the present application provides an RDF data query optimization method and system, storage medium, and computer device for distributed systems, by storing resource relationship data in the RDF database in each slave node combination, and storing resource attribute data in the relational database, when performing subsequent data queries, first use the RDF database to query the relationship between resources, and then retrieve the corresponding attributes of the resources from the relational data table based on these relationships. In this way, the mixed query involving resource attributes and resource relationships in the SPARQL query is converted into a nested filtering SPARQL query for optimization. This method greatly improves the query speed by rewriting the multiple connection queries of SPARQL into the intersection operation of the multi-field result set in the relational database, and the performance is improved several times, which reduces the complexity of data query.

[0005] According to one aspect of the present application, a method for optimizing RDF data queries for a distributed system is provided, which is applied to a master node in the distributed system. The method includes:

[0006] In response to a query instruction input by a user, the query instruction is parsed, and all sub-query instructions corresponding to the query instruction are generated according to the parsing result and the data storage situation corresponding to each slave node combination in the distributed system, wherein the query instruction is generated based on SPARQL, and each slave node combination includes an RDF database and a relational database;

[0007] Sending each sub-query instruction to a corresponding slave node combination, so that the slave node combination obtains a resource relationship query result from the RDF database and / or obtains a resource attribute query result from the relational database based on the sub-query instruction;

[0008] Receive the query results returned by each slave node combination, and integrate the query results to obtain the final query result corresponding to the query instruction.

[0009] According to another aspect of the present application, there is provided an RDF data query optimization system for a distributed system, comprising:

[0010] The master node is used to respond to the query instruction input by the user, parse the query instruction, generate all sub-query instructions corresponding to the query instruction according to the parsing result and the data storage situation corresponding to each slave node combination in the distributed system, and send each sub-query instruction to the corresponding slave node combination, wherein the query instruction is generated based on SPARQL, and each slave node combination includes an RDF database and a relational database;

[0011] A slave node combination, for obtaining a resource relationship query result from the RDF database and / or a resource attribute query result from the relational database based on the sub-query instruction;

[0012] The master node is further used to receive the query results returned by each slave node combination, and integrate the query results to obtain the final query results corresponding to the query instruction.

[0013] According to another aspect of the present application, a storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the above-mentioned RDF data query optimization method for a distributed system is implemented.

[0014] According to another aspect of the present application, a computer device is provided, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor implements the above-mentioned RDF data query optimization method for a distributed system when executing the program.

[0015] By means of the above technical scheme, the present application provides an RDF data query optimization method and system, storage medium, and computer device for a distributed system. After receiving the query instruction input by the user, the master node first parses it. While parsing the query instruction, the master node can also understand the data storage situation of each slave node combination in the distributed system, and then can guide the master node to assign different query conditions of the user to different slave node combinations. Then, the master node can generate all sub-query instructions corresponding to the query instruction based on the parsing result of the query instruction and the data storage situation of each slave node combination. After that, the master node can send each generated sub-query instruction to the corresponding slave node combination. The slave node combination can obtain the resource relationship query result from the RDF database and / or obtain the resource attribute query result from the relational database according to the sub-query instruction. After each slave node combination executes the sub-query instruction, the query result can be returned to the master node. After the master node receives the query result returned by each slave node combination, the received query result can be integrated. After integration, the master node returns the final query result to the user. The embodiment of the present application stores resource relationship data in the RDF database in each slave node combination, and stores resource attribute data in the relational database. When performing subsequent data queries, the RDF database is first used to query the relationship between resources, and then the corresponding attributes of the resources are retrieved from the relational data table based on these relationships. In this way, the mixed query involving resource attributes and resource relationships in the SPARQL query is converted into a nested filtering SPARQL query for optimization. This method greatly improves the query speed and performance by rewriting the multiple connection queries of SPARQL into the intersection operation of the multi-field result set in the relational database, which reduces the complexity of data query.

[0016] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0018] Figure 1 A schematic diagram of a flow chart of an RDF data query optimization method for a distributed system provided in an embodiment of the present application is shown;

[0019] Figure 2 A schematic diagram of the structure of an RDF data query optimization system for a distributed system provided by an embodiment of the present application is shown;

[0020] Figure 3 A schematic diagram of the device structure of a computer device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0021] The present application will be described in detail below with reference to the accompanying drawings and in combination with embodiments. It should be noted that the embodiments and features in the embodiments of the present application can be combined with each other without conflict.

[0022] In this embodiment, a method for optimizing RDF data query for a distributed system is provided. Figure 1 As shown, the method includes:

[0023] Step 101, in response to a query instruction input by a user, the query instruction is parsed, and all sub-query instructions corresponding to the query instruction are generated according to the parsing result and the data storage situation corresponding to each slave node combination in the distributed system, wherein the query instruction is generated based on SPARQL, and each slave node combination includes an RDF database and a relational database.

[0024] An RDF data query optimization method for a distributed system provided in an embodiment of the present application can greatly reduce the number of connection queries in the RDF data query process and improve data query efficiency. In this embodiment, the distributed system may include a master node and a plurality of slave node combinations, each of which may include an RDF database node and a relational database node. The RDF database is used to store all resource relationship data, and the relational database is used to store a set of resource attribute data. Among them, the resource relationship data is used to represent the association relationship between resources, usually stored in the form of triples, and the triples include a subject, a predicate and an object. The subject and the object can be two specific resources, and the predicate can be used to represent the association relationship between the two specific resources; the resource attribute data is used to represent the specific attribute values ​​of each resource. It should be noted that in the embodiment of the present application, the resource relationship data stored in the RDF database in different slave node combinations are the same, and the resource attribute data stored in the relational database in different slave node combinations are different.

[0025] The user can input a SPARQL-based query instruction through a certain interface on the client (such as a Web interface, a command line interface, etc.). The query instruction describes the specific data that the user wants to query. After that, after the master node receives the query instruction input by the user, it first parses it. The parsing process includes identifying each query condition in the query instruction and understanding the logical relationship between them. While parsing the query instruction, the master node can also understand the data storage situation of each slave node combination in the distributed system, and then guide the master node to assign different query conditions of the user to different slave node combinations. Then, the master node can generate all sub-query instructions corresponding to the query instruction based on the parsing results of the query instruction and the data storage situation of each slave node combination. These sub-query instructions can be executed on a specific slave node combination to obtain the required resource relationship query results and resource attribute query results. It should be noted that each sub-query instruction corresponds to a slave node combination, which is used to obtain query results from the corresponding slave node combination. For example, the slave node combinations include combination A, combination B, combination C, combination D, and combination E. After parsing the query instructions input by the user, it is found that the data the user wants to query are stored on slave node combination A and slave node combination C, respectively. At this time, based on the parsing results, a sub-query instruction corresponding to slave node combination A and a sub-query instruction corresponding to slave node combination C can be generated.

[0026] Step 102: Send each sub-query instruction to a corresponding slave node combination, so that the slave node combination obtains resource relationship query results from the RDF database and / or obtains resource attribute query results from the relational database based on the sub-query instruction.

[0027] In this embodiment, the master node can then send each generated sub-query instruction to the corresponding slave node combination. These slave node combinations are selected to execute specific sub-query instructions based on the data content stored in them. The slave node combination can obtain resource relationship query results from the RDF database and / or resource attribute query results from the relational database according to the sub-query instructions. Here, if the sub-query instruction only indicates to query resource relationship data, then the resource relationship query results can be obtained from the RDF database; if the sub-query instruction only indicates to query resource attribute data, then the resource attribute query results can be obtained from the relational database; if it indicates to query both resource relationship data and resource attribute data, then the resource relationship query results can be obtained from the RDF database, and the resource attribute query results can be obtained from the relational database.

[0028] Step 103, receiving the query results returned by each slave node combination, and integrating the query results to obtain a final query result corresponding to the query instruction.

[0029] In this embodiment, after each slave node combination executes the sub-query instruction, the query result can be returned to the master node. Here, the query result can include resource relationship query results and resource attribute query results. After the master node receives the query results returned by each slave node combination, it can integrate the received query results. The integration process may include operations such as deduplication, sorting, and merging of the query results, which are not limited here, to ensure the accuracy and completeness of the final query results. After integration, the master node returns the final query result to the user, and the final query result contains all the information requested by the user through the SPARQL query instruction.

[0030] By applying the technical solution of this embodiment, after the master node receives the query instruction input by the user, it first parses it. While parsing the query instruction, the master node can also understand the data storage situation of each slave node combination in the distributed system, and then can guide the master node to assign different query conditions of the user to different slave node combinations. Then, the master node can generate all sub-query instructions corresponding to the query instruction based on the parsing result of the query instruction and the data storage situation of each slave node combination. After that, the master node can send each generated sub-query instruction to the corresponding slave node combination. The slave node combination can obtain the resource relationship query result from the RDF database and / or obtain the resource attribute query result from the relational database according to the sub-query instruction. After each slave node combination executes the sub-query instruction, the query result can be returned to the master node. After the master node receives the query result returned by each slave node combination, the received query result can be integrated. After integration, the master node returns the final query result to the user. The embodiment of the present application stores resource relationship data in the RDF database in each slave node combination, and stores resource attribute data in the relational database. When performing subsequent data queries, the RDF database is first used to query the relationship between resources, and then the corresponding attributes of the resources are retrieved from the relational data table based on these relationships. In this way, the mixed query involving resource attributes and resource relationships in the SPARQL query is converted into a nested filtering SPARQL query for optimization. This method greatly improves the query speed and performance by rewriting the multiple connection queries of SPARQL into the intersection operation of the multi-field result set in the relational database, which reduces the complexity of data query.

[0031] In an embodiment of the present application, optionally, the RDF database in the slave node combination receives the sub-query instruction, parses the sub-query instruction, and determines whether the sub-query instruction is a query instruction of a hybrid query mode, wherein the hybrid query mode is a query mode that includes both resource attribute query and resource relationship query; when the sub-query instruction is a query instruction of the hybrid query mode, the RDF database generates a SPARQL query statement and an SQL query statement according to the sub-query instruction, obtains a resource relationship query result from the RDF database according to the SPARQL query statement, and sends the resource relationship query result and the SQL query statement to the corresponding relational database, so that the relational database obtains the resource attribute query result from the relational database based on the resource relationship query result and the SQL query statement, and returns the resource attribute query result to the RDF database. query result; the RDF database returns the resource relationship query result and the resource attribute query result to the master node; when the sub-query instruction is a query instruction for resource attribute query, the RDF database generates an SQL query statement according to the sub-query instruction, and sends the SQL query statement to the corresponding relational database, so that the relational database obtains the resource attribute query result from the relational database based on the SQL query statement, and returns the resource attribute query result to the RDF database; the RDF database returns the resource attribute query result to the master node; when the sub-query instruction is a query instruction for resource relationship query, the RDF database generates a SPARQL query statement according to the sub-query instruction, obtains the resource relationship query result from the RDF database according to the SPARQL query statement, and returns the resource relationship query result to the master node.

[0032] In this embodiment, the RDF database in the slave node combination is used to receive the sub-query instruction sent by the master node. After receiving the sub-query instruction, the RDF database parses the sub-query instruction to determine whether it is a query instruction of a mixed query mode. The mixed query mode refers to a query mode that involves both resource attribute query (such as the specific attribute value of a resource) and resource relationship query (such as the association relationship between resources). In one embodiment, the process of parsing the sub-query instruction can be implemented by regular expression matching or SPARQL parser. If the sub-query instruction contains query statements for querying resource attributes and querying resource relationships at the same time, the sub-query instruction is determined to be a sub-query instruction of the mixed query mode.

[0033] If it is found through analysis that the sub-query instruction is a query instruction of a hybrid query mode, then the corresponding SPARQL query statement and SQL query statement can be generated respectively according to the query instruction of the hybrid query mode. Among them, the SPARQL query statement is used to process the resource relationship query part, and the SQL query statement is used to process the resource attribute query part. Specifically, a SPARQL query generator can be used to generate a SPARQL query statement according to the resource relationship query part in the sub-query instruction; according to the resource attribute query part in the sub-query instruction, combined with the table structure of the relational database, a corresponding SQL query statement is generated. For example, if the sub-query instruction is to query all attribute values ​​of a resource and other resources associated with it, the generated SPARQL query statement is used to query all triples related to the resource, and the generated SQL query statement is used to query the specific attribute values ​​of the resource in the relational database. Further, the RDF database can use the SPARQL query engine to execute the SPARQL query statement, determine the resource relationship query result from the RDF database, and then send the resource relationship query result and the SQL statement to the relational database corresponding to the RDF database. Next, the relational database searches for resource attribute query results in itself based on the resource relationship query results and SQL statements, and returns the resource attribute query results to the RDF database. After receiving the resource attribute query results, the RDF database returns the resource attribute query results and resource relationship query results to the master node.

[0034] If it is found through analysis that the sub-query instruction is a query instruction for resource attribute query, then the RDF database can generate a corresponding SQL query statement based on the sub-query instruction. Specifically, the corresponding SQL query statement can be generated based on the resource attribute query part in the sub-query instruction and the table structure of the relational database. Further, the RDF database can send the SQL statement to the relational database corresponding to the RDF database. Then, based on the SQL statement, the relational database searches for the resource attribute query result in itself and returns the resource attribute query result to the RDF database. After receiving the resource attribute query result, the RDF database returns the resource attribute query result to the master node.

[0035] If the sub-query instruction is found to be a query instruction for resource relationship query after analysis, then the RDF database can generate a corresponding SPARQL query statement according to the sub-query instruction. Specifically, a SPARQL query generator can be used to generate a SPARQL query statement according to the resource relationship query part in the sub-query instruction. Further, the RDF database can use a SPARQL query engine to execute the SPARQL query statement and determine the resource relationship query result from the RDF database. Finally, the RDF database returns the resource relationship query result to the master node.

[0036] The embodiment of the present application optimizes the mixed query involving resource attributes and resource relationships in the SPARQL query by converting it into a nested filtering SPARQL query. This method rewrites the multiple connection query of SPARQL into the intersection operation of the multi-field result set in the relational database, greatly improving the query speed and improving the performance several times. This optimization method first uses the RDF database to query the relationship between resources, and then retrieves the corresponding attributes of the resources from the relational data table based on these relationships. In this way, the powerful grouping and statistical capabilities of the relational database can be used to process the resource attribute query part, which is several times faster than the native grouping and statistical capabilities of SPARQL, and can greatly improve the query efficiency of RDF data. In addition, since the resource relationship data is stored in each RDF database, when the master node executes the nested query, it parses the query instruction and distributes it to each RDF database for resource relationship query. After each RDF database completes the query independently, it performs a nested query with the relational database. Finally, all query results are aggregated to the master node. This design avoids data transmission between nodes and improves query performance.

[0037] In an embodiment of the present application, optionally, before step 101, the method further includes: acquiring original RDF data, parsing the original RDF data, and obtaining resource attribute data and resource relationship data corresponding to the original RDF data; storing the resource relationship data in the RDF database in each slave node combination; and grouping the resource attribute data and storing each group of resource attribute data in the relational databases in different slave node combinations.

[0038] In this embodiment, before receiving the query instruction input by the user, the original RDF data can be parsed to obtain resource attribute data and resource relationship data. Afterwards, the resource relationship data is stored in each RDF database in the slave node combination, that is, each RDF database stores the same resource relationship data. In addition, the resource attribute data can also be grouped to obtain multiple grouping results, and each grouping result is stored in a relational database, that is, the resource attribute data stored in the relational databases of different slave node combinations are different, so as to achieve the "slicing" (that is, sub-table storage) of the resource attribute data. Only then is the corresponding RDF data query optimization method executed. Specifically, the original RDF data can be processed by the following steps. First, the original RDF data can be obtained. Here, the original RDF data can be data in the form of triples, but each resource in the triples also has attribute data. Afterwards, the received original RDF data is parsed and decomposed into resource attribute data and resource relationship data. Resource attribute data describes the specific attributes of resources, such as the name, price, release time, etc. of the commodity; resource relationship data describes the association relationship between resources, such as the association between commodities and categories, the association between people and works, etc.

[0039] Next, the resource attribute data obtained by the analysis can be grouped, and each grouping result can be stored in a relational database. A relational database is a table-based database that can efficiently store and query structured data. In an embodiment of the present application, the resource attribute data is stored in a table in a relational database, each attribute corresponds to a field, and each resource corresponds to a record. For example, for product information, a database table named "Product Table" can be created, which contains fields such as product ID, product name, price, and release time. The product attribute data obtained by the analysis can be filled into this table.

[0040] At the same time, the resource relationship data obtained by parsing can be stored in the RDF database in the form of triples. Triples are the basic units of RDF data, consisting of a subject, a predicate, and an object, and are used to describe the association relationship between resources. RDF database is a database specifically used to store and query RDF data. In an embodiment of the present application, resource relationship data is stored in a triple table of the RDF database, and each triple corresponds to a record. For example, for the association relationship between a commodity and a category, it can be stored as a triple (commodity ID, is, category ID). The resource relationship data obtained by parsing can be inserted into the RDF database in the form of triples.

[0041] The embodiment of the present application can efficiently process the original RDF data, decompose it into resource attribute data and resource relationship data, and store them in the relational database and the RDF database respectively. Since the resource attribute data sub-tables (shards) are stored in the relational database table, the mixed query of resource attributes and resource relationships can be converted into a nested query, thereby reducing the number of connection queries and improving query efficiency.

[0042] In an embodiment of the present application, optionally, after receiving the resource relationship data, the RDF database converts the resource relationship data according to a preset triple form to obtain multiple target triples, and adjusts the positions of the subject, predicate and object in each target triple according to each variant triple form of the preset triple form to obtain a variant triple corresponding to the target triple; the RDF database stores the multiple target triples in an RDF table corresponding to the preset triple form, and stores the variant triples in the RDF tables corresponding to the variant triple forms respectively, and constructs an index of the RDF table according to the triple form corresponding to each RDF table; after the RDF database is started, the index is loaded into the memory, and after receiving the SPARQL query statement, based on the SPARQL query statement, a matching target index is determined from the loaded index, and a query is performed based on the SPARQL query statement and the target index to obtain a resource relationship query result.

[0043] In this embodiment, when storing resource relationship data, after receiving the resource relationship data, the RDF database can convert the received resource relationship data according to a preset triple form (e.g., a subject-predicate-object form), that is, simplify the complex data structure into a triple form that is easy to process and query. Through the conversion, the RDF database can obtain multiple target triples. Each target triple clearly represents a resource relationship. In order to support more flexible queries, the RDF database adjusts the positions of the subject, predicate and object in each target triple according to each preset variant triple form (e.g., subject-object-predicate, object-subject-predicate, etc.). Through the position adjustment, the RDF database generates corresponding variant triples for each target triple. These variant triples provide additional flexibility during query. For example, the target triple is (author JK Rowling-creation-"Harry Potter"), and the variant triple forms include subject-object-predicate, object-subject-predicate, and object-predicate-subject. Then the variant triples corresponding to the target triple can be (author JK Rowling-"Harry Potter")-creation), ("Harry Potter"-author JK Rowling-creation), ("Harry Potter"-creation-author JK Rowling). After that, the RDF database stores the obtained target triple in the RDF table corresponding to the preset triple form. At the same time, the generated variant triples are stored in the RDF tables of the corresponding variant triple forms. For example, the RDF table corresponding to the variant triple form "subject-object-predicate" stores all the variant triples of the subject-object-predicate form, and the RDF table corresponding to the variant triple form "object-subject-predicate" stores all the variant triples of the object-subject-predicate form, and so on. For each RDF table, the RDF database determines the index construction method according to the triple form stored in it (whether it is a target triple or a transformed triple). Specifically, based on the determined triple form, the RDF database can construct an index for each RDF table, such as an SPO (subject-predicate-object) index, a SOP (subject-object-predicate) index, an OSP (object-subject-predicate) index, an OPS (object-predicate-subject) index, a PSO (predicate-subject-object) index, and a POS (predicate-object-subject) index. These indexes help to speed up the query process and improve query efficiency.

[0044] After all resource relationship data is stored in each RDF database, the previously constructed index can be loaded into memory when the RDF database is started. This ensures that the index can be quickly accessed during query, further improving the query speed. After the RDF database receives the SPARQL query statement input by the user, it can determine the matching target index from the loaded index based on the received SPARQL query statement. According to the SPARQL query statement and the determined target index, the RDF database performs a query operation. Specifically, the corresponding RDF table can be located according to the target index, and then the query operation can be quickly performed from the located RDF table to retrieve the resource relationship query results that match the SPARQL query statement, which contains all resource relationship information requested by the user through the SPARQL query statement.

[0045] The embodiment of the present application constructs a target triple and a deformed triple corresponding to the target triple, and constructs indexes for triples of various forms. These indexes arrange the subject, predicate and object in different orders to support various possible query modes. For example, when all instances of a specific relationship need to be queried, the corresponding index can be used directly to quickly locate these instances. In this way, different resource relationship query situations can be flexibly responded to, and the query process can be accelerated through different indexes, including exact match queries, range queries, etc., thereby improving query efficiency. In addition, the resource attribute data and resource relationship data are stored separately, and only the resource relationship data is loaded into the memory. This method greatly reduces the usage of memory, while avoiding the problem of frequent data transmission between nodes caused by data fragmentation, and effectively improves query efficiency.

[0046] It should be noted that the process of constructing target triples and transformed triples when storing resource relationship data in an RDF database in the embodiment of the present application can be first executed in one RDF database, and then the target triples and transformed triples can be copied to the remaining RDF databases.

[0047] In an embodiment of the present application, optionally, before the "converting the resource relationship data according to a preset triple form to obtain multiple target triples", the method also includes: constructing a first dictionary, a second dictionary and a third dictionary, wherein the first dictionary is used to store the mapping relationship between each subject string and an integer unique identifier, the second dictionary is used to store the mapping relationship between each predicate string and an integer unique identifier, and the third dictionary is used to store the mapping relationship between each object string and an integer unique identifier; the "converting the resource relationship data according to a preset triple form to obtain multiple target triples" includes: converting the resource relationship data according to a preset triple form to obtain multiple initial triples; for each initial triple, converting the initial triple into a target triple consisting of three integer unique identifiers according to the first dictionary, the second dictionary and the third dictionary.

[0048] In this embodiment, before starting to process the resource relationship data, three dictionaries are first constructed to help convert the subject, predicate and object in the resource relationship data in the form of strings into unique identifiers in the form of integers. Specifically, a first dictionary is constructed, and the task of the first dictionary is to assign unique integer identifiers to subject strings. The subject strings corresponding to various subject resources can be stored in the first dictionary in advance, and an integer unique identifier is assigned to each subject string and recorded in the first dictionary. For example, if the subject strings are "Alice", "Bob", "Charlie", etc., an integer unique identifier 1 can be assigned to "Alice", an integer unique identifier 2 can be assigned to "Bob", an integer unique identifier 3 can be assigned to "Charlie", and so on. In this way, the subject string can be converted into the corresponding integer unique identifier through the first dictionary later.

[0049] Construct a second dictionary. The task of the second dictionary is to assign unique integer identifiers to predicate strings. Predicate strings corresponding to various predicates can be stored in the second dictionary in advance, and an integer unique identifier is assigned to each predicate string and recorded in the second dictionary. For example, if the predicate strings are "knows", "likes", "hates", etc., an integer unique identifier 101 can be assigned to "knows", an integer unique identifier 102 can be assigned to "likes", and an integer unique identifier 103 can be assigned to "hates". In this way, the predicate string can be converted into the corresponding integer unique identifier through the second dictionary later.

[0050] Construct a third dictionary, the task of which is to assign a unique integer identifier to the object string. The object strings corresponding to various objects can be stored in the third dictionary in advance, and an integer unique identifier is assigned to each object string and recorded in the third dictionary. The object string can be a name, a place, etc., and an integer unique identifier can be assigned to each name and each place. For example, an integer unique identifier 201 is assigned to "New York". In this way, the object string can be converted into a corresponding integer unique identifier through the third dictionary later.

[0051] After constructing the three dictionaries, the resource relationship data can be processed next. First, the resource relationship data can be converted according to the preset triple form (such as subject-predicate-object), so that all the initial triples included in the resource relationship data can be obtained. Assuming that there is a resource relationship data as follows: "Books written by J.K.Rowling", then first convert this resource relationship data according to the preset triple form (such as subject-predicate-object) to obtain an initial triple: "JKRowling-written by-Books". Then, using the three dictionaries constructed previously, replace the string in the initial triple with the corresponding integer unique identifier. Specifically, according to the first dictionary, find the integer unique identifier corresponding to "JKRowling"; according to the second dictionary, find the integer unique identifier corresponding to "written by"; according to the third dictionary, find the integer unique identifier corresponding to "Books". Therefore, the initial triple can be converted into a target triple consisting of three integer unique identifiers. For each initial triple in the resource relationship data, the above conversion process is repeated until all initial triples are converted into target triples.

[0052] The embodiment of the present application successfully obtains multiple target triples by constructing three dictionaries and converting the subject, predicate and object in the form of strings in the resource relationship data into unique identifiers in the form of integers. Compared with integer storage, string storage takes up a large space and is slower to query. Therefore, this method of the embodiment of the present application not only reduces the storage space, but also improves the query efficiency of subsequent resource relationship data.

[0053] It should be noted that since the target triplet is composed of three integer unique identifiers, the deformation triplet corresponding to each subsequent target triplet is actually also composed of these integer unique identifiers, so the storage space can be greatly reduced.

[0054] In an embodiment of the present application, optionally, the “querying based on the SPARQL query statement and the target index to obtain a resource relationship query result” includes: determining a query identifier corresponding to the resource relationship query from the first dictionary, the second dictionary, and the third dictionary based on the SPARQL query statement; determining an initial query result from the RDF table corresponding to the target index based on the query identifier, and converting the initial query result into a resource relationship query result according to the first dictionary, the second dictionary, and the third dictionary.

[0055] In this embodiment, the RDF database can determine the query identifier corresponding to the resource relationship query from the constructed first dictionary, second dictionary and third dictionary according to the SPARQL query statement. That is, the resource relationship query part included in the SPARQL query statement is converted into an integer unique identifier form through the first dictionary, the second dictionary and the third dictionary, and the combination of these three integer unique identifiers can be understood as the query identifier. For example, the SPARQL query statement indicates "find all books written by 'JKRowling'". First, the SPARQL query statement is parsed to identify the subject, predicate and object therein. After identification, its subject is "JKRowling", the predicate is "writing", and the objects are "books" respectively. Next, according to the first dictionary, the second dictionary and the third dictionary, these strings are converted into corresponding integer unique identifiers. For example, the integer unique identifier of "JKRowling" is 1001, the integer unique identifier of "writing" is 1002, and the integer unique identifier of "books" is 1003. Then the query identifier can be (1001-1002-1003).

[0056] After determining the query identifier, the RDF table corresponding to the target index can be queried according to the query identifier to obtain the initial query result. Among them, the RDF table corresponding to each index can include multiple, and different RDF tables are used to store the relationship between different resources. For example, the RDF table corresponding to the SOP index can include RDF tables such as author-writing-books, author-hobby-books, etc., so for the above example, the RDF table corresponding to "author-writing-books" can be found according to (1001-1002-1003), and the resource relationship query result can be found according to (1001-1002-1003). Here, since books are variables, all records with the subject "1001" (that is, JKRowling) and the predicate "1002" (that is, writing) can be found in the RDF table corresponding to "author-writing-books", and the object in each record is the result you want to find. Since integer unique identifiers are stored in the RDF table, these initial query results are still composed of integer unique identifiers.

[0057] Afterwards, the initial query result can be converted into a resource relationship query result. Specifically, for each combination of integer unique identifiers in the initial query result, the corresponding first dictionary, second dictionary, and third dictionary are searched to convert them back to the original string form. For example, if the initial query result contains a triple (1,1002,X) consisting of integer unique identifiers, where X is an integer unique identifier representing a book, the third dictionary can be searched to find the book name corresponding to X.

[0058] Through the above steps, all integer unique identifiers in the initial query result are converted back to the original string form. Then, the resource relationship query result is determined according to the restored triples, which includes all resource relationship records that satisfy the SPARQL query statement.

[0059] Through this embodiment, the resource relationship query result can be successfully retrieved in the relational database using the SPARQL query statement and the target index. The resource relationship query in this process is actually a query of integer unique identifiers, which can greatly reduce the complexity of the query compared to the original string query, which is conducive to improving the query efficiency and ensuring the query performance.

[0060] In an embodiment of the present application, optionally, the "integrating the query results to obtain a final query result corresponding to the query instruction" in step 103 includes: parsing the query instruction to determine whether there is query result processing information; when there is query result processing information, determining a processing method for the query result according to the query result processing information, and processing the query result according to the processing method to obtain the final query result, wherein the processing method includes at least one of aggregation processing, statistical processing, and sorting processing; when there is no query result processing information, merging the query results to obtain the final query result corresponding to the query instruction.

[0061] In this embodiment, after the master node receives the query results returned by each slave node combination, it can further parse the query instruction again to determine whether there is query result processing information. The query instruction may include query conditions and query result processing information. The query condition is used to determine which data to retrieve from the database. According to the query condition, a SPARQL query statement or an SQL query statement can be generated, and the query result processing information is used to describe how to process the retrieved data. The query result processing information may include aggregation processing (such as sum, average value, etc.), statistical processing (such as counting, maximum value, minimum value, etc.) or sorting processing (such as arranging in ascending or descending order). Assume that the query instruction is: "Find all products with sales exceeding 1000, and sort them from high to low by sales, and calculate the total sales." At this time, "sales exceeding 1000" is the query condition, and "sort by sales from high to low" and "calculate the total sales" are the query result processing information.

[0062] If there is query result processing information, a specific processing method can be determined based on the query result processing information. For example, if the query result processing information requires sorting, the processing method can be to call a sorting algorithm; if the query result processing information requires calculating the total sales, the processing method can be to call a sum function. After the processing methods are determined, these processing methods can be applied to the query results for processing.

[0063] If the query instruction does not contain any query result processing information, then the query results can be simply merged into a single output. Suppose the query instruction is: "Find all products belonging to the 'Electronic Products' category." Then the query instruction does not contain any query result processing information. Therefore, the query results (i.e. all product records belonging to the 'Electronic Products' category) can be merged into a list and returned as the final query result.

[0064] When the query instruction includes query result processing information in the embodiment of the present application, corresponding processing is performed on the query results retrieved from each slave node combination. While improving the efficiency of data post-processing, the flexibility of data query can be further improved, which is beneficial to improving the user's data query experience.

[0065] Further, as Figure 1 The specific implementation of the method, the embodiment of the present application provides an RDF data query optimization system for a distributed system, such as Figure 2 As shown, the system includes:

[0066] The master node is used to respond to the query instruction input by the user, parse the query instruction, generate all sub-query instructions corresponding to the query instruction according to the parsing result and the data storage situation corresponding to each slave node combination in the distributed system, and send each sub-query instruction to the corresponding slave node combination, wherein the query instruction is generated based on SPARQL, and each slave node combination includes an RDF database and a relational database;

[0067] A slave node combination, for obtaining a resource relationship query result from the RDF database and / or a resource attribute query result from the relational database based on the sub-query instruction;

[0068] The master node is further used to receive the query results returned by each slave node combination, and integrate the query results to obtain the final query results corresponding to the query instruction.

[0069] Optionally, the RDF database in the slave node combination is used to receive the sub-query instruction, parse the sub-query instruction, and determine whether the sub-query instruction is a query instruction of a mixed query mode, wherein the mixed query mode is a query mode that includes both resource attribute query and resource relationship query;

[0070] The RDF database in the slave node combination is further used to generate a SPARQL query statement and an SQL query statement according to the sub-query instruction when the sub-query instruction is a query instruction of a mixed query mode, obtain a resource relationship query result from the RDF database according to the SPARQL query statement, and send the resource relationship query result and the SQL query statement to the corresponding relational database; the relational database in the slave node combination is used to obtain a resource attribute query result from the relational database based on the resource relationship query result and the SQL query statement, and return the resource attribute query result to the RDF database; the RDF database is also used to return the resource relationship query result and the resource attribute query result to the master node;

[0071] The RDF database in the slave node combination is further used to generate an SQL query statement according to the sub-query instruction when the sub-query instruction is a query instruction for resource attribute query, and send the SQL query statement to the corresponding relational database; the relational database is further used to obtain a resource attribute query result from the relational database based on the SQL query statement, and return the resource attribute query result to the RDF database; the RDF database is further used to return the resource attribute query result to the master node;

[0072] The RDF database in the slave node combination is also used to generate a SPARQL query statement according to the sub-query instruction when the sub-query instruction is a query instruction for resource relationship query, obtain a resource relationship query result from the RDF database according to the SPARQL query statement, and return the resource relationship query result to the master node.

[0073] Optionally, the master node is further used to:

[0074] In response to a query instruction input by a user, before parsing the query instruction, original RDF data is acquired, and the original RDF data is parsed to obtain resource attribute data and resource relationship data corresponding to the original RDF data;

[0075] The resource relationship data is stored in the RDF database in each slave node combination, and the resource attribute data is grouped and each group of resource attribute data is stored in the relational database in different slave node combinations.

[0076] Optionally, the RDF database is further used for:

[0077] After receiving the resource relationship data, the resource relationship data is converted according to a preset triple form to obtain a plurality of target triples, and according to each variant triple form of the preset triple form, the positions of the subject, predicate and object in each target triple are adjusted to obtain a variant triple corresponding to the target triple;

[0078] The plurality of target triples are stored in the RDF table corresponding to the preset triple form, and the deformed triples are respectively stored in the RDF table corresponding to the deformed triple form, and according to the triple form corresponding to each RDF table, an index of the RDF table is constructed;

[0079] After the RDF database is started, the index is loaded into the memory, and after receiving the SPARQL query statement, a matching target index is determined from the loaded index based on the SPARQL query statement, and a query is performed based on the SPARQL query statement and the target index to obtain a resource relationship query result.

[0080] Optionally, the RDF database is further used for:

[0081] Before converting the resource relationship data into a preset triple form to obtain a plurality of target triples, a first dictionary, a second dictionary and a third dictionary are constructed, wherein the first dictionary is used to store a mapping relationship between each subject string and an integer unique identifier, the second dictionary is used to store a mapping relationship between each predicate string and an integer unique identifier, and the third dictionary is used to store a mapping relationship between each object string and an integer unique identifier;

[0082] The RDF database is also used for:

[0083] Converting the resource relationship data into a preset triple form to obtain a plurality of initial triples;

[0084] For each initial triple, the initial triple is converted into a target triple consisting of three integer unique identifiers according to the first dictionary, the second dictionary and the third dictionary.

[0085] Optionally, the RDF database is further used for:

[0086] Based on the SPARQL query statement, determining a query identifier corresponding to the resource relationship query from the first dictionary, the second dictionary, and the third dictionary;

[0087] Based on the query identifier, an initial query result is determined from the RDF table corresponding to the target index, and the initial query result is converted into a resource relationship query result according to the first dictionary, the second dictionary and the third dictionary.

[0088] Optionally, the master node is further used to:

[0089] Parsing the query instruction to determine whether there is query result processing information;

[0090] When there is query result processing information, determining a processing method for the query result according to the query result processing information, and processing the query result according to the processing method to obtain the final query result, wherein the processing method includes at least one of aggregation processing, statistical processing, and sorting processing;

[0091] When there is no query result processing information, the query results are merged to obtain a final query result corresponding to the query instruction.

[0092] It should be noted that for other corresponding descriptions of the functional units involved in the RDF data query optimization system for a distributed system provided in the embodiment of the present application, reference can be made to Figure 1 The corresponding description in the method will not be repeated here.

[0093] The present application also provides a computer device, which may be a personal computer, a server, a network device, etc. Figure 3 As shown, the computer device includes a bus, a processor, a memory and a communication interface, and may also include an input and output interface and a display device. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store location information. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the steps in each method embodiment are implemented.

[0094] Those skilled in the art will understand that Figure 3 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0095] In one embodiment, a computer-readable storage medium is provided. The computer-readable storage medium may be non-volatile or volatile, and stores a computer program thereon. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0096] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0097] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0098] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0099] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0100] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A RDF data query optimization method for a distributed system, characterized in that: Applied to a master node in a distributed system, the method comprises: In response to a query instruction input by a user, the query instruction is parsed, and all sub-query instructions corresponding to the query instruction are generated according to the parsing result and the data storage situation corresponding to each slave node combination in the distributed system, wherein the query instruction is generated based on SPARQL, and each slave node combination includes an RDF database and a relational database; Sending each sub-query instruction to a corresponding slave node combination, so that the slave node combination obtains a resource relationship query result from the RDF database and / or obtains a resource attribute query result from the relational database based on the sub-query instruction; Receive the query results returned by each slave node combination, and integrate the query results to obtain the final query result corresponding to the query instruction.

2. The method according to claim 1, characterized in that The sub-query instruction is received from the RDF database in the node combination, and the sub-query instruction is parsed to determine whether the sub-query instruction is a query instruction of a mixed query mode, wherein the mixed query mode is a query mode that includes both resource attribute query and resource relationship query; When the sub-query instruction is a query instruction of a hybrid query mode, the RDF database generates a SPARQL query statement and an SQL query statement according to the sub-query instruction, obtains a resource relationship query result from the RDF database according to the SPARQL query statement, and sends the resource relationship query result and the SQL query statement to a corresponding relational database, so that the relational database obtains a resource attribute query result from the relational database based on the resource relationship query result and the SQL query statement, and returns the resource attribute query result to the RDF database; the RDF database returns the resource relationship query result and the resource attribute query result to the master node; When the sub-query instruction is a query instruction for resource attribute query, the RDF database generates an SQL query statement according to the sub-query instruction, and sends the SQL query statement to the corresponding relational database, so that the relational database obtains the resource attribute query result from the relational database based on the SQL query statement, and returns the resource attribute query result to the RDF database; the RDF database returns the resource attribute query result to the master node; When the sub-query instruction is a query instruction for resource relationship query, the RDF database generates a SPARQL query statement according to the sub-query instruction, obtains a resource relationship query result from the RDF database according to the SPARQL query statement, and returns the resource relationship query result to the master node.

3. The method according to claim 1, characterized in that In response to the query instruction input by the user, before parsing the query instruction, the method further includes: Acquire original RDF data, parse the original RDF data, and obtain resource attribute data and resource relationship data corresponding to the original RDF data; The resource relationship data is stored in the RDF database in each slave node combination, and the resource attribute data is grouped and each group of resource attribute data is stored in the relational database in different slave node combinations.

4. The method according to claim 3, characterized in that: After receiving the resource relationship data, the RDF database converts the resource relationship data according to a preset triple form to obtain a plurality of target triples, and adjusts the positions of the subject, predicate and object in each target triple according to each variant triple form of the preset triple form to obtain a variant triple corresponding to the target triple; The RDF database stores the multiple target triples in the RDF tables corresponding to the preset triple forms, and stores the deformed triples in the RDF tables corresponding to the deformed triple forms, respectively, and constructs an index of the RDF table according to the triple form corresponding to each RDF table; After the RDF database is started, the index is loaded into the memory, and after receiving the SPARQL query statement, a matching target index is determined from the loaded index based on the SPARQL query statement, and a query is performed based on the SPARQL query statement and the target index to obtain a resource relationship query result.

5. The method according to claim 4, characterized in that Before converting the resource relationship data into a preset triple form to obtain a plurality of target triples, the method further includes: Constructing a first dictionary, a second dictionary and a third dictionary, wherein the first dictionary is used to store a mapping relationship between each subject string and an integer unique identifier, the second dictionary is used to store a mapping relationship between each predicate string and an integer unique identifier, and the third dictionary is used to store a mapping relationship between each object string and an integer unique identifier; The resource relationship data is converted into a preset triple form to obtain a plurality of target triples, including: Converting the resource relationship data into a preset triple form to obtain a plurality of initial triples; For each initial triple, the initial triple is converted into a target triple consisting of three integer unique identifiers according to the first dictionary, the second dictionary and the third dictionary.

6. The method according to claim 5, characterized in that The querying based on the SPARQL query statement and the target index to obtain a resource relationship query result includes: Based on the SPARQL query statement, determining a query identifier corresponding to the resource relationship query from the first dictionary, the second dictionary, and the third dictionary; Based on the query identifier, an initial query result is determined from the RDF table corresponding to the target index, and the initial query result is converted into a resource relationship query result according to the first dictionary, the second dictionary and the third dictionary.

7. The method according to claim 1, characterized in that The step of integrating the query results to obtain a final query result corresponding to the query instruction includes: Parsing the query instruction to determine whether there is query result processing information; When there is query result processing information, determining a processing method for the query result according to the query result processing information, and processing the query result according to the processing method to obtain the final query result, wherein the processing method includes at least one of aggregation processing, statistical processing, and sorting processing; When there is no query result processing information, the query results are merged to obtain a final query result corresponding to the query instruction.

8. An RDF data query optimization system for distributed systems, characterized in that: include: The master node is used to respond to the query instruction input by the user, parse the query instruction, generate all sub-query instructions corresponding to the query instruction according to the parsing result and the data storage situation corresponding to each slave node combination in the distributed system, and send each sub-query instruction to the corresponding slave node combination, wherein the query instruction is generated based on SPARQL, and each slave node combination includes an RDF database and a relational database; A slave node combination, for obtaining a resource relationship query result from the RDF database and / or a resource attribute query result from the relational database based on the sub-query instruction; The master node is further used to receive the query results returned by each slave node combination, and integrate the query results to obtain the final query results corresponding to the query instruction.

9. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. A computer device comprising a storage medium, a processor, and a computer program stored in the storage medium and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

Images