Method for obtaining data and related device

The method employs a fusion database to consolidate data from multiple databases, enabling efficient data retrieval in a unified query language, addressing the complexity of querying across different database models and improving performance by reducing the need for separate database searches and merging operations.

WO2024228636A1PCT designated stage expired Publication Date: 2024-11-07HUAWEI CLOUD COMPUTING TECHNOLOGIES CO LTD +1
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
PCT/RU2023/000133
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-02
Publication Date
2024-11-07

AI Technical Summary

Technical Problem

Retrieving data from databases organized in different models is complex and time-consuming due to the need for constructing query statements in specific database languages, such as SQL, OQL, and XQuery, which requires searching and merging data across multiple databases.

Method used

A method that uses a fusion database to consolidate data from multiple databases, allowing queries to be executed in a graph query language, which simplifies data retrieval by eliminating the need for separate searches across multiple databases and reduces the complexity of merging data, and includes mechanisms for updating data and relationships based on metadata changes.

Benefits of technology

This approach saves time on querying data by allowing queries to be executed in a unified manner across multiple databases, reducing the need for expensive JOIN operations and ensuring accurate results through metadata-driven updates, thereby improving performance and efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure RU2023000133_07112024_PF_FP_ABST
    Figure RU2023000133_07112024_PF_FP_ABST
Patent Text Reader

Abstract

Embodiments of this application provide a method for obtaining data and a related device. The method includes: obtaining a first query request, where the first query request is configured to retrieve data from N databases; and searching, according to the first query request, a fusion database to obtain a first result, where the fusion database includes N data sets, the N data sets are in one-to-one correspondence with the N databases, an ith data set among the N data sets includes a portion of characteristics belonging to at least one data in an ith database among the N databases, each characteristic in the portion of the characteristics has at least one corresponding characteristic in other N-1 database(s), and the fusion database further includes a characteristics relationship information describing a correspondence of characteristics of the N databases. This approach may save time on querying data.
Need to check novelty before this filing date? Find Prior Art

Description

METHOD FOR OBTAINING DATA AND RELATED DEVICETECHNICAL FIELD

[0001] Embodiments of the present invention relate to the field of information technologies, and more specifically, to a method for obtaining data and a related device.BACKGROUND

[0002] A database (DB) is a collection of data that is organized and stored in a manner that allows for efficient retrieval, modification, and management of the data. Databases can be used for a wide range of purposes, from storing customer information for a small business to managing complex systems for large organizations. Related information may be stored in different databases. Retrieving data from the databases is complex and time consuming as the databases may be organized in different models. Different database languages may be designed for accessing data from databases organized in the different models. For example, a structured query language (SQL) is specific to a SQL database, an object query language (OQL) is specific to an object-oriented database, and an XQuery is specific to an extensible markup language (XML) database. In addition, some SQL databases based on different relational database management systems (RDBMS) may also use different SQL languages. If a user wants to search for data from the databases organized in the different models, he / she has to construct query statements using different database languages. Therefore, how to retrieve data from the databases in the different models is one of the most important issues to be resolved.SUMMARY

[0003] Embodiments of this application provide a method for obtaining data and a related device. This approach may save time on querying data.

[0004] According to a first aspect, an embodiment of the present application provides a imethod for obtaining data. The method includes: obtaining a first query request, where the first query request is configured to retrieve data from N databases, and N is a positive integer greater than one; and searching, according to the first query request, a fusion database to obtain a first result, where the fusion database includes N data sets, the N data sets are in one-to-one correspondence with the N databases, an ithdata set among the N data sets includes a portion of characteristics belonging to at least one data in an ithdatabase among the N databases, each characteristic in the portion of the characteristics has at least one corresponding characteristic in other N-l database(s), and the fusion database further includes a characteristics relationship information describing a correspondence of characteristics of the N databases, i is a positive integer greater than or equal to one and less than or equal to N.

[0005] According to the above-mentioned technical solution, the fusion database only includes a portion of data from the N databases, and the first query request may retrieve data from the fusion database. Therefore, there is no need to search N databases separately or to merge all data from N databases into one database. This approach may save time on querying data and require no significant time spent on merging data across N databases.

[0006] In a possible design, the fusion database is modeled as a graph, and the obtaining a first query request, includes: obtaining a second query request, where the second query request is configured to retrieve the data from the N databases, and a language of the second query request is described in a different query language other than a graph query language; and transforming the second query request into the first query request, where the first query request is described in the graph query language.

[0007] Graph query languages are designed to handle complex data structures and relationships, which makes them more flexible than SQL. Graph query languages use a visual, graph-based approach that allows users to more intuitively query and explore data. Further, graph databases are optimized for querying and traversing relationships between nodes, making them more performant than traditional SQL databases when working with complex, interconnected data. The reason is that graph databases do not require expensive JOIN operations to relate data across tables, which can significantly slow down query response times.

[0008] In a possible design, N is greater than a first threshold, and / or, a total size of the N databases is greater than a second threshold.

[0009] In a possible design, the method further includes: determining at least one third query request according to the first result, where each of the at least one third query request is configured to retrieve data from one of the N databases; searching, according to the at least one third query request, at least one database of the N databases in parallel to obtain a second result; and combining the first result and the second result to obtain a final result.

[0010] As mentioned above, the fusion database includes only a portion of the data from the N databases. According to the above-mentioned technical solution, additional data that is not present in the fusion database may be retrieved.

[0011] In a possible design, the method further includes: determining whether data in the N data sets has changed according to metadata of the N databases; and when data in the N data sets has changed, updating the N data sets contained in the fusion database according to the changed data.

[0012] According to the above-mentioned technical solution, the data contained in the fusion database may be updated according to metadata of the N databases. This ensures that the results retrieved from the fusion database are accurate.

[0013] In a possible design, the method further includes: determining whether a correspondence of one or more of the characteristics of the N databases has changed according to metadata of the N databases; and when the correspondence of one or more of the characteristics of the N databases has changed, updating the characteristics relation information contained in the fusion database according to the changed correspondence.

[0014] According to the above-mentioned technical solution, the correspondence of the characteristics contained in the fusion database may be updated according to metadata of the N databases. This ensures that the results retrieved from the fusion database are accurate.

[0015] According to a second aspect, an embodiment of the present application provides an electronic device, and the electronic device has a function of implementing the method in the first aspect. The function may be implemented by hardware, or may be implemented by hardware executing corresponding software. The hardware of the software includes one or more modules corresponding to the function.

[0016] According to a third aspect, an embodiment of the present application provides a computer readable storage medium, which includes instructions. When the instructions run ona computer, the computer is enabled to perform the method in the first aspect or any possible implementation of the first aspect.

[0017] According to a fourth aspect, an electronic device is provided, including a processor and a memory. The processor is connected to the memory. The memory is configured to store instructions, and the processor is configured to execute the instructions. When the processor executes the instructions stored in the memory, the processor is enabled to perform the method in the first aspect or any possible implementation of the first aspect.

[0018] According to a fifth aspect, a chip system is provided, where the chip system includes a memory and a processor, the memory is configured to store a computer program, and the processor is configured to invoke the computer program from the memory and run the computer program, so that an electronic device on which the chip system is disposed performs the method in the first aspect or any possible implementation of the first aspect.

[0019] According to a sixth aspect, a computer program product is provided, where when the computer program product runs on an electronic device, the electronic device is enabled to perform the method in the first aspect or any possible implementation of the first aspect.

[0020] According to a seventh aspect, a computing device cluster is provided. The computing device cluster comprises at least one computing device, wherein the computing device comprises a processor and a memory coupled with the processor. The memory is configured to store a computer program, and the processor is configured to invoke and run the computer program stored in the memory, so that the method according to the first aspect or any implementation manners of the first aspect is executed.DESCRIPTION OF DRAWINGS

[0021] FIG. 1 illustrates a diagram of a method for obtaining data in accordance with some embodiments of the present application.

[0022] FIG. 2 illustrates a diagram of a system for implementing a method for obtaining data in accordance with some embodiments of the present application.

[0023] FIG. 3 illustrates an electronic device according to embodiments of the present application.

[0024] FIG. 4 illustrates an electronic device according to embodiments of the present application.DESCRIPTION OF EMBODIMENTS

[0025] The following describes the technical solutions in this application with reference to the accompanying drawings.

[0026] FIG. 1 illustrates a diagram of a method for obtaining data in accordance with some embodiments of the present application. The method shown in FIG. 1 may be implemented by a computer, a component of the computer (e.g., a circuit or a chip in the computer) or a computing device cluster including at least one computing device. The computer may be a personal computer, a server, or the like. In some embodiments, the method shown in FIG. 1 may be performed by a computer cluster or a cloud system. Different steps of the method shown in FIG. 1 may be performed by different hardware of the computer cluster or the cloud system. For convenience, it is assumed that the method shown in FIG. 1 is performed by a computer for detailed description in the following.

[0027] 101, the computer obtains a first query request.

[0028] The first query request is configured to retrieve data from N databases, N is a positive integer greater than one.

[0029] A model of each of the N databases is not limited in the present application. For example, models of the N databases may be at least one of the followings: a rational database, a key-value (KV) database, a graph database, or the like.

[0030] In some embodiments, models of the N databases are the same.

[0031] In some embodiments, models of the N databases are not identical. In other words, at least one of the N databases has a model that is different from models of the other databases.

[0032] In some embodiments, the different models mentioned in the present application may refer to a difference in a broad sense.

[0033] In some other embodiments, the different models mentioned in the present application may refer to a difference in a narrow sense.

[0034] The three databases mentioned above are also used as an example. The first database and a second database are relational databases, and the first database and the second database are based on different RDBMS. For example, the first database is based on Oracle database, and the second database is based on MySQL. The third database is a non-relational database. For example, the third database may be based on MongoDB. Obviously, the model of the third database is different from that of either the first database or the second database. However, in a broad sense, the model of the first database is the same as the model of the second database; and in a narrow sense, the model of the first database is different from the model of the second database.

[0035] For convenience, it is assumed that the N databases are different in a narrow sense for detailed description in the following.

[0036] 102, the computer searches a fusion database to obtain first result according to the first query request.

[0037] The fusion database includes N data sets, and the N data sets are in one-to-one correspondence with the N databases. Each of the N data sets includes one or more groups of data in a corresponding database. An ithdata set among the N data sets includes a portion of characteristics belonging to at least one data in an ithdatabase among the N databases, each characteristic in the portion of the characteristics has at least one corresponding characteristic in other N-l database(s), and the fusion database further includes a characteristics relationship information describing a correspondence of characteristics of the N databases, i is a positive integer greater than or equal to one and less than or equal to N.

[0038] The fusion data base further includes values of the characteristics contained in the fusion database. In other words, the ithdata set among the N data sets includes a value(s) of a portion of characteristics belonging to at least one data in the ithdatabase among the N databases, each characteristic in the portion of the characteristics has at least one corresponding characteristic in other N-l database(s).

[0039] For example, if the ithdatabase is an SQL database, a data in the i* database may be a record or a row in the SQL database, and a characteristic of the data may be an attribute or a field of the record. If the ithdatabase is a KV database (e.g., Redis, RocksDB, LevelDB or the like), a data in the ithdatabase may be a key in the KV database, and a characteristic of the datamay be an attribute in a value of the key. If the ithdatabase is a graph database, a data in the ithdatabase may be a node in the KV database, and a characteristic of the data may be a property of the node.

[0040] For convenience, a characteristic stored in the fusion database may be referred to as a first characteristic, and a characteristic not stored in the fusion database may be referred to as a second characteristic. Each of the N databases may include at least one first characteristic and at least one second characteristic. Each of the first characteristics of the N databases may be corresponding to at least one of another first characteristic of the N databases. When one of the first characteristics is determined, all of the rest first characteristic(s) may be located according to the corresponding relationship of the first characteristics. Correspondingly, when the first query request matches a first characteristic, then all first characteristics may be found according to the corresponding relationship.

[0041] The three databases mentioned above are also used as an example. It is assumed that the first database is based on MySQL, the second database is based on Oracle database, and the third database is based on SQL Server. Based on this assumption, data stored in the three databases may be organized as a table, and a table is a collection of related data entries and it consists of columns and rows. For convenience, a table including data in the first database may be referred to as a first table, a table including data in the second database may be referred to as a second table, and a table including data in the third database may be referred to as a third table. Each of three tables (that is, the first table, the second table, and the third table) includes a plurality of columns. The fusion database may also be a table (hereinafter referred to as a “fusion table”). Columns of the fusion table may be grouped into three data sets, and the three data sets are in one-to-one correspondence with the three tables. Each of the three data sets includes one or more columns from a corresponding table. For convenience, a data set corresponding to the first table may be referred to as a first data set, a data set corresponding to the second table may be referred to as a second data set, and a data set corresponding to the third table may be referred to as a third data set. It is assumed that the first table includes Xi columns, the second table includes X2 columns, the third table includes X3 columns, and Xi, X2, and X3 are positive integers. The fusion table may include K columns, where K is a positive integer less than X, and X is a sum of Xi, X2, and X3. The K columns may be grouped into threedata sets, the first data set includes Ki column(s) from the first table, where Ki is a positive integer less than Xi; the second data set includes K2 column(s) from the second table, where K2 is a positive integer less than X2; and the third data set includes K3 column(s) from the third table, where K3 is a positive integer less than X3. Each column of the Ki column(s) has at least one corresponding column, and the at least one corresponding column belongs to the K2 column(s) and / or the K3 columns. Similarly, each column of the K2 column(s) has at least one corresponding column, and the at least one corresponding column belongs to the Ki column(s) and / or the K3 columns. Each column of the K3 column(s) has at least one corresponding column, and the at least one corresponding column belongs to the Ki column(s) and / or the K2 columns.

[0042] For example, in some embodiment, the fusion table may include three columns, and the three columns are primary keys of the three databases. In another word, the fusion table includes a primary key of the first table (hereinafter referred to as a first primary key), a primary of the second table (hereinafter referred to as a second primary key), and a primary key of the third table (hereinafter referred to as a third primary key). Each of the three primary keys corresponds to at least one of other 2 primary keys except itself. For example, the first primary key corresponds to the second primary key, and the second primary key corresponds to the third primary key. For another example, the first primary key corresponds to both the second primary key and the third primary key.

[0043] A primary key of a database is used to uniquely identify each record in the database. Therefore, a primary key column or set of columns must contain unique values and cannot contain NULL values. Therefore, when the fusion database include the primary keys of the N databases, each column of the fusion database may not include identical values.

[0044] The fusion database further includes a characteristics relationship information describing a correspondence of characteristics of the N databases

[0045] As mentioned above, a characteristic corresponding to the value(s) included in one data set corresponds to at least one characteristic corresponding to the value(s) included in other data set(s). According to the correspondence of the characteristics, by using the value(s) found in the fusion database, one can locate other database(s) that includes characteristic values that are not in the fusion database.

[0046] Take two database as an example. It is assumed that each record in the first databaseincludes the following fields: c_id, region, salesperson, time; each record in the second database includes the following field: c_id, w_id, w_region, phone number, and w_address. A primary key of the first database is c_id, and a primary key of the second database is w_id. The fusion database includes the primary key of the first database and the primary key of the second database, which means that c_id correspons to w_ID. According to the first query request, a record including w_id = 50 and c_id = 100 may be retrieved from the fusion database. According to the characteristics relationship information describing a correspondence of c_id and w_id, values of other fields for a record with c_id=100 may be obtained from the first database, and values of other fields for a record with w_id=50 may be obtained from the second database.

[0047] In some embodiments, a language of a query request obtained by the computer differs from a language of the fusion database. Under this condition, the computer has to transform the obtained query request into a query request in the language of the fusion database. The first query request is the query request in the language of the fusion database. A second query request is the query request obtained by the computer, and a language of the second query request is different from the language of the fusion database.

[0048] For example, the fusion database is modeled as a graph (in other words, a model of the fusion database is a graph database), but the language of the second query request is described in a different query language other than a graph query language. Under this condition, the computer may transform the second query request into the first query request described in the graph query language. For example, if the second query request is described in the SQL query language and the fusion database is a graph database, the computer may transform the second query request described in the SQL query language into the first query request described in the graph query language.

[0049] In some embodiments, before searching the fusion database, the computer may determine that the first query request meets a preset condition. For example, the computer may determine that the first query request is a deep relation query or a highly selective query. In other words, the computer only searches the fusion database when the first query request is the deep relation query or the highly selective query.

[0050] In some embodiments, the computer may determine that a query request is the deeprelation query or the highly selective query when the number of join statements in the query request is greater than a first threshold. In some embodiments, the first threshold may be a preset value, and the preset value may be a positive integer greater than two. For example, the first threshold may be 2, 3, 4, or 5. In some other embodiments, the first threshold is related with the N databases. If a total size of the N databases is larger, the first threshold is smaller; or if the total size of the N databases is smaller, the first threshold is larger. For example, if the total size of the N databases includes more than one million records, the first threshold may be 3; or if the total size of the target tables includes more than ten million records, the first threshold may be 2.

[0051] In some embodiments, the computer may determine that a query request is the deep relation query or the highly selective query when a total size of the N databases is greater than a second threshold. For example, in some embodiments, if the total size of the N databases is greater than one hundred million records, the computer may determine that the query request is the deep relation query or the highly selective query, and the computer may search the fusion database to obtain a result corresponding to the query; in contrast, if the total size of the N databases is less than or equal to one hundred million records, the computer may determine that the query request is not the deep relation query or the highly selective query, and the computer may search each of the N databases to obtain the result of the query.

[0052] In some other embodiments, the computer may determine at least one query request according to the first result. To distinguish between the query request for searching the fusion database and the query request determined according to the first result, the query request determined according to the first result may be referred to as a third query request. In other words, the computer may determine at least one third query request according to the first result. The at least one third query request is used to retrieve data in at least one database among the N databases in parallel.

[0053] For convenience, letter M is used to represent the number of the third query request. Further, the N databases are SQL databases in a narrow sense for detailed description in the following.

[0054] In some embodiments, M is equal to N. The first query request may retrieve data from all of the N data sets in the fusion database. Therefore, the first result may include at leastN items from the N data sets. Under this condition, M is equal to N. As mentioned above, the N data sets are in one-to-one correspondence with the N databases, and each of the N data sets includes a part of data in a corresponding database. The first result may miss one or more fields of records in the N databases. Therefore, the computer may search the N databases to obtain the missing properties.

[0055] The three databases mentioned above are also used as an example. It is also assumed that the three databases are three SQL databases (that is, the three tables) and that the fusion table includes the K columns, where the K columns may be grouped into the three data sets. As mentioned above, the first data set includes the Ki column(s) from the first table, the second data set includes the K2 column(s) from the second table, and the third data set includes the K3 column(s) from the third table. The first query request may retrieve data from at least one column from each of the three data sets. In other words, the first query request may retrieve data from one or more columns of the Ki column(s), retrieve data from one or more columns of the K2 column(s), and retrieve data from one or more columns of the K3 column(s). The first table includes Xi columns, but the fusion table includes only Ki column(s) of the Xi columns. Therefore, the first result obtained and returned by the first query request may at best include value(s) in the Ki column(s) of the first table. The first query request cannot retrieve value(s) belonging to X1-K1 column(s) of the first table from the fusion table. In order to obtain missing value(s) in the X1-K1 column(s), the computer may determine a third query request to search the first table to obtain the missing value(s) in the X1-K1 column(s). The third query request may include value(s) of the Ki column(s) obtained according to the first query request. Similarly, the computer may determine a third query request to search the second table to obtain missing value(s) in X2-K2 column(s) of the X2 columns, and the computer may determine a third query request to search the third table to obtain missing value(s) in X3-K3 column(s) of the X3 columns.

[0056] In some other embodiments, the first query request may only retrieve data from a part of the N data sets. The three databases mentioned above are also used as an example. The first query request may only retrieve data from the first data set and the second data set. Under this condition, M is a positive integer less than N.

[0057] According to the at least one third query request, the computer may retrieve data from the at least one database of the N databases in parallel. The at least one third query requestis in one-to-one correspondence with the at least one database. The computer may obtain at least one field value from each of the at least one database according to a corresponding third query request. In order to distinguish between the first result (that is, the result obtained according to the first query request) and the result obtained according to the third query request, the result obtained according to the at least one third query request may be referred to as a second result. The second result may include at least one third result. The at least one third result is in one-to-one correspondence with the at least one third query request, and each of the at least one third result includes field values obtained from a database based on a corresponding third query request. In other words, the computer may obtain at least one third result from the at least one database according to the at least one third query request respectively.

[0058] After obtaining the second result, the computer may combine the first result and the second result to obtain a final result as a return of the first query request. Each of the first result may include several characteristic values, and each of the several characteristic values is retrieved from one of at least one data set in the fusion database, where the number of the at least one data set is a positive integer less than or equal to N. Therefore, according to a source of the several characteristic values, characteristic values in the first result may be grouped into at least one characteristic group, at least one M characteristic group is in one-to-one correspondence with the at least one data set, and each of the at least one characteristic group includes at least one characteristic value obtained from a corresponding data set. As the at least one data set is in one-to-one correspondence with the at least one database, the at least one characteristic group is also in one-to-one correspondence with the at least one database, and each of the at least one characteristic group includes at least one characteristic value in a corresponding database. As mentioned above, each of the at least one data set only includes a part of data in the corresponding database. Therefore, each of the at least one characteristic group may not include each characteristic value in the corresponding database. The missing characteristic value(s) may be obtained by the second result. The second result includes at least one third result, and the at least one third result is also in one-to-one correspondence with the at least one database. The at least one characteristic group is in one-to-one correspondence with the at least one third result, each of the at least one characteristic group includes at least one characteristic value in the corresponding database, and each of the second result includesmissing characteristic value in the corresponding database.

[0059] The three databases mentioned above are also used as an example. The first result R1 includes Ki field value(s) in the first table, and the second result includes Xi-Ki field value(s) in the first table. Consequently, a combination of the first result and the second result includes all filed value(s) of at least one record that satisfied the first query request in the first table.

[0060] In some other embodiments, the first result obtained from the fusion table may satisfy a search requirement. In other words, the computer may only want to obtain data in the fusion database and do not want to retrieve the missing properties. Under this condition, the first result may be returned as a final result of the first query request.

[0061] In some embodiments, the computer may track changes from the N databases. When data in the N data sets has changed, the computer may update the N data sets contained in the fusion database according to the changed data.

[0062] FIG. 2 illustrates a diagram of a system for implementing a method for obtaining data in accordance with some embodiments of the present application.

[0063] Referring to FIG. 2, the system 200 includes four modules, a query generator module 210, a serverless service module 220, a database engine module 230, and a graph engine module 240.

[0064] The query generator module 210 is configured to generate a query request. The query generator module 210 may include at least one query builder. Each of the at least one query builder corresponds to a program language and generates a query request in the corresponding program language. For example, the query generator module 210 includes three builders, a JavaScript query builder sub-module 211 , a SQL query builder sub-module 212, and a Gremlin query builder sub-module 213. The JavaScript query builder sub-module 211 may generate a query request described in JavaScript, the SQL query builder sub-module 212 may generate a query request described in SQL, and the Gremlin query builder sub-module 213 may generate a query request described in Gremlin. Gremlin is a query language used for graph databases. It is a powerful tool that allows users to traverse and manipulate graph data using a concise and expressive syntax. Gremlin is designed to work with a variety of graph databases, including Apache TinkerPop, Neo4j, and Amazon Neptune. It is commonly used for tasks such as graph traversal, pattern matching, and graph analysis. With Gremlin, users can easily createcomplex queries that extract specific information from their graph data, making it a valuable tool for data analysis and management. Gremlin is just an example of a graph language, and the present application is not limited thereto. For example, other graph languages may also be used in the present application, such as graph query language (GQL), cypher query language (CQL), or the like.

[0065] The query request generated by the query generator module 210 may be transmitted to the serverless service module 220. The serverless service module 220 includes a non-graph connection sub-module 221, and a graph connection sub-module 222. The non-graph connection sub-module 221 may receive a query request in a non-graph language from the query generator module 210 and forward the query request to the database engine module 230. The graph connection sub-module 222 may receive a query request in a graph language from the query generator module 210 and forward the query request to the graph engine module 240.

[0066] The database engine module 230 includes a coordinator sub-module 231, a distributed engine sub-module 232, an optimized engine sub-module 233, an index service submodule 234, and a metadata sub-module 235.

[0067] The coordinator sub-module 231 is configured to receive the query request from the serverless service module 220. The coordinator sub-module 231 may determine whether the received query request is a deep relation query or a highly selective query. If the query request is the deep relation query or the highly selective query, the coordinator sub-module 231 may forward the query request to the optimized engine sub-module 233; or if the query request is neither the deep relation query nor the highly selective query, the coordinator sub-module 231 may forward the query request to the distributed engine sub-module 232. The method for determining whether the query request is the deep relation query or the highly selective query may refer to the above-mentioned embodiments and will not be repeated here.

[0068] After receiving the query request, the distributed engine sub-module 232 may search a database corresponding to the query request and retrieve data from the corresponding database. Details on how to retrieve data from the corresponding database may refer to the prior art and will not be described here.

[0069] The optimized engine sub-module 233 retrieves data from a fusion database that combines data from several databases. This is done using the received query request, which isused to retrieve from the several databases. Details about the fusion database may refer to the above-mentioned embodiments and will not be described here.

[0070] The fusion database may be modeled as a graph. In other words, the fusion database is a graph database. Since the language of the query request obtained by the database engine module 230 is a non-graph language, the optimized engine sub-module 233 may transform the language of the received query request from a non-graph language into a graph language, e.g., Gremlin.

[0071] For example, the query request obtained by the database engine 230 may be the followings:1 select *2 from bmsql_warehose3 left join bmsql_district on (w_id=d_w_id)4 left join bmsql_customer on (d_w_id=c_w_id and d_id=c_id_id)5 where w_id = 50 and c_id = 1006 and d_w_name =‘a’

[0072] The optimized engine sub-module 233 may transform the above-mentioned query request into the followings:1 g.V().has(‘bmsql_customer’, ‘c_id’, 100).2 ,out(‘bmsql_customer_2_bmsql_district’). .out(‘bmsql_district_2_ bmsql_warehouse’)3 .has(‘w_id’, 50)4 .path().by(valueMap())

[0073] For convenience, the query request transformed by the optimized engine submodule 233 (that is, the query request described in the graph language) may be referred to as a first query request, and the query request obtained by the database engine 230 (that is, the query request described in a different query language other than a graph query language) may be referred to as a second query request. Moreover, it should be understood that the graph language of the first query request may match the graph language of the fusion database. For example, if the graph language used by the fusion database is Gremlin, the graph language of the first query request may also be Gremlin.

[0074] After obtaining the first query request, the optimized engine sub-module 233 may transmit the first query request to the graph engine module 240.

[0075] Referring to FIG. 2, the graph engine module 240 includes a metadata sub-module 241, a coordinator sub-module 242, a relation graph sub-module 243, and a data enrich submodule 244.

[0076] The relation graph sub-module 243 may be used to store the fusion database. Therefore, the first query request may be transmitted to the relation graph sub-module 243, which may obtain a result corresponding to the first query request from the fusion database. As mentioned above, the fusion database includes only a portion of the data from the several databases. In some embodiments, the relation graph sub-module 243 may retrieve additional data that is not present in the fusion database from the metadata sub-module 241 that stores all the data from the several databases. For convenience, the data retrieved from the fusion database may be referred to as a first result, and the addition data retrieved from the several databases corresponding to the fusion database may be referred to as a second result. In other words, the relation graph sub-module 243 may obtain the first result from the fusion database and obtain the second result from the metadata sub-module 241.

[0077] The relation graph sub-module 243 may determine a query request for retrieving the at least one second result from the several databases based on the first result. For convenience, the query request for retrieving the at least one second result may be referred to as a third query request. Similarly, a language of the third query request may match a language of the several databases. For example, if the language of the several database is SQL, then the language of the third query request is SQL. The query request in SQL mentioned above is also used as an example. The relation graph sub-module 243 may determine the followings as a third query request:1 select * from bmsql_warehouse where w_id in (50)2 select * from bmsql_customer where c_id in (100)

[0078] After obtaining the second result, the relation graph sub-module 243 may combine the first result and the second result to obtain a final result as a return of the first query request.

[0079] The metadata sub-module 235 is configured to store metadata of the several databases. The metadata sub-module 241 may obtain the several databases from the metadata sub-module 235. The metadata sub-module 241 may be configured to synchronize its databases with the one stored in the metadata sub-module 235. For example, if a user modifies a recordin one of the databases, the record in the database stored in the metadata sub-module 235 should be modified at first, and then the metadata sub-module 241 synchronizes the record stored in the metadata sub-module 241 with the modified record stored in the metadata sub-module 235. The fusion database is determined according to the databases saved in the metadata sub-module 241. For convenience, the metadata sub-module 235 may be referred to as a data source, and the databases stored in the metadata sub-module 235 may be referred to as metadata.

[0080] The index service sub-module 234 is configured to monitor the metadata to track changes in the data source. For example, the index service sub-module 234 may read an information schema to track changes in the data source. The information schema stores various metadata information about all databases, tables, columns, constraints, indexes, and other objects within the databases. This information schema may be used to analyze the structure and content of the databases, which helps in managing and maintaining the databases. When the index service sub-module 234 determines that one or more records in the metadata has been modified, the index service sub-module 234 may inform the metadata sub-module 241 to update the one or more records in the databases stored in the metadata sub-module 241.

[0081] The index service sub-module 234 may also be configured to monitor relationships of the databases stored in the data source. The fusion database includes a plurality of data sets, the plurality of data sets are in one-to-one correspondence with the databases. Each of the plurality of data sets is in correspondence with another one or more data sets among the plurality of data sets. For example, a first data set is anyone of the plurality of data sets, and a first database is a database corresponding to the first data set. The first data set include a portion of characteristics belonging to at least one data in the first database. Each characteristic in the portion of characteristics has at least one corresponding characteristic in other databases except the first database itself. Meanwhile, a characteristic(s) corresponding to the characteristic(s) of the first database is also comprised in the fusion database.

[0082] For example, all of the several databases are SQL databases, and each of the several databases includes a primary key. In some embodiments, the fusion database includes the several primary keys. Each of the primary keys of the several databases is in correspondence with another one or more primary keys of the several databases.

[0083] For convenience, if a characteristic is stored in the fusion database, the characteristicmay be referred to as a first characteristic. Correspondingly, if a characteristic is not stored in the fusion database, the characteristic may be referred to as a second characteristic. The fusion database may also store values of the first characteristics.

[0084] The fusion database may include information about a relationship of the first characteristics, and the information may be referred to as a characteristics relationship information. The corresponding relationship of the first characteristics may be modified. Therefore, if a correspondence of one or more of the first characteristics is modified, it may influence the characteristic relationship information for describing a correspondence of characteristics, where the characteristic relationship information is stored in the fusion database. Therefore, when the index service sub-module 234 observes that one or more of the first characteristics of the databases stored in the data source have been modified, the index service sub-module 234 may inform the relation graph sub-module 243 to update the characteristics relationship information contained in the fusion database according to the changed correspondence.

[0085] As mentioned above, the serverless service module 220 includes the graph connection sub-module 222 for forwarding a graph query request (that is, a query request described in the graph query language) to the graph engine module 230. The coordinator submodule 242 may be configured to receive the graph query request. If the graph query request is used for retrieving data from the fusion database, the coordinator sub-module 242 may transmit the graph query request to the relation graph sub-module 243, and the relation graph submodule 243 may retrieve data from the fusion database according to the graph query request. Similarly, a result obtained from the fusion database may miss one or more properties. The result with miss properties may be transmitted to the data enrich engine sub-module 244. The data enrich engine sub-module 244 may collect the missing properties from a corresponding database. In some embodiments, the data enrich engine sub-module 244 may obtain a primary key of the result and determine a primary key-based query request to obtain the missing properties. For example, the query request for obtaining the missing properties may be: select * from bmsql_warehouse where w_id in (50). After obtaining the missing properties, the data enrich engine sub-module 244 may combine the result obtained from the fusion database and the missing properties as a final result of the graph query request. The graph query request maybe the first query request in the above-mentioned embodiments, the result obtained from the fusion database according to the graph query request may be the first result in the above- mentioned embodiments, and the missing properties may be the second result in the above- mentioned embodiments.

[0086] In some embodiments, the modules in FIG. 2 may be implemented by different computers. The computer mentioned in the present application maybe a personal computer, a server, a workstation, a computer cluster or the like.

[0087] In some embodiments, the four modules in FIG. 2 may be implemented by four different computers.

[0088] In some embodiments, one computer may implement two or more modules in FIG. 2. For example, a computer A may be configured to implement the query generator module 210 and the serverless service module 220, a computer B may be configured to implement the database engine module 230, and a computer C may be configured to implement the graph engine module 240.

[0089] In some embodiments, one or more sub-modules in FIG. 2 may be implemented by a computer. For example, the metadata sub-module 235 may be implemented by a computer, and other sub-modules in the database engine module 230 may be implemented by another computer.

[0090] In some embodiments, the modules in FIG. 2 may be implemented by a computing device cluster. The computing cluster includes at least one computing device. The computing device maybe a personal computer, a server, a workstation, a computer cluster or the like.

[0091] FIG. 3 illustrates an electronic device according to embodiments of the present application. Referring to FIG. 3, the electronic device 300 includes an obtaining module 301, a storage module 302, and a processing module 303.

[0092] The obtaining module 301 is configured to obtain a first query request, where the first query request is configured to retrieve data from N databases, and N is a positive integer greater than one.

[0093] The storage module 302 is configured to store a fusion database.

[0094] The processing module 303 is configured to search, according to the first query request, the fusion database to obtain a first result, where the fusion database includes N datasets, the N data sets are in one-to-one correspondence with the N databases, an ithdata set among the N data sets includes a portion of characteristics belonging to at least one data in an ithdatabase among the N databases, each characteristic in the portion of the characteristics has at least one corresponding characteristic in other N-l database(s), and the fusion database further includes a characteristics relationship information describing a correspondence of characteristics of the N databases, i is a positive integer greater than or equal to one and less than or equal to N.

[0095] In some embodiments, the fusion database is modeled as a graph, and the obtaining module 301 is specifically configured to: obtain a second query request, where the second query request is configured to retrieve the data from the N databases, and a language of the second query request is described in a different query language other than a graph query language; and transform the second query request into the first query request, where the first query request is described in the graph query language.

[0096] In some embodiments, N is greater than a first threshold, and / or, a total size of the N databases is greater than a second threshold.

[0097] In some embodiments, the processing module 303 is further configured to: determine at least one third query request according to the first result, where each of the at least one third query request is configured to retrieve data from one of the N databases; search, according to the at least one third query request, at least one database of the N databases in parallel to obtain a second result; and combining the first result and the second result to obtain a final result.

[0098] In some embodiments, the processing module 303 is further configured to: determine whether data in the N data sets has changed according to metadata of the N databases; and when data in one or more of the N data sets has changed, update the N data sets contained in the fusion database according to the changed data.

[0099] In some embodiments, the processing module 303 is further configured to: determine whether a correspondence of one or more of the characteristics of the N databases has changed according to metadata of the N databases; and when the correspondence of one or more of the characteristics of the N databases has changed, update the characteristics relation information contained in the fusion database according to the changed correspondence.

[0100] In some embodiments, the electronic device 300 may be a computer.

[0101] In some embodiments, the electronic device 300 may be a component of a computer(e.g., a circuit, a chip, or a processor).

[0102] In some embodiments, the electronic device 300 may the system 200 shown in FIG. 2.

[0103] Details of the functions of the modules in FIG. 3 may be referred to the above- mentioned embodiments. For example, a function of the obtaining module 301 may refer to the functions of the optimized engine sub-module 233 and the data enrich sub-module 244; a function of the processing module 303 may refer to the function of the relation graph submodule 243; and a function of the storage module 303 may refer to the function of the relation graph sub-module 243.

[0104] As shown in FIG.4, an electronic device 400 may include a transceiver 401, a processor 402, and a memory 403. The transceiver 401 may be configured to receive a query request. The memory 403 may be configured to store code, instructions, and the like executed by the processor 402. The memory 403 may be further configured to store a fusion database, and databases corresponding to the fusion database.

[0105] It should be understood that the processor 402 may be an integrated circuit chip and has a signal processing capability. In an implementation process, steps of the foregoing method embodiments may be completed by using a hardware integrated logic circuit in the processor, or by using instructions in a form of software. The processor may be a general purpose processor, a central processing unit (CPU), a graphics processing unit (GPU), a neural processing unit (NPU), a system on chip (SoC) or another programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component. The processor may implement or perform the methods, the steps, and the logical block diagrams that are disclosed in the embodiments of the present invention. The general purpose processor may be a microprocessor, or the processor may be any conventional processor or the like. The steps of the methods disclosed with reference to the embodiments of the present invention may be directly performed and completed by a hardware decoding processor, or may be performed and completed by using a combination of hardware in the decoding processor and a software module. The software module may be located in a mature storage medium in the art, such as a random access memory,a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, or a register. The storage medium is located in the memory, and the processor reads information in the memory and completes the steps of the foregoing methods in combination with hardware in the processor.

[0106] It may be understood that the memory 403 in the embodiments of the present invention may be a volatile memory or a nonvolatile memory, or may include both a volatile memory and a nonvolatile memory. The nonvolatile memory may be a read-only memory (Read-Only Memory, ROM), a programmable read-only memory (Programmable ROM, PROM), an erasable programmable read-only memory (Erasable PROM, EPROM), an electrically erasable programmable read-only memory (Electrically EPROM, EEPROM), or a flash memory. The volatile memory may be a random access memory (Random Access Memory, RAM) and is used as an external cache. By way of example rather than limitation, many forms of RAMs may be used, and are, for example, a static random access memory (Static RAM, SRAM), a dynamic random access memory (Dynamic RAM, DRAM), a synchronous dynamic random access memory (Synchronous DRAM, SDRAM), a double data rate synchronous dynamic random access memory (Double Data Rate SDRAM, DDR SDRAM), an enhanced synchronous dynamic random access memory (Enhanced SDRAM, ESDRAM), a synchronous link dynamic random access memory (Synchronous link DRAM, SLDRAM), and a direct rambus random access memory (Direct Rambus RAM, DR RAM).

[0107] It should be noted that the memory in the systems and the methods described in this specification includes but is not limited to these memories and a memory of any other appropriate type.

[0108] An embodiment of the present application further provides a chip system, where the chip includes an input / output interface, at least one processor, at least one memory, and a bus. The at least one memory is configured to store instructions, and the at least one processor is configured to invoke the instructions of the at least one memory to perform operations in the methods in the foregoing embodiments.

[0109] An embodiment of the present application further provides a computer storage medium, where the computer storage medium may store a program instruction for performing the steps in the foregoing methods.

[0110] Optionally, the storage medium may be specifically the memory 403.

[0111] An embodiment of the present application further provides a computer program product, where when the computer program product runs on an electronic device, the electronic device is enabled to perform the steps in the foregoing methods.

[0112] The present application also provides a computing device cluster. The computing device cluster comprises at least one computing device, and the computing device including at least one processor coupled to at least one memory, and the at least one processor is configured to execute computer programs or instructions stored in the at least one memory to cause the device to execute the method as provided in the foregoing embodiments.

[0113] A person of ordinary skill in the art may be aware that, in combination with the examples described in the embodiments disclosed in this specification, units and algorithm steps can be implemented by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are performed by hardware or software depends on particular applications and design constraints of the technical solutions. A person skilled in the art may use different methods to implement the described functions for each particular application, but it should not be considered that the implementation goes beyond the scope of the present application.

[0114] It may be clearly understood by a person skilled in the art that, for the purpose of convenient and brief description, a detailed working process of the foregoing system, apparatus, and unit, refers to a corresponding process in the foregoing method embodiment. Details are not described herein again.

[0115] In the embodiments of the present application, “at least one” means one or more, and “a plurality of’ means two or more. The term “and / or” describes an association relationship between associated objects and represents that three relationships may exist. For example, A and / or B may represent the following three cases: only A exists, both A and B exist, and only B exists, where A and B may be singular or plural. The character “ / ” generally indicates an “or” relationship between the associated objects. “At least one of the following” and a similar expression thereof refer to any combination of these items, including any combination of one item or a plurality of items. For example, at least one of a, b, and c may indicate: a, b, c, a and b, a and c, b and c, or a, b, and c, where a, b, and c may be singular or plural.

[0116] In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus, and method may be implemented in other manners. For example, the described apparatus embodiment is merely an example. For example, the unit division is merely logical function division and may be other division in actual implementation. For example, a plurality of units or components may be combined or integrated into another system, or some features may be ignored or not performed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections may be implemented through some interfaces. The indirect couplings or communication connections between the apparatuses or units may be implemented in electronic, mechanical, or other forms.

[0117] The units described as separate parts may be or may not be physically separated, and parts displayed as units may be or may not be physical units, may be located in one position, or may be distributed on a plurality of network units. Some or all of the units may be selected based on actual requirements to achieve the objectives of the solutions of the embodiments.

[0118] In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each of the units may exist alone physically, or two or more units are integrated into one unit.

[0119] When the functions are implemented in a form of a software functional unit and sold or used as an independent product, the functions may be stored in a computer readable storage medium. Based on such an understanding, the technical solutions in the present application essentially, or the part contributing to the prior art, or some of the technical solutions may be implemented in a form of a software product. The computer software product is stored in a storage medium, and includes several instructions for instructing a computer device (which may be a personal computer, a server, a network device, or the like) to perform all or some of the steps of the methods described in the embodiments of the present application. The foregoing storage medium includes: any medium that can store program code, such as a USB flash drive, a removable hard disk, a read-only memory (ROM), a random-access memory (RAM), a magnetic disk, or an optical disc.

[0120] The foregoing descriptions are merely specific implementations of the present application, but are not intended to limit the protection scope of the present application. Any variation or replacement readily figured out by a person skilled in the art within the technicalscope disclosed in the present application shall fall within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

CLAIMSWhat is claimed is:

1. A method for obtaining data, wherein the method comprises: obtaining a first query request, wherein the first query request is configured to retrieve data from N databases, and N is a positive integer greater than one; and searching, according to the first query request, a fusion database to obtain a first result, wherein the fusion database comprises N data sets, the N data sets are in one-to-one correspondence with the N databases, an ithdata set among the N data sets comprises a portion of characteristics belonging to at least one data in an ithdatabase among the N databases, each characteristic in the portion of the characteristics has at least one corresponding characteristic in other N-l database(s), and the fusion database further comprises a characteristics relationship information describing a correspondence of characteristics of the N databases, i is a positive integer greater than or equal to one and less than or equal to N.

2. The method according to claim 1, wherein the fusion database is modeled as a graph, and the obtaining a first query request, comprises: obtaining a second query request, wherein the second query request is configured to retrieve the data from the N databases, and a language of the second query request is described in a different query language other than a graph query language; and transforming the second query request into the first query request, wherein the first query request is described in the graph query language.

3. The method according to claim 1 or 2, wherein N is greater than a first threshold, and / or, a total size of the N databases is greater than a second threshold.

4. The method according to any one of claims 1 to 3, wherein the method further comprises: determining at least one third query request according to the first result, wherein each of the at least one third query request is configured to retrieve data from one of the N databases; searching, according to the at least one third query request, at least one database of the N databases in parallel to obtain a second result; andcombining the first result and the second result to obtain a final result.

5. The method according to any one of claims 1 to 4, wherein the method further comprises: determining whether data in the N data sets has changed according to metadata of the N databases; and when data in the N data sets has changed, updating the N data sets contained in the fusion database according to the changed data.

6. The method according to any one of claims 1 to 5, wherein the method further comprises: determining whether a correspondence of one or more of the characteristics of the N databases has changed according to metadata of the N databases; and when the correspondence of one or more of the characteristics of the N databases has changed, updating the characteristics relation information contained in the fusion database according to the changed correspondence.

7. An electronic device, wherein the electronic device comprises: an obtaining module, configured to obtain a first query request, wherein the first query request is configured to retrieve data from N databases, and N is a positive integer greater than one; a storage module, configured to store a fusion database; a processing module, configured to search, according to the first query request, the fusion database to obtain a first result, wherein the fusion database comprises N data sets, the N data sets are in one-to-one correspondence with the N databases, an ithdata set among the N data sets comprises a portion of characteristics belonging to at least one data in an ithdatabase among the N databases, each characteristic in the portion of the characteristics has at least one corresponding characteristic in other N-l database(s), and the fusion database further comprises a characteristics relationship information describing a correspondence of characteristics of the N databases, i is a positive integer greater than or equal to one and less than or equal to N.

8. The electronic device according to claim 7, wherein the fusion database is modeled as a graph, and the obtaining module is specifically configured to: obtain a second query request, wherein the second query request is configured to retrieve the data from the N databases, and a language of the second query request is described in a different query language other than a graph query language; andtransform the second query request into the first query request, wherein a language of the first query request is described in the graph query language.

9. The electronic device according to claim 7 or 9, wherein N is greater than a first threshold, and / or, a total size of the N databases is greater than a second threshold.

10. The electronic device according to any one of claims 7 to 9, wherein the processing module is further configured to: determine at least one third query request according to the first result, wherein each of the third query request is configured to retrieve data from one of the N databases; search, according to the at least one third query request, at least one database of the N databases in parallel to obtain a second result; and combine the first result and the second result to obtain a final result.

11. The electronic device according to any one of claims 7 to 10, wherein the processing module is further configured to: determine whether data in the N data sets has changed according to metadata of the N databases; and when data in one or more of the N data sets has changed, update the N data sets contained in the fusion database according to the changed data.

12. The electronic device according to any one of claims 7 to 11, wherein the processing module is further configured to: determine whether a correspondence of one or more of the characteristics of the N databases has changed according to metadata of the N databases; and when the correspondence of one or more of the characteristics of the N databases has changed, update the characteristics relation information contained in the fusion database according to the changed correspondence.

13. A computing device cluster, comprising at least one computing device, wherein the computing device comprising a processor and a memory coupled with the processor, wherein the memory is configured to store a computer program, and the processor is configured to invoke and run the computer program stored in the memory, so that the computing device executes the method according to any one of claims 1 to 6.

14. A computer readable storage medium, wherein the computer readable storage medium stores instructions, and when the instructions run on an electronic device, the electronic device is enabled to perform the method according to any one of claims 1 to 6.

15. An electronic device, comprising a memory and a processor, wherein the memory is configured to store a computer program, and the processor is configured to invoke the computer program from the memory and run the computer program, so that the electronic device is enabled to perform the method according to any one of claims 1 to 6.

16. A computer program product, wherein when the computer program product runs on an electronic device, the electronic device is enabled to perform the method according to any one of claims 1 to 6.

17. A chip system, comprising a memory and a processor, wherein the memory is configured to store a computer program, and the processor is configured to invoke the computer program from the memory and run the computer program, so that an electronic device on which the chip system is disposed performs the method according to any one of claims 1 to 6.

Citation Information

Patent Citations

  • Relational data base processor and method therefor

    JP1993204983A

  • Verifying graph-based queries

    US9378239B1