Method for data query based on ignite cache architecture and system thereof
Patent Information
- Application Number
- CA3168298
- Authority / Receiving Office
- CA · CA
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2019-01-16
- Filing Date
- 2019-09-20
- Publication Date
- 2026-09-22
- Estimated Expiration
- 2039-09-20
Abstract
Description
METHOD FOR DATA QUERY BASED ON IGNITE CACHE ARCHITECTURE AND SYSTEM THEREOF BACKGROUND OF THE INVENTION Technical Field
[0001] The present invention relates to the technical field of data processing, and more particularly to a method for data query based on Ignite cache architecture and a system thereof. Description of Related Art
[0002] With the development of the business scales of internet-related companies, querying massive data usually takes considerable time. In particular, for complicated hotspot data query, high concurrency often means great pressure on the computing layer and the storage layer, and tends to lead to delayed query results. Additionally, in practical applications, storage of underlying data in complicated projects can involve various external data source, such as RDBMS, NoSQL, and HDFS. Since the storage media are diverse, different query instructions are touted to different external data sources. In the prior art, for accelerating query results, one common solution is to cache hotspot data in each data storage layer itself. However, such a known solution is only suitable for queries of real-time hotspot data. If a user wants to query non-hotspot data, such as same-period data, he / she has to additionally access external data sources. In this case, by far, the query efficiency is still low. SUMMARY OF THE INVENTION
[0003] The objective of the present invention is to provide a method for data query based on Ignite cache architecture and a system thereof, which use an intermediate cache layer to effectively speed up the query process.
[0004] In order to achieve the foregoing objective, in one aspect, the present invention provides a method for data query based on Ignite cache architecture, which comprises:
[0005] S1, setting a cache algorithm, and caching query objects obtained using the cache algorithm into a buffer pool in a priority order;
[0006] S2, based on the query objects, activating a scheduling task so as to extract the cache data from corresponding external data sources and storing the cache data into an Ignite distributed node;
[0007] S4, receiving a query request initiated by a user, and when there is a said query object matching the query request stored in the Ignite distributed node, associating the corresponding cache data and feeding-back the same through outputting; otherwise, directly accessing the external data sources to acquire result data matching the query object and feeding-back the result data by outputting.
[0008] Preferably, between S2 and S4, the method further comprises:
[0009] S3: estimating query requests from users, and regularly updating the cache data stored in the Ignite distributed node.
[0010] More preferably, in S1, setting a cache algorithm, and caching query objects obtained using the cache algorithm into a buffer pool in a priority order comprises:
[0011] using the cache algorithm to identify the query objects having high query frequencies and long query elapses; and
[0012] caching the query objects into the buffer pool based on weight results of the identified query objects according to a TopN sort order, in which the weight result is a sum of a weight of the query frequency and a weight of the query elapse.
[0013] Preferably, in S2, based on the query objects, activating a scheduling task so as to extract the cache data from corresponding external data sources and storing the data into an Ignite distributed node comprises:
[0014] creating plural storage tables for caching data sets using SQL syntax by means of creating classes; and
[0015] based on the query objects, activating scheduling tasks to extract the cache data from the corresponding external data sources, respectively, and storing the cache data in one said node or in different said nodes, in which the node comprises at least one storage table.
[0016] Further, after the step of, based on the query objects, activating scheduling tasks to extract the cache data from the corresponding external data sources, respectively, and storing the cache data in one said node or in different said nodes, the method further comprises:
[0017] performing a cache persistence operation on the cache data in the node, so that the cache data can be automatically loaded in a memory of the node during reactivating services.
[0018] Preferably, the step of, when there is a said query object matching the query request stored in the Ignite distributed node, associating the corresponding cache data and feeding-back the same through outputting comprises:
[0019] according to the query request initiated by the user, matching query objects from the corresponding node; and
[0020] associating and matching the query objects from the different external data sources, performing analysis using the SQL syntax, and then outputting feed-back results.
[0021] As compared to the prior art, the method for data query based on Ignite cache architecture of the present invention provides the following beneficial effects:
[0022] The method for data query based on Ignite cache architecture of the present invention begins with the step of setting a cache algorithm. Then query objects are acquired from the corresponding external data sources and cached in a buffer pool in a priority order. Afterward, different storage tables are created using the SQL syntax by means of creating classes for caching data sets. This step is essentially about creating a cache intermediate layer. Further, according to the acquired query objects, scheduling tasks are activated to extract the cache data from the corresponding external data sources. The extracted data are then stored in storage tables of nodes so that in response to a query request from a user, the method determines whether there is any query object matching the query request existing in the cache intermediate layer. If yes, it indicates that the query hits a target, and the corresponding cache data are associated and output as feedback. If there is not any matching query object, it means that the query does not hit any target. In this case, a query can be made using the known OLAP approach to directly access the external data sources to acquire result data matching the query object and feedback the result data by outputting.
[0023] It is thus clear that with the method of the present invention, cache data from various types of external data sources can be drawn into the cache intermediate layer in advance and stored in the Ignite distributed nodes. The cache intermediate layer can, downward, extract the cache data in different types of external data sources, including RDBMS, NoSQL, HDFS, etc., and can, upward, provide unified query services to users. Thereby, when a user initiates a query request, if the query hits a target in the cache intermediate layer, the corresponding cache data can be directly associated and output as feedback, so as to significantly improve query efficiency; and if the query does not hit any target in the cache intermediate layer, the known OLAP approach can be used to generate an output for the query, thereby ensuring feedback of query results.
[0024] In another aspect, the present invention provides a system for data query based on Ignite cache architecture, used in the method for data query based on Ignite cache architecture of the foregoing technical scheme the. The system comprises:
[0025] an algorithm setting unit, for setting a cache algorithm, and caching query objects obtained using the cache algorithm into a buffer pool in a priority order;
[0026] a cache creating unit, for, based on the query objects, activating a scheduling task so as to extract the cache data from corresponding external data sources and storing the cache data into an Ignite distributed node;
[0027] a result outputting unit, for receiving a query request initiated by a user, and when there is a said query object matching the query request stored in the Ignite distributed node, associating the corresponding cache data and feeding-back the same through outputting; otherwise, directly accessing the external data sources to acquire result data matching the query object and feeding-back the result data by outputting.
[0028] Preferably, further comprises:
[0029] a cache updating unit, for estimating query requests from users, and regularly updating the cache data stored in the Ignite distributed node.
[0030] More preferably, the algorithm setting unit comprises:
[0031] a query object module, for identifying the query objects having high query frequencies and long query elapses;
[0032] an identifying module, for caching the query objects into the buffer pool based on weight results of the identified query objects according to a TopN sort order, in which the weight result is a sum of a weight of the query frequency and a weight of the query elapse.
[0033] Preferably, the cache creating unit comprises:
[0034] a storage table creating module, for creating plural storage tables for caching data sets using SQL syntax by means of creating classes; and
[0035] a cache data extracting module, for based on the query objects, activating scheduling tasks to extract the cache data from the corresponding external data sources, respectively, and storing the cache data in one said node or in different said nodes, in which the node at least comprises one said storage table.
[0036] As compared to the prior art, the disclosed system for data query based on Ignite cache architecture provides beneficial effects that are similar to those provided by the disclosed method for data query based on Ignite cache architecture as enumerated above, and thus no repetitions are made herein. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] The accompanying drawings are provided herein for better understanding of the present invention and form a part of this disclosure. The illustrative embodiments and their descriptions are for explaining the present invention and by no means form any improper limitation to the present invention, wherein:
[0038] FIG. 1 is a flowchart of a method for data query based on Ignite cache architecture according to Embodiment 1 of the present invention;
[0039] FIG. 2 is a block diagram of the method of Embodiment 1 of the present invention showing related components; and
[0040] FIG. 3 is a structural diagram of a system for data query based on Ignite cache architecture according to Embodiment 2 of the present invention.
[0041] List of reference numbers:
[0042] 1- algorithm setting unit, 2- cache creating unit;
[0043] 3- cache updating unit, 4- result outputting unit. DETAILED DESCRIPTION OF THE INVENTION
[0044] To make the foregoing objectives, features, and advantages of the present invention clearer and more understandable, the following description will be directed to some embodiments as depicted in the accompanying drawings to detail the technical schemes disclosed in these embodiments. It is, however, to be understood that the embodiments referred herein are only a part of all possible embodiments and thus not exhaustive. Based on the embodiments of the present invention, all the other embodiments can be conceived without creative labor by people of ordinary skill in the art, and all these and other embodiments shall be encompassed in the scope of the present invention.
[0045] Embodiment 1
[0046] Referring to FIG. 1, the present embodiment provides a method for data query based on Ignite cache architecture, which comprises:
[0047] S1, setting a cache algorithm, and caching query objects obtained using the cache algorithm into a buffer pool in a priority order; S2, based on query object activating scheduling task so as to extract the cache data from corresponding external data sources and storing the cache data into an Ignite distributed node; and S4, receiving a query request initiated by a user, and when there is a said query object matching the query request stored in the Ignite distributed node, associating the corresponding cache data and feeding-back the same through outputting; otherwise, directly accessing the external data sources to acquire result data matching the query object and feeding-back the result data by outputting.
[0048] The method for data query based on Ignite cache architecture of the present embodiment, as shown in FIG. 2, begins with the step of setting a cache algorithm. Then query objects are acquired from the corresponding external data sources and cached in a buffer pool in a priority order. Afterward, different storage tables are created using the SQL syntax by means of creating classes for caching data sets. This step is essentially about creating a cache intermediate layer. Further, according to the acquired query objects, scheduling tasks are activated to extract the cache data from the corresponding external data sources. The extracted data are then stored in storage tables of nodes so that in response to a query request from a user, the method determines whether there is any query object matching the query request existing in the cache intermediate layer. If yes, it indicates that the query hits a target, and the corresponding cache data are associated and output as feedback. If there is not any matching query object, it means that the query does not hit any target. In this case, a query can be made using the known OLAP approach to directly access the external data sources to acquire result data matching the query object and feedback the result data by outputting.
[0049] It is thus clear that with the method of the present invention, cache data from various types of external data sources can be drawn into the cache intermediate layer in advance and stored in the Ignite distributed nodes. The cache intermediate layer can, downward, extract the cache data in different types of external data sources, including RDBMS, NoSQL, HDFS, etc., and can, upward, provide unified query services to users. Thereby, when a user initiates a query request, if the query hits a target in the cache intermediate layer, the corresponding cache data can be directly associated and output as feedback, so as to significantly improve query efficiency; and if the query does not hit any target in the cache intermediate layer, the known OLAP approach can be used to generate an output for the query, thereby ensuring feedback of query results. In practical implementations, dynamic addition of external data sources can be simply achieved adding corresponding interfaces for the external data source services, so as to perfectly support transverse expansion of external data sources.
[0050] Optionally, in order to facilitate making queries, the present embodiment further involves providing a unified query services interface for the cache intermediate layers. With the query services interface, data in different external data sources can be queried in a unified manner, which means that users need not to care about the different query syntaxes of different storage media and data storage locations at all, and the desired query result can be easily obtained by calling the query services interface, thereby improving query efficiency and query experience.
[0051] For further expanding the range of data queries, in the embodiment described above, between S2 and S4, the method further comprises: S3: estimating query requests from users, and regularly updating the cache data stored in the Ignite distributed node. For example, if it is expected that there will be many users making the same query request about some hotspot data and the matching same-period data tomorrow, the cache data corresponding to the hotspot data and the matching same-period data can be acquired from the relevant external data sources and stored in the in Ignite distributed nodes in advance for later use. In this manner, once a user makes this expected query request, a query result can be generated very soon, and associated data from different sources can be provided as feedback at the same time. Moreover, this helps estimate the access volume caused by the query request tomorrow, so that the Ignite distributed nodes can update the related cache data in advance. Exemplarily, the cache data may be updated by means cleaning and insertion successively.
[0052] Specifically, in the embodiment described above, the step of S1, setting a cache algorithm, and caching query objects obtained using the cache algorithm into a buffer pool in a priority order comprises:
[0053] using the cache algorithm to identify the query objects having high query frequencies and long query elapses; and caching the query objects into the buffer pool based on weight results of the identified query objects according to a TopN sort order, in which the weight result is the sum of the weight values of query frequency and query elapse.
[0054] In practical implementations, the cache algorithm may have diverse cache strategies. Preferably, the cache strategy is about targeting high-frequency and / or long-elapse objects as the query objects, and assigning weight values to them according to their respective frequencies and / or elapses. The weight results of the query objects are sorted in the TopN order and cached into a buffer pool. It is to be noted that the screening criteria like the query frequencies and / or elapses can be set according to applications in practical implementations, and the present embodiment makes no limitations thereto.
[0055] Further, in the embodiment described previously, the step of, based on the query objects, activating scheduling task so as to extract the cache data from corresponding external data sources and storing the data into an Ignite distributed node comprises:
[0056] creating plural storage tables for caching data sets using SQL syntax by means of creating classes; and based on the query objects, activating scheduling task, respectively, to extract the cache data from the corresponding external data source and storing the cache data in one said node or in different said nodes, in which the node at least comprises one said storage table.
[0057] Specifically, nodes are created through the process described blow. First, setting is made through setIndexedTypes. Afterward, nodes are created using the SQL syntax supported by Ignite, such as CREATE TABLE CashA(FILED1 INT, FIELD2 VARCHAR, FIELD3 VARCHAR, PRIMARY KEY (FIELD1)) WITH \"BACKUPS=1, AFFINITY KEY=FILED1\"). In addition, the table CashA may be set with an index and a primary key. CashA represents the query object of the cache data. Later queries made on the cache data for CashA can be operated using the SQL syntax. It is also to be noted that the number of the nodes may be adapted to the size of the cache data. For a small amount of cache data, only one node may be enough. In this case, all the cache data are stored in the storage table of the node. If the amount of cache data is relatively large, more nodes may be set. In this case, the cache data are distributed across the nodes. Storage of the cache data may be in the form of classes, such as Cache.put(1, new CashA (1,2,3)). Alternatively, an asynchronous form may be used, such as Cache.putAsync(1, new CashA (1,2,3)). The present embodiment makes no limitations thereto.
[0058] Optionally, in the embodiment described previously, after the step of based on the query objects, activating scheduling tasks, respectively, to extract the cache data from the corresponding external data source and storing the cache data in one said node or in different said nodes, the method further comprises: performing a persistence operation on the cache data in the node, so that the cache data can be automatically loaded in a memory of the node during activation of services. In practical implementations, with the persistence operation for the Ignite cache data activated, the cache data are automatically loaded to the memory of the node during reboot of the services. If the persistence operation for the Ignite cache data is deactivated, reboot of the services causes loss of the cache data. It is understandable that the cache data in the node contain the entire data set including indicators and dimensions. The entire cluster comprises the whole data set.
[0059] Preferably, in the embodiment described previously, if the Ignite distributed node stores a query object matching the query request, the step of associating and feeding-back the corresponding cache data through outputting comprises:
[0060] according to the query request initiated by the user, identifying the matching query objects from the corresponding node; and associating and matching the query objects from the different external data sources before performing analysis using the SQL syntax, and then outputting feed-back results.
[0061] In practical business scenarios, a user usually needs to acquire cache data from different external data sources and then associate the queries. For addressing this problem, the present embodiment uses a cache intermediate layer for associated queries of data from different external data sources. For example, purchase order data are stored in ElasticSearch, and member data are stored in PostgreSql. When a user wants to acquire sales data of new and existing buyers, the cache intermediate layer caches the sales data from ElasticSearch to the node, and names this data set as Table A, and caches the member data from PostgreSql to the node, and name this data set as Table B. In response to a query request initiated by the user through the query services interface, the cache intermediate layer performs secondary processing on the cache data based on the query request instruction. In other words, the two parts of cache data from different data sources are associated and eventually only an SQL associated analysis result of Table A and Table Bis output. Hence, with the cache intermediate layer of the present embodiment, data logical processing across data sources can be conveniently achieved and the output query result is more accurate. Exemplarily, the secondary processing for the cache data comprises filtering of the cache results, screening of data rights, and secondary filtering after aggregation.
[0062] Embodiment 2
[0063] Referring to FIG. 1 and FIG. 3, the present embodiment provides a system for data query based on Ignite cache architecture, which comprises:
[0064] an algorithm setting unit 1, for setting a cache algorithm, and caching query objects obtained using the cache algorithm into a buffer pool in a priority order;
[0065] a cache creating unit 2, based on query object activating scheduling task so as to extract the cache data from corresponding external data sources and storing the cache data into an Ignite distributed node; and
[0066] a result outputting unit 4, for receiving a query request initiated by a user, and when there is a said query object matching the query request stored in the Ignite distributed node, associating the corresponding cached data and feeding-back the same through outputting; otherwise, directly accessing the external data sources to acquire result data matching the query object and feeding-back the result data by outputting.
[0067] Preferably, the system further comprises:
[0068] a cache updating unit 3, for estimating query requests from users, and regularly updating the cache data stored in the Ignite distributed node.
[0069] Preferably, algorithm setting unit 1 comprises:
[0070] a query object module, for identifying the query objects having high query frequencies and long query elapses;
[0071] an identifying module, for caching the query objects into the buffer pool based on weight results of the identified query objects according to a TopN sort order, in which the weight result is a sum of a weight of the query frequency and a weight of the query duration.
[0072] Preferably, the cache creating unit 2 comprises:
[0073] a storage table creating module, for creating plural storage tables for caching data sets using SQL syntax by means of creating classes; and
[0074] a cache data extracting module, for based on the query objects, activating scheduling tasks to extract the cached data from the corresponding external data sources, respectively, and storing the cached data in one said node or in different said nodes, in which the node at least comprises one said storage table.
[0075] As compared to the prior art, the disclosed system for data query based on Ignite cache architecture provides beneficial effects that are similar to those provided by the disclosed method for data query based on Ignite cache architecture as enumerated in Embodiment 1, and thus no repetitions are made herein.
[0076] As will be appreciated by people of ordinary skill in the art, implementation of all or a part of the steps of the method of the present invention as described previously may be realized by having a program instruct related hardware components. The program may be stored in a computer-readable storage medium, and the program is about performing the individual steps of the methods described in the foregoing embodiments. The storage medium may be a ROM / RAM, a hard drive, an optical disk, a memory card or the like.
[0077] The present invention has been described with reference to the preferred embodiments and it is understood that the embodiments are not intended to limit the scope of the present invention. Moreover, as the contents disclosed herein should be readily understood and can be implemented by a person skilled in the art, all equivalent changes or modifications which do not depart from the concept of the present invention should be encompassed by the appended claims. Hence, the scope of the present invention shall only be defined by the appended claims.
Claims
1. A system comprising: an algorithm setting unit, configured to: set a cache algorithm; cache query objects obtained using the cache algorithm into a buffer pool in a priority order; a cache creating unit, configured to, based on the query objects, activate a scheduling task to extract cache data from external data sources and store the cache data into a distributed node wherein the cache creating unit includes a storage table creating module, configured to create plural storage tables for caching data sets using structured query language (SQL) syntax by creating classes; a result outputting unit, configured to receive a query request initiated by a user; where there is a query object matching the query request stored in the distributed node, associating the cache data and feeding-back through outputting; and where there is no query object matching the query request stored in the distributed node, directly accessing the external data sources to acquire result data matching the query object and feeding-back the result data by outputting.
2. The system of claim 1, further comprising: a cache updating unit, configured to: estimate query requests from the user; and regularly update the cache data stored in the distributed node.
3. The system of claim 1, wherein the algorithm setting unit comprises: a query object module, configured to identify the query objects having high query frequencies and long query elapses; and an identifying module, configured to cache the query objects into the buffer pool based on weight results of identified query objects according to a TopN sort order, wherein the weight results is a sum of a weight of the query frequency and the weight of the query elapse.
4. The system of claim 1, wherein the cache creating unit comprises: a cache data extracting module, configured to, based on the query objects: activate the scheduling tasks to extract the cache data from the external data sources; and store the cache data in one node or in different node, wherein the node at least comprises of one storage table.
5. The system of claim 4, further comprises: performing a cache persistence operation on the cache data in the node, for the cache data to automatically load in a memory of the node during reactivating services.
6. The system of claim 1, wherein the result outputting unit is further configured to: according to the query request initiated by the user, match the query objects from node; associate and match the query objects from different external data sources; perform analysis using the SQL syntax; and output feed-back results.
7. The system of any one of claims 1 to 6, wherein a cache intermediate layer downwards, extracts the cache data in different types of the external data sources, including relational database management system (RDBMS), NoSQL, hadoop distributed file system (HDFS) and upwards, provide unified query services to the user.
8. The system of any one of claims 1 to 7, wherein the cache data from various types of the external data sources are drawn into the cache intermediate layer in advance and stored in the distributed node.
9. The system of any one of claims 1 to 8, provides a unified query services interface for the cache intermediate layers.
10. The system of any one of claims 1 to 8, wherein a unified query services interface, data in the different external data sources are queried in a unified manner.
11. The system of any one of claims 1 to 10, wherein there are many user making same query request about some hotspot data and matching same-period data tomorrow, the cache data corresponding to the hotspot data and the matching same-period data are acquired from the external data sources and stored in the distributed node in advance.
12. The system of any one of claims 1 to 11, wherein the cache data is updated by means of cleaning and insertion successively.
13. The system of any one of claims 1 to 12, wherein the cache algorithm has diverse cache strategies.
14. The system of any one of claims 1 to 13, wherein cache strategy is targeting high-frequency and / or long-elapse objects as the query objects, and assigning weight values to them according to their respective frequencies and / or elapses.
15. The system of any one of claims 1 to 14, wherein screening criteria are set according to applications in practical implementations.
16. The system of any one of claims 1 to 15, wherein the node are created with setting made through setIndexedTypes, wherein the node are created using the SQL syntax supported by Ignite, including CREATE TABLE CashA(FILED1 INT, FIELD2 VARCHAR, FIELD3 VARCHAR, PRIMARY KEY (FIELD1)) WITH \"BACKUPS=1, AFFINITY KEY=FILED1\").
17. The system of claim 16, wherein the table CashA is set with an index and a primary key.
18. The system of any one of claims 16 to 17, wherein CashA represents the query object of the cache data.
19. The system of any one of claims 16 to 18, wherein queries made on the cache data for the CashA is operated using the SQL syntax.
20. The system of any one of claims 1 to 19, wherein number of the node are adapted to size of the cache data, wherein with a small amount of the cache data, one node is enough.
21. The system of any one of claims 1 to 20, wherein all the cache data are stored in storage table of the node.
22. The system of any one of claims 1 to 21, wherein amount of the cache data is relatively large, more node are set, wherein the cache data are distributed across the node.
23. The system of any one of claims 1 to 22, wherein storage of the cache data is in form of classes, including Cache.put(1, new CashA (1,2,3)), wherein an asynchronous form is used, including Cache.putAsync(1, new CashA (1,2,3)).
24. The system of any one of claims 1 to 23, wherein the cache data are automatically loaded to memory of the node during reboot of services.
25. The system of any one of claims 1 to 24, wherein the cache data in the node contain entire data set including indicators and dimensions, wherein entire cluster comprises whole data set.
26. A device comprising: an algorithm setting unit, configured to: set a cache algorithm; cache query objects obtained using the cache algorithm into a buffer pool in a priority order; a cache creating unit, configured to, based on the query objects, activate a scheduling task to extract cache data from external data sources and store the cache data into a distributed node wherein the cache creating unit includes a storage table creating module, configured to create plural storage tables for caching data sets using structured query language (SQL) syntax by creating classes; a result outputting unit, configured to receive a query request initiated by a user; where there is a query object matching the query request stored in the distributed node, associating the cache data and feeding-back through outputting; and where there is no query object matching the query request stored in the distributed node, directly accessing the external data sources to acquire result data matching the query object and feeding-back the result data by outputting.
27. The device of claim 26, further comprising: a cache updating unit, configured to: estimate query requests from the user; and regularly update the cache data stored in the distributed node.
28. The device of claim 26, wherein the algorithm setting unit comprises: a query object module, configured to identify the query objects having high query frequencies and long query elapses; and an identifying module, configured to cache the query objects into the buffer pool based on weight results of identified query objects according to a TopN sort order, wherein the weight results is a sum of a weight of the query frequency and the weight of the query elapse.
29. The device of claim 26, wherein the cache creating unit comprises: a cache data extracting module, configured to, based on the query objects: activate the scheduling tasks to extract the cache data from the external data sources; and store the cache data in one node or in different node, wherein the node at least comprises of one storage table.
30. The device of claim 29, further comprises: performing a cache persistence operation on the cache data in the node, for the cache data to automatically load in a memory of the node during reactivating services.
31. The device of claim 26, wherein the result outputting unit is further configured to: according to the query request initiated by the user, match the query objects from node; associate and match the query objects from different external data sources; perform analysis using the SQL syntax; and output feed-back results.
32. The device of any one of claims 26 to 31, wherein a cache intermediate layer downwards, extracts the cache data in different types of the external data sources, including relational database management system (RDBMS), NoSQL, hadoop distributed file system (HDFS) and upwards, provide unified query services to the user.
33. The device of any one of claims 26 to 32, wherein the cache data from various types of the external data sources are drawn into the cache intermediate layer in advance and stored in the distributed node.
34. The device of any one of claims 26 to 33, provides a unified query services interface for the cache intermediate layers.
35. The device of any one of claims 26 to 33, wherein a unified query services interface, data in the different external data sources are queried in a unified manner.
36. The device of any one of claims 26 to 35, wherein there are many user making same query request about some hotspot data and matching same-period data tomorrow, the cache data corresponding to the hotspot data and the matching same-period data are acquired from the external data sources and stored in the distributed node in advance.
37. The device of any one of claims 26 to 36, wherein the cache data is updated by means of cleaning and insertion successively.
38. The device of any one of claims 26 to 37, wherein the cache algorithm has diverse cache strategies.
39. The device of any one of claims 26 to 38, wherein cache strategy is targeting high- frequency and / or long-elapse objects as the query objects, and assigning weight values to them according to their respective frequencies and / or elapses.
40. The device of any one of claims 26 to 39, wherein screening criteria are set according to applications in practical implementations.
41. The device of any one of claims 26 to 40, wherein the node are created with setting made through setIndexedTypes, wherein the node are created using the SQL syntax supported by Ignite, including CREATE TABLE CashA(FILED1 INT, FIELD2 VARCHAR, FIELD3 VARCHAR, PRIMARY KEY (FIELD1)) WITH \"BACKUPS=1, AFFINITY KEY=FILED1\").
42. The device of claim 41, wherein the table CashA is set with an index and a primary key.
43. The device of any one of claims 41 to 42, wherein CashA represents the query object of the cache data.
44. The device of any one of claims 41 to 43, wherein queries made on the cache data for the CashA is operated using the SQL syntax.
45. The device of any one of claims 26 to 44, wherein number of the node are adapted to size of the cache data, wherein with a small amount of the cache data, one node is enough.
46. The device of any one of claims 26 to 45, wherein all the cache data are stored in storage table of the node.
47. The device of any one of claims 26 to 46, wherein amount of the cache data is relatively large, more node are set, wherein the cache data are distributed across the node.
48. The device of any one of claims 26 to 47, wherein storage of the cache data is in form of classes, including Cache.put(1, new CashA (1,2,3)), wherein an asynchronous form is used, including Cache.putAsync(1, new CashA (1,2,3)).
49. The device of any one of claims 26 to 48, wherein the cache data are automatically loaded to memory of the node during reboot of services.
50. The device of any one of claims 26 to 49, wherein the cache data in the node contain entire data set including indicators and dimensions, wherein entire cluster comprises whole data set.
51. A method comprising: setting a cache algorithm; caching query objects obtained using the cache algorithm into a buffer pool in a priority order; based on the query objects, activating a scheduling task to extract cache data from external data sources; storing the cache data into a distributed node; creating plural storage tables for caching data sets using structured query language (SQL) syntax by creating classes; receiving a query request initiated by a user; where there is a query object matching the query request stored in the distributed node, associating the cache data and feeding-back through outputting; and where there is no query object matching the query request stored in the distributed node, directly accessing the external data sources to acquire result data matching the query object and feeding-back the result data by outputting.
52. The method of claim 51 further comprises: estimating the query requests from the user; regularly updating the cache data stored in the distributed node.
53. The method of claim 51 or 52, wherein setting the cache algorithm, and caching the query objects obtained using the cache algorithm into the buffer pool in the priority order comprises: using the cache algorithm to identify the query objects having high query frequencies and long query elapses; and caching the query objects into the buffer pool based on weight results of identified query objects according to a TopN sort order, wherein the weight results is a sum of a weight of the query frequency and the weight of the query elapse.
54. The method of claim 51, wherein based on the query objects, activating the scheduling task to extract the cache data from the external data sources and storing the cache data into the distributed node comprises: based on the query objects, activating the scheduling tasks to extract the cache data from the external data sources and storing the cache data in one node or in different node, wherein the node comprises at least one storage table.
55. The method of claim 54, further comprises: performing a cache persistence operation on the cache data in the node, for the cache data to automatically load in a memory of the node during reactivating services.
56. The method of claim 51, wherein when the query object matching the query request stored in the distributed node, associating the cache data and feeding-back through outputting comprises: according to the query request initiated by the user, matching the query objects from node; associating and matching the query objects from different external data sources; performing analysis using the SQL syntax; and outputting feed-back results.
57. The method of any one of claims 51 to 56, wherein a cache intermediate layer downwards, extracts the cache data in different types of the external data sources, including relational database management system (RDBMS), NoSQL, hadoop distributed file system (HDFS) and upwards, provide unified query services to the user.
58. The method of any one of claims 51 to 57, wherein the cache data from various types of the external data sources are drawn into the cache intermediate layer in advance and stored in the distributed node.
59. The method of any one of claims 51 to 58, provides a unified query services interface for the cache intermediate layers.
60. The method of any one of claims 51 to 58, wherein a unified query services interface, data in the different external data sources are queried in a unified manner.
61. The method of any one of claims 51 to 60, wherein there are many user making same query request about some hotspot data and matching same-period data tomorrow, the cache data corresponding to the hotspot data and the matching same-period data are acquired from the external data sources and stored in the distributed node in advance.
62. The method of any one of claims 51 to 61, wherein the cache data is updated by means of cleaning and insertion successively.
63. The method of any one of claims 51 to 62, wherein the cache algorithm has diverse cache strategies.
64. The method of any one of claims 51 to 63, wherein cache strategy is targeting high- frequency and / or long-elapse objects as the query objects, and assigning weight values to them according to their respective frequencies and / or elapses.
65. The method of any one of claims 51 to 64, wherein screening criteria are set according to applications in practical implementations.
66. The method of any one of claims 51 to 65, wherein the node are created with setting made through setIndexedTypes, wherein the node are created using the SQL syntax supported by Ignite, including CREATE TABLE CashA(FILED1 INT, FIELD2 VARCHAR, FIELD3 VARCHAR, PRIMARY KEY (FIELD1)) WITH \"BACKUPS=1, AFFINITY_KEY=FILED1\").
67. The method of claim 66, wherein the table CashA is set with an index and a primary key.
68. The method of any one of claims 66 to 67, wherein CashA represents the query object of the cache data.
69. The method of any one of claims 66 to 68, wherein queries made on the cache data for the CashA is operated using the SQL syntax.
70. The method of any one of claims 51 to 69, wherein number of the node are adapted to size of the cache data, wherein with a small amount of the cache data, one node is enough.
71. The method of any one of claims 51 to 70, wherein all the cache data are stored in storage table of the node.
72. The method of any one of claims 51 to 71, wherein amount of the cache data is relatively large, more node are set, wherein the cache data are distributed across the node.
73. The method of any one of claims 51 to 72, wherein storage of the cache data is in form of classes, including Cache.put(1, new CashA (1,2,3)), wherein an asynchronous form is used, including Cache.putAsync(1, new CashA (1,2,3)).
74. The method of any one of claims 51 to 73, wherein the cache data are automatically loaded to memory of the node during reboot of services.
75. The method of any one of claims 51 to 74, wherein the cache data in the node contain entire data set including indicators and dimensions, wherein entire cluster comprises whole data set.
76. An electronic equipment comprising: one or more processors; a memory, associated with the one or more processors and used for storing a program instruction; the program instruction executed by the one or more processors configured to: set a cache algorithm; cache query objects obtained using the cache algorithm into a buffer pool in a priority order; based on the query objects, activate a scheduling task to extract cache data from external data sources; store the cache data into a distributed node; creating plural storage tables for caching data sets using structured query language (SQL) syntax by creating classes receive a query request initiated by a user; where there is a query object matching the query request stored in the distributed node, associate the cache data and feeding-back through outputting; and where there is no query object matching the query request stored in the distributed node, directly access the external data sources to acquire result data matching the query object and feeding-back the result data by outputting.
77. The equipment of claim 76, further comprises: estimating the query requests from the user; regularly updating the cache data stored in the distributed node.
78. The equipment of claim 76 or 77, wherein setting the cache algorithm, and caching the query objects obtained using the cache algorithm into the buffer pool in the priority order comprises: using the cache algorithm to identify the query objects having high query frequencies and long query elapses; and caching the query objects into the buffer pool based on weight results of identified query objects according to a TopN sort order, wherein the weight results is a sum of a weight of the query frequency and the weight of the query elapse.
79. The equipment of claim 76, wherein based on the query objects, activating the scheduling task to extract the cache data from the external data sources and storing the cache data into the distributed node comprises: based on the query objects, activating the scheduling tasks to extract the cache data from the external data sources and storing the cache data in one node or in different node, wherein the node comprises at least one storage table.
80. The equipment of claim 79, further comprises: performing a cache persistence operation on the cache data in the node, for the cache data to automatically load in memory of the node during reactivating services.
81. The equipment of claim 76, wherein when the query object matching the query request stored in the distributed node, associating the cache data and feeding-back through outputting comprises: according to the query request initiated by the user, matching the query objects from node; associating and matching the query objects from different external data sources; performing analysis using the SQL syntax; and outputting feed-back results.
82. The equipment of any one of claims 76 to 81, wherein a cache intermediate layer downwards, extracts the cache data in different types of the external data sources, including relational database management system (RDBMS), NoSQL, hadoop distributed file system (HDFS) and upwards, provide unified query services to the user.
83. The equipment of any one of claims 76 to 82, wherein the cache data from various types of the external data sources are drawn into the cache intermediate layer in advance and stored in the distributed node.
84. The equipment of any one of claims 76 to 83, provides a unified query services interface for the cache intermediate layers.
85. The equipment of any one of claims 76 to 83, wherein a unified query services interface, data in the different external data sources are queried in a unified manner.
86. The equipment of any one of claims 76 to 85, wherein there are many user making same query request about some hotspot data and matching same-period data tomorrow, the cache data corresponding to the hotspot data and the matching same-period data are acquired from the external data sources and stored in the distributed node in advance.
87. The equipment of any one of claims 76 to 86, wherein the cache data is updated by means of cleaning and insertion successively.
88. The equipment of any one of claims 76 to 87, wherein the cache algorithm has diverse cache strategies.
89. The equipment of any one of claims 76 to 88, wherein cache strategy is targeting high- frequency and / or long-elapse objects as the query objects, and assigning weight values to them according to their respective frequencies and / or elapses.
90. The equipment of any one of claims 76 to 89, wherein screening criteria are set according to applications in practical implementations.
91. The equipment of any one of claims 76 to 90, wherein the node are created with setting made through setIndexedTypes, wherein the node are created using the SQL syntax supported by Ignite, including CREATE TABLE CashA(FILED1 INT, FIELD2 VARCHAR, FIELD3 VARCHAR, PRIMARY KEY (FIELD1)) WITH \"BACKUPS=1, AFFINITY KEY=FILED1\").
92. The equipment of claim 91, wherein the table CashA is set with an index and a primary key.
93. The equipment of any one of claims 91 to 92, wherein CashA represents the query object of the cache data.
94. The equipment of any one of claims 91 to 93, wherein queries made on the cache data for the CashA is operated using the SQL syntax.
95. The equipment of any one of claims 76 to 94, wherein number of the node are adapted to size of the cache data, wherein with a small amount of the cache data, one node is enough.
96. The equipment of any one of claims 76 to 95, wherein all the cache data are stored in storage table of the node.
97. The equipment of any one of claims 76 to 96, wherein amount of the cache data is relatively large, more node are set, wherein the cache data are distributed across the node.
98. The equipment of any one of claims 76 to 97, wherein storage of the cache data is in form of classes, including Cache.put(1, new CashA (1,2,3)), wherein an asynchronous form is used, including Cache.putAsync(1, new CashA (1,2,3)).
99. The equipment of any one of claims 76 to 98, wherein the cache data are automatically loaded to memory of the node during reboot of services.
100. The equipment of any one of claims 76 to 99, wherein the cache data in the node contain entire data set including indicators and dimensions, wherein entire cluster comprises whole data set.
101. A non-transitory computer-readable medium storing computer-executable instructions that, when executed by a computer is configured to: set a cache algorithm; cache query objects obtained using the cache algorithm into a buffer pool in a priority order; based on the query objects, activate a scheduling task to extract cache data from external data sources; store the cache data into a distributed node; creating plural storage tables for caching data sets using structured query language (SQL) syntax by creating classes; receive a query request initiated by a user; where there is a query object matching the query request stored in the distributed node, associate the cache data and feeding-back through outputting; and where there is no query object matching the query request stored in the distributed node, directly access the external data sources to acquire result data matching the query object and feeding-back the result data by outputting.
102. The computer-readable medium of claim 101, further comprises: estimating the query requests from the user; regularly updating the cache data stored in the distributed node.
103. The computer-readable medium of claim 101 or 102, wherein setting the cache algorithm, and caching the query objects obtained using the cache algorithm into the buffer pool in the priority order comprises: using the cache algorithm to identify the query objects having high query frequencies and long query elapses; and caching the query objects into the buffer pool based on weight results of identified query objects according to a TopN sort order, wherein the weight results is a sum of a weight of the query frequency and the weight of the query elapse.
104. The computer-readable medium of claim 101, wherein based on the query objects, activating the scheduling task to extract the cache data from the external data sources and storing the cache data into the distributed node comprises: based on the query objects, activating the scheduling tasks to extract the cache data from the external data sources and storing the cache data in one node or in different node, wherein the node comprises at least one storage table.
105. The computer-readable medium of claim 104, further comprises: performing a cache persistence operation on the cache data in the node, for the cache data to automatically load in a memory of the node during reactivating services.
106. The computer-readable medium of claim 101, wherein when the query object matching the query request stored in the distributed node, associating the cache data and feeding-back through outputting comprises: according to the query request initiated by the user, matching the query objects from node; associating and matching the query objects from different external data sources; performing analysis using the SQL syntax; and outputting feed-back results.
107. The computer-readable medium of any one of claims 101 to 106, wherein a cache intermediate layer downwards, extracts the cache data in different types of the external data sources, including relational database management system (RDBMS), NoSQL, 32adoop distributed file system (HDFS) and upwards, provide unified query services to the user.
108. The computer-readable medium of any one of claims 101 to 107, wherein the cache data from various types of the external data sources are drawn into the cache intermediate layer in advance and stored in the distributed node.
109. The computer-readable medium of any one of claims 101 to 108, provides a unified query services interface for the cache intermediate layers.
110. The computer-readable medium of any one of claims 101 to 108, wherein a unified query services interface, data in the different external data sources are queried in a unified manner.
111. The computer-readable medium of any one of claims 101 to 110, wherein there are many user making same query request about some hotspot data and matching same-period data tomorrow, the cache data corresponding to the hotspot data and the matching same-period data are acquired from the external data sources and stored in the distributed node in advance.
112. The computer-readable medium of any one of claims 101 to 111, wherein the cache data is updated by means of cleaning and insertion successively.
113. The computer-readable medium of any one of claims 101 to 112, wherein the cache algorithm has diverse cache strategies.
114. The computer-readable medium of any one of claims 101 to 113, wherein cache strategy is targeting high-frequency and / or long-elapse objects as the query objects, and assigning weight values to them according to their respective frequencies and / or elapses.
115. The computer-readable medium of any one of claims 101 to 114, wherein screening criteria are set according to applications in practical implementations.
116. The computer-readable medium of any one of claims 101 to 115, wherein the node are created with setting made through setIndexedTypes, wherein the node are created using the SQL syntax supported by Ignite, including CREATE TABLE CashA(FILED1 INT, FIELD2 VARCHAR, FIELD3 VARCHAR, PRIMARY KEY (FIELD1)) WITH \"BACKUPS=1, AFFINITY_KEY=FILED1\").
117. The computer-readable medium of claim 116, wherein the table CashA is set with an index and a primary key.
118. The computer-readable medium of any one of claims 116 to 117, wherein CashA represents the query object of the cache data.
119. The computer-readable medium of any one of claims 116 to 118, wherein queries made on the cache data for the CashA is operated using the SQL syntax.
120. The computer-readable medium of any one of claims 101 to 119, wherein number of the node are adapted to size of the cache data, wherein with a small amount of the cache data, one node is enough.
121. The computer-readable medium of any one of claims 101 to 120, wherein all the cache data are stored in storage table of the node.
122. The computer-readable medium of any one of claims 101 to 121, wherein amount of the cache data is relatively large, more node are set, wherein the cache data are distributed across the node.
123. The computer-readable medium of any one of claims 101 to 122, wherein storage of the cache data is in form of classes, including Cache.put(1, new CashA (1,2,3)), wherein an asynchronous form is used, including Cache.putAsync(1, new CashA (1,2,3)).
124. The computer-readable medium of any one of claims 101 to 123, wherein the cache data are automatically loaded to memory of the node during reboot of services.
125. The computer-readable medium of any one of claims 101 to 124, wherein the cache data in the node contain entire data set including indicators and dimensions, wherein entire cluster comprises whole data set.
126. A system comprising: an algorithm setting unit, configured to: set a cache algorithm; cache query objects obtained using the cache algorithm into a buffer pool in a priority order; and a cache creating unit, configured to, based on the query objects, activate a scheduling task to extract cache data from external data sources and store the cache data into a distributed node wherein the cache creating unit includes a storage table creating module, configured to create plural storage tables for caching data sets using structured query language (SQL) syntax by creating classes.
127. The system of claim 126, further comprising: a result outputting unit, configured to receive a query request initiated by a user; where there is a query object matching the query request stored in the distributed node, associating the cache data and feeding-back through outputting; and there is no query object matching the query request stored in the distributed node, directly accessing the external data sources to acquire result data matching the query object and feeding-back the result data by outputting.
128. The system of claim 127, further comprising: a cache updating unit, configured to: estimate query requests from the user; and regularly update the cache data stored in the distributed node.
129. The system of claim 127, wherein the algorithm setting unit comprises: a query object module, configured to identify the query objects having high query frequencies and long query elapses; and an identifying module, configured to cache the query objects into the buffer pool based on weight results of identified query objects according to a TopN sort order, wherein the weight results is a sum of a weight of the query frequency and the weight of the query elapse.
130. The system of claim 127, wherein the cache creating unit comprises: a cache data extracting module, configured to, based on the query objects: activate the scheduling tasks to extract the cache data from the external data sources; and store the cache data in one node or in different node, wherein the node at least comprises of one storage table.
131. The system of claim 130, further comprises: performing a cache persistence operation on the cache data in the node, for the cache data to automatically load in a memory of the node during reactivating services.
132. The system of claim 127, wherein the result outputting unit is further configured to: according to the query request initiated by the user, match the query objects from node; associate and match the query objects from different external data sources; perform analysis using the SQL syntax; and output feed-back results.
133. The system of any one of claims 126 to 132, wherein a cache intermediate layer downwards, extracts the cache data in different types of the external data sources, including relational database management system (RDBMS), NoSQL, 36adoop distributed file system (HDFS) and upwards, provide unified query services to the user.
134. The system of any one of claims 126 to 133, wherein the cache data from various types of the external data sources are drawn into the cache intermediate layer in advance and stored in the distributed node.
135. The system of any one of claims 126 to 134, provides a unified query services interface for the cache intermediate layers.
136. The system of any one of claims 126 to 134, wherein a unified query services interface, data in the different external data sources are queried in a unified manner.
137. The system of any one of claims 126 to 136, wherein there are many user making same query request about some hotspot data and matching same-period data tomorrow, the cache data corresponding to the hotspot data and the matching same-period data are acquired from the external data sources and stored in the distributed node in advance.
138. The system of any one of claims 126 to 137, wherein the cache data is updated by means of cleaning and insertion successively.
139. The system of any one of claims 126 to 138, wherein the cache algorithm has diverse cache strategies.
140. The system of any one of claims 126 to 139, wherein cache strategy is targeting high- frequency and / or long-elapse objects as the query objects, and assigning weight values to them according to their respective frequencies and / or elapses.
141. The system of any one of claims 126 to 140, wherein screening criteria are set according to applications in practical implementations.
142. The system of any one of claims 126 to 141, wherein the node are created with setting made through setIndexedTypes, wherein the node are created using the SQL syntax supported by Ignite, including CREATE TABLE CashA(FILED1 INT, FIELD2 VARCHAR, FIELD3 VARCHAR, PRIMARY KEY (FIELD1)) WITH \"BACKUPS=1, AFFINITY KEY=FILED1\").
143. The system of claim 142, wherein the table CashA is set with an index and a primary key.
144. The system of any one of claims 142 to 143, wherein CashA represents the query object of the cache data.
145. The system of any one of claims 142 to 144, wherein queries made on the cache data for the CashA is operated using the SQL syntax.
146. The system of any one of claims 126 to 145, wherein number of the node are adapted to size of the cache data, wherein with a small amount of the cache data, one node is enough.
147. The system of any one of claims 126 to 146, wherein all the cache data are stored in storage table of the node.
148. The system of any one of claims 126 to 147, wherein amount of the cache data is relatively large, more node are set, wherein the cache data are distributed across the node.
149. The system of any one of claims 126 to 148, wherein storage of the cache data is in form of classes, including Cache.put(1, new CashA (1,2,3)), wherein an asynchronous form is used, including Cache.putAsync(1, new CashA (1,2,3)).
150. The system of any one of claims 126 to 149, wherein the cache data are automatically loaded to memory of the node during reboot of services.
151. The system of any one of claims 126 to 150, wherein the cache data in the node contain entire data set including indicators and dimensions, wherein entire cluster comprises whole data set.
152. A device comprising: an algorithm setting unit, configured to: set a cache algorithm; cache query objects obtained using the cache algorithm into a buffer pool in a priority order; and a cache creating unit, configured to, based on the query objects, activate a scheduling task to extract cache data from external data sources and store the cache data into a distributed node wherein the cache creating unit includes a storage table creating module, configured to create plural storage tables for caching data sets using structured query language (SQL) syntax by creating classes.
153. The device of claim 152, further comprising: a result outputting unit, configured to receive a query request initiated by a user; where there is a query object matching the query request stored in the distributed node, associating the cache data and feeding-back through outputting; and there is no query object matching the query request stored in the distributed node, directly accessing the external data sources to acquire result data matching the query object and feeding-back the result data by outputting.
154. The device of claim 153, further comprising: a cache updating unit, configured to: estimate query requests from the user; and regularly update the cache data stored in the distributed node.
155. The device of claim 153, wherein the algorithm setting unit comprises: a query object module, configured to identify the query objects having high query frequencies and long query elapses; and an identifying module, configured to cache the query objects into the buffer pool based on weight results of identified query objects according to a TopN sort order, wherein the weight results is a sum of a weight of the query frequency and the weight of the query elapse.
156. The device of claim 153, wherein the cache creating unit comprises: a cache data extracting module, configured to, based on the query objects: activate the scheduling tasks to extract the cache data from the external data sources; and store the cache data in one node or in different node, wherein the node at least comprises of one storage table.
157. The device of claim 156, further comprises: performing a cache persistence operation on the cache data in the node, for the cache data to automatically load in a memory of the node during reactivating services.
158. The device of claim 153, wherein the result outputting unit is further configured to: according to the query request initiated by the user, match the query objects from node; associate and match the query objects from different external data sources; perform analysis using the SQL syntax; and output feed-back results.
159. The device of any one of claims 152 to 158, wherein a cache intermediate layer downwards, extracts the cache data in different types of the external data sources, including relational database management system (RDBMS), NoSQL, 40adoop distributed file system (HDFS) and upwards, provide unified query services to the user.
160. The device of any one of claims 152 to 159, wherein the cache data from various types of the external data sources are drawn into the cache intermediate layer in advance and stored in the distributed node.
161. The device of any one of claims 152 to 160, provides a unified query services interface for the cache intermediate layers.
162. The device of any one of claims 152 to 160, wherein a unified query services interface, data in the different external data sources are queried in a unified manner.
163. The device of any one of claims 152 to 162, wherein there are many user making same query request about some hotspot data and matching same-period data tomorrow, the cache data corresponding to the hotspot data and the matching same-period data are acquired from the external data sources and stored in the distributed node in advance.
164. The device of any one of claims 152 to 163, wherein the cache data is updated by means of cleaning and insertion successively.
165. The device of any one of claims 152 to 164, wherein the cache algorithm has diverse cache strategies.
166. The device of any one of claims 152 to 165, wherein cache strategy is targeting high- frequency and / or long-elapse objects as the query objects, and assigning weight values to them according to their respective frequencies and / or elapses.
167. The device of any one of claims 152 to 166, wherein screening criteria are set according to applications in practical implementations.
168. The device of any one of claims 152 to 167, wherein the node are created with setting made through setIndexedTypes, wherein the node are created using the SQL syntax supported by Ignite, including CREATE TABLE CashA(FILED1 INT, FIELD2 VARCHAR, FIELD3 VARCHAR, PRIMARY KEY (FIELD1)) WITH \"BACKUPS=1, AFFINITY KEY=FILED1\").
169. The device of claim 168, wherein the table CashA is set with an index and a primary key.
170. The device of any one of claims 168 to 169, wherein CashA represents the query object of the cache data.
171. The device of any one of claims 168 to 170, wherein queries made on the cache data for the CashA is operated using the SQL syntax.
172. The device of any one of claims 152 to 171, wherein number of the node are adapted to size of the cache data, wherein with a small amount of the cache data, one node is enough.
173. The device of any one of claims 152 to 172, wherein all the cache data are stored in storage table of the node.
174. The device of any one of claims 152 to 173, wherein amount of the cache data is relatively large, more node are set, wherein the cache data are distributed across the node.
175. The device of any one of claims 152 to 174, wherein storage of the cache data is in form of classes, including Cache.put(1, new CashA (1,2,3)), wherein an asynchronous form is used, including Cache.putAsync(1, new CashA (1,2,3)).
176. The device of any one of claims 152 to 175, wherein the cache data are automatically loaded to memory of the node during reboot of services.
177. The device of any one of claims 152 to 176, wherein the cache data in the node contain entire data set including indicators and dimensions, wherein entire cluster comprises whole data set.
178. A method comprising: setting a cache algorithm; caching query objects obtained using the cache algorithm into a buffer pool in a priority order; based on the query objects, activating a scheduling task to extract cache data from external data sources; storing the cache data into a distributed node; and creating plural storage tables for caching data sets using structured query language (SQL) syntax by creating classes.
179. The method of claim 178, further comprises: receiving a query request initiated by a user; where there is a query object matching the query request stored in the distributed node, associating the cache data and feeding-back through outputting; and where there is no query object matching the query request stored in the distributed node, directly accessing the external data sources to acquire result data matching the query object and feeding-back the result data by outputting.
180. The method of claim 179, further comprises: estimating the query requests from the user; regularly updating the cache data stored in the distributed node.
181. The method of claim 179 or 180, wherein setting the cache algorithm, and caching the query objects obtained using the cache algorithm into the buffer pool in the priority order comprises: using the cache algorithm to identify the query objects having high query frequencies and long query elapses; and caching the query objects into the buffer pool based on weight results of identified query objects according to a TopN sort order, wherein the weight results is a sum of a weight of the query frequency and the weight of the query elapse.
182. The method of claim 179, wherein based on the query objects, activating the scheduling task to extract the cache data from the external data sources and storing the cache data into the distributed node comprises: based on the query objects, activating the scheduling tasks to extract the cache data from the external data sources and storing the cache data in one node or in different node, wherein the node comprises at least one storage table.
183. The method of claim 182, further comprises: performing a cache persistence operation on the cache data in the node, for the cache data to automatically load in a memory of the node during reactivating services.
184. The method of claim 179, wherein when the query object matching the query request stored in the distributed node, associating the cache data and feeding-back through outputting comprises: according to the query request initiated by the user, matching the query objects from node; associating and matching the query objects from different external data sources; performing analysis using the SQL syntax; and outputting feed-back results.
185. The method of any one of claims 178 to 184, wherein a cache intermediate layer downwards, extracts the cache data in different types of the external data sources, including relational database management system (RDBMS), NoSQL, 44 adoop distributed file system (HDFS) and upwards, provide unified query services to the user.
186. The method of any one of claims 178 to 185, wherein the cache data from various types of the external data sources are drawn into the cache intermediate layer in advance and stored in the distributed node.
187. The method of any one of claims 178 to 186, provides a unified query services interface for the cache intermediate layers.
188. The method of any one of claims 178 to 186, wherein a unified query services interface, data in the different external data sources are queried in a unified manner.
189. The method of any one of claims 178 to 188, wherein there are many user making same query request about some hotspot data and matching same-period data tomorrow, the cache data corresponding to the hotspot data and the matching same-period data are acquired from the external data sources and stored in the distributed node in advance.
190. The method of any one of claims 178 to 189, wherein the cache data is updated by means of cleaning and insertion successively.
191. The method of any one of claims 178 to 190, wherein the cache algorithm has diverse cache strategies.
192. The method of any one of claims 178 to 191, wherein cache strategy is targeting high- frequency and / or long-elapse objects as the query objects, and assigning weight values to them according to their respective frequencies and / or elapses.
193. The method of any one of claims 178 to 192, wherein screening criteria are set according to applications in practical implementations.
194. The method of any one of claims 178 to 193, wherein the node are created with setting made through setIndexedTypes, wherein the node are created using the SQL syntax supported by Ignite, including CREATE TABLE CashA(FILED1 INT, FIELD2 VARCHAR, FIELD3 VARCHAR, PRIMARY KEY (FIELD1)) WITH \"BACKUPS=1, AFFINITY KEY=FILED1\").
195. The method of claim 194, wherein the table CashA is set with an index and a primary key.
196. The method of any one of claims 194 to 195, wherein CashA represents the query object of the cache data.
197. The method of any one of claims 194 to 196, wherein queries made on the cache data for the CashA is operated using the SQL syntax.
198. The method of any one of claims 178 to 197, wherein number of the node are adapted to size of the cache data, wherein with a small amount of the cache data, one node is enough.
199. The method of any one of claims 178 to 198, wherein all the cache data are stored in storage table of the node.
200. The method of any one of claims 178 to 199, wherein amount of the cache data is relatively large, more node are set, wherein the cache data are distributed across the node.
201. The method of any one of claims 178 to 200, wherein storage of the cache data is in form of classes, including Cache.put(1, new CashA (1,2,3)), wherein an asynchronous form is used, including Cache.putAsync(1, new CashA (1,2,3)).
202. The method of any one of claims 178 to 201, wherein the cache data are automatically loaded to memory of the node during reboot of services.
203. The method of any one of claims 178 to 202, wherein the cache data in the node contain entire data set including indicators and dimensions, wherein entire cluster comprises whole data set.
204. An electronic equipment comprising: one or more processors; a memory, associated with the one or more processors and used for storing a program instruction; the program instruction executed by the one or more processors configured to: set a cache algorithm; cache query objects obtained using the cache algorithm into a buffer pool in a priority order; based on the query objects, activate a scheduling task to extract cache data from external data sources; store the cache data into a distributed node; and creating plural storage tables for caching data sets using structured query language (SQL) syntax by creating classes.
205. The equipment of claim 204, further comprises: receiving a query request initiated by a user; where there is a query object matching the query request stored in the distributed node, associating the cache data and feeding-back through outputting; and where there is no query object matching the query request stored in the distributed node, directly accessing the external data sources to acquire result data matching the query object and feeding-back the result data by outputting.
206. The equipment of claim 205, further comprises: estimating the query requests from the user; regularly updating the cache data stored in the distributed node.
207. The equipment of claim 205 or 206, wherein setting the cache algorithm, and caching the query objects obtained using the cache algorithm into the buffer pool in the priority order comprises: using the cache algorithm to identify the query objects having high query frequencies and long query elapses; and caching the query objects into the buffer pool based on weight results of identified query objects according to a TopN sort order, wherein the weight results is a sum of a weight of the query frequency and the weight of the query elapse.
208. The equipment of claim 205, wherein based on the query objects, activating the scheduling task to extract the cache data from the external data sources and storing the cache data into the distributed node comprises: based on the query objects, activating the scheduling tasks to extract the cache data from the external data sources and storing the cache data in one node or in different node, wherein the node comprises at least one storage table.
209. The equipment of claim 208, further comprises: performing a cache persistence operation on the cache data in the node, for the cache data to automatically load in memory of the node during reactivating services.
210. The equipment of claim 205, wherein when the query object matching the query request stored in the distributed node, associating the cache data and feeding-back through outputting comprises: according to the query request initiated by the user, matching the query objects from node; associating and matching the query objects from different external data sources; performing analysis using the SQL syntax; and outputting feed-back results.
211. The equipment of any one of claims 204 to 210, wherein a cache intermediate layer downwards, extracts the cache data in different types of the external data sources, including relational database management system (RDBMS), NoSQL, hadoop distributed file system (HDFS) and upwards, provide unified query services to the user.
212. The equipment of any one of claims 204 to 211, wherein the cache data from various types of the external data sources are drawn into the cache intermediate layer in advance and stored in the distributed node.
213. The equipment of any one of claims 204 to 212, provides a unified query services interface for the cache intermediate layers.
214. The equipment of any one of claims 204 to 212, wherein a unified query services interface, data in the different external data sources are queried in a unified manner.
215. The equipment of any one of claims 204 to 214, wherein there are many user making same query request about some hotspot data and matching same-period data tomorrow, the cache data corresponding to the hotspot data and the matching same-period data are acquired from the external data sources and stored in the distributed node in advance.
216. The equipment of any one of claims 204 to 215, wherein the cache data is updated by means of cleaning and insertion successively.
217. The equipment of any one of claims 204 to 216, wherein the cache algorithm has diverse cache strategies.
218. The equipment of any one of claims 204 to 217, wherein cache strategy is targeting high- frequency and / or long-elapse objects as the query objects, and assigning weight values to them according to their respective frequencies and / or elapses.
219. The equipment of any one of claims 204 to 218, wherein screening criteria are set according to applications in practical implementations.
220. The equipment of any one of claims 204 to 219, wherein the node are created with setting made through setIndexedTypes, wherein the node are created using the SQL syntax supported by Ignite, including CREATE TABLE CashA(FILED1 INT, FIELD2 VARCHAR, FIELD3 VARCHAR, PRIMARY KEY (FIELD1)) WITH \"BACKUPS=1, AFFINITY KEY=FILED1\").
221. The equipment of claim 220, wherein the table CashA is set with an index and a primary key.
222. The equipment of any one of claims 220 to 221, wherein CashA represents the query object of the cache data.
223. The equipment of any one of claims 220 to 222, wherein queries made on the cache data for the CashA is operated using the SQL syntax.
224. The equipment of any one of claims 204 to 223, wherein number of the node are adapted to size of the cache data, wherein with a small amount of the cache data, one node is enough.
225. The equipment of any one of claims 204 to 224, wherein all the cache data are stored in storage table of the node.
226. The equipment of any one of claims 204 to 225, wherein amount of the cache data is relatively large, more node are set, wherein the cache data are distributed across the node.
227. The equipment of any one of claims 204 to 226, wherein storage of the cache data is in form of classes, including Cache.put(1, new CashA (1,2,3)), wherein an asynchronous form is used, including Cache.putAsync(1, new CashA (1,2,3)).
228. The equipment of any one of claims 204 to 227, wherein the cache data are automatically loaded to memory of the node during reboot of services.
229. The equipment of any one of claims 204 to 228, wherein the cache data in the node contain entire data set including indicators and dimensions, wherein entire cluster comprises whole data set.
230. A non-transitory computer-readable medium storing computer-executable instructions that, when executed by a computer is configured to: set a cache algorithm; cache query objects obtained using the cache algorithm into a buffer pool in a priority order; based on the query objects, activate a scheduling task to extract cache data from external data sources; store the cache data into a distributed node; and creating plural storage tables for caching data sets using structured query language (SQL) syntax by creating classes.
231. The computer-readable medium of claim 230, further comprises: receiving a query request initiated by a user; where there is a query object matching the query request stored in the distributed node, associating the cache data and feeding-back through outputting; and where there is no query object matching the query request stored in the distributed node, directly accessing the external data sources to acquire result data matching the query object and feeding-back the result data by outputting.
232. The computer-readable medium of claim 231, further comprises: estimating the query requests from the user; regularly updating the cache data stored in the distributed node.
233. The computer-readable medium of claim 231 or 232, wherein setting the cache algorithm, and caching the query objects obtained using the cache algorithm into the buffer pool in the priority order comprises: using the cache algorithm to identify the query objects having high query frequencies and long query elapses; and caching the query objects into the buffer pool based on weight results of identified query objects according to a TopN sort order, wherein the weight results is a sum of a weight of the query frequency and the weight of the query elapse.
234. The computer-readable medium of claim 231, wherein based on the query objects, activating the scheduling task to extract the cache data from the external data sources and storing the cache data into the distributed node comprises: based on the query objects, activating the scheduling tasks to extract the cache data from the external data sources and storing the cache data in one node or in different node, wherein the node comprises at least one storage table.
235. The computer-readable medium of claim 234, further comprises: performing a cache persistence operation on the cache data in the node, for the cache data to automatically load in a memory of the node during reactivating services.
236. The computer-readable medium of claim 231, wherein when the query object matching the query request stored in the distributed node, associating the cache data and feeding-back through outputting comprises: according to the query request initiated by the user, matching the query objects from node; associating and matching the query objects from different external data sources; performing analysis using the SQL syntax; and outputting feed-back results.
237. The computer-readable medium of any one of claims 230 to 236, wherein a cache intermediate layer downwards, extracts the cache data in different types of the external data sources, including relational database management system (RDBMS), NoSQL, hadoop distributed file system (HDFS) and upwards, provide unified query services to the user.
238. The computer-readable medium of any one of claims 230 to 237, wherein the cache data from various types of the external data sources are drawn into the cache intermediate layer in advance and stored in the distributed node.
239. The computer-readable medium of any one of claims 230 to 238, provides a unified query services interface for the cache intermediate layers.
240. The computer-readable medium of any one of claims 230 to 238, wherein a unified query services interface, data in the different external data sources are queried in a unified manner.
241. The computer-readable medium of any one of claims 230 to 240, wherein there are many user making same query request about some hotspot data and matching same-period data tomorrow, the cache data corresponding to the hotspot data and the matching same-period data are acquired from the external data sources and stored in the distributed node in advance.
242. The computer-readable medium of any one of claims 230 to 241, wherein the cache data is updated by means of cleaning and insertion successively.
243. The computer-readable medium of any one of claims 230 to 242, wherein the cache algorithm has diverse cache strategies.
244. The computer-readable medium of any one of claims 230 to 243, wherein cache strategy is targeting high-frequency and / or long-elapse objects as the query objects, and assigning weight values to them according to their respective frequencies and / or elapses.
245. The computer-readable medium of any one of claims 230 to 244, wherein screening criteria are set according to applications in practical implementations.
246. The computer-readable medium of any one of claims 230 to 245, wherein the node are created with setting made through setIndexedTypes, wherein the node are created using the SQL syntax supported by Ignite, including CREATE TABLE CashA(FILED1 INT, FIELD2 VARCHAR, FIELD3 VARCHAR, PRIMARY KEY (FIELD1)) WITH \"BACKUPS=1, AFFINITY KEY=FILED1\").
247. The computer-readable medium of claim 246, wherein the table CashA is set with an index and a primary key.
248. The computer-readable medium of any one of claims 246 to 247, wherein CashA represents the query object of the cache data.
249. The computer-readable medium of any one of claims 246 to 248, wherein queries made on the cache data for the CashA is operated using the SQL syntax.
250. The computer-readable medium of any one of claims 230 to 249, wherein number of the node are adapted to size of the cache data, wherein with a small amount of the cache data, one node is enough.
251. The computer-readable medium of any one of claims 230 to 250, wherein all the cache data are stored in storage table of the node.
252. The computer-readable medium of any one of claims 230 to 251, wherein amount of the cache data is relatively large, more node are set, wherein the cache data are distributed across the node.
253. The computer-readable medium of any one of claims 230 to 252, wherein storage of the cache data is in form of classes, including Cache.put(1, new CashA (1,2,3)), wherein an asynchronous form is used, including Cache.putAsync(1, new CashA (1,2,3)).
254. The computer-readable medium of any one of claims 230 to 253, wherein the cache data are automatically loaded to memory of the node during reboot of services.
255. The computer-readable medium of any one of claims 230 to 254, wherein the cache data in the node contain entire data set including indicators and dimensions, wherein entire cluster comprises whole data set.