Data query method and device and computer equipment

Abstract

Embodiments of the invention provide a data query method and apparatus, and a computer device. The method comprises the steps of receiving a data query request; analyzing the data query request to obtain a query dimension in the data query request; judging whether the query dimension is matched with a pre-constructed index dimension of data stored in a database or not; if yes, the query request is distributed to a first engine; if not, the query request is distributed to a second engine; and obtaining a query result of the data query request returned by the first engine or the second engine. According to the method, the query dimension is matched with the index dimension, the advantages of the first engine and the second engine are utilized to the maximum extent, it is ensured that each query request can be processed by the most appropriate engine, the query efficiency is improved to the maximum extent, the data expansion rate is reduced, and storage resources are saved.

Description

Technical Field

[0001] The embodiments of this specification relate to the field of big data technology, and in particular to a data query method, apparatus, and computer device. Background Art

[0002] In the prior art, for multidimensional model queries, the data model is usually built in Kylin in advance and index data is built by batch building. When the BI (Business Intelligence) tool queries data, the Kylin model responds to the query to return the query results and display the query results in the BI tool. However, due to the diversity of BI tools and the complexity of application scenarios, it is difficult to prejudge the actual query method, so two solutions are often used when building index data. One of the methods is to build indexes for all dimensions of the data. However, this method will result in low query efficiency, and due to the construction of too many indexes, it will lead to a high data expansion rate, which will cause a great waste of storage resources. Another way is to build partial indexes based on business experience. However, since some query scenarios cannot be predicted in advance, some queries may not hit the model, which will greatly reduce query efficiency.

[0003] In view of this, the embodiments of this specification aim to provide a data query method, apparatus and computer device. Summary of the invention

[0004] In view of the above problems in the prior art, the purpose of the embodiments of this specification is to provide a data query method, apparatus and computer device to solve the problem that the existing data query technology cannot take into account both query efficiency and high expansion rate due to building too many indexes.

[0005] In order to solve the above technical problems, the specific technical solutions of the embodiments of this specification are as follows:

[0006] In a first aspect, an embodiment of the present specification provides a data query method, including:

[0007] Receive data query requests;

[0008] Analyze the data query request to obtain a query dimension in the data query request;

[0009] Determining whether the query dimension matches a pre-built index dimension of data stored in the database;

[0010] If there is a match, the query request is dispatched to the first engine;

[0011] If there is no match, the query request is sent to the second engine;

[0012] Obtain a query result of the data query request returned by the first engine or the second engine.

[0013] Specifically, after dispatching the query request to the second engine, the method further includes:

[0014] A first historical data query request dispatched to the second engine during a collection monitoring period;

[0015] Calculating the usage frequency of each query dimension in the first historical data query request;

[0016] Determining whether the usage frequency is greater than or equal to a preset first frequency threshold;

[0017] If so, the query dimension whose usage frequency is greater than or equal to the first frequency threshold is used as the recommended index dimension to be constructed.

[0018] Specifically, after dispatching the query request to the first engine, the method further includes:

[0019] A second historical data query request dispatched to the first engine during the acquisition monitoring period;

[0020] Calculate the usage frequency of each index dimension corresponding to the second historical data query request;

[0021] Determining whether the usage frequency is less than a preset second frequency threshold;

[0022] If so, the index dimension whose usage frequency is less than the preset second frequency threshold is used as the recommended index dimension to be deleted.

[0023] Preferably, the first engine is the Kylin engine, and the second engine is the ClickHouse engine.

[0024] Specifically, before receiving the data query request, the method further includes:

[0025] Receive access documents;

[0026] Collecting data in the access document and converting the format;

[0027] The format-converted data is stored in the database.

[0028] Furthermore, the method further comprises:

[0029] Perform preliminary aggregation on the format-converted data to obtain the pre-built index dimensions.

[0030] In a second aspect, an embodiment of the present specification provides a data query device, including:

[0031] A receiving module, used for receiving a data query request;

[0032] An analysis module, used to analyze the data query request to obtain a query dimension in the data query request;

[0033] A determination module, used to determine whether the query dimension matches a pre-built index dimension of the data stored in the database;

[0034] A first dispatching module, configured to dispatch the query request to a first engine if there is a match;

[0035] A second dispatching module, configured to dispatch the query request to a second engine if there is no match;

[0036] The acquisition module is used to acquire the query result of the data query request returned by the first engine or the second engine.

[0037] In a third aspect, an embodiment of the present specification provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method provided by the above technical solution when executing the computer program.

[0038] In a fourth aspect, an embodiment of the present specification provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method provided by the above technical solution is implemented.

[0039] In a fifth aspect, an embodiment of the present specification provides a computer program product, comprising at least one instruction or at least one program, wherein the at least one instruction or the at least one program is loaded and executed by a processor to implement the method provided by the above technical solution.

[0040] By adopting the above technical solution, a data query method, apparatus and computer device provided in the embodiments of this specification maximize the advantages of the first engine and the second engine by matching the query dimension with the index dimension, ensuring that each query request can be processed by the most appropriate engine, while maximizing the query efficiency, reducing the data expansion rate and saving storage resources.

[0041] In order to make the above and other purposes, features and advantages of the embodiments of this specification more obvious and easy to understand, the following specifically cites preferred embodiments and describes them in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the embodiments of this specification or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0043] Figure 1 A schematic diagram showing the steps of a data query method provided by an embodiment of this specification is shown;

[0044] Figure 2 A schematic diagram of the steps of determining the recommended index temperature to be constructed in the embodiment of this specification is shown;

[0045] Figure 3 A schematic diagram showing the steps of determining the recommended index temperature to be deleted in an embodiment of this specification is shown;

[0046] Figure 4 Another schematic diagram of the steps of the data query method in the embodiment of this specification is shown;

[0047] Figure 5 A schematic diagram of the structure of a data query device provided in an embodiment of this specification is shown;

[0048] Figure 6 A schematic diagram of the structure of a computer device provided in an embodiment of this specification is shown.

[0049] Description of the accompanying symbols:

[0050] 51. Receiving module;

[0051] 52. Analysis module;

[0052] 53. Judgment module;

[0053] 54. The first distribution module;

[0054] 55. The second distribution module;

[0055] 56. Get module;

[0056] 602. Computer equipment;

[0057] 604, processor;

[0058] 606. Memory;

[0059] 608, driving mechanism;

[0060] 610, input / output module;

[0061] 612. Input devices;

[0062] 614. Output device;

[0063] 616. Presentation equipment;

[0064] 618. Graphical user interface;

[0065] 620, network interface;

[0066] 622, communication link;

[0067] 624. Communication bus. DETAILED DESCRIPTION

[0068] The following will be combined with the drawings in the embodiments of this specification to clearly and completely describe the technical solutions in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this specification.

[0069] It should be noted that the terms "first", "second", etc. in this specification and claims and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of this specification described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, device, product or equipment that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0070] In order to solve the above problems, the embodiments of this specification provide a data query method, apparatus and computer device, which can solve the problem that the existing data query technology cannot take into account both query efficiency and high expansion rate due to building too many indexes. Figure 1 This is a step diagram of a data query method provided in an embodiment of this specification. This specification provides method operation steps as described in the embodiment or flowchart, but may include more or fewer operation steps based on conventional or non-creative labor. The order of steps listed in the embodiment is only one way of executing the steps among many orders, and does not represent the only order of execution. When the actual system or device product is executed, it can be executed in the order of the method shown in the embodiment or the accompanying drawings or in parallel. Specifically, Figure 1 As shown, the method may include:

[0071] S101: receiving a data query request.

[0072] In some specific embodiments, an OLAP (Online Analytical Processing) integration engine may receive a data query request sent through a BI (Business Intelligence) tool.

[0073] S102: Analyze the data query request to obtain a query dimension in the data query request.

[0074] S103: Determine whether the query dimension matches a pre-built index dimension of data stored in the database.

[0075] S104: If there is a match, the query request is dispatched to the first engine.

[0076] S105: If there is no match, the query request is sent to the second engine.

[0077] Furthermore, the OLAP integration engine automatically routes the data query request to the Kylin engine or the Clickhouse engine according to the matching result. In the embodiment of this specification, the first engine is the Kylin engine, and the second engine is preferably the Clickhouse engine.

[0078] Kylin engine and Clickhouse engine are both excellent OLAP engines in the field of big data processing. Among them, Kylin engine uses multidimensional data storage and pre-computation technology, supports multidimensional OLAP analysis, and can perform complex queries and aggregation operations on large-scale data sets; thus, when the query dimension in the data query request matches the pre-built index dimension, fast query can be achieved. Clickhouse uses columnar storage, which can significantly reduce I / O and data reading when executing queries involving only some columns; thus, when the query dimension in the data query request does not match the pre-built index dimension, the Clickhouse engine can be used to query only some fields in the data table instead of all fields, thereby improving query speed.

[0079] S106: Receive a query result of the first engine or the second engine in response to the data query request.

[0080] By matching the query dimension with the index dimension, the data query method provided in the embodiments of this specification maximizes the use of the aggregate query advantages of the Kylin engine and the detailed query advantages of the Clickhouse engine, thereby ensuring that each query request can be processed by the most appropriate engine to maximize the query efficiency, thereby improving the comprehensive query efficiency of the OLAP integrated engine by an average of 3-10 times.

[0081] like Figure 2 As shown, in the embodiment of this specification, in step S105, after dispatching the query request to the second engine, the method further includes:

[0082] S201: collecting a first historical data query request sent to the second engine during a monitoring period.

[0083] S202: Calculate the first usage frequency of each query dimension in the first historical data query request.

[0084] In the embodiments of this specification, the monitoring period may be one week, one month, one quarter, etc. The first usage frequency is adapted to the monitoring period and may be set according to actual applications, and after the setting is completed, it may also be adjusted according to actual applications.

[0085] S203: Determine whether the first usage frequency is greater than or equal to a preset first frequency threshold.

[0086] S204: If yes, taking the query dimension whose first usage frequency is greater than or equal to the first frequency threshold as a recommended index dimension to be constructed.

[0087] The data query method provided in the embodiments of this specification can intelligently recommend query dimensions with higher user query frequency as index dimensions to be constructed and construct them according to query requirements. In subsequent data queries, if there is a query dimension that matches the newly constructed index dimension, the data query request will be dispatched to the Kylin engine to improve query performance.

[0088] In some feasible embodiments, the method may further include:

[0089] Input the recommended index dimensions to be constructed into the pre-trained recommendation prediction model, and output the recommendation index of each recommended index dimension to be constructed;

[0090] Then, according to the recommendation index, index dimensions are constructed for at least part of the recommended index dimensions to be constructed.

[0091] The recommendation prediction model is trained based on historical data query request data and query business environment data. The business environment data at least includes one or a combination of resource occupancy data, index statistical information, and operation indicator data generated during the execution of the data query request.

[0092] Among them, the resource occupancy data indicates the load performance data of the data query request during the execution process, and the resource occupancy data at least includes one or a combination of several of the data such as processor speed, IO retrieval time, IO transmission speed, maximum IO throughput, IO number, memory occupancy rate, cache hit rate, processor occupancy rate, etc.

[0093] Index statistics indicate how index dimensions are used by data query requests. Optimizing index dimensions can be used to optimize query request execution plans and improve query performance. Index statistics include at least one or a combination of index query times and index hit rates.

[0094] The operation indicator data is used to indicate the data generated by executing data query requests and the data measuring the operation indicators of the execution of data query requests. The operation indicator data at least includes one or a combination of the number of parsing calls, the number of disk reads, the number of cache reads, the number of processed rows, the number of executions, the processing runtime, the runtime of the query statement, the average parsing time, the number of sorting times, the shared memory, the number of physical read requests, the number of physical read bytes, the physical write requests, and the number of physical write bytes.

[0095] By predicting the recommendation index for the recommended index dimensions to be built, the accuracy of index construction can be improved, and the efficiency of index dimension update optimization can be improved; thereby increasing the hit rate of data query requests for the built index dimensions and ensuring query efficiency.

[0096] Furthermore, if Figure 3 As shown, in the embodiment of this specification, in step S104, after dispatching the query request to the first engine, the method further includes:

[0097] S301: collecting a second historical data query request sent to the first engine during a monitoring period.

[0098] S302: Calculate the second usage frequency of each index dimension corresponding to the second historical data query request.

[0099] S303: Determine whether the second usage frequency is less than a preset second frequency threshold.

[0100] In the embodiments of this specification, the monitoring period for collecting the second historical data query request can be different from or the same as the monitoring period for collecting the first historical data query request; if they are the same, the first historical data query request and the second historical data query request are collected at the same time in the same monitoring period. Similar to the first usage frequency, the second usage frequency can also be set according to actual applications, and after the setting is completed, it can also be adjusted according to actual applications.

[0101] S304: If yes, taking the index dimension whose second usage frequency is less than the preset second frequency threshold as the recommended index dimension to be deleted.

[0102] The data query method provided in the embodiments of this specification can reduce the data expansion rate and save storage resources by deleting the index dimensions with the highest usage frequency.

[0103] In summary, the data query method provided in the embodiments of this specification realizes dynamic adjustment and optimization of index dimensions and improves query performance by recommending index dimensions to be constructed and index dimensions to be deleted.

[0104] In some feasible embodiments, the method may further include:

[0105] Input the recommended index dimensions to be deleted into the pre-trained deletion prediction model, and output the deletion index of each recommended index dimension to be deleted;

[0106] Then, according to the deletion index, at least some index dimensions are selected from the recommended index dimensions to be deleted for deletion.

[0107] By predicting the deletion index of the recommended index dimension to be deleted, the accuracy of index dimension deletion can be improved, the efficiency of index dimension update optimization can be improved, and the occupation of storage resources by the index dimension can be avoided.

[0108] like Figure 4 As shown, before step S101: receiving a data query request, the method further includes:

[0109] S401: Receive access document.

[0110] S402: Collect data in the access document and perform format conversion.

[0111] S403: storing the format-converted data in a database.

[0112] That is, the data query method provided in the embodiment of this specification also includes a data access parsing stage before receiving the data query request. This stage needs to parse the access document submitted by the business, which specifically includes an automatic modeling process and a data import process. Automatic modeling is to convert the data in the access document into a data model suitable for analysis. Data import is to import the data into the corresponding data warehouse. Usually, the data warehouse can be selected as Clickhouse, and Kylin is connected to Clickhouse.

[0113] Furthermore, the method may also include:

[0114] S404: Preliminary aggregation is performed on the format-converted data to obtain pre-built index dimensions.

[0115] That is, the data access and analysis stage can further include a basic index construction process, which performs preliminary aggregation on the data to obtain constructed index dimensions so that subsequent queries are more efficient.

[0116] It should be noted that, in some feasible embodiments, the basic index construction process may not be performed after the data in the access document is stored in the database, that is, the initial index dimension is empty. At this time, when responding to the user's data query request, it will be determined that the query dimension in each data query request cannot be matched to the corresponding index dimension, so each data query request is dispatched to the second engine (i.e., Clickhouse engine). And in subsequent applications, after determining the recommended index dimension to be constructed using the collected first historical data query request, the index dimension is constructed according to the recommended index dimension to be constructed.

[0117] In summary, the data query method provided by the embodiments of this specification can achieve efficient and rapid response to business query requests without building an index in advance or building only a small number of indexes; it greatly reduces the requirements for developer capabilities and improves query efficiency. Subsequently, the index dimensions to be built and the index dimensions to be deleted are intelligently recommended, and the index dimensions are dynamically updated. As time goes by, the query performance will continue to improve, taking into account the query efficiency of query requests and the economic efficiency of data expansion rate and storage.

[0118] Based on the above-described data query method, the embodiments of this specification also provide a data query device. The device may include a system (including a distributed system), software (application), module, component, server, client, etc. using the method described in the embodiments of this specification and a device combined with necessary implementation hardware. Based on the same innovative concept, the device in one or more embodiments provided in the embodiments of this specification is as described in the following embodiments. Since the implementation scheme and method for solving the problem of the device are similar, the implementation of the specific device in the embodiments of this specification can refer to the implementation of the aforementioned method, and the repetitions will not be repeated. As used below, the term "unit" or "module" can implement a combination of software and / or hardware of a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.

[0119] like Figure 5 As shown, a data query device provided in an embodiment of this specification includes:

[0120] A receiving module 51, used for receiving a data query request;

[0121] An analysis module 52, configured to analyze the data query request to obtain a query dimension in the data query request;

[0122] A determination module 53, used to determine whether the query dimension matches a pre-built index dimension of the data stored in the database;

[0123] A first dispatching module 54, configured to dispatch the query request to a first engine if there is a match;

[0124] A second dispatching module 55, configured to dispatch the query request to a second engine if there is no match;

[0125] The acquisition module 56 is used to acquire the query result of the data query request returned by the first engine or the second engine.

[0126] The beneficial effects obtained by the device provided in the embodiments of this specification are consistent with the beneficial effects obtained by the above method and will not be repeated here.

[0127] like Figure 6As shown, a computer device provided by an embodiment of this specification, the data query device in this specification can be a computer device in this embodiment, and the above method of this specification is executed. The computer device 602 may include one or more processors 604, such as one or more central processing units (CPUs), and each processing unit may implement one or more hardware threads. The computer device 602 may also include any memory 606, which is used to store any kind of information such as code, settings, data, etc. Non-restrictive, for example, the memory 606 may include any one or more combinations of the following: any type of RAM, any type of ROM, flash memory device, hard disk, optical disk, etc. More generally, any memory can use any technology to store information. Further, any memory can provide volatile or non-volatile retention of information. Further, any memory can represent a fixed or removable component of the computer device 602. In one case, when the processor 604 executes an associated instruction stored in any memory or a combination of memories, the computer device 602 can perform any operation of the associated instruction. The computer device 602 also includes one or more drive mechanisms 608 for interacting with any storage, such as a hard disk drive mechanism, an optical disk drive mechanism, etc.

[0128] The computer device 602 may also include an input / output module 610 (I / O) for receiving various inputs (via input devices 612) and for providing various outputs (via output devices 614). A specific output mechanism may include a presentation device 616 and an associated graphical user interface (GUI) 618. In other embodiments, the input / output module 610 (I / O), input device 612, and output device 614 may not be included, and the computer device 602 may be used as a computer device in a network. The computer device 602 may also include one or more network interfaces 620 for exchanging data with other devices via one or more communication links 622. One or more communication buses 624 couple the components described above together.

[0129] The communication link 622 may be implemented in any manner, for example, through a local area network, a wide area network (e.g., the Internet), a point-to-point connection, etc., or any combination thereof. The communication link 622 may include any combination of hardwired links, wireless links, routers, gateway functions, name servers, etc. governed by any protocol or combination of protocols.

[0130] Corresponding to Figures 1 to 4 In addition to the method shown, an embodiment of the specification also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above method are executed.

[0131] The embodiment of the present specification also provides a computer-readable instruction, wherein when the processor executes the instruction, the program therein causes the processor to execute the following Figures 1 to 4 The method shown.

[0132] The embodiments of the present specification also provide a computer program product, including at least one instruction or at least one program, wherein the at least one instruction or the at least one program is loaded and executed by a processor to implement the following Figures 1 to 4 The method shown.

[0133] It should be understood that in the various embodiments of this specification, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this specification.

[0134] It should also be understood that in the embodiments of this specification, the term "and / or" is only a description of the association relationship of the associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this specification generally indicates that the associated objects before and after are in an "or" relationship.

[0135] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in this specification can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this specification.

[0136] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0137] In the several embodiments provided in this specification, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, or it can be an electrical, mechanical or other form of connection.

[0138] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of this specification.

[0139] In addition, each functional unit in each embodiment of this specification may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0140] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this specification is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of this specification. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0141] Specific embodiments are used in this specification to illustrate the principles and implementation methods of this specification. The description of the above embodiments is only used to help understand the methods and core ideas of this specification. At the same time, for those skilled in the art, according to the ideas of this specification, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on this specification.

Claims

1. A data query method, characterized in that: The method comprises: Receive data query requests; Analyze the data query request to obtain a query dimension in the data query request; Determining whether the query dimension matches a pre-built index dimension of data stored in the database; If there is a match, the query request is dispatched to the first engine; If there is no match, the query request is sent to the second engine; Obtain a query result of the data query request returned by the first engine or the second engine.

2. The method according to claim 1, characterized in that: After dispatching the query request to the second engine, the method further includes: A first historical data query request dispatched to the second engine during a collection monitoring period; Calculating a first usage frequency of each query dimension in the first historical data query request; Determining whether the first usage frequency is greater than or equal to a preset first frequency threshold; If so, the query dimension whose first usage frequency is greater than or equal to the first frequency threshold is used as a recommended index dimension to be constructed.

3. The method according to claim 1, characterized in that: After dispatching the query request to the first engine, the method further includes: A second historical data query request dispatched to the first engine during the acquisition monitoring period; Calculating a second usage frequency of each index dimension corresponding to the second historical data query request; Determining whether the second usage frequency is less than a preset second frequency threshold; If so, the index dimension whose second usage frequency is less than the preset second frequency threshold is taken as the recommended index dimension to be deleted.

4. The method according to claim 1, characterized in that: The first engine is the Kylin engine, and the second engine is the ClickHouse engine.

5. The method according to claim 1, characterized in that Before receiving the data query request, the method further includes: Receive access documents; Collecting data in the access document and converting the format; The format-converted data is stored in the database.

6. The method according to claim 5, characterized in that The method further comprises: Perform preliminary aggregation on the format-converted data to obtain the pre-built index dimensions.

7. A data query device, characterized in that: The device comprises: A receiving module, used for receiving a data query request; An analysis module, used to analyze the data query request to obtain a query dimension in the data query request; A determination module, used to determine whether the query dimension matches a pre-built index dimension of the data stored in the database; A first dispatching module, configured to dispatch the query request to a first engine if there is a match; A second dispatching module, configured to dispatch the query request to a second engine if there is no match; The acquisition module is used to acquire the query result of the data query request returned by the first engine or the second engine.

8. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

10. A computer program product, characterized in that The method comprises at least one instruction or at least one program, wherein the at least one instruction or the at least one program is loaded and executed by a processor to implement the method according to any one of claims 1 to 6.

Images