Data storage method and device and data query method and device

Abstract

The invention provides a data storage method and device and a data query method and device, and the method comprises the steps: obtaining detail data which comprises a plurality of pieces of data under each dimension, and sending the detail data to an ODS layer in a Dores database; performing data summarization and data cleaning on the detail data in the ODS layer to obtain to-be-stored data, writing multiple pieces of data under different dimensions into different tables based on attribute information and content of the to-be-stored data, and converting the to-be-stored data into preset structure data based on a preset calculation method; integrating a plurality of pieces of data under the same dimension into one piece of aggregated data based on the table corresponding to each dimension and each piece of preset structure data, and writing the aggregated data into a Dores database. The storage space can be saved, and the data query efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of data analysis technology, and in particular to a data storage method, a query method and a device. Background Art

[0002] Massively Parallel Processing is a computing method and architecture for processing large-scale data, providing high performance and scalability through parallel processing and distributed computing.

[0003] In large-scale data analysis, it is necessary to flexibly perform deduplication statistics on the data according to the statistical dimension based on the needs of users, and it is necessary to retain the data before and after the calculation at the same time. At present, the most widely used data storage is OLTP database (Mysql, Oracle) and OLAP database (Hive, Doris); Doris is an open source MPP database that focuses on low-latency query analysis under large data volumes, can provide efficient OLAP analysis performance, and can retain the data before and after the calculation in the data at the same time. At present, when using the Doris database for large-scale data analysis, it is generally necessary to store the detailed data before the calculation, obtain the calculated data after the calculation is completed, and then splice the data before and after the calculation. Both calculation and query need to be performed on the database; at present, although data storage and query can be realized based on the Doris database, when the data volume is large, there are many problems in data storage based on the existing data storage method. This is because when performing large-scale data storage, the storage space occupied is too large due to the large amount of data, and when performing data query, it will also lead to low data query efficiency. Therefore, when conducting large-scale data analysis, how to save storage space and improve data query efficiency is a technical problem that needs to be solved urgently. Summary of the invention

[0004] In view of this, an embodiment of the present invention provides a data storage method, a query method and a device to eliminate or improve one or more defects in the prior art.

[0005] One aspect of the present invention provides a data storage method, the method comprising the following steps:

[0006] Obtain detailed data, the detailed data including multiple pieces of data under each dimension, and send the detailed data to the ODS layer in the Doris database;

[0007] Summarize and clean the detailed data in the ODS layer to obtain data to be stored, write multiple data in different dimensions into different tables based on the attribute information and content of the data to be stored, and convert the data to be stored into preset structure data based on a preset calculation method;

[0008] Based on the tables corresponding to each dimension and each of the preset structure data, multiple pieces of data under the same dimension are integrated into one piece of aggregated data, and the aggregated data is written into the Doris database.

[0009] In some embodiments of the present invention, the detailed data in the ODS layer is aggregated and cleaned to obtain the data to be stored, including:

[0010] Based on the computing engine, the detailed data in the ODS layer is aggregated and cleaned to obtain the data to be stored.

[0011] In some embodiments of the present invention, the preset structure data is Json structure data or Bitmap structure data; and / or,

[0012] Obtain detailed data, including: obtaining log data and embedded log data sent by each business system.

[0013] In some embodiments of the present invention, when the preset structure data is Bitmap structure data,

[0014] Based on the tables corresponding to each dimension and each of the preset structure data, multiple pieces of data under the same dimension are integrated into one piece of aggregated data, and the aggregated data is written into the Doris database, including:

[0015] Determine the Bitmap structure data corresponding to each data under the same dimension based on the table corresponding to each dimension;

[0016] The Bitmap structure data corresponding to each data under the same dimension is recorded in each bit position in the Bitmap.

[0017] The aggregated data with each bit recorded in the Bitmap structure data is written into the Doris database.

[0018] In some embodiments of the present invention, the computing engine is a Spark engine or a Flink engine.

[0019] According to another aspect of the present invention, a data query method is also provided, the method comprising the following steps:

[0020] Obtaining a query request from a user, and generating a query statement based on the query request, wherein the query request includes dimension information of the data to be queried;

[0021] Extracting aggregated data corresponding to the dimension information from the Doris database based on the query statement;

[0022] Determine the preset structure data corresponding to the data to be queried based on the aggregated data;

[0023] The preset structure data is parsed into a data query result by using a preset parsing algorithm.

[0024] In some embodiments of the present invention, the preset structure data is Json structure data or Bitmap structure data.

[0025] Correspondingly, the present invention also provides a data storage device, including a processor, a memory and a computer program stored in the memory, wherein the processor is used to execute the computer program, and when the computer program is executed, the device implements the steps of the method described in any of the above embodiments.

[0026] According to another aspect of the present invention, a computer-readable storage medium is disclosed, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method described in any of the above embodiments are implemented.

[0027] According to another aspect of the present invention, the present invention discloses a computer program product, including a computer program, which implements the steps of the method described in any of the above embodiments when executed by a processor.

[0028] The data storage method, query method and device disclosed in the embodiment of the present invention write multiple data in different dimensions into different tables based on the attribute information and content of the data to be stored, convert the data to be stored into preset structure data based on a preset calculation method, and integrate multiple data in the same dimension into one aggregated data based on the tables corresponding to each dimension and each preset structure data, and finally write the aggregated data into the Doris database. This method enables multiple data in the same dimension to be aggregated into one data for storage, which greatly reduces the amount of stored data and saves the storage space occupied by the data; and when querying data, the amount of scanned data is reduced, which also speeds up the data query speed, thereby improving the data query efficiency.

[0029] In addition, this application entrusts data calculation to the computing engine, that is, the computing engine is used to clean, calculate, summarize and other operations on the data before writing the calculation results into the database, avoiding the phenomenon of calculation failure caused by users using temporary database calculations.

[0030] Additional advantages, purposes, and features of the present invention will be described in part in the following description, and will become apparent to those skilled in the art after studying the following, or may be learned from the practice of the present invention. The purposes and other advantages of the present invention may be achieved and obtained by the structures specifically indicated in the specification and the accompanying drawings.

[0031] Those skilled in the art will appreciate that the objectives and advantages that can be achieved with the present invention are not limited to the above specific description, and the above and other objectives that can be achieved by the present invention will be more clearly understood from the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of this application, and do not constitute a limitation of the present invention. The components in the drawings are not drawn to scale, but are only for illustrating the principles of the present invention. In order to facilitate the illustration and description of some parts of the present invention, the corresponding parts in the drawings may be enlarged, that is, they may become larger relative to other components in the exemplary device actually manufactured according to the present invention. In the drawings:

[0033] Figure 1 The figure is a flowchart of a data storage method according to an embodiment of the present invention.

[0034] Figure 2 The figure is a flowchart of a data query method according to an embodiment of the present invention.

[0035] Figure 3 The figure is a schematic diagram of storing int type data and Bitmap type data according to an embodiment of the present invention.

[0036] Figure 4 FIG. 1 is a schematic diagram of the architecture of a data storage device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments and the accompanying drawings. Here, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0038] It should also be noted that, in order to avoid obscuring the present invention due to unnecessary details, only structures and / or processing steps closely related to the solutions according to the present invention are shown in the accompanying drawings, while other details that are not closely related to the present invention are omitted.

[0039] It should be emphasized that the term “include / comprises” when used herein refers to the presence of features, elements, steps or components, but does not exclude the presence or addition of one or more other features, elements, steps or components.

[0040] It should also be noted that, unless otherwise specified, the term “connection” herein may refer not only to a direct connection but also to an indirect connection with an intermediate.

[0041] The meanings of terms related to the technical solution of this application are explained as follows:

[0042] OLAP refers to Online Analytical Processing, a processing method used for decision support and data analysis, mainly used for multidimensional data analysis and complex queries; typical OLAP systems include data warehouses and business intelligence tools.

[0043] TPS refers to the number of transactions completed in one second.

[0044] Bitmap, regarded as a huge bit array, each bit can store a value of 0 or 1, which makes Bitmap very suitable for storing and operating a large number of Boolean values ​​or performing efficient statistical counting.

[0045] At present, the OLTP database (Mysql, Oracle) and OLAP database (Hive, Doris) used for data storage are based on the basic type of data when performing data storage, which has the following disadvantages: the amount of data stored in the database is very large, and these data occupy extra disk space and resources; when performing data query, the corresponding scanned data is relatively large, so the scanning time is relatively long, and the query efficiency is also low. Especially in large-scale data scenarios, due to the large amount of data, storing data according to the existing data storage method not only causes the problem of occupying a large storage space and low data query efficiency, but also may cause data calculation failure. In order to solve the above problems existing in the existing data storage method and data query method, the present application discloses a data storage method, a query method and a device, that is, when performing large-scale data analysis, the basic type of data is converted into complex type of data for storage, so that multiple basic type data under the same dimension are compressed into one complex type of data, thereby greatly reducing the amount of data, thereby saving storage space, and correspondingly improving the data query efficiency when performing data query. In addition, for large amounts of detailed data that need to be stored, in some embodiments of the present application, data is pre-calculated based on the computing engine, that is, an architecture that separates computing and storage is adopted, so that the database is only responsible for simple calculations during queries, thereby further ensuring the computing stability under large-scale data.

[0046] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the accompanying drawings, the same reference numerals represent the same or similar components, or the same or similar steps.

[0047] Figure 1 FIG. 1 is a flow chart of a data storage method according to an embodiment of the present invention. Figure 1 As shown, the data storage method includes steps S10 to S30.

[0048] Step S10: Obtain detailed data, which includes multiple pieces of data under each dimension, and send the detailed data to the ODS layer in the Doris database.

[0049] In this step, the detailed data obtained includes data of multiple dimensions, and each dimension has multiple data items. For example, taking users under multiple dimensions (time / region / login platform, etc.) as an example, the detailed data that needs to be stored is all dimensional information and all user IDs under each dimension.

[0050] Exemplarily, when obtaining detailed data, the specific method is to obtain log data and embedded point log data sent by each business system. In this embodiment, the database (oracle, mysql) logs of the upstream business system are collected and synchronized to the ODS layer of the Doris database; for the front-end embedded point data, it is obtained through the back-end service and then synchronized to the ODS layer of the downstream database Doris through kafka.

[0051] Step S20: Aggregate and clean the detailed data in the ODS layer to obtain the data to be stored, write multiple data in different dimensions into different tables based on the attribute information and content of the data to be stored, and convert the data to be stored into preset structure data based on a preset calculation method.

[0052] In this step, the acquired detailed data is aggregated and calculated to obtain the data to be stored, and then the data under the same dimension is written into the same table, so that the data in the same table is aggregated into one piece of data and saved. The dimensions of the data in the same table are the same. For example, when the dimensions of the detailed data are time, region, and login platform, all the data to be stored are written into three tables, and the attribute information of the data to be stored can also be understood as the dimension information to which the data to be stored belongs.

[0053] Since the data acquired in step S10 is detailed data, and the log data corresponding to different business systems and tracking logs correspond to different tables, the purpose of data aggregation and data cleaning of the detailed data is to pre-calculate the log data in different tables into the table corresponding to the data to be stored. In a specific embodiment, the detailed data in the ODS layer can be aggregated and cleaned based on the computing engine to obtain the data to be stored, that is, the detailed data is pre-calculated as the data to be stored based on the computing engine.

[0054] Exemplarily, the computing engine is a Spark engine or a Flink engine. In some embodiments, the computing engine reads different ods tables and dim tables in the doris database, merges them into one table after aggregation calculation, and then the merged table is written into doris. It can be understood that in this embodiment, setting the computing engine as a Spark engine or a Flink engine is only some examples and does not constitute a limitation on the technical solution of the present application. In some other embodiments, the computing engine can also be other engines besides Spark or Flink.

[0055] In addition, the preset structure data can be Json structure data or Bitmap structure data. In this embodiment, the basic type of data is converted into Json structure data or Bitmap structure data based on the preset calculation method, and the basic data types in the database generally refer to Long, Int, Double, decimal, Char, Varchar, String, Boolen and other data types used to describe numbers, strings, and yes and no. The preset structure data is a complex data type relative to the basic data type. The complex data types supported by the doris database are Json, Map, Struct, Bitmap, Variant and other data types containing complex structures. Therefore, it can be understood that the embodiment sets the preset structure data as Json structure data or Bitmap structure data is only some preferred examples. In other embodiments, the preset structure data can also be other complex types of data except Json structure data and Bitmap structure data.

[0056] Step S30: Based on the tables corresponding to each dimension and each of the preset structure data, multiple pieces of data under the same dimension are integrated into one piece of aggregated data, and the aggregated data is written into the Doris database.

[0057] This step realizes the aggregation of multiple data under each dimension and the writing of aggregated data. It can be seen from this step that the present application aggregates multiple data of the same dimension into one data and then writes it into the doris database, which can greatly reduce the amount of stored data. Correspondingly, when querying data, the time for scanning data is reduced, and the amount of calculated data is also reduced accordingly. Therefore, aggregating multiple data under the same dimension not only greatly improves the efficiency and stability of the query, but also reduces the pressure on the back-end service.

[0058] Exemplarily, the preset structure data is Bitmap structure data. At this time, based on the tables corresponding to each dimension and the preset structure data, multiple data under the same dimension are integrated into one aggregate data, and the aggregate data is written into the Doris database, which specifically includes the following steps: based on the tables corresponding to each dimension, determine the Bitmap structure data corresponding to each data under the same dimension; record the Bitmap structure data corresponding to each data under the same dimension into each bit in the Bitmap, and write the aggregate data with each bit recorded in the Bitmap structure data into the Doris database. In this embodiment, the bitmap data structure can reduce the storage space of the data itself and the number of data when storing INT / BIGINT data; Figure 3 FIG. 1 is a schematic diagram of storing int type data and Bitmap type data according to an embodiment of the present invention. Figure 3 As shown, for INT type data, 32 bits are required to store one INT data, while Bitmap type data only requires 1 bit to store one INT data. Therefore, it can be seen that converting INT type data into Bitmap type data for storage can reduce the amount of data storage while retaining detailed data. It can be understood that the INT type data in this embodiment is the basic type data mentioned in this article, and the Bitmap type data is correspondingly the complex type data.

[0059] Similar to Bitmap type data, the Json data structure can also reduce the number of data to be stored and reduce the storage space. Specifically, Table 1 is a table corresponding to when multiple data in the prior art are stored as basic data types, while Table 2 is a table corresponding to when multiple data disclosed in this application are stored as complex data types. Figure 1 and Figure 2 It can be seen that the data storage method of the present application greatly reduces the amount of data storage when storing large-scale data compared to the prior art.

[0060] Table 1 Corresponding table when multiple data in the prior art are stored as basic data types

[0061]

[0062]

[0063] Table 2 Corresponding table when multiple data disclosed in this application are stored as complex data types

[0064] KEY_COL VALUE_COL KEY VALUE1+VALUE2+VALUE3+…

[0065] Specifically, the present application uses complex data types (Json, Bitmap, etc.) to store the detailed data required by the user, which can reduce the amount of data storage while retaining the detailed data. For example, taking the calculation of user information under multiple dimensions (time / region / login platform, etc.) as an example, since each different dimension is specified by the user when querying, and the user index needs to deduplicate the user ID, the prior art needs to store all dimensions and all user IDs under each dimension, and the data volume is the number of dimensions * the number of users under each dimension; while the present invention uses the Bitmap field to store user IDs, so different user IDs under the same dimension can be stored in the same data, and the data volume is the number of dimensions; in practice, it is found that the data storage method of the present application can reduce the occupied storage space by at least 90%.

[0066] It is understandable that in the data analysis process of the prior art, scanning, calculating and transmitting data all require the use of database resources (CPU, memory, disk space, disk IO, network bandwidth, etc.), and the larger the amount of data, the more complex the calculation, the more resources it occupies and the more time it takes to return the data. When the resources it occupies and the time it takes exceed the threshold set by the database, an error will be reported, resulting in calculation failure. The present application receives detailed data and first pre-calculates it based on a computing engine designed specifically for data calculation (Spark, Flink, etc.), and imports the calculation results into the database; it avoids complex data calculations of large amounts of data in the database, so that the database only performs data storage and data query, and achieves a storage and calculation separation architecture design, thereby avoiding the occurrence of data calculation failures and improving the efficiency of data storage.

[0067] Correspondingly, the present application also discloses a data query method, such as Figure 2 As shown, the data query method at least includes the following steps S01 to S04.

[0068] Step S01: Obtain a query request from a user, and generate a query statement based on the query request, wherein the query request includes dimension information of the data to be queried.

[0069] For example, according to different dimensions (time, region, device, etc.), the number of users / number of new users under the custom combination of each dimension is queried. At this time, the user will pass the dimension information to the backend service through the interface, and the backend service will pass the dimension information to the database.

[0070] Step S02: extracting aggregated data corresponding to the data to be queried from the Doris database based on the query statement.

[0071] Since the data stored based on the above data storage method is aggregated data of multiple data under each dimension, when extracting the data to be queried, firstly, the aggregated data corresponding to the dimension information is extracted from the doris database based on the dimension information contained in the query request. Since the number of data stored in the doris database is the same as the number of dimensions, the amount of data that needs to be scanned when querying data is greatly reduced.

[0072] Step S03: Determine preset structure data corresponding to the data to be queried based on the aggregated data.

[0073] After the aggregated data corresponding to the dimension information in the query request is extracted, the preset structure data corresponding to the data to be queried is further determined based on the aggregated data. Similar to the data storage method, the preset structure data in the aggregated data is complex type data, and the type of complex type data is not limited. Exemplarily, the preset structure data may be Json structure data or Bitmap structure data. It can be understood that in this embodiment, setting the preset structure data to Json structure data and Bitmap structure data are only some examples, and other complex types of data can also be used. In addition, it should be understood that the type of the preset structure data in this step is consistent with the type of aggregated data stored in the database. For example, when the aggregated data stored in the doris database is Bitmap structure data, then in this step, the Bitmap structure data corresponding to the data to be queried is determined based on the aggregated data.

[0074] Step S04: parsing the preset structure data into data query results using a preset parsing algorithm.

[0075] This step is to further parse the preset structure data into data query results, which are consistent with the basic type data mentioned in the above data storage method, that is, this step parses the complex type data into basic type data. For example, Bitmap structure data is parsed into INT type data through a parsing algorithm.

[0076] Based on the above content, it can be seen that the amount of data is reduced by preprocessing the detailed data during data storage (compressing different data of the same dimension and storing them in complex type data of the same data). The aggregated data contains both the calculated data (complex type data such as Json and Bitmap) and the detailed data before calculation (basic type data). When querying data, the amount of data scanned decreases, the complexity of calculation also decreases, and the latency of the query also decreases accordingly.

[0077] Through the above embodiments, it can be found that the data storage method disclosed in the present application uses the Bitmap data type in combination with the Aggregate model table building when performing large-scale data storage, and the int / bigint data of the original detailed data can be retained when the amount of stored data is reduced; the Json data type is combined with the Unique model table building, and the string data of the original detailed data can be retained when the amount of stored data is reduced. That is, by using complex data types to store large-scale data, multiple data under the same dimension can be compressed into one data, and the differences of different data under the same dimension can be stored as a complex data structure; such as the original user ID integer data, if its difference is retained based on the basic data type (such as int), the number of data under the same dimension needs to be retained as many users as there are; but if a complex data type (such as Bitmap) is used, different user IDs can be stored in the same Bitmap structure, so that multiple data under the same dimension are compressed into one data. This method converts the detailed data originally to be retained into complex type data, reduces the actual amount of data that needs to be stored, and thus reduces the utilization rate of hard disk space.

[0078] To sum up, the data storage method and data query method disclosed in this application are based on Spark as the main computing engine, use Dorsi as the storage and query database, and use complex data types such as Json and Bitmap to save data storage space and improve data query capabilities. In addition, it also improves the calculation success rate and reduces query latency.

[0079] Correspondingly, the present application also discloses a data storage device, including a processor, a memory, and a computer program stored in the memory, wherein the processor is used to execute the computer program, and when the computer program is executed, the device implements the steps of the method described in any of the above embodiments.

[0080] Figure 4 FIG. 1 is a schematic diagram of the architecture of a data storage device according to an embodiment of the present invention. Figure 4 As shown in the figure, after receiving the detailed data, it is first calculated based on a computing engine designed for data computing (Spark, Flink, etc.), and the calculation results after the calculation by the computing engine are imported into the database. This method realizes the separation of computing and storage, so that in large-scale data analysis scenarios, the query capability and storage capacity of the database can also be ensured.

[0081] The embodiment of the present invention also provides a computer-readable storage medium and a computer program product, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the aforementioned edge computing server deployment method are implemented. The computer-readable storage medium can be a tangible storage medium, such as a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a floppy disk, a hard disk, a removable storage disk, a CD-ROM, or any other form of storage medium known in the technical field.

[0082] It should be understood by those skilled in the art that the exemplary components, systems and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software or a combination of the two. Whether it is performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present invention are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link via a data signal carried in a carrier.

[0083] It should be clear that the present invention is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present invention.

[0084] In the present invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with features of other embodiments or replace features of other embodiments.

[0085] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the embodiments of the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A data storage method, characterized in that: The method comprises the following steps: Obtain detailed data, the detailed data including multiple pieces of data under each dimension, and send the detailed data to the ODS layer in the Doris database; Summarize and clean the detailed data in the ODS layer to obtain data to be stored, write multiple data in different dimensions into different tables based on the attribute information and content of the data to be stored, and convert the data to be stored into preset structure data based on a preset calculation method; Based on the tables corresponding to each dimension and each of the preset structure data, multiple pieces of data under the same dimension are integrated into one piece of aggregated data, and the aggregated data is written into the Doris database.

2. The data storage method according to claim 1, characterized in that: The detailed data in the ODS layer is aggregated and cleaned to obtain the data to be stored, including: Based on the computing engine, the detailed data in the ODS layer is aggregated and cleaned to obtain the data to be stored.

3. The data storage method according to claim 1, characterized in that: The preset structure data is Json structure data or Bitmap structure data; and / or, Obtain detailed data, including: obtaining log data and embedded log data sent by each business system.

4. The data storage method according to claim 3, characterized in that: When the preset structure data is Bitmap structure data, based on the tables corresponding to each dimension and each preset structure data, multiple pieces of data under the same dimension are integrated into one piece of aggregated data, and the aggregated data is written into the Doris database, including: Determine the Bitmap structure data corresponding to each data under the same dimension based on the table corresponding to each dimension; The Bitmap structure data corresponding to each data under the same dimension is recorded in each bit position in the Bitmap. The aggregated data with each bit recorded in the Bitmap structure data is written into the Doris database.

5. The data storage method according to claim 2, characterized in that: The computing engine is a Spark engine or a Flink engine.

6. A data query method, characterized in that: The method comprises the following steps: Obtaining a query request from a user, and generating a query statement based on the query request, wherein the query request includes dimension information of the data to be queried; Extracting aggregated data corresponding to the dimension information from the Doris database based on the query statement; Determine the preset structure data corresponding to the data to be queried based on the aggregated data; The preset structure data is parsed into a data query result by using a preset parsing algorithm.

7. The data query method according to claim 6, characterized in that: The preset structure data is Json structure data or Bitmap structure data.

8. A data storage device comprising a processor, a memory and a computer program stored in the memory, characterized in that: The processor is used to execute the computer program. When the computer program is executed, the device implements the steps of the method according to any one of claims 1 to 5.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images