A data increment statistics method, device, equipment and storage medium for doris
By using the aggregation model and data statistics functions in Doris, the problem of group by operation taking too long in Doris data increment statistics is solved, real-time statistical efficiency at the minute level is achieved, and real-time requirements are met.
Patent Information
- Application Number
- CN202311447019.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-11-01
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2043-11-01
AI Technical Summary
During the data increment statistics process of Doris, the group by operation from the DWS layer to the ADS layer takes too long, resulting in the real-time statistics requirements of the minute level cannot be met.
By using an aggregation model in Doris, the statistical indicator fields and dimension fields are divided according to the table structure of the statistical report, the data is aggregated and the statistical results are calculated using the preset data statistics function to avoid group by operations and directly count the dimension columns number of data in.
It significantly reduces the time of data statistics, improves the efficiency of data increment statistics, meets the real-time statistics requirements at the minute level, and reduces the operation and maintenance burden of the system.
Smart Images

Figure CN117520408B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of data processing, and in particular to a data increment statistics method, device, equipment and storage medium for DORIS. Background Art
[0002] The existing data warehouse is implemented based on Doris, which is a real-time analytical database with MPP architecture. It is layered inside Doris and divided into three layers, including: ODS (detail layer) stores the original detailed data, and the data comes from the business database; DWS (data service layer) contains multiple dimension fields for the reuse of different reports, and the data comes from the ODS layer; ADS (application layer) re-aggregates the group by dimension fields from the DWS layer according to the statistical requirements of different reports to obtain aggregated statistical results. The final user displays the report by querying the data in the ADS layer.
[0003] Each layer extracts data from the lower layer to the upper layer by means of timed SQL scheduling. The scheduling tasks between the layers are divided into two types of tasks. One is incremental statistics. To ensure real-time performance, incremental statistics are run once a minute. The other is full statistics. To ensure data accuracy and program robustness, it is run once a day and re-runs data from the previous few days.
[0004] During incremental statistics, from the ODS layer to the DWS layer, since only the data of the previous few minutes is counted each time, the statistical speed is very fast, and it can be completed in an average of a few seconds, which can meet the minute-level statistical requirements. However, from the DWS layer to the ADS layer, it is necessary to perform group by according to the required statistical dimensions, and call the bitmap_union_count function to obtain the statistical results. The bitmap_union_count function first aggregates the bitmap data structures with the same dimension values according to the group by field to obtain the intersection. This process requires re-aggregation of data and parsing of the bitmap to obtain the intersection of data, which takes a lot of time, reduces the efficiency of incremental data statistics, and cannot meet the minute-level real-time statistical requirements. Summary of the invention
[0005] In order to solve the above technical problems, the present invention discloses a data incremental statistics method, device, equipment and storage medium for Doris, which improves the efficiency of data incremental statistics.
[0006] In order to achieve the above object, in a first aspect, the present invention discloses a data increment statistics method for Doris, comprising:
[0007] Divide the fields according to the acquired statistical report to determine the statistical indicator fields and statistical dimension fields corresponding to the statistical report;
[0008] According to the statistical dimension field and the statistical indicator field, the data stored in the detail layer of Doris is aggregated by calling the aggregation model preset in Doris to obtain the statistical data corresponding to the statistical report;
[0009] According to the statistical data, the number of the statistical data is calculated by calling the data statistical function preset in the Doris, and the data statistical result corresponding to the statistical report is obtained.
[0010] The present invention discloses a data increment statistics method for Doris, which customizes statistical indicator fields and statistical dimension fields according to table structures of different current statistical reports, obtains data from the ODS layer of Doris according to the statistical indicator fields and statistical dimension fields, and aggregates the data using the aggregation model of Doris. Since the aggregation model is used, when data is newly added, data with the same dimension column, that is, data belonging to the same statistical dimension field, will be aggregated into a row first. When the data statistical function of Doris is used to calculate the number of statistical data, since the same dimension column already contains the fields required for statistics, it is not necessary to aggregate the dimension column. It is only necessary to count how many statistical data there are in the row corresponding to the current statistical dimension field to obtain the statistical result, thereby reducing the time for data statistics and improving the efficiency of data statistics.
[0011] As a preferred example, aggregating the data stored in the detail layer of Doris by calling the aggregation model preset in Doris includes:
[0012] Incrementally synchronize the data within a period of time before the current moment from the detail layer of Doris to obtain the data to be aggregated;
[0013] Traversing the data to be aggregated through the aggregation model and acquiring a dimension field of each first data in the data to be aggregated;
[0014] According to the statistical dimension field, the first data corresponding to when the dimension field is the same as the statistical dimension field is selected through the aggregation model to obtain statistical data corresponding to the statistical report.
[0015] The present invention utilizes the aggregation model and the statistical dimension field to aggregate data, so that the data obtained after aggregation has been classified according to the statistical dimension field, thereby solving the group by operation in the prior art, reducing the time of data parsing, and improving the efficiency of data statistics.
[0016] As a preferred example, in the step of aggregating the data stored in the detail layer of Doris by calling the aggregation model preset in Doris according to the statistical dimension field and the statistical indicator field, it also includes:
[0017] The statistical data are classified according to the statistical indicator fields to obtain a plurality of statistical data corresponding to each statistical indicator field in different statistical indicator fields.
[0018] The present invention classifies the statistical data according to the statistical indicator field, so that when statistics of different indicator data are performed in the subsequent process, it is no longer necessary to parse the data structure, and only the number of data contained in the current indicator field needs to be counted, which can effectively improve the efficiency of data statistics.
[0019] As a preferred example, in calculating the number of the statistical data by calling the data statistical function preset in the Doris, obtaining the data statistical result corresponding to the statistical report includes:
[0020] The number of data corresponding to the plurality of first statistical data is calculated by the bitmap_count function preset in the Doris, and the first data statistical results corresponding to each statistical indicator field in different statistical indicator fields are obtained;
[0021] According to the statistical dimension field, the statistical indicator field and the first data statistical result, statistical data corresponding to the statistical report is obtained.
[0022] In the present invention, data is divided into dimensions according to statistical dimension fields, and each dimension row corresponds to data of the same dimension obtained after aggregation, so that group by operation is no longer required, reducing this time-consuming process, thereby improving the speed of data statistics.
[0023] In a second aspect, the present invention further discloses a data increment statistics device for Doris, the device comprising a field division module, a data aggregation module and a data statistics module;
[0024] The field division module is used to divide the fields according to the acquired statistical report, and determine the statistical indicator field and the statistical dimension field corresponding to the statistical report;
[0025] The data aggregation module is used to aggregate the data stored in the detail layer of Doris according to the statistical dimension field and the statistical indicator field by calling the aggregation model preset in Doris to obtain the statistical data corresponding to the statistical report;
[0026] The data statistics module is used to calculate the number of the statistical data based on the statistical data by calling the data statistics function preset in the Doris, so as to obtain the data statistical results corresponding to the statistical report.
[0027] The present invention discloses a data increment statistics device for Doris, which customizes statistical indicator fields and statistical dimension fields according to the table structures of different current statistical reports, obtains data from the ODS layer of Doris according to the statistical indicator fields and statistical dimension fields, and aggregates the data using the aggregation model of Doris. Since the aggregation model is used, when data is added, data with the same dimension column, that is, data belonging to the same statistical dimension field, will be aggregated into a row first. When the data statistical function of Doris is used to calculate the number of statistical data, since the same dimension column already contains the fields required for statistics, it is not necessary to aggregate the dimension column. It is only necessary to count how many statistical data there are in the row corresponding to the current statistical dimension field to obtain the statistical result, thereby reducing the time for data statistics and improving the efficiency of data statistics.
[0028] As a preferred example, the data aggregation module includes an incremental synchronization unit and a dimension division unit;
[0029] The incremental synchronization unit is used to incrementally synchronize data within a period of time before the current moment from the detail layer of Doris to obtain data to be aggregated;
[0030] The dimension division unit is used to traverse the data to be aggregated through the aggregation model and obtain the dimension field of each first data in the data to be aggregated; according to the statistical dimension field, the first data corresponding to the dimension field when the dimension field is the same as the statistical dimension field is selected through the aggregation model to obtain the statistical data corresponding to the statistical report.
[0031] The present invention utilizes the aggregation model and the statistical dimension field to aggregate data, so that the data obtained after aggregation has been classified according to the statistical dimension field, thereby solving the group by operation in the prior art, reducing the time of data parsing, and improving the efficiency of data statistics.
[0032] As a preferred example, the data aggregation module further includes an indicator division unit;
[0033] The indicator classification unit is used to classify the statistical data according to the statistical indicator fields, and obtain a plurality of first statistical data corresponding to each statistical indicator field in different statistical indicator fields.
[0034] The present invention classifies the statistical data according to the statistical indicator field, so that when statistics of different indicator data are performed in the subsequent process, it is no longer necessary to parse the data structure, and only the number of data contained in the current indicator field needs to be counted, which can effectively improve the efficiency of data statistics.
[0035] As a preferred example, the data statistics module includes an indicator statistics unit and a combination unit;
[0036] The indicator statistics unit is used to calculate the number of data corresponding to the plurality of first statistical data by using the bitmap_count function preset in the Doris, and obtain the first data statistical results corresponding to each statistical indicator field in different statistical indicator fields;
[0037] The combining unit is used to obtain statistical data corresponding to the statistical report according to the statistical dimension field, the statistical indicator field and the first data statistical result.
[0038] In the present invention, data is divided into dimensions according to statistical dimension fields, and each dimension row corresponds to data of the same dimension obtained after aggregation, so that group by operation is no longer required, reducing this time-consuming process, thereby improving the speed of data statistics.
[0039] In a third aspect, the present invention discloses a data incremental statistics device for Doris, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; the memory is used to store computer programs; the processor is used to implement a data incremental statistics method for Doris as described in the first aspect when executing the program stored in the memory.
[0040] In a fourth aspect, the present invention discloses a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the data incremental statistics method for Doris as described in the first aspect is implemented. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] Figure 1 : A flow chart of a data increment statistics method for Doris disclosed in an embodiment of the present invention;
[0042] Figure 2 : A structural diagram of a data increment statistics device for Doris disclosed in an embodiment of the present invention. DETAILED DESCRIPTION
[0043] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0044] Example
[0045] This embodiment discloses a data increment statistical method for Doris. The specific implementation process of the statistical method can be referred to Figure 1 , mainly including step 101 to step 103, the steps are:
[0046] Step 101: Divide the fields according to the acquired statistical report, and determine the statistical indicator fields and statistical dimension fields corresponding to the statistical report.
[0047] In this embodiment, this step includes: dividing the statistical indicator field and the statistical dimension field according to the structure of the table to be counted.
[0048] Specifically, in an implementable method provided in this embodiment, in order to solve the technical problem of slow data statistics efficiency from the DWS layer to the ADS layer, an acceleration layer is added to replace the original incremental DWS layer, and the incremental statistics method provided in this embodiment is implemented through the acceleration layer. In the acceleration layer, referring to the statistical requirements of the table to be counted, the user id and channel id are used as statistical dimension fields, and according to different states, the number of success (state=0) and failure (state=0) indicators are counted. The success amount and the failure amount are the statistical indicator fields, and a data statistical structure table is constructed according to the statistical indicator fields and the statistical dimension fields, and data aggregation is performed according to the data statistical structure table.
[0049] Step 102: According to the statistical dimension field and the statistical indicator field, the data stored in the detail layer of Doris is aggregated by calling the aggregation model preset in Doris to obtain the statistical data corresponding to the statistical report.
[0050] In this embodiment, this step includes: incrementally synchronizing data within a period of time before the current moment from the detail layer of Doris to obtain the data to be aggregated; traversing the data to be aggregated through the aggregation model and obtaining the dimension field of each first data in the data to be aggregated; according to the statistical dimension field, selecting the first data corresponding to the dimension field when the dimension field is the same as the statistical dimension field through the aggregation model to obtain the statistical data corresponding to the statistical report; at the same time, classifying the statistical data according to the statistical indicator field to obtain a number of first statistical data corresponding to each statistical indicator field in different statistical indicator fields.
[0051] Specifically, in an implementable method provided in this embodiment, first, the data within a period of time before the current time is incrementally synchronized from the ODS layer to the acceleration layer. In this implementation method, the data of the previous 10 minutes is selected for increment. Since only 10 minutes of data are incrementally synchronized, the speed is very fast and usually completed within 5 seconds. Then the acceleration layer calls the aggregation model pre-stored in Doris, and then refers to the statistical dimension fields in the data statistical structure table constructed by the acceleration layer in the above steps to aggregate the data belonging to the same statistical dimension field into a row, and then the data in the same row are divided according to their corresponding statistical indicator fields.
[0052] Specifically, referring to the user ID divided in the above steps, the channel ID is a statistical dimension field, and the data identical to the statistical dimension field is found from the data of the incremental synchronization of the ODS layer, and the data is stored in the form of a table. The table structure of the searched data is as shown in Table 1, and Table 1 is as follows:
[0053] id User-id Channel-id state 11 1 4 0 22 1 4 0 33 1 4 1 44 1 4 1 55 3 6 0 66 3 6 0
[0054] Table 1
[0055] Referring to the data obtained in the above table, the User-id and the Channel-id are used as statistical dimension fields, and the data with the same statistical dimension fields are divided into the same row. Then, the data in the same row is classified according to the statistical indicator fields to which it belongs. From the ODS layer to the acceleration layer, according to different states, the id is converted into a bitmap and the data is inserted into the fields of success and failure respectively. The data of the same dimension will be aggregated. The table structure of the statistical data obtained by aggregating the searched data can be referred to Table 2, which is as follows:
[0056] Bitmap-success Bitmap-fail User-id Channel-id 11,22 33,44 1 4 55,66 3 6
[0057] Table 2
[0058] As shown in Table 2, User-id and the Channel-id of 1 and 4 are grouped into one row, and User-id and the Channel-id of 3 and 6 are grouped into one row. It can be seen that the ids corresponding to User-id and the Channel-id of 1 and 4 include 11, 22, 33 and 44. Then, the ids are grouped according to the state value corresponding to the ids. It can be seen that ids 11 and 22 belong to state = 0, and ids 33 and 44 belong to state = 1. Similarly, the User-id and the Channel-id of 3 and 6 can be grouped.
[0059] The present invention utilizes the aggregation model and the statistical dimension field to aggregate data, so that the data obtained after aggregation has been classified according to the statistical dimension field, thereby solving the group by operation in the prior art, reducing the time of data parsing, and improving the efficiency of data statistics. At the same time, the statistical data is classified according to the statistical indicator field, so that when statistics of different indicator data are performed in the subsequent process, it is no longer necessary to parse the data structure, but only the number of data contained in the current indicator field needs to be counted, which can effectively improve the efficiency of data statistics.
[0060] Step 103: According to the statistical data, the number of the statistical data is calculated by calling the data statistical function preset in the Doris to obtain the data statistical result corresponding to the statistical report.
[0061] In this embodiment, this step includes: calculating the number of data corresponding to the several first statistical data through the bitmap_count function preset in the Doris, and obtaining the first data statistical results corresponding to each statistical indicator field in different statistical indicator fields; according to the statistical dimension field, the statistical indicator field and the first data statistical results, obtaining the statistical data corresponding to the statistical report.
[0062] Specifically, in an implementable manner provided in this embodiment, the data obtained by the acceleration layer after aggregation through the aggregation model can refer to Table 2. It can be seen from Table 2 that the dimension column is already the statistical dimension required for statistics, and the bitmap field corresponds to the statistical indicator, that is, the corresponding id. Therefore, the bitmap_count function can be directly called to obtain the number of bitmaps contained in each different statistical indicator field. The bitmap_count function does not involve parsing the bitmap data, but only needs to count how many bitmaps there are in each row, avoiding the group by operation. The group by operation must be aggregated first, and then the bitmap is parsed to find the intersection. This process is very time-consuming. By reducing this process, the speed is improved, usually completed within 10-20 seconds. Specifically, referring to Table 2, the bitmap_count function is used for statistics, and the statistical results can be referred to Table 3. The structure of Table 3 is as follows:
[0063] Success-count fail-count User-id Channel-id 2 2 1 4 2 0 3 6
[0064] Table 3
[0065] According to the above table, the acceleration layer directly uses the bitmap_count function to obtain statistical results from the ADS layer. Furthermore, when the acceleration layer uses the method for incremental statistics, only the data of the current day is retained, and the original incremental statistics DWS layer does not need to be executed. Before the full statistics, the acceleration layer data other than the current day is deleted, and the original full statistics process remains unchanged. In this way, the acceleration layer only needs to save the data of the current day, and the time consumption will not increase as the amount of data increases.
[0066] In this embodiment, this step divides the data into dimensions according to the statistical dimension fields. Each dimension row corresponds to the data of the same dimension obtained after aggregation, so the group by operation is no longer required, which reduces this time-consuming process and improves the speed of data statistics.
[0067] On the other hand, the present invention also discloses a data increment statistics device for Doris, and the specific structure of the device can be referred to Figure 2 The device includes a field division module 201, a data aggregation module 202 and a data statistics module 203.
[0068] The field division module 201 is used to divide fields according to the acquired statistical report, and determine the statistical indicator fields and statistical dimension fields corresponding to the statistical report.
[0069] The data aggregation module 202 is used to aggregate the data stored in the detail layer of Doris according to the statistical dimension field and the statistical indicator field by calling the aggregation model preset in Doris to obtain the statistical data corresponding to the statistical report.
[0070] The data statistics module 203 is used to calculate the number of the statistical data according to the statistical data by calling the data statistics function preset in the Doris, so as to obtain the data statistical results corresponding to the statistical report.
[0071] In this embodiment, the data aggregation module 202 includes an incremental synchronization unit and a dimension division unit.
[0072] The incremental synchronization unit is used to incrementally synchronize data within a period of time before the current moment from the detail layer of Doris to obtain data to be aggregated.
[0073] The dimension division unit is used to traverse the data to be aggregated through the aggregation model and obtain the dimension field of each first data in the data to be aggregated; according to the statistical dimension field, the first data corresponding to the dimension field when the dimension field is the same as the statistical dimension field is selected through the aggregation model to obtain the statistical data corresponding to the statistical report.
[0074] In this embodiment, the data aggregation module 202 further includes an indicator division unit.
[0075] The indicator classification unit is used to classify the statistical data according to the statistical indicator fields, and obtain a plurality of first statistical data corresponding to each statistical indicator field in different statistical indicator fields.
[0076] In this embodiment, the data statistics module 203 includes an indicator statistics unit and a combining unit.
[0077] The indicator statistics unit is used to calculate the number of data corresponding to the plurality of first statistical data through the bitmap_count function preset in the Doris, and obtain the first data statistical results corresponding to each statistical indicator field in different statistical indicator fields.
[0078] The combining unit is used to obtain statistical data corresponding to the statistical report according to the statistical dimension field, the statistical indicator field and the first data statistical result.
[0079] In addition to the above-mentioned method and apparatus, this embodiment also discloses a data incremental statistics device for Doris, wherein the device includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; the memory is used to store computer programs; the processor is used to implement a data incremental statistics method for Doris described in this embodiment when executing the program stored in the memory.
[0080] Furthermore, a computer-readable storage medium is disclosed, wherein a computer program is stored in the computer-readable storage medium. When the computer program is executed by a processor, the data increment statistics method for Doris described in this embodiment is implemented.
[0081] The data incremental statistics method, device, equipment and storage medium for Doris disclosed in this embodiment are performed in Doris of the original system, and no additional operation and maintenance deployment work is required. At the same time, the incremental statistics time based on the newly added acceleration layer is reduced from the original 5 minutes to 20 seconds, meeting the real-time statistics requirements at the minute level. Furthermore, since the solution only involves the transformation of incremental statistics, the original process is slightly changed, and the flexibility of the original DWS layer is retained. Moreover, since the acceleration layer only retains the data of the day, each report occupies 2-3G, so it does not take up a lot of space for storage, and the subsequent increase in the amount of data will not lead to an increase in the statistical time.
[0082] The specific embodiments described above further illustrate the purpose, technical solutions and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. It is particularly pointed out that for those skilled in the art, any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of protection of the present invention.
Claims
1. A data increment statistics method for Doris, characterized in that: include: Dividing the fields according to the acquired statistical report to determine the statistical indicator field and the statistical dimension field corresponding to the statistical report; wherein the statistical indicator field and the statistical dimension field are divided according to the structure of the statistical report; According to the statistical dimension field and the statistical indicator field, the data stored in the detail layer of Doris is aggregated by calling the aggregation model preset in Doris to obtain the statistical data corresponding to the statistical report; wherein, the data within a period of time before the current moment is incrementally synchronized from the detail layer of Doris to obtain the data to be aggregated; the data to be aggregated is traversed through the aggregation model and the dimension field of each first data in the data to be aggregated is obtained; according to the statistical dimension field, the first data corresponding to the dimension field and the statistical dimension field are selected through the aggregation model to obtain the statistical data corresponding to the statistical report; the statistical data is classified according to the statistical indicator field to obtain a number of first statistical data corresponding to each statistical indicator field in different statistical indicator fields; According to the statistical data, the number of the statistical data is calculated by calling the data statistical function preset in the Doris, and the data statistical result corresponding to the statistical report is obtained.
2. A data increment statistics method for Doris as claimed in claim 1, characterized in that: The method of calculating the number of the statistical data by calling the data statistical function preset in the Doris to obtain the data statistical result corresponding to the statistical report includes: The number of data corresponding to the plurality of first statistical data is calculated by the bitmap_count function preset in the Doris, and the first data statistical results corresponding to each statistical indicator field in different statistical indicator fields are obtained; According to the statistical dimension field, the statistical indicator field and the first data statistical result, statistical data corresponding to the statistical report is obtained.
3. A data increment statistics device for Doris, characterized in that: The device includes a field division module, a data aggregation module and a data statistics module; The field division module is used to divide the fields according to the acquired statistical report, and determine the statistical indicator field and the statistical dimension field corresponding to the statistical report; wherein the statistical indicator field and the statistical dimension field are divided according to the structure of the statistical report; The data aggregation module is used to aggregate the data stored in the detail layer of Doris according to the statistical dimension field and the statistical indicator field by calling the aggregation model preset in Doris to obtain the statistical data corresponding to the statistical report; wherein, the data aggregation module includes an incremental synchronization unit and a dimension division unit; the incremental synchronization unit is used to incrementally synchronize the data within a period of time before the current moment from the detail layer of Doris to obtain the data to be aggregated; the dimension division unit is used to traverse the data to be aggregated through the aggregation model and obtain the dimension field of each first data in the data to be aggregated; according to the statistical dimension field, the first data corresponding to the dimension field when the dimension field is the same as the statistical dimension field is selected through the aggregation model to obtain the statistical data corresponding to the statistical report; the data aggregation module also includes an indicator division unit; the indicator division unit is used to classify the statistical data according to the statistical indicator field, and obtain a number of first statistical data corresponding to each statistical indicator field in different statistical indicator fields; The data statistics module is used to calculate the number of the statistical data based on the statistical data by calling the data statistics function preset in the Doris, and obtain the data statistical results corresponding to the statistical report.
4. A data increment statistics device for Doris as claimed in claim 3, characterized in that: The data statistics module includes an indicator statistics unit and a combination unit; The indicator statistics unit is used to calculate the number of data corresponding to the plurality of first statistical data by using the bitmap_count function preset in the Doris, and obtain the first data statistical results corresponding to each statistical indicator field in different statistical indicator fields; The combining unit is used to obtain statistical data corresponding to the statistical report according to the statistical dimension field, the statistical indicator field and the first data statistical result.
5. A data increment statistics device for Doris, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; the memory is used to store computer programs; the processor is used to implement a data increment statistics method for Doris as described in any one of claims 1-2 when executing the program stored in the memory.
6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, the data increment statistics method for Doris described in any one of claims 1-2 is implemented.
Citation Information
Patent Citations
Data construction and management method of data warehouse and terminal
CN114328759A
Industrial equipment data storage method and device
CN114721601A
Data monitoring method and device based on aggregation model, equipment and storage medium
CN114781961A
Data real-time updating method and device, equipment and storage medium
CN114936214A
Vehicle data analysis method and device, storage medium and vehicle
CN115630127A
