Multi-level cache aggregation calculation method and device

Abstract

The invention discloses a multi-level cache aggregation computing method and device, and relates to the technical field of cache aggregation computation.The method comprises the steps that a message bus is connected, and a corresponding first-level cache located at the top of a stack is obtained; when subscribing to the data with the same time dimension and the same equipment dimension, carrying out accumulative aggregation calculation on the data and the original data; judging whether the first event task belongs to a defined time interval range or not; when a timed task corresponding to the first event task is triggered, delay waiting time is set; and meanwhile, transmitting the data of the first-level cache to external cache middleware, and updating and merging the data of the same time dimension and equipment dimension. The technical problems that in the prior art, a time series data aggregation calculation cache mode is single, an aggregation mode is limited, and data storage and processing efficiency is low are solved, and the technical effects that time series data processing efficiency is improved, memory occupation and delay are reduced, multiple aggregation modes are supported, and system performance and expandability are enhanced are achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of cache aggregation computing, and in particular to a multi-level cache aggregation computing method and device. Background Art

[0002] In the field of industrial big data, a large amount of time series data will be collected. The processing of these time series data is a key issue, which directly affects the monitoring, analysis and decision-making of the industrial production process. At present, there are two common processing methods. One is to store the time series data directly in the time series database for aggregation calculation. However, when a large number of calculations are required, the reading performance of the time series database will become a bottleneck, resulting in low computing efficiency. The other is to use a single machine cache to store the intermediate state, but this method also has many limitations when facing a complex industrial data environment. When the time dimension is defined as a short recent time, it is a common, simple and efficient method to store the time series data in the database and aggregate it directly. However, when the time dimension is too long, using the database for aggregation will generate a large amount of calculations and the data response time will be very slow.

[0003] The existing technology has technical problems such as a single time series data aggregation calculation cache method, limited aggregation method, and low data storage and processing efficiency. Summary of the invention

[0004] The present application provides a multi-level cache aggregation calculation method and device for solving the technical problems in the prior art of single cache method for time series data aggregation calculation, limited aggregation method, and low data storage and processing efficiency.

[0005] In view of the above problems, the present application provides a multi-level cache aggregation calculation method and device.

[0006] In a first aspect of the present application, a multi-level cache aggregation calculation method is provided, the method comprising:

[0007] Connect to the message bus, obtain the first-level cache corresponding to the top of the stack, and the subscription time series data corresponding to the message bus are arranged in order; wherein, each data of the first-level cache contains a time dimension, a device dimension and a numerical value; when subscribing to data with the same time dimension and device dimension, perform cumulative aggregation calculation with the original data; define a scheduled task, the scheduled task is the same as the time interval defined by the time dimension of the first-level cache, and determine whether the first event task belongs to the time interval defined by the time dimension of the first-level cache; when the scheduled task corresponding to the first event task is triggered, set a delayed waiting time according to the set watermark; at the same time, transfer the data of the first-level cache to the external cache middleware, define a second-level cache, and if there is initial cache data in the second-level cache, update and merge the data with the same time dimension and device dimension.

[0008] A second aspect of the present application provides a multi-level cache aggregation computing device, the device comprising:

[0009] A first-level cache acquisition module, the first-level cache acquisition module is used to connect to the message bus and obtain the first-level cache corresponding to the top of the stack, and the subscription time series data corresponding to the message bus is arranged in order; a first-level cache content module, wherein each piece of data in the first-level cache includes a time dimension, a device dimension and a value; a cumulative aggregation calculation module, the cumulative aggregation calculation module is used to perform cumulative aggregation calculation with the original data when subscribing to data with the same time dimension and device dimension; a scheduled task definition module, the scheduled task definition module is used to define a scheduled task, the scheduled task is the same as the time interval defined by the time dimension of the first-level cache, and judge whether the first event task belongs to the time interval defined by the time dimension of the first-level cache; a delay waiting time setting module, the delay waiting time setting module is used to set the delay waiting time according to the set watermark when the scheduled task corresponding to the first event task is triggered; a second-level cache definition module, the second-level cache definition module is used to simultaneously transmit the data of the first-level cache to the external cache middleware, define the second-level cache, and if there is initial cache data in the second-level cache, update and merge the data with the same time dimension and device dimension.

[0010] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0011] Connect to the message bus and obtain the first-level cache corresponding to the top of the stack; when subscribing to data with the same time dimension and device dimension, perform cumulative aggregation calculations with the original data; define a scheduled task, the scheduled task is the same as the time interval defined by the time dimension of the first-level cache, and determine whether the first event task belongs to the time interval defined by the time dimension of the first-level cache; when the scheduled task corresponding to the first event task is triggered, set the delay waiting time according to the set watermark; at the same time, transfer the data of the first-level cache to the external cache middleware, define the second-level cache, and if there is initial cache data in the second-level cache, update and merge the data with the same time dimension and device dimension. The technical effect of improving the efficiency of time series data processing, reducing memory usage and latency, supporting multiple aggregation methods, and enhancing system performance and scalability is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0013] Figure 1 A schematic diagram of a multi-level cache aggregation calculation method flow chart provided in an embodiment of the present application;

[0014] Figure 2 A schematic diagram of the structure of a multi-level cache aggregation computing device provided in an embodiment of the present application.

[0015] Explanation of reference numerals: first-level cache acquisition module 10 , first-level cache content module 20 , cumulative aggregation calculation module 30 , scheduled task definition module 40 , delay waiting time setting module 50 , second-level cache definition module 60 . DETAILED DESCRIPTION

[0016] The present application provides a multi-level cache aggregation calculation method and device, which is used to solve the technical problems in the prior art of time series data aggregation calculation with a single cache method, limited aggregation method, and low data storage and processing efficiency.

[0017] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0018] Embodiment 1, as Figure 1 As shown, the present application provides a multi-level cache aggregation calculation method, the method comprising:

[0019] Step S100: Connect to the message bus, obtain the first-level cache corresponding to the top of the stack, and the subscription time series data corresponding to the message bus is arranged in order.

[0020] Specifically, the message bus, as a key intermediary for data transmission, is responsible for orderly transmission of time series data. By establishing a connection with the message bus, the first-level cache at the top of the stack can be obtained. The first-level cache here is the basic part of the entire multi-level cache system. The message bus ensures that the subscribed time series data is arranged in a specific order. This orderliness provides the necessary conditions for the accurate processing and analysis of subsequent data. For example, in an industrial production environment, the time series data generated by a large number of devices is continuously transmitted through the message bus, and this step can accurately obtain the part of the data cache that needs to be processed first, laying the foundation for subsequent calculations and analysis.

[0021] Step S200: wherein each piece of data in the first-level cache includes a time dimension, a device dimension and a value.

[0022] Specifically, each piece of data in the first-level cache has a specific structure, including three key parts: time dimension, device dimension, and value. The time dimension is a definition of data in time, which is defined as a specific time period. This time period is generally between 1 minute and 10 minutes. For example, when it is defined as 5 minutes, 12:00 on May 1, 2024 represents the time interval from 11:55 to 12:00 on May 1, 2024. Through such a time dimension definition, the time range to which the data belongs can be clearly determined, providing a time reference standard for subsequent data processing and analysis. The device dimension is used to distinguish the source device of the data. In a complex industrial environment, there are many different devices that generate data at the same time. The existence of the device dimension makes it clear which specific device generates each piece of data, which is of great significance for subsequent data processing and analysis of different devices. The value part is the specific data value generated by the device in the corresponding time dimension. This value reflects a certain state or indicator of the device in a specific time interval, such as the operating parameters of the device, the production quantity, etc. It is a quantitative description of the operation of the device, providing the original data basis for further data analysis.

[0023] Step S300: When subscribing to data with the same time dimension and device dimension, perform cumulative aggregation calculation with the original data.

[0024] Specifically, when performing data subscription operations in the message bus, the acquired data is carefully judged. Once the data with the same time dimension and device dimension is subscribed, the key operation of cumulative aggregation calculation will be triggered. From the perspective of the time dimension, the newly acquired data meets the conditions for cumulative aggregation only when the time interval matches the existing data. For example, if the time dimension is defined as a 5-minute interval, only data within the same 5-minute interval will be considered for aggregation. From the perspective of the device dimension, it must be ensured that the data comes from the same device. This is because the data generated by different devices may have different characteristics and meanings. Only data from the same device can be aggregated. When the conditions of the same time dimension and device dimension are met, the newly acquired data will be aggregated with the original data. This aggregation calculation method includes multiple methods, such as averaging (AVG), summing (SUM), counting points (COUNT), finding the maximum (MAX), finding the minimum (MIN), finding the first value (FIRST), finding the last value (LAST), finding the second integral (INTEGRAL_S), finding the hour integral (INTEGRAL_H), and finding the difference (DIFF). Through these aggregation computing methods, the data of the same device within a specific time period can be integrated and analyzed to obtain more valuable information, providing strong support for subsequent industrial big data processing and decision-making.

[0025] Step S400: defining a scheduled task, wherein the scheduled task has the same time interval as that defined by the time dimension of the first-level cache, and determining whether a first event task falls within the time interval defined by the time dimension of the first-level cache.

[0026] Specifically, defining a scheduled task is a key operation. The time setting of this scheduled task is closely related to the time interval defined by the time dimension of the first-level cache, and the two remain the same. For example, if the time dimension of the first-level cache is defined as a 5-minute interval, then the scheduled task will be set to execute every 5 minutes. During the execution of the scheduled task, it is necessary to judge the first event task to determine whether the first event task belongs to the time interval defined by the time dimension of the first-level cache. This step is crucial. It ensures that the relevant operations will only be executed if the time interval requirements are met. This judgment mechanism can effectively avoid confusion and errors in data processing and ensure the accuracy and orderliness of the entire system in the time dimension. For example, in industrial big data processing, if the first-level cache processes device data according to a time interval of every 5 minutes, then only the first event task within the corresponding 5-minute interval will be processed correctly, thereby providing an accurate time benchmark for subsequent data transmission and aggregation calculations.

[0027] Step S500: When the scheduled task corresponding to the first event task is triggered, a delay waiting time is set according to the set watermark.

[0028] Specifically, when the scheduled task corresponding to the first event task is triggered, the delay waiting time is determined according to the pre-set watermark. The watermark plays a role of time regulation here. For example, assuming that the watermark is set to 1 minute, then when the scheduled task is triggered, the subsequent operations will not be executed immediately, but will wait for 1 minute. This delay waiting is designed to ensure that there is enough time window in the data stream. In the industrial big data environment, the generation and transmission of data is a complex process with various uncertain factors. By setting the watermark and the corresponding delay waiting time, data loss or inaccurate calculation caused by data transmission delay or other reasons can be avoided. It provides a buffer mechanism for data between different processing stages, making the entire data processing process more robust and reliable.

[0029] Step S600: At the same time, the data of the first-level cache is transferred to the external cache middleware, and the second-level cache is defined. If there is initial cache data in the second-level cache, the data of the same time dimension and device dimension are updated and merged.

[0030] Specifically, the data of the first-level cache is transferred to the external cache middleware, and the second-level cache is defined in this process. The external cache middleware provides another storage level for the data, which helps to further optimize the data processing process. For example, it uses technologies such as Redis to achieve more efficient storage and management. Secondly, if the initial cache data already exists in the second-level cache, the data of the same time dimension and device dimension will be updated and merged. This step ensures the consistency and accuracy of the data. From the perspective of the time dimension, only data belonging to the same time interval will be merged. For example, if the first-level cache processes data with a time dimension of 5 minutes, then in the second-level cache, the data in the same 5-minute time interval will be correctly merged. From the perspective of the device dimension, only data from the same device will be merged to ensure the pertinence and effectiveness of the data. Through this update and merge operation, data can be better integrated, providing a better data foundation for subsequent data analysis and processing.

[0031] In a possible implementation, step S300 further includes:

[0032] Step S310: The corresponding indicators of the cumulative aggregation calculation include AVG value, SUM value, COUNT value, MAX value, MIN value, FIRST value, LAST value, INTEGRAL_S value, INTEGRAL_H value, and DIFF value.

[0033] Specifically, cumulative aggregation calculation covers a variety of important indicators, which quantify and analyze data from different angles to meet the diverse needs of industrial big data processing. Among them, the AVG value is the average value, which is obtained by summing the data of the same device in a specific time dimension and dividing it by the number of data. This indicator can reflect the average operating level or performance of the device in this time period. The SUM value is the sum value, which is to directly add all the data of the same device in the same time dimension, which is used to understand the overall output or cumulative effect of the device in this time period. The COUNT value represents the number of data, that is, the number of data points under a specific time dimension and device dimension. It is of great significance to understand the integrity and sample quantity of the data. The MAX value is the maximum value, which is used to find the maximum value of the data generated by the same device in a given time dimension, which helps to find the optimal operating state or limit of the device in this time period. The MIN value is the minimum value. In contrast, it determines the minimum value of the data of the same device in the same time dimension. The MIN value is used to understand the lowest operating level of the equipment or the lower limit of possible abnormalities. The FIRST value is the first value, that is, the first data point generated by the same equipment in a specific time dimension. It is helpful for analyzing the situation or initial state of the equipment startup phase. The LAST value is the last value, which represents the last data point generated by the same equipment in this time dimension. This is of reference value for studying the state change of the equipment at the end of this time period. The INTEGRAL_S value is the second-level integration. It processes the data of the equipment on the second-level time scale from the perspective of integration, and can more carefully analyze the dynamic changes of the equipment in a short period of time. The INTEGRAL_H value is the hour-level integration, which is similar to the second-level integration, but integrates the data on the hourly time scale, which is suitable for analyzing the overall trend of the equipment over a longer period of time. The DIFF value is the difference, which is usually calculated between adjacent data points. It is used to analyze the rate of change or fluctuation of equipment data, so as to understand the stability and dynamic characteristics of the equipment. By calculating and analyzing these indicators, we can fully and deeply understand the operation of the equipment in a specific time dimension, and provide rich information support for the processing and decision-making of industrial big data.

[0034] In a possible implementation, step S300 further includes:

[0035] Step S320: Update the first-level cache by accumulating and aggregating the corresponding indicators.

[0036] Specifically, the first-level cache is updated by processing each indicator corresponding to the cumulative aggregation calculation. For each indicator, there is a specific calculation method and impact on the cache update. For example, when calculating the AVG value, the summation result and the number of data need to be cached separately so that the average value can be updated in real time. This means that in the first-level cache, in addition to the original data, additional space is required to store these intermediate results used to calculate the average value. For the SUM value, since all data of the same device in the same time dimension are directly added, when updating the first-level cache, it is only necessary to replace the original corresponding value with the newly calculated SUM value. The update of the COUNT value is based on the original data quantity and is accumulated according to the newly acquired data. This ensures that the first-level cache always stores accurate data quantity information for subsequent calculations of indicators such as the average value. The update of the MAX value and the MIN value requires comparing the newly acquired data with the maximum and minimum values ​​already in the cache. If the new data is greater than the MAX value, the MAX value is updated; if the new data is less than the MIN value, the MIN value is updated. The update of FIRST and LAST values ​​is relatively simple. When new data is generated, the FIRST value (if it is the first data in the time dimension) or LAST value (if it is the last data in the time dimension) is updated according to its position in the time dimension. The update of INTEGRAL_S and INTEGRAL_H values ​​needs to include the new data in the integral calculation range according to their specific integral calculation method, and update the corresponding integral value in the cache. The update of DIFF value needs to calculate the difference between adjacent data points and store it in the cache for subsequent analysis of the change rate or fluctuation of device data. Through such processing of various indicators and corresponding update of the first-level cache, the first-level cache can reflect the latest operation status of the device in a specific time dimension in real time, providing accurate and timely basic data for subsequent data processing and analysis.

[0037] In a possible implementation, step S400 further includes:

[0038] Step S410: The time period interval defined in the first-level cache is a time window.

[0039] Step S420: The second-level cache uses Redis's zset for storage.

[0040] Specifically, an important feature of the first-level cache is clarified. In the first-level cache, the defined time interval is regarded as a time window, which is a way to divide data in the time dimension. For example, the time window can be defined as a 5-minute interval, which means that the data within this time period will be processed and analyzed as a whole. By setting the time window, the scope of the data can be more clearly defined, which is convenient for subsequent operations such as cumulative aggregation calculations on the data. It also provides a time benchmark for the setting of scheduled tasks.

[0041] As the data processing flow progresses, the storage method of the second-level cache is involved. The second-level cache uses Redis's Zset for storage. Zset is an ordered set with a unique storage structure and characteristics. Among them, the score of Zset can be used for sorting, usually used to store timestamps, so that data can be arranged in chronological order to ensure the orderliness of data in the time dimension, while the value is used to store the value of the data, and the value of Zset is a non-repeatable element. In order to ensure the uniqueness of the data, it is also necessary to add some uniqueness constraints, such as adding timestamps as uniqueness constraints. This storage method enables the second-level cache to efficiently store and manage data transmitted from the first-level cache, providing a good foundation for subsequent data query and analysis.

[0042] In a possible implementation, step S420 further includes:

[0043] Step S421: the score of zset can be sorted to store timestamps, and the value is used to store values. The value of zset is a non-repeatable element.

[0044] Specifically, the score of zset has the feature of being sortable, which is cleverly used to store timestamps. By storing timestamps in the score, data can be arranged in chronological order. For example, in the industrial big data environment, data generated at different time points can be stored in zset in order according to their corresponding timestamps, so that when querying and analyzing data, data can be easily obtained in chronological order, providing a time dimension order basis for data processing. Secondly, the value of zset is used to store the value of data, which carries the specific data content transmitted from the first-level cache. However, it should be noted that the value of zset is a non-repeatable element. This feature ensures the uniqueness of the data and avoids repeated storage and confusion of data. In order to further ensure the uniqueness of the data, it is usually necessary to add some uniqueness constraints. For example, a timestamp can be added as a uniqueness constraint, that is, when two data have the same value but different timestamps, they can still be distinguished by the timestamp, thereby ensuring the accuracy and uniqueness of the data during storage. This unique storage structure and constraint mechanism enables zset to efficiently store and manage data in the second-level cache, providing a reliable foundation for subsequent data processing and analysis.

[0045] In a possible implementation, step S421 further includes:

[0046] Step S4211: Add a timestamp as a uniqueness constraint based on value.

[0047] Specifically, according to the characteristics of the value in zset, a measure of adding a timestamp as a unique constraint is taken. Since the value of zset is a non-repeatable element, in actual data processing, there will be situations where the data value is the same but generated at different times. In order to ensure the uniqueness and accuracy of the data, an effective way of distinguishing is required. As a marker that can identify the time when the data is generated, the timestamp is used as a unique constraint added to the value. For example, when there are two data with the same value, but they are generated at different time points, they can be clearly distinguished by adding a corresponding timestamp to each value. In this way, during data storage and subsequent processing, even if the data value is repeated, different data can be accurately identified based on the timestamp, avoiding data confusion and erroneous processing, thereby ensuring the accuracy and reliability of the data in the second-level cache and providing a solid foundation for the entire data processing process.

[0048] In a possible implementation, step S600 further includes:

[0049] Step S610: The time interval defined by the time dimension of the second-level cache is larger than the time interval defined by the first-level cache.

[0050] Specifically, the important relationship between the second-level cache and the first-level cache in the time dimension is clarified. The time interval defined by the time dimension of the second-level cache is larger than that defined by the first-level cache. This design is based on the overall architecture and data processing requirements of the multi-level cache. The first-level cache usually processes data within a shorter time interval, for example, its time dimension may be defined as a 5-minute interval, and is mainly used to quickly obtain and preliminarily process data generated by the device within a short time period. The second-level cache is responsible for integrating and storing data in a longer time range. The time interval defined by its time dimension is larger, for example, it may be 15 minutes or longer. This design enables data to be reasonably stored and processed in caches at different levels according to the progressive relationship of the time dimension. From the perspective of the data processing process, after the data in the first-level cache is initially processed, it is transferred to the second-level cache for further integration and storage. As the time dimension expands, the second-level cache can accommodate more data within the time range, thereby providing a more comprehensive data foundation for subsequent more macro data analysis and decision-making. This hierarchical time dimension design helps to improve the efficiency and accuracy of data processing and better meet the complex needs of industrial big data processing.

[0051] In a possible implementation, step S600 further includes:

[0052] Step S620: The time dimension interval of each level of cache is larger than the time interval of the previous level, and the size of the time interval defined by the first level of cache defines the minimum update frequency of data.

[0053] Specifically, the time dimension interval of each level of cache is larger than that of the previous level. This is a hierarchical progressive design concept. For example, the time dimension interval of the first-level cache may be 5 minutes, the second-level cache may be 15 minutes, and the third-level cache may be 30 minutes. This design allows the time range covered by data in caches at different levels to gradually expand as the cache level increases. At the same time, the size of the time interval defined by the first-level cache defines the minimum update frequency of the data. This means that the data is updated in the first-level cache according to this time interval. Taking the first-level cache time dimension of 5 minutes as an example, the data will be updated every 5 minutes, including data acquisition, aggregation calculation, and cache update. This minimum update frequency setting provides a basic time rhythm for the entire data processing process, which ensures that the data can be processed and updated in a timely manner to avoid data obsolescence or untimely processing. Moreover, this design based on the time dimension interval allows caches at each level to perform their respective duties, from the first-level cache that quickly processes short-term data to the higher-level cache that integrates long-term data, jointly building an efficient and orderly industrial big data processing system.

[0054] Embodiment 2 is based on the same inventive concept as a multi-level cache aggregation calculation method in the above embodiment. Figure 2 As shown, the present application provides a multi-level cache aggregation computing device, and the device and method embodiments in the present application embodiments are based on the same inventive concept. The device includes:

[0055] The first-level cache acquisition module 10 is used to connect to the message bus and acquire the first-level cache corresponding to the top of the stack. The subscription time series data corresponding to the message bus are arranged in order.

[0056] The first-level cache content module 20, wherein each piece of data in the first-level cache includes a time dimension, a device dimension and a value.

[0057] The cumulative aggregation calculation module 30 is used to perform cumulative aggregation calculation with the original data when subscribing to data with the same time dimension and device dimension.

[0058] The scheduled task definition module 40 is used to define a scheduled task, which has the same time interval as that defined by the time dimension of the first-level cache, and to determine whether the first event task falls within the time interval defined by the time dimension of the first-level cache.

[0059] The delay waiting time setting module 50 is used to set the delay waiting time according to the set watermark when the scheduled task corresponding to the first event task is triggered.

[0060] The second-level cache definition module 60 is used to simultaneously transfer the data of the first-level cache to the external cache middleware and define the second-level cache. If there is initial cache data in the second-level cache, the data of the same time dimension and device dimension will be updated and merged.

[0061] Furthermore, the cumulative aggregation calculation module 30 further includes:

[0062] The indicators corresponding to the cumulative aggregation calculation include AVG value, SUM value, COUNT value, MAX value, MIN value, FIRST value, LAST value, INTEGRAL_S value, INTEGRAL_H value, and DIFF value.

[0063] Furthermore, the cumulative aggregation calculation module 30 further includes:

[0064] The first-level cache updating unit is used to update the first-level cache by accumulating and aggregating corresponding indicators.

[0065] Furthermore, the scheduled task definition module 40 also includes:

[0066] A time window determination unit is used to define a time period interval in the first-level cache as a time window.

[0067] The second-level cache storage unit is used for the second-level cache and uses the zset of redis for storage.

[0068] Furthermore, the second-level cache storage unit also includes:

[0069] A timestamp storage unit, wherein the score of a zset can be sorted to store timestamps, and the value is used to store values. The value of a zset is a non-repeatable element.

[0070] Furthermore, the timestamp storage unit further includes:

[0071] A uniqueness constraint determination unit, wherein the uniqueness constraint determination unit is used to add a timestamp as a uniqueness constraint according to the value.

[0072] Furthermore, the second-level cache definition module 60 also includes:

[0073] The time interval defined by the time dimension of the second-level cache is larger than the time interval defined by the first-level cache.

[0074] Furthermore, the second-level cache definition module 60 also includes:

[0075] The time dimension interval of each level of cache is larger than the time interval of the previous level, and the size of the time interval defined by the first level of cache defines the minimum update frequency of data.

[0076] It should be noted that the above-mentioned sequence of the embodiments of the present application is only for description and does not represent the advantages and disadvantages of the embodiments. And the above-mentioned specific embodiments of this specification are described. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0077] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

[0078] This specification and the drawings are merely exemplary illustrations of the present application and are deemed to cover any and all modifications, variations, combinations or equivalents within the scope of the present application. Obviously, a person skilled in the art may make various modifications and variations to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the present application and its equivalents, the present application intends to include these modifications and variations.

Claims

1. A multi-level cache aggregation calculation method, characterized in that: The method comprises: Connect to the message bus, obtain the first-level cache corresponding to the top of the stack, and the subscription time series data corresponding to the message bus is arranged in order; Wherein, each piece of data in the first-level cache includes a time dimension, a device dimension and a value; When subscribing to data with the same time dimension and device dimension, the data is aggregated and calculated with the original data; Defining a scheduled task, wherein the scheduled task has the same time interval as that defined by the time dimension of the first-level cache, and determining whether the first event task falls within the time interval defined by the time dimension of the first-level cache; When the scheduled task corresponding to the first event task is triggered, a delay waiting time is set according to the set watermark; At the same time, the data of the first-level cache is transferred to the external cache middleware, and the second-level cache is defined. If there is initial cache data in the second-level cache, the data with the same time dimension and device dimension are updated and merged.

2. A multi-level cache aggregation calculation method as claimed in claim 1, characterized in that: The indicators corresponding to the cumulative aggregation calculation include AVG value, SUM value, COUNT value, MAX value, MIN value, FIRST value, LAST value, INTEGRAL_S value, INTEGRAL_H value, and DIFF value.

3. A multi-level cache aggregation calculation method as claimed in claim 2, characterized in that: The method comprises: The first-level cache is updated by accumulating and aggregating the corresponding indicators.

4. A multi-level cache aggregation calculation method as claimed in claim 3, characterized in that: The method comprises: The time period interval defined in the first-level cache is the time window; The second-level cache uses redis's zset for storage.

5. A multi-level cache aggregation calculation method as claimed in claim 4, characterized in that: The score of zset can be sorted to store timestamps, and the value is used to store values. The value of zset is a non-repeatable element.

6. A multi-level cache aggregation calculation method as claimed in claim 5, characterized in that: Add a timestamp as a unique constraint based on the value.

7. A multi-level cache aggregation calculation method as claimed in claim 6, characterized in that: The time interval defined by the time dimension of the second-level cache is larger than the time interval defined by the first-level cache.

8. A multi-level cache aggregation calculation method as claimed in claim 7, characterized in that: The time dimension interval of each level of cache is larger than the time interval of the previous level, and the size of the time interval defined by the first level of cache defines the minimum update frequency of data.

9. A multi-level cache aggregation computing device, characterized in that: The device comprises: A first-level cache acquisition module, the first-level cache acquisition module is used to connect to the message bus and acquire the first-level cache corresponding to the top of the stack, and the subscription time series data corresponding to the message bus is arranged in order; A first-level cache content module, wherein each piece of data in the first-level cache includes a time dimension, a device dimension, and a value; A cumulative aggregation calculation module, which is used to perform cumulative aggregation calculation with the original data when subscribing to data with the same time dimension and device dimension; A scheduled task definition module, the scheduled task definition module is used to define a scheduled task, the scheduled task has the same time interval as that defined by the time dimension of the first-level cache, and determines whether the first event task falls within the time interval defined by the time dimension of the first-level cache; A delay waiting time setting module, wherein the delay waiting time setting module is used to set the delay waiting time according to the set watermark when the scheduled task corresponding to the first event task is triggered; The second-level cache definition module is used to simultaneously transfer the data of the first-level cache to the external cache middleware and define the second-level cache. If there is initial cache data in the second-level cache, the data of the same time dimension and device dimension will be updated and merged.

Images