A multi-level cache aggregation computing method and device

Abstract

The application discloses a kind of multistage cache aggregation computing method and device, it is related to cache aggregation computing technical field, the method includes: connecting message bus, obtains the first level cache corresponding to stack top;When subscribing to the data of same time dimension and device dimension, cumulative aggregation calculation is carried out with original data;Whether the first event task belongs to the time interval defined in range is judged;When the timing task corresponding to the first event task triggers, set delay waiting time;At the same time, the data of the first level cache is transmitted to external cache middleware, and the data of same time dimension, device dimension is updated and merged.The application solves the technical problems of single cache mode, limited aggregation mode, low data storage and processing efficiency in the prior art, improves the efficiency of time series data processing, reduces memory usage and delay, supports multiple aggregation modes, and enhances the technical effects of system performance and scalability.

Description

Technical Field

[0001] This invention relates to the field of cache aggregation computing technology, and specifically to a multi-level cache aggregation computing method and apparatus. Background Technology

[0002] In the field of industrial big data, a large amount of time-series data is collected. Processing this time-series data is a critical issue, directly impacting monitoring, analysis, and decision-making in industrial production processes. Currently, there are two main processing methods. One is to directly store the time-series data in a time-series database for aggregation calculations. However, when large-scale calculations are required, the read performance of the time-series database becomes a bottleneck, leading to low computational efficiency. The other method is to use a single-machine cache to store intermediate states, but this approach also has many limitations when facing complex industrial data environments. When the time dimension is defined as a relatively recent period, storing the time-series data directly in a database for aggregation is a common, simple, and efficient method. However, when the time dimension is too long, using a database for aggregation will generate a large amount of computation, and the data response time will be very slow.

[0003] Existing technologies suffer from problems such as a single caching method for time-series data aggregation and computation, limited aggregation methods, and low data storage and processing efficiency. Summary of the Invention

[0004] This application provides a multi-level cache aggregation calculation method and apparatus to address the technical problems of single caching method, limited aggregation method, and low data storage and processing efficiency in the prior art for time-series data aggregation calculation.

[0005] In view of the above problems, this application provides a multi-level cache aggregation calculation method and apparatus.

[0006] A first aspect of this application provides a multi-level cache aggregation calculation method, the method comprising:

[0007] Connect to the message bus and retrieve the first-level cache corresponding to the top of the stack. The subscription time-series data corresponding to the message bus is arranged in order. Each piece of data in the first-level cache includes a time dimension, a device dimension, and a value. When data with the same time dimension and device dimension is subscribed to, it is aggregated and calculated with the original data. Define a scheduled task. The time interval defined by the time dimension of the scheduled task is the same as that defined by the time dimension of the first-level cache. Determine whether the first event task belongs to the time interval defined by the time dimension of the first-level cache. When the scheduled task corresponding to the first event task is triggered, a delay waiting time is set according to the set watermark. At the same time, the data of the first-level cache is transmitted to an external cache middleware to define a second-level cache. If the second-level cache contains initial cache data, the data with the same time dimension and device dimension are updated and merged.

[0008] A second aspect of this application provides a multi-level cache aggregation computing apparatus, the apparatus comprising:

[0009] The first-level cache acquisition module connects to the message bus and acquires the first-level cache corresponding to the top of the stack. The subscription time-series data corresponding to the message bus are arranged in order. The first-level cache content module contains a time dimension, a device dimension, and a numerical value for each piece of data in the first-level cache. The cumulative aggregation calculation module performs cumulative aggregation calculations with existing data when data with the same time and device dimensions is subscribed to. The scheduled task definition module defines scheduled tasks with the same time interval defined by the time dimension of the first-level cache, and determines whether the first event task falls within the time interval defined by the time dimension of the first-level cache. The delay waiting time setting module sets a delay waiting time based on a set watermark when the scheduled task corresponding to the first event task is triggered. The second-level cache definition module simultaneously transmits the data from the first-level cache to an external cache middleware to define a second-level cache. If the second-level cache contains initial cache data, it updates and merges data with the same time and device dimensions.

[0010] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0011] The system connects to the message bus and retrieves the first-level cache corresponding to the top of the stack. When data with the same time and device dimensions is subscribed to, it performs cumulative aggregation calculations with the existing data. A scheduled task is defined, with the time interval defined by the time dimension of the first-level cache being the same. It determines whether the first event task falls within the time interval defined by the time dimension of the first-level cache. When the scheduled task corresponding to the first event task is triggered, a delay waiting time is set according to the set watermark. Simultaneously, the data from the first-level cache is transmitted to an external cache middleware, defining a second-level cache. If the second-level cache contains initial cache data, data with the same time and device dimensions are updated and merged. This achieves the technical effects of improving the efficiency of time-series data processing, reducing memory usage and latency, supporting multiple aggregation methods, and enhancing system performance and scalability. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This application provides a schematic flowchart of a multi-level cache aggregation calculation method.

[0014] Figure 2 This is a schematic diagram of a multi-level cache aggregation computing device provided in an embodiment of this application.

[0015] Explanation of reference numerals in the attached diagram: Level 1 cache acquisition module 10, Level 1 cache content module 20, cumulative aggregation calculation module 30, scheduled task definition module 40, delay waiting time setting module 50, and Level 2 cache definition module 60. Detailed Implementation

[0016] This application provides a multi-level cache aggregation calculation method and apparatus to address the technical problems in the prior art, such as the single caching method, limited aggregation method, and low data storage and processing efficiency of time-series data aggregation calculation.

[0017] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0018] Example 1, as Figure 1 As shown, this application provides a multi-level cache aggregation calculation method, the method comprising:

[0019] Step S100: Connect to the message bus and obtain the first-level cache corresponding to the top of the stack. The subscription timing data corresponding to the message bus is arranged in order.

[0020] Specifically, the message bus, as a key intermediary for data transmission, is responsible for the orderly delivery of time-series data. By establishing a connection with the message bus, the first-level cache at the top of the stack can be accessed. This first-level cache is the foundation of the entire multi-level caching system. The message bus ensures that the subscribed time-series data is arranged in a specific order; this orderliness provides the necessary conditions for accurate processing and analysis of subsequent data. For example, in an industrial production environment, time-series data generated by a large number of devices is continuously transmitted through the message bus. This step can accurately acquire the cached data that needs to be processed first, laying the foundation for subsequent calculations and analysis.

[0021] Step S200: Each piece of data in the first-level cache includes a time dimension, a device dimension, and a value.

[0022] Specifically, each piece of data in the first-level cache has a specific structure, comprising three key parts: time dimension, device dimension, and numerical value. The time dimension defines the data within a specific time period. This period typically ranges from 1 to 10 minutes. For example, if defined as 5 minutes, 12:00 on May 1, 2024 represents the time interval from 11:55 to 12:00 on May 1, 2024. This time dimension definition clearly identifies the time range to which the data belongs, providing a temporal reference standard for subsequent data processing and analysis. The device dimension distinguishes the source device of the data. In complex industrial environments, numerous different devices generate data simultaneously. The device dimension clarifies which specific device generated each piece of data, which is crucial for subsequent data processing and analysis targeting different devices. The numerical value is the specific data value generated by the device within the corresponding time dimension. This value reflects a certain state or indicator of the device within a specific time interval, such as operating parameters or production quantity. It is a quantitative description of the device's operating status, providing the initial data foundation for further data analysis.

[0023] Step S300: When subscribing to data with the same time dimension and device dimension, perform cumulative aggregation calculation with the original data.

[0024] Specifically, when performing data subscription operations in the message bus, the acquired data undergoes careful evaluation. Once data with the same time and device dimensions is subscribed to, the crucial cumulative aggregation calculation is triggered. From a time perspective, cumulative aggregation is only performed when the newly acquired data matches existing data within a specific time interval. For example, if the time dimension is defined as a 5-minute interval, only data within the same 5-minute interval will be considered for aggregation. From a device perspective, it is essential to ensure that the data originates from the same device. This is because data from different devices may possess different characteristics and meanings, and aggregation of data from the same device is only meaningful. Once the conditions of identical time and device dimensions are met, the newly acquired data is cumulatively aggregated with the existing data. This aggregation calculation includes various methods, such as averaging (AVG), summing (SUM), counting (COUNT), finding the maximum (MAX), finding the minimum (MIN), finding the first value (FIRST), finding the last value (LAST), calculating the second-level integral (INTEGRAL_S), calculating the hourly integral (INTEGRAL_H), and calculating the difference (DIFF). These aggregation computing methods integrate and analyze data from the same device within a specific time interval to obtain more valuable information, providing strong support for subsequent industrial big data processing and decision-making.

[0025] Step S400: Define a scheduled task, wherein the scheduled task is the same as the time interval defined by the time dimension of the first-level cache, and determine whether the first event task belongs to the time interval defined by the time dimension of the first-level cache.

[0026] Specifically, defining a scheduled task is a crucial operation. The timing of this scheduled task is closely related to the time interval defined by the time dimension of the first-level cache, and the two must be identical. For example, if the time dimension of the first-level cache is defined as a 5-minute interval, then the scheduled task will be set to execute every 5 minutes. During the execution of the scheduled task, it is necessary to determine whether the first event task falls within the time interval defined by the time dimension of the first-level cache. This step is critical, ensuring that only operations are executed if the time interval requirements are met. This determination mechanism effectively avoids data processing chaos and errors, guaranteeing the accuracy and orderliness of the entire system in the time dimension. For example, in industrial big data processing, if the first-level cache processes equipment data according to a 5-minute time interval, then only the first event task within the corresponding 5-minute interval will be correctly processed, thus providing an accurate time reference for subsequent data transmission and aggregation calculations.

[0027] Step S500: When the timed task corresponding to the first event task is triggered, the delay waiting time is set according to the watermark.

[0028] Specifically, when the scheduled task corresponding to the first event task is triggered, a delay time is determined based on a pre-set watermark. The watermark acts as a time control mechanism. For example, assuming a watermark of 1 minute, subsequent operations will not be executed immediately after the scheduled task is triggered; instead, a 1-minute delay will occur. This delay design ensures sufficient time windows within the data stream. In industrial big data environments, data generation and transmission are complex processes with various uncertainties. By setting watermarks and corresponding delay times, data loss or inaccurate calculations due to data transmission delays or other reasons can be avoided. It provides a buffer mechanism between different processing stages, making the entire data processing flow more robust and reliable.

[0029] Step S600: Simultaneously, the data of the first-level cache is transferred to the external cache middleware, and a second-level cache is defined. If the second-level cache contains initial cache data, the data with the same time dimension and device dimension are updated and merged.

[0030] Specifically, the data from the first-level cache is transferred to an external caching middleware, which defines the second-level cache. The external caching middleware provides another storage layer for the data, helping to further optimize the data processing flow. For example, it uses technologies like Redis to achieve more efficient storage and management. Secondly, if the initial cached data already exists in the second-level cache, data with the same time and device dimensions will be updated and merged. This step ensures data consistency and accuracy. From a time perspective, only data belonging to the same time interval will be merged. For example, if the first-level cache processes data in 5-minute intervals, then data within the same 5-minute interval will be correctly merged in the second-level cache. From a device perspective, only data from the same device will be merged to ensure data relevance and validity. Through this update and merge operation, data can be better integrated, providing a higher-quality data foundation for subsequent data analysis and processing.

[0031] In one possible implementation, step S300 further includes:

[0032] Step S310: The cumulative aggregation calculation includes the following indicators: AVG value, SUM value, COUNT value, MAX value, MIN value, FIRST value, LAST value, INTEGRAL_S value, INTEGRAL_H value, and DIFF value.

[0033] Specifically, cumulative aggregation calculations encompass a variety of important indicators, which quantify and analyze data from different perspectives to meet the diverse needs of industrial big data processing. Among them, the AVG value, or average value, is obtained by summing the data from the same device over a specific time dimension and dividing by the number of data points. This indicator reflects the average operating level or performance of the device during that time period. The SUM value is a summation value, directly adding all data from the same device over the same time dimension, used to understand the overall output or cumulative effect of the device during that time period. The COUNT value represents the number of data points, i.e., the number of data points counted in a specific time and device dimension, which is important for understanding data completeness and sample size. The MAX value is the maximum value, used to find the maximum value of the data generated by the same device within a given time dimension, which helps to discover the optimal operating state or extreme conditions of the device during that time period. The MIN value is the minimum value, conversely, determining the minimum value of the data from the same device within the same time dimension. The MIN value reveals the minimum operating level or potential lower limit of anomalies for the equipment. The FIRST value is the first value, representing the first data point generated by the same equipment within a specific time dimension, helpful for analyzing the equipment's startup phase or initial state. The LAST value is the last value, representing the last data point generated by the same equipment within that time dimension, valuable for studying changes in the equipment's state at the end of that time period. The INTEGRAL_S value is a second-level integral, processing equipment data on a second-level time scale from an integral perspective, enabling more detailed analysis of the equipment's dynamic changes over a short period. The INTEGRAL_H value is an hour-level integral, similar to the second-level integral, but integrating data on an hourly time scale, suitable for analyzing the overall trend of the equipment over a longer period. The DIFF value is the difference, typically calculating the difference between adjacent data points, used to analyze the rate of change or fluctuation of equipment data, thereby understanding the equipment's stability and dynamic characteristics. Through the calculation and analysis of these indicators, a comprehensive and in-depth understanding of the equipment's operation within a specific time dimension can be achieved, providing rich information support for industrial big data processing and decision-making.

[0034] In one possible implementation, step S300 further includes:

[0035] Step S320: Update the first-level cache by accumulating and aggregating the corresponding metrics.

[0036] Specifically, the first-level cache is updated by processing the various metrics corresponding to the cumulative aggregation calculation. Each metric has a specific calculation method and its impact on cache updates. For example, when calculating the AVG value, the summation result and the number of data points need to be cached separately for real-time updates of the average. This means that in the first-level cache, in addition to the original data, extra space is needed to store these intermediate results used to calculate the average. For the SUM value, since it directly adds up all data from the same device within the same time dimension, updating the first-level cache simply requires replacing the original corresponding value with the newly calculated SUM value. The COUNT value is updated by accumulating the newly acquired data based on the existing data quantity. This ensures that the first-level cache always retains accurate data quantity information for use in subsequent calculations of metrics such as the average. The MAX and MIN values ​​are updated by comparing the newly acquired data with the existing maximum and minimum values ​​in the cache. If the new data is greater than the MAX value, the MAX value is updated; if the new data is less than the MIN value, the MIN value is updated. Updating the FIRST and LAST values ​​is relatively simple. When new data is generated, the FIRST value (if it's the first data point in that time dimension) or the LAST value (if it's the last data point in that time dimension) is updated based on its position in the time dimension. Updating the INTEGRAL_S and INTEGRAL_H values ​​requires incorporating the new data into the integration calculation based on their specific integration methods and updating the corresponding integration values ​​in the cache. Updating the DIFF value requires calculating the difference between adjacent data points and storing it in the cache for subsequent analysis of the rate of change or fluctuation of device data. Through this processing of various indicators and corresponding updates to the first-level cache, the first-level cache can reflect the latest operating status of the device in a specific time dimension in real time, providing accurate and timely basic data for subsequent data processing and analysis.

[0037] In one possible implementation, step S400 further includes:

[0038] Step S410: The time period defined in the first-level cache is the time window.

[0039] Step S420: The second-level cache uses Redis's zset for storage.

[0040] Specifically, a key characteristic of the first-level cache is that the defined time interval is considered a time window. This time window is a way of dividing data in the time dimension. For example, a time window can be defined as a 5-minute interval, which means that data within this time period will be processed and analyzed as a whole. By setting a time window, the range of data can be more clearly defined, facilitating subsequent operations such as cumulative aggregation calculations, and also providing a time benchmark for setting scheduled tasks.

[0041] As the data processing flow progresses, the storage method for the second-level cache becomes relevant. The second-level cache uses Redis's sorted set (zset) for storage. A zset is an ordered set with unique storage structure and characteristics. The zset's score can be used for sorting, typically for storing timestamps, ensuring data is ordered chronologically. The value stores the data's information, and zset values ​​are unique elements. To guarantee data uniqueness, some uniqueness constraint is added, such as using timestamps. This storage method allows the second-level cache to efficiently store and manage data transferred from the first-level cache, providing a solid foundation for subsequent data queries and analysis.

[0042] In one possible implementation, step S420 further includes:

[0043] Step S421: The score of a zset can be sorted to store timestamps, and the value is used to store values. The value of a zset is a unique element.

[0044] Specifically, the sortable nature of a zset's score is cleverly utilized to store timestamps. By storing timestamps in the score, data can be arranged chronologically. For example, in an industrial big data environment, data generated at different points in time can be stored in an ordered zset based on their corresponding timestamps. This allows for convenient chronological retrieval of data during querying and analysis, providing a temporal order basis for data processing. Secondly, the zset's value stores the data's intrinsic value, carrying the specific data content transferred from the first-level cache. However, it's important to note that zset values ​​are non-repeatable elements. This characteristic ensures data uniqueness, preventing duplicate storage and confusion. To further guarantee data uniqueness, some uniqueness constraint is usually added. For example, a timestamp can be added as a uniqueness constraint; that is, when two data items have the same value but different timestamps, they can still be distinguished by the timestamp, thus ensuring the accuracy and uniqueness of the data during storage. This unique storage structure and constraint mechanism enable zsets to efficiently store and manage data in the second-level cache, providing a reliable foundation for subsequent data processing and analysis.

[0045] In one possible implementation, step S421 further includes:

[0046] Step S4211: Add a timestamp as a unique constraint based on the value.

[0047] Specifically, based on the characteristics of values ​​in a zset, a timestamp was added as a uniqueness constraint. Since zset values ​​are non-repeatable, in actual data processing, there may be cases where data values ​​are identical but generated at different times. To ensure data uniqueness and accuracy, an effective way to distinguish them is needed. A timestamp, as a marker that identifies when data was generated, is added to the value as a uniqueness constraint. For example, when two data values ​​are identical but generated at different times, adding a corresponding timestamp to each value can clearly distinguish them. In this way, even if data values ​​are duplicated during data storage and subsequent processing, different data can be accurately identified based on the timestamp, avoiding data confusion and incorrect handling. This ensures the accuracy and reliability of data in the second-level cache, providing a solid foundation for the entire data processing flow.

[0048] In one possible implementation, step S600 further includes:

[0049] Step S610: The time interval defined by the time dimension of the second-level cache is larger than the time interval defined by the first-level cache.

[0050] Specifically, the important relationship between the second-level cache and the first-level cache in terms of time dimension is clarified. The time interval defined by the second-level cache is larger than that defined by the first-level cache. This design is based on the overall architecture of multi-level caching and data processing needs. The first-level cache typically processes data within a shorter time interval, such as a 5-minute interval, mainly used for quickly acquiring and initially processing data generated by devices within a short period. The second-level cache, on the other hand, undertakes the task of integrating and storing data over a longer time range. Its time interval is defined as larger, such as 15 minutes or longer. This design allows data to be rationally stored and processed in different cache levels according to the progressive relationship of the time dimension. From the data processing flow perspective, after initial processing, the data in the first-level cache is transferred to the second-level cache for further integration and storage. As the time dimension expands, the second-level cache can accommodate data within a larger time range, thus providing a more comprehensive data foundation for subsequent macro-level data analysis and decision-making. This hierarchical time dimension design helps improve the efficiency and accuracy of data processing and better meets the complex needs of industrial big data processing.

[0051] In one possible implementation, step S600 further includes:

[0052] Step S620: The time dimension interval of each level of cache is larger than that of the previous level. The size of the time interval defined by the first level of cache defines the minimum update frequency of the data.

[0053] Specifically, each level of cache has a larger time interval than the level above it, reflecting a hierarchical design philosophy. For example, the first-level cache might have a 5-minute time interval, the second-level cache 15 minutes, and the third-level cache 30 minutes, and so on. This design ensures that the time range covered by data gradually expands as the cache level increases. Simultaneously, the size of the time interval defined by the first-level cache defines the minimum update frequency. This means that data in the first-level cache is updated according to this time interval. Taking a 5-minute time interval for the first-level cache as an example, an update operation is performed every 5 minutes, including data acquisition, aggregation calculations, and cache updates. This minimum update frequency setting provides a basic time rhythm for the entire data processing flow, ensuring that data is processed and updated in a timely manner, avoiding data staleness or delayed processing. Moreover, this time interval-based design allows each level of cache to perform its specific function, from the first-level cache quickly processing short-term data to the higher-level cache integrating long-term data, collectively building an efficient and orderly industrial big data processing system.

[0054] Example 2, based on the same inventive concept as the multi-level cache aggregation calculation method in the foregoing examples, such as... Figure 2 As shown, this application provides a multi-level cache aggregation computing device. The device and method embodiments in this application are based on the same inventive concept. The device includes:

[0055] The first-level cache acquisition module 10 is used to connect to the message bus and acquire the first-level cache corresponding to the top of the stack. The subscription timing data corresponding to the message bus is arranged in order.

[0056] The first-level cache content module 20, wherein each piece of data in the first-level cache includes a time dimension, a device dimension, and a value.

[0057] The cumulative aggregation calculation module 30 is used to perform cumulative aggregation calculation with the original data when subscribing to data with the same time dimension and device dimension.

[0058] The scheduled task definition module 40 is used to define scheduled tasks. The scheduled task has the same time interval as the time dimension defined by the first-level cache. The module determines whether the first event task belongs to the time interval defined by the time dimension of the first-level cache.

[0059] The delay waiting time setting module 50 is used to set a delay waiting time according to the set watermark when the timed task corresponding to the first event task is triggered.

[0060] The second-level cache definition module 60 is used to simultaneously transmit the data of the first-level cache to an external cache middleware and define the second-level cache. If there is initial cache data in the second-level cache, the data with the same time dimension and device dimension will be updated and merged.

[0061] Furthermore, the cumulative aggregation calculation module 30 also includes:

[0062] The cumulative aggregation calculations correspond to the following metrics: AVG value, SUM value, COUNT value, MAX value, MIN value, FIRST value, LAST value, INTEGRAL_S value, INTEGRAL_H value, and DIFF value.

[0063] Furthermore, the cumulative aggregation calculation module 30 also includes:

[0064] The first-level cache update unit is used to update the first-level cache by accumulating and aggregating the corresponding indicators.

[0065] Furthermore, the scheduled task definition module 40 also includes:

[0066] A time window determination unit is used to define a time period interval as the time window in the first-level cache.

[0067] The second-level cache storage unit is used for storing data in Redis's sorted set.

[0068] Furthermore, the second-level cache storage unit also includes:

[0069] The timestamp storage unit is used to store timestamps in the zset's score, which can be sorted, and to store values. The zset's value is a unique element.

[0070] Furthermore, the timestamp storage unit also includes:

[0071] A uniqueness constraint determination unit is used to add a timestamp as a uniqueness constraint based on the value.

[0072] Furthermore, the second-level cache definition module 60 also includes:

[0073] The time interval defined by the time dimension of the second-level cache is larger than the time interval defined by the first-level cache.

[0074] Furthermore, the second-level cache definition module 60 also includes:

[0075] Each level of cache has a larger time interval than the previous level, and the size of the time interval defined by the first level cache defines the minimum update frequency of the data.

[0076] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, the above description focuses on specific embodiments of this specification. Additionally, the processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired results. In some implementations, multitasking and parallel processing are possible or may be advantageous.

[0077] The above description is only a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

[0078] This specification and accompanying drawings are merely illustrative examples of this application and are intended to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Clearly, those skilled in the art can make various alterations and modifications to this application without departing from its scope. Therefore, if such modifications and variations fall within the scope of this application and its equivalents, this application intends to include such modifications and variations.

Claims

1. A multi-level cache aggregation calculation method, characterized in that, The method includes: Connect to the message bus and obtain the first-level cache corresponding to the top of the stack. The subscription timing data corresponding to the message bus is arranged in order. Each piece of data in the first-level cache includes a time dimension, a device dimension, and a numerical value; When subscribing to data with the same time and device dimensions, perform cumulative aggregation calculations with the existing data. Define a scheduled task, wherein the scheduled task is in the same time interval as the time dimension defined by the first-level cache, and determine whether the first event task belongs to the time interval defined by the time dimension of the first-level cache; When the timed task corresponding to the first event task is triggered, the delay waiting time is set according to the watermark setting. At the same time, the data of the first-level cache is transferred to the external cache middleware to define the second-level cache. If the second-level cache contains the initial cache data, the data with the same time dimension and device dimension will be updated and merged. The time interval defined by the time dimension of the second-level cache is larger than the time interval defined by the first-level cache. Each level of cache has a larger time interval than the previous level, and the size of the time interval defined by the first level cache defines the minimum update frequency of the data.

2. The multi-level cache aggregation calculation method as described in claim 1, characterized in that, The cumulative aggregation calculations correspond to the following metrics: AVG value, SUM value, COUNT value, MAX value, MIN value, FIRST value, LAST value, INTEGRAL_S value, INTEGRAL_H value, and DIFF value.

3. The multi-level cache aggregation calculation method as described in claim 2, characterized in that, The method includes: The first-level cache is updated by accumulating and aggregating the corresponding metrics.

4. The multi-level cache aggregation calculation method as described in claim 3, characterized in that, The method includes: The time interval defined in the first-level cache is the time window; The second-level cache uses Redis's sorted set for storage.

5. The multi-level cache aggregation calculation method as described in claim 4, characterized in that, The zset's score can be sorted to store timestamps, and its value is used to store values. The zset's value is a unique element.

6. The multi-level cache aggregation calculation method as described in claim 5, characterized in that, Add a timestamp as a uniqueness constraint based on the value.

7. A multi-level cache aggregation computing device, characterized in that, The apparatus is used to implement the multi-level cache aggregation calculation method according to any one of claims 1-6, the apparatus comprising: The first-level cache acquisition module is used to connect to the message bus and acquire the first-level cache corresponding to the top of the stack. The subscription timing data corresponding to the message bus is arranged in order. The first-level cache content module, wherein each piece of data in the first-level cache includes a time dimension, a device dimension, and a value; The cumulative aggregation calculation module is used to perform cumulative aggregation calculation with the original data when subscribing to data that has the same time dimension and device dimension. A scheduled task definition module is used to define scheduled tasks. The scheduled tasks are in the same time interval as the time dimension defined by the first-level cache. The module determines whether the first event task belongs to the time interval defined by the time dimension of the first-level cache. A delay waiting time setting module is used to set a delay waiting time according to the set watermark when the timed task corresponding to the first event task is triggered. The second-level cache definition module is used to simultaneously transmit the data of the first-level cache to the external cache middleware and define the second-level cache. If there is initial cache data in the second-level cache, the data with the same time dimension and device dimension will be updated and merged.

Images