Multi-level eigenvalue layered rendering method for massive time series data of Internet of Things

Abstract

The invention discloses a multilevel eigenvalue layered rendering method for massive time series data of the Internet of Things. The method comprises the following steps: a back end predefines multi-stage time granularity, regularly calculates the maximum value and the minimum value of original data in each time window, and persistently stores the maximum value and the minimum value as a multi-layer characteristic value data set; the front end intelligently matches the optimal hierarchy according to the current view time span and requests the feature value of the hierarchy; and the front end draws a vertical line segment or an envelope line for rendering by using the maximum value and the minimum value of each window. And during user interaction, data hierarchies are dynamically switched to realize smooth drilling. Through the core technology of extreme value pre-calculation, on-demand matching and envelope line rendering, the fluctuation range and key event characteristics of data are completely reserved while front-end single-time rendering data points are strictly controlled within thousand levels, high-performance and high-fidelity Web visualization of ten-billion-level time series data is achieved, and the real-time performance of the system is improved. The technical problem that performance, precision and interactive experience cannot be achieved at the same time in a traditional scheme is solved.

Description

Technical Field

[0001] This invention relates to the fields of industrial Internet of Things (IoT) and web front-end visualization technology, specifically a multi-level feature value layered rendering method for massive IoT time-series data. Background Technology

[0002] In industrial IoT systems, a single device can generate tens to hundreds of time-series data points per second, and the operation of millions of indicators across the entire plant for 24 hours will generate tens of billions of data points. Traditional web visualization solutions typically employ the following strategies:

[0003] Full data retrieval + front-end downsampling:

[0004] All raw data is transmitted to the browser, where JavaScript performs sampling or aggregation.

[0005] Drawbacks: Long network transmission time, high memory consumption (prone to crashes), and slow initial screen rendering.

[0006] Real-time backend aggregated query:

[0007] Each time the user zooms or drags, a GROUP BY time() aggregate query is sent to the database.

[0008] Drawbacks: Under high concurrency, the database pressure increases dramatically, response latency is high, and smooth interaction cannot be achieved.

[0009] Fixed particle size prepolymerization:

[0010] Pre-calculate only a single granularity (e.g., 1-minute average);

[0011] Defects: It cannot take into account both macro trends and micro details, and "sawtooth distortion" or information loss may occur during drilling.

[0012] Furthermore, existing solutions generally overlook the crucial role of "highs and lows" in trend reconstruction—using only the average value smooths out peaks / troughs, making alert events invisible. Directly transmitting the original extreme values, however, fails to address the data volume issue.

[0013] Therefore, there is an urgent need for a hierarchical feature value supply mechanism that takes into account accuracy, performance, and interactive experience. Summary of the Invention

[0014] The purpose of this invention is to provide a multi-level feature value layered rendering method for massive IoT time-series data in order to overcome the shortcomings of the prior art.

[0015] To achieve the above objectives, the present invention adopts the following technical solution: a multi-level feature value hierarchical rendering method for massive IoT time-series data, comprising the following steps:

[0016] S1 backend pre-computation process: Predefine N standard time granularity levels, where N≥2, and each level corresponds to a fixed time interval length; for each level, periodically divide the raw indicator data in the IoT time series database into sliding or fixed windows according to the corresponding time interval length, and calculate at least one key feature value within each window, wherein the key feature value includes at least the maximum and minimum values ​​of the raw data within the window; persist the calculation results as the feature value dataset of the corresponding level;

[0017] S2 Front-end Matching and Request Process: The front-end visualization module automatically calculates the total time span ΔT based on the user's currently set time range to be visualized, and selects an optimal target level from N standard time granularity levels according to preset matching rules; the matching rules make the time interval length of the selected target level fit the total time span ΔT, so that the number of data points to be rendered is lower than a preset threshold.

[0018] S3 Data Supply and Rendering Process: The front end initiates a data request to the back end service, the request carrying the identifier of the target level and the time range to be visualized; the back end service retrieves all data records falling within the time range from the corresponding feature value dataset and returns them to the front end; based on the received data records, the front end performs graphical drawing for each time interval using its corresponding maximum and minimum values ​​to restore the data trend within the time range to be visualized.

[0019] As a further description of the above technical solution:

[0020] The front-end performs graphical drawing based on the received data records as follows: For each time interval, a vertical line segment is drawn in the chart coordinate system with the time identifier of the interval as the horizontal axis, starting from the minimum value and ending at the maximum value; or, a line is drawn by connecting all the minimum value points in the interval first and then all the maximum value points to form an envelope.

[0021] As a further description of the above technical solution:

[0022] The method also includes an interactive drill-down process, which includes the following steps: responding to the user's drill-down interactive operation on the visualization interface, the drill-down interactive operation including a selection operation or a zoom operation; and re-executing step S2 front-end matching and request process and step S3 data supply and rendering process according to the new time range determined by the drill-down interactive operation, wherein the total time span corresponding to the new time range will be matched to a level with a finer time interval length.

[0023] As a further description of the above technical solution:

[0024] When the drill-down interaction operation is a selection operation, if the time span of the selected area is less than the time interval length of the target level used for rendering, drill-down will be automatically triggered, and the next finer-grained level will be used as the new target level.

[0025] As a further description of the above technical solution:

[0026] The N standard time granularity levels are in an exponential or multiple relationship, covering at least interval lengths in units of seconds, minutes, hours, and days, wherein the time interval length of the finest granular level matches the acquisition frequency of the original data.

[0027] As a further description of the above technical solution:

[0028] In the backend pre-calculation process of step S1, the key feature values ​​to be calculated also include at least one of the following: average value, first value, last value, and number of samples.

[0029] As a further description of the above technical solution:

[0030] The preset threshold is 1000 data points; the matching rule is: among all levels whose time interval length is not greater than the quotient obtained by dividing the total time span ΔT by the preset threshold, the level with the largest time interval length is selected as the target level.

[0031] As a further description of the above technical solution:

[0032] The persistent storage creates an independent database table for each level. The database table contains at least the fields of indicator ID, window start time, maximum value and minimum value, and uses indicator ID and window start time as a composite primary key.

[0033] In summary, due to the adoption of the above technical solution, the present invention has the following beneficial effects compared with the prior art:

[0034] This invention defines N standard time granularity levels and pre-calculates the maximum and minimum values ​​within their corresponding time windows for each level, storing them persistently to achieve multi-level feature value pre-calculation. The front-end automatically matches an optimal level based on the total span of the user-defined visualization time range, achieving intelligent level matching. The front-end requests feature data for the matched level, and the back-end quickly retrieves and returns it from the corresponding storage, achieving dynamic data request and supply. The front-end uses the maximum and minimum values ​​of each time window to draw vertical line segments or connect them into envelopes to restore data trends, achieving extreme value envelope rendering. Responding to user zooming, selection, and other operations, the system dynamically switches to finer or coarser granularity levels, re-executing the above process to achieve smooth data exploration and interactive drill-down. This approach, through a 7-level standard time granularity plus a pre-aggregation model of high and low point feature values, balances the macro-level of trends with the micro-level of events; it completely solves the browser's big data rendering bottleneck, compressing hundreds of millions of raw data points into thousands of feature points; backend pre-computation plus frontend on-demand requests significantly reduces the pressure on real-time database queries; it naturally supports smooth drill-down and bounding box selection interactions, providing a user experience close to native applications; and it forms a clear decoupled architecture of raw data layer → multi-layer feature value intermediate layer → frontend matching and rendering layer, with strong system scalability and easy integration with existing high-availability, high-throughput IoT platforms. Attached Figure Description

[0035] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0036] Figure 1 This is a flowchart illustrating the workflow of a multi-level feature value layered rendering method for massive IoT time-series data. Detailed Implementation

[0037] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0038] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0039] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0040] In the description of the embodiments of the present invention, it should be noted that the terms "upper" and "inner" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship in which the product of the invention is usually placed when in use. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limiting the present invention.

[0041] In the description of this invention, it should also be noted that, unless otherwise explicitly specified and limited, the terms "set," "install," "connect," and "link" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a direct connection, an indirect connection through an intermediate medium, or a connection within two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0042] Example:

[0043] Please see Figure 1 This invention provides a technical solution: a multi-level feature value layered rendering method for massive IoT time-series data. This method pre-calculates key feature values ​​(such as maximum and minimum values) at multiple time granularities in the backend and dynamically returns data at the corresponding level based on user interaction behavior. This ensures the accuracy of trend restoration while keeping the amount of data rendered at the frontend within the thousands. It is suitable for high-concurrency, long-time-span industrial data visualization scenarios such as SCADA, energy monitoring, and smart manufacturing. Specifically, it includes the following steps:

[0044] S1 backend pre-computation process: Predefine N standard time granularity levels, where N≥2, and each level corresponds to a fixed time interval length; for each level, periodically divide the raw indicator data in the IoT time series database into sliding or fixed windows according to the corresponding time interval length, and calculate at least one key feature value within each window, wherein the key feature value includes at least the maximum and minimum values ​​of the raw data within the window; persist the calculation results as the feature value dataset of the corresponding level;

[0045] The N standard time granularity levels are in an exponential or multiple relationship, covering at least interval lengths in units of seconds, minutes, hours, and days, wherein the time interval length of the finest granular level matches the acquisition frequency of the original data.

[0046] Specifically, this system defines 7 standard time levels.

[0047] hierarchy Time interval length Typical Adapted View Span Main uses L0 24 hours ≥24 hours (e.g., 7 days) Year / Month Overview L1 12 hours 12h–24h Daily Trends L2 1 hour 1h–12h Schedule Analysis L3 30 minutes 30min–1h Process section monitoring L4 15 minutes 15min–30min Equipment start-up and shutdown analysis L5 5 minutes 5min–15min Alarm Association L6 1 second ≤5min Original waveform

[0048] In the backend pre-calculation process of step S1, the key feature values ​​to be calculated also include at least one of the following: average value, first value, last value, and number of samples.

[0049] The persistent storage creates an independent database table for each level. Each database table contains at least three fields: indicator ID, window start time, maximum value, and minimum value, with the indicator ID and window start time serving as a composite primary key. Because many indicators in industrial scenarios are Boolean or integer types, the maximum and minimum value fields accurately reflect state transitions.

[0050] The specific workflow is as follows: The backend uses Java Spring Scheduler or a distributed task scheduling framework (such as XXL-Job) to scan the original time-series data table periodically (e.g., hourly). For each level (e.g., L2), tasks are grouped by metric_id (metric ID) and window_start (window start time, aligned to 1 hour), and the maximum value max(val) and minimum value min(val) within each group are calculated. Optional fields such as sample count and average value avg are also calculated. The calculation results are stored in a separate feature value table.

[0051] S2 Front-end Matching and Request Process: The front-end visualization module automatically calculates the total time span ΔT based on the user's currently set time range to be visualized, and selects an optimal target level from N standard time granularity levels according to preset matching rules; the matching rules make the time interval length of the selected target level fit the total time span ΔT, so that the number of data points to be rendered is lower than a preset threshold.

[0052] The preset threshold is 1000 data points; the matching rule is: among all levels whose time interval length is not greater than the quotient obtained by dividing the total time span ΔT by the preset threshold, the level with the largest time interval length is selected as the target level.

[0053] The specific workflow is as follows: The user selects a device metric on the front-end interface and sets an initial time range (e.g., "last 8 hours"). The front-end logic calculates the time span ΔT = 8 hours.

[0054] According to the matching rules (e.g., selecting the longest level with a interval length ≤ ΔT / 1000), we calculate: 8h / 1000 = 28.8 seconds. Looking at the definition table, is the longest interval with a length less than 28.8 seconds L5 (5 minutes = 300 seconds)? No, we should select the level that covers the entire span and has the fewest points. A more reasonable rule is: select the smallest level that satisfies interval length * 1000 ≥ ΔT. For 8h (28800 seconds), L2 (1h = 3600 seconds) satisfies 3600 * 1000 >> 28800, and has only 8 points, therefore it matches the L2 level.

[0055] S3 Data Supply and Rendering Process: The front end initiates a data request to the back end service, the request carrying the identifier of the target level and the time range to be visualized; the back end service retrieves all data records falling within the time range from the corresponding feature value dataset and returns them to the front end; based on the received data records, the front end performs graphical drawing for each time interval using its corresponding maximum and minimum values ​​to restore the data trend within the time range to be visualized.

[0056] Preferably, the front-end performs graphic drawing based on the received data records as follows: for each time interval, draw a vertical line segment in the chart coordinate system with the time identifier of the interval as the horizontal axis, starting from the minimum value and ending at the maximum value; or, connect all the minimum value points in the interval first, and then draw all the maximum value points to form an envelope line.

[0057] The specific workflow is as follows: The frontend sends a request to the backend API. After receiving the request, the backend service directly queries the metric_feature_tier_L2 table and quickly returns records with metric_id=1001 and window_start between the start and end times (e.g., 8 records).

[0058] After receiving the data, the front end plots a vertical line segment on the chart, with `window_start` as the x-axis, from `min_val` to `max_val`, for each record (representing a 1-hour interval). These 8 line segments are arranged sequentially, clearly outlining the fluctuation range of this indicator within each hour over the past 8 hours. Compared to a smoothed curve plotted using hourly averages, this method preserves abnormally low troughs in the early morning and peaks in the morning, which could be important alert events.

[0059] The method also includes an interactive drill-down process, which includes the following steps: responding to the user's drill-down interactive operation on the visualization interface, the drill-down interactive operation including a selection operation or a zoom operation; and re-executing step S2 front-end matching and request process and step S3 data supply and rendering process according to the new time range determined by the drill-down interactive operation, wherein the total time span corresponding to the new time range will be matched to a level with a finer time interval length.

[0060] When the drill-down interaction operation is a selection operation, if the time span of the selected area is less than the time interval length of the target level used for rendering, drill-down will be automatically triggered, and the next finer-grained level will be used as the new target level.

[0061] The specific workflow is as follows: A user notices abnormal fluctuations in the early morning and selects the hourly range from 2:00 AM to 3:00 AM using the mouse. The front-end logic checks if the selected time span (1 hour) is less than the current level L2 interval length (1 hour). The usual rule is to trigger drill-down when the selected span is less than 1 / 2 or 1 / 3 of the current level interval length. Assuming the rule is drill-down if less than 1 / 2, and 1 hour is not less than 0.5 hours, no drill-down is triggered. However, the user can force drill-down by double-clicking the area.

[0062] After the user double-clicks, the front-end updates the view's time range to 02:00-03:00, and ΔT becomes 1 hour. The matching logic is re-executed: within the 1-hour span, both L3 (30 minutes) and L4 (15 minutes) meet the point requirement. Based on the rule of "selecting the finest granularity when the point count is less than the threshold," L4 (15 minutes) might be matched. The front-end then requests feature data at the L4 level (resulting in four extreme value pairs for each 15-minute interval) and re-renders a finer envelope, allowing the user to further observe the detailed fluctuations within this abnormal hour. Drilling down to L6 can approximate the waveform of the original second-level data.

[0063] Through the above steps, this invention achieves high-performance, high-fidelity, and highly interactive visualization of massive amounts of data, effectively solving the core pain points in the field of industrial IoT.

[0064] This method allows for control over the amount of rendering data, ensuring that the number of rendering points per iteration by the front end always meets the following requirements, regardless of the size of the original data:

[0065] Maximum number of intervals ≈ Total time span / Length of interval at the current level;

[0066] For example: viewing 24-hour data using L2 (1-hour interval) → only 24 intervals → a maximum of 48 points (min+max); even viewing 7-day data (L0, 24-hour interval) → only 7 intervals → 14 points. This ensures that the front-end always processes <1000 data points, guaranteeing smooth rendering at 60 FPS.

[0067] This method pre-calculates and persists the maximum and minimum values ​​(i.e., "feature values") within each interval of seven standard time granularities (24h, 12h, 1h, 30min, 15min, 5min, 1s) through a scheduled task in the backend; the frontend automatically matches the optimal level based on the current view's time range and only requests the feature value sequence of that level to draw a line chart; when the user selects or drills down, it dynamically switches to a finer granular level to achieve "on-demand supply and thousand-point rendering".

[0068] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A multi-level feature value hierarchical rendering method for massive IoT time-series data, characterized in that, Includes the following steps: S1 backend pre-computation process: Predefine N standard time granularity levels, where N≥2, and each level corresponds to a fixed time interval length; For each level, the raw indicator data in the IoT time series database is periodically divided into sliding or fixed windows according to the length of the corresponding time interval, and at least one key feature value is calculated in each window. The key feature value includes at least the maximum and minimum values ​​of the raw data in the window. The calculation results are persistently stored as a feature value dataset of the corresponding level; S2 Front-end Matching and Request Process: The front-end visualization module automatically calculates the total time span ΔT based on the user's currently set time range to be visualized, and selects the optimal target level from the N standard time granularity levels according to the preset matching rules. The matching rule makes the length of the time interval of the selected target level fit the total time span ΔT, so that the number of data points to be rendered is lower than a preset threshold. S3 Data Supply and Rendering Process: The front end initiates a data request to the back end service, and the request carries the identifier of the target level and the time range to be visualized; The backend service retrieves all data records falling within the time range from the corresponding feature value dataset and returns them to the frontend. Based on the received data records, the frontend uses the corresponding maximum and minimum values ​​to draw a graph for each time interval to restore the data trend within the time range to be visualized.

2. The multi-level feature value layered rendering method for massive IoT time-series data according to claim 1, characterized in that, The front-end performs graphical drawing based on the received data records as follows: For each time interval, a vertical line segment is drawn in the chart coordinate system with the time identifier of the interval as the horizontal axis, starting from the minimum value and ending at the maximum value; or, a line is drawn by connecting all the minimum value points in the interval first and then all the maximum value points to form an envelope.

3. A multi-level feature value layered rendering method for massive IoT time-series data according to claim 1 or 2, characterized in that, It also includes an interactive drill-down process, which includes the following steps: responding to the user's drill-down interactive operation on the visual interface, the drill-down interactive operation including a selection operation or a zoom operation; according to the new time range determined by the drill-down interactive operation, re-execute step S2 front-end matching and request process and step S3 data supply and rendering process, wherein the total time span corresponding to the new time range will be matched to a level with a finer time interval length.

4. The multi-level feature value layered rendering method for massive IoT time-series data according to claim 3, characterized in that, When the drill-down interaction operation is a selection operation, if the time span of the selected area is less than the time interval length of the target level used for rendering, drill-down will be automatically triggered, and the next finer-grained level will be used as the new target level.

5. The multi-level feature value layered rendering method for massive IoT time-series data according to claim 1, characterized in that, The N standard time granularity levels are in an exponential or multiple relationship, covering at least interval lengths in units of seconds, minutes, hours, and days, wherein the time interval length of the finest granular level matches the acquisition frequency of the original data.

6. The multi-level feature value layered rendering method for massive IoT time-series data according to claim 1, characterized in that, In the backend pre-calculation process of step S1, the key feature values ​​to be calculated also include at least one of the following: average value, first value, last value, and number of samples.

7. The multi-level feature value layered rendering method for massive IoT time-series data according to claim 1, characterized in that, The preset threshold is 1000 data points; the matching rule is: among all levels whose time interval length is not greater than the quotient obtained by dividing the total time span ΔT by the preset threshold, the level with the largest time interval length is selected as the target level.

8. The multi-level feature value layered rendering method for massive IoT time-series data according to claim 1, characterized in that, The persistent storage creates an independent database table for each level. The database table contains at least the fields of indicator ID, window start time, maximum value and minimum value, and uses indicator ID and window start time as a composite primary key.

Images