A processing method and device based on Internet of Things time series data, equipment and medium

Abstract

A kind of processing method, device, equipment and medium based on thing networking time series data are disclosed, applied to thing networking device, method includes: the message object of current period is sequentially structured and visualized processing, obtains first information;According to first information, from the visual information corresponding to visual window in last period in second information is determined and deleted, obtains residual information, wherein, the time span of visual window is greater than the time span of first information, the time span of first information is equal to the time span of second information, and second information is the earliest piece of information in time sequence in the visual information corresponding to visual window in last period;According to residual information and first information, generate the visual information corresponding to visual window in current period.The method can realize real-time visual processing and visual display to new data.

Description

Technical Field

[0001] This invention relates to the field of data processing, and in particular to a method, apparatus, device, and medium for processing time-series data based on the Internet of Things. Background Technology

[0002] In current IoT and digital twin systems, massive numbers of heterogeneous devices continuously generate high-frequency real-time time-series data through message queue protocols. This data is widely used in scenarios such as device monitoring, condition assessment, trend analysis, and predictive maintenance.

[0003] In order to enable operations and maintenance personnel and business decision-makers to make real-time judgments and responses to business based on the data obtained, front-end or back-end developers will visualize the obtained data in the form of code.

[0004] However, the above-mentioned data visualization processing can only visualize historical data that has already been obtained. This will prevent operations and maintenance personnel and business decision-makers from responding quickly to business changes or new data requirements. Summary of the Invention

[0005] This invention provides a method, apparatus, device, and medium for processing IoT time-series data, which solves the problem that current methods can only visualize historical data, making it difficult for staff to quickly respond to business changes or new data requirements. It enables real-time visualization processing and display of newly acquired data.

[0006] According to one aspect of the present invention, a method for processing Internet of Things (IoT) time-series data is provided, applied to IoT devices, the method comprising:

[0007] The message objects of the current period are processed sequentially through structured processing and visualization processing to obtain the first information.

[0008] Based on the first information, the second information is determined and deleted from the visualization information corresponding to the previous cycle in the visualization window to obtain the remaining information. The time span of the visualization window is greater than the time span of the first information, the time span of the first information is equal to the time span of the second information, and the second information is the earliest segment of information in the visualization information corresponding to the previous cycle in the visualization window.

[0009] Based on the remaining information and the first information, generate the visualization information corresponding to the current period for the visualization window.

[0010] The IoT time-series data processing method provided in this invention performs structured and visualized processing on message objects in the current period to obtain first information; based on the first information, second information is determined and deleted from the visualized information corresponding to the previous period in the visualization window to obtain remaining information; based on the remaining information and the first information, visualized information corresponding to the current period in the visualization window is generated. In this technical solution, on the one hand, performing structured and visualized processing on message objects in the current period enables rapid visualization of real-time data, facilitating the direct display of real-time visualized information to maintenance personnel and decision-makers. On the other hand, based on the time span of the first information, the second information is determined and deleted from the visualized information corresponding to the previous period in the visualization window, and the remaining information displayed in the visualization window of the previous period is merged with the first information of the current period corresponding to the newly obtained message object in the current period to generate visualized information corresponding to the current period in the visualization window. This solves the problem that currently only historical data can be visualized, leading to staff being unable to quickly respond to business changes or new data requirements, and achieves real-time visualization processing and display of newly obtained data. Furthermore, the visualization information generated in the visualization window includes not only the first information corresponding to the newly added message object in the current period, but also some information from the visualization window in the previous period, namely the remaining information. This realizes incremental refresh of the data visualization and ensures the continuity of data visualization information when displaying new and old data.

[0011] According to another aspect of the present invention, a processing apparatus based on Internet of Things (IoT) time-series data is provided, applied to IoT devices, the apparatus comprising:

[0012] The preprocessing module is used to perform structured and visual processing on the message objects of the current period in sequence to obtain the first information.

[0013] The processing module is used to determine and delete the second information from the visualization information corresponding to the previous cycle in the visualization window based on the first information, and obtain the remaining information. The time span of the visualization window is greater than the time span of the first information, the time span of the first information is equal to the time span of the second information, and the second information is the earliest segment of information in the visualization information corresponding to the previous cycle in the visualization window.

[0014] The generation module is used to generate visualization information for the current period based on the remaining information and the first information.

[0015] According to another aspect of the present invention, an Internet of Things (IoT) device is provided, the IoT device comprising:

[0016] At least one processor; and

[0017] A memory that is communicatively connected to at least one processor; wherein,

[0018] The memory stores a computer program that can be executed by at least one processor, such that the at least one processor is able to perform the IoT-based time-series data processing method according to any embodiment of the present invention.

[0019] According to another aspect of the present invention, a computer-readable storage medium is provided, which stores computer instructions for causing a processor to execute and implement the processing method based on Internet of Things time-series data according to any embodiment of the present invention.

[0020] According to another aspect of the present invention, a computer program product is provided, comprising a computer program that, when executed by a processor, implements a processing method based on Internet of Things time-series data according to any embodiment of the present invention.

[0021] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 A flowchart illustrating a method for processing IoT time-series data according to an embodiment of the present invention;

[0024] Figure 2 A flowchart illustrating another method for processing IoT time-series data provided in an embodiment of the present invention;

[0025] Figure 3 This is a schematic diagram of the structure of a processing device based on Internet of Things time-series data provided in an embodiment of the present invention;

[0026] Figure 4 This is a schematic diagram of the structure of an Internet of Things (IoT) device provided in an embodiment of the present invention. Detailed Implementation

[0027] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0028] It should be noted that the terms "current," "previous," etc., used in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0029] Figure 1 This is a flowchart illustrating a method for processing IoT time-series data according to an embodiment of the present invention. This embodiment is applicable to situations where real-time data is visualized. The method can be executed by an IoT time-series data processing device, which can be implemented in hardware and / or software and can be configured in an IoT device. In this embodiment, the IoT device is equivalent to an IoT system. The IoT device is presented in the form of an electronic device. An electronic device can refer to a computer or terminal device. Figure 1 As shown, the method includes:

[0030] S101. Perform structured processing and visualization processing on the message object of the current period in sequence to obtain the first information.

[0031] Here, a message object refers to data received by an IoT device from a target topic or queue, according to message protocol rules. In this embodiment, a cycle is determined based on a pre-set data refresh frequency. Structured processing refers to the method of converting data into structured objects. Visualization processing refers to the process of visualizing the data.

[0032] Specifically, IoT devices listen to pre-defined target topics or queues via connectors and receive data in real time according to message protocol rules. Data enters the access channel in the form of message objects. The period can be determined based on a pre-set data refresh frequency. For example, if the refresh frequency is 10 seconds, the message objects for the current period are: all message objects received between the last refresh and the next refresh, based on the current time. After obtaining the message objects for the current period, they are first structured to obtain structured objects. Then, the structured objects are visualized to obtain the first information. This first information is the visualized data after visualizing the message objects for the current period.

[0033] In this embodiment, message objects are divided into message objects of different periods according to a preset refresh frequency, providing a foundation for refreshing the visualized information according to the refresh frequency in the future. Furthermore, by performing structured processing and visualization processing on the message objects of the current period, it is possible to quickly visualize real-time data, making it convenient to directly display real-time visualized information to operation and maintenance personnel and decision-making managers.

[0034] S102. Based on the first information, determine and delete the second information from the visualization information corresponding to the previous cycle in the visualization window to obtain the remaining information.

[0035] Among them, the time span of the visualization window is greater than the time span of the first information, the time span of the first information is equal to the time span of the second information, and the second information is the earliest segment of information in the visualization information corresponding to the previous cycle.

[0036] Specifically, the visualization window is the window that displays the visualized data. Upon receiving the first piece of information, it can be determined whether visualized data is already displayed in the visualization window. For example, if the "message object of the current period" is the first batch of message objects received by the IoT device, then it can be determined that no visualized data is displayed in the visualization window. That is, as long as the "message object of the current period" is not the first batch of message objects received by the IoT device, the visualization information corresponding to the previous period in the visualization window can be obtained. Based on the time span of the first piece of information, the earliest segment of information displayed in the visualization window can be determined and deleted, i.e., the second piece of information is deleted. After deleting the second piece of information, the remaining visualization information corresponding to the previous period displayed in the visualization window is the remaining information.

[0037] For example, assume the visualization window spans one hour. That is, the visualization information displayed in the window is data from one hour. We then obtain the first piece of information, whose time span, i.e., the period corresponding to the current cycle, is 20 minutes. First, we determine that the visualization information in the previous cycle spanned from 10:10 AM to 11:10 AM. Then, based on the 20-minute time span of the first piece of information, we can delete the visualization information from 10:10 AM to 10:30 AM in the previous cycle, i.e., the second piece of information. The remaining visualization information from 10:30 AM to 11:10 AM is the residual information.

[0038] In this embodiment, based on the time span of the first information, the second information is determined and deleted from the visualization information corresponding to the previous period in the visualization window. This enables the first information corresponding to the newly introduced current period to retain the position to be filled in the visualization window. While achieving consistency of the time span of the visualization window, it also enables the removal of old data based on the time span of the new data after obtaining the new data of the current period, thus providing a basis for adding new data visualization information to the visualization window.

[0039] S103. Based on the remaining information and the first information, generate the visualization information corresponding to the visualization window in the current cycle.

[0040] Specifically, after deleting the second information and obtaining the remaining information, the remaining information can be merged with the first information to generate the corresponding visual information in the current period's visualization window after obtaining the message object of the current period.

[0041] In this embodiment, the remaining information displayed in the visualization window of the previous period is merged with the first information of the current period corresponding to the newly obtained message object, generating the visualization information corresponding to the current period. This solves the problem that currently only historical data can be visualized, which prevents staff from quickly responding to business changes or new data requests. It enables real-time visualization processing and display of newly obtained data. Furthermore, the visualization information generated in the visualization window includes not only the first information corresponding to the newly added message object of the current period but also some information from the visualization window of the previous period, i.e., the remaining information. This achieves incremental refresh of data visualization and ensures the continuity of data visualization information when displaying new and old data.

[0042] The IoT time-series data processing method provided in this invention performs structured and visualized processing on message objects in the current period to obtain first information; based on the first information, second information is determined and deleted from the visualized information corresponding to the previous period in the visualization window to obtain remaining information; based on the remaining information and the first information, visualized information corresponding to the current period in the visualization window is generated. In this technical solution, on the one hand, performing structured and visualized processing on message objects in the current period enables rapid visualization of real-time data, facilitating the direct display of real-time visualized information to maintenance personnel and decision-makers. On the other hand, based on the time span of the first information, the second information is determined and deleted from the visualized information corresponding to the previous period in the visualization window, and the remaining information displayed in the visualization window of the previous period is merged with the first information of the current period corresponding to the newly obtained message object in the current period to generate visualized information corresponding to the current period in the visualization window. This solves the problem that currently only historical data can be visualized, leading to staff being unable to quickly respond to business changes or new data requirements, and achieves real-time visualization processing and display of newly obtained data. Furthermore, the visualization information generated in the visualization window includes not only the first information corresponding to the newly added message object in the current period, but also some information from the visualization window in the previous period, namely the remaining information. This realizes incremental refresh of the data visualization and ensures the continuity of data visualization information when displaying new and old data.

[0043] Figure 2 This is a flowchart illustrating another method for processing IoT time-series data according to an embodiment of the present invention. Based on the above embodiments and other examples, this embodiment provides a detailed explanation of "structuring and visualizing message objects" and "generating visualization information corresponding to the current period in the visualization window." Figure 2 As shown, the method includes:

[0044] S201. Perform structured processing on the message object of the current period to obtain the structured data of the current period.

[0045] Optionally, the following operations need to be performed before retrieving the message object:

[0046] IoT devices include multiple connectors, all based on a common connection adapter mechanism, implementing corresponding connectors for different message queue protocols. For example, a connector is instantiated initially and establishes a persistent connection to the corresponding message middleware. Before establishing a connection, information can be defined, such as specifying the message queue topic or channel to be listened to, and mapping fields in the original message body to standard fields within the device. It supports extracting timestamps from specified fields in the message, configuring time formats, and a fallback mechanism based on the current device generating timestamps when the message does not contain timestamps. After defining the above configuration, each connector on the device continuously listens to the target topic or queue according to the configuration and receives new data in real time according to the message protocol rules. At this time, the data enters the access channel in the form of "message objects." Optionally, multi-channel concurrent reception is supported, utilizing asynchronous coroutines / thread pools to improve throughput.

[0047] Specifically, after receiving the message object of the current period according to the preset period, the message object can be processed according to the pre-set structured processing method to obtain the structured data of the current period.

[0048] For example, the steps to obtain structured data for the current period include:

[0049] (i) Determine if the message object in the current period has a corresponding parsing function; if it does, execute (ii); if it does not, execute (iii).

[0050] Specifically, upon receiving message data for the current period, the device determines whether a corresponding parsing function exists for that message object. Parsing functions are only used when the manufacturer has customized the encoding or when the message structure is complex. Therefore, it's advisable to first determine whether a corresponding parsing function exists for the message object in the current period.

[0051] (ii) Use parsing functions to parse the message object of the current period to obtain the structured data of the current period.

[0052] Specifically, when it is determined that a parsing function exists, the parsing function can be used to parse the message object of the current period to obtain a structured object, that is, the structured data of the current period.

[0053] For example, the logic for structuring a message object using a parsing function is as follows: D t =P(M raw ), where M raw For the message object, P(·) is the parsing function, and D is the message object. t It is a standard structured time series data, containing timestamp t, value v, and tag.

[0054] (iii) Standardize the message objects of the current period using preset data fields to obtain the structured data of the current period.

[0055] Specifically, if there is no parsing function corresponding to the message data, the message object is standardized using pre-set data fields, that is, the message object is standardized into a unified "time-series data structure".

[0056] For example, the preset data fields generally include: timestamp: data timestamp; metric: indicator identifier (such as temperature, voltage); value: numerical value; tags: optional context tags such as device ID, floor, room number, etc.

[0057] Optionally, when standardizing data timestamps, the device also needs to perform integrity and timing checks on the timestamp field. For example, if the message does not have a timestamp field, the receiving time is added; if the time format is not standard, it is converted to Unix timestamp milliseconds; for out-of-order data, the device supports a lightweight reordering buffer based on timestamps (e.g., sorting is delayed by up to 5 seconds).

[0058] After structuring, the structured data for the current period can be represented as: D t ={t, v, tag}.

[0059] In this embodiment, the message object is processed in a structured manner to obtain structured time-series data. This ensures data consistency and provides a foundation for subsequent data visualization. For example, standardized time-series data is processed more efficiently, facilitates data analysis and mining, and can be displayed more intuitively and accurately during visualization.

[0060] S202. Perform data aggregation and visualization processing on the structured data of the current period to generate visualized first information.

[0061] Among them, data aggregation and visualization processing involves processing data using aggregation methods and then performing visualization operations on the data.

[0062] Specifically, data aggregation and visualization can be performed in the following ways:

[0063] (i) Process the structured data of the current period according to the preset aggregation method to obtain aggregated data.

[0064] The aggregation methods include at least one of the following: time-dimensional aggregation, sliding window aggregation, and trend line fitting.

[0065] Specifically:

[0066] (1) If the aggregation method includes time dimension aggregation, then the time bucket corresponding to the structured data of the current period is determined according to the timestamp of the structured data of the current period, the aggregation result in each time bucket is obtained, and the aggregation result in each time bucket is determined as the aggregated data.

[0067] Among them, time-dimensional aggregation refers to aggregating time-series data according to a specified time granularity (such as by minute, hour, etc.).

[0068] For example, suppose there is a set of time series data {(t1, v1), (t2, v2), ..., (t... n v n )}, where t i For timestamps, v i For time point t i The value on the time scale. The time granularity is Δt, for example, per minute or per hour. For each data point t i Rounding down gives the corresponding time bucket: t bucket (t i )=|t i / △t|*△t. Then, all data points in the same time bucket are aggregated. Common aggregation operations include summation, averaging, finding the maximum or minimum value, etc. Assuming we choose to calculate the average, the aggregation formula is: v agg (t bucket )=1 / N∑ i＝1 N *v i Where N is the number of data points in the same time bucket. Finally, the aggregation result within each time bucket is output: {[t] bucket v agg (t bucket )]}.

[0069] (2) If the aggregation method includes sliding window aggregation, the smoothed data value corresponding to the structured data of the current period is determined according to the structured data of the current period and the preset sliding window range, and the smoothed data value corresponding to the structured data of the current period is determined as the aggregated data.

[0070] Sliding window aggregation is used to smooth real-time data streams. Common operations include moving average, maximum value, minimum value, etc.

[0071] For example, suppose there is a set of time series data {(t1, v1), (t2, v2), ..., (t... n v n )}, where t i For timestamps, v i For time point t i The value is given above. The sliding window size is W, meaning that the window can store a maximum of W data points. For each data point v...i The window containing the data points is [i-W+1, i]. Aggregate all data points within the window.

[0072] For example, to calculate the sliding average within the window:

[0073] Alternatively, if you want to calculate the maximum or minimum value, you can replace the aggregate function with a maximum or minimum value function: or Ultimately, the output is the time point t. i Smoothed data value v smooth (i).

[0074] (3) If the aggregation method includes trend line fitting, then the fitting trend line is determined based on the structured data of the current period and the fitting method, and the fitting trend line is determined as the aggregated data.

[0075] Trendline fitting involves using methods such as linear regression to fit a trendline to the data based on past data points.

[0076] For example, suppose there is a set of time series data {(t1, v1), (t2, v2), ..., (t... n v n )}, where t i For timestamps, v i For time point t i The value on.

[0077] Calculate the average of the data:

[0078] Calculate the slope m and intercept b:

[0079] Calculate the fitted value of the trend line: v trend (t i )=m*t i +b.

[0080] At this point, the predicted value of the trend line can be output: {[t i v trend (t i )]}.

[0081] Optionally, extreme value analysis can also be performed to identify the maxima and minima in the data, ultimately determining the local extreme points.

[0082] Optionally, after processing the structured data of the current period according to a preset aggregation method to obtain aggregated data, branching steps can also be executed synchronously:

[0083] (1) Determine the target message middleware based on the data access configuration.

[0084] One connector establishes a connection with a message middleware. Data access configuration refers to the configuration performed by the user on the device before data access is determined. For example, setting the message queue protocol type, data topic, field mapping relationship, timestamp parsing rules, and selecting the default adapter, pre-setting the data access configuration, and enabling data stream subscription.

[0085] Specifically, if the target message middleware has not changed, that is, the user has not changed the "message protocol type", then execute (2) directly; if the target message middleware has changed, then the target message middleware needs to be adjusted according to the data access configuration.

[0086] (2) Obtain the message object for the next cycle based on the target connector connected to the target message middleware.

[0087] Specifically, after initially identifying the target message middleware, the device connects to it using a connector; this connected connector becomes the target connector. At this point, the device can directly obtain the message object for the next cycle based on the target connector.

[0088] Furthermore, after obtaining the message object for the next cycle, the message object for the next cycle can be directly used as the message object for the current cycle, and the execution of step S301 can be returned.

[0089] Please note that the "synchronous execution branch step" is executed synchronously. That is, it continues to perform the following operations on the original "structured data of the current period," while simultaneously receiving the message object of the next period and returning it as the message object of the current period to execute step S201. As described above, it is possible to continuously perform dynamic visualization processing on the message objects acquired "in real time" without any interval. Optionally, acquiring the "message object of the next period" can also be done after "processing the structured data of the current period using any aggregation method."

[0090] (ii) Visualize the aggregated data according to the preset visualization configuration to obtain the first information.

[0091] Specifically, the preset visualization configuration is for visualizing data. For example, each floor of a building is equipped with energy collection devices, which report real-time electricity consumption data from each meter at regular intervals via a certain protocol. In this case, the preset visualization configuration is as follows: select the access source configured in the previous step as the data source; set the display dimensions: floor number and meter ID; select kilowatt-hours as the display field; set the chart type to a line chart; set the chart style with blue as the main color and "kilowatt-hours (kWh)" as the unit, and enable dual Y-axis; refresh frequency: set to automatically refresh every 5 seconds; time window: select "past 1 hour" data as the display interval; data processing method: enable the "moving average" processing algorithm, and set the sliding window width to 5 minutes; other interactive settings: enable chart zoom, hover tooltip, export function, etc.

[0092] S203. Based on the first information, determine and delete the second information from the visualization information corresponding to the previous cycle in the visualization window to obtain the remaining information.

[0093] Specifically, upon receiving the first information, the visualization information corresponding to the previous cycle in the visualization window can be obtained. Based on the time span of the first information, the earliest segment of information displayed in the visualization window is determined and deleted, i.e., the second information is deleted. After deleting the second information, the remaining visualization information corresponding to the previous cycle displayed in the visualization window is the remaining information.

[0094] S204. Determine the information access point based on the start timestamp of the first information and the end timestamp of the remaining information.

[0095] Specifically, since each data point in the first information contains a timestamp, the timestamp that is furthest from the current time can be determined; this timestamp is the start timestamp. Furthermore, the end time of the remaining information, i.e., the end timestamp, can be determined. Based on these two timestamps, the information access point can be identified.

[0096] For example, continuing the above example, if the time span of the remaining information is from 10:30 AM to 11:10 AM, then the time recorded by the ending timestamp is 11:10 AM. Also, if the starting timestamp of the first information is 11:11 AM, then the information access point can be determined to be between 11:10 AM and 11:11 AM. The word "between" is used here because the example uses a relatively large time frame; in reality, the information access point may be only a second or even a smaller point in time.

[0097] S205. Based on the information access point, the first information and the remaining information are spliced ​​together to obtain the visualization information corresponding to the visualization window in the current period.

[0098] Specifically, after determining the information access point, the first piece of information can be appended to the remaining information at the location of the access point to achieve incremental data updates. Furthermore, this information is displayed in a visualization window, showing the visualization information corresponding to the current period.

[0099] In this embodiment, by concatenating the newly acquired first information with the remaining information corresponding to the previous cycle and displaying them in the visualization window, the user can see the continuously concatenated and visualized first information while viewing the remaining information displayed in the visualization window. This achieves dynamic visualization of the data based on the updated data. Furthermore, based on the above steps, since the message object for each cycle is automatically generated after the first information is obtained, dynamic data updates can be viewed without refreshing the visualization window. Also, because the visualization processing is performed on each obtained message object first, the visualization information corresponding to the incremental message object is directly and dynamically generated when the visualization window is incrementally refreshed.

[0100] Optionally, in this embodiment, the acquisition of all message objects, the structuring and visualization of data can all be displayed on the browser side of the IoT device. Data updates, i.e., the setting of a cycle, can be accurate to the second or at a higher frequency, depending on the refresh rate setting.

[0101] Optionally, during the above process, users can modify the time window span, moving average period, and fitting function of the visualization window (e.g., changing to linear fitting, multinomial fitting, etc.) as needed. The device will immediately recalculate the data, push the new calculation results, and display them in the visualization window. The visualization window will automatically refresh to display the visualization information under the new algorithm. Furthermore, users can switch chart types (e.g., change line charts to bar charts, area charts, pie charts, dashboards, etc.), modify colors, axis labels, units, etc., in the visualization window. Users can also modify the refresh rate (e.g., from 5 seconds to 1 second). The methods for modifying the visualization interface are not limited to dragging, dropdowns, templates, etc. Additionally, style parameter configuration items are provided, such as preset themes and custom color schemes in the color scheme; axis range, labels, and whether to display grid lines; coordinate unit and precision configuration; display legends, data lines, and annotation lines, etc. Moreover, the data refresh rate and time window span of the visualization window can be changed according to user needs.

[0102] In this embodiment, the various configurations of the visualization window described above can be directly selected or set by operations and maintenance personnel and business decision-makers based on the visualization window. After reconfiguration, the visualization information displayed in the visualization window will respond quickly and dynamically according to the configuration, realizing rapid response to business changes or new requirements. Furthermore, since the above configurations do not rely on front-end or back-end developers, they are suitable for users without a technical background. Additionally, because the visualization settings allow for various data aggregation processes and data extreme value determination, the visualized information provides a deeper level of data presentation.

[0103] Optionally, in this embodiment, during the process of receiving message objects based on multiple connectors in the IoT device, the IoT device supports multiple mainstream message queue protocols. Furthermore, the user can select a target message queue from the registered data source of the IoT device. Fuzzy matching of input is supported, and hierarchical topics can also be selected from a tree structure. The backend retrieves the latest data, i.e., the message object, from the corresponding message queue through configuration parameters.

[0104] Optionally, the visualization window in this embodiment can support linked operations between charts. For example, multiple charts can share the same timeline, making it easier to observe the linked changes across multiple dimensions. When a user hovers over a time point on Chart 1, Charts 2 and 3 will automatically highlight the data at the corresponding time point. When the time range of one chart is zoomed, other charts will automatically respond.

[0105] Figure 3 This is a schematic diagram of the structure of a processing device based on Internet of Things time-series data provided in an embodiment of the present invention. Figure 3 As shown, this device is applied to Internet of Things (IoT) devices and includes:

[0106] The preprocessing module 301 is used to perform structured processing and visualization processing on the message objects of the current period in sequence to obtain the first information.

[0107] The processing module 302 is used to determine and delete the second information from the visualization information corresponding to the previous cycle of the visualization window based on the first information, so as to obtain the remaining information. The time span of the visualization window is greater than the time span of the first information, the time span of the first information is equal to the time span of the second information, and the second information is the earliest segment of information in the visualization information corresponding to the previous cycle of the visualization window.

[0108] The generation module 303 is used to generate visualization information corresponding to the visualization window in the current period based on the remaining information and the first information.

[0109] Optionally, the preprocessing module 301 is specifically used for:

[0110] The message objects of the current period are processed in a structured manner to obtain the structured data of the current period; the structured data of the current period is then aggregated and visualized to generate the first visualized information.

[0111] Optionally, the message object of the current period is processed in a structured manner to obtain the structured data of the current period. The preprocessing module 301 is specifically used for:

[0112] Determine if a corresponding parsing function exists for the message object in the current period; if it exists, use the parsing function to parse the message object in the current period to obtain the structured data for the current period; if it does not exist, use preset data fields to standardize the message object in the current period to obtain the structured data for the current period.

[0113] Optionally, the structured data of the current period is aggregated and visualized to generate visualized first information. The preprocessing module 301 is specifically used for:

[0114] The structured data for the current period is processed according to the preset aggregation method to obtain aggregated data; the aggregated data is then visualized according to the preset visualization configuration to obtain the first information.

[0115] Optionally, the preset aggregation method includes at least one of time-dimensional aggregation, sliding window aggregation, and trend line fitting; the structured data of the current period is processed according to the preset aggregation method to obtain aggregated data, and the preprocessing module 301 is specifically used for:

[0116] If the aggregation method includes time-dimensional aggregation, then the time bucket corresponding to the structured data of the current period is determined based on the timestamp of the structured data of the current period, and the aggregation result within each time bucket is obtained, and the aggregation result within each time bucket is determined as the aggregated data; if the aggregation method includes sliding window aggregation, then the smoothed data value corresponding to the structured data of the current period is determined based on the structured data of the current period and the preset sliding window range, and the smoothed data value corresponding to the structured data of the current period is determined as the aggregated data; if the aggregation method includes trendline fitting, then the fitted trendline is determined based on the structured data of the current period and the fitting method, and the fitted trendline is determined as the aggregated data.

[0117] Optionally, the device further includes: a connection module; the IoT device includes multiple connectors; after performing structured processing and visualization processing on the message object of the current period in sequence to obtain the first information, the connection module is specifically used for:

[0118] The target message middleware is determined based on the data access configuration, wherein a connector establishes a connection relationship with a message middleware; based on the target connector connected to the target message middleware, the message object of the next cycle is obtained.

[0119] Optionally, the generation module 303 is specifically used for:

[0120] The information access point is determined based on the start timestamp of the first information and the end timestamp of the remaining information; the first information and the remaining information are then concatenated based on the information access point to obtain the visualization information corresponding to the visualization window in the current period.

[0121] The IoT-based time-series data processing device provided in this embodiment of the invention can execute the IoT-based time-series data processing method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the method execution.

[0122] Figure 4 This is a schematic diagram of the structure of an Internet of Things (IoT) device provided in an embodiment of the present invention. In this embodiment, the IoT device is presented in the form of an electronic device. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0123] like Figure 4 As shown, the IoT device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer programs stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the IoT device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0124] Multiple components in the IoT device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, optical disk, etc.; and a communication unit 19, such as a network interface card, modem, wireless transceiver, etc. The communication unit 19 allows the IoT device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0125] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as processing methods based on Internet of Things (IoT) time-series data.

[0126] In some embodiments, the IoT-based time-series data processing method can be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on the IoT device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the IoT-based time-series data processing method described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to perform the IoT-based time-series data processing method by any other suitable means (e.g., by means of firmware).

[0127] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0128] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0129] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0130] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0131] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0132] A computing system can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0133] This invention also provides a computer program product, including a computer program that, when executed by a processor, implements the IoT time-series data processing method provided in any embodiment of this invention.

[0134] In implementing the computer program product, computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0135] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0136] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for processing time-series data based on the Internet of Things, characterized in that, Applied to Internet of Things (IoT) devices, the method includes: The message objects of the current period are sequentially processed through structured processing and visualization processing to obtain the first information, including: processing the message objects of the current period into structured data; processing the structured data of the current period into aggregated data according to a preset aggregation method; and visualizing the aggregated data according to a preset visualization configuration to obtain the first information. The visualization chart types include line charts, bar charts, area charts, pie charts, and dashboards. Based on the first information, the second information is determined and deleted from the visualization information corresponding to the previous cycle in the visualization window, and the remaining information is obtained; the time span of the visualization window is greater than the time span of the first information, the time span of the first information is equal to the time span of the second information, and the second information is the earliest segment of information in the visualization information corresponding to the previous cycle in the visualization window; the setting of a cycle is accurate to the second level; Based on the remaining information and the first information, generate the visualization information corresponding to the current period for the visualization window; The preset aggregation methods include time-dimensional aggregation; the structured data of the current period is processed according to the preset aggregation method to obtain aggregated data, including: determining the time bucket corresponding to the structured data of the current period based on the timestamp of the structured data of the current period, obtaining the aggregation result in each time bucket, and determining the aggregation result in each time bucket as the aggregated data; time-dimensional aggregation refers to aggregating time-series data according to a specified time granularity; If a set of time series data is {(t1, v1), (t2, v2), ..., (t... n v n )},t i For timestamps, v i For in t i The value above determines the time bucket corresponding to the structured data in the current period, calculated using the following formula: t bucket (t i )=|t i / △t|×△t; where △t is the time granularity.

2. The method for processing IoT time-series data according to claim 1, characterized in that, The step of performing structured processing on the message object of the current period to obtain structured data for the current period includes: Determine whether the message object of the current period has a corresponding parsing function; If it exists, the message object of the current period is parsed using the parsing function to obtain the structured data of the current period; If it does not exist, the message object of the current period is standardized using preset data fields to obtain the structured data of the current period.

3. The method for processing IoT time-series data according to claim 1, characterized in that, The preset aggregation method also includes at least one of sliding window aggregation and trend line fitting; The process of processing the structured data of the current period according to a preset aggregation method to obtain aggregated data includes: If the aggregation method includes sliding window aggregation, then the smoothed data value corresponding to the structured data of the current period is determined according to the structured data of the current period and the preset sliding window range, and the smoothed data value corresponding to the structured data of the current period is determined as the aggregated data; If the aggregation method includes trendline fitting, then the fitted trendline is determined based on the structured data of the current period and the fitting method, and the fitted trendline is determined as the aggregated data.

4. The method for processing IoT time-series data according to claim 1, characterized in that, The IoT device includes multiple connectors; after performing structured and visual processing on the message objects of the current period to obtain the first information, it also includes: The target message middleware is determined based on the data access configuration, wherein one connector establishes a connection relationship with one message middleware; The message object for the next cycle is obtained based on the target connector connected to the target message middleware.

5. The method for processing IoT time-series data according to claim 1, characterized in that, The step of generating visualization information for the current period based on the remaining information and the first information includes: The information access point is determined based on the start timestamp of the first information and the end timestamp of the remaining information. By concatenating the first information and the remaining information based on the information access point, the visualization information corresponding to the visualization window in the current period is obtained.

6. A processing device for Internet of Things (IoT) time-series data, characterized in that, Applied to Internet of Things (IoT) devices, the device includes: The preprocessing module is used to perform structured processing and visualization processing on the message objects of the current period in sequence to obtain the first information, including: performing structured processing on the message objects of the current period to obtain structured data of the current period; processing the structured data of the current period according to a preset aggregation method to obtain aggregated data; and performing visualization processing on the aggregated data according to a preset visualization configuration to obtain the first information; the visualization chart types include line charts, bar charts, area charts, pie charts, and dashboards; The processing module is used to determine and delete the second information from the visualization information corresponding to the previous cycle in the visualization window based on the first information, and obtain the remaining information. The time span of the visualization window is greater than the time span of the first information, the time span of the first information is equal to the time span of the second information, and the second information is the earliest segment of information in the visualization information corresponding to the previous cycle in the visualization window; the setting of a cycle is accurate to the second level. The generation module is used to generate the visualization information corresponding to the visualization window in the current period based on the remaining information and the first information; The preset aggregation methods include time-dimensional aggregation, which processes the structured data of the current period according to the preset aggregation method to obtain aggregated data. This includes: determining the time bucket corresponding to the structured data of the current period based on the timestamp of the structured data of the current period, obtaining the aggregation result in each time bucket, and determining the aggregation result in each time bucket as the aggregated data; time-dimensional aggregation refers to aggregating time-series data according to a specified time granularity. If a set of time series data is {(t1, v1), (t2, v2), ..., (t... n v n )},t i For timestamps, v i For in t i The value above determines the time bucket corresponding to the structured data in the current period, calculated using the following formula: t bucket (t i )=|t i / △t|×△t; where △t is the time granularity.

7. An Internet of Things (IoT) device, characterized in that, include: One or more processors; Memory, used to store one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the processing method based on IoT time-series data as described in any one of claims 1 to 5.

8. A readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the processing method based on IoT time-series data as described in any one of claims 1 to 5.

Images