Kubeard architecture-based edge end Internet of Things data processing method and system

Abstract

The invention belongs to the technical field of Internet of Things, and particularly relates to an edge end Internet of Things data processing method and system based on a Kubeard architecture. According to the edge end Internet of Things data processing method based on the Kubeard architecture, a Mapper layer in the Kubeard architecture comprises a data preprocessing assembly, and the data preprocessing assembly executes the following steps that a data stream is obtained; allocating an event timestamp and a water level mark W (te) for the data stream; and judging whether the event timestamp behind the water level mark W (te) is smaller than or equal to the timestamp te of the water level mark W (te), and if the event timestamp behind the water level mark W (te) is smaller than or equal to the timestamp te, correcting the out-of-order event in the data stream. According to the method, the data preprocessing component is expanded in the Mapper layer in the Kuebound architecture, and after being processed by the data preprocessing component, the data is directly provided for the application deployed at the edge end, so that the real-time performance is greatly improved.

Description

Technical Field

[0001] This invention belongs to the field of Internet of Things (IoT) technology, specifically relating to an edge IoT data processing method and system. Background Technology

[0002] In the process of IoT data collection, most IoT platforms perform data aggregation and completion in the cloud, allowing IoT devices to... Figure 1 As shown, data can be uploaded directly via the protocol or as... Figure 2 The diagram shows data being uploaded to the cloud for processing after a relay at the edge gateway. In both methods, if device data requires aggregation calculations, such as calculating an average over a period of time, or if data uploaded by the device is lost due to network fluctuations and needs to be replenished, this is handled uniformly in the cloud service. However, due to network transmission latency, cloud computing cannot achieve real-time processing. In some industry scenarios with high real-time requirements, such as water quality testing in water plants, near real-time processing and real-time feedback of calculation results are needed, which cloud processing cannot meet.

[0003] Kubeedge is an open-source system that extends native containerized business processes and device management capabilities to edge nodes. Built on Kubernetes, it provides core infrastructure support for network communication, application deployment, and metadata synchronization between the cloud and edge, facilitating management between the cloud and the edge. However, in its architecture, the edge does not support sensor data aggregation and completion capabilities.

[0004] Data aggregation and completion are essential data processing operations in the Internet of Things (IoT). Since incomplete IoT data is common, latency, jitter, and packet loss are unavoidable, leading to timing discrepancies and data gaps. Without proper processing, downstream business services will experience numerous unpredictable errors. The current cloud-based processing method used by Kubeedge is unsuitable for scenarios with high real-time requirements, such as water treatment plants. Device data is uploaded directly to the cloud without processing for centralized cloud processing. However, because the application is deployed at the edge, [see...] Figure 3 and Figure 4 The difference between cloud and edge deployments means that after data is processed in the cloud, it still needs to flow back to the edge application, resulting in extremely poor timeliness. Summary of the Invention

[0005] To address the aforementioned problems, the present invention aims to provide an edge IoT data processing method based on the Kubeedge architecture;

[0006] Another objective of this invention is to provide an edge IoT data processing system based on the Kubeedge architecture.

[0007] Edge - side Internet of Things data processing method based on the Kubeedge architecture. The Mapper layer in the Kubeedge architecture includes a data pre - processing component, and the data pre - processing component performs the following steps:

[0008] Obtain the data stream;

[0009] Assign an event timestamp and a watermark identifier W(t

[0018] ) to the data stream;

[0010] Judge whether the event timestamp behind the watermark identifier W(t e ) is less than or equal to the timestamp t of the watermark identifier W(t e ), and if it is less than or equal to the timestamp t e , correct the out - of - order events in the data stream.

[0011] In the edge - side Internet of Things data processing method of the present invention, multiple watermark identifiers are set along the data stream, and the algorithm formula of the watermark identifier is: W(t e ) = max(t e ) - Δ;

[0012] Where, max(t e ) is the maximum event time observed currently;

[0013] Δ is the maximum allowable out - of - order delay;

[0014] t e is the event time, equal to t1, t2, t3, t4, t5, t6, and t1 < t2 < t3 < t4 < t5 < t6. <000,093>In the edge - side Internet of Things data processing method of the present invention, the data pre - processing component also performs the following steps of aggregation calculation:

[0016] Extract the event time of device marking from the data stream to obtain a data stream with event timestamps;

[0017] Assign an adaptive sliding window to the data stream with event timestamps to obtain an adaptively sampled data stream. The algorithm formula of the adaptive sliding window is: Windows(t) = {[iS, iS + W')|iS ≤ T < iS + W'}, where t is the current time, W' is the window size, S is the step size, i is a positive integer variable, iS is the start point of the adaptive sliding window, and iS + W' is the end point of the adaptive sliding window;

[0018] Perform aggregation calculation on the adaptively sampled data stream to obtain the aggregation calculation result of each window.​​

[0019] The edge IoT data processing method of the present invention includes a Mapper layer comprising a DataPanel component, and the DataPanel component comprising a Pushdata data upload unit.

[0020] The aggregation calculation result is pushed to the local MQTT server through the Pushdata data upload unit.

[0021] The aggregation calculation includes at least one of the following: average value calculation, maximum value calculation, and minimum value calculation.

[0022] The edge IoT data processing method of the present invention includes a data preprocessing component comprising a data completion step:

[0023] Missing data in the data stream is imputed based on the following exponential smoothing formula:

[0024] EMA t =α·Y t +(1-α)·EMA t-1

[0025] Where t is the current time;

[0026] Y t This refers to the actual observed value at time point t;

[0027] EMA t This is the exponential moving average calculated at time point t;

[0028] EMA t-1 This is the exponential moving average of the previous time point;

[0029] α is the smoothing coefficient, 0 < α ≤ 1.

[0030] The edge IoT data processing method of the present invention refers to the edge terminal as the edge gateway of a direct drinking water device, and the data stream as the water flow rate of the direct drinking water device.

[0031] An edge IoT data processing system based on the Kubeedge architecture is used to execute the aforementioned edge IoT data processing method based on the Kubeedge architecture. The Mapper layer in the Kubeedge architecture includes...

[0032] A data interaction component that interacts with IoT devices to obtain data streams from the IoT devices;

[0033] A data preprocessing component, connected to the data interaction component, assigns event timestamps and watermark identifiers W(t) to the data stream. e), determine whether the event timestamp behind the water level line identifier W(t e ) is less than or equal to the timestamp t of the water level line identifier W(t e ). e , if it is less than or equal to the timestamp t e , correct the out-of-order events in the data stream.

[0034] In the edge-side Internet of Things data processing system of the present invention, the data preprocessing component extracts the event time of device ticking from the data stream to obtain a data stream with event timestamps;

[0035] An adaptive sliding window is allocated to the data stream with event timestamps to obtain an adaptively sampled data stream. The algorithm formula of the adaptive sliding window is: Windows(t) = {[iS, iS + W')|iS ≤ T < iS + W'}, where t is the current time, W' is the window size, S is the step size, i is a positive integer variable, iS is the start point of the adaptive sliding window, and iS + W' is the end point of the adaptive sliding window;

[0036] Aggregate calculation is performed on the adaptively sampled data stream to obtain the aggregate calculation result of each window;

[0037] The aggregate calculation result is pushed to the local MQTT server through the Pushdata data upload unit in the DataPanel component of the Mapper layer.

[0038] In the edge-side Internet of Things data processing system of the present invention, the data preprocessing component also fills in the missing data in the data stream based on the following exponential smoothing calculation formula:

[0039] EMA t = α·Y t +(1 - α)·EMA t-1

[0040] where, t is the current time point;

[0041] Y t is the actual observed value at time point t;

[0042] EMA t is the exponentially weighted moving average calculated at time point t;

[0043] EMA t-1 is the exponentially weighted moving average of the previous time point;

[0044] α is the smoothing coefficient, 0 < α ≤ 1.

[0045] The edge-end IoT data processing system of the present invention refers to the edge gateway of a direct drinking water device, and the data stream refers to the water flow rate of the direct drinking water device.

[0046] Beneficial effects: This invention extends the data preprocessing component in the Mapper layer of the Kuebeedge architecture. After processing by the data preprocessing component, the data is directly provided to the application deployed at the edge, which greatly improves real-time performance. Attached Figure Description

[0047] Figure 1 This is a diagram illustrating how existing IoT devices directly upload data to the cloud via a protocol.

[0048] Figure 2 This is a schematic diagram illustrating how existing IoT devices upload data to the cloud after a single relay at an edge gateway.

[0049] Figure 3 This is a schematic diagram illustrating the deployment of existing technologies in edge gateways;

[0050] Figure 4 This is a diagram illustrating the deployment of existing technologies on cloud servers;

[0051] Figure 5 This is a schematic diagram of the Kubeedge architecture of the present invention;

[0052] Figure 6 This is a flowchart illustrating the edge IoT data processing method based on the Kubeedge architecture of the present invention.

[0053] Figure 7 This is a schematic diagram of the aggregation calculation process of the edge IoT data processing method based on the Kubeedge architecture of the present invention;

[0054] Figure 8 This is a schematic diagram of the present invention for allocating event timestamps and water level identifiers W(t) to the data stream;

[0055] Figure 9 This is a schematic diagram illustrating the present invention for determining the existence of out-of-order data streams;

[0056] Figure 10 This is a schematic diagram of the present invention for allocating aggregation windows to a data stream;

[0057] Figure 11 This is a schematic diagram illustrating the data completion effect of the present invention;

[0058] Figure 12 This is a schematic diagram illustrating the use of a one-dimensional Kalman filter to complete missing data;

[0059] Figure 13This is a schematic diagram illustrating the use of LightGBM / XGBoost regression to complete missing data.

[0060] Figure 14 This is a data completion line chart according to a specific embodiment of the present invention;

[0061] Figure 15 This is a schematic diagram of a water flow time window according to a specific embodiment of the present invention. Detailed Implementation

[0062] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0063] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other.

[0064] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, but this is not intended to limit the scope of the invention.

[0065] Reference Figure 5 , Figure 6 The edge IoT data processing method based on the Kubeedge architecture includes a data preprocessing component called DataPreProcess in the Mapper layer of the Kubeedge architecture. The DataPreProcess component performs the following steps:

[0066] Acquire data stream;

[0067] Assign event timestamps and watermark identifiers W(t) to the data stream. e );

[0068] Determine the water level indicator W(t) e Is the event timestamp following the waterline identifier W(t) less than or equal to the waterline identifier? e The timestamp t e If less than or equal to timestamp t e This corrects out-of-order events in the data stream.

[0069] This invention extends the DataPreProcess component to the Mapper layer in the Kuebeedge architecture, addressing the pain points of out-of-order data (data delays, packet loss, jitter) and poor real-time performance in edge scenarios. After processing by the data preprocessing component, the data is directly provided to applications deployed at the edge. This changes the original cloud computing model to edge-based processing, allowing applications to directly access the data after deployment at the edge, saving network transmission time (from device to cloud) and greatly improving real-time performance. The watermark mechanism ensures that subsequent processed data is not out of order.

[0070] Figure 5 The Mapper Framework includes the PKg component and the _template component. The Mapper layer includes the DMIv1beta1 API component, the Panel component, and the Driver component. The DMIv1beta1 API component includes the grpcClient component and the grpcServer component.

[0071] The _template component includes the devPanel unit and the driver unit.

[0072] The Panel component includes the devPanel component and the DataPanel component. The devPanel component includes:

[0073] The startDev unit is used to start a specified device and trigger the device to begin working.

[0074] The RemoveDev unit is used to remove a specified device from the system and stop communication and management with that device.

[0075] The UpdateDev unit is used to update the device's configuration information or parameters.

[0076] The GetDev unit is used to obtain detailed information about the device, such as device status and configuration parameters.

[0077] The UpdateDevTwins unit is used to update the device's twin data to ensure that the device twin is consistent with the actual device state.

[0078] The DevModel unit is used to define the model of a device, including its properties, events, etc.

[0079] DataPanel components include

[0080] The Pushdata data upload unit sends the data collected by the device to EdgeCore or a specified target to achieve data upload.

[0081] The PullData unit retrieves data from EdgeCore or other data sources to download the data.

[0082] The Driver component includes:

[0083] The DeviceType unit is used to define the type of device, such as a sensor;

[0084] The GetDeviceData unit is used to read data from the device, such as process data collected by sensors and operating parameters of the device.

[0085] The GetDeviceStatus unit is used to obtain the current status of the device, such as whether the device is online or working properly.

[0086] The InitDevice unit is used to perform device initialization operations, such as setting the device's initial parameters and establishing a communication connection with the device.

[0087] The SetDeviceData unit is used to send data to devices, such as sending control commands to device 1, device 2, device N, etc.

[0088] The StopDevice unit is used to stop the operation of a device, such as turning off the device's power or communication connection.

[0089] As the core node of edge computing, EdgeCore is responsible for managing and coordinating communication and data processing between edge devices and the cloud platform. It connects to the grpcClient and grpcServer components through the DMI device management interface.

[0090] The grpcClient component includes:

[0091] The RegisterMapper unit is used to register Mappers with Edgecore, informing Edgecore of the existence of the Mapper and related information so that Edgecore can manage and schedule it.

[0092] The ReportStatus unit is used to report the running status of the Mapper to Edgecore;

[0093] The ReportState unit is used to report device status information to Edgecore.

[0094] The gRPC Server components include:

[0095] The RegisterDevice unit is used to receive device registration requests and add device information to the system's device list.

[0096] The RemoveDevice unit is used to handle device removal requests;

[0097] The UpdateDevice unit is used to update basic information about the device.

[0098] The GetDevice unit is used to obtain detailed information about the device based on the device identifier.

[0099] The UpdateTwin unit is used to update the twin data of the device.

[0100] The edge IoT data processing method of the present invention sets multiple water level markers along the data stream, and the algorithm formula for the water level markers is: W(t) e ) = max(t e )-Δ;

[0101] Where max(t) e The maximum event time currently observed is 0.

[0102] Δ is the maximum allowed out-of-order delay;

[0103] t e The event time is t1, t2, t3, t4, t5, t6, where t1 is less than t2, less than t3, less than t4, less than t5, and less than t6.

[0104] One specific embodiment, referring to Figure 8 The water level markers W(20) and W(10) are shown. The event timestamps following the water level markers are all greater than the water level markers, therefore the data stream is not out of order. (Refer to...) Figure 9 The water level markers W(20), W(17), W(13), W(11), W(7), and W(3) are shown, where the event timestamp 19 is out of order and needs to be corrected. This invention ensures that events within the pipeline are ordered through a water level mechanism, effectively handling network jitter, packet loss, and delay issues. A water level W(t) e This indicates that the event time in the current stream has reached timestamp t. e This means t e All the previous data has been collected; the event timestamp 't' will no longer appear in the stream. e '≤t e The data.

[0105] Reference Figure 7 The DataPreProcess component also performs the following steps:

[0106] Extract the event timestamps from the data stream to obtain a data stream with event timestamps;

[0107] An adaptive sliding window is assigned to the data stream with event timestamps to obtain an adaptively sampled data stream. The algorithm formula of the adaptive sliding window is: Windows(t) = {[iS, iS + W')|iS ≤ T < iS + W'}, where t is the current time, W' is the window size, S is the step size, i is an incrementing positive integer variable, iS is the start point of the adaptive sliding window, and iS + W' is the end point of the adaptive sliding window;

[0108] Aggregate calculations are performed on the adaptively sampled data stream to obtain the aggregate calculation results for each window.

[0109] A specific embodiment: W' = 2min, S = 1min, and the value range of i is i ≤ 12:01 < i + 2, thus obtaining the window range. All data in the window can be sampled through the window for subsequent mathematical calculations

[0110] Figure 10 Schematic diagrams of the strategies representing Fixed Window (Fixed), Sliding Window (Sliding), and Sessions Window (Sessions).

[0111] When using a 5s fixed window, let the allowed latency be Δ (such as 10–​15s), and watermark = the maximum event time currently observed - Δ. The window result is output only when the watermark is reached. Subsequently arriving "late data" is either discarded or corrected by bypass.

[0112] In this invention, an adaptive sliding window is assigned, AssignWindows(Sliding(W', 1m)), where W' is a variable value that can control the expansion and contraction of the window. By listening for custom events, the window is adaptively adjusted according to event driving, and adaptively sampled. The sampling window is driven in real time according to the resource ratio, reducing memory usage. [[ID=!9]]

[0113] The data within the adaptive sliding window supports various aggregate data operations, such as average calculation, maximum value calculation, minimum value calculation, etc. It supports extending ways such as EL expressions to write formulas for calculation. When the window slides, the calculation can be automatically triggered, or triggered according to event driving. Then, the aggregate calculation results are pushed to the local MQTT server through the Pushdata data upload unit. Programs such as water plant applications can listen to the MQTT server to obtain the aggregate calculation results.

[0114] For the edge - side Internet of Things data processing method of this invention, the data pre - processing component DataPreProcess also fills in the missing data in the data stream based on the following exponential smoothing calculation formula:

[0115] EMA t=α·Y t +(1-α)·EMA t-1

[0116] Where t is the current time;

[0117] Y t This refers to the actual observed value at time point t;

[0118] EMA t This is the exponential moving average calculated at time t. This value can be seen as either a smoothed value at the current time or a predicted value for the next time t+1.

[0119] EMA t-1 It is the exponential moving average of the previous time point, which contains all historical information up to time t-1;

[0120] α is the smoothing coefficient, 0 < α ≤ 1. Based on the attributes of IoT devices in the water industry, the coefficient is 0.7 here, giving the model a greater weight on recent data, making it more sensitive to new changes, and able to quickly reflect the latest changes. For scenarios such as water plants, the effect of data completion is more correlated with recent data, and the predicted value can more accurately express expectations.

[0121] Reference Figure 11 , Figure 12 , Figure 13 Several data completion algorithms were compared: MA (Moving Average), EMA (Exponential Easing), Kalman Filter, XGBoost, and LightGBM. Tests were conducted using actual IoT data. From the completion effect perspective, XGBoost and EMA performed best. Kalman Filter can express trends but has a significant lag on the x-axis. However, XGBoost is computationally complex, requires substantial computing power, and has a slow output speed (0.2–1 second), making it unsuitable for low-computing-power, high-real-time scenarios at the edge. EMA was used in combination for rapid data completion. The default smoothing coefficient for EMA in the water industry is 0.7, which can be adjusted according to the specific equipment requirements.

[0122] The edge IoT data processing method of the present invention refers to the edge terminal as the edge gateway of the direct drinking water device, and the data stream as the water flow rate of the direct drinking water device.

[0123] An edge IoT data processing system based on the Kubeedge architecture is used to execute the aforementioned edge IoT data processing method based on the Kubeedge architecture. The Mapper layer in the Kubeedge architecture includes...

[0124] Data interaction components interact with IoT devices to obtain data streams from them;

[0125] The data preprocessing component, connected to the data interaction component, assigns event timestamps and watermark identifiers W(t) to the data stream. e ), determine the water level marker W(t) e Is the event timestamp following the waterline identifier W(t) less than or equal to the waterline identifier? e The timestamp t e If less than or equal to timestamp t e This corrects out-of-order events in the data stream.

[0126] The edge IoT data processing system of the present invention includes a data preprocessing component that extracts the event timestamps of the device from the data stream to obtain a data stream with event timestamps.

[0127] An adaptive sliding window is assigned to a data stream with event timestamps to obtain an adaptively sampled data stream.

[0128] The aggregate calculation results for each window are obtained by performing aggregation calculations on the adaptively sampled data stream;

[0129] The aggregation calculation results are pushed to the local MQTT server through the Pushdata data upload unit in the DataPanel component of the Mapper layer.

[0130] The edge IoT data processing system of the present invention further includes a data preprocessing component that fills in missing data in the data stream based on the following exponential smoothing formula:

[0131] EMA t =α·Y t +(1-α)·EMA t-1

[0132] Where t is the current time point;

[0133] Y t The actual observed value at time point t;

[0134] EMA t This is the exponential moving average calculated at time point t;

[0135] EMA t-1 This is the exponential moving average of the previous time point;

[0136] α is the smoothing coefficient, 0 < α ≤ 1.

[0137] In one specific embodiment, the edge end is the edge gateway of the direct drinking water device, and the data stream is the water flow rate of the direct drinking water device.

[0138] The water plant management system is a program for water plant management. Originally deployed on a server, it was changed to deploy the application directly to the edge gateway via the cloud through the Kubeedge framework due to real-time requirements and the complex environment of the site. The application obtains water quality data, such as pH value, through sensors. However, pH data is occasionally missing due to sensor and transmission reasons. But pH rarely shows abrupt changes in a very short time and the trend of change is relatively smooth. Therefore, the data is directly supplemented by data preprocessing components.

[0139] Reference Figure 14 The supplemented data accurately expresses trends compared to the original data, and its predictions are more accurate than those of the original data. The predicted data can be directly used for subsequent business processing.

[0140] The pH data table contains 10,800 data points, with 1,683 missing data points.

[0141] Data Quality Assessment Report:

[0142] Total data points: 10800;

[0143] Missing data points: 1683 (15.58%);

[0144] Data points within the standard range: 10753 (99.56%);

[0145] EMA smoothing factor: 0.7.

[0146] Reference Figure 15 A direct drinking water management system is a management program for direct drinking water equipment. Since direct drinking water equipment deployment sites generally lack the conditions for server installation, an edge gateway is installed on the equipment to process real-time data. Controlling the water flow rate of the direct drinking water equipment is a crucial function. Using the aggregation function of a data preprocessing component, the water flow rate within a certain time window can be accurately calculated. By subtracting the inflow value at the beginning of the time window from the current flow rate, the accurate water flow rate for that period can be obtained. When the water flow rate exceeds a set value, the sensor can be shut off via the Kubeedge framework.

[0147] The description and accompanying drawings provide typical embodiments of specific structures for specific implementations. Other modifications are possible based on the spirit of the invention. While the above-described invention presents preferred embodiments, these are not intended to be limiting.

[0148] For those skilled in the art, various changes and modifications will undoubtedly be apparent after reading the above description. Therefore, the appended claims should be construed as covering all changes and modifications that encompass the true intent and scope of the invention. Any and all equivalent scope and content within the scope of the claims should be considered to remain within the intent and scope of the invention.

Claims

1. An edge IoT data processing method based on the Kubeedge architecture, characterized in that, The Mapper layer in the Kubeedge architecture includes a data preprocessing component, and the data preprocessing component performs the following steps: Obtain a data stream; Assign event timestamps and watermark identifiers W(t) to the data stream. e ); Determine the water level indicator W(t) e Is the event timestamp following the water level identifier W(t) less than or equal to the water level identifier W(t)? e The timestamp t e If less than or equal to the timestamp t e This corrects the out-of-order events in the data stream.

2. The edge IoT data processing method according to claim 1, characterized in that, Multiple water level markers are set along the data stream, and the algorithm formula for the water level markers is: W(t) e ) = max(t e )-Δ; Where max(t) e The maximum event time currently observed is 0. Δ is the maximum delay allowing out-of-order; t e The event time is t1, t2, t3, t4, t5, t6, where t1 is less than t2, less than t3, less than t4, less than t5, and less than t6.

3. The edge IoT data processing method according to claim 1, characterized in that, The data preprocessing component also performs the following steps of aggregation calculation: Extract the event time of device ticking from the data stream to obtain a data stream with event timestamps; Allocate an adaptive sliding window to the data stream with event timestamps to obtain an adaptively sampled data stream. The algorithm formula of the adaptive sliding window is: Windows(t) = {[iS, iS + W')|iS ≤ T < iS + W′}, where t is the current time, W′ is the window size, S is the step size, i is a positive integer variable, iS is the start point of the adaptive sliding window, and iS + W' is the end point of the adaptive sliding window; Perform aggregation calculation on the adaptively sampled data stream to obtain the aggregation calculation result of each window.

4. The edge IoT data processing method according to claim 3, characterized in that, The Mapper layer includes a DataPanel component, and the DataPanel component includes a Pushdata data upload component; Push the aggregation calculation result to the local MQTT server through the Pushdata data upload component; The aggregation calculation includes at least one of average value calculation, maximum value calculation, and minimum value calculation.

5. The edge IoT data processing method according to claim 1, characterized in that, The data preprocessing component includes a data completion step: Complete the missing data in the data stream based on the following exponential smoothing calculation formula: EMA t =α·Y t +(1-a)·EMA t-1 where t is the current time; Y t The actual observed value at time point t; EMA t This is the exponential moving average calculated at time point t; EMA t-1 This is the exponential moving average of the previous time point; α is the smoothing coefficient, 0 < α ≤ 1.

6. The edge IoT data processing method according to claim 1, characterized in that, The edge side is the edge gateway of the direct drinking water device, and the data stream is the water flow of the direct drinking water device.

7. An edge-end IoT data processing system based on the Kubeedge architecture, characterized in that: For implementing the edge-side Internet of Things data processing method based on the Kubeedge architecture according to any one of claims 1-6, the Mapper layer in the Kubeedge architecture includes, A data interaction component that interacts with the Internet of Things device to obtain the data stream of the Internet of Things device; A data preprocessing component, connected to the data interaction component, assigns event timestamps and watermark identifiers W(t) to the data stream. e ), determine the water level marker W(t) e Is the event timestamp following the water level identifier W(t) less than or equal to the water level identifier W(t)? e The timestamp t e If less than or equal to the timestamp t e This corrects the out-of-order events in the data stream.

8. The edge IoT data processing system according to claim 7, characterized in that, The data preprocessing component extracts the event time of device ticking from the data stream to obtain a data stream with event timestamps; Allocate an adaptive sliding window to the data stream with event timestamps to obtain an adaptively sampled data stream. The algorithm formula of the adaptive sliding window is: Windows(t) = {[iS, iS + W')|iS ≤ T < iS + W'}, where t is the current time, W' is the window size, S is the step size, i is a positive integer variable, iS is the start point of the adaptive sliding window, and iS + W' is the end point of the adaptive sliding window; Perform aggregation calculation on the adaptively sampled data stream to obtain the aggregation calculation result of each window; Push the aggregation calculation result to the local MQTT server through the Pushdata data upload component in the DataPanel component of the Mapper layer.

9. The edge IoT data processing system according to claim 7, characterized in that, The data preprocessing component also completes the missing data in the data stream based on the following exponential smoothing calculation formula: EMA t =α·Y t +(1-a)·EMA t-1 where t is the current time; Y t The actual observed value at time point t; EMA t This is the exponential moving average calculated at time point t; EMA t-1 This is the exponential moving average of the previous time point; α is the smoothing coefficient, 0 < α ≤ 1.

10. The edge IoT data processing system according to claim 7, characterized in that, The edge terminal is the edge gateway of the direct drinking water device, and the data stream is the water flow rate of the direct drinking water device.

Images