An internet of things data distribution engine algorithm

Abstract

The application relates to the technical field of data processing distribution, and discloses an Internet of Things data distribution engine algorithm, which comprises the following steps: firstly, original power data collected is cleaned and standardized; then, feature extraction and semantic analysis are carried out; dynamic priority scores representing data real-time urgency and static importance factors representing inherent importance of equipment are generated by calculating a plurality of dynamic feature indexes of the equipment and combining business metadata of the equipment; further, a hierarchical scheduling strategy is adopted, and secondary judgment based on change trend is started on uncertain data, so that accurate routing of data to different priority channels is realized; finally, the system dynamically adjusts distribution thresholds according to actual business utility feedback of data distribution, so that a closed-loop optimization is formed; the application effectively solves the problems of key information submergence, response delay and extensive resource allocation in massive Internet of Things data, and realizes low-delay transmission of high-value data and adaptive optimization of system resources.

Description

Technical Field

[0001] This invention relates to the field of data processing and distribution, and more particularly to an Internet of Things (IoT) data distribution engine algorithm. Background Technology

[0002] In industrial park energy management systems, thousands of key process devices continuously generate massive amounts of power time-series data to achieve precise monitoring; however, the value density of these data streams is extremely uneven, with the vast majority of data representing only background information for normal operation, and only a few containing high-value signals such as equipment malfunctions, energy efficiency degradation, or early signs of failure.

[0003] Existing data acquisition and transmission schemes typically treat all data equally, resulting in valuable network bandwidth and computing resources being occupied by a large amount of low-value data, causing truly critical information to be submerged in the data deluge and difficult to identify and process in real time.

[0004] Furthermore, traditional systems typically employ static and preset data distribution strategies, failing to self-optimize based on real-time data load and business feedback. When faced with dynamic load changes due to shift changes and operating conditions, fixed distribution thresholds easily lead to system congestion during peak hours and resource idleness during off-peak hours. The entire system lacks a closed-loop control mechanism capable of sensing its own distribution efficiency and dynamically optimizing it, making it difficult to consistently ensure low-latency transmission and processing of critical data during long-term operation, thus hindering further improvements in the intelligence level of the energy management system.

[0005] Therefore, there is an urgent need for an IoT data distribution engine algorithm to achieve highly reliable scheduling between multiple devices and multiple controllers. Summary of the Invention

[0006] The objective of this invention can be achieved through the following technical solution: an Internet of Things (IoT) data distribution engine algorithm, comprising the following steps;

[0007] S1. Collect raw power data and equipment metadata of key process equipment in each workshop of the industrial zone in real time, and clean and standardize them to obtain a standard power data stream;

[0008] S2. Based on standard power data streams, calculate the instantaneous power anomaly, current deviation, voltage deviation, power trend, energy efficiency health, and energy consumption anomaly of the equipment to form a power feature set; calculate dynamic priority scores based on the power feature set, and determine static importance factors according to the equipment level and business type in the equipment metadata;

[0009] S3. Combine the static importance factor and dynamic priority score to calculate the comprehensive distribution confidence. Use a hierarchical scheduling strategy to divide the data to be distributed into high-priority data, data to be observed and regular data. Send the high-priority data, data to be observed and regular data to the high-priority distribution channel, the second-priority distribution channel and the regular distribution channel respectively.

[0010] S4. Based on the business utility feedback of data distribution, dynamically adjust the distribution threshold to optimize the process, realize adaptive closed-loop optimization, and adjust the entry conditions of the data distribution channel according to the load of the data distribution channel.

[0011] Preferably, the specific steps of S1 are as follows:

[0012] A unique identifier is constructed for each device. Raw power data and device metadata are collected periodically or event-triggered through IoT gateways. Duplicate data removal, outlier identification, clock synchronization, and missing value filling are performed to output a standard power data stream with a unified timestamp, no duplicates, and no missing values.

[0013] Device metadata includes device location, device level, and service type.

[0014] The preferred set of power characteristics is calculated using the following steps:

[0015] Based on standard power data streams, the instantaneous power anomaly and current deviation are calculated using the Z-Score method, and the voltage deviation is calculated using the percentage deviation relative to the rated voltage. A short-time sliding window is set, and the slope of the linear fitting of instantaneous power within the window is calculated to obtain the power trend. The normalized ratio of the average reactive power ratio is calculated to obtain the energy efficiency health. The Z-Score of the average active power sequence relative to the historical baseline is calculated to obtain the energy consumption anomaly.

[0016] The final output includes a set of power characteristics, including instantaneous power anomaly, current deviation, voltage deviation, power trend, energy efficiency health, and energy consumption anomaly.

[0017] The preferred method is dynamic priority scoring, specifically analyzed as follows:

[0018] The maximum value among instantaneous power anomaly, current deviation, and voltage deviation is used as the instantaneous anomaly level factor, energy efficiency health is used as the energy efficiency anomaly level factor, and the average value of power trend and energy consumption anomaly is used as the trend anomaly level factor. The dynamic priority score is obtained by comprehensively calculating the instantaneous anomaly level factor, energy efficiency anomaly level factor, and trend anomaly level factor.

[0019] The preferred static importance factor is analyzed as follows:

[0020] The weights of the device level and the service type are obtained according to the predefined mapping table. The weights of the device level and the service type are combined and the maximum value is obtained by an aggregation function. The result of this aggregation function is the static importance factor.

[0021] The preferred hierarchical scheduling strategy is as follows:

[0022] By jointly calculating static importance factors and dynamic priority scores, and combining dynamic threshold ranges and trend re-judgment mechanisms, high-priority data is processed immediately, data to be observed is further analyzed, and routine data is aggregated and sent with a delay, thereby achieving a dynamic balance between resource utilization and response efficiency.

[0023] Preferably, further analysis of the observed data includes:

[0024] The dynamic priority scores of the target device at the most recent M sampling times are obtained to form a dynamic priority score sequence. The dynamic priority score sequence is linearly fitted based on the least squares method to obtain a linear function, where the slope of the linear function is the slope of the dynamic priority score change trend.

[0025] The slope of the dynamic priority score change trend is compared with the preset positive and negative priority slope thresholds to obtain the corresponding data distribution channels.

[0026] The positive and negative priority slope thresholds are determined based on the slope distribution of the historical dynamic priority score change trend during normal equipment operation. The positive priority slope threshold is the 95th percentile of the slope distribution of the historical dynamic priority score change trend, and the negative priority slope threshold is the 5th percentile of the slope distribution of the historical dynamic priority score change trend.

[0027] The preferred data distribution channel is as follows:

[0028] Data is sent in a strict priority order of high priority distribution channel > second priority distribution channel > normal distribution channel; data in the high priority distribution channel is always sent first, data in the second priority distribution channel is sent only when the high priority distribution channel is empty, and data in the normal distribution channel is processed last.

[0029] Preferably, the entry conditions for the data distribution channel load adjustment data distribution channel specifically include:

[0030] When the preset energy management system detects that the high-priority distribution channel has been overloaded for a long time and the current response time exceeds the preset response time, the energy management system automatically raises the dynamic distribution threshold range and increases the standard for entering the high-priority distribution channel to ensure the instantaneous response capability of the most critical data.

[0031] When the energy management system detects that a high-priority distribution channel has been idle for an extended period and that critical alarms have an acceptable preset response time, the energy management system automatically lowers the dynamic distribution threshold range, allowing more potentially important data to enter the priority channel and improving the sensitivity of the energy management system.

[0032] The preset response time is set based on the maximum acceptable response latency under the analysis of key business scenarios.

[0033] Preferably, the distribution threshold is dynamically adjusted, specifically as follows:

[0034] The automatic adjustment of the distribution threshold is implemented based on the proportional-integral controller logic. The adjustment range is calculated by combining the current instantaneous error and the historical cumulative error. The proportional component is calculated based on the error between the current response time and the preset response time and is used to adjust the distribution threshold in real time. The integral component is calculated based on the cumulative calculation of a period of historical response error and is used to correct long-term deviations.

[0035] The final distribution threshold adjustment is the weighted sum of the proportional and integral components, with the proportional component having a greater weight than the integral component, to ensure that the energy management system achieves a balance between rapid response and long-term stability.

[0036] The beneficial effects of this invention are:

[0037] (1) This invention extracts abnormal features from instantaneous data and transforms them into dynamic priority and static importance labels with clear business significance. Further analysis of the observed data enables the prediction of equipment status from instantaneous monitoring to continuous trend prediction. It achieves a leap from raw data to intelligent decision-making basis, laying a solid foundation for subsequent accurate data distribution.

[0038] (2) This invention achieves precise on-demand allocation of system resources through an innovative hierarchical scheduling mechanism. This mechanism performs rapid initial screening of data and initiates secondary judgment based on changing trends for uncertain data, which can keenly capture potential risks; it ensures that system bandwidth and computing power always prioritize the most urgent and important data, thereby ensuring instantaneous response to critical events even under high load;

[0039] (3) This invention dynamically adjusts the internal decision threshold by monitoring the actual effect of data distribution in real time; this closed-loop design enables the system to actively adapt to changing loads and maintain the best operating state for a long time without human intervention, which significantly improves the adaptability, long-term stability and efficiency of the entire system. Attached Figure Description

[0040] The invention will now be further described with reference to the accompanying drawings.

[0041] Figure 1This is a flowchart illustrating an IoT data distribution engine algorithm according to the present invention. Detailed Implementation

[0042] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0043] In modern industrial park energy management, real-time monitoring and intelligent processing of massive amounts of equipment power consumption data are crucial for ensuring production safety and energy efficiency. However, traditional energy management systems face a core pain point: they cannot distinguish between routine data and high-value alarm data indicating equipment anomalies, performance degradation, or sudden failures at the data source. This results in all data being transmitted indiscriminately to the cloud, causing critical alarms to be overwhelmed by the data deluge, leading to response delays, and the energy management system's resources being consumed by a large amount of invalid data.

[0044] This invention aims to solve this core problem. The data distribution engine built based on this algorithm can calculate the dynamic urgency and static importance of each device data point in real time. This enables the energy management system to intelligently classify data at the network edge and achieve precise diversion and scheduling.

[0045] Taking the energy management system of a modern industrial park as an example, the systems described below are all energy management systems. For solutions to the aforementioned technical problems, please refer to [link / reference needed]. Figure 1 As shown, this invention is an IoT data distribution engine algorithm, comprising:

[0046] S1. Collect raw power data and equipment metadata of key process equipment in each workshop of the industrial zone in real time, and perform preliminary cleaning and standardization to provide a high-quality data foundation for subsequent feature quantification and priority determination.

[0047] S1 includes the following steps:

[0048] S11. During the system initialization phase, a unique identifier is assigned to the key process equipment of all monitored workshops in the industrial zone. Specifically, each workshop and key process equipment is assigned a number i, where i = 1, 2, 3, ..., I, and I represents the total number of all workshops and key process equipment. This number i, along with metadata such as the physical location of the workshop equipment, equipment level, and business type, is stored in the system equipment registry for automatic association and identification of data sources during data collection.

[0049] S12. Through the Internet of Things gateway and data interface, periodically or event-triggeredly collect raw power data from smart meters and sensors on each workshop and key process equipment numbered i (from number 1 to I); the raw power data includes at least: instantaneous power, current, voltage, reactive power, active energy value and reactive energy value.

[0050] S13. Clean and standardize the collected raw power data stream to ensure the uniqueness, consistency, and integrity of the data. This step includes:

[0051] S131. Duplicate Data Removal: The system identifies and removes completely duplicate data packets based on the device number-timestamp combination key of the data, ensuring the uniqueness of each data item.

[0052] S132. Abnormal Data Detection and Filtering: The system uses at least one of the following methods to identify and remove abnormal data:

[0053] Threshold method: The data value is compared with a preset reasonable upper and lower limit (for example, the current value should be between 0A and the maximum capacity of the circuit breaker). Data that exceeds the range is considered abnormal.

[0054] Rate of change method: Calculate the rate of change of data between adjacent periods. If the rate of change exceeds the physically possible threshold for abrupt change (e.g., power changes by more than 50% of the rated value within 1 second), it is considered abnormal data.

[0055] Isolate the marked abnormal data and log it, so that it does not enter the subsequent processing flow;

[0056] S133. The device clock at the data acquisition point is used as the reference, and the IoT gateway performs clock synchronization correction during data transmission. If the device does not have a reliable clock, the system receiving timestamp of the data arriving at the acquisition module shall be used as the reference, and this shall be noted in the device metadata. The device metadata includes the device location, device level, and service type.

[0057] For data loss caused by transmission loss or other reasons, the system uses linear interpolation or forward filling to fill the missing data. Specifically, in a continuous time series, if data is missing at a certain time point, the data from the previous valid time point is used to fill the missing data (forward filling), or linear calculation is performed based on the data from the previous and next valid time points to fill the missing data (linear interpolation).

[0058] S14. After the above steps, the raw power data is processed to obtain a set of standard power data streams with a unified timestamp, no duplication, no anomalies, and complete data.

[0059] In this embodiment, by assigning a unique logical identifier to all devices and collecting their multi-parameter raw power data, a systematic cleaning and standardization process is performed, including duplicate data removal, outlier filtering, timestamp alignment, and missing value imputation. Finally, a high-quality, uniform, and well-organized standard power data stream is output, laying a solid data foundation for subsequent in-depth feature analysis and value mining.

[0060] S2. Perform feature extraction and semantic analysis on the standard power data stream to obtain dynamic priority scores and static importance factors;

[0061] S2 includes the following steps:

[0062] S21. For each device i, the instantaneous parameters (instantaneous power Pi, current Li, voltage Yi) are maintained by the system. A sliding time window of length N (e.g., N=300, corresponding to 5 minutes of 1Hz data) is calculated. The mean and standard deviation of instantaneous power Pi, current Li and voltage Yi within the sliding time window are calculated and used as the dynamic reference baseline for the parameter to operate normally in the near future.

[0063] S22. Anomaly calculation based on instantaneous parameters:

[0064] Calculate the instantaneous power anomaly Z1: For the acquired instantaneous power Pi, the Z-Score method is used to calculate the standardized score of the instantaneous power Pi's deviation from the baseline within the sliding time window, which is taken as the instantaneous power anomaly Z1, where the instantaneous power anomaly is: Where Pμ and Pσ are the mean and standard deviation of instantaneous power; the larger the instantaneous power anomaly Z1 value, the greater the deviation of the current power level from its normal fluctuation range, and the higher the possibility of load anomaly.

[0065] Calculate the current deviation Z2: For the acquired parameter current Li, the Z-Score method is used to calculate the standardized fraction of the current Li's deviation from the baseline within the sliding time window, which is taken as the current deviation Z2. The calculation formula is as follows: , where Lμ and Lσ are the mean and standard deviation of the current, respectively; the current anomaly can identify overcurrent or undercurrent conditions, can be used as an auxiliary verification of power anomalies, and can detect potential problems of low power factor;

[0066] Calculate voltage deviation Z3: For the collected parameter voltage Yi, since the voltage should be stable near the rated value, the percentage deviation from the rated voltage is used as the voltage deviation. The calculation formula is as follows: Y1 is the rated voltage, and Z3 is a percentage value that directly reflects the degree to which the current voltage deviates from the rated value. The larger the value, the worse the power supply quality.

[0067] S23. Determine whether the power consumption level of the device is in a continuous increasing or decreasing trend. Perform linear fitting on the instantaneous power sequence in a sliding time window of length N to obtain its slope. Use the slope as the power trend degree Z4. Z4 is the slope of the fitted line. If it is positive, it indicates that the power consumption is continuously increasing, which may indicate a change in the device's operating conditions or a decrease in efficiency.

[0068] To quantify the energy utilization efficiency of equipment and identify ineffective operating states with high energy consumption and low output, the average active power and average reactive power within a fixed time interval (e.g., 5 minutes) are obtained by using the collected cumulative energy values ​​and calculating the difference. Subsequently, the approximate proportion of average reactive power in the sum of average active power and average reactive power is calculated, and this energy efficiency proportion is denoted as energy efficiency health Z5. Z5 is a dimensionless ratio between 0 and 1. The larger the value, the more the electrical energy is used for ineffective electromagnetic energy exchange rather than actual work, the lower the power factor of the system, and the worse the energy efficiency.

[0069] To detect persistent anomalies at a more macro level of energy consumption (rather than instantaneous power), the system maintains a sequence of average active power over a continuous time window and calculates the Z-Score of the current average active power relative to the historical baseline of this sequence as the energy consumption anomaly degree Z6. Z6 reflects the anomaly of the average energy consumption level and can detect energy efficiency anomalies that are not high in absolute power but continue to deviate from the normal baseline.

[0070] Through the above steps, the original multidimensional power data is output as a set of power characteristics {Z1,Z2,Z3,Z4,Z5,Z6}, which includes instantaneous power anomaly, current deviation, voltage deviation, power trend, energy efficiency health, and energy consumption anomaly. This lays a solid data foundation for the next stage of data analysis.

[0071] S24. The power feature set {Z1, Z2, Z3, Z4, Z5, Z6} is fused and mapped using a preset aggregation rule base; specifically:

[0072] S241. This step aims to generate a comprehensive dynamic priority score F, which ranges from 0 to 1. A higher value indicates a higher real-time urgency of the data. The system determines the value of the dynamic priority score F by executing the following sub-steps:

[0073] Calculate the instantaneous anomaly level factor: Select the maximum value from the instantaneous power anomaly degree Z1, current deviation degree Z2 and voltage deviation degree Z3 as the instantaneous anomaly level factor to ensure that any serious anomaly of any critical electrical parameter can be quickly captured;

[0074] Calculate the energy efficiency anomaly level factor: Obtain the value of energy efficiency health Z5 as the energy efficiency anomaly level factor. The higher the value of Z5, the lower the equipment operating efficiency and the greater the negative impact of its abnormal condition on the system energy efficiency.

[0075] Calculate the trend anomaly level factor: take the average of the power trend degree Z4 and the energy consumption anomaly degree Z6 as the trend anomaly level factor, and use this to comprehensively judge whether there is a trend of continuous deterioration in the power consumption of the equipment.

[0076] Finally, the instantaneous anomaly level factor, energy efficiency anomaly level factor and trend anomaly level factor are weighted and fused according to the preset multi-factor weighted model: F = instantaneous anomaly level factor * a1 + energy efficiency anomaly level factor * a2 + trend anomaly level factor * a3, and the final dynamic priority score F is calculated, where a1, a2 and a3 are preset weight coefficients, and a1 + a2 + a3 = 1.

[0077] It should be noted that the values ​​of the weight coefficients a1, a2, and a3 are based on the principle of maximizing historical utility and are objective results obtained through linear regression calculation. Specifically, the system collects a large number of historical abnormal event samples, and domain experts label each sample with its actual urgency level according to the actual impact of the event. Subsequently, a linear regression algorithm is used to solve for a set of optimal weights that minimize the mean square error between the model's predicted score and the expert-labeled level. This set of weights is the value mentioned above. It is not subjectively set, but rather a data-driven fit to the historical decision-making patterns of experts, thereby ensuring the objectivity and accuracy of the model's judgment.

[0078] S242. Obtain the metadata of device number i and query the predefined business importance rule base; wherein the business importance rule base performs a comprehensive mapping of the device based on the device's hierarchical weight and business type weight;

[0079] Equipment level weights determine the breadth of influence based on the equipment's physical location within the energy supply network. Equipment levels can be categorized into overall site overview, workshop, production line, and individual equipment levels. The overall site overview level has a weight of 1.0; data anomalies at this level could trigger a plant-wide power outage or energy interruption, having the largest impact. The workshop level has a weight of 0.8; data anomalies at this level would cause the entire production workshop or a large area to shut down, having a significant impact. The production line level has a weight of 0.6; data anomalies at this level would affect a complete production line, having a moderate impact. The individual equipment level has a weight of 0.3; data anomalies at this level would only affect the operation of a single piece of equipment, having a localized impact.

[0080] The business type weight determines the value density of equipment based on its role in production. Business types can be divided into core production equipment, critical plant support equipment, general production equipment, and non-critical equipment. Core production equipment has a weight of 1.0, directly affecting the output of major products; downtime will result in significant output value loss. Critical plant support equipment has a weight of 0.8, although it does not directly produce, it provides the necessary power and environment for production (such as air compressors and chillers); its downtime may lead to production interruption. General production equipment has a weight of 0.5, participating in the production process, but may have backups or short-term downtime has a smaller impact. Non-critical equipment has a weight of 0.2, with low correlation to core production (such as general lighting and office air conditioning), and its energy consumption anomalies have a low priority.

[0081] The system automatically performs the following automated steps to calculate the static importance factor G for each device i:

[0082] Metadata retrieval: The system retrieves the pre-stored device metadata, namely device level and service type, based on device number i.

[0083] Query weight values: Based on the retrieved device level, query the corresponding level weight from the above device level weight mapping rules. At the same time, based on the business type, query the corresponding type weight from the business type weight mapping rules.

[0084] Calculate the static importance factor G: The system combines the weight values ​​of device level weight and business type weight, and uses an aggregation function that takes the maximum value to capture the highest importance of the device in any dimension: G = max(device level weight, business type weight), to obtain the static importance factor G;

[0085] The final outputs are a dynamic priority score F and a static importance factor G;

[0086] In this embodiment, by extracting features from standard power data streams, dynamic priority scores and static importance factors are generated for each device based on the extracted features. Specifically, this step first maintains a sliding time window for the device and extracts six features with clear physical and statistical significance from its instantaneous parameters (power, current, voltage) and cumulative energy value, quantifying the device's operating status from three dimensions: instantaneous anomalies, operating trends, and energy efficiency health. Subsequently, these numerical features are aggregated into a dynamic priority score reflecting real-time urgency through a preset weighted model. At the same time, a static importance factor reflecting inherent business value is mapped based on the device's metadata and business rule base, thus providing a core basis for subsequent data priority scheduling and decision-making.

[0087] S3. Combine the static importance factor and dynamic priority score to calculate the comprehensive distribution confidence. Use a hierarchical scheduling strategy to divide the data to be distributed into high-priority data, data to be observed and regular data. Send the high-priority data, data to be observed and regular data to the high-priority distribution channel, the second-priority distribution channel and the regular distribution channel respectively.

[0088] S3 includes the following steps:

[0089] S31. Calculate a comprehensive distribution confidence level Q for each device, substituting the dynamic priority score F and the static importance factor G into the set formula. The overall distribution confidence level Q is obtained, where β1 and β2 are the set weight coefficients, and β1+β2=1;

[0090] It should be noted that the weight coefficients β1 and β2 are determined as follows: The system collects a training set containing N historical data samples, each sample containing its (F,G) value. Domain experts label each sample with a true distribution priority label based on the actual business impact of the event (e.g., must be processed immediately = 1, can be delayed = 0). The system aims to minimize the cross-entropy loss between the predicted priority and the true priority. It uses a grid search method to traverse the range β1∈[0,1] with a step size of 0.01. For each β1, β2=1-β1 is calculated, and its loss value on the validation set is calculated. Finally, the combination of β1 and β2 that minimizes the loss on the validation set is selected as the optimal weight.

[0091] A preset dynamic distribution threshold range [U] 低, U 高 The system compares and analyzes the obtained comprehensive priority confidence level with the dynamic distribution threshold range. When the comprehensive priority confidence level is greater than or equal to the maximum value of the dynamic distribution threshold range, the data of the device is marked as high-priority data and immediately enters the high-priority distribution channel for transmission. The high-priority distribution channel has the highest bandwidth and computing resources to ensure that the data in it is distributed immediately, so as to achieve instantaneous response to confirmed high-urgency events.

[0092] When the overall priority confidence level is less than or equal to the minimum value of the dynamic distribution threshold range, the data of the device is marked as regular data and sent using the regular distribution channel. The regular distribution channel adopts optimization strategies such as batch aggregation and delayed sending to efficiently process massive amounts of non-urgent regular data.

[0093] When the overall priority confidence level is within the dynamic distribution threshold range, the data of the device is marked as data to be observed and sent to S32 to initiate a trend analysis process based on dynamic priority scoring.

[0094] The dynamic distribution threshold range is based on the system resource capacity planning. It uses statistical analysis to comprehensively analyze the historical distribution of the distribution confidence Q and selects a specific percentile as the boundary to achieve a balanced distribution of the load on each priority channel.

[0095] S32. Obtain the dynamic priority scores of device i at the most recent M times, forming a dynamic priority score sequence Fm={F1,F2,F3,…,FM} at the most recent M consecutive sampling times, where m=1,2,3,…M, M is the total number of sampling times, and m is the time index of any time. Using the time index m as the independent variable and the dynamic priority score Fm as the dependent variable, perform linear fitting on the dynamic priority score sequence using the least squares method to obtain a linear function of the form Fm=C*m+B. The slope C of this linear function is the slope of the dynamic priority score change trend of the device; B is the preset intercept on the y-axis of this linear function. The slope of the dynamic priority score change trend of the device represents the pure rate of change of the device's abnormal urgency, and is independent of static importance.

[0096] A positive priority slope threshold and a negative priority slope threshold are set. A secondary judgment is made based on the slope of the dynamic priority score change trend. When the slope of the dynamic priority score change trend is greater than the set positive priority slope threshold, it indicates that the dynamic priority score is deteriorating rapidly, and the data is promoted to the high priority distribution channel.

[0097] When the slope of the dynamic priority score change trend is less than the set negative threshold of the priority slope, it indicates that the abnormal situation is mitigating itself, and it is safe to reduce its processing priority and adjust the data to the regular distribution channel.

[0098] When the slope of the dynamic priority score change trend is within the positive and negative thresholds of the priority slope, the surface data trend is stable and remains in the second priority distribution channel.

[0099] The setting of the positive priority slope threshold and the negative priority slope threshold is based on the principle of statistical significance test: During the normal operation of the equipment, the system collects a large number of dynamic priority score sequences, calculates the slope of the dynamic priority score change trend, and obtains the distribution of the historical dynamic priority score change trend slope under normal fluctuations; the 95th percentile of the normal historical dynamic priority score change trend slope distribution is taken as the positive priority slope threshold, and the 5th percentile is taken as the negative priority slope threshold, to ensure that the system only responds to statistically significant trend changes that exceed normal fluctuations;

[0100] S33. The system maintains the above three distribution channels and sends data in a strict priority order of high priority distribution channel > second priority distribution channel > normal distribution channel. Data in the high priority distribution channel is always sent first. Data in the second priority distribution channel is sent only when the high priority distribution channel is empty. Data in the normal distribution channel is processed last.

[0101] In this embodiment, the core is the proposed three-level hierarchical scheduling and distribution strategy of initial screening, observation, and fine judgment. This strategy first calculates a comprehensive confidence level based on the static importance and dynamic urgency of the data to achieve initial data diversion. Then, for the data with pending priority, the changing trend of its urgency is further analyzed to accurately identify potential risks that are deteriorating. Finally, through strict scheduling of high, medium, and low priority channels, system resources are intelligently and finely allocated to the data that needs them most, achieving accurate data distribution.

[0102] S4. Based on the business utility feedback of data distribution, dynamically adjust the distribution threshold to optimize the process, realize adaptive closed-loop optimization, and adjust the channel entry conditions according to the data distribution channel load.

[0103] S4 includes the following steps:

[0104] S41. The system continuously monitors the business effectiveness feedback generated by the data distributed through the S3 module; specifically:

[0105] For data entering the high-priority distribution channel, the system tracks the time from distribution to triggering a response from the backend system (such as monitoring alarms or automatic control). Ideally, high-priority data should trigger a rapid response.

[0106] The system records the frequency and distribution of data from various devices being classified as high priority over a period of time.

[0107] S42. Based on the monitoring data from S41, the system dynamically adjusts the distribution threshold and adjusts the entry conditions of the data distribution channel according to the load of the data distribution channel to optimize the process.

[0108] When the system detects that the high-priority distribution channel is overloaded for a long time (for example, more than 80% of the data is backlogged) and the current response time exceeds the preset response time, the system automatically raises the dynamic distribution threshold range and increases the standard for entering the high-priority distribution channel to ensure the instantaneous response capability of the most critical data.

[0109] When the system detects that a high-priority distribution channel has been idle for an extended period (e.g., utilization rate is below 20%), and critical alarms have an acceptable preset response time, the system automatically lowers the dynamic distribution threshold range, allowing more potentially important data to enter the priority channel and improving system sensitivity.

[0110] It should be noted that the preset response time is set based on the maximum acceptable response delay under the analysis of key business scenarios. For example, security alarms should respond within 100ms, and energy efficiency optimization instructions should respond within 500ms. The most stringent value among these business requirements is taken as the benchmark, and a 20% safety margin is reserved. That is, preset response time = business requirement response time * 0.8.

[0111] The automatic adjustment range of the distribution threshold is calculated based on the proportional-integral controller logic, combining the current instantaneous error with the historical cumulative error to achieve a balance between rapid response and long-term stability. The proportional adjustment component is calculated based on the relative error between the current response time and the preset response time, and this error is multiplied by a proportional weighting coefficient (e.g., 0.1) as the basic adjustment component. The integral adjustment component is calculated based on the sum of the continuous errors between the current response time and the preset response time over a historical period, and this sum of continuous errors is multiplied by an integral weighting coefficient (e.g., 0.01) as a supplementary component to correct long-term deviations.

[0112] The final adjustment range is the weighted sum of the proportional adjustment component and the integral adjustment component, wherein the weight coefficient of the proportional component is much larger than that of the integral component, to ensure that the system can respond quickly to emergencies and avoid over-adjustment due to small fluctuations.

[0113] The specific implementation of dynamic distribution threshold adjustment is as follows:

[0114] Floating conditions and operations: When the system detects that the current response time continues to exceed the preset response time and the data backlog rate of the high-priority distribution channel remains high (e.g., exceeding 5 sampling periods), the system will perform a floating of the distribution threshold. The new upper bound of the distribution threshold is increased by the entire calculated adjustment range on the basis of the original upper bound; while the new lower bound of the distribution threshold is increased by a smaller proportion on the basis of the original lower bound, so as to maintain the structure of the distribution threshold range and avoid excessive narrowing of the range.

[0115] Downgrading conditions and operations: When the system detects that the current response time is much lower than the preset response time, and the data backlog rate of the high-priority distribution channel remains low (e.g., exceeding 10 sampling periods), the system will perform a downgrading of the distribution threshold; the new upper bound of the distribution threshold is reduced by a small percentage from the original upper bound; the new lower bound of the distribution threshold is reduced by an even smaller percentage from the original lower bound; this asymmetric and cautious downgrading strategy aims to avoid the system standard from decreasing too quickly and to maintain the stability of the scheduling strategy.

[0116] In this embodiment, a closed-loop adaptive optimization mechanism based on business utility feedback is established. By continuously monitoring the system response time and channel backlog rate of high-priority data, and using this as a feedback signal, the dynamic distribution threshold range in the S3 module is dynamically adjusted. This adjustment process simulates the classic proportional-integral control logic, achieving a balance between response speed and long-term stability. This allows the entire data distribution system to evolve from a static, preset strategy into an intelligent system capable of self-calibrating according to actual operating conditions, thereby continuously ensuring the timeliness of critical data transmission under varying loads.

[0117] The above description is merely an example and illustration of the structure of the present invention. Those skilled in the art can make various modifications or additions to the specific embodiments described, or use similar methods to replace them, as long as they do not deviate from the structure of the invention or exceed the scope defined in the claims, all of which should fall within the protection scope of the present invention.

Claims

1. A method for an Internet of Things (IoT) data distribution engine, characterized in that, Includes the following steps; S1. Collect raw power data and equipment metadata of key process equipment in each workshop of the industrial zone in real time, and clean and standardize them to obtain a standard power data stream; S2. Based on standard power data streams, calculate the instantaneous power anomaly, current deviation, voltage deviation, power trend, energy efficiency health, and energy consumption anomaly of the equipment to form a power feature set; calculate dynamic priority scores based on the power feature set, and determine static importance factors according to the equipment level and business type in the equipment metadata; The dynamic priority scoring is analyzed as follows: The maximum value among instantaneous power anomaly, current deviation, and voltage deviation is used as the instantaneous anomaly level factor, energy efficiency health is used as the energy efficiency anomaly level factor, and the average value of power trend and energy consumption anomaly is used as the trend anomaly level factor. The dynamic priority score is obtained by comprehensively calculating the instantaneous anomaly level factor, energy efficiency anomaly level factor, and trend anomaly level factor. The static importance factor is specifically analyzed as follows: The weights of the device level and the business type are obtained according to the predefined mapping table. The weights of the device level and the business type are combined and the maximum value is taken by the aggregation function. The result of the aggregation function is the static importance factor. S3. Combine the static importance factor and dynamic priority score to calculate the comprehensive distribution confidence. Use a hierarchical scheduling strategy to divide the data to be distributed into high-priority data, data to be observed and regular data. Send the high-priority data, data to be observed and regular data to the high-priority distribution channel, the second-priority distribution channel and the regular distribution channel respectively. The hierarchical scheduling strategy is as follows: By jointly calculating static importance factors and dynamic priority scores, and combining dynamic threshold intervals and trend re-judgment mechanisms, high-priority data is processed in real time, data to be observed is further analyzed, and routine data is aggregated and sent with a delay, thereby achieving a dynamic balance between resource utilization and response efficiency. Further analysis of the data to be observed includes: The dynamic priority scores of the target device at several recent sampling times are obtained to form a dynamic priority score sequence. The dynamic priority score sequence is linearly fitted based on the least squares method to obtain a linear function, where the slope of the linear function is the slope of the dynamic priority score change trend. The slope of the dynamic priority score change trend is compared with the preset positive and negative priority slope thresholds to obtain the corresponding data distribution channels. The positive and negative priority slope thresholds are determined based on the slope distribution of the historical dynamic priority score change trend during normal equipment operation. The positive priority slope threshold is the 95th percentile of the slope distribution of the historical dynamic priority score change trend, and the negative priority slope threshold is the 5th percentile of the slope distribution of the historical dynamic priority score change trend. S4. Based on the business utility feedback of data distribution, dynamically adjust the distribution threshold to optimize the process, realize adaptive closed-loop optimization, and adjust the entry conditions of the data distribution channel according to the load of the data distribution channel.

2. The IoT data distribution engine method according to claim 1, characterized in that, The specific steps of S1 are as follows: A unique identifier is built for each device. Raw power data and device metadata are collected periodically or event-triggered through IoT gateways. Duplicate data removal, outlier identification, clock synchronization, and missing value filling are performed to output a standard power data stream with a unified timestamp, no duplicates, and no missing values. Device metadata includes device location, device level, and service type.

3. The IoT data distribution engine method according to claim 1, characterized in that, The specific calculation steps for the set of power characteristics are as follows: Based on standard power data streams, the instantaneous power anomaly and current deviation are calculated using the Z-Score method, and the voltage deviation is calculated using the percentage deviation relative to the rated voltage. Set a short-term sliding window, calculate the slope of the instantaneous power linear fitting within the window to obtain the power trend degree; calculate the normalized ratio of the average reactive power ratio to obtain the energy efficiency health degree; calculate the Z-Score of the average active power sequence relative to the historical baseline to obtain the energy consumption anomaly degree. The final output includes a set of power characteristics, including instantaneous power anomaly, current deviation, voltage deviation, power trend, energy efficiency health, and energy consumption anomaly.

4. The IoT data distribution engine method according to claim 1, characterized in that, The data distribution channel is specifically as follows: Data is sent in a strict priority order of high priority distribution channel > second priority distribution channel > normal distribution channel; data in the high priority distribution channel is always sent first, data in the second priority distribution channel is sent only when the high priority distribution channel is empty, and data in the normal distribution channel is processed last.

5. The IoT data distribution engine method according to claim 1, characterized in that, The adjustment of the entry conditions for the data distribution channel based on the data distribution channel load specifically includes: When the preset energy management system detects that the high-priority distribution channel has been overloaded for a long time and the current response time exceeds the preset response time, the energy management system automatically raises the dynamic distribution threshold range and increases the standard for entering the high-priority distribution channel to ensure the instantaneous response capability of the most critical data. When the energy management system detects that the high-priority distribution channel has been idle for a long time and that critical alarms have an acceptable preset response time, the energy management system automatically lowers the dynamic distribution threshold range, so that more potentially important data can enter the high-priority distribution channel and improve the sensitivity of the energy management system. The preset response time is set based on the maximum acceptable response latency under the analysis of key business scenarios.

6. The IoT data distribution engine method according to claim 5, characterized in that, The dynamic adjustment of the distribution threshold specifically refers to: The automatic adjustment of the distribution threshold is implemented based on the proportional-integral controller logic, which calculates the adjustment range by combining the current instantaneous error with the historical cumulative error; the proportional component is calculated based on the error between the current response time and the preset response time, and is used to adjust the distribution threshold in real time. The integral component is calculated based on the cumulative error of a pre-defined historical response period and is used to correct long-term deviations. The final distribution threshold adjustment is the weighted sum of the proportional and integral components, with the proportional component having a greater weight than the integral component, to ensure that the energy management system achieves a balance between rapid response and long-term stability.

Images