A data complexity evaluation method, system, electronic device, and medium
By calculating multi-dimensional data features in industrial edge computing, constructing a comprehensive complexity index and quantifying abnormal feature vectors, the problem of imprecise resource scheduling in existing technologies is solved, achieving efficient allocation of computing resources and low-power data processing.
Patent Information
- Application Number
- CN202611313858.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-27
- Publication Date
- 2026-11-17
AI Technical Summary
Existing technologies cannot accurately capture the abrupt changes, trend changes, and coupling anomalies between multiple parameters in industrial edge computing scenarios, resulting in imprecise resource scheduling, wasted computing power, and high power consumption.
By acquiring the data to be evaluated, multi-dimensional data including data mutation degree, trend complexity and multi-parameter coupling anomaly degree are calculated, a comprehensive complexity index is constructed, hierarchical decision-making is carried out, and anomaly feature vectors are quantified to dynamically allocate heterogeneous computing resources.
It achieves precise interception and diversion of data events, reduces computing power waste, lowers system power consumption, and improves response efficiency and model matching accuracy.
Smart Images

Figure CN123116010A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of electronics, and more particularly to a data complexity assessment method, system, electronic device, and medium. Background Technology
[0002] With the rapid development of the Internet of Things (IoT) in water conservancy and water affairs, massive amounts of field instrument monitoring data have been generated in industrial edge computing scenarios. To achieve real-time monitoring and early warning of anomalies in pipeline network operations, computing resources are typically deployed on edge devices to process the data. Data complexity assessment refers to determining the complexity of current data events by extracting and quantifying features from the collected multidimensional monitoring data, thereby providing a basis for subsequent computing resource scheduling and anomaly pattern recognition.
[0003] In existing technologies, in industrial edge computing scenarios, to address the challenges of massive data processing and computing resource allocation, it is impossible to accurately capture the abrupt changes, trend shifts, and coupling anomalies between multiple parameters, leading to misjudgments of complex water data events and hindering truly refined resource scheduling. Furthermore, because resource scheduling granularity remains at the "node level" (i.e., between edge devices and the cloud), it lacks a micro-scheduling mechanism for heterogeneous processors within edge devices (such as microprocessors and neural network processors). This results in industrial communication adapters equipped with neural network processors either sending all data to the neural network processor for full analysis, causing extreme waste of computing power and continuous high power consumption, or having everything executed by the microprocessor, leading to untimely processing of complex tasks. Summary of the Invention
[0004] In view of the shortcomings of the prior art described above, the present invention provides a data complexity evaluation method, system, electronic device and medium to solve the above technical problems.
[0005] Firstly, this application provides a data complexity assessment method applied to industrial communication adapters, comprising: acquiring data to be assessed, the data to be assessed including monitoring parameters; calculating multi-dimensional data based on the data to be assessed to obtain a comprehensive complexity index, the multi-dimensional data including data mutation degree, trend complexity, and multi-parameter coupling anomaly degree; performing hierarchical decision-making based on the comprehensive complexity index, the hierarchical decision-making including matching the comprehensive complexity index with a preset complexity threshold, different complexity thresholds corresponding to different levels of processing strategies; quantifying the data to be assessed to obtain an anomaly feature vector used to characterize the anomaly contribution of each monitoring parameter, and performing data complexity assessment based on the anomaly feature vector and the pre-configured weights of the multi-dimensional data corresponding to each monitoring parameter to determine the final processing strategy.
[0006] Optionally, the data mutation rate is obtained based on the degree of deviation between the current data and the baseline of historical data; the formula is as follows: F1=
[0007] Where F1 represents the data mutation degree, The sampled value at the current moment. This is the historical baseline mean. The historical baseline standard deviation It is a very small constant.
[0008] Optionally, the trend complexity is obtained based on the density of trend inflection points in the data to be evaluated within a time window.
[0009] Optionally, the multi-parameter coupling anomaly degree is obtained based on the synergistic relationship between the interrelated parameters; the formula is as follows:
[0010] in, For multi-parameter coupling anomaly degree, This represents the actual correlation coefficient between parameters i and j within the current window. The standard correlation coefficient is the historical correlation coefficient under normal operating conditions. Weights for different parameter pairs.
[0011] Optionally, the data complexity assessment based on the abnormal feature vector and the weights of the multi-dimensional data corresponding to each pre-configured parameter includes: calculating the abnormal contribution components of each monitoring parameter in the three dimensions of data mutation degree, trend complexity, and multi-parameter coupling anomaly degree; performing a weighted summation of the abnormal contribution components in the three dimensions to obtain the abnormal contribution of each monitoring parameter, and constructing a parameter-level abnormal feature vector; calculating the similarity between the current abnormal feature vector and the feature vector template in the pre-constructed abnormal pattern library using the reciprocal of the weighted Euclidean distance, and performing abnormal pattern matching based on the similarity results.
[0012] Optionally, after performing abnormal pattern matching based on the similarity results, the method further includes: selecting a corresponding analysis model based on the matching results; if the similarity with each abnormal pattern in the abnormal pattern library is less than a preset first threshold, it is determined to be an unknown abnormality, and a full analysis model is used for analysis; if the similarity with an abnormal pattern in the abnormal pattern library is greater than a preset multiple of the similarity with other abnormal patterns, the analysis model corresponding to the abnormal pattern is used for analysis; if the similarity with multiple abnormal patterns in the abnormal pattern library is greater than a preset second threshold, the union of the sets of analysis models corresponding to the multiple abnormal patterns is used for analysis.
[0013] Optionally, historical data statistical features are periodically distributed from the cloud to the edge, and the edge adjusts the weights of data mutation degree, trend complexity, and multi-parameter coupling anomaly degree according to the historical data statistical features.
[0014] Secondly, this application provides a data complexity assessment system applied to an industrial communication adapter, comprising: a data acquisition module for acquiring data to be assessed, the data including monitoring parameters; a complexity calculation module for calculating multi-dimensional data based on the data to be assessed to obtain a comprehensive complexity index, the multi-dimensional data including data mutation degree, trend complexity, and multi-parameter coupling anomaly degree; a hierarchical decision-making module for making hierarchical decisions based on the comprehensive complexity index, the hierarchical decision-making including matching the comprehensive complexity index with a preset complexity threshold, different complexity thresholds corresponding to different levels of processing strategies; an anomaly feature vector module for quantifying the data to be assessed to obtain an anomaly feature vector characterizing the anomaly contribution of each monitoring parameter; and an assessment module for assessing data complexity based on the anomaly feature vector and the pre-configured weights of the multi-dimensional data corresponding to each monitoring parameter, to determine the final processing strategy.
[0015] Thirdly, this application provides an electronic device, including: one or more processors; and a storage device for storing one or more programs, which, when executed by one or more processors, cause the electronic device to implement the data complexity evaluation method as described in any optional embodiment of the first aspect.
[0016] Fourthly, this application provides a computer-readable storage medium storing computer-readable instructions that, when executed by a computer's processor, cause the computer to perform the data complexity evaluation method as described in any optional embodiment of the first aspect.
[0017] The beneficial effects of this invention are as follows: The data complexity assessment method provided in this application acquires data to be assessed containing monitoring parameters, and calculates three multi-dimensional data based on this data: data mutation degree, trend complexity, and multi-parameter coupling anomaly degree. This comprehensively and accurately characterizes the complex features of the data, thereby obtaining a comprehensive complexity index that objectively reflects the overall complexity of the data. Furthermore, based on this comprehensive complexity index, hierarchical decision-making is performed, matching data events of different complexity to different levels of processing strategies, enabling the system to effectively intercept and divert data in the early stages. For high-complexity data events, anomaly feature vectors representing the abnormal contribution of each monitoring parameter are obtained by quantifying the data to be assessed, and data complexity is assessed by combining pre-configured weights, achieving fine-grained decomposition and precise location of the anomaly source. Through the multi-dimensional data complexity quantification and evaluation mechanism in this application, the industrial communication adapter can proactively perceive the complexity of data events, thereby dynamically and optimally allocating internal heterogeneous computing resources. This reduces the waste of computing power caused by indiscriminately sending all data into high-power analysis models, lowers the overall power consumption of the system, and improves the response efficiency to complex abnormal events and the accuracy of model matching. Attached Figure Description
[0018] Figure 1 This is a flowchart illustrating the data complexity evaluation method in an embodiment of the present invention.
[0019] Figure 2 This is a schematic diagram illustrating the specific workflow of the data complexity evaluation method in this embodiment of the invention. Detailed Implementation
[0020] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, unless otherwise specified, the following embodiments and features described therein can be combined with each other.
[0021] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.
[0022] Before delving into the specific implementation methods of this application, we first analyze the technical application scenarios of this application and the technical deficiencies existing in the prior art. The technical solution of this application is mainly applied to the edge computing scenario of the Internet of Things (IoT) in water conservancy and water affairs. In the modern industrial IoT architecture, industrial communication adapters (i.e., edge computing gateways) undertake the tasks of collecting, preprocessing, and edge analyzing massive amounts of field instrument data. With the popularization of heterogeneous computing architectures, industrial communication adapters typically integrate microprocessors (MPUs) that are adept at conventional logic control and lightweight computing, as well as neural network processors (NPUs) that are adept at deep learning inference and high-concurrency matrix operations.
[0023] Existing technologies often face the problem of rigid allocation of computing resources when handling complex calculations such as flow statistics and water quality analysis in water pipe networks. The root cause of this technical problem is that existing edge computing systems lack the ability to proactively perceive and multi-dimensionally quantify the "complexity" of input data events. This leads the system into two extreme processing dilemmas: the first is relying solely on microprocessors for fixed threshold judgments. This approach cannot capture complex trend changes in water pipe networks (such as slow pipe leakage) or hidden coupling anomalies of multiple parameters, resulting in frequent missed reports and untimely responses. The second is indiscriminately feeding all sliding window data into the neural network processor to run a massive artificial intelligence analysis model. Since water data is stable and normal under most steady-state conditions, this indiscriminate full analysis directly causes the neural network processor to operate under high load for a long time, resulting not only in a huge waste of computing power but also in causing excessive power consumption, severe overheating, and even system crashes in edge devices.
[0024] To address the inherent trade-off between "timely response" and "power consumption and computing power control," this application provides a data complexity assessment method. This method innovatively introduces a pre-emptive multi-dimensional data complexity quantification and assessment mechanism. Through statistical feature calculations with extremely low computing power consumption, it accurately identifies the complexity of the current data event, and then dynamically determines whether to wake up the neural network processor, and which specific analysis model to execute after waking it up. This approach effectively reduces the computing power waste caused by indiscriminately feeding all data into high-power analysis models, achieving dynamic optimal allocation of heterogeneous computing resources and intelligent and accurate matching of analysis models.
[0025] In the following description, numerous details are explored to provide a more thorough explanation of embodiments of the invention. However, it will be apparent to those skilled in the art that embodiments of the invention may be practiced without these specific details. In other embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring embodiments of the invention.
[0026] like Figure 1As shown, the data complexity evaluation method in this embodiment includes the following steps: S110. Obtain the data to be evaluated, which includes monitoring parameters.
[0027] In one embodiment, the data to be evaluated can be acquired in real time and continuously using field instruments. The data to be evaluated includes multiple monitoring parameters, such as instantaneous flow rate, instantaneous pressure, pH value, turbidity, and residual chlorine. Furthermore, a microprocessor can perform data preprocessing to deduplicate the raw sampled data, remove outliers, and fill in missing values, outputting standardized sliding window data. This data preprocessing ensures the accuracy of subsequent complexity assessments.
[0028] S120. Calculate multi-dimensional data based on the data to be evaluated to obtain a comprehensive complexity index. The multi-dimensional data includes data mutation degree, trend complexity, and multi-parameter coupling anomaly degree.
[0029] In one embodiment, a microprocessor can perform complex quantifier calculations to compute multi-dimensional data based on the data to be evaluated. To comprehensively characterize the complex features of water data and overcome the limitations of single-dimensional identification, the multi-dimensional data includes data variability, trend complexity, and multi-parameter coupling anomaly. Data variability measures the deviation of current data from historical baselines to capture transient and drastic changes; trend complexity measures whether the data's change pattern within a time window is abnormal to distinguish between stable fluctuations and high-frequency oscillations; and multi-parameter coupling anomaly assesses the physical correlation characteristics of water scenarios to determine whether the collaborative logic between parameters such as flow and pressure has been disrupted. These three dimensions, from the perspectives of single-parameter static, single-parameter dynamic, and multi-parameter correlation, respectively, constitute a rigorous data feature profile.
[0030] In one embodiment, after obtaining multi-dimensional data, a comprehensive complexity index can be calculated using a microprocessor. Specifically, the calculation results of the three dimensions can be normalized and then weighted and summed to output a quantified comprehensive complexity index. The comprehensive complexity index can intuitively reflect the overall complexity of the current data window.
[0031] S130. Make hierarchical decisions based on the comprehensive complexity index. The hierarchical decision includes matching the comprehensive complexity index with a preset complexity threshold. Different complexity thresholds correspond to different levels of processing strategies.
[0032] In this embodiment, the hierarchical decision-making process includes matching the overall complexity index with a preset complexity threshold, with different complexity thresholds corresponding to different levels of processing strategies. The microprocessor can classify data events into three levels—low, medium, and high / urgent—based on the overall complexity index. For low-complexity events, only caching can be performed, without reporting to the cloud or triggering deep analysis; the neural network processor remains in deep sleep. For medium-complexity events, threshold detection can be performed; in the threshold-exceeding branch, if it is determined to be yes, a local alarm is triggered (medium complexity). Only when the overall complexity index reaches the high / urgent level is the deep analysis link initiated. This hierarchical decision-making mechanism ensures that the neural network processor is only awakened when necessary, significantly reducing the system's average power consumption.
[0033] S140. Quantify the data to be evaluated to obtain an anomaly feature vector that characterizes the anomaly contribution of each monitoring parameter. Based on the anomaly feature vector and the weights of the multi-dimensional data corresponding to each monitoring parameter that are pre-configured, evaluate the data complexity to determine the final processing strategy.
[0034] In one embodiment, the data to be evaluated is quantized to obtain an anomaly feature vector. Based on the anomaly contribution of each monitoring parameter in the anomaly feature vector and the pre-configured weights of each monitoring parameter in three dimensions, when the complexity is determined to be high / urgent, the microprocessor executes the construction of an anomaly feature vector, quantizes the data to be evaluated, and obtains an anomaly feature vector used to characterize the anomaly contribution of each monitoring parameter.
[0035] In this embodiment, while the overall complexity index can determine the overall complexity, it cannot pinpoint which specific parameter caused the complexity. By decomposing the source of complexity to each specific monitoring parameter, a parameter-level anomaly feature vector is constructed, enabling precise location of the anomaly source. Based on the anomaly feature vector and the pre-configured weights of multi-dimensional data corresponding to each monitoring parameter, data complexity is evaluated to determine the final processing strategy.
[0036] In this embodiment, the microprocessor can perform a matching process with an anomaly pattern library, comparing the real-time constructed anomaly feature vectors with the preset anomaly pattern library for similarity. In the branch determining whether a matching pattern exists, if a match is successful, the corresponding analysis model is selected; if the match fails, a full analysis is initiated. Subsequently, the microprocessor activates the NPU running model, sending activation instructions and essential core data to the dormant neural network processor. The neural network processor is awakened and loads the selected analysis model for deep inference. In the branch determining whether the analysis result is abnormal, if an anomaly is confirmed, a local alarm (high complexity) is triggered and the data is reported to the cloud. The cloud platform performs cloud-based evolutionary learning, clustering analysis on the reported data, optimizing the pattern library, and then updating it as needed.
[0037] In one implementation, to accurately measure the deviation of current data from historical normal conditions, the data variability is obtained based on the deviation of current data from the baseline of historical data. Specifically, the formula for calculating the data variability is as follows: F1= Equation (1) Where F1 represents the data mutation degree, This refers to the sampled value at the current moment (such as instantaneous flow rate or pH value). This is the historical baseline mean (e.g., the mean of samples taken at the same time in the previous 7 days). The historical baseline standard deviation It is a very small constant to prevent division by zero.
[0038] In this embodiment, the data mutation rate (F1) can measure the degree of deviation of the current data from the historical baseline (such as the deviation from the average value of the sampled values of the same period of the previous 7 days). When the flow rate suddenly exceeds the normal fluctuation range, it may indicate a pipe burst or equipment abnormality, which increases the complexity.
[0039] Regarding the calculation mechanism for the aforementioned data abrupt changes, in the actual operation of water conservancy and waterworks networks, monitoring parameters such as instantaneous flow and network pressure typically fluctuate around a relatively stable baseline under normal operating conditions. However, in emergency situations such as main pipeline bursts or sensor hardware failures, the current sampling data can instantly deviate significantly from its normal historical fluctuation range. If only absolute thresholds are used for judgment, it is easily affected by seasonal water consumption changes or peak morning and evening water usage, leading to false alarms. In the above formula, the numerator calculates the absolute deviation between the current sampling value and the historical baseline mean. The historical baseline mean is dynamically maintained by the system; for example, it is taken as the arithmetic mean of the sampling values from the same period of the previous 7 days, thus adaptively absorbing the periodic patterns of network operation. The denominator is the historical baseline standard deviation, representing the normal fluctuation range of the parameter within the same historical period. Dividing the absolute deviation by the standard deviation is essentially a standardization process. This method assesses the relative drasticness of data change. If a parameter typically fluctuates very little, even a small absolute deviation will result in a significantly increased abrupt change, thus enabling the system to sensitively detect minute anomalies. Conversely, if a parameter typically fluctuates drastically, the system's tolerance for its absolute deviation will increase accordingly, reducing the probability of false alarms. Furthermore, the inclusion of a very small constant in the denominator effectively avoids the problem of division-by-zero crashes when historical data is completely stationary, leading to a zero standard deviation. This improves the long-term stability of the algorithm on edge devices. The formula for calculating the abrupt change eliminates the differences between different units of measurement, providing a reliable static single-parameter dimension for identifying sudden anomalies.
[0040] In one embodiment, in addition to considering the static abrupt changes in data at a given moment, it is also necessary to characterize the complexity of the data from the perspective of single-parameter dynamic evolution. Trend complexity is obtained based on the density of trend inflection points in the data to be evaluated within a time window. It can be calculated using the following formula: F2 = Hearst exponent × trend inflection point density (2) Where F2 is the trend complexity.
[0041] In actual monitoring using the Internet of Things (IoT) for water affairs, the changing patterns of data sequences contain rich information about operating conditions. For example, a slow and continuous rise in the water tank level is a normal unilateral trend, while frequent and drastic fluctuations in pipeline pressure may be due to abnormal operating conditions caused by pump cavitation or water hammer effects in the pipeline. Traditional threshold detection methods struggle to distinguish between these two states. This embodiment uses the density of trend inflection points as the core indicator for evaluating trend complexity. The Hearst exponent in this embodiment is based on the rescaled range (R / S) analysis method. For a target window length n, several typical H values (such as 0.1, 0.2, ..., 0.9) are taken at fixed steps within the Hearst exponent range of 0.1 to 0.9. For each H value, a fractal Brownian motion model is used to generate 100,000 time series of length n. During the generation process, common operating characteristic constraints in the water industry are adopted, including the normal fluctuation range of parameters, fluctuation amplitude, sampling interval, etc., to ensure that the statistical characteristics of the simulated sequence are consistent with those of real water monitoring data. For each set of simulation sequences, calculate its R / S value. Then, take the arithmetic mean of the R / S values of 100,000 sequences corresponding to the same H value to obtain the standard average R / S value of that H value under the current window length n. Organize all H values and their corresponding standard average R / S values into a mapping table. The mapping table structure is as follows, where each set of brackets represents a mapping point, and it is pre-built into the device. [(R / S1,H1),(R / S2,H2),(R / S3,H3),…,(R / S n H n )] In actual operation, the corresponding mapping table can be selected according to the current sliding window length n, and then the real-time calculated R / S value can be matched with the standard average P / S value in the table. The Hearst exponent value of the current sequence can be obtained by looking up the nearest table.
[0042] In this embodiment, the calculation process of trend inflection point density is as follows: First, the microprocessor performs a first-order difference operation on the time series data of length n within the current sliding window to generate a difference sequence. This step calculates the amount and direction of change between two adjacent sampling points. Subsequently, the algorithm traverses the difference sequence and counts the number of times the difference sign changes, T. The sign change is either a change in the difference value from positive to negative (representing a change in the data sequence from rising to falling, forming a local peak) or a change in the difference value from negative to positive (representing a change in the data sequence from falling to rising, forming a local trough). Finally, the number of sign changes is divided by the maximum number of data points within the window that may cause a turning point, to obtain the normalized trend inflection point density, calculated using the formula D = T / (n-2), where D is the inflection point density. Specifically, the first-order difference sequence ΔX is calculated for the time series within the window. k = x k+1 -x k .
[0043] In this embodiment, the value of the trend inflection point density ranges between [0,1]. If the value is low, it indicates that the data within the sliding window exhibits a stable unilateral trend or remains constant, which is a low-complexity state; if the value is high, it means that the data has frequently reversed up and down within a very short time window, reflecting severe turbulence in the water flow in the physical pipe network or serious interference with sensor signals, which is a high-complexity abnormal condition.
[0044] In one embodiment, in addition to the dimensions described above, the microprocessor also needs to examine the system's operating state from a global perspective of multi-parameter correlation. The multi-parameter coupling anomaly degree is obtained based on the cooperative relationship between interrelated parameters.
[0045] Equation (3) in, For multi-parameter coupling anomaly degree, This represents the actual correlation coefficient between parameters i and j within the current window. The standard correlation coefficient is the historical correlation coefficient under normal operating conditions. Weights for different parameter pairs (such as flow-pressure parameter pairs, flow-turbidity parameter pairs, which can be adjusted in the cloud).
[0046] In this embodiment, parameter coupling anomaly can be used to determine the logical consistency between related parameter coefficients in a water utility scenario, identify the intrinsic relationship between parameters such as flow rate, pressure, pH, and turbidity, and determine whether the synergistic relationship between parameters is normal. Under normal circumstances, pressure should decrease when flow rate increases. If an abnormal coupling occurs where both flow rate and pressure increase, it may indicate sensor malfunction or changes in the pipeline topology, representing high complexity. In complex municipal water utility networks, various physical monitoring parameters follow fluid dynamics or water chemistry laws, exhibiting inherent logical consistency and synergistic relationships. For example, within the same pipe segment, when the instantaneous flow rate increases significantly, the instantaneous pressure at that point should typically show a decreasing trend, with the two exhibiting a negative correlation. If a phenomenon of a sharp increase in flow rate and an abnormal surge in pressure occurs during actual operation, it often signifies a systemic, hidden fault such as sensor probe damage or illegal alteration of the pipeline topology. If only threshold detection is performed on a single parameter, it is easy to cause missed detections. The approach described in this embodiment can transform physical coordination relationships into quantifiable mathematical indicators, enabling the system to identify complex operating conditions where individual parameters appear normal but their combination is abnormal, thus achieving in-depth identification of pipeline topology changes or sensor group failures.
[0047] In one embodiment, the three dimensions are normalized and then weighted and summed: Equation (4) The sigmoid function normalizes the calculation results of three dimensions to the 0-1 interval, eliminating the dimensional differences between different dimensions. The function formula is: Equation (5) Among them, α, β, and γ are configurable weights with initial default values of α=0.4, β=0.3, and γ=0.3. They can be dynamically adjusted according to the scenario (e.g., if the proportion of F1 anomalies increases based on the historical average of the site over the past 7 days, the weights of α, β, and γ will be increased).
[0048] In one embodiment, after the industrial communication adapter determines that the current data event is of high / urgent complexity, since the overall complexity index is only a macroscopic scalar, it is impossible to determine which parameter caused the anomaly. For example, even though it is a high-complexity event, it may be a leakage warning caused by a sudden increase in flow or a pollution warning caused by water quality deterioration. Different causes require different analysis models to be activated.
[0049] Table 1 shows the grading and processing examples for typical scenarios of different complexities:
[0050] Table 1 like Figure 2As shown, in one embodiment, the MPU is the main control processor of the industrial communication adapter, and the NPU is an on-demand activated coprocessor. The specific rules for scheduling the computing resources of the two are as follows: The NPU is in a low-power sleep state by default, which is triggered by the MPU based on the value of the comprehensive complexity index C. When C≥0.6, the MPU sends an activation command to the NPU. After the task is completed, the NPU automatically goes into sleep mode. When the MPU transmits data to the NPU, it only transmits core necessary data such as the abnormal feature vector E, the matched pattern ID, and the key sampled value of the current window, and does not transmit the original full data. After the NPU completes the analysis model, it only feeds back the core results such as analysis conclusions, anomaly levels, and handling suggestions to the MPU, which then performs local alarms, cloud reporting, data caching, and other operations. When a multi-mode fusion is matched, the NPU runs and analyzes the models in sequence according to their priority, which is set by the recognition accuracy of the models in the mode library.
[0051] In one embodiment, the overall complexity index C reflects the overall complexity of the current data event and is used for hierarchical decision-making. However, it cannot represent the contribution of each of the three dimensions of parameters, thus making it impossible to determine the corresponding analysis model. For example, a high-complexity event with C=0.6 might be a leakage warning caused by a sudden increase in flow or a pollution warning caused by water quality deterioration. Warnings caused by different reasons require the activation of different analysis models. To address this, the present invention further constructs an anomaly feature vector E to represent the contribution of anomalies, decomposing the source of complexity to each monitoring parameter and realizing the location of the anomaly source (parameter). The overall complexity C and the anomaly feature vector E together constitute a complete decision-making chain from anomaly detection to model scheduling.
[0052] The abnormal feature vector E is composed of the abnormal contribution e of all monitored parameters (such as instantaneous flow rate, instantaneous pressure, pH, turbidity, residual chlorine, etc.). i Composition, used to describe the distribution of anomaly sources in the current window: E = [e1,e2,...,e n ] in, This represents the abnormal contribution of the i-th monitoring parameter.
[0053] in , , The weights are for the three dimensions, with initial default values. =0.5、 =0.3、 =0.2, which can be dynamically adjusted according to the scenario. : Represents the contribution of parameter i to the dimension of data mutation rate. If A value close to 1 indicates that the anomaly of the current window is mainly caused by a sudden change in parameter i.
[0054] Equation (6) in, The sampled value of parameter i at the current moment (real-time acquisition). The historical baseline mean of parameter i (updated daily). The historical baseline standard deviation of parameter i (updated daily). This is the sum of the deviations of all parameters, used for normalization.
[0055] In one embodiment, during the anomaly feature vector construction stage, the microprocessor decomposes the macroscopic complexity into each microscopic monitoring parameter.
[0056] This represents the contribution of parameter i to the trend complexity dimension. If A value close to 1 indicates that the anomaly in the current window is mainly caused by the abnormal fluctuation pattern of parameter i.
[0057] Equation (7) in, Let be the Hurst exponent of parameter i, derived from the time series of i using a simplified Hurst algorithm. Let be the inflection point density of parameter i, which is obtained from the time series of i using a differential change detection algorithm. The summation of the trend complexity of all parameters is used for normalization.
[0058] This represents the contribution of parameter i to the multi-parameter coupling anomaly dimension. If A value close to 1 indicates a significant change in the coupling relationship between this parameter and other parameters, which may be a source of failure.
[0059] Equation (8) in, The weights of parameter pairs (i,j) characterize the importance of the coupling relationship. This represents the actual correlation coefficients (calculated in real time) of parameters i and j within the current window. The historical standard correlation coefficients of parameters i and j (representing the coupling relationship under normal operating conditions) are given. The weighted sum of the coupling deviations of parameter i with all other parameters. The weights of the parameter pair (k,j) are related to w in the molecule. ij The meaning is the same, only the index is different. Let (k,j) be the actual correlation coefficient between the parameter pairs. Let (k,j) be the standard correlation coefficient of the parameter pair. The weighted sum of the coupling biases of all parameter pairs including k and j is used for normalization.
[0060] In one embodiment, the non-volatile memory of the industrial communication adapter contains a pre-stored abnormal pattern library, as shown in Table 2, wherein the feature vector template T k =[t k1 ,t k2 ,...,t kn ] represents the typical anomaly contribution distribution of each monitoring parameter under the k-th mode; the set of analysis models M k This represents the list of deep analysis models that need to be executed when pattern k is identified.
[0061]
[0062] Table 2 When the abnormal feature vector E=[e1,e2,...,e] of the current window is obtained n After that, calculate E and each pattern template T. k The similarity between E and a certain template T is... k The higher the similarity value, the more likely the current working condition is to be that T. k The more typical the scenario, the more likely the set of analytical models M corresponding to this pattern will be adopted. k This invention uses the reciprocal of the weighted Euclidean distance as a similarity metric.
[0063] Equation (9) in, The weight of parameter i can be preset according to the importance of the parameter (e.g., the weight of flow rate and pressure is higher than that of other parameters); add 1 to the denominator to avoid division by zero and make the similarity range between (0,1].
[0064] In one embodiment, based on the abnormal pattern similarity results, the matching analysis model can be performed according to the following process rules: (1) No matching pattern: If the similarity with all patterns is less than the threshold (default value 0.3), it is considered an "unknown anomaly". In this case, the full analysis model is used, and the current E vector is reported to the cloud for subsequent expansion of the pattern library. Otherwise, proceed to step 2).
[0065] (2) Single best match: If there exists a pattern k* with a significantly higher similarity than other patterns (e.g., greater than the default value of 1.5 times for the second highest similarity), then the analysis model set Mk* corresponding to that pattern is used. Otherwise, proceed to step 3).
[0066] (3) Multi-mode fusion: If there are multiple modes with similarity close to each other (e.g., the difference is less than the default value of 0.1), then the current anomaly is considered to be a mixed mode, and the union of the analysis model sets corresponding to these modes is adopted. Otherwise, proceed to step 4).
[0067] (4) Default matching: The analysis model corresponding to the highest similarity is used. If there are multiple models with the same highest similarity, the first model in the analysis model library is used by default.
[0068] In the above steps, step (1) is a fallback defense mechanism. When the similarity is less than the preset first threshold (e.g., the default value of 0.3), it indicates that the current abnormal event characteristics do not match the known typical working conditions. The system determines it to be an unknown abnormality, wakes up the neural network processor to load all deep analysis models for a comprehensive investigation, in order to prevent the omission of potential hidden dangers. At the same time, the microprocessor will report the feature vector of the unknown abnormality to the cloud for cloud evolutionary learning to expand the future pattern library. Step (2) is a precision mechanism. When the similarity of a certain abnormal pattern is significantly greater than that of other patterns (e.g., the highest similarity is more than 1.5 times the default value of the second highest similarity), it indicates that the current working condition has distinct characteristics and the abnormal source is clear. The system only activates the neural network processor to run the specific set of analysis models corresponding to the single abnormal pattern, saving computing power and response time to the greatest extent. Step (3) is a collaborative analysis mechanism. When the similarity of multiple abnormal patterns is high and they are close to each other, it usually occurs in a complex chain failure scenario, presenting mixed failure characteristics. The system extracts the union of the analysis model sets corresponding to these high-similarity patterns and passes it to the neural network processor for execution, ensuring the comprehensiveness of the analysis perspective and preventing misjudgments due to insufficient coverage of a single model. Through the above multi-level decision branches, based on the statistical characteristics of the similarity distribution, it dynamically and intelligently switches between saving computing power, comprehensive investigation, and security backup, taking into account the comprehensive analysis of complex mixed faults while ensuring the ability to capture unknown new anomalies.
[0069] In one embodiment, to address the issue that fixed-weighted complexity assessments at the edge device are prone to false alarms or missed alarms due to seasonal variations or shifts in operating conditions, a dynamic compensation mechanism involving cloud-edge collaboration can be employed. Specifically, historical data statistical features are periodically distributed from the cloud to the edge device, and the edge device adjusts the weights of data mutation rate, trend complexity, and multi-parameter coupling anomaly based on these historical data statistical features.
[0070] In the actual operation of water supply networks, operational characteristics can drift with seasonal changes or shifts in the surrounding water usage structure. If edge devices consistently use fixed weights to evaluate data complexity, false alarms or missed alarms are likely to occur when operational conditions change. The cloud platform continuously receives operational data reported by various edge sites and periodically performs statistical feature mining on historical data. The cloud then distributes the latest historical data statistical features extracted to the corresponding edge industrial communication adapters. Upon receiving the statistical features, the edge device autonomously triggers adaptive adjustment logic for complexity weights. For example, if the statistical features indicate that a pumping station node has experienced frequent compliant high-frequency pressure fluctuations over the past week, the edge device will automatically increase the weight of the trend complexity dimension, enhancing the system's sensitivity to abnormal trend patterns.
[0071] For newly installed industrial communication adapters, due to the lack of accumulated historical data, reliable baseline means and standard deviations cannot be calculated. In this case, the equipment uses factory-preset parameters from a general water management model as the initial baseline. These parameters are statistically derived from typical municipal water management operating characteristics (such as the normal fluctuation range of flow rate, pressure, and water quality), ensuring effective complexity assessment during the initial installation phase. After 24 hours of operation, the system automatically switches to a dynamic baseline calculated based on local actual sampling data, achieving a gradual improvement in assessment accuracy.
[0072] To ensure the stable operation of the above complexity assessment method under the limited computing power of edge industrial communication adapters, all core algorithms in this embodiment adopt a lightweight design, and their computational complexity is as follows: (1) Complexity quantification algorithm (calculation and weighted fusion of data mutation degree, trend complexity, multi-parameter coupling anomaly degree): time complexity O(n), space complexity O(1), where n is the number of sampling points in the sliding window.
[0073] (2) Algorithm for constructing abnormal feature vectors (calculation and normalization of the abnormal contribution components of each parameter): time complexity O(n), space complexity O(n), where n is the number of monitoring parameters.
[0074] (3) Abnormal pattern matching algorithm (similarity calculation and sorting of abnormal feature vectors and templates in the pattern library): time complexity O(m×n), space complexity O(m×n), where m is the number of preset patterns in the pattern library and n is the number of monitoring parameters.
[0075] The above algorithms are all implemented on a microprocessor (MPU), without activating a neural network processor (NPU), ensuring extremely low computational overhead before hierarchical decision-making and not affecting the real-time performance of the main control task. Based on the same inventive concept, this application also provides a data complexity evaluation system. This system is applied to industrial communication adapters and is a system-level implementation of the aforementioned data complexity evaluation method. The system includes: The data acquisition module is used to acquire the data to be evaluated, including monitoring parameters. This module is responsible for interfacing with various sensors in the field using low-level protocols, acquiring and preprocessing the data to provide a standard data stream for subsequent analysis.
[0076] The complexity calculation module is used to calculate multi-dimensional data based on the data to be evaluated, and obtain a comprehensive complexity index. The multi-dimensional data includes data mutation degree, trend complexity, and multi-parameter coupling anomaly degree. This module is deployed on a low-power microprocessor, and completes the preliminary assessment of the complexity of data events with relatively small computing power overhead.
[0077] The hierarchical decision-making module is used to make hierarchical decisions based on the comprehensive complexity index. This involves matching the comprehensive complexity index with preset complexity thresholds, with different thresholds corresponding to different levels of processing strategies. This module acts as a scheduling hub, directly routing low-complexity data to the local cache without waking the neural network processor, thus controlling unnecessary power consumption at the system architecture level.
[0078] The anomaly feature vector module is used to quantify the data to be evaluated, obtaining anomaly feature vectors that characterize the anomaly contribution of each monitoring parameter. This module breaks down the macroscopic complexity into microscopic monitoring parameters.
[0079] The evaluation module assesses data complexity based on the anomaly feature vectors and the weights of multi-dimensional data corresponding to each pre-configured monitoring parameter, thereby determining the final processing strategy. This module uses a distance algorithm to compare real-time feature vectors with a pre-set pattern library, decides on an analysis model suitable for the current operating conditions, and sends activation instructions and core data to the neural network processor.
[0080] This system architecture achieves coarse-grained interception and fine-grained precise allocation of computing power through an interactive control mechanism that integrates microprocessor main control and neural network processor on-demand co-processing. This provides the physical and logical foundation for the heterogeneous dual-core collaborative operation of industrial communication adapters.
[0081] The following example of a specific water pumping station application scenario further illustrates this application.
[0082] In a water pumping station scenario during the summer peak water usage period, the industrial communication adapter collects parameters such as flow rate, pressure, and turbidity in real time. In Scenario A (normal fluctuations), due to variations in water consumption between morning and evening, flow rate and pressure exhibit normal fluctuations, with the magnitude of these fluctuations remaining within the normal range of the historical baseline. After data preprocessing by the microprocessor (MPU), the data enters the complexity quantifier to calculate the components of each dimension: the data abrupt change degree F1 is at a low level, the trend complexity F2 shows stable fluctuation characteristics, and the multi-parameter coupling anomaly degree F3 is normal (increased flow rate accompanied by a moderate decrease in pressure). After weighted fusion, the overall complexity index C=0.2. The hierarchical decision module determines this to be low complexity, and the system only caches the data locally without triggering any alarms or reporting actions. The neural network processor (NPU) remains dormant, saving a significant amount of power. In Scenario B (main pipe burst), a sudden pipe burst occurs, causing a rapid surge in flow and a sharp drop in pressure. During complex quantification calculations, the microprocessor (MPU) detects: the data abrupt change component F1 is extremely high, primarily driven by drastic deviations in flow and pressure; the multi-parameter coupling anomaly component F3 is also significantly elevated due to a severe deviation in the correlation coefficient between actual flow and pressure from historical standard correlation coefficients (normally a negative correlation, currently exhibiting abnormal coupling); and the trend complexity component F2 is also elevated due to the continuous downward pressure trend. After weighted fusion of the three dimensions, the overall complexity index C = 0.85, which the hierarchical decision model classifies as urgent complexity. The system then switches to a high / urgent processing path: the microprocessor constructs an anomaly feature vector E based on the contribution sources of each dimension recorded during the complexity quantification process, decomposing the anomaly contribution to each monitored parameter. Matching with a pre-set anomaly pattern library reveals that the current E has an extremely high similarity to the "flow surge" pattern template (P01) and is significantly greater than other patterns, indicating a single best match. Based on this, the microprocessor activates the neural network processor (NPU), which loads only the set of "pipeline hydraulics and leakage location models" corresponding to this mode for precise analysis and quickly outputs pipe burst alarm information, avoiding the waste of computing power caused by full analysis.
[0083] In scenario C (pressure sensor malfunction or illegal bypass), the flow rate surges abnormally while the pressure rises synchronously instead of decreasing according to physical laws, exhibiting an abnormal state that violates the negative correlation law. When the microprocessor (MPU) performs complex quantification calculations, it finds that although the data mutation degree component F1 is high, the multi-parameter coupling anomaly component F3 becomes dominant, with its contribution far exceeding other dimensions. This is because the actual flow-pressure correlation coefficient deviates significantly from the historical standard correlation coefficient (normally it should be negative, but currently it shows an abnormal positive correlation). After weighted fusion, the overall complexity index C ≥ 0.6, and the hierarchical decision module determines it to be high complexity. The system switches to a high-complexity processing path: the microprocessor constructs an anomaly feature vector E, in which the anomaly contribution of pressure and flow parameters is significantly higher than that of other parameters. Matching with the anomaly pattern library reveals that the current E has the highest similarity to the "pressure-flow decoupling" pattern template (P04), and is determined to be the single best match. Based on this, the microprocessor activates the neural network processor (NPU), loads the set of "device diagnostic models" corresponding to this mode for precise analysis, accurately identifies non-real hydraulic changes as originating from sensor failures or illegal bypasses, and avoids erroneous pipeline scheduling instructions.
[0084] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in the present invention should still be covered by the claims of the present invention.
[0085] In the corresponding figures of the above embodiments, connecting lines can represent the connection relationship between various components, indicating more constitutive signal paths and / or one or more ends of some lines having arrows to indicate the main information flow direction. Connecting lines are an identifier and are not a limitation on the scheme itself, but rather the use of these lines in combination with one or more exemplary embodiments helps to more easily connect circuits or logic units. Any signal represented (determined by design requirements or preferences) can actually include one or more signals that can be transmitted in any direction and can be implemented in any suitable type of signal scheme.
[0086] In the above embodiments, unless otherwise specified, the use of serial numbers such as "first" and "second" to describe common objects only indicates that they refer to different instances of the same object, rather than indicating that the objects being described must be in a given order, whether temporally, spatially, sequentially, or in any other way.
[0087] In the above embodiments, references to "this embodiment," "an embodiment," "another embodiment," or "other embodiments" in the specification indicate that a specific feature, structure, or characteristic described in connection with the embodiments is included in at least some embodiments, but not necessarily all embodiments. Multiple appearances of "this embodiment," "an embodiment," or "another embodiment" do not necessarily all refer to the same embodiment. If the specification describes a component, feature, structure, or characteristic as "may," "may," or "can" be included, then that specific component, feature, structure, or characteristic is not necessarily required to be included. If the specification or claims refer to an element "a," it does not mean that there is only one element. If the specification or claims refer to "an additional" element, it does not exclude the existence of more than one additional element.
[0088] In the above embodiments, although the invention has been described in conjunction with specific embodiments thereof, many substitutions, modifications, and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory structures (e.g., dynamic RAM (DRAM)) may be used with the embodiments discussed. The embodiments of the invention are intended to cover all such substitutions, modifications, and variations falling within the broad scope of the appended claims.
[0089] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.
[0090] This invention can be used in a wide range of general-purpose or special-purpose computing system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices, etc.
[0091] This invention can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This invention can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.
Claims
1. A data complexity evaluation method, applied to industrial communication adapters, characterized in that, include: Acquire the data to be evaluated, which includes monitoring parameters; Based on the data to be evaluated, multi-dimensional data is calculated to obtain a comprehensive complexity index. The multi-dimensional data includes data mutation degree, trend complexity, and multi-parameter coupling anomaly degree. A hierarchical decision is made based on the comprehensive complexity index, wherein the hierarchical decision includes matching the comprehensive complexity index with a preset complexity threshold, and different complexity thresholds correspond to different levels of processing strategies. The data to be evaluated is quantified to obtain an anomaly feature vector that characterizes the anomaly contribution of each monitoring parameter. Based on the anomaly contribution of each monitoring parameter in the anomaly feature vector and the pre-configured weights of the multi-dimensional data corresponding to each monitoring parameter, the data complexity is evaluated to determine the final processing strategy.
2. The data complexity evaluation method according to claim 1, characterized in that, The data mutation rate is obtained based on the degree of deviation between the current data and the baseline of historical data; the formula is as follows: F1= Where F1 represents the data mutation degree, The sampled value at the current moment. This is the historical baseline mean. The historical baseline standard deviation It is a very small constant.
3. The data complexity evaluation method according to claim 1, characterized in that, The trend complexity is obtained based on the density of trend inflection points in the data to be evaluated within the time window.
4. The data complexity evaluation method according to claim 1, characterized in that, The multi-parameter coupling anomaly degree is obtained based on the synergistic relationship between the interrelated parameters; the formula is as follows: in, For multi-parameter coupling anomaly degree, This represents the actual correlation coefficient between parameters i and j within the current window. The standard correlation coefficient is the historical correlation coefficient under normal operating conditions. Weights for different parameter pairs.
5. The data complexity evaluation method according to claim 1, characterized in that, The data complexity evaluation based on the anomaly feature vector and the weights of the multi-dimensional data corresponding to each pre-configured parameter includes: Calculate the anomaly contribution components of each monitoring parameter in the three dimensions of the data mutation degree, the trend complexity, and the multi-parameter coupling anomaly degree; The anomaly contribution components of the three dimensions are weighted and summed to obtain the anomaly contribution of each monitoring parameter, and a parameter-level anomaly feature vector is constructed. The similarity between the current abnormal feature vector and the feature vector template in the pre-built abnormal pattern library is calculated using the reciprocal of the weighted Euclidean distance, and abnormal pattern matching is performed based on the similarity results.
6. The data complexity evaluation method according to claim 5, characterized in that, After performing abnormal pattern matching based on the similarity results, the process also includes: Select the corresponding analysis model based on the matching results. If the similarity with each abnormal pattern in the abnormal pattern library is less than a preset first threshold, it is determined to be an unknown abnormality, and a full analysis model is used for analysis. If the similarity with an abnormal pattern in the abnormal pattern library is greater than a preset multiple of the similarity with other abnormal patterns, then the analysis model corresponding to the abnormal pattern is used for analysis. If the similarity with multiple abnormal patterns in the abnormal pattern library is greater than a preset second threshold, then the analysis is performed using the union of the analysis model sets corresponding to the multiple abnormal patterns.
7. The data complexity evaluation method according to any one of claims 1-6, characterized in that, Historical data statistical features are periodically distributed from the cloud to the edge, and the edge adjusts the weights of data mutation degree, trend complexity, and multi-parameter coupling anomaly degree according to the historical data statistical features.
8. A data complexity evaluation system, applied to industrial communication adapters, characterized in that, include: The data acquisition module is used to acquire the data to be evaluated, which includes monitoring parameters; The complexity calculation module is used to calculate multi-dimensional data based on the data to be evaluated to obtain a comprehensive complexity index. The multi-dimensional data includes data mutation degree, trend complexity, and multi-parameter coupling anomaly degree. A hierarchical decision-making module is used to make hierarchical decisions based on the comprehensive complexity index. The hierarchical decision-making includes matching the comprehensive complexity index with a preset complexity threshold, and different complexity thresholds correspond to different levels of processing strategies. An anomaly feature vector module is used to quantify the data to be evaluated to obtain an anomaly feature vector that characterizes the anomaly contribution of each monitoring parameter. The evaluation module is used to evaluate the data complexity based on the anomaly feature vector and the weights of the multi-dimensional data corresponding to each pre-configured monitoring parameter, so as to determine the final processing strategy.
9. An electronic device, characterized in that, include: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, cause the electronic device to implement the data complexity evaluation method as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that, It stores computer-readable instructions that, when executed by the computer's processor, cause the computer to perform the data complexity evaluation method according to any one of claims 1 to 7.
