A communication detection and early warning system based on big data

Abstract

This invention relates to the field of industrial communication detection and early warning, aiming to solve the problems of poor adaptability of multi-source heterogeneous data in ICPS, high false alarm rate of anomaly detection, and insufficient targeted early warning, and provides an intelligent system. This invention includes modules for data acquisition, adaptive preprocessing of multi-source heterogeneous communication data, spatiotemporal correlation dynamic calibration anomaly detection and hierarchical early warning, cloud storage, and early warning push. Multi-protocol communication data is collected through distributed edge nodes. After adaptive weighted threshold filtering preprocessing, an improved spatiotemporal correlation dynamic calibration LSTM model is used to extract spatiotemporal features, dynamically calibrate and quantify anomaly scores, and issue hierarchical early warnings according to three threshold levels. The warnings are pushed to corresponding maintenance personnel through multiple channels, and the data is synchronously stored in the cloud. This invention improves the accuracy and response efficiency of industrial communication anomaly detection, adapts to multi-protocol heterogeneous scenarios, provides a guarantee for the safe and stable operation of ICPS, and is applicable to industrial scenarios such as water treatment and power dispatching.

Description

Technical Field

[0001] This invention relates to the field of communication detection and early warning, specifically to a communication detection and early warning system based on big data. Background Technology

[0002] Industrial Cyber-Physical Systems (ICPS) integrate industrial equipment, communication networks, and cloud management platforms, and are widely used in critical industrial scenarios such as water treatment, power dispatching, and metal smelting. Their communication stability directly determines the safety and efficiency of industrial production. Current ICPS communication detection and early warning systems suffer from several prominent issues specific to these fields, making it difficult to meet the high reliability and low latency requirements of industrial scenarios. Specifically: Poor adaptability of multi-source heterogeneous data preprocessing: ICPS contains multiple communication protocols such as Modbus, OPC UA, and MQTT. The data formats, transmission frequencies, and noise characteristics of different protocols vary significantly. Existing preprocessing algorithms mostly use fixed threshold filtering and uniform standardization methods, without considering the differences in protocol types and the importance of data, resulting in incomplete noise filtering and loss of effective data, which affects the accuracy of subsequent anomaly detection. The anomaly detection and early warning systems lack specificity: existing systems mostly use single-dimensional anomaly detection algorithms, without taking into account the spatiotemporal correlation characteristics of ICPS devices (such as the collaborative correlation of communication data between devices in the same workshop), and the early warnings are not graded, so they cannot provide accurate handling guidance based on the severity of the anomalies, increasing operation and maintenance costs.

[0003] Currently, there are no effective solutions to the specific problems encountered in the aforementioned ICPS communication scenarios. Existing communication detection and early warning systems are mostly general-purpose designs that are not optimized for the multi-protocol heterogeneity and time-series distribution drift characteristics of ICPS, thus failing to meet the high security and high reliability requirements of industrial production. Therefore, developing a targeted and innovative big data-based communication detection and early warning system that can solve these specific problems has become an urgent technical challenge. Summary of the Invention

[0004] The purpose of this invention is to provide a communication detection and early warning system based on big data to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a communication detection and early warning system based on big data, comprising a data acquisition module, a multi-source heterogeneous communication data adaptive preprocessing module, a spatiotemporal correlation dynamic calibration anomaly detection and hierarchical early warning module, a cloud storage module, and an early warning push module; the data acquisition module adopts an edge node distributed acquisition method, deployed on the terminals of various industrial equipment (PLC, sensors, industrial robots, etc.) of ICPS, for acquiring equipment communication data (including data frames, transmission delay, packet loss rate, signal strength, etc.), supporting multi-protocol data acquisition, and dynamically adjusting the acquisition frequency according to the equipment type (1Hz-100Hz). After acquisition, the raw data is transmitted to the multi-source heterogeneous communication data adaptive preprocessing module in real time; The multi-source heterogeneous communication data adaptive preprocessing module receives multi-source heterogeneous communication data transmitted by the data acquisition module, and uses an improved adaptive weighted threshold filtering algorithm to complete data classification, noise estimation, weighted filtering, and standardization processing, solving the problems of poor adaptability of multi-protocol data and large noise interference. The processed data is then transmitted to the spatiotemporal correlation dynamic calibration anomaly detection and hierarchical early warning module. The spatiotemporal correlation dynamic calibration anomaly detection and hierarchical early warning module receives communication data preprocessed by the multi-source heterogeneous communication data adaptive preprocessing module, and uses an improved spatiotemporal correlation dynamic calibration LSTM model (ST-DC-LSTM) to complete spatiotemporal feature extraction, dynamic calibration, anomaly scoring, and hierarchical early warning. This solves the problems of cloud-edge communication latency, high bandwidth costs, and false alarms and missed detections caused by distribution drift. The early warning information is transmitted to the early warning push module, and the detection results and preprocessed data are transmitted to the cloud storage module. The cloud storage module adopts a cloud-native architecture, supports distributed storage and fast retrieval of multi-source data, and stores raw communication data, preprocessed data, anomaly detection results, model parameters, and calibration data, providing data support for model iteration and optimization. The early warning push module receives early warning information transmitted by the spatiotemporal correlation dynamic calibration anomaly detection and hierarchical early warning module. Based on the anomaly classification results, it pushes the early warning information to the corresponding level of operation and maintenance personnel and management platform through methods including but not limited to SMS, APP push, and platform pop-up windows. At the same time, it records the early warning log for easy follow-up.

[0006] Preferably, the specific implementation logic of the early warning data acquisition module is as follows: Step A1: Edge Acquisition Node Deployment: Based on the ICPS equipment distribution scenario (e.g., industrial water treatment workshop, power dispatch station), deploy edge acquisition nodes one-to-one with each industrial equipment terminal to ensure that each device (PLC controller, sensor, industrial robot, etc.) has a dedicated acquisition node, achieving distributed full-coverage acquisition. High-performance edge acquisition nodes (supporting high-frequency data acquisition and multi-protocol compatibility) are used for critical equipment such as PLC controllers and transformers; general-purpose edge acquisition nodes are used for conventional equipment such as water pumps and ordinary monitoring terminals to reduce deployment costs. All edge nodes are interconnected via an industrial LAN and synchronously connected to the system's unified clock to ensure the accuracy of subsequent data time alignment. Step A2, Adaptive Configuration of Acquisition Parameters: Based on device type, communication protocol, and industrial scenario requirements, complete the initial configuration of acquisition parameters to ensure the relevance and effectiveness of the acquired data. Specific configuration includes: Protocol Adaptation Configuration: Configure a corresponding protocol parsing module for each edge acquisition node according to the communication protocol used by the device, i.e., Modbus, OPC UA, or MQTT, to ensure accurate parsing of raw data frames from different protocols and avoid data acquisition failures due to protocol incompatibility; Acquisition Frequency Configuration: Dynamically set the acquisition frequency (range 1Hz-100Hz) according to the importance of the device and the real-time requirements of the data—OPC. For critical devices using the UA protocol (such as PLC controllers and transformers), the acquisition frequency is set to 50-100Hz to ensure the real-time performance of critical data. For conventional devices using the Modbus protocol (such as water pumps and industrial robots), the acquisition frequency is set to 10-30Hz to balance acquisition efficiency and resource consumption. For low-priority devices using the MQTT protocol (such as ordinary water quality sensors), the acquisition frequency is set to 1-10Hz to reduce unnecessary resource consumption. Data acquisition item configuration: Clearly define the data acquisition items for each device and standardize the core communication parameters of the acquisition devices, including but not limited to data frames (frame length, checksum), transmission delay, packet loss rate, and signal strength. This ensures that the acquired data is consistent with the requirements of subsequent preprocessing modules, providing a complete data foundation for noise filtering and standardization. Step A3: Start Distributed Data Acquisition: After completing the parameter configuration, all edge acquisition nodes synchronously start the acquisition process, adopting a real-time sampling plus local caching mode to achieve continuous acquisition of raw communication data: Each acquisition node acquires the communication data of the corresponding device in real time according to a preset frequency. No data processing is performed during the acquisition process (to avoid affecting the real-time performance of the acquisition). The raw data is temporarily stored in the node's local cache (the cache capacity is adapted to the amount of data acquired in 24 hours to prevent data overflow). At the same time, the timestamp corresponding to each data frame is recorded (accurate to the millisecond level) to provide support for data time alignment in the subsequent preprocessing module. Step A4: Preliminary Verification of Data Validity: To reduce interference from invalid data to subsequent preprocessing modules, before data transmission, the edge acquisition nodes perform a preliminary validity verification of the locally cached raw data, removing obviously invalid data: data missing key fields (such as missing timestamps or incorrect data frame checksums); data exceeding reasonable ranges (such as abnormal values ​​with signal strengths exceeding -100dBm to 0dBm or extreme values ​​with transmission delays greater than 1000ms); and duplicate data frames (through data frame checksum deduplication). After verification, valid raw data is retained, and the invalid data cache is cleared to ensure the quality of data transmitted to the preprocessing module. Step A5: Real-time Transmission of Raw Data: Verified raw data is transmitted in real-time from the edge acquisition nodes to the multi-source heterogeneous communication data adaptive preprocessing module via the industrial LAN. The transmission process combines batch transmission with real-time retransmission: data is transmitted in batches every 10 acquisition time steps, while lost data packets are retransmitted in real-time to ensure data transmission integrity and real-time performance, preventing deviations in subsequent preprocessing and anomaly detection due to data loss. After transmission, the edge acquisition nodes continue to execute acquisition tasks, forming a closed-loop workflow of "acquisition-verification-transmission," continuously providing raw data support to the preprocessing module.

[0007] Preferably, the specific implementation logic of the multi-source heterogeneous communication data adaptive preprocessing module is as follows: Step B1, Data Classification: Receive the raw communication data transmitted by the data acquisition module and classify it according to the communication protocol type (Modbus, OPC UA, MQTT). Simultaneously, extract the feature dimensions of each data type, including but not limited to data frame length, transmission delay, packet loss rate, signal strength, and checksum. For missing data, interpolation is performed (using linear interpolation for continuous data; and mode interpolation for discrete data) to obtain the complete classified data set. ,in For Modbus protocol data, For OPC UA protocol data, For MQTT protocol data, each data set contains One sample Each feature dimension, i.e. , ; Step B2, Adaptive Noise Estimation: For each feature dimension of each type of protocol data, a fusion estimation method based on data fluctuation characteristics and protocol type weights is used to calculate the noise intensity of that dimension. The calculation formula is as follows: ,in For the first Class of communication data, the first Noise estimates for each feature dimension This is the weight of the fluctuation coefficient, with a value ranging from 0.3 to 0.7. It is dynamically adjusted according to the noise characteristics of the industrial scenario (0.7 for scenarios with large noise fluctuations and 0.3 for scenarios with stable noise). The standard deviation of the data in this dimension reflects the degree of data fluctuation. The protocol type weight is set according to the noise sensitivity of different protocols (OPC UA protocol is used for communication of critical equipment, with the highest noise sensitivity, so it is set to 1.0; Modbus protocol is used for communication of general equipment, so it is set to 0.8; MQTT protocol is used for low-priority data transmission, so it is set to 0.7). The mean of the absolute values ​​of the data in this dimension helps to reflect the overall intensity of the noise. This noise estimation method integrates the data's own fluctuations and protocol type characteristics. Compared with the traditional single standard deviation estimation method, it can more accurately adapt to the noise characteristics of different protocol data and avoid poor filtering effects caused by noise estimation bias. Step B3, Weighted Threshold Filtering: Based on the noise estimate calculated in Step 2 Combining the importance weights of data features (The weighting is set according to the degree of influence of the feature on the communication anomaly, such as packet loss rate and transmission delay with an importance weight of 0.9, signal strength with 0.7, and checksum with 0.8), calculate the weighted filtering threshold for each feature dimension. The formula is as follows: ,in, This is the threshold coefficient, ranging from 2.5 to 3.5, adjusted according to the false detection tolerance in the industrial scenario (3.5 for scenarios with low false detection tolerance, and 2.5 for scenarios with high false detection tolerance); during the filtering process, for each data sample... (No. Class data, the first The sample, the first (a feature), if it is equal to the mean of that feature dimension. The absolute value of the difference is greater than If the difference is less than or equal to the moving average of that feature dimension (with a moving window size of 5-10 samples), it is considered noise data and replaced. If the original data is not found, the original data is retained. The innovation of this weighted threshold filtering lies in the fact that the threshold is dynamically adjusted according to the noise intensity, protocol type, and feature importance. Compared with traditional fixed threshold filtering, it can effectively filter noise and avoid the loss of valid data, thereby improving the accuracy of data preprocessing. Step B4, Standardization Processing: For the filtered protocol data, an adaptive standardization method is used. Based on the value range and distribution characteristics of different protocol data, standardization processing is performed separately, mapping the data to the [0,1] interval. The formula is as follows: ,in, For the standardized first Class data, the first The sample, the first 1 eigenvalue, , The first Class Data The minimum and maximum values ​​of each feature dimension are determined; after standardization, various protocol data are integrated into a preprocessed data set in a unified format and transmitted to the spatiotemporal correlation dynamic calibration anomaly detection and hierarchical early warning module.

[0008] Preferably, the specific implementation logic of the spatiotemporal correlation dynamic calibration anomaly detection and hierarchical early warning module is as follows: Step C1, Spatiotemporal Feature Extraction: Receive preprocessed data from the multi-source heterogeneous communication data adaptive preprocessing module. First, extract temporal features using a sliding window (10-30 time steps) to extract the temporal characteristics (including temporal trends, fluctuation frequencies, and abrupt change points) of each device's communication data. Second, extract spatial features by dividing the ICPS devices into groups based on their deployment location (devices in the same workshop or on the same production line are grouped together). Calculate the correlation between the current device's communication data and that of other neighboring devices within the group to obtain spatial correlation features. Finally, fuse the temporal and spatial features to construct a spatiotemporal correlation feature vector. ,in For the current time step, For the spatiotemporal feature dimension, the formula for calculating the spatiotemporal correlation weight is: ,in, for Current device and The spatiotemporal correlation weights of devices in the time neighborhood The covariance of the two data sets reflects spatial correlation. , These are the variances of the two sets of data, The time decay coefficient, ranging from 5 to 10, is used to reduce the impact of data from neighboring devices with long time intervals, ensuring the timeliness of spatiotemporal correlation features. This spatiotemporal feature extraction combines the spatial deployment characteristics of ICPS devices with the temporal characteristics of data. Compared with traditional single-time dimension feature extraction, it can more comprehensively capture the characteristics of communication anomalies and improve the accuracy of anomaly detection. Step C2, Dynamic Calibration: To address the distribution drift problem in ICPS time-series data, a sliding window calibration mechanism is employed to construct a dynamic calibration set. Combined with the consistent form prediction (CP) method, the output of the improved spatiotemporal correlation dynamic calibration LSTM model is dynamically calibrated to avoid increased false alarm rates caused by distribution drift. Specifically, the sliding calibration window size is set to 50-100 time steps. The window dynamically moves forward over time, and each time it moves forward, abnormal samples within the window are removed (based on historical anomaly detection results), retaining normal samples as the calibration set. ( (where is the number of samples in the calibration set). The CP value is calculated using the calibration set to correct the output of the improved spatiotemporal correlation dynamic calibration LSTM model, ensuring that the model can adapt to changes in data distribution. The calibration formula is as follows: ,in, This is an indicator function; when the condition inside the parentheses is true, ,otherwise ; To improve the spatiotemporal correlation dynamic calibration LSTM model for the calibration set... The output value of each sample To improve the spatiotemporal correlation dynamic calibration of the LSTM model's output value for the current time step data; Step C3, Anomaly Scoring: The spatiotemporal correlation feature vector extracted in Step C1 is input into the improved spatiotemporal correlation dynamic calibration LSTM model to obtain the basic output value of the improved spatiotemporal correlation dynamic calibration LSTM model. Combined with the CP value calculated in Step C2, a weighted fusion method is used to obtain the anomaly score of the communication data at the current time step. The formula is as follows: ,in, The fusion weight is set to a value of 0.6-0.8 to balance the influence of model output and calibration results (0.8 for scenarios with high model output reliability, and 0.6 for scenarios with large data distribution fluctuations). The output value of the LSTM model incorporating spatiotemporal weights, namely the improved spatiotemporal correlation dynamic calibration LSTM model, reflects the initial judgment on the degree of anomaly in the current data. This is the mean of the CP values, reflecting the degree of anomaly in the current data relative to the calibration set. Anomaly Score The value range is [0,1], and the higher the score, the greater the possibility of communication anomalies. This anomaly scoring method integrates model prediction and statistical calibration results. Compared with the traditional single model output score, it can effectively reduce the false alarm rate caused by distribution drift and improve the accuracy of anomaly scoring. Step C4, Tiered Early Warning: Based on anomaly scoring The value is set to a three-level warning threshold to achieve tiered warning. The specific threshold division is as follows: Level 1 warning (minor anomaly): The corresponding communication data shows slight fluctuations, which may be due to temporary noise interference. No downtime is required; simply alert the maintenance personnel. Level 2 Warning (Moderate Anomaly): The corresponding communication exhibits significant anomalies, leading to unstable data transmission. Maintenance personnel promptly investigated the equipment and network; Level 3 alert (severe anomaly): If a serious communication anomaly is detected, indicating equipment failure or malicious attack, the system should be shut down immediately for investigation to prevent industrial production accidents. After classification, the warning level, abnormal data, abnormal characteristics, and investigation suggestions will be transmitted to the warning push module, and the detection results will be transmitted to the cloud storage module for subsequent model iteration and optimization.

[0009] Compared with the prior art, the beneficial effects of the present invention are as follows: The present invention adopts an improved adaptive weighted threshold filtering algorithm, which dynamically adjusts the noise estimation and filtering threshold by combining protocol type weight and feature importance, to achieve accurate noise reduction for multi-protocol data such as Modbus, OPC UA, and MQTT, effectively filtering interference noise in the industrial environment, improving the accuracy of data preprocessing, and improving the effective data retention rate compared with traditional fixed threshold filtering, thus providing high-quality data support for subsequent anomaly detection.

[0010] This invention classifies and processes data according to protocol type and adopts an adaptive standardization method to uniformly map heterogeneous data with different value ranges and distribution characteristics to the [0,1] interval. At the same time, it supplements missing data, eliminates semantic and format differences between multi-protocol data, improves data integration consistency, and solves the problem of difficult collaborative analysis of heterogeneous data in traditional systems.

[0011] This invention integrates time-series trends and fluctuation frequencies with spatial device group correlation features to construct a spatiotemporal correlation feature vector, comprehensively capturing the spatiotemporal coupling characteristics of ICPS communication anomalies. Compared with single-dimensional detection, the anomaly identification coverage is improved, and it can accurately identify single-point device anomalies and group collaborative anomalies.

[0012] This invention constructs a dynamic calibration set through a sliding window, combines the CP method to correct the model output in real time, adapts to data distribution drift, avoids the increase of false alarm rate in long-term operation, reduces the false alarm rate of anomaly detection, stabilizes detection accuracy, and adapts to the time-series dynamic change characteristics of industrial scenario data.

[0013] This invention uses a weighted fusion method to calculate anomaly scores, balances model predictions and statistical calibration results, quantifies the degree of communication anomalies, and provides a precise basis for graded early warning. Compared with traditional qualitative detection, the objectivity and operability of anomaly judgment are greatly improved.

[0014] This invention uses a three-tiered alert system based on anomaly scores, sending targeted alerts to the corresponding levels of operations and maintenance (O&M) personnel. Minor anomalies only require notification, moderate anomalies are investigated promptly, and severe anomalies require immediate shutdown. This avoids the waste of O&M resources caused by a one-size-fits-all approach to alerts, thus improving O&M response efficiency. Alert information is simultaneously pushed through multiple channels, including SMS, app push notifications, and platform pop-ups, combined with a timeout retry mechanism to ensure that O&M personnel receive anomaly information as soon as possible. A complete alert log is also recorded for subsequent fault tracing and review. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the system structure of the present invention; Figure 2 This is a schematic diagram of the data acquisition module workflow of the present invention; Figure 3 This is a schematic diagram of the workflow of the adaptive preprocessing module for multi-source heterogeneous communication data of the present invention; Figure 4 This is a schematic diagram of the workflow of the spatiotemporal correlation dynamic calibration anomaly detection and hierarchical early warning module of the present invention. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] Please see Figure 1-4 This invention provides a technical solution: a communication detection and early warning system based on big data, comprising a data acquisition module, a multi-source heterogeneous communication data adaptive preprocessing module, a spatiotemporal correlation dynamic calibration anomaly detection and hierarchical early warning module, a cloud storage module, and an early warning push module; the data acquisition module adopts a distributed acquisition method at edge nodes, deployed on various industrial equipment (PLC, sensors, industrial robots, etc.) terminals of ICPS, for acquiring equipment communication data (including data frames, transmission delay, packet loss rate, signal strength, etc.), supporting multi-protocol data acquisition, and dynamically adjusting the acquisition frequency according to the equipment type (1Hz-100Hz). After acquisition, the raw data is transmitted in real time to the multi-source heterogeneous communication data adaptive preprocessing module; the specific implementation logic is as follows: Step A1: Edge Acquisition Node Deployment: Based on the ICPS equipment distribution scenario (e.g., industrial water treatment workshop, power dispatch station), deploy edge acquisition nodes one-to-one with each industrial equipment terminal to ensure that each device (PLC controller, sensor, industrial robot, etc.) has a dedicated acquisition node, achieving distributed full-coverage acquisition. High-performance edge acquisition nodes (supporting high-frequency data acquisition and multi-protocol compatibility) are used for critical equipment such as PLC controllers and transformers; general-purpose edge acquisition nodes are used for conventional equipment such as water pumps and ordinary monitoring terminals to reduce deployment costs. All edge nodes are interconnected via an industrial LAN and synchronously connected to the system's unified clock to ensure the accuracy of subsequent data time alignment. Step A2, Adaptive Configuration of Acquisition Parameters: Based on device type, communication protocol, and industrial scenario requirements, complete the initial configuration of acquisition parameters to ensure the relevance and effectiveness of the acquired data. Specific configuration includes: Protocol Adaptation Configuration: Configure a corresponding protocol parsing module for each edge acquisition node according to the communication protocol used by the device, i.e., Modbus, OPC UA, or MQTT, to ensure accurate parsing of raw data frames from different protocols and avoid data acquisition failures due to protocol incompatibility; Acquisition Frequency Configuration: Dynamically set the acquisition frequency (range 1Hz-100Hz) according to the importance of the device and the real-time requirements of the data—OPC. For critical devices using the UA protocol (such as PLC controllers and transformers), the acquisition frequency is set to 50-100Hz to ensure the real-time performance of critical data. For conventional devices using the Modbus protocol (such as water pumps and industrial robots), the acquisition frequency is set to 10-30Hz to balance acquisition efficiency and resource consumption. For low-priority devices using the MQTT protocol (such as ordinary water quality sensors), the acquisition frequency is set to 1-10Hz to reduce unnecessary resource consumption. Data acquisition item configuration: Clearly define the data acquisition items for each device and standardize the core communication parameters of the acquisition devices, including but not limited to data frames (frame length, checksum), transmission delay, packet loss rate, and signal strength. This ensures that the acquired data is consistent with the requirements of subsequent preprocessing modules, providing a complete data foundation for noise filtering and standardization. Step A3: Start Distributed Data Acquisition: After completing the parameter configuration, all edge acquisition nodes synchronously start the acquisition process, adopting a real-time sampling plus local caching mode to achieve continuous acquisition of raw communication data: Each acquisition node acquires the communication data of the corresponding device in real time according to a preset frequency. No data processing is performed during the acquisition process (to avoid affecting the real-time performance of the acquisition). The raw data is temporarily stored in the node's local cache (the cache capacity is adapted to the amount of data acquired in 24 hours to prevent data overflow). At the same time, the timestamp corresponding to each data frame is recorded (accurate to the millisecond level) to provide support for data time alignment in the subsequent preprocessing module. Step A4: Preliminary Verification of Data Validity: To reduce interference from invalid data to subsequent preprocessing modules, before data transmission, the edge acquisition nodes perform a preliminary validity verification of the locally cached raw data, removing obviously invalid data: data missing key fields (such as missing timestamps or incorrect data frame checksums); data exceeding reasonable ranges (such as abnormal values ​​with signal strengths exceeding -100dBm to 0dBm or extreme values ​​with transmission delays greater than 1000ms); and duplicate data frames (through data frame checksum deduplication). After verification, valid raw data is retained, and the invalid data cache is cleared to ensure the quality of data transmitted to the preprocessing module. Step A5: Real-time Transmission of Raw Data: Verified raw data is transmitted in real-time from the edge acquisition nodes to the multi-source heterogeneous communication data adaptive preprocessing module via the industrial LAN. The transmission process combines batch transmission with real-time retransmission: data is transmitted in batches every 10 acquisition time steps, while lost data packets are retransmitted in real-time to ensure data transmission integrity and real-time performance, preventing deviations in subsequent preprocessing and anomaly detection due to data loss. After transmission, the edge acquisition nodes continue to execute acquisition tasks, forming a closed-loop workflow of "acquisition-verification-transmission," continuously providing raw data support to the preprocessing module. Multi-source heterogeneous communication data adaptive preprocessing module: Receives multi-source heterogeneous communication data transmitted from the data acquisition module, and uses an improved adaptive weighted threshold filtering algorithm to complete data classification, noise estimation, weighted filtering, and standardization processing. This addresses the problems of poor compatibility and high noise interference in multi-protocol data. The processed data is then transmitted to the spatiotemporal correlation dynamic calibration anomaly detection and hierarchical early warning module. The specific implementation logic is as follows: Step B1, Data Classification: Receive the raw communication data transmitted by the data acquisition module and classify it according to the communication protocol type (Modbus, OPC UA, MQTT). Simultaneously, extract the feature dimensions of each data type, including but not limited to data frame length, transmission delay, packet loss rate, signal strength, and checksum. For missing data, interpolation is performed (using linear interpolation for continuous data; and mode interpolation for discrete data) to obtain the complete classified data set. ,in For Modbus protocol data, For OPC UA protocol data, For MQTT protocol data, each data set contains One sample Each feature dimension, i.e. , ; Step B2, Adaptive Noise Estimation: For each feature dimension of each type of protocol data, a fusion estimation method based on data fluctuation characteristics and protocol type weights is used to calculate the noise intensity of that dimension. The calculation formula is as follows: ,in For the first Class of communication data, the first Noise estimates for each feature dimension This is the weight of the fluctuation coefficient, with a value ranging from 0.3 to 0.7. It is dynamically adjusted according to the noise characteristics of the industrial scenario (0.7 for scenarios with large noise fluctuations and 0.3 for scenarios with stable noise). The standard deviation of the data in this dimension reflects the degree of data fluctuation. The protocol type weight is set according to the noise sensitivity of different protocols (OPC UA protocol is used for communication of critical equipment, with the highest noise sensitivity, so it is set to 1.0; Modbus protocol is used for communication of general equipment, so it is set to 0.8; MQTT protocol is used for low-priority data transmission, so it is set to 0.7). The mean of the absolute values ​​of the data in this dimension helps to reflect the overall intensity of the noise. This noise estimation method integrates the data's own fluctuations and protocol type characteristics. Compared with the traditional single standard deviation estimation method, it can more accurately adapt to the noise characteristics of different protocol data and avoid poor filtering effects caused by noise estimation bias. Step B3, Weighted Threshold Filtering: Based on the noise estimate calculated in Step 2 Combining the importance weights of data features (The weighting is set according to the degree of influence of the feature on the communication anomaly, such as packet loss rate and transmission delay with an importance weight of 0.9, signal strength with 0.7, and checksum with 0.8), calculate the weighted filtering threshold for each feature dimension. The formula is as follows: ,in, This is the threshold coefficient, ranging from 2.5 to 3.5, adjusted according to the false detection tolerance in the industrial scenario (3.5 for scenarios with low false detection tolerance, and 2.5 for scenarios with high false detection tolerance); during the filtering process, for each data sample... (No. Class data, the first The sample, the first (a feature), if it is equal to the mean of that feature dimension. The absolute value of the difference is greater than If the difference is less than or equal to the moving average of that feature dimension (with a moving window size of 5-10 samples), it is considered noise data and replaced. If the original data is not found, the original data is retained. The innovation of this weighted threshold filtering lies in the fact that the threshold is dynamically adjusted according to the noise intensity, protocol type, and feature importance. Compared with traditional fixed threshold filtering, it can effectively filter noise and avoid the loss of valid data, thereby improving the accuracy of data preprocessing. Step B4, Standardization Processing: For the filtered protocol data, an adaptive standardization method is used. Based on the value range and distribution characteristics of different protocol data, standardization processing is performed separately, mapping the data to the [0,1] interval. The formula is as follows: ,in, For the standardized first Class data, the first The sample, the first 1 eigenvalue, , The first Class Data The minimum and maximum values ​​of each feature dimension are determined; after standardization, various protocol data are integrated into a preprocessed data set in a unified format and transmitted to the spatiotemporal correlation dynamic calibration anomaly detection and hierarchical early warning module.

[0018] The spatiotemporal correlation dynamic calibration anomaly detection and hierarchical early warning module receives communication data preprocessed by the multi-source heterogeneous communication data adaptive preprocessing module. It employs an improved spatiotemporal correlation dynamic calibration LSTM model (ST-DC-LSTM) to complete spatiotemporal feature extraction, dynamic calibration, anomaly scoring, and hierarchical early warning. This addresses the issues of cloud-edge communication latency, high bandwidth costs, and false alarms / missed detections caused by distribution drift. The early warning information is transmitted to the early warning push module, while the detection results and preprocessed data are simultaneously transmitted to the cloud storage module. The specific implementation logic is as follows: Step C1, Spatiotemporal Feature Extraction: Receive preprocessed data from the multi-source heterogeneous communication data adaptive preprocessing module. First, extract temporal features using a sliding window (10-30 time steps) to extract the temporal characteristics (including temporal trends, fluctuation frequencies, and abrupt change points) of each device's communication data. Second, extract spatial features by dividing the ICPS devices into groups based on their deployment location (devices in the same workshop or on the same production line are grouped together). Calculate the correlation between the current device's communication data and that of other neighboring devices within the group to obtain spatial correlation features. Finally, fuse the temporal and spatial features to construct a spatiotemporal correlation feature vector. ,in For the current time step, As a spatiotemporal feature dimension, the specific construction steps are as follows: Step 1: Preprocessing Data Verification and Alignment. First, the preprocessed data from the adaptive preprocessing module receiving multi-source heterogeneous communication data is validated. Abnormal redundant data not fully filtered during preprocessing (such as data exceeding the [0,1] interval after standardization, and duplicate data frames) is removed. A small number of residual missing values ​​are supplemented using linear interpolation to ensure data integrity. Then, based on the timestamps at the time of data acquisition, time alignment processing is performed on the communication data from all devices—using the system's unified clock as a reference, communication data from different protocols and devices are synchronized to the same time step (the time step interval matches the data acquisition frequency; 1Hz acquisition corresponds to 1s / time step, and 100Hz acquisition corresponds to 10ms / time step), avoiding spatiotemporal feature deviations caused by time misalignment, resulting in a time-aligned standardized data set. ,in To unify the time step, This represents the total number of ICPS devices. for Time of the first Standardized communication feature vectors of individual devices; Step 2, Time Dimension Feature Extraction (Adaptive Sliding Window): For each device's communication data after time alignment, an adaptive sliding window mechanism is used to extract time-series features. This eliminates the limitations of the traditional fixed window and dynamically adjusts the window size based on the device's communication protocol type and data fluctuation characteristics. For critical devices corresponding to the OPC UA protocol (such as PLC controllers and transformers), where data real-time requirements are high, the sliding window size is set to 10-15 time steps. For conventional devices corresponding to the Modbus and MQTT protocols (such as water pumps and monitoring terminals), where data fluctuations are relatively mild, the sliding window size is set to 16-30 time steps. The window size can be dynamically fine-tuned based on the historical data fluctuation amplitude (when the fluctuation amplitude exceeds 0.2, the window is reduced by 2 time steps; when the fluctuation amplitude is less than 0.05, the window is expanded by 2 time steps).

[0019] Based on an adaptive sliding window, three core time-series features are extracted and quantified for the communication data of each device: Time-series trend features: The trend of data change within the window is fitted by linear regression, and the trend slope is calculated. A positive slope indicates an upward trend in communication parameters (such as increased latency), while a negative slope indicates a downward trend. The larger the absolute value of the slope, the more pronounced the trend. Frequency fluctuation characteristics: Fast Fourier Transform (FFT) is used to perform frequency domain analysis on the data within the window to extract the main fluctuation frequencies. Reflecting the degree of fluctuation in communication data, the higher the frequency, the worse the data stability; abrupt change point characteristics: the difference between adjacent time steps within a window is calculated using the first-order difference method, and a difference threshold is set. (Based on standardized data range settings), when the absolute value of the difference is greater than... When a mutation point is identified, the number of mutation points within the count window is determined. The more mutation points there are, the higher the risk of communication anomalies. After quantifying the three types of time-series features, a model is constructed for each device. Time feature vector at time step This enables accurate extraction of time-dimensional features; Step 3, Spatial Dimension Feature Extraction (Device Grouping + Correlation Calculation): Combining the actual deployment scenarios and functional relationships of ICPS devices, first complete the device grouping, then calculate the correlation of communication data between the current device and neighboring devices, and extract spatial correlation features: (1) Device group division: The dual division standard of location distance + function association is adopted. First, the physical deployment coordinates of all ICPS devices are obtained and the distance threshold is set. (Industrial workshop scene) Power dispatching scenario ), the distance is less than or equal to The equipment was initially divided into groups; subsequently, based on the functional relevance of the equipment, the initial groups were adjusted—equipment on the same production line and with the same control unit were grouped into the same group (e.g., in an industrial water treatment scenario, the PLC controller, water quality sensor, and water pump in the inlet area were grouped into one group, while the equipment in the treatment area was grouped into another group), avoiding spatial interference caused by proximity but unrelated functions, ultimately resulting in several equipment groups. Each group contains 3-10 devices.

[0020] (2) Spatial correlation calculation: for the current equipment Filter the groups they belong to Calculate the current device's value from all neighboring devices (excluding itself). Real-time communication data with each neighboring device Correlation of real-time communication data – Pearson correlation coefficient combined with protocol type weights is used to optimize correlation calculation results and adapt to the characteristics of heterogeneous data from multiple protocols. The calculation formula is as follows: ,in, for Current device With neighboring devices The spatial correlation coefficient ranges from [-1, 1], and the larger the absolute value, the stronger the correlation. , The protocol type weights for the current device and neighboring devices are respectively set (1.0 for OPC UA protocol, 0.8 for Modbus protocol, and 0.7 for MQTT protocol), which are used to strengthen the relevance weights of key devices; For both Covariance of time-matter data , These represent the variances of the two sets of data.

[0021] (3) Spatial feature vector construction: The correlation coefficients between the current device and all neighboring devices are statistically analyzed, and the mean of the correlation coefficients is extracted. Maximum value ,variance These three statistical values ​​reflect the overall correlation, strongest correlation, and correlation stability between the current device and its neighboring devices, respectively. These three values ​​are used as core spatial features to construct the current device's spatial structure. Spatial feature vector at time step .

[0022] Step 4: Spatiotemporal Feature Fusion. To achieve effective synergy between temporal and spatial features, a weighted adaptive fusion mechanism is adopted to construct a spatiotemporally correlated feature vector. ( That is, three dimensions of time features + three dimensions of spatial features), the fusion weights are dynamically adjusted according to the data distribution, and the specific process is as follows: First, calculate the time feature weights. Spatial feature weights ,satisfy When the current device data fluctuates drastically (absolute value of time-series trend slope > 0.5 or number of abrupt change points ≥ 2), the time characteristics have a greater impact on anomaly detection. Therefore, setting... , When the current device has a very strong correlation with neighboring devices (mean correlation coefficient ≥ 0.8), the spatial characteristics have higher reference value. , In typical scenarios, the settings are as follows: , .

[0023] Subsequently, the time feature vector With spatial feature vectors The weighted fusion is calculated using the following formula: ; Right now This completes the construction of the spatiotemporal correlation feature vector. This vector contains both the temporal variation pattern of device communication data and the spatial correlation characteristics between devices, which can comprehensively capture the spatiotemporal characteristics of ICPS communication anomalies, providing high-quality feature input for subsequent anomaly prediction and dynamic calibration of the ST-DC-LSTM model.

[0024] The formula for calculating the spatiotemporal correlation weight is: ,in, for Current device and The spatiotemporal correlation weights of devices in the time neighborhood The covariance of the two data sets reflects spatial correlation. , These are the variances of the two sets of data, The time decay coefficient, ranging from 5 to 10, is used to reduce the impact of data from neighboring devices with long time intervals, ensuring the timeliness of spatiotemporal correlation features. This spatiotemporal feature extraction combines the spatial deployment characteristics of ICPS devices with the temporal characteristics of data. Compared with traditional single-time dimension feature extraction, it can more comprehensively capture the characteristics of communication anomalies and improve the accuracy of anomaly detection. Step C2, Dynamic Calibration: To address the distribution drift problem in ICPS time-series data, a sliding window calibration mechanism is employed to construct a dynamic calibration set. Combined with the consistent form prediction (CP) method, the output of the improved spatiotemporal correlation dynamic calibration LSTM model is dynamically calibrated to avoid increased false alarm rates caused by distribution drift. Specifically, the sliding calibration window size is set to 50-100 time steps. The window dynamically moves forward over time, and each time it moves forward, abnormal samples within the window are removed (based on historical anomaly detection results), retaining normal samples as the calibration set. ( (where is the number of samples in the calibration set). The CP value is calculated using the calibration set to correct the output of the improved spatiotemporal correlation dynamic calibration LSTM model, ensuring that the model can adapt to changes in data distribution. The calibration formula is as follows: ,in, This is an indicator function; when the condition inside the parentheses is true, ,otherwise ; To improve the spatiotemporal correlation dynamic calibration LSTM model for the calibration set... The output value of each sample To improve the spatiotemporal correlation dynamic calibration of the LSTM model's output value for the current time step data; Step C3, Anomaly Scoring: The spatiotemporal correlation feature vector extracted in Step C1 is input into the improved spatiotemporal correlation dynamic calibration LSTM model to obtain the basic output value of the improved spatiotemporal correlation dynamic calibration LSTM model. Combined with the CP value calculated in Step C2, a weighted fusion method is used to obtain the anomaly score of the communication data at the current time step. The formula is as follows: ,in, The fusion weight is set to a value of 0.6-0.8 to balance the influence of model output and calibration results (0.8 for scenarios with high model output reliability, and 0.6 for scenarios with large data distribution fluctuations). The output value of the LSTM model incorporating spatiotemporal weights, namely the improved spatiotemporal correlation dynamic calibration LSTM model, reflects the initial judgment on the degree of anomaly in the current data. This is the mean of the CP values, reflecting the degree of anomaly in the current data relative to the calibration set. Anomaly Score The value range is [0,1], and the higher the score, the greater the possibility of communication anomalies. This anomaly scoring method integrates model prediction and statistical calibration results. Compared with the traditional single model output score, it can effectively reduce the false alarm rate caused by distribution drift and improve the accuracy of anomaly scoring. Step C4, Tiered Early Warning: Based on anomaly scoring The value is set to a three-level warning threshold to achieve tiered warning. The specific threshold division is as follows: Level 1 warning (minor anomaly): The corresponding communication data shows slight fluctuations, which may be due to temporary noise interference. No downtime is required; simply alert the maintenance personnel. Level 2 Warning (Moderate Anomaly): The corresponding communication exhibits significant anomalies, leading to unstable data transmission. Maintenance personnel promptly investigated the equipment and network; Level 3 alert (severe anomaly): If a serious communication anomaly is detected, indicating equipment failure or malicious attack, the system should be shut down immediately for investigation to prevent industrial production accidents. After classification, the warning level, abnormal data, abnormal characteristics, and investigation suggestions will be transmitted to the warning push module, and the detection results will be transmitted to the cloud storage module for subsequent model iteration and optimization.

[0025] Cloud storage module: Adopting a cloud-native architecture, it supports distributed storage and rapid retrieval of multi-source data. Stored content includes raw communication data, preprocessed data, anomaly detection results, model parameters, and calibration data, providing data support for model iterative optimization. The specific implementation logic of this module is as follows: Step D1: Initial Configuration of Cloud-Native Architecture: Build a distributed storage architecture based on cloud-native technologies, completing basic configurations to adapt to the storage needs of ICPS multi-source data: Deploy distributed storage nodes. Based on the scale of the industrial scenario (e.g., industrial water treatment workshop, power dispatch station), configure several cloud storage nodes. The nodes are interconnected through a high-speed network to achieve distributed data backup and avoid data loss due to single point of failure; Containerize the storage service deployment, using Docker containers to encapsulate storage components and combining them with Kubernetes for container orchestration to achieve elastic scaling of storage resources (automatically expanding storage nodes when data volume surges and shrinking resources when data volume decreases), reducing operation and maintenance costs; Configure security protection policies, including data transmission encryption (using SSL / TLS protocol), storage encryption (AES-256 encryption algorithm), and access control (assigning read and write permissions according to roles, such as operation and maintenance personnel can only read data, while administrators can manage data), to prevent data leakage or tampering.

[0026] Step D2, Multi-source Data Reception and Classification: Establish a data receiving interface to synchronously receive data transmitted from various modules of the system, ensuring the real-time nature and integrity of data reception. Simultaneously, classify the data according to data type to adapt to subsequent storage and retrieval needs: Receive raw communication data transmitted from the data acquisition module (valid data after preliminary verification, including timestamps, device identifiers, protocol types, etc.); Receive preprocessed data transmitted from the multi-source heterogeneous communication data adaptive preprocessing module (standardized, noise-reduced unified format data, including characteristic information of various protocol data); Receive anomaly detection results (anomaly scores, warning levels, anomaly features, etc.), model parameters (ST-DC-LSTM model weights, filtering algorithm parameters, etc.), and calibration data (dynamic calibration set samples, CP value calculation results, etc.) transmitted from the spatiotemporal correlation dynamic calibration anomaly detection and hierarchical early warning module; Classify and label the received data, marking the data source (corresponding module), data type (raw data / preprocessed data, etc.), device identifier, timestamp, and other key information to avoid data confusion.

[0027] Step D3, Data Storage and Partition Management: A partitioned storage strategy is adopted, combining data type, time dimension, and device type for partitioning to improve storage efficiency and retrieval speed. Specific partitioning methods are as follows: Partitioning by data type: Divided into raw data area, preprocessed data area, anomaly detection result area, model parameter area, and calibration data area. Each partition is managed independently to avoid interference between different data types; Partitioning by time dimension: Within each data partition, sub-partitions are made by day / week (e.g., raw data area partitioned by day, preprocessed data area partitioned by week), adapting to the time-series characteristics of ICPS data and facilitating data retrieval by time range; Partitioning by device type: In the raw data area and preprocessed data area, further subdivisions are made by device type (PLC controller, sensor, industrial robot, etc.) and communication protocol (Modbus, OPC UA, MQTT) to ensure refined data classification. During storage, a distributed file system (such as HDFS) is used to store massive time-series data, and a relational database (such as MySQL) is used to store structured data (such as anomaly detection results and model parameters), balancing the efficient management of massive data storage and structured data.

[0028] Step D4, Data Index Construction and Rapid Retrieval: To meet the needs of rapid data traceability and analysis in industrial scenarios, a multi-dimensional index is constructed for the stored multi-source data to achieve efficient retrieval: Time Index Construction: Based on the data's timestamp, a time index is created for each partition of data, supporting rapid retrieval of communication data and detection results for the corresponding time period by time range (e.g., a specific day or week); Device and Protocol Index Construction: Based on device identifiers and communication protocol types, a device-protocol index is constructed, supporting retrieval of corresponding data by device and protocol type (e.g., retrieving all original communication data of a PLC controller); Feature Index Construction: For preprocessed data and anomaly detection results, a feature index is constructed based on data features (e.g., packet loss rate, transmission latency, anomaly score), supporting retrieval by feature conditions (e.g., retrieving severely abnormal data with anomaly scores ≥ 0.8). Simultaneously, the retrieval algorithm is optimized to ensure a single retrieval request response time ≤ 100ms, meeting the needs of maintenance personnel for rapid data traceability and fault diagnosis.

[0029] Step D5, Data Lifecycle Management: Based on the data retention needs of industrial scenarios, formulate a data lifecycle management strategy to achieve rational utilization of storage resources: Raw Data: The retention period is set at 6-12 months. Raw data exceeding the retention period will be compressed and archived after administrator confirmation (using the LZ4 compression algorithm, compression ratio ≥80%). Archived data can be restored as needed. Preprocessed Data and Anomaly Detection Results: The retention period is set at 12-24 months for subsequent fault tracing and data analysis. Model Parameters and Calibration Data: Long-term retention until model iteration updates. The latest three versions of parameters and calibration data will be retained, while older versions will be archived to ensure the traceability of model iterations. Regular data cleaning and verification will be performed to remove invalid and redundant data, check data integrity, and repair damaged data (based on distributed backup data) to ensure the quality of stored data.

[0030] Step D6: Data Support for Model Iteration and Optimization: Regularly push relevant data to the spatiotemporal correlation dynamic calibration anomaly detection and hierarchical early warning module to assist in model optimization: Export historical preprocessed data, anomaly detection results, and calibration data according to a preset cycle (e.g., weekly) and transmit them to the anomaly detection module for iterative training of the model; respond to data call requests from the anomaly detection module in real time, providing services such as model parameter query and historical calibration data retrieval to ensure that the model can quickly obtain the required data during dynamic calibration; store the new parameters and new calibration data after model iteration to form a model iteration archive, which facilitates subsequent comparative analysis of the performance of different version models and optimization of iteration strategies.

[0031] Early warning push module: Receives early warning information from the spatiotemporal correlation dynamic calibration anomaly detection and hierarchical early warning module. Based on the anomaly classification results, it pushes the early warning information to the corresponding level of operation and maintenance personnel and management platform through methods including but not limited to SMS, APP push, and platform pop-ups. Simultaneously, it records early warning logs for subsequent traceability. The specific implementation logic is as follows: Step E1, Module Initialization Configuration: Based on existing message push technology, complete the basic configuration of the early warning push module to ensure communication compatibility with the anomaly detection module and cloud storage module. Specific configuration content includes: Communication Interface Configuration: Build a dedicated communication interface with the spatiotemporal correlation dynamic calibration anomaly detection and hierarchical early warning module to ensure real-time reception of early warning information, supporting both batch reception and single message reception modes to avoid loss of early warning information; Push Channel Configuration: Enable three core push channels: SMS, APP push, and platform pop-up, and complete the parameter configuration for each channel—the SMS channel is bound to the mobile phone numbers of maintenance personnel and managers, the APP push channel is connected to the industrial maintenance dedicated APP, and the platform pop-up channel is connected to the display terminal of the workshop monitoring center and enterprise management platform; Hierarchical Push Rule Configuration: Preset the receiving objects and push channels corresponding to the three levels of early warning, fully synchronized with the hierarchical standards of the anomaly detection module to ensure the accuracy of hierarchical push, and configure a push timeout retry mechanism (timeout time set to 10 seconds, retries 3 times) to avoid push failure due to network fluctuations.

[0032] Step E2, Warning Information Reception and Parsing: Monitor the information output of the spatiotemporal correlation dynamic calibration anomaly detection and hierarchical warning module in real time, receiving the transmitted warning information to ensure the integrity and real-time nature of the information reception. After reception, parse the warning information to extract core key information, specifically including: warning level (Level 1, Level 2, Level 3), abnormal device identifier (e.g., PLC controller number, sensor model), anomaly timestamp, anomaly score, anomaly characteristics (e.g., excessive packet loss rate, abnormal transmission delay), and troubleshooting suggestions (targeted maintenance guidance based on anomaly characteristics). Organize the parsed information into standardized warning data packets to avoid information confusion and provide a clear data foundation for subsequent hierarchical push notifications and log recording.

[0033] Step E3, Tiered Matching and Recipient Confirmation: Based on the parsed warning level and the preset tiered push rules, match the corresponding recipients and push channels to ensure that the warning information is accurately pushed to the relevant responsible persons. The specific matching rules are as follows: Level 1 Warning (minor anomaly, ): The recipients are on-site maintenance personnel of the corresponding abnormal equipment. Push notifications are only enabled via the app to avoid excessive disruption; Level 2 alert (moderate abnormality, The recipients are on-site maintenance personnel and workshop monitoring center staff. The push notifications utilize both app push notifications and platform pop-ups to ensure staff are aware in real time. Level 3 alerts (serious anomalies, ...) The recipients are on-site maintenance personnel, workshop monitoring center staff, and enterprise management platform administrators. The push notification channels utilize SMS, app push notifications, and platform pop-ups to achieve comprehensive multi-channel coverage and ensure immediate response from relevant personnel. Simultaneously, the communication status of all recipients is verified (e.g., phone number validity, app online status, platform terminal online status). For recipients with communication issues, the push notification is redirected to a backup channel (e.g., if a maintenance personnel's phone number is unreachable, the notification is sent to their backup phone number).

[0034] Step E4, Multi-channel Precise Push: Based on the hierarchical matching results, activate the corresponding push channels to push standardized early warning data packets to the recipients. During the push process, ensure the integrity and readability of the information. Specific push specifications are as follows: SMS Push: Concise and clear content, including the early warning level, abnormal device, abnormal time, core abnormal characteristics, and emergency contact information, adapted for mobile reading scenarios; APP Push: Includes complete early warning information (including investigation suggestions and abnormal data trend charts), supporting online confirmation of receipt and feedback on processing progress by maintenance personnel; Platform Pop-up: A prominent early warning prompt pops up on the display terminals of the workshop monitoring center and enterprise management platform, indicating the early warning level (blue for level 1, yellow for level 2, and red for level 3), while playing a slight alert sound (an emergency alert sound is played for level 3 early warning) to attract the attention of on-duty personnel. After the push is completed, record the push status (success / failure) of each channel. For failed push tasks, trigger a timeout retry mechanism to ensure that the early warning information is effectively delivered.

[0035] Step E5: Warning Log Recording and Synchronization: Upon completion of the warning push, a standardized warning log is automatically generated. The log content must comprehensively cover all information throughout the warning process, specifically including: a unique warning identifier (for traceability), warning level, abnormal device information, abnormal time, abnormal score, abnormal characteristics, push channel, recipient, push status, and push time, ensuring log traceability. After log generation, the log is synchronized in real-time to the anomaly detection results area of ​​the cloud storage module, and stored in association with anomaly detection results and device communication data. This facilitates subsequent maintenance personnel in tracing the cause of the anomaly and reviewing the handling process, while also providing warning-related data support for model iteration and optimization.

[0036] Step E6: Push Result Feedback and Operational Closed Loop. Receive push confirmation messages and processing progress feedback from each recipient (e.g., operations personnel confirm receipt via the app, providing feedback such as "investigated" or "anomaly handled"). Synchronize this feedback to the cloud storage module in real time and update the processing status in the warning log. For Level 3 warnings, track processing progress in real time. If a preset processing time limit is exceeded (e.g., no processing progress feedback within 30 minutes), automatically push a reminder message to higher-level management personnel to ensure timely handling of serious anomalies. Once the anomaly is handled, receive the "anomaly cleared" signal from the anomaly detection module and automatically push an anomaly cleared notification to all recipients, completing the closed-loop process of warning push and operational handling.

[0037] This invention relates to the field of industrial communication detection and early warning, aiming to solve the problems of poor adaptability of multi-source heterogeneous data in ICPS, high false alarm rate of anomaly detection, and insufficient targeted early warning, and provides an intelligent system. This invention includes modules for data acquisition, adaptive preprocessing of multi-source heterogeneous communication data, spatiotemporal correlation dynamic calibration anomaly detection and hierarchical early warning, cloud storage, and early warning push. Multi-protocol communication data is collected through distributed edge nodes. After adaptive weighted threshold filtering preprocessing, an improved spatiotemporal correlation dynamic calibration LSTM model is used to extract spatiotemporal features, dynamically calibrate and quantify anomaly scores, and issue hierarchical early warnings according to three threshold levels. The warnings are pushed to corresponding maintenance personnel through multiple channels, and the data is synchronously stored in the cloud. This invention improves the accuracy and response efficiency of industrial communication anomaly detection, adapts to multi-protocol heterogeneous scenarios, provides a guarantee for the safe and stable operation of ICPS, and is applicable to industrial scenarios such as water treatment and power dispatching.

[0038] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A communication detection and early warning system based on big data, characterized in that, It includes a data acquisition module, a multi-source heterogeneous communication data adaptive preprocessing module, a spatiotemporal correlation dynamic calibration anomaly detection and hierarchical early warning module, a cloud storage module, and an early warning push module. The data acquisition module adopts a distributed acquisition method at edge nodes and is deployed on various industrial equipment terminals of ICPS to collect equipment communication data. It supports multi-protocol data acquisition, and the acquisition frequency is dynamically adjusted according to the equipment type. After the acquisition is completed, the raw data is transmitted to the multi-source heterogeneous communication data adaptive preprocessing module in real time. The multi-source heterogeneous communication data adaptive preprocessing module receives multi-source heterogeneous communication data transmitted by the data acquisition module, and uses an improved adaptive weighted threshold filtering algorithm to complete data classification, noise estimation, weighted filtering, and standardization. The processed data is then transmitted to the spatiotemporal correlation dynamic calibration anomaly detection and hierarchical early warning module. The spatiotemporal correlation dynamic calibration anomaly detection and hierarchical early warning module receives communication data preprocessed by the multi-source heterogeneous communication data adaptive preprocessing module, adopts an improved spatiotemporal correlation dynamic calibration LSTM model, completes spatiotemporal feature extraction, dynamic calibration, anomaly scoring, and hierarchical early warning, transmits the early warning information to the early warning push module, and transmits the detection results and preprocessed data to the cloud storage module. The cloud storage module adopts a cloud-native architecture, supports distributed storage and fast retrieval of multi-source data, and stores raw communication data, preprocessed data, anomaly detection results, model parameters, and calibration data. The early warning push module receives early warning information transmitted by the spatiotemporal correlation dynamic calibration anomaly detection and hierarchical early warning module. Based on the anomaly classification results, it pushes the early warning information to the corresponding level of operation and maintenance personnel and management platform through methods including but not limited to SMS, APP push, and platform pop-up windows. At the same time, it records the early warning log for easy follow-up.

2. The communication detection and early warning system based on big data according to claim 1, characterized in that: The specific implementation logic of the early warning data acquisition module is as follows: Step A1, Edge Acquisition Node Deployment: Based on the ICPS device distribution scenario, deploy edge acquisition nodes one by one with each industrial equipment terminal to ensure that each device has a dedicated acquisition node and achieve distributed full-coverage acquisition; Step A2, Adaptive Configuration of Acquisition Parameters: Based on device type, communication protocol, and industrial scenario requirements, complete the initial configuration of acquisition parameters to ensure the relevance and effectiveness of the acquired data. Specific configuration content includes: Protocol Adaptation Configuration: Configure a corresponding protocol parsing module for each edge acquisition node according to the communication protocol used by the device, namely Modbus, OPC UA, and MQTT; Acquisition Frequency Configuration: Dynamically set the acquisition frequency according to the importance of the device and the real-time requirements of the data; Acquisition Data Item Configuration: Define the acquisition data items for each device and unify the core communication parameters of the acquisition devices, including but not limited to data frames, transmission latency, packet loss rate, and signal strength. Step A3: Start Distributed Data Acquisition: After completing the parameter configuration, all edge acquisition nodes start acquisition synchronously, adopting a real-time sampling plus local caching mode to achieve continuous acquisition of raw communication data: Each acquisition node acquires the communication data of the corresponding device in real time according to a preset frequency, temporarily stores the raw data in the node's local cache area, and records the timestamp corresponding to each data frame to provide support for data time alignment in the subsequent preprocessing module; Step A4: Preliminary validity check of collected data: Before data transmission, the edge acquisition node performs a preliminary validity check on the locally cached raw data, removing obviously invalid data: data missing key fields; data exceeding reasonable range; duplicate data frames; after the check is completed, valid raw data is retained, and invalid data cache is cleared to ensure the quality of data transmitted to the preprocessing module. Step A5: Real-time transmission of raw data: The verified raw data is transmitted in real time from the edge acquisition node to the multi-source heterogeneous communication data adaptive preprocessing module via the industrial LAN. During the transmission, a combination of batch transmission and real-time retransmission is adopted: data is transmitted in batches every 10 acquisition time steps, and data packets lost during the transmission are retransmitted in real time to ensure the integrity and real-time performance of data transmission and avoid deviations in subsequent preprocessing and anomaly detection due to data loss.

3. The communication detection and early warning system based on big data according to claim 1, characterized in that: The specific implementation logic of the multi-source heterogeneous communication data adaptive preprocessing module is as follows: Step B1, Data Classification: Receive the raw communication data transmitted by the data acquisition module, classify it according to the communication protocol type, and extract the feature dimensions of each data type, including but not limited to data frame length, transmission delay, packet loss rate, signal strength, and checksum. Interpolate and supplement missing data to obtain the complete classified data set. ,in For Modbus protocol data, For OPC UA protocol data, For MQTT protocol data, each data set contains One sample Each feature dimension, i.e. , ; Step B2, Adaptive Noise Estimation: For each feature dimension of each type of protocol data, a fusion estimation method based on data fluctuation characteristics and protocol type weights is used to calculate the noise intensity of that dimension. The calculation formula is as follows: ,in For the first Class of communication data, the first Noise estimates for each feature dimension This is the weight for the volatility coefficient, with a value ranging from 0.3 to 0.

7. The standard deviation of the data in this dimension reflects the degree of data volatility. Protocol type weight; This is the mean of the absolute values ​​of the data in this dimension, which helps to reflect the overall intensity of the noise. Step B3, Weighted Threshold Filtering: Based on the noise estimate calculated in Step B2 Combining the importance weights of data features Calculate the weighted filtering threshold for each feature dimension. The formula is as follows: ,in, This is the threshold coefficient, with a value ranging from 2.5 to 3.5; during the filtering process, for each data sample... If it is equal to the mean of that feature dimension The absolute value of the difference is greater than If the difference is less than or equal to [value], it is considered noise data and replaced with the moving average of that feature dimension; if the absolute value of the difference is less than or equal to [value], it is considered noise data. If so, the original data will be retained; Step B4, Standardization Processing: For the filtered protocol data, an adaptive standardization method is used. Based on the value range and distribution characteristics of different protocol data, standardization processing is performed separately, mapping the data to the [0,1] interval. The formula is as follows: ,in, For the standardized first Class data, the first The sample, the first 1 eigenvalue, , The first Class Data The minimum and maximum values ​​of each feature dimension are determined; after standardization, various protocol data are integrated into a preprocessed data set in a unified format and transmitted to the spatiotemporal correlation dynamic calibration anomaly detection and hierarchical early warning module.

4. The communication detection and early warning system based on big data according to claim 1, characterized in that: The specific implementation logic of the spatiotemporal correlation dynamic calibration anomaly detection and hierarchical early warning module is as follows: Step C1, Spatiotemporal Feature Extraction: Receive preprocessed data transmitted by the multi-source heterogeneous communication data adaptive preprocessing module, first perform time dimension feature extraction, and use a sliding window to extract the temporal features of each device's communication data; Next, spatial dimension features are extracted. Based on the deployment location of the ICPS devices, device groups are divided, and the correlation between the current device and the communication data of other neighboring devices in the group is calculated to obtain spatial association features. Finally, by integrating temporal and spatial features, a spatiotemporal correlation feature vector is constructed. ,in For the current time step, For the spatiotemporal feature dimension, the formula for calculating the spatiotemporal correlation weight is: ,in, for Current device and The spatiotemporal correlation weights of devices in the time neighborhood The covariance of the two data sets reflects spatial correlation. , These are the variances of the two sets of data. This is the time decay coefficient, with a value ranging from 5 to 10; Step C2, Dynamic Calibration: A sliding window calibration mechanism is employed to construct a dynamic calibration set. Combined with the consistent form prediction (CP) method, the output of the improved spatiotemporal correlation dynamic calibration LSTM model is dynamically calibrated to avoid increased false alarm rates caused by distribution drift. Specifically, the sliding calibration window size is set to 50-100 time steps. The window dynamically moves forward over time. Each time it moves forward, abnormal samples within the window are removed, and normal samples are retained as the calibration set. ( (where is the number of samples in the calibration set). The CP value is calculated using the calibration set to correct the output of the improved spatiotemporal correlation dynamic calibration LSTM model, ensuring that the model can adapt to changes in data distribution. The calibration formula is as follows: ,in, This is an indicator function; when the condition inside the parentheses is true, ,otherwise ; To improve the spatiotemporal correlation dynamic calibration LSTM model, the calibration set was... The output value of each sample To improve the spatiotemporal correlation, dynamically calibrate the output value of the LSTM model for the current time step data; Step C3, Anomaly Scoring: The spatiotemporal correlation feature vector extracted in Step C1 is input into the improved spatiotemporal correlation dynamic calibration LSTM model to obtain the basic output value of the improved spatiotemporal correlation dynamic calibration LSTM model. Combined with the CP value calculated in Step C2, a weighted fusion method is used to obtain the anomaly score of the communication data at the current time step. The formula is as follows: ,in, The fusion weight is set to a value between 0.6 and 0.

8. The output value of the LSTM model incorporating spatiotemporal weights, namely the improved spatiotemporal correlation dynamic calibration LSTM model, reflects the initial judgment on the degree of anomaly in the current data. This is the mean of the CP values, reflecting the degree of anomaly in the current data relative to the calibration set. Anomaly Score The value range is [0,1]; Step C4, Tiered Early Warning: Based on anomaly scoring The value of is set to establish a three-level warning threshold, thereby achieving tiered warning. The specific threshold division is as follows: Level 1 warning: There are slight fluctuations in the corresponding communication data; please remind maintenance personnel to pay attention. Level 2 alert: The corresponding communication showed obvious abnormalities, resulting in unstable data transmission. Maintenance personnel promptly investigated the equipment and network; Level 3 alert: If a serious communication anomaly is detected, indicating equipment failure or malicious attack, the system should be shut down immediately for investigation to prevent industrial production accidents. After classification, the warning level, abnormal data, abnormal characteristics, and investigation suggestions will be transmitted to the warning push module, and the detection results will be transmitted to the cloud storage module.

Images