A multi-source heterogeneous data stream real-time fusion and lightweight analysis method for IIoT

Abstract

This invention discloses a real-time fusion and lightweight analysis method for multi-source heterogeneous data streams in the Industrial Internet of Things (IIoT) field. The method includes: multi-source heterogeneous data stream access and protocol parsing; hierarchical preprocessing (noise filtering, standardization, and spatiotemporal alignment); multimodal feature extraction and adaptive fusion; lightweight analysis model construction and edge deployment; redundant data filtering and incremental updates; and result output and visualization. This invention adopts an edge-cloud collaborative architecture and solves the problems of poor real-time performance, low accuracy, and difficult edge deployment in existing IIoT multi-source heterogeneous data fusion technologies through hierarchical preprocessing, attention mechanism adaptive fusion, and knowledge distillation lightweight modeling. It achieves standardized processing, real-time fusion, and lightweight analysis of multi-source heterogeneous data, adapts to edge device resource constraints, improves industrial data processing efficiency and decision reliability, and is applicable to IIoT scenarios such as intelligent manufacturing and energy / chemical engineering.

Description

Technical Field

[0001] This invention relates to the field of industrial Internet of Things (IIoT) data processing technology, and in particular to a method for real-time fusion and lightweight analysis of multi-source heterogeneous data streams for IIoT. Background Technology

[0002] The Industrial Internet of Things (IIoT) enables the interconnection of all elements in an industrial scenario through sensors, smart devices, and industrial gateways, generating massive, multi-source, heterogeneous data streams. These streams encompass structured sensor data (time-series values ​​such as temperature, pressure, vibration, and current), semi-structured log data (equipment operation logs, alarm records), unstructured data (industrial images, voice, and video), and heterogeneous protocol data (Modbus, OPC UA, MQTT, CoAP, etc.). This data suffers from problems such as large differences in sampling frequencies, inconsistent formats, inconsistent spatiotemporal references, strong noise interference, and high redundancy. Traditional data processing methods face the following pain points: 1. Insufficient real-time performance: Traditional centralized cloud processing requires the transmission of massive amounts of raw data, resulting in high bandwidth consumption and large latency, which cannot meet the millisecond-level real-time analysis requirements of industrial scenarios; 2. Low fusion accuracy: The lack of a unified heterogeneous data representation mechanism makes it difficult to align multimodal features, easily leads to the loss of key information, and results in poor reliability of the fusion results; 3. Limited computing resources: Edge devices (industrial gateways, embedded terminals) have limited computing power, storage, and power consumption, making it difficult to deploy complex deep learning models and highlighting the contradiction between lightweight design and accuracy; 4. Redundancy and noise interference: Industrial field data has a lot of noise and high redundancy, and direct analysis is prone to model overfitting and low inference efficiency.

[0003] In existing technologies, some solutions employ edge computing for data preprocessing, but these only support single-modal data and cannot adapt to multi-source heterogeneous scenarios. Some fusion methods are based on deep learning, but the models are large and inference is time-consuming, making it difficult to run in real time at the edge. Other solutions filter redundant data using simple rules, lacking dynamic feature selection and adaptive fusion mechanisms, resulting in poor generalization. Therefore, there is an urgent need for a real-time fusion and lightweight analysis method for multi-source heterogeneous data streams in the IIoT, which can achieve unified representation, real-time fusion, lightweight analysis, and efficient edge deployment of multimodal data, balancing accuracy, real-time performance, and resource consumption. Summary of the Invention

[0004] 1. Purpose of the invention This invention aims to address the problems of poor real-time performance, low accuracy, difficult edge deployment, and severe redundant noise interference in the fusion of multi-source heterogeneous data streams in the existing IIoT technology. It provides a real-time fusion and lightweight analysis method for multi-source heterogeneous data streams in IIoT, realizing standardized preprocessing, spatiotemporal alignment, adaptive feature fusion, lightweight modeling, and real-time analysis of multi-source heterogeneous data. It adapts to the resource constraints of edge devices and improves the efficiency of industrial data processing and the reliability of decision-making.

[0005] 2. Technical Solution To achieve the above objectives, this invention provides a real-time fusion and lightweight analysis method for multi-source heterogeneous data streams in the IIoT field, comprising the following steps. The overall architecture adopts an edge-cloud collaborative model, with the edge serving as the core processing node and the cloud responsible for model training, updating, and global optimization: Step 1: Multi-source heterogeneous data stream access and protocol parsing • Access to multi-source heterogeneous data streams in IIoT scenarios, including: structured sensor data (temperature, pressure, vibration, current, etc.), semi-structured log data (device operation logs, alarm records), unstructured data (industrial images, voice), and heterogeneous protocol data (Modbus, OPC UA, MQTT, CoAP). • A multi-protocol adapter gateway is used to parse heterogeneous protocol data, extract metadata such as data payload, timestamp, device identifier, and data type, and convert it into a standardized intermediate format (such as JSON / Protobuf) to complete the initial data normalization.

[0006] Step 2: Layered preprocessing (noise filtering - normalization - spatiotemporal alignment) • Noise filtering and anomaly detection: Differentiated noise reduction strategies are adopted for different modalities of data: Numerical sensor data is denoised using sliding window wavelet thresholding + median filtering to remove high-frequency noise and impulse interference; Log data is filtered using regular expression matching + keyword filtering to remove invalid characters and redundant logs; Image data is denoised using Gaussian blur + edge detection to remove noise and retain key features; Anomalies are detected and marked based on the 3σ criterion + isolated forest algorithm to avoid outliers interfering with subsequent fusion.

[0007] • Data standardization and normalization: For numerical data, Z-score standardization (mean 0, variance 1) or Min-Max normalization (mapped to the [0,1] interval) is used to eliminate dimensional differences; for semi-structured log data, TF-IDF word vectors + semantic hashing are used to convert them into low-dimensional dense vectors; for image data, a lightweight CNN (MobileNetV2) is used to extract fixed-dimensional feature vectors to achieve same-dimensional representation of cross-modal data.

[0008] • Spatiotemporal benchmark unification and alignment: GPS / BeiDou timestamps are used as the global synchronization benchmark. Sliding time windows and linear interpolation are used to fill in missing time point data and correct clock drift between devices. A spatial correlation matrix is ​​constructed based on the spatial location information of the devices (such as workshop coordinates and device network topology) to achieve dual alignment in spatiotemporal dimensions and ensure time synchronization and spatial correlation consistency of multi-source data. Step 3: Multimodal Feature Extraction and Adaptive Fusion • Hierarchical feature extraction: Underlying features: Extract basic features from various data sources, such as mean, variance, peak factor, and spectral features of sensor data; word frequency and semantic vectors of log data; and texture, edge, and shape features of image data. High-level features: A lightweight attention mechanism is introduced to assign importance weights to low-level features, highlighting key features and suppressing redundant features, thereby generating high-level semantic features for each modality.

[0009] • Dynamic adaptive feature fusion: A multimodal feature fusion matrix is ​​constructed, and the fusion weights of each data source are dynamically adjusted based on attention weight and feature correlation analysis to solve the problem of decreased fusion accuracy caused by data quality fluctuations and changes in operating conditions. An improved sparse autoencoder is used to reduce the dimensionality of the fused features, remove redundant feature dimensions, retain core related features, and generate a low-dimensional fused feature vector, thereby reducing the amount of subsequent computation.

[0010] Step 4: Lightweight Analysis Model Construction and Edge Deployment • Lightweight model design: Employing a cloud-edge collaborative modeling strategy: Cloud-based: Train high-precision teacher models (such as ResNet-50, BERT-base) based on fused feature datasets to learn deep data correlation patterns; At the edge: Build lightweight student models (such as MobileNetV3, SqueezeNet, and lightweight Transformer). Through knowledge distillation, model pruning, and quantization compression techniques, knowledge from the teacher model is transferred to the student model. At the same time, redundant network connections are removed and 32-bit floating-point parameters are quantized into 16-bit / 8-bit integers to achieve model miniaturization, low computing power, and low power consumption. Model optimization: Hardware-aware optimization is introduced to adapt to the computing power characteristics of edge devices (ARM architecture, FPGA), and inference is accelerated through TensorRT / ONNX Runtime to improve the running efficiency at the edge.

[0011] • Edge Deployment and Real-time Inference: Deploy lightweight student models to industrial gateways and embedded edge terminals, integrate feature streams, and achieve local real-time inference. Output analysis results such as device status monitoring, fault diagnosis, anomaly alarms, and quality prediction, with response latency controlled in milliseconds.

[0012] Step 5: Redundant Data Filtering and Incremental Update • Redundant feature filtering: Based on feature importance scores (such as XGBoost feature weights and mutual information), low-contribution redundant features are eliminated to build a streamlined feature library and reduce data storage and transmission overhead; • Incremental model update: New data is collected in real time at the edge, generating incremental feature samples, which are uploaded to the cloud periodically (e.g., hourly / daily). The cloud fine-tunes the teacher model based on the incremental samples and updates the model parameters. The updated teacher model is then distilled into a lightweight student model and distributed to the edge, enabling dynamic iteration and continuous optimization of the model to adapt to dynamic changes in industrial operating conditions.

[0013] Step 6: Results Output and Visualization • The edge device converts real-time analysis results (equipment status, fault type, anomaly level, and predicted value) into a standardized format and uploads them to the industrial platform via the MQTT protocol; • The platform provides a visual interface to display the data fusion process, feature distribution, analysis results, and equipment operation status. It supports alarm reminders, historical data review, and trend analysis to assist industrial decision-making.

[0014] 3. Beneficial effects Compared with the prior art, the present invention has the following advantages: 1. Strong real-time performance: Adopting an edge computing architecture, data preprocessing, fusion, and analysis are all completed at the edge, eliminating the need to transmit massive amounts of raw data, reducing bandwidth usage by more than 60%, and controlling response latency to the millisecond level, meeting the real-time requirements of industry. 2. High fusion accuracy: Through hierarchical preprocessing, spatiotemporal alignment, lightweight attention mechanism and adaptive fusion strategy, unified representation and accurate fusion of multi-source heterogeneous data are achieved, and the fusion accuracy is improved to over 92%, effectively suppressing noise and redundant interference; 3. Excellent edge adaptability: Based on knowledge distillation, model pruning, and quantization compression, the lightweight modeling method reduces the model size by more than 80% and improves the inference speed by more than 5 times. It can be efficiently deployed on edge devices with limited computing power, balancing accuracy and resource consumption. 4. Strong generalization and scalability: Supports multi-protocol and multi-modal data access, dynamically adapting to different industrial scenarios (intelligent manufacturing, energy and chemical industry, intelligent logistics); adopts modular design, which can flexibly expand data source types and analysis functions to meet the personalized needs of enterprises; 5. Low cost and high security: Reduces investment in cloud computing power and bandwidth leasing costs, while local data processing reduces the risk of raw data leakage and improves the security level of industrial data. Attached Figure Description

[0015] Figure 1 This invention relates to a system of steps and processes for a real-time fusion and lightweight analysis method of multi-source heterogeneous data streams for IIoT. Figure 1 .

[0016] Figure 2 This invention relates to a system of steps and processes for a real-time fusion and lightweight analysis method of multi-source heterogeneous data streams for IIoT. Figure 2 .

[0017] Figure 3 This invention relates to a system of steps and processes for a real-time fusion and lightweight analysis method of multi-source heterogeneous data streams for IIoT. Figure 3 .

[0018] Figure 4 This invention relates to a system of steps and processes for a real-time fusion and lightweight analysis method of multi-source heterogeneous data streams for IIoT. Figure 4 .

[0019] Figure 5 This invention relates to a system of steps and processes for a real-time fusion and lightweight analysis method of multi-source heterogeneous data streams for IIoT. Figure 5 .

[0020] Figure 6 This invention relates to a system of steps and processes for a real-time fusion and lightweight analysis method of multi-source heterogeneous data streams for IIoT. Figure 6 .

[0021] Figure 7 This invention relates to a system of steps and processes for a real-time fusion and lightweight analysis method of multi-source heterogeneous data streams for IIoT. Figure 7 .

[0022] Figure 8 This invention relates to a system of steps and processes for a real-time fusion and lightweight analysis method of multi-source heterogeneous data streams for IIoT. Figure 8 . Detailed Implementation

[0023] 1. System Architecture The architecture diagram of this invention for a real-time fusion and lightweight analysis system of multi-source heterogeneous data streams for IIoT adopts a three-layer collaborative architecture of edge-cloud: • Sensing end: including industrial sensors (temperature, pressure, vibration), smart devices, industrial cameras, and log servers, responsible for collecting multi-source heterogeneous data streams; • Edge: Includes multi-protocol adapter gateways and edge computing nodes (industrial gateways, embedded terminals), deploying data preprocessing, feature fusion, and lightweight analysis modules to achieve real-time data processing and local inference; • Cloud: Includes model training servers, data storage centers, and industrial management platforms, responsible for high-precision model training, knowledge distillation, model updates, data persistence, result visualization, and decision support.

[0024] 2. Detailed Implementation Steps Taking the health monitoring scenario of equipment in a smart manufacturing workshop as an example, the implementation process of the method of the present invention is as follows: Step 1: Multi-source heterogeneous data stream access and protocol parsing • Data collected by the sensing end: ① Structured sensor data: equipment vibration (200kHz sampling), motor current (1kHz sampling), bearing temperature (1Hz sampling); ② Semi-structured log data: equipment operation log, fault alarm record; ③ Unstructured data: equipment surface images collected by industrial cameras; ④ Heterogeneous protocol data: Modbus (sensor data), OPC UA (equipment status), MQTT (log data); • The edge-end multi-protocol adapter gateway parses Modbus / OPC UA / MQTT protocol data, extracts information such as timestamps, device IDs, data values, log content, and image frames, and converts them into JSON format to complete the initial data normalization.

[0025] Step 2: Layered Preprocessing • Noise filtering and anomaly detection: ① Vibration data: Sliding window wavelet threshold denoising (window size 256) + median filtering to remove high-frequency noise; ② Current data: 3σ criterion to detect outliers, marking outliers exceeding the mean ± 3σ; ③ Log data: Regular expression matching to remove invalid characters, retaining time, equipment, and fault keywords; ④ Image data: Gaussian blur (3×3 convolution kernel) + edge detection to remove noise and retain surface cracks and wear features. • Data standardization and normalization: ① Vibration, current, and temperature data: Min-Max normalization mapping to the [0,1] interval; ② Log data: TF-IDF word vectors are converted into 128-dimensional dense vectors; ③ Image data: MobileNetV2 extracts 256-dimensional feature vectors to achieve cross-modal same-dimensional representation. • Spatiotemporal benchmark unification and alignment: Based on GPS timestamps, linear interpolation with a sliding time window (window size 1s) is used to complete missing time point data; a spatial association matrix is ​​constructed based on the workshop equipment network topology to ensure time synchronization of different data sources for the same equipment and consistent spatial association between adjacent equipment.

[0026] Step 3: Multimodal Feature Extraction and Adaptive Fusion • Hierarchical Feature Extraction: ① Low-level features: Extract peak factor, kurtosis, and spectral features from vibration data; extract mean, variance, and harmonic features from current data; extract trend slope and fluctuation amplitude from temperature data; extract keyword frequency and semantic features from log data; extract texture, edge, and shape features from image data; ② High-level features: A lightweight attention mechanism assigns weights to low-level features (e.g., 0.4 for vibration data, 0.3 for current, 0.15 for temperature, 0.1 for logs, and 0.05 for images) to highlight key features; • Dynamic adaptive feature fusion: Construct a multimodal fusion feature matrix and dynamically adjust attention weights (increase vibration and image weights when operating conditions are abnormal); improve the dimensionality reduction of the sparse autoencoder, remove redundant features, and generate a 128-dimensional low-dimensional fusion feature vector.

[0027] Step 4: Lightweight Analysis Model Construction and Edge Deployment • Cloud-based teacher model training: Historical fusion feature datasets (including labels for normal equipment, wear, cracks, and faults) are collected in the cloud to train the ResNet-50 teacher model, which learns the correlation between equipment health status and fusion features, achieving an accuracy of 96%. • Lightweight edge student model construction: A MobileNetV3 student model is built, and knowledge distillation transfers knowledge from the teacher model to the student model; redundant convolutional and fully connected layers are removed by model pruning; 8-bit quantization compression parameters are used to compress the model size from 98MB to 12MB, and the inference speed is improved by 6 times. • Edge deployment and real-time inference: Deploy the lightweight model to the industrial gateway, connect and integrate feature streams, infer the health status of the equipment in real time, and output four types of results: "normal, slight wear, heavy wear, and crack failure", with an inference latency of 8ms.

[0028] Step 5: Redundant Data Filtering and Incremental Update • Redundant feature filtering: Based on XGBoost feature importance scoring, redundant features with a contribution of less than 0.01 are removed, reducing the feature library from 128 dimensions to 64 dimensions and reducing storage and computational overhead; • Incremental model updates: New data is collected daily at the edge, and incremental feature samples are generated and uploaded to the cloud. The teacher model is fine-tuned in the cloud, and after updating the parameters, it is distilled again into a lightweight model and distributed to the edge. The model is iterated once a week to adapt to the dynamic changes in equipment wear and tear.

[0029] Step 6: Results Output and Visualization • The edge device uploads the analysis results (device ID, health status, confidence level, inference time) to the industrial platform via MQTT; • Platform visual interface display: real-time equipment status dashboard, vibration / current / temperature trend curves, fusion feature distribution, fault alarm list, historical health status backtracking, supports abnormal alarm pop-up reminders, assisting maintenance personnel in handling equipment faults in a timely manner.

[0030] 3. Effect Verification In this embodiment, the method of the present invention is compared with traditional cloud processing methods and single-modal edge analysis methods, and the results are as follows: As shown in the table above, the method of the present invention is significantly superior to existing methods in terms of real-time performance, fusion accuracy, model lightweighting, bandwidth usage, and deployment difficulty, and is fully adapted to the needs of multi-source heterogeneous data processing scenarios in the Industrial Internet of Things.

Claims

1. A IIoT-oriented multi-source heterogeneous data stream real-time fusion and lightweight analysis method, characterized in that, Includes the following steps: Step 1: Multi-source heterogeneous data stream access and protocol parsing: Access multi-source heterogeneous data streams in the Industrial Internet of Things (IIoT) scenario, use a multi-protocol adapter gateway to parse heterogeneous protocol data, extract metadata and convert it into a standardized intermediate format; Step 2: Layered preprocessing: Noise filtering and anomaly detection, data standardization and normalization, and spatiotemporal benchmark unification and alignment are performed sequentially to achieve multi-source data quality optimization and representation normalization; Step 3: Multimodal feature extraction and adaptive fusion: Layered extraction of basic features at the bottom level and semantic features at the top level; dynamic adjustment of fusion weights and dimensionality reduction based on a lightweight attention mechanism and an improved sparse autoencoder to generate a low-dimensional fusion feature vector. Step 4: Lightweight analysis model construction and edge deployment: A cloud-edge collaborative modeling strategy is adopted. A high-precision teacher model is trained in the cloud, and a lightweight student model is built at the edge through knowledge distillation, model pruning, and quantization compression. The model is then deployed to edge devices to achieve local real-time inference. Step 5: Redundant data filtering and incremental update: Redundant features are removed based on feature importance scores to build a simplified feature library; Incremental data is collected at the edge, and the model is dynamically updated and distributed in the cloud to achieve continuous model optimization; Step Six: Results Output and Visualization: The edge device uploads the analysis results to the industrial platform, and displays the data fusion process, analysis results, and equipment operation status through a visualization interface.

2. The method of claim 1, wherein, In step 1), the multi-source heterogeneous data stream includes structured sensor data, semi-structured log data, unstructured data, and heterogeneous protocol data; the heterogeneous protocols include Modbus, OPC UA, MQTT, and CoAP; and the standardized intermediate format is JSON or Protobuf.

3. The method of claim 1, wherein, In step 2), the noise filtering and anomaly detection specifically include: using sliding window wavelet threshold denoising + median filtering for numerical sensor data; using regular expression matching + keyword filtering for log data; using Gaussian blur + edge detection for image data; and detecting and marking abnormal data based on the 3σ criterion + isolated forest algorithm.

4. The method of claim 1, wherein, In step 2), the data standardization and normalization specifically include: numerical data is normalized using Z-score or Min-Max normalization; log data is converted into dense vectors using TF-IDF word vectors + semantic hashing; and image data is extracted using MobileNetV2 to extract fixed-dimensional feature vectors.

5. The method of claim 1, wherein, In step 2), the spatiotemporal benchmark unification and alignment specifically includes: using GPS / BeiDou timestamps as global synchronization benchmarks, and using sliding time windows + linear interpolation to complete missing data; constructing a spatial correlation matrix based on device spatial location information to achieve dual alignment in spatiotemporal dimensions.

6. The method of claim 1, wherein, In step 3), the hierarchical feature extraction specifically includes: extracting statistical features and spectral features of sensor data, word frequency and semantic features of log data, and texture and edge features of image data at the bottom level; and introducing a lightweight attention mechanism to assign feature importance weights to generate high-level semantic features at the top level.

7. The method of claim 1, wherein, In step 3), the dynamic adaptive feature fusion specifically includes: constructing a multimodal feature fusion matrix, dynamically adjusting the fusion weights based on attention weights and feature correlation analysis; and using an improved sparse autoencoder to reduce the dimensionality of the fused features and remove redundant feature dimensions.

8. The method according to claim 1, characterized in that, In step 4), the lightweight analysis model construction specifically includes: training a high-precision teacher model of ResNet-50 or BERT-based model in the cloud; constructing a student model of MobileNetV3, SqueezeNet or lightweight Transformer at the edge; transferring knowledge from the teacher model through knowledge distillation; and achieving model lightweighting by combining model pruning and 8-bit quantization compression.

9. The method according to claim 1, characterized in that, In step 4), the edge deployment and real-time inference specifically include: using TensorRT / ONNX Runtime to optimize model inference speed, deploying to industrial gateways or embedded edge terminals to achieve millisecond-level local real-time inference, and outputting device status monitoring, fault diagnosis, and abnormal alarm analysis results.

10. The method according to claim 1, characterized in that, In step 5), the redundant data filtering specifically includes: calculating feature importance scores based on XGBoost feature weights or mutual information, removing low-contribution redundant features, and constructing a simplified feature library; the incremental model update specifically includes: periodically uploading incremental feature samples at the edge, fine-tuning the teacher model in the cloud and distilling it into a lightweight model, and distributing it to the edge to achieve dynamic model iteration.

Images