Data fusion method and processing system based on industrial internet

Abstract

The invention discloses a data fusion method and processing system based on the industrial internet, and relates to the technical field of industrial internet and edge intelligence. Stream segmentation processing is used for replacing global calculation, mutual information analysis and resampling are only carried out on latest data segments, time consumed by alignment of hundreds of data streams is compressed to a millisecond level, and the data fusion efficiency is improved. The real-time control requirement of the flexible production line is met; the dynamic offset generation is combined with the equipment sampling rate, the communication protocol and the type weight, the compensation intensity is adaptively adjusted, and alignment distortion caused by equipment heterogeneity is avoided; a noise perception mechanism is introduced, a gradient median and absolute deviation are utilized to construct a self-adaptive threshold value, pulse noise points are accurately marked, weight penalty is applied, the integrity of fault features is kept while interference is suppressed, and signal smoothness caused by traditional filtering is prevented; tensor decomposition under a federated learning framework supports distributed data privacy protection, and a noise suppression module preposed design avoids pollution to a core feature extraction process.

Description

Technical Field

[0001] The present invention relates to the field of industrial Internet and edge intelligence technology, and in particular to a data fusion method and processing system based on the industrial Internet. Background Art

[0002] In discrete manufacturing production lines, such as automotive welding and 3C electronics assembly, dozens to hundreds of industrial devices concurrently upload time series data at millisecond frequencies. The sampling cycles, communication protocols, and network jitter of different devices lead to timing offsets in multi-source data streams. In such scenarios, equipment status monitoring requires the integration of multi-dimensional data such as vibration and current. However, millisecond-level misalignments can mask the correlation between key events. For example, a phase misalignment between the robot arm's motion trajectory and the welding gun's current may lead to a false positive.

[0003] Most current solutions use the dynamic time warping (DTW) optimization algorithm combined with a sliding window mechanism. The specific process includes: intercepting the data stream in segments with a fixed duration, such as 200ms, executing an improved DTW within the window, such as FastDTW, to align multi-device data, and finally inputting the aligned segments into the fusion model. Although such solutions alleviate global offsets, the complexity of FastDTW is still O(n), and the alignment of 100 data streams takes more than 50ms. The fixed window cannot adapt to sudden network fluctuations.

[0004] When the number of devices surges in high-concurrency scenarios or network anomalies occur, the window truncation of existing solutions leads to cross-window event interruptions. The alignment algorithm relies on a preset similarity function and cannot adapt to sudden changes in device sampling rates. The contradiction between real-time requirements and computing resources is exacerbated. Such problems are particularly prominent during the dynamic reorganization of flexible production lines. Existing methods have not yet achieved millisecond-level dynamic adaptive fusion. Summary of the Invention

[0005] In view of the above existing problems, the present invention is proposed.

[0006] The present invention provides a data fusion method and processing system based on the industrial Internet to solve the problem that the data stream of industrial high-concurrency equipment has timing misalignment due to sampling rate differences and network jitter, and the existing dynamic time warping algorithm has high calculation delay and cannot adapt to sudden fluctuations.

[0007] In order to solve the above technical problems, the present invention provides the following technical solutions: In a first aspect, an embodiment of the present invention provides a data fusion method based on the industrial Internet, which includes: step S1, receiving concurrent data streams from multiple industrial devices, wherein the data streams include timestamps and device identifiers; Step S2, caching the data stream into an independent buffer according to the device identification; Step S3: Perform streaming segment timing alignment on the data in each buffer: a) Calculate the local mutual information between the latest data segment and the historical data in the buffer; b) generating a dynamic time offset according to the mutual information value; c) resampling and aligning the data segments based on the time offset; Step S4: input the aligned multi-device data into the fusion model to generate joint features.

[0008] As a preferred solution of the data fusion method based on the industrial Internet described in the present invention, the dynamic time offset generation in step S3 satisfies: The offset calculation formula is: offset = reference offset × (1-mutual information value), where the reference offset is the preset maximum tolerance offset value.

[0009] As a preferred solution of the data fusion method based on industrial Internet described in the present invention, in step S3, the dynamic time offset calculation in the streaming segment timing alignment includes: Within, the latest data segment is considered as a random variable , the corresponding historical segment is regarded as , calculate the normalized discrete mutual information, the formula is: , in, Representation device At the moment The mutual information of For device index, is the time index, is the number of histogram bins, for Fall into The joint probability of the bins, 、 are their respective marginal probabilities; For device sampling rate and protocol correction factors , giving the reference offset coefficient as follows: , in, For equipment The reference offset coefficient, is the protocol correction factor, Profinet takes , other protocols take For equipment Sampling rate; According to the inherent delay characteristics of different functional units, type weights are introduced: , in, is the equipment type weight, the value is determined based on empirical tests; Then the dynamic time offset and skip threshold are determined, and the final time compensation is located as follows: , in, To be applied to the latest segment Time compensation; like , , then directly pass the data segment, otherwise enter the sparse resampling window; where , is the skip interpolation threshold.

[0010] As a preferred solution of the data fusion method based on the industrial Internet described in the present invention, the resampling alignment in step S3 adopts the cubic spline interpolation method, and only the data within the offset time window is interpolated.

[0011] As a preferred solution of the data fusion method based on the industrial Internet described in the present invention, the fusion model in step S4 is a tensor decomposition model under the federated learning framework, and a noise detection module is deployed at the input end of the model: The noise detection module identifies impulse noise points by detecting gradient mutations; The noise points are subjected to weight penalty factors and then input into the tensor decomposition model.

[0012] As a preferred solution of the data fusion method based on industrial Internet described in the present invention, in step S4, after applying weight penalty factor to noise points and inputting into tensor decomposition model, each device Aligned data sequence Follow the following steps in sequence: For devices The number of aligned sample points in the current batch, unitless: With a unified sampling period Calculate the gradient magnitude of adjacent samples: , in, Representation device In the index The gradient amplitude at , is the sample index, For the Sample values, is the previous sample value, is the unified sampling period after alignment; Then to Performing robust statistics, we get: , ; in, is the median gradient, is the median absolute deviation of the gradient; The adaptive threshold is set as: , in, is the gradient mutation threshold, is the threshold amplification factor; Define the noise indicator function: , in, is the gradient mutation mark, 1 represents the impulse noise point; Apply weight penalty to detected noise points: , , in, is the sample weight, is the penalty coefficient, is the weighted sample value, As tensor decomposition model input.

[0013] In a second aspect, the present invention provides a data processing system based on the Industrial Internet, comprising: The device access layer is configured to connect to multi-protocol industrial devices and parse raw data streams; Stream processing engine, including: Buffer management module, which isolates and stores data according to device identification; Data alignment module, which performs streaming segment timing alignment operations; Integrate the computing layer and deploy the federated learning framework and noise suppression tensor decomposition model.

[0014] As a preferred solution of the data processing system based on the industrial Internet described in the present invention, the data alignment module includes: Mutual information calculation unit, which calculates the correlation of data segments in real time; A dynamic offset compensation unit outputs a time offset instruction based on mutual information; Sparse resampling unit, performs interpolation alignment according to offset instructions.

[0015] As a preferred solution of the data processing system based on the industrial Internet described in the present invention, the tensor decomposition model of the fusion computing layer includes: Noise marking unit, which locates impulse noise through real-time gradient analysis; The weighted decomposition unit assigns preset weight coefficients to noise points.

[0016] As a preferred solution of the data processing system based on the Industrial Internet described in the present invention, the device access layer supports protocols including OPC UA, Modbus, and Profinet, and performs semantic conflict resolution during protocol conversion: Calculate KL divergence based on the distribution of historical data of the device; When the KL divergence is greater than the preset threshold, the fusion weight is dynamically allocated according to the device confidence; The confidence allocation in semantic conflict resolution satisfies: Device confidence = 1 - the false alarm rate of the device in the past 24 hours. The fusion weight is normalized and allocated according to the confidence.

[0017] The beneficial effects of the present invention are as follows: the present invention replaces global calculation with streaming segmentation processing, only performs mutual information analysis and resampling on the latest data segment, compresses the time consumption of aligning hundreds of data streams to milliseconds, and meets the real-time control requirements of flexible production lines; dynamic offset generation combines the device sampling rate, communication protocol and type weight, adaptively adjusts the compensation strength, and avoids alignment distortion caused by device heterogeneity; introduces a noise perception mechanism to use the gradient median and absolute deviation to construct an adaptive threshold, accurately marks pulse noise points and applies weight penalties, while suppressing interference, retains the integrity of fault features, and prevents signal smoothing caused by traditional filtering; tensor decomposition under the federated learning framework supports distributed data privacy protection, and the noise suppression module is designed in advance to avoid contaminating the core feature extraction process.

[0018] The dynamic resolution of semantic conflicts in the present invention is based on KL divergence to quantify protocol differences, and combines the historical false alarm rate of the equipment to allocate fusion weights, eliminating implicit conflicts between protocols such as OPC UA and Modbus, and reducing manual maintenance costs. In addition, buffer isolation management and sparse resampling are used to reduce redundant calculations, and the stream processing engine is lightweight and deployed on edge nodes to alleviate cloud load pressure. Interpolation thresholds are skipped to avoid invalid operations under micro-shifts, thereby improving resource utilization. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0020] Figure 1 This is a flow chart of the data fusion method based on the industrial Internet in Example 1.

[0021] Figure 2 This is a schematic diagram of the framework of the data processing system based on the industrial Internet in Example 1. DETAILED DESCRIPTION

[0022] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0023] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0024] Secondly, the term "one embodiment" or "embodiment" herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in various places throughout this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive of other embodiments.

[0025] Example 1, with reference to Figure 1 and Figure 2 , this embodiment provides a data fusion method based on the industrial Internet, comprising the following steps: Step S1, receiving concurrent data streams from multiple industrial devices, the data streams including timestamps and device identifiers; Step S2, caching the data stream into an independent buffer according to the device identification; Step S3: Perform streaming segment timing alignment on the data in each buffer: a) Calculate the local mutual information between the latest data segment and the historical data in the buffer; b) generating a dynamic time offset according to the mutual information value; c) resampling and aligning the data segments based on the time offset; The dynamic time offset generation in step S3 satisfies: The offset calculation formula is: offset = reference offset × (1-mutual information value), where the reference offset is the preset maximum tolerance offset value; The rules for determining the base offset value include: For devices with a sampling rate ≥ 50 Hz, the reference offset = 20 ms; For devices with sampling rates < 50 Hz, the baseline offset = 50 ms; If the device communication protocol is Profinet, the baseline offset value is reduced by 40%; In step S3, the calculation of the dynamic time offset in the streaming segment timing alignment includes: In the sliding window Within, the latest data segment is considered as a random variable , the corresponding historical segment is regarded as , calculate the normalized discrete mutual information, the formula is: , in, Representation device At the moment The mutual information of For device index, is the time index, is the number of histogram bins, for Fall into The joint probability of the bins, 、 are their respective marginal probabilities; For device sampling rate and protocol correction factors , giving the reference offset coefficient as follows: , in, For equipment The reference offset coefficient, is the protocol correction factor, Profinet takes , other protocols take For equipment Sampling rate; According to the inherent delay characteristics of different functional units, type weights are introduced: , in, is the equipment type weight, the value is determined based on empirical tests; Then the dynamic time offset and skip threshold are determined, and the final time compensation is located as follows: , in, To be applied to the latest segment Time compensation; like , , then directly pass the data segment, otherwise enter the sparse resampling window; where , is the skip interpolation threshold.

[0026] Specifically, mutual information measures the correlation between the latest segment and historical segments using binned probabilities, avoiding any assumptions about signal amplitude and thus adapting to both analog and discrete scenarios. The reference offset coefficient introduces sampling rate and protocol factors to unify the measurement of physical layer jitter and network layer jitter. The type weight further distinguishes actuator hysteresis from controller feedback differences, making the compensation strategy robust to scenario changes. The resampling alignment in step S3 uses cubic spline interpolation, and only interpolates the data within the offset time window; Step S4, inputting the aligned multi-device data into the fusion model to generate joint features; The fusion model in step S4 is a tensor decomposition model under the federated learning framework, and a noise detection module is deployed at the model input: The noise detection module identifies impulse noise points through gradient mutation detection; Apply weight penalty factors to noise points and input them into tensor decomposition model; In step S4, when the noise points are input into the tensor decomposition model after applying the weight penalty factor, in order to suppress the impulse noise before tensor decomposition, each device Aligned data sequence Follow the following steps in sequence: For devices The number of aligned sample points in the current batch, unitless: With a unified sampling period Calculate the gradient magnitude of adjacent samples: , in, Representation device In the index The gradient amplitude at , is the sample index, For the Sample values, is the previous sample value, is the unified sampling period after alignment; Then to Performing robust statistics, we get: , ; in, is the median gradient, is the median absolute deviation of the gradient; The adaptive threshold is set as: , in, is the gradient mutation threshold, is the threshold amplification factor; Define the noise indicator function: , in, is the gradient mutation mark, 1 represents the impulse noise point; Apply weight penalty to detected noise points: , , in, is the sample weight, is the penalty coefficient, is the weighted sample value, As tensor decomposition model input.

[0027] Specifically, gradient mutation detection uses the median and the median of absolute deviation to construct a threshold, without making any prior assumptions about the signal distribution, and remains robust to industrial process data containing spike interference; the threshold amplification factor and the penalty coefficient are designed separately, and the noise suppression strength can be independently adjusted without recalibrating the gradient statistics; weighted samples avoid destroying the time series integrity through dense noise suppression rather than hard deletion.

[0028] This embodiment also provides a data processing system based on the Industrial Internet, including: The device access layer is configured to connect to multi-protocol industrial devices and parse raw data streams; Stream processing engine, including: Buffer management module, which isolates and stores data according to device identification; Data alignment module, which performs streaming segment timing alignment operations; The data alignment module includes: Mutual information calculation unit, which calculates the correlation of data segments in real time; A dynamic offset compensation unit outputs a time offset instruction based on mutual information; Sparse resampling unit, performs interpolation alignment according to offset instructions; Integrate the computing layer and deploy the federated learning framework and noise suppression tensor decomposition model; The tensor decomposition model of the fusion computing layer includes: Noise marking unit, which locates impulse noise through real-time gradient analysis; A weighted decomposition unit assigns a preset weight coefficient of less than 0.1 to noise points; The device access layer supports protocols including OPC UA, Modbus, and Profinet, and performs semantic conflict resolution during protocol conversion: Calculate KL divergence based on the distribution of historical data of the device; When the KL divergence is greater than the preset threshold, the fusion weight is dynamically allocated according to the device confidence; The confidence allocation in semantic conflict resolution satisfies: Device confidence = 1 - the device's false alarm rate in the past 24 hours. The fusion weight is normalized and allocated according to the confidence level. The above threshold parameters (e.g., 20 ms, 50 ms, 0.6, 1.0, 2 ms, etc.) are calibrated based on large-sample experimental statistics and industry experience on typical discrete manufacturing production lines, and can be equivalently adjusted by those skilled in the art according to actual scenarios through experiments or automated parameter adjustment methods (e.g., grid search, Bayesian optimization). The present invention is not limited to specific numerical values.

[0029] This embodiment addresses the problem of data stream timing misalignment caused by the concurrent uploading of data at millisecond frequencies by dozens to hundreds of industrial devices in discrete manufacturing production lines. A dynamic adaptive data fusion method and system based on the Industrial Internet is proposed. This method first isolates cached data by device identifier, calculates the local mutual information between the latest and historical segments within a sliding window, and combines the sampling rate, protocol correction factor, and device type weight to generate a time compensation that is inversely proportional to the correlation. When the compensation amount exceeds the skip threshold, cubic spline interpolation is used only for samples within the offset window, avoiding the O(n) complexity brought by global DTW.

[0030] The aligned multi-source data is fed into a tensor decomposition model within the federated learning framework. A gradient-MAD-based impulse noise detection module is deployed at the model entry point, assigning low-weight penalties to outliers before participating in feature decomposition, thereby balancing privacy, security, and robustness. The supporting system comprises a device access layer, a stream processing engine, and a fusion computing layer, encompassing functional units such as protocol parsing, mutual information calculation, dynamic compensation, sparse interpolation, and weighted tensor decomposition. Key threshold parameters are derived from large-sample statistics and can be adjusted through automated parameter tuning.

[0031] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A data fusion method based on industrial Internet, characterized in that: The following steps are involved: Step S1, receiving concurrent data streams from multiple industrial devices, wherein the data streams include timestamps and device identifiers; Step S2, caching the data stream into an independent buffer according to the device identification; Step S3: Perform streaming segment timing alignment on the data in each buffer: a) Calculate the local mutual information between the latest data segment and the historical data in the buffer; b) generating a dynamic time offset according to the mutual information value; c) resampling and aligning the data segments based on the time offset; Step S4: input the aligned multi-device data into the fusion model to generate joint features.

2. The data fusion method based on the industrial Internet according to claim 1, characterized in that: The dynamic time offset generation in step S3 satisfies: The offset calculation formula is: offset = reference offset × (1-mutual information value), where the reference offset is the preset maximum tolerance offset value.

3. The data fusion method based on the industrial Internet according to claim 2, characterized in that: In step S3, the dynamic time offset calculation in the streaming segment timing alignment includes: Within, the latest data segment is considered as a random variable , the corresponding historical segment is regarded as , calculate the normalized discrete mutual information, the formula is: ， in, Representation device At the moment The mutual information of For device index, is the time index, is the number of histogram bins, for Fall into The joint probability of the bins, 、 are their respective marginal probabilities; For device sampling rate and protocol correction factors , giving the reference offset coefficient as follows: ， in, For devices The reference offset coefficient, is the protocol correction factor, Profinet takes , other protocols take For devices Sampling rate; According to the inherent delay characteristics of different functional units, type weights are introduced: ， in, is the equipment type weight, the value is determined based on empirical tests; Then the dynamic time offset and skip threshold are determined, and the final time compensation is located as follows: ， in, To be applied to the latest segment Time compensation; like , , then directly pass the data segment, otherwise enter the sparse resampling window; where , is the skip interpolation threshold.

4. The data fusion method based on the industrial Internet according to claim 1, characterized in that: The resampling alignment in step S3 uses the cubic spline interpolation method, and only interpolates the data within the offset time window.

5. The data fusion method based on the industrial Internet according to claim 1, characterized in that: The fusion model in step S4 is a tensor decomposition model under the federated learning framework, and a noise detection module is deployed at the model input: The noise detection module identifies impulse noise points by detecting gradient mutations; The noise points are subjected to weight penalty factors and then input into the tensor decomposition model.

6. The data fusion method based on the industrial Internet according to claim 5, characterized in that: In step S4, when the noise points are input into the tensor decomposition model after applying the weight penalty factor, each device Aligned data sequence Follow the following steps in sequence: For devices The number of aligned sample points in the current batch, unitless: With a unified sampling period Calculate the gradient magnitude of adjacent samples: ， in, Representation device In the index The gradient amplitude at , is the sample index, For the Sample values, is the previous sample value, is the unified sampling period after alignment; Then to Performing robust statistics, we get: ， ； in, is the median gradient, is the median absolute deviation of the gradient; The adaptive threshold is set as: ， in, is the gradient mutation threshold, is the threshold amplification factor; Define the noise indicator function: ， in, is the gradient mutation mark, 1 represents the impulse noise point; Apply weight penalty to detected noise points: ， ， in, is the sample weight, is the penalty coefficient, is the weighted sample value, As tensor decomposition model input.

7. A data processing system based on the industrial Internet, based on the data fusion method based on the industrial Internet according to any one of claims 1 to 6, characterized in that: include: The device access layer is configured to connect to multi-protocol industrial devices and parse raw data streams; Stream processing engine, including: Buffer management module, which isolates and stores data according to device identification; Data alignment module, which performs streaming segment timing alignment operations; Integrate the computing layer and deploy the federated learning framework and noise suppression tensor decomposition model.

8. The data processing system based on the Industrial Internet according to claim 7, characterized in that: The data alignment module includes: Mutual information calculation unit, which calculates the correlation of data segments in real time; A dynamic offset compensation unit outputs a time offset instruction based on mutual information; Sparse resampling unit, performs interpolation alignment according to offset instructions.

9. The data processing system based on the Industrial Internet according to claim 7, characterized in that: The tensor decomposition model of the fusion computing layer includes: Noise marking unit, which locates impulse noise through real-time gradient analysis; The weighted decomposition unit assigns preset weight coefficients to noise points.

10. The data processing system based on the Industrial Internet according to claim 7, characterized in that: The device access layer supports protocols including OPC UA, Modbus, and Profinet, and performs semantic conflict resolution during protocol conversion: Calculate KL divergence based on the distribution of historical data of the device; When the KL divergence is greater than the preset threshold, the fusion weight is dynamically allocated according to the device confidence; The confidence allocation in semantic conflict resolution satisfies: Device confidence = 1 - the false alarm rate of the device in the past 24 hours. The fusion weight is normalized and allocated according to the confidence.

Images