Industrial internet of things sensor data prediction method based on multi-modal data augmentation

Abstract

The application relates to the field of industrial big data, and relates to an industrial Internet of Things sensor data prediction method based on multi-modal data enhancement, which comprises the following steps: preprocessing industrial sensor data to obtain standard time series data and standard text data; performing linear projection on the standard time series data to obtain time series embedding features, and performing vector conversion on the standard text data to obtain text embedding features; obtaining independent enhanced time series embedding features, independent enhanced text embedding features, fusion enhanced time series embedding features, fusion enhanced text embedding features, time series trend embedding features, time series seasonality embedding features, text trend embedding features and text seasonality embedding features, and then performing cross-modal fusion to obtain cross-modal fusion features; and finally, the cross-modal fusion algorithm is used to predict the industrial sensor data to obtain the final industrial sensor data prediction result. The application can improve the prediction accuracy of the industrial Internet of Things sensor data based on multi-modal data enhancement.

Description

Technical Field

[0001] This invention belongs to the field of industrial big data technology, and in particular relates to an industrial Internet of Things sensor data prediction method based on multimodal data enhancement. Background Technology

[0002] Industrial sensor data prediction technology in the Industrial Internet of Things (IIoT) is a core support for intelligent manufacturing, equipment health management, and energy efficiency optimization. In industrial production environments, accurate prediction of time-series information such as equipment sensor data, process parameter flows, and energy consumption sequences enables key applications such as early fault warning, dynamic optimization of production plans, and intelligent resource scheduling, directly empowering industrial systems to reduce costs, increase efficiency, and upgrade decision-making. However, industrial sites generally face the severe challenge of a scarcity of high-quality training samples: on the one hand, newly commissioned equipment or new processes lack historical data accumulation; on the other hand, abnormal operating conditions such as equipment failures and sensor interruptions lead to fragmented effective data segments. Traditional single-modal time series models are significantly limited in such small-sample scenarios, struggling to capture the dynamic patterns of multi-factor coupling in complex industrial systems, severely restricting prediction accuracy and engineering practicality.

[0003] In general, for existing technical solutions, time series data comes from sensors and device logs in industrial Internet of Things, including but not limited to numerical data such as temperature, pressure, vibration frequency, etc. These data are arranged in chronological order and can reflect the operating state of devices or systems. Time series data usually has noise and missing values, and needs to be preprocessed to improve data quality. Text data comes from device maintenance logs, operation records, news reports, etc., and is unstructured data. Since text data contains rich semantic information, it can supplement the context information that is difficult to capture in time series data. Therefore, text data needs to be preprocessed, including operations such as word segmentation, stemming, and stop word removal. In the preprocessing module, operations such as normalizing time series data are performed, such as scaling the data to a specific range (such as 0 to 1); interpolating missing values to fill in the missing data. Some word segmentation operations are performed on text data to break sentences into words or phrases; stop words such as "de", "he" and other words without practical meaning are removed; stemming is performed to restore words to their root forms. In the feature extraction module, statistical features (such as mean, standard deviation) and periodic features (such as daily cycle, weekly cycle) of time series are extracted. These features can capture the basic rules and dynamic changes of time series. For text data, a pre-trained language model (such as BERT) is used to convert the text into an embedding vector. The embedding vector can capture the semantic information of the text and provide useful feature representations for the subsequent fusion module. Then, in the fusion module, time series features and text features are fused, usually using methods such as concatenation or weighted summation. Among them, the concatenation method directly connects the two features together to form a comprehensive feature vector; while the weighted summation method assigns different weights according to the importance of the features and then performs a summation operation. The fused features can synthesize time and semantic information and provide a more comprehensive input for the prediction module. Finally, in the prediction module, machine learning or deep learning models (such as LSTM, Transformer) are used to predict industrial sensor data. LSTM can capture the long-term dependencies of time series, and Transformer can process data in parallel and capture global features. The prediction module outputs the prediction results for future time steps, providing a basis for enterprise decision-making and resource optimization. The advantages of the existing technology are that it fuses multi-source data, fully utilizes multi-source information by combining time series and text data to improve prediction accuracy, and has strong expressive ability, that is, the pre-trained language model can capture the semantic information of the text, and the deep learning model can process complex time series patterns. However, there are still problems such as data scarcity, high model complexity, and overly simple fusion methods.In practical applications, it may be difficult to obtain a large amount of time series and text data for training. Furthermore, deep learning models (such as Transformer) have high computational complexity and high hardware resource requirements. At the same time, existing fusion methods (such as splicing and weighted summation) may not be able to fully explore the correlation and complementary information between different modal data, resulting in low prediction accuracy of industrial sensor data. Summary of the Invention

[0004] This invention provides a method and apparatus for predicting industrial IoT sensor data based on multimodal data augmentation, which can improve the accuracy of industrial sensor data prediction.

[0005] To achieve the above objectives, this invention provides an industrial IoT sensor data prediction method based on multimodal data augmentation, comprising:

[0006] Acquire industrial sensor data, and preprocess the time series data and text data in the industrial sensor data respectively to obtain standard time series data and standard text data;

[0007] The standard time series data is linearly projected to obtain time series embedding features, and the standard text data is vectorized using a pre-trained language model to obtain text embedding features.

[0008] The time series embedding features are input into a pre-built multimodal augmentation encoder for sample-level independent augmentation to obtain independently augmented time series embedding features. The text embedding features are input into a pre-built multimodal augmentation encoder for sample-level independent augmentation to obtain independently augmented text embedding features.

[0009] The time series embedding features and text embedding features are input into a pre-built multimodal augmented encoder for feature-level fusion augmentation to obtain fused features. The fused features are then segmented in the time series dimension to obtain fused augmented time series embedding features, and the fused features are then segmented in the text dimension to obtain fused augmented text embedding features.

[0010] The time series embedding features are decomposed into time series trend embedding features and time series seasonal embedding features by performing trend-seasonal decomposition. The text embedding features are decomposed into text trend embedding features and text seasonal embedding features by performing trend-seasonal decomposition.

[0011] The independent enhanced time series embedding features, fused enhanced time series embedding features, time series trend embedding features, and time series seasonal embedding features are fused with the independent enhanced text embedding features, fused enhanced text embedding features, text trend embedding features, and text seasonal embedding features to obtain cross-modal fused features. Based on the cross-modal fused features, a pre-built cross-modal fusion algorithm is used to predict industrial sensor data to obtain the final industrial sensor data prediction results.

[0012] To address the aforementioned problems, the present invention also provides an industrial IoT sensor data prediction device based on multimodal data augmentation, the device comprising:

[0013] The data acquisition module is used to acquire industrial sensor data and preprocess the time series data and text data in the industrial sensor data to obtain standard time series data and standard text data.

[0014] The feature processing module is used to linearly project standard time series data to obtain time series embedding features, and to perform vector transformation on standard text data using a pre-trained language model to obtain text embedding features. The time series embedding features are then input into a pre-built multimodal augmentation encoder for sample-level independent augmentation to obtain independently augmented time series embedding features. Similarly, the text embedding features are input into the pre-built multimodal augmentation encoder for sample-level independent augmentation to obtain independently augmented text embedding features. The time series and text embedding features are then input into the pre-built multimodal augmentation encoder for feature-level fusion augmentation to obtain fused features. These fused features are then segmented along the time series dimension to obtain fused augmented time series embedding features, and along the text dimension to obtain fused augmented text embedding features. Finally, the time series embedding features are decomposed into trend-seasonal features to obtain time series trend embedding features and time series seasonal embedding features, and the text embedding features are decomposed into trend-seasonal features to obtain text trend embedding features and text seasonal embedding features.

[0015] The industrial sensor data prediction module is used to perform cross-modal fusion of independent enhanced time series embedding features, fused enhanced time series embedding features, time series trend embedding features, and time series seasonal embedding features with independent enhanced text embedding features, fused enhanced text embedding features, text trend embedding features, and text seasonal embedding features to obtain cross-modal fused features. Based on the cross-modal fused features, a pre-built cross-modal fusion algorithm is used to predict industrial sensor data to obtain the final industrial sensor data prediction result.

[0016] To address the above problems, the present invention also provides an electronic device, the electronic device comprising:

[0017] At least one processor; and,

[0018] A memory communicatively connected to the at least one processor; wherein,

[0019] The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the above-described industrial IoT sensor data prediction method based on multimodal data augmentation.

[0020] To address the aforementioned issues, the present invention also provides a computer-readable storage medium storing at least one computer program, which is executed by a processor in an electronic device to implement the aforementioned method for predicting industrial IoT sensor data based on multimodal data augmentation.

[0021] This invention preprocesses time-series data and text data from industrial sensor data separately to ensure data quality and consistency. Standard time-series data is linearly projected to obtain time-series embedding features, and a pre-trained language model is used to perform vector transformation on standard text data to obtain text embedding features. This enhances feature representation while reducing computational burden. The time-series embedding features are input into a pre-built multimodal augmentation encoder for sample-level independent augmentation to obtain independently augmented time-series embedding features. Similarly, the text embedding features are input into a pre-built multimodal augmentation encoder for sample-level independent augmentation to obtain independently augmented text embedding features. Sample-level independent augmentation improves the robustness of both time-series and text embedding features. Furthermore, by further enhancing the time-series embedding... Features and text embedding features are input into a pre-built multimodal augmented encoder for feature-level fusion enhancement, which can achieve cross-modal fusion and extract deep correlations between time series and text, and extract complementary information. In addition, trend-seasonal decomposition of time series embedding features yields time series trend embedding features and time series seasonal embedding features, and trend-seasonal decomposition of text embedding features yields text trend embedding features and text seasonal embedding features, which can capture the systematic patterns of industrial sensor data changes over time. Finally, based on the cross-modal fusion features, a pre-built cross-modal fusion algorithm is used to predict industrial sensor data, and the final industrial sensor data prediction result is obtained. All augmented features can be integrated for high-order fusion and prediction, improving the prediction accuracy of industrial sensor data. Attached Figure Description

[0022] Figure 1 This is a flowchart illustrating an industrial IoT sensor data prediction method based on multimodal data augmentation, provided in an embodiment of the present invention.

[0023] Figure 2 A structural flowchart illustrating an example of an industrial IoT sensor data prediction method based on multimodal data augmentation provided in an embodiment of the present invention;

[0024] Figure 3 This is a schematic diagram illustrating the acquisition of time-series embedding features in an industrial IoT sensor data prediction method based on multimodal data augmentation, provided in an embodiment of the present invention.

[0025] Figure 4 A schematic diagram illustrating the working principle of the trend-seasonal decomposition module in an industrial IoT sensor data prediction method based on multimodal data augmentation, provided in an embodiment of the present invention.

[0026] Figure 5 This is a functional block diagram of an industrial IoT sensor data prediction device based on multimodal data augmentation, provided in an embodiment of the present invention.

[0027] Figure 6 This is a schematic diagram of the structure of an electronic device that implements the industrial Internet of Things sensor data prediction method based on multimodal data augmentation, according to an embodiment of the present invention.

[0028] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0029] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0030] This application provides an industrial IoT sensor data prediction method based on multimodal data augmentation. The execution entity of this method includes, but is not limited to, at least one of the following electronic devices that can be configured to execute the method provided in this application: a server, a terminal, etc. In other words, the method can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cluster of cloud servers. The server can be an independent server or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0031] Reference Figure 1The diagram shown is a flowchart illustrating an industrial IoT sensor data prediction method based on multimodal data augmentation according to an embodiment of the present invention. In this embodiment, the industrial IoT sensor data prediction method based on multimodal data augmentation includes:

[0032] S1. Acquire industrial sensor data, and preprocess the time series data and text data in the industrial sensor data to obtain standard time series data and standard text data.

[0033] Understandably, industrial sensor data refers to the collection of data generated by various sensors in the field of Industrial Internet of Things (IIoT). These sensors include, but are not limited to, temperature sensors and humidity sensors that measure the environment in which the IIoT system operates, temperature sensors that measure the operating temperature of equipment in the IIoT, and IIoT equipment status sensors, which are used to acquire the electrical power, current, or energy of the equipment.

[0034] Understandably, time-series data is a sequence of time-dependent data points collected by industrial sensors, with each data point associated with a timestamp. For example, time-series data may include ambient temperature time-series data, ambient humidity time-series data, device temperature time-series data for one or more devices, and device status time-series data for one or more devices.

[0035] Understandably, text data typically comes from the equipment's control system, log files, or manual records. For example, text data includes, but is not limited to, equipment maintenance logs, operation records, and fault reports.

[0036] Specifically, the time-series data and text data in the industrial sensor data are preprocessed to obtain standard time-series data and standard text data, including:

[0037] Preprocessing time series data includes: normalizing the time series data to obtain normalized time series data; imputing missing values ​​in the normalized time series data using linear interpolation to obtain complete time series data; and performing data consistency processing on the complete time series data to obtain the standard time series data.

[0038] Preprocessing of text data includes: cleaning the text data by characters, segmenting the cleaned text data into words to obtain a text word set; and standardizing the text word set to obtain standard text data.

[0039] For example, normalizing time series data to obtain normalized time series data can be achieved using the following implementation steps:

[0040] (1) For data whose distribution is approximately Gaussian (such as temperature), Z-Score standardization is adopted. For example, the numerical ranges of motor vibration sensors (unit: mm / s²) and temperature sensors (unit: °C) are very different, and they can only be compared after standardization.

[0041]

[0042] in, It is the sliding window mean. It is the standard deviation of the sliding window. The original time series data, This is standardized time series data.

[0043] (2) For data with clear boundaries (such as humidity readings of 0% to 100%), use Min-Max normalization. Note that this method is sensitive to outliers and outliers need to be processed before use to avoid being affected by noise.

[0044]

[0045] in, The minimum value of the sliding window. This represents the maximum value of the sliding window.

[0046] (3) Robust scaling is adopted for industrial data with outliers (such as instantaneous current spikes).

[0047]

[0048] in, This represents the median value in the sliding window, while This represents the interquartile range (75th percentile - 25th percentile).

[0049] Understandably, quartiles are an important concept in statistics used to describe the distribution and dispersion of data. They are particularly suitable for data with outliers or non-normal distributions, and can divide an ordered dataset into four roughly equal parts.

[0050] Understandably, the interquartile range (IQR) is IQR = Q3 - Q1, which measures the dispersion of the middle 50% of the main data.

[0051] For example, to impute missing values ​​in normalized time series data using linear interpolation to obtain complete time series data, the following implementation steps can be adopted:

[0052] In handling missing values, the most commonly used method is linear interpolation, when the missing point is located at time point... At the appointed time In between,

[0053]

[0054] in, The number of consecutive missing points. For at a certain point in time Missing values.

[0055] It is understood that, in embodiments of the present invention, when the data is a higher-order smoothed data, i.e. a continuous signal (such as flow velocity), spline interpolation can be used; or statistical filling methods, such as moving average (MA) and exponential smoothing (EMA), can be used.

[0056]

[0057] In the moving average method, Indicates the size of the movable window.

[0058]

[0059] In exponential smoothing, As a smoothing factor, [0,1], where, For at a certain point in time Historical forecast values ​​at that location For at a certain point in time The actual observed value at that location.

[0060] Understandably, the following implementation steps can be used to perform data consistency processing on complete time series data to obtain the standard time series data:

[0061] To ensure data consistency, methods such as timestamp alignment, logical consistency verification, and outlier detection (auxiliary consistency) can be adopted. When ensuring timestamp alignment, resampling can be performed, such as upsampling (low frequency → high frequency) or downsampling (high frequency → low frequency). When performing logical consistency verification, physical constraint rules can be established, such as the pump power P and flow rate Q satisfying certain conditions. If a deviation occurs, it is marked as abnormal, or a device status association is established. For example, if the motor speed is 0 in the "running" state, it is marked as contradictory data. The 3σ principle is often used in outlier detection; if a deviation occurs... This indicates an abnormality, or a dynamic threshold can be set to adjust the threshold based on the operating conditions (such as the temperature threshold increasing under full load).

[0062] S2. Linearly project the standard time series data to obtain time series embedding features, and use a pre-trained language model to perform vector transformation on the standard text data to obtain text embedding features.

[0063] Specifically, time series embedding features are obtained by linearly projecting standard time series data, including:

[0064] Time series embedding features are obtained by linearly projecting standard time series data, including:

[0065] Based on the preset time segment length and step size, the standard time series data is divided into multiple different time series data segments;

[0066] Time series data segments are mapped to a multidimensional latent space using a linear projection matrix to obtain time series embedding features.

[0067] Understandably, a linear projection matrix is ​​a mathematical tool that maps original time series segments to a low-dimensional latent space. It achieves linear transformation of data through matrix multiplication, with the aim of extracting key features and reducing feature dimensionality.

[0068] Understandably, the preset time segment length refers to the duration of each time series segment.

[0069] Understandably, step size refers to the time interval between the starting points of adjacent segments.

[0070] For example, the following implementation steps can be used to obtain time series embedding features by linearly projecting standard time series data:

[0071] For the time series data of the i-th channel:

[0072]

[0073] Given a preset time segment length P and step size S, divide X into multiple segments. For the first The total length of the time series for each channel is given, and each segment has a length of P, meaning it contains P consecutive observations. The number of segments N is calculated as follows:

[0074]

[0075] For example, time series X={ , ,…, If the segment length P=3 and the step size S=2, then the segmented segments N=4, specifically:

[0076] {{ , , },{ , , },{ , ,},{ , , }}

[0077] Each segment is projected through a linear projection matrix. Mapping to a d-dimensional latent space yields the time-series embedding feature representation. .

[0078]

[0079] in, It is the segmented matrix.

[0080] It is understood that in the embodiments of the present invention, a channel refers to multiple different sensors, with each sensor constituting one channel.

[0081] Specifically, text embedding features are obtained by vector transformation of standard text data using a pre-trained language model, including:

[0082] The text data corresponding to each time step of the text data is converted into a multi-dimensional embedding vector to obtain the text data embedding vector corresponding to each time step.

[0083] The text data embedding vectors corresponding to each time step are summarized to obtain the text data embedding matrix;

[0084] The text data is divided into multiple text data segments of the same length as the preset time segment, and the text data embedding vectors in the text data segments are concatenated to obtain the concatenated text data embedding vector.

[0085] Attention pooling is performed on the concatenated text data embedding vector, and the pooled concatenated text data embedding vector is mapped to a multidimensional latent space with the same dimension as the time series embedding features to obtain the text embedding features.

[0086] It is understood that a pre-trained language model refers to a language understanding and generation model that is pre-trained using large-scale unlabeled text. In this embodiment of the invention, the pre-trained language model includes, but is not limited to, the BERT model.

[0087] Understandably, attention pooling is a technique that aggregates sequential information by dynamically assigning weights, potentially merging input elements based on contextual importance.

[0088] For example, text embedding features can be obtained by vector transformation of standard text data using a pre-trained language model, and the following implementation steps can be adopted:

[0089] For text data:

[0090] Each text data is processed using a pre-trained language model. Convert to a The dimensional embedding vector is as follows:

[0091]

[0092] This yields the text embedding matrix for each time step t:

[0093]

[0094] in, For the first Time step in each channel The A text data, The output embedding dimension of the pre-trained language model. For time step The total number of text data.

[0095] Then, the text data is divided into multiple slices, each slice covering P time steps. For time ranges from... arrive (in All text data within this interval are collected, and the text embedding vectors are as follows:

[0096]

[0097] Here, ⊕ represents a vertical splicing operation.

[0098] Next, attention pooling is used to perform weighted aggregation of the text embedding vectors, as shown in the following formula:

[0099]

[0100] in, The first text parameter that can be learned. The second text parameter is learnable. For learnable third text parameters, For activation function,

[0101] Understandably, the learnable first, second, and third text parameters are used to weight the text segment of the i-th channel.

[0102] Finally, the attention-pooled text embedding vectors are mapped into a d-dimensional latent space using a learnable linear projection matrix. The text embedding feature representation is obtained. .

[0103] S3. Input the time series embedding features into the pre-built multimodal augmentation encoder for sample-level independent augmentation to obtain independently augmented time series embedding features. Input the text embedding features into the pre-built multimodal augmentation encoder for sample-level independent augmentation to obtain independently augmented text embedding features.

[0104] Understandably, a pre-built multimodal augmentation encoder refers to an encoder used to uniformly process data from multiple modalities, which can enhance and integrate complementary information from different modalities to generate more robust feature representations. For example, a multimodal augmentation encoder includes, but is not limited to, the Transformer Encoder architecture.

[0105] It is understandable that sample-level independent augmentation refers to performing single-modal augmentation on a specific sample type, without considering the correlation between multiple sample types. For example, in this embodiment of the invention, time series embedding features and text embedding features are augmented separately.

[0106] For example, inputting time series embedding features into a pre-built multimodal augmentation encoder for sample-level independent augmentation yields independently augmented time series embedding features, and inputting text embedding features into a pre-built multimodal augmentation encoder for sample-level independent augmentation yields independently augmented text embedding features. The following implementation steps can be adopted:

[0107] Embedsions of time series and text data are processed as independent training samples.

[0108]

[0109]

[0110] in, To independently enhance time series embedding features, To independently enhance text embedding features, A location embedding matrix for time series data is used to capture the location information of the time series. A position embedding matrix is ​​used to capture the positional information of the text data.

[0111] Understandably, by treating data from different modalities as independent samples, the diversity of training data is increased, thereby improving the model's generalization ability.

[0112] S4. Input the time series embedding features and text embedding features into the pre-built multimodal enhancement encoder for feature-level fusion enhancement to obtain fused features. Then, perform feature segmentation on the time series dimension to obtain fused enhanced time series embedding features, and perform feature segmentation on the text dimension to obtain fused enhanced text embedding features.

[0113] For example, the time series embedding features and text embedding features are input into a pre-built multimodal augmented encoder for feature-level fusion enhancement to obtain fused features. The fused features are then segmented along the time series dimension to obtain fused enhanced time series embedding features, and the fused features are further segmented along the text dimension to obtain fused enhanced text embedding features. The implementation steps can be as follows:

[0114] By concatenating the embeddings of time series and text data, cross-modal interaction can be facilitated.

[0115]

[0116] in, This indicates a splicing operation.

[0117] The concatenated input contains 2N fragments. To re-segment the concatenated representation into time series and text representations, the following method is used:

[0118]

[0119]

[0120] in, This is a splitting operation.

[0121] This invention enhances the model's ability to understand multimodal data by allowing feature interactions between time series and text data. After processing by the multimodal augmentation encoder, a cross-modal representation can be obtained, and its output includes: Cross-modal representation: ,in, To fuse and enhance time series embedding features, To enhance text embedding features, a multimodal augmented encoder can effectively integrate and enhance the representation of time series and text data, providing a rich information foundation for subsequent prediction tasks.

[0122] S5. Perform trend-seasonal decomposition on the time series embedding features to obtain time series trend embedding features and time series seasonal embedding features. Perform trend-seasonal decomposition on the text embedding features to obtain text trend embedding features and text seasonal embedding features.

[0123] Understandably, trend-seasonality is used to describe the systematic patterns in which data changes over time.

[0124] Specifically, trend-seasonal decomposition is performed on time series embedding features to obtain time series trend embedding features and time series seasonal embedding features, including trend-seasonal decomposition of time series embedding features using the moving average method.

[0125] For example, the following implementation steps can be used to perform trend-seasonal decomposition on time series embedded features using the moving average method;

[0126] Trend-Seasonal Decomposition of Time Series: Use standard trend-seasonal decomposition methods, such as moving averages.

[0127]

[0128]

[0129] in, It is a time series trend component. It is a seasonal component of the time series.

[0130] Specifically, the text embedding features are decomposed into trend-seasonal features to obtain text trend embedding features and text seasonal embedding features. This includes extracting text trend embedding features from text embedding features using a preset trend attention parameter, and extracting text seasonal embedding features from text embedding features using a preset seasonal attention parameter.

[0131] For example, extracting text seasonality embedding features from text embedding features using preset seasonality attention parameters can be achieved through the following implementation steps:

[0132] Trend-Seasonal Decomposition of Text Data: Using two different and independent sets of attention parameters to extract trend- and seasonally relevant text information.

[0133] For the trend components of text data:

[0134]

[0135] For the seasonal component of text data:

[0136]

[0137] in, The first attention parameter used for the trend component. This is the second attention parameter used for the trend component. This is the third attention parameter used for the trend component. As the first attention parameter for seasonal components, For the second attention parameter used for seasonal components, This is the third attention parameter used for seasonal components.

[0138] Furthermore, after trend-seasonal decomposition of the time series and the text data, the output of the joint trend-seasonal decomposition module is obtained:

[0139] (1) Embedding of trend components in time series:

[0140] (2) Embedding of seasonal components in time series:

[0141] (3) Embedding of trend components in text data:

[0142] (4) Embedding of seasonal components in text data:

[0143] Understandably, joint trend-seasonal decomposition can effectively extract trend and seasonal features from time series and text data, providing a richer information foundation for subsequent forecasting tasks.

[0144] S6. Perform cross-modal fusion of independent enhanced time series embedding features, fused enhanced time series embedding features, time series trend embedding features, time series seasonal embedding features, independent enhanced text embedding features, fused enhanced text embedding features, text trend embedding features, and text seasonal embedding features to obtain cross-modal fused features. Based on the cross-modal fused features, use the pre-built cross-modal fusion algorithm to predict industrial sensor data and obtain the final industrial sensor data prediction results.

[0145] Specifically, based on cross-modal fusion characteristics, a pre-built cross-modal fusion algorithm is used to predict industrial sensor data, resulting in the final industrial sensor data prediction results, including:

[0146] Based on the time series dimension and the text dimension, the independent enhanced time series embedding features, the fused enhanced time series embedding features, the time series trend embedding features, and the time series seasonal embedding features are combined with the independent enhanced text embedding features, the fused enhanced text embedding features, the text trend embedding features, and the text seasonal embedding features to obtain multiple sets of cross-modal fusion features;

[0147] Each set of cross-modal fusion features is used to predict industrial sensor data using a pre-built cross-modal fusion algorithm, and the prediction results of industrial sensor data corresponding to each set of cross-modal fusion features are summarized to obtain multiple sets of industrial sensor data prediction results.

[0148] The prediction results of each group of industrial sensor data are weighted and fused to obtain the final industrial sensor data prediction result.

[0149] For example, the following implementation steps can be used to predict industrial sensor data based on cross-modal fusion features and a pre-built cross-modal fusion algorithm to obtain the final industrial sensor data prediction result:

[0150] (1) Combinatorial representation: embedding time series trend components by combining them Embedding of seasonal components in time series All representations in the dataset are used to generate multiple predictions. Specifically, for each trend component representation... ∈ and each seasonal component represents ∈ They are combined together for prediction, where... , The specific formula is as follows:

[0151]

[0152] in, and It is a single-layer linear decoder used for trend and seasonal representation. This is the actual value. It is the loss for each channel i.

[0153] (2) Independent Prediction: Each combination of representations generates an independent prediction. Independent prediction allows the model to learn from a wider range of representation combinations, thus providing a more comprehensive understanding of the interaction between trend and seasonal components. Since data from different modalities may reflect different information and characteristics, independent prediction can better utilize the complementarity of this information. Each modality may offer different insights and characteristics on a particular dataset; therefore, utilizing independent prediction for each modality can enable the model to have a more comprehensive understanding of the underlying dynamics of the time series.

[0154] (3) Offline Synthesis: During the inference phase, a simple offline linear aggregation module is used to weight and fuse multiple independent prediction results to optimize the final prediction result based on the quality of the input modality. This method effectively improves the prediction stability and accuracy under multimodal data fusion. Simultaneously, the offline-trained prediction synthesis module can more effectively assign different weights to the results, avoiding interference with the prediction weights during model training, thereby improving prediction accuracy. The final prediction results are generated as follows:

[0155]

[0156] in, ,…, Here, b is the weight parameter, and b is the bias term. This is the kth independent prediction result. As the output of this module, it represents the final industrial sensor data prediction result. This result integrates representations from different modalities and components, resulting in more accurate and robust predictions of industrial sensor data.

[0157] This invention preprocesses time-series data and text data from industrial sensor data separately to ensure data quality and consistency. Standard time-series data is linearly projected to obtain time-series embedding features, and a pre-trained language model is used to perform vector transformation on standard text data to obtain text embedding features. This enhances feature representation while reducing computational burden. The time-series embedding features are input into a pre-built multimodal augmentation encoder for sample-level independent augmentation to obtain independently augmented time-series embedding features. Similarly, the text embedding features are input into a pre-built multimodal augmentation encoder for sample-level independent augmentation to obtain independently augmented text embedding features. Sample-level independent augmentation improves the robustness of both time-series and text embedding features. Furthermore, by further enhancing the time-series embedding... Features and text embedding features are input into a pre-built multimodal augmented encoder for feature-level fusion enhancement, which can achieve cross-modal fusion and extract deep correlations between time series and text, and extract complementary information. In addition, trend-seasonal decomposition of time series embedding features yields time series trend embedding features and time series seasonal embedding features, and trend-seasonal decomposition of text embedding features yields text trend embedding features and text seasonal embedding features, which can capture the systematic patterns of industrial sensor data changes over time. Finally, based on the cross-modal fusion features, a pre-built cross-modal fusion algorithm is used to predict industrial sensor data, and the final industrial sensor data prediction result is obtained. All augmented features can be integrated for high-order fusion and prediction, improving the prediction accuracy of industrial sensor data.

[0158] Reference Figure 2 The diagram shown is a structural flowchart of an example of an industrial IoT sensor data prediction method based on multimodal data augmentation provided in an embodiment of the present invention.

[0159] Reference Figure 3 The diagram shown is a schematic diagram of time series embedding feature acquisition for an industrial IoT sensor data prediction method based on multimodal data augmentation, provided in an embodiment of the present invention.

[0160] Reference Figure 4 The diagram shown illustrates the working principle of the trend-seasonal decomposition module in an industrial IoT sensor data prediction method based on multimodal data augmentation, according to an embodiment of the present invention.

[0161] like Figure 5 The diagram shown is a functional block diagram of an industrial IoT sensor data prediction device based on multimodal data augmentation, provided in an embodiment of the present invention.

[0162] The industrial IoT sensor data prediction device 100 based on multimodal data augmentation described in this invention can be installed in an electronic device. Depending on the functions implemented, the industrial IoT sensor data prediction device 100 based on multimodal data augmentation may include a data acquisition module 101, a feature processing module 102, and an industrial sensor data prediction module 103.

[0163] The module described in this invention can also be called a unit, which refers to a series of computer program segments that can be executed by the processor of an electronic device and can perform a fixed function, and are stored in the memory of the electronic device.

[0164] In this embodiment, the functions of each module / unit are as follows:

[0165] The data acquisition module 101 is used to acquire industrial sensor data and preprocess the time series data and text data in the industrial sensor data to obtain standard time series data and standard text data.

[0166] The feature processing module 102 is used to linearly project standard time series data to obtain time series embedding features, and use a pre-trained language model to perform vector transformation on standard text data to obtain text embedding features; input the time series embedding features into a pre-built multimodal augmentation encoder for sample-level independent augmentation to obtain independently augmented time series embedding features, and input the text embedding features into a pre-built multimodal augmentation encoder for sample-level independent augmentation to obtain independently augmented text embedding features; input the time series embedding features and text embedding features into a pre-built multimodal augmentation encoder for feature-level fusion augmentation to obtain fused features, and perform feature segmentation on the time series dimension to obtain fused augmented time series embedding features, and perform feature segmentation on the text dimension to obtain fused augmented text embedding features; perform trend-seasonal decomposition on the time series embedding features to obtain time series trend embedding features and time series seasonal embedding features, and perform trend-seasonal decomposition on the text embedding features to obtain text trend embedding features and text seasonal embedding features.

[0167] The industrial sensor data prediction module 103 is used to perform cross-modal fusion of independent enhanced time series embedding features, fused enhanced time series embedding features, time series trend embedding features, time series seasonal embedding features, independent enhanced text embedding features, fused enhanced text embedding features, text trend embedding features, and text seasonal embedding features to obtain cross-modal fused features. Based on the cross-modal fused features, a pre-built cross-modal fusion algorithm is used to predict industrial sensor data to obtain the final industrial sensor data prediction result.

[0168] like Figure 6The diagram shown is a schematic representation of an electronic device that implements an industrial Internet of Things (IoT) sensor data prediction method based on multimodal data augmentation, according to an embodiment of the present invention.

[0169] The electronic device may include a processor 10, a memory 11, a communication bus 12, and a communication interface 13. It may also include a computer program stored in the memory 11 and capable of running on the processor 10, such as a method program for predicting industrial IoT sensor data based on multimodal data augmentation.

[0170] In some embodiments, the processor 10 may be composed of integrated circuits, such as a single packaged integrated circuit or multiple integrated circuits with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The processor 10 is the control unit of the electronic device, connecting various components of the entire electronic device through various interfaces and lines. It executes programs or modules stored in the memory 11 (e.g., executing a program for predicting industrial IoT sensor data based on multimodal data augmentation), and calls data stored in the memory 11 to perform various functions of the electronic device and process data.

[0171] The memory 11 includes at least one type of readable storage medium, including flash memory, portable hard drive, multimedia card, card-type memory (e.g., SD or DX memory), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of an electronic device, such as a portable hard drive. In other embodiments, the memory 11 can be an external storage device of the electronic device, such as a plug-in portable hard drive, Smart Media Card (SMC), Secure Digital (SD) card, Flash Card, etc. Furthermore, the memory 11 can include both internal and external storage units of the electronic device. The memory 11 can be used not only to store application software and various types of data installed on the electronic device, such as the code of an industrial IoT sensor data prediction method program based on multimodal data augmentation, but also to temporarily store data that has been output or will be output.

[0172] The communication bus 12 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into an address bus, a data bus, a control bus, etc. The bus is configured to enable communication between the memory 11 and at least one processor 10, etc.

[0173] The communication interface 13 is used for communication between the aforementioned electronic device and other devices, including a network interface and a user interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a Wi-Fi interface, Bluetooth interface, etc.), typically used to establish communication connections between the electronic device and other electronic devices. The user interface may be a display, an input unit (such as a keyboard), or optionally, a standard wired or wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen, etc. The display may also be appropriately referred to as a screen or display unit, used to display information processed in the electronic device and to display a visual user interface.

[0174] Figure 6 Only electronic devices with components are shown; it will be understood by those skilled in the art that... Figure 6 The structure shown does not constitute a limitation on the electronic device and may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0175] For example, although not shown, the electronic device may also include a power supply (such as a battery) to power the various components. Preferably, the power supply can be logically connected to the at least one processor 10 through a power management device, thereby enabling functions such as charging management, discharging management, and power consumption management. The power supply may also include one or more DC or AC power supplies, recharging devices, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components. The electronic device may also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.

[0176] It should be understood that the embodiments described are for illustrative purposes only and are not limited to this structure in the scope of the patent application.

[0177] The program for predicting industrial IoT sensor data based on multimodal data augmentation, stored in the memory 11 of the electronic device, is a combination of multiple instructions. When run in the processor 10, it can achieve the following:

[0178] Acquire industrial sensor data, and preprocess the time series data and text data in the industrial sensor data respectively to obtain standard time series data and standard text data;

[0179] The standard time series data is linearly projected to obtain time series embedding features, and the standard text data is vectorized using a pre-trained language model to obtain text embedding features.

[0180] The time series embedding features are input into a pre-built multimodal augmentation encoder for sample-level independent augmentation to obtain independently augmented time series embedding features. The text embedding features are input into a pre-built multimodal augmentation encoder for sample-level independent augmentation to obtain independently augmented text embedding features.

[0181] The time series embedding features and text embedding features are input into a pre-built multimodal augmented encoder for feature-level fusion augmentation to obtain fused features. The fused features are then segmented in the time series dimension to obtain fused augmented time series embedding features, and the fused features are then segmented in the text dimension to obtain fused augmented text embedding features.

[0182] The time series embedding features are decomposed into time series trend embedding features and time series seasonal embedding features by performing trend-seasonal decomposition. The text embedding features are decomposed into text trend embedding features and text seasonal embedding features by performing trend-seasonal decomposition.

[0183] The independent enhanced time series embedding features, fused enhanced time series embedding features, time series trend embedding features, and time series seasonal embedding features are fused with the independent enhanced text embedding features, fused enhanced text embedding features, text trend embedding features, and text seasonal embedding features to obtain cross-modal fused features. Based on the cross-modal fused features, a pre-built cross-modal fusion algorithm is used to predict industrial sensor data to obtain the final industrial sensor data prediction results.

[0184] Specifically, the specific implementation method of the processor 10 for the above instructions can be referred to the description of the relevant steps in the corresponding embodiment of the accompanying drawings, and will not be repeated here.

[0185] Furthermore, if the modules / units integrated in the electronic device 1 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile. For example, the computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM).

[0186] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor of an electronic device, can perform the following:

[0187] Acquire industrial sensor data, and preprocess the time series data and text data in the industrial sensor data respectively to obtain standard time series data and standard text data;

[0188] The standard time series data is linearly projected to obtain time series embedding features, and the standard text data is vectorized using a pre-trained language model to obtain text embedding features.

[0189] The time series embedding features are input into a pre-built multimodal augmentation encoder for sample-level independent augmentation to obtain independently augmented time series embedding features. The text embedding features are input into a pre-built multimodal augmentation encoder for sample-level independent augmentation to obtain independently augmented text embedding features.

[0190] The time series embedding features and text embedding features are input into a pre-built multimodal augmented encoder for feature-level fusion augmentation to obtain fused features. The fused features are then segmented in the time series dimension to obtain fused augmented time series embedding features, and the fused features are then segmented in the text dimension to obtain fused augmented text embedding features.

[0191] The time series embedding features are decomposed into time series trend embedding features and time series seasonal embedding features by performing trend-seasonal decomposition. The text embedding features are decomposed into text trend embedding features and text seasonal embedding features by performing trend-seasonal decomposition.

[0192] The independent enhanced time series embedding features, fused enhanced time series embedding features, time series trend embedding features, and time series seasonal embedding features are fused with the independent enhanced text embedding features, fused enhanced text embedding features, text trend embedding features, and text seasonal embedding features to obtain cross-modal fused features. Based on the cross-modal fused features, a pre-built cross-modal fusion algorithm is used to predict industrial sensor data to obtain the final industrial sensor data prediction results.

[0193] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0194] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0195] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0196] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0197] Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within the invention. No appended diagram markings in the claims should be construed as limiting the scope of the claims.

[0198] The blockchain referred to in this invention is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked together using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include an underlying blockchain platform, a platform product service layer, and an application service layer.

[0199] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0200] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in a system claim may also be implemented by a single unit or device through software or hardware. The terms "first," "second," etc., are used to indicate names and do not indicate any specific order.

[0201] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for predicting industrial IoT sensor data based on multimodal data augmentation, characterized in that, The method includes: Acquire industrial sensor data, and preprocess the time series data and text data in the industrial sensor data respectively to obtain standard time series data and standard text data; The standard time series data is linearly projected to obtain time series embedding features, and a pre-trained language model is used to perform vector transformation on the standard text data to obtain text embedding features. The step of using a pre-trained language model to perform vector transformation on the standard text data to obtain text embedding features includes: The text data corresponding to each time step of the text data is converted into a multi-dimensional embedding vector to obtain the text data embedding vector corresponding to each time step. The text data embedding vectors corresponding to each time step are summarized to obtain the text data embedding matrix; The text data is divided into multiple text data segments of the same length as the preset time segment, and the text data embedding vectors in the text data segments are concatenated to obtain the concatenated text data embedding vector. Attention pooling is performed on the concatenated text data embedding vector, and the pooled concatenated text data embedding vector is mapped to a multidimensional latent space with the same dimension as the time series embedding features to obtain the text embedding features; The time series embedding features are input into a pre-built multimodal augmentation encoder for sample-level independent augmentation to obtain independently augmented time series embedding features. The text embedding features are input into a pre-built multimodal augmentation encoder for sample-level independent augmentation to obtain independently augmented text embedding features. The time series embedding features and text embedding features are input into a pre-built multimodal augmented encoder for feature-level fusion augmentation to obtain fused features. The fused features are then segmented in the time series dimension to obtain fused augmented time series embedding features, and the fused features are then segmented in the text dimension to obtain fused augmented text embedding features. The time series embedding features are decomposed into time series trend embedding features and time series seasonal embedding features by performing trend-seasonal decomposition. The text embedding features are decomposed into text trend embedding features and text seasonal embedding features by performing trend-seasonal decomposition. The independent enhanced time series embedding features, fused enhanced time series embedding features, time series trend embedding features, and time series seasonal embedding features are fused with the independent enhanced text embedding features, fused enhanced text embedding features, text trend embedding features, and text seasonal embedding features to obtain cross-modal fused features. Based on the cross-modal fused features, a pre-built cross-modal fusion algorithm is used to predict industrial sensor data to obtain the final industrial sensor data prediction results.

2. The industrial IoT sensor data prediction method based on multimodal data augmentation as described in claim 1, characterized in that, The preprocessing of time-series data and text data from industrial sensor data to obtain standard time-series data and standard text data includes: Preprocessing time series data includes: normalizing the time series data to obtain normalized time series data; imputing missing values ​​in the normalized time series data using linear interpolation to obtain complete time series data; and performing data consistency processing on the complete time series data to obtain the standard time series data. Preprocessing of text data includes: cleaning the text data by characters, segmenting the cleaned text data into words to obtain a text word set; and standardizing the text word set to obtain standard text data.

3. The industrial IoT sensor data prediction method based on multimodal data augmentation as described in claim 1, characterized in that, The step of linearly projecting standard time series data to obtain time series embedding features includes: Based on the preset time segment length and step size, the standard time series data is divided into multiple different time series data segments; Time series data segments are mapped to a multidimensional latent space using a linear projection matrix to obtain time series embedding features.

4. The industrial IoT sensor data prediction method based on multimodal data augmentation as described in claim 1, characterized in that, The process of performing trend-seasonal decomposition on time series embedding features to obtain time series trend embedding features and time series seasonal embedding features includes using the moving average method to perform trend-seasonal decomposition on time series embedding features.

5. The industrial IoT sensor data prediction method based on multimodal data augmentation as described in claim 1, characterized in that, The step of performing trend-seasonal decomposition on text embedding features to obtain text trend embedding features and text seasonal embedding features includes extracting text trend embedding features from text embedding features using preset trend attention parameters, and extracting text seasonal embedding features from text embedding features using preset seasonal attention parameters.

6. The industrial IoT sensor data prediction method based on multimodal data augmentation as described in any one of claims 1-5, characterized in that, The step of using a pre-built cross-modal fusion algorithm based on cross-modal fusion features to predict industrial sensor data and obtain the final industrial sensor data prediction result includes: Based on the time series dimension and the text dimension, the independent enhanced time series embedding features, the fused enhanced time series embedding features, the time series trend embedding features, and the time series seasonal embedding features are combined with the independent enhanced text embedding features, the fused enhanced text embedding features, the text trend embedding features, and the text seasonal embedding features to obtain multiple sets of cross-modal fusion features; Each set of cross-modal fusion features is used to predict industrial sensor data using a pre-built cross-modal fusion algorithm, and the prediction results of industrial sensor data corresponding to each set of cross-modal fusion features are summarized to obtain multiple sets of industrial sensor data prediction results. The prediction results of each group of industrial sensor data are weighted and fused to obtain the final industrial sensor data prediction result.

7. An industrial IoT sensor data prediction device based on multimodal data augmentation, characterized in that, The apparatus is used to implement the industrial IoT sensor data prediction method based on multimodal data augmentation as described in any one of claims 1 to 6, the apparatus comprising: The data acquisition module is used to acquire industrial sensor data and preprocess the time series data and text data in the industrial sensor data to obtain standard time series data and standard text data. The feature processing module is used to linearly project standard time series data to obtain time series embedding features, and to perform vector transformation on standard text data using a pre-trained language model to obtain text embedding features. The time series embedding features are then input into a pre-built multimodal augmentation encoder for sample-level independent augmentation to obtain independently augmented time series embedding features. Similarly, the text embedding features are input into the pre-built multimodal augmentation encoder for sample-level independent augmentation to obtain independently augmented text embedding features. The time series and text embedding features are then input into the pre-built multimodal augmentation encoder for feature-level fusion augmentation to obtain fused features. These fused features are then segmented along the time series dimension to obtain fused augmented time series embedding features, and along the text dimension to obtain fused augmented text embedding features. Finally, the time series embedding features are decomposed into trend-seasonal features to obtain time series trend embedding features and time series seasonal embedding features, and the text embedding features are decomposed into trend-seasonal features to obtain text trend embedding features and text seasonal embedding features. The industrial sensor data prediction module is used to perform cross-modal fusion of independent enhanced time series embedding features, fused enhanced time series embedding features, time series trend embedding features, and time series seasonal embedding features with independent enhanced text embedding features, fused enhanced text embedding features, text trend embedding features, and text seasonal embedding features to obtain cross-modal fused features. Based on the cross-modal fused features, a pre-built cross-modal fusion algorithm is used to predict industrial sensor data to obtain the final industrial sensor data prediction result.

8. An electronic device, characterized in that, The electronic device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the industrial IoT sensor data prediction method based on multimodal data augmentation as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the industrial IoT sensor data prediction method based on multimodal data augmentation as described in any one of claims 1 to 6.

Images