Industrial Control Timing Data Abnormality Detection Method

Abstract

This application relates to the field of industrial control security technology and computer technology, and discloses an industrial control time series data anomaly detection method, including: effectively integrating a novel autoencoder, a generative adversarial network, and a self-supervised contrastive learning network. The novel autoencoder uses an improved long short-term memory network to model long-term dependencies by integrating long-term and short-term feature analysis techniques, and combines causal convolution and dilated convolution of a temporal convolutional network to extract short-term features. The self-attention mechanism is adopted in the last layer to accurately locate key features in the time series. A dynamic dual optimization strategy of the generative adversarial network and the self-supervised contrastive learning network is also introduced to optimize the parameters of the novel autoencoder, which can effectively identify various types of contaminated data. The generative adversarial network can further learn time series features and generate diverse anomaly sequences, significantly improving the detection accuracy, thereby not only improving the accuracy of anomaly detection but also enhancing the adaptability to complex industrial control environments.

Description

Technical Field

[0001] This application relates to the field of industrial control security technology and computer technology, and particularly to a method for detecting anomalies in industrial control time series data. Background Art

[0002] With the wide application of Industrial Control System (ICS) technology, in the industrial control environment, the time series data of network traffic generated, sensor states, and time series data of other key operation parameters have become important bases for monitoring and diagnosis. These data not only reflect the operating status of the equipment, but also carry a large amount of valuable information about the production process, environmental conditions, and operation behaviors, providing solid data support for real-time monitoring, fault prediction, and system optimization.

[0003] However, due to the complexity and diversity of the industrial environment, these time series data are often affected by various pollution factors such as sensor noise, periodic fluctuations, short-term data loss, and fluctuations caused by maintenance operations. These effects not only increase the difficulty of data analysis, but also pose higher requirements for accuracy and reliability. On this basis, the detection accuracy of abnormal data is also affected. Summary of the Invention

[0004] In view of this, this application provides a method for detecting anomalies in industrial control time series data. By dynamically adjusting the parameter values involved in the time series feature learning part, the purpose of effectively distinguishing various data pollutions and real anomalies is achieved, and the problems of decreased accuracy and weak adaptability of most time series detection methods in the face of multi-type data pollution are effectively solved.

[0005] According to one aspect of this application, a method for detecting anomalies in industrial control time series data is provided, which is applied to an enhanced neural network framework. The enhanced neural network framework includes a new autoencoder and an anomaly detection module. The new autoencoder includes a new encoder and a new decoder with a reciprocal structure. The new encoder includes a long short-term memory network, a temporal convolutional network, and a self-attention mechanism module. The method includes:

[0006] Obtain the industrial control time series data to be detected and input it into the enhanced neural network framework, and perform data anomaly detection through the following steps executed by the enhanced neural network framework:

[0007] Using each gate in the long short-term memory network with a gated mechanism, based on the activation value dynamic adjustment factor, cross-verify the optimal activation value corresponding to each gate of the industrial control time series data to be detected under various data pollution conditions, and adjust each gate through the optimal activation value and then output the long-term dependence features;

[0008] After dynamically adjusting the contribution of the residual part of the temporal convolutional network based on the residual connection scaling factor using the temporal convolutional network, short-term features are output.

[0009] The self-attention mechanism module uses the self-attention mechanism to locate key features in the short-term features and output them, where the output key features are characterized by the reconstructed time series and the low-dimensional feature representation form.

[0010] The novel decoder performs reverse decoding on the key features to obtain the industrial control time series data to be compared.

[0011] The anomaly detection module obtains the anomaly detection result by comparing the industrial control time series data to be detected and the industrial control time series data to be compared.

[0012] By means of the above technical solutions, an industrial control time series data anomaly detection method provided by the present application effectively integrates a novel autoencoder, a generative adversarial network, and a self-supervised contrast learning network to form an efficient and robust time series anomaly detection model, and effectively solves problems such as weak adaptability, single application scenario, and decreased accuracy in the face of multi-type data pollution in industrial environment time series anomaly detection through a dynamic contrast loss mechanism.

[0013] The above description is only an overview of the technical solution of the present application. In order to be able to understand the technical means of the present application more clearly, it can be implemented according to the content of the specification. And in order to make the above and other purposes, features, and advantages of the present application more obvious and understandable, the specific embodiments of the present application are specifically given below. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The drawings described herein are used to provide a further understanding of the present application and form a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation to the present application. In the drawings:

[0015] Figure 1 Shows a schematic diagram of an enhanced neural network framework for applying the industrial control time series data anomaly detection method provided by an embodiment of the present application;

[0016] Figure 2 Shows a schematic diagram of a novel encoder in a novel autoencoder provided by an embodiment of the present application;

[0017] Figure 3 Shows a schematic diagram of a temporal convolutional network provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0018] The present application will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments. It should be noted that, without conflict, the embodiments in the present application and the features in the embodiments may be combined with each other.

[0019] In this embodiment, an industrial control timing data anomaly detection method is provided, which is applied to an enhanced neural network framework. The enhanced neural network framework includes a novel autoencoder and an anomaly detection module. The novel autoencoder is as Figure 1 shown, and includes a novel encoder 101 and a novel decoder 102 with a reciprocal structure; the novel encoder 101 is as Figure 2 shown and includes a long short-term memory network 1011, a temporal convolutional network 1012, and a self-attention mechanism module 1013; the method includes:

[0020] Obtain the industrial control timing data to be detected and input it into the enhanced neural network framework, so as to perform data anomaly detection by the enhanced neural network framework to execute the following steps:

[0021] Step 101, use the gates in the long short-term memory network with a gated mechanism, based on the activation value dynamic adjustment factor, cross-verify the optimal activation value corresponding to each gate of the industrial control timing data to be detected under various data contamination situations, and adjust each gate through the optimal activation value and then output the long-term dependence feature.

[0022] Step 102, use the gates in the long short-term memory network with a gated mechanism, based on the activation value dynamic adjustment factor, cross-verify the optimal activation value corresponding to each gate of the industrial control timing data to be detected under various data contamination situations, and adjust each gate through the optimal activation value and then output the long-term dependence feature.

[0023] Step 103, use the temporal convolutional network based on the residual connection scaling factor, dynamically adjust the contribution of the residual part of the temporal convolutional network according to the long-term dependence feature, and then output the short-term feature.

[0024] Step 104, the self-attention mechanism module uses the self-attention mechanism to locate the key features in the short-term features and output them. Among them, the output key features are characterized by the reconstructed time series and the low-dimensional feature representation form.

[0025] Step 105, the novel decoder performs reverse decoding on the key features to obtain the industrial control timing data to be compared.

[0026] Step 106, the anomaly detection module obtains the anomaly detection result by comparing the industrial control timing data to be detected and the industrial control timing data to be compared.

[0027] In the above embodiments of the present application, the industrial control time series data to be detected is obtained and input into the enhanced neural network framework, and the following steps are performed through the enhanced neural network framework to achieve data anomaly detection: Using each gate in the long short-term memory network with a gated mechanism, based on the activation value dynamic adjustment factor, cross-verify the optimal activation value corresponding to each gate of the industrial control time series data to be detected under various data contamination conditions, and adjust each gate through the optimal activation value and then output the long-term dependence features; Using the temporal convolutional network based on the residual connection scaling factor, after dynamically adjusting the contribution of the residual part of the temporal convolutional network according to the long-term dependence features, output the short-term features, that is, through the temporal convolutional network receiving the long-term dependence features output by the long short-term memory network, combining causal convolution and dilated convolution, constructing the short-term feature extraction part of the new encoder, ensuring that while capturing the short-term feature relationship, future information leakage is avoided. The self-attention mechanism module then uses the self-attention mechanism to locate the key features in the short-term features and output them. By introducing the self-attention mechanism, the contaminated features are dynamically identified and their weights are adjusted, thereby weakening the influence of noise and focusing on the key time segments with truly abnormal features.

[0028] Specifically, the output of the self-attention mechanism module can be , which is the feature dimension after the attention weight adjustment. Finally, the new encoder combines the features of short-term, long-term dependence, and self-attention mechanism, and outputs the reconstructed time series and its low-dimensional feature representation as . Then, the new decoder performs reverse decoding on the key features to obtain the industrial control time series data to be compared; the anomaly detection module then obtains the anomaly detection result by comparing the industrial control time series data to be detected and the industrial control time series data to be compared.

[0029] Industrial Control Time Series Data refers to a series of data points recorded over time in industrial automation and control systems. These data are usually used for monitoring, analyzing, and optimizing industrial processes. Specifically, industrial control time series data includes, for example:

[0030] 1. Sensor data, including: temperature sensors, which record the temperature changes of equipment or the environment. Pressure sensors, which monitor the pressure changes in pipelines, containers, or systems. Flow sensors, which measure the flow rate of liquids or gases. Humidity sensors, which record the humidity level in the air.

[0031] 2. Equipment status data, including: motor operating status, such as rotational speed, current, voltage, etc. Pump operating status, including flow rate, pressure, power, etc. Machine vibration data, which is used to monitor the mechanical health status of equipment.

[0032] 3. Process control data, including: controlling valve opening, recording the position or opening change of the valve. Liquid level control, such as the liquid level change in a storage tank or container. Parameters in a chemical reaction process, such as temperature, pressure, reactant concentration, etc.

[0033] 4. Energy management data, including: electricity consumption, recording the electricity usage in different time periods. Gas or steam consumption, monitoring the flow rate and consumption rate of gas or steam.

[0034] 5. Environmental monitoring data, including: air quality monitoring, such as PM2.5, PM10, carbon dioxide concentration, etc. Water quality monitoring, including pH value, dissolved oxygen, turbidity, etc.

[0035] 6. Production data, including: the output rate of the production line, recording the number of products produced per unit time. Product quality data, such as size, weight, defect rate, etc.

[0036] 7. Safety system data, including: fire alarm system, recording the time, location and status of fire alarms. Security access control system, monitoring the opening and closing status of access control and personnel entry and exit records.

[0037] Industrial control time series data can be collected through a data acquisition system (such as a SCADA system, PLC, sensor network, etc.) and stored in a database for subsequent analysis. By analyzing these industrial control time series data, engineers and operators can understand the operating status of industrial processes, identify potential problems, optimize production efficiency, and ensure the safety and reliability of the system.

[0038] For the new encoder 101, it can be constructed by combining long-term dependence modeling, short-term feature extraction and self-attention mechanism. The new encoder 101 can effectively distinguish various data contaminations and real anomalies, enhance the accuracy and robustness of anomaly sequence detection. The constructed new encoder is as Figure 2 shown.

[0039] Figure 2 The leftmost in... represents the input industrial control time series data x to be detected 0 ...x n, the time step (i.e., moment) t ranges from 0 to n, and the input industrial control time series data to be detected can be represented by (B, T, F), where B represents the batch size (BatchSize), T represents the time step length (Time Steps), and F represents the feature dimension (Feature Dimension). Next, there is a long short-term memory network (LSTM) 1011, which is used to process sequence data (industrial control time series data to be detected), has a memory and forgetting mechanism, can effectively capture long-term dependencies in the time series, and output long-term dependency features. Then, there is a temporal convolutional network (TCN) 1012, which is used to improve the deep learning performance of the network. It is a deep learning model specifically designed for processing time series data, and can effectively capture features in the time series. It is applicable to time series prediction and classification, especially suitable for time series prediction and sequence classification tasks. Through its unique convolutional operation, it can accurately predict future data points or classify sequences, and solve the problem of vanishing gradients in deep networks by introducing skip connections, enhancing the expressive power and training stability of the network. In particular, the temporal convolutional network 1012 also includes three residual sub-modules, namely the first residual sub-module 10121, the second residual sub-module 10122, and the third residual sub-module 10123, and their functions are similar, that is, to enhance the expressive power of the network. More specifically, the TCN captures local features in the time series through convolutional operations, and these features are crucial for understanding the dynamic behavior of the time series. By stacking multiple layers of convolutions, the TCN can capture more complex time dependencies, thereby improving the expressive power and prediction accuracy of the model and outputting short-term features. Next is the self-attention mechanism module 1013, which is used to focus on the key time segments that truly have abnormal features. Finally, through the fully connected layer, the key feature x is output 0 ...x n , the output key feature can be represented by (B, T, 32). In particular, the self-attention mechanism is a special form of the attention mechanism, which is mainly used to process the relationships between elements in sequence data or set data. In the self-attention mechanism, each element calculates the correlation with all other elements and assigns weights according to this correlation.

[0040] The gating mechanism of the long short-term memory network (LSTM) is its core design, mainly used to control the retention, update, and output of information. Specifically, the LSTM can include the following three key gating mechanisms:

[0041] 1. Forget Gate, which is responsible for determining which historical information to discard from the cell state. It generates a value between 0 and 1 through the sigmoid function, where 0 means complete forgetting and 1 means complete retention.

[0042] 2. Input Gate (or Update Gate), which is used to filter the new information in the current input that needs to be stored in the cell state. It consists of two parts: determining which information to update through the sigmoid function and generating candidate memory content through the tanh function.

[0043] 3. Output Gate, which controls how much information in the cell state needs to be exposed to the hidden state at the current moment, thereby affecting the final output.

[0044] In particular, the forget gate and the input gate work together to complete the update of the cell state, forming a dynamic control of long-term memory. This gating mechanism enables the LSTM to effectively solve the vanishing gradient problem of traditional RNN (Recurrent Neural Network) and capture long-term dependencies through the cell state.

[0045] To this end, by improving the long short-term memory network and adding an activation value dynamic adjustment factor, so that when the long short-term memory network uses the gating mechanism, it can cross-validate the best activation values corresponding to each gate for the industrial control time series data to be detected under various data contamination situations for each gate of the gating mechanism. After adjusting each gate through the best activation value, the long short-term memory network outputs long-term dependence features, which can enable the long short-term memory network to better learn features.

[0046] In particular, when detecting whether there are abnormalities in industrial control time series data, the data contamination situations can include the following:

[0047] 1. Sensor noise. Sensors may be subject to various random interferences during the measurement process, such as electromagnetic interference, temperature fluctuations, etc., resulting in deviations or fluctuations in the measurement data. This kind of noise can be manifested as random jitters or high-frequency fluctuations at data points, affecting the accuracy and reliability of the data.

[0048] 2. Periodic fluctuations. There are often periodic factors in the industrial production process, such as the periodic vibration of equipment operation, the periodic start and stop of production lines, etc. These periodic fluctuations may mask the true abnormal signals, making anomaly detection more difficult.

[0049] 3. Short-term data loss: Due to communication failures, sensor failures, or data transmission problems, some data may be lost in a short period of time. Data loss may cause breaks or discontinuities in time series data, affecting data integrity and the accuracy of analysis results.

[0050] 4. Fluctuations caused by maintenance operations: In industrial control systems, equipment maintenance and repair are inevitable. Maintenance operations may cause temporary changes in equipment status, thus introducing abnormal fluctuations or mutations in time series data.

[0051] Optionally, before using each gate in the long short-term memory network with a gating mechanism in step 101 to dynamically adjust the factor based on the activation value and cross-validate the optimal activation value corresponding to each gate for the industrial control time series data to be detected under various data contamination situations, and then adjusting each gate and outputting long-term dependence features through the optimal activation value, the method further includes:

[0052] Step 107, determining the data scenario to which the industrial control time series data to be detected belongs, where the data scenario includes a data high real-time requirement scenario, a data high contamination scenario, and a data insufficiency scenario.

[0053] Step 108, constructing the total loss function of the self-supervised contrastive learning network and the generative adversarial network, and by dynamically adjusting the loss dynamic adjustment factors corresponding to the self-supervised contrastive learning network and the generative adversarial network respectively in the constructed total loss function, optimizing the parameters of the new autoencoder, where the total loss function is:

[0054] ,

[0055] is the total loss function, is the contrastive loss of the self-supervised contrastive learning network, is the adversarial loss of the generative adversarial network, is the loss dynamic adjustment factor of the self-supervised contrastive learning network, is the loss dynamic adjustment factor of the generative adversarial network.

[0056] Step 109, when the data scenario to which the industrial control time series data to be detected belongs is a data high real-time requirement scenario, setting the loss dynamic adjustment factor of the generative adversarial network to the minimum preset threshold approaching 0; when the data scenario to which the industrial control time series data to be detected belongs is a data high contamination scenario, setting the loss dynamic adjustment factor of the self-supervised contrastive learning network to be less than the loss dynamic adjustment factor of the generative adversarial network; and when the data scenario to which the industrial control time series data to be detected belongs is a data insufficiency scenario, setting the loss dynamic adjustment factor of the generative adversarial network to 0.

[0057] In the above embodiments of the present application, to enhance the learning and discrimination capabilities for multi-type data contamination and abnormal patterns, an optimization mechanism with dynamic adaptation capabilities is designed for different data environments and detection requirements (scenarios with high real-time data requirements, scenarios with high data contamination, or scenarios with insufficient data). By dynamically adjusting the weight parameters of the contrast loss and the adversarial loss, and combining the generative adversarial network and the self-supervised contrast learning network, time series anomaly detection can be achieved in real and complex industrial control scenarios. By combining a novel autoencoder, a self-supervised contrast learning network, and a generative adversarial network for time series anomaly detection, it can well adapt to the problems of multi-type data contamination and anomaly identification in the industrial control environment, and effectively integrating the novel autoencoder, the generative adversarial network, and the self-supervised contrast learning network can form an efficient and robust time series anomaly detection model.

[0058] Autoencoders originate from the fields of artificial intelligence and machine learning and play an important role especially in the research of neural networks. An autoencoder is a neural network for unsupervised learning, whose main goal is to learn a compressed representation (i.e., encoding) of the input data, and at the same time be able to reconstruct the original data from this compressed representation (i.e., decoding). An autoencoder consists of two parts: an encoder and a decoder. The encoder converts the input data into a representation in the latent space (or called encoding), while the decoder tries to convert this representation back to the original data space to reconstruct the input data as accurately as possible. Through training, the autoencoder can learn an effective representation of the data, and this representation can be used for various tasks such as data dimensionality reduction, feature extraction, denoising, and anomaly detection.

[0059] In the above embodiments of the present application, by combining the long short-term memory network, the temporal convolutional network, and the self-attention mechanism to reconstruct the "novel autoencoder", and at the same time adding an activation value dynamic adjustment factor and a residual connection scaling factor for real-time dynamic adjustment, the constructed novel autoencoder can better learn the characteristics of industrial control time series data, thereby improving the subsequent anomaly detection accuracy for industrial control time series data.

[0060] In the self-supervised contrast learning network, self-supervised learning is a machine learning method that uses the inherent information in the data itself as a supervision signal to train the model without the need for manually labeled tags. Contrast learning is a technique in self-supervised learning that learns the representation of data by comparing similar and dissimilar samples. In contrast learning, the model is trained to distinguish positive samples (similar) and negative samples (dissimilar). The self-supervised contrast learning network combines the ideas of self-supervised learning and contrast learning, and uses unlabeled data to learn an effective representation of the data by comparing positive and negative samples.

[0061] In a generative adversarial network (GAN), the generative model is a type of machine learning model that can generate new data similar to the training data. Adversarial training is a training method that includes two models (a generator and a discriminator), and the two models compete with each other and co-evolve. Specifically, the generator attempts to generate realistic fake data, while the discriminator attempts to distinguish between real data and fake data. A generative adversarial network (GAN) is a neural network architecture composed of a generator and a discriminator, which generates new data similar to the real data distribution through adversarial training.

[0062] Specifically, generative adversarial networks (GANs) consist of two main parts: a generator and a discriminator. The generator attempts to generate realistic data samples, while the discriminator attempts to distinguish between real data and generated data. Through this adversarial training, the generator can gradually learn to generate more and more realistic data. Self-Supervised Contrastive Learning Networks, that is, networks that use self-supervised learning and contrastive learning techniques. Self-supervised learning is a learning method that does not require manually labeled data, and it uses the intrinsic information of the data itself to generate supervision signals. Contrastive learning is a method of self-supervised learning that learns the representation of data by comparing similar and dissimilar samples. In such a network, a certain form of contrastive loss function is usually used to optimize the model so that it can better distinguish between similar and dissimilar samples.

[0063] In the above embodiments of the present application, specifically, in the design of the total loss function, a loss dynamic adjustment factor and are constructed to separately adjust the weights of the contrastive loss and the adversarial loss . The overall (total) loss function is defined as follows: .

[0064] Wherein, and values are dynamically adjusted according to the characteristics of the input data and the requirements of the scenario. is used to optimize the feature representation ability and enhance the model's ability to distinguish complex abnormal patterns, and improve the model's robustness to data contamination and the ability to capture real anomalies through the generative adversarial network.

[0065] Furthermore, for scenarios with high real-time data requirements, that is, when the demand for detection results is to be output quickly, the reliance on generative adversarial training can be reduced (this part takes a long time but has higher accuracy), and only the self-supervised contrast learning network part is retained, that is, set , so that the enhanced neural network framework can fully focus on , thereby reducing the computational complexity and meeting the high real-time detection requirements.

[0066] For scenarios with highly polluted data, that is, when there is a large amount of polluted data and further improvement in detection accuracy is required without concerning about the time-consuming issue, can be set to increase the optimization weight for , strengthen the generative adversarial training, and thus enhance the robustness to data pollution.

[0067] For scenarios with insufficient data, the self-supervised contrast learning network can be fully adopted, that is, fully rely on the contrast loss for optimization, corresponding to set to 0.

[0068] Regarding the generative adversarial training network, the generative adversarial training network includes a generator and a discriminator. Among them, the generator shares the structure with the new encoder. This design not only simplifies the structure of the generator but also enables the generator to better capture the long-term and short-term dependencies in the time series through the shared feature extraction ability, generating abnormal samples that are more in line with the timing law. At the same time, a pure convolutional neural network is selected as the discriminator. The discriminator uses a multi-layer convolutional neural network for hierarchical time dependence detection to detect the features of short-term dependencies, and uses the loss function of the Wasserstein generative adversarial network (WGAN) for optimization. Different from the structure of the generator that combines the long short-term memory network and the temporal convolutional neural network, the discriminator is more focused on extracting short-term temporal dependencies through the multi-layer convolutional neural network, avoiding excessive sharing of structural information between the generator and the discriminator. The generative adversarial network part learns the time series features of various data pollutions and abnormal patterns, generates different types of abnormal samples through the generator, reduces overfitting to a certain type of feature, and improves the robustness of anomaly detection; at the same time, it can learn the time series change laws caused by different pollutions or faults from the adversarial training, and has stronger anomaly classification ability in the face of severely polluted situations. In particular, the parameter range of the long short-term memory network unit of the generator can be [32, 256], and the dilation rate setting of the discriminator can be [1, 2, 4], and the learning effects at different levels are determined through hyperparameter search.

[0069] Regarding the self-supervised contrastive learning network, the Euclidean distance between the features of abnormal samples is minimized through the contrastive loss function, while the distance between the features of normal samples and abnormal samples is maximized, guiding the new encoder to better learn the feature differences of different abnormal patterns and quickly improving the new encoder's learning ability for abnormalities. In addition to using contrastive learning alone, contrastive learning can also be combined with a generative adversarial network to compare the feature of the abnormal sample generated by the generator with the feature of normal data.

[0070] For this reason, the weights determined through dynamic search and can be set in the range of [0.1, 1.0]. If the real-time requirement is high, set = 0; if the data pollution is severe, set to enhance the contrastive learning ability.

[0071] Optionally, there are multiple types of gates, including input gates, forget gates, and output gates. In step 102, each gate in the long short-term memory network with a gated mechanism is used to dynamically adjust the factor based on the activation value, and cross-validate the optimal activation value corresponding to each gate of the industrial control time series data to be detected under various data pollution conditions. After adjusting each gate through the optimal activation value, long-term dependence features are output, specifically including:

[0072] Step 1021, for any one of the various gates in the gated mechanism, based on the activation value dynamic adjustment factor and the optimal activation value determination formula, cross-validate to obtain the optimal activation value corresponding to the gate of the industrial control time series data to be detected under various data pollution conditions, where the optimal activation value determination formula is:

[0073] ,

[0074] ,

[0075] is the optimal activation value of the gate at time t, is the weight matrix, is the industrial control time series data to be detected input at the current moment and the hidden state at the previous moment t - 1 concatenation, is the bias vector, is the optimal activation value of the gate adjusted based on the activation value dynamic adjustment factor, is the Sigmoid activation function that makes the optimal activation value between 0 and 1.

[0076] Step 1022, for any one of various gates, control the output of the gate through the gate control calculation formula corresponding to the gate and the optimal activation value of the gate until the outputs of various gates are respectively controlled, and then output the long-term dependence feature, where the gate control calculation formulas corresponding to different types of gates are respectively:

[0077] ,

[0078] ,

[0079] ,

[0080] ,

[0081] , , are respectively the input gate, forget gate, and output gate calculated through the Sigmoid activation function . is the candidate cell state calculated through the function. , , and are respectively the weight matrices of the input gate, forget gate, output gate, and candidate cell state. , , and are respectively the bias terms of the input gate, forget gate, output gate, and candidate cell state. is the input industrial control timing data at the current moment and the hidden state at the previous moment t - 1 , and are respectively the optimal activation values of the input gate, forget gate, and output gate at time t. is the Sigmoid activation function that makes the optimal activation value between 0 and 1.

[0082] Step 1023, utilize the information newly stored in the memory unit and the industrial control timing data to be detected input at the current moment, and output the long-term dependence feature through the output gate, where the output long-term dependence feature includes the information at the current moment and the long-term dependence feature at the previous moment.

[0083] In the above embodiments of the present application, a long short-term memory network with a gating mechanism is used to construct the long-term dependence modeling part in the new encoder, that is, the long short-term memory network 1011. This part introduces an activation value dynamic adjustment factor (the value range can be set to [0.1, 1.0]), and cross-validates the optimal activation value of the industrial control time series data to be detected under various data contamination conditions, so that the model (long short-term memory network) can select appropriate activation values for the input gate, output gate, and forget gate according to different types of data contamination, effectively avoiding over-reliance on contaminated data and reducing unnecessary noise interference. At the same time, the activation value dynamic adjustment factor will automatically adjust the activation value of each gate according to the current input features at each time step (moment) t. For example, when the temperature jumps from 25°C to 40°C, the activation value dynamic adjustment factor automatically adjusts, thereby reducing the impact of extreme values on the model (long short-term memory network).

[0084] In the gating calculation formulas corresponding to different types of gates, represents the input gate, which determines how much of the input information at the current moment will be incorporated into the cell state and is calculated through the sigmoid function (σ). This function outputs a value between 0 and 1, which is used to smoothly control the inflow of information. represents the forget gate, which determines how much information in the cell state at the previous moment will be forgotten. Similarly, this is also calculated through the sigmoid function, and outputs a value between 0 and 1, which is used to control the degree of information forgetting. represents the output gate, which determines how much information in the cell state at the current moment will be output to the hidden state. This is also calculated through the sigmoid function, and outputs a value between 0 and 1, which is used to control the output of information. is the candidate cell state, which is calculated through the tanh function. This function outputs a value between -1 and 1. The candidate cell state contains the comprehensive information of the input information at the current moment and the hidden state at the previous moment, and is the basis for updating the cell state. 、 、 and are weight matrices, which respectively correspond to the calculations of the input gate, forget gate, output gate, and candidate cell state. 、 、 and are bias terms, which are used to adjust the reference value of the calculation. 、 and respectively serve as the optimal activation values of the input gate, forget gate, and output gate, and can further fine-tune the opening degrees of the input gate, forget gate, and output gate.

[0085] By applying the above embodiments of the present application, the long short-term memory network (LSTM) 1011 can process industrial control time-series data more flexibly by constructing an activation value dynamic adjustment factor, especially in the face of data pollution. Specifically, the activation value dynamic adjustment factor automatically adjusts the optimal activation value of each gate (such as the input gate, forget gate, output gate) according to the current input features at each time step (moment) t. This dynamic adjustment helps the LSTM network better adapt to the data characteristics at different moments, especially when the data is polluted or there is noise. At the same time, through the cross-validation method, the LSTM network can optimize the activation values corresponding to each gate in the case of pollution to obtain the optimal activation values. This process aims to find the optimal activation value combination that can maintain the best performance for each gate under different degrees (situations) of data pollution. The core of the LSTM network lies in its ability to capture and maintain long-term dependence features. This benefits from its internal memory unit and gating mechanism. The memory unit is responsible for storing and updating information, while the gating mechanism (input gate, forget gate, output gate) controls the inflow, outflow, and forgetting of information. By dynamically adjusting the optimal activation values of these gates, the LSTM network can more precisely control the flow of information, thereby better capturing and maintaining long-term dependence features. When processing industrial control time-series data, the LSTM network gradually accumulates and updates information through its memory unit and gating mechanism. When it is necessary to output long-term dependence features, the LSTM network will use its internal state (i.e., the information in the memory unit) and the current input features to generate an output through the output gate. At the same time, this output not only contains the information at the current moment but also integrates the long-term dependence features of the previous moments.

[0086] In summary, by constructing an activation value dynamic adjustment factor and cross-validating the optimal activation values at each moment based on the activation value dynamic adjustment factor, the LSTM network can process industrial control time-series data more flexibly and accurately output its long-term dependence features. This mechanism enables the LSTM network to maintain high performance and stability in the face of data pollution and noise.

[0087] Optionally, the temporal convolutional network includes a first convolutional layer, a second convolutional layer, and a third convolutional layer, and the dilation rates corresponding to the first convolutional layer to the third convolutional layer increase layer by layer. Step 103 uses the temporal convolutional network based on the residual connection scaling factor to dynamically adjust the contribution of the residual part of the temporal convolutional network according to the long-term dependence features and then outputs short-term features, which specifically includes:

[0088] Step 1031, sequentially connect the first convolutional layer, the second convolutional layer, and the third convolutional layer through a dynamic residual, and use the residual connection scaling factor to dynamically adjust the contribution of the residual part of the temporal convolutional network according to the long-term dependence features and then output short-term features, where the output short-term features are represented by the following formula:

[0089] ,

[0090] is the short-term feature output after the contribution of the convolutional operation and the residual part of the temporal convolutional network is dynamically adjusted, is the long-term dependence feature at time t, is the weight of the convolutional kernel of the temporal convolutional network, is the dilation rate that determines the sampling interval when determining the wide-base operation, is the residual connection scaling factor. In the above embodiments of the present application, the temporal convolutional network (Convolutional Neural Networks, CNNs) is a deep learning model, especially suitable for processing data with a grid structure. Each layer of the temporal convolutional network is composed of a residual sub-module, and the dilation rate is increased layer by layer to enhance the receptive field and adapt to complex time series patterns. To optimize the gradient flow and improve the model performance, dynamic residual connections are used between layers, and the contribution of the residual part is dynamically adjusted through the residual connection scaling factor to distinguish abnormal short-term changes and effectively filter out the influence of noise, focusing on short-term features with actual abnormal patterns. In addition, by reasonably combining Layer Norm (layer normalization) and Batch Norm (batch normalization), and streamlining Dropout (random inactivation layer), the training stability and expression ability of the temporal convolutional network can be significantly improved. The temporal convolutional network is shown in

[0091] where the random inactivation layer is Dropout, the rectified linear unit variant is ReLU*, ReLU is Rectified Linear Unit, which is the rectified linear unit. The batch normalization is Batch Norm. The dilated causal convolution is Dilated Causal Conv, and the layer normalization is Layer Norm. Finally, the low-dimensional feature dimension output by the temporal convolutional network 1012 can be expressed as Figure 3 , , is the dimension of the extracted short-term feature.

[0092] Residual connection is a special design of the temporal convolutional network, first proposed in the Deep Residual Network (ResNet). By adding shortcuts (or skip connections) between certain layers of the network, residual connection allows the original input information to be directly passed to subsequent layers, thus helping to alleviate the problem of vanishing gradients or exploding gradients during the training of deep networks. A residual sub-module usually consists of two or more convolutional layers. The input is directly added to the output of these convolutional layers through the shortcut (or merged in other forms, such as concatenation), and then this result is used as the input for the next residual sub-module. This design enables the network to more easily learn the identity mapping, that is, if the network believes that certain layers do not need to change the input, it can directly learn to make the output of these layers close to the input.

[0093] Regarding Batch Normalization (Batch Norm), Batch Norm is a normalization technique used to improve the speed and stability of neural network training. It normalizes the features of each mini-batch of data so that the mean of each feature is 0 and the variance is 1, and then scales and offsets through learnable parameters to restore the representational ability of the data.

[0094] Regarding Layer Normalization (Layer Norm), Layer Norm is another normalization technique. It normalizes not in the batch dimension but in the feature dimension. This means it is independent of the batch size and is thus more suitable for scenarios such as mini-batches or recurrent neural networks (RNNs).

[0095] By applying the above embodiments of the present application, that is, by reasonably combining Layer Norm and Batch Norm, the performance of the model (temporal convolutional network) can be further improved. For example, using Batch Norm in some layers and Layer Norm in other layers (especially those layers that benefit from processing independent of the batch size). This combination can be adjusted according to the specific task and model architecture to achieve the best results.

[0096] Regarding the Dropout layer, Dropout is a regularization technique used to prevent neural network overfitting. By randomly discarding some neurons in the network during training (i.e., setting their outputs to 0), Dropout can force the network to learn more robust feature representations.

[0097] In the above implementation of this application, Dropout is streamlined. Streamlining the Dropout layer means using the Dropout layer carefully in the design to avoid overuse that may cause difficulties in model learning or performance degradation. Regarding streamlining the Dropout layer, it can be achieved by selecting an appropriate Dropout rate (i.e., the proportion of neurons to be discarded) and determining where to use Dropout in the network. In particular, Dropout can be used only in certain parts of the network (such as fully connected layers), while not used in other parts such as convolutional layers.

[0098] When outputting short-term features through a temporal convolutional network, that is, in the formula where represents the output features. This is the output feature result (short-term feature) after the convolutional operation of the temporal convolutional network and the contribution of the residual part are dynamically adjusted. is the long-term dependence feature at time t and is the original feature data processed by the temporal convolutional network. represents the weights of the convolutional kernels. These weights are used to extract specific patterns in the input features during the convolutional operation. d represents the dilation rate, which determines the sampling interval during the convolutional operation and can affect the scale and range of feature extraction. β is the residual connection scaling factor, which is used to introduce the dynamic residual part. It determines the weight of the residual connection, making the residual connection more flexible and helping to improve the training stability and expressive ability of the model. In particular, the dilation rate range of the temporal convolutional network can be {1, 2, 4}, and the dynamic adjustment range of the residual connection scaling factor can be set to [0.1, 1.0]. Random search is used for parameter adjustment, and the optimal value is found through experiments. For example, when processing power consumption data, the momentary startup of a device may cause the voltage to rise, which in turn causes the temperature reading to briefly soar to 50°C. The temporal convolutional network filters out this transient phenomenon by adjusting the dynamic trade-off of the importance of short-term features, ensuring that the model focuses on real abnormal patterns.

[0099] Optionally, the self-attention mechanism module in step 104 uses the self-attention mechanism to locate key features in the short-term features and output them, specifically including:

[0100] Step 1041, using the self-attention mechanism, dynamically identify the contaminated features in the short-term features, and adjust the attention weights of the identified contaminated features to obtain key features and output them.

[0101] In the above embodiments of the present application, the self-attention mechanism module 1013 can represent short-term features in the form of vectors or tensors, that is, the features can be time series data. Ensure that all features are in the same dimensional space for subsequent processing. Then, generate query (Query), key (Key), and value (Value) vectors for each feature, which can be achieved, for example, through linear transformation (such as a fully connected layer). Calculate the dot product of the query vector and all key vectors, and then apply the softmax function to obtain the attention scores. These scores represent the importance of each feature for the current query feature. By analyzing the attention scores, features with extremely high or low scores can be identified, which may be contaminated features or noise. Then, design a strategy to dynamically adjust the attention weights of the identified contaminated features. For example, lower weights can be assigned to these features, or the weights can be adjusted based on the comparison between their scores and a preset threshold. Apply the adjusted weights, with the weight range set to [0.0, 1.0], and recalculate the attention scores for each feature to ensure that key features receive more attention. Use the adjusted attention weights to perform weighted summation on the value vectors to obtain the weighted representation of each feature. Based on the weighted representation, select the features with higher weights and significant contributions to the task as key features. Integrate the selected key features into a compact representation, which can be, for example, a vector, matrix, or tensor, depending on the task requirements. To this end, output the integrated key features to subsequent processing layers or models for further analysis, prediction, or decision-making. In particular, the self-attention mechanism is usually implemented as a layer or module of a neural network, can be used in combination with other layers (such as convolutional layers, fully connected layers), and the model can also be trained through optimization algorithms such as backpropagation and gradient descent to minimize the loss function and improve performance. At the same time, regularization techniques (such as dropout, L2 regularization) can be applied to prevent overfitting and improve the generalization ability of the model.

[0102] Optionally, the anomaly detection module in step 106 obtains the anomaly detection result by comparing the industrial control time series data to be detected and the industrial control time series data to be compared, specifically including:

[0103] In step 1061, the anomaly detection module compares the industrial control time series data to be detected and the industrial control time series data to be compared. When the comparison result between the industrial control time series data to be detected and the industrial control time series data to be compared shows a difference exceeding the preset difference threshold, there is abnormal industrial control time series data.

[0104] In the above embodiments of the present application, by comparing the input and output, the difference between the input data and the decoded output data (such as MSE) can be calculated, and a threshold (preset difference threshold) can be set to determine whether the difference is significant. If the difference exceeds the threshold, it is determined as abnormal; otherwise, it is normal. The threshold can be set through historical data.

[0105] Optionally, before obtaining the industrial control time series data to be detected and inputting it into the enhanced neural network framework to perform the following steps through the enhanced neural network framework to implement data anomaly detection, the method further includes:

[0106] Step 110, obtain the original industrial control time series data in real time, and after performing data preprocessing on the original industrial control time series data, obtain the industrial control time series data to be detected, where the data preprocessing includes at least one of data outlier removal and data normalization.

[0107] In the above embodiments of the present application, to ensure the quality and consistency of the input data, the KNN interpolation method or the linear interpolation method can be used to process data missing, the interquartile range method or the Z-score standardization method can be used to remove extreme outliers, and linear normalization can be used to normalize the time series data. The finally processed data can be expressed as (B, T, F), where B is the batch size, T is the time step, and F is the number of features.

[0108] Specifically, for example, select the actual time series data in the industrial control system, including sensor data, device network load flow data, etc., and use the KNN interpolation method or the linear interpolation method to fill in the missing values. For example, in the industrial control system, sensors may occasionally lose power briefly, resulting in missing data. The KNN interpolation method is used to infer the missing values based on the spatial similarity of adjacent points to ensure the continuity and integrity of the time series.

[0109] Regarding data outlier removal, the interquartile range method can be introduced to remove extreme outliers. On each time series feature dimension, define the upper and lower quartiles Q1 and Q3, and define the outlier threshold as Q1 - 1.5×IQR, Q3 + 1.5×IQR, and remove the data outside this range. For example, in the temperature sensor data, if some readings exceed the normal range (such as above 40°C), they are considered abnormal and removed to avoid false alarms, where IQR is the abbreviation of "Interquartile Range", which means "interquartile range". IQR is a statistic that describes the data distribution and represents the range of the middle 50% of the data in the dataset. Specifically, IQR is the difference between the upper quartile (Q3) and the lower quartile (Q1). IQR can be used to measure the dispersion of the data. A larger IQR indicates a more dispersed data distribution, while a smaller IQR indicates a more concentrated data distribution.

[0110] Regarding data normalization, the time series data can be converted to between 0 and 1 through linear normalization to ensure the dimensional consistency of data in different dimensions, as shown in the following formula:

[0111] ,

[0112] is the industrial control time series data to be detected after data normalization processing, is the original industrial control time series data, and are the minimum and maximum values in the original industrial control time series data respectively. Therefore, the time series dimension of the industrial control time series data to be detected after normalization is (B, T, F).

[0113] By applying the technical solution of this embodiment, by dynamically adjusting the parameter values involved in the time series feature learning part, the purpose of effectively distinguishing various data contaminations and real anomalies is achieved, and the problem that most time series detection methods have a decline in accuracy and weak adaptability in the face of multi-type data contaminations is effectively solved.

[0114] Through the description of the above embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus a necessary general hardware platform, or can be implemented by hardware. By dynamically adjusting the parameter values involved in the time series feature learning part, the purpose of effectively distinguishing various data contaminations and real anomalies is achieved, and the problem that most time series detection methods have a decline in accuracy and weak adaptability in the face of multi-type data contaminations is effectively solved.

[0115] Those skilled in the art can understand that the drawings are only schematic diagrams of a preferred embodiment scenario, and the modules or processes in the drawings are not necessarily essential for implementing this application. Those skilled in the art can understand that the modules in the device in the embodiment scenario can be distributed in the device in the embodiment scenario according to the description of the embodiment scenario, or can be correspondingly changed and located in one or more devices different from this embodiment scenario. The modules in the above embodiment scenario can be combined into one module, or can be further split into multiple sub-modules.

[0116] The above serial numbers of this application are only for description and do not represent the advantages or disadvantages of the embodiment scenario. The above disclosure is only several specific embodiment scenarios of this application. However, this application is not limited thereto, and any changes made by those skilled in the art should fall within the protection scope of this application.

Claims

1. A method for detecting anomalies in industrial control time series data, characterized in that: Applied to an enhanced neural network framework, the enhanced neural network framework includes a novel autoencoder and an anomaly detection module, the novel autoencoder includes a novel encoder and a novel decoder with a reciprocal structure, the novel encoder includes a long short-term memory network, a temporal convolutional network and a self-attention mechanism module; the method includes: The industrial control time series data to be detected is obtained and input into the enhanced neural network framework, so as to implement data anomaly detection by executing the following steps through the enhanced neural network framework: By using each gate in the long short-term memory network with a gating mechanism, based on the dynamic adjustment factor of the activation value, cross-validate the optimal activation value corresponding to each gate of the industrial control time series data to be tested under various data pollution conditions, and adjust each gate through the optimal activation value to output the long-term dependency feature; After dynamically adjusting the contribution of the residual part of the temporal convolutional network according to the long-term dependency feature based on the residual connection scaling factor using the temporal convolutional network, the short-term feature is output; The self-attention mechanism module uses a self-attention mechanism to locate key features in the short-term features and output them, wherein the output key features are represented by reconstructed time series and low-dimensional feature representations; The novel decoder reversely decodes the key features to obtain the industrial control timing data to be compared; The anomaly detection module obtains an anomaly detection result by comparing the industrial control timing data to be detected with the industrial control timing data to be compared.

2. The method for detecting anomalies in industrial control time series data according to claim 1, characterized in that: The enhanced neural network framework also includes a self-supervised contrastive learning network and a generative adversarial network; the gates in the long short-term memory network with the added gating mechanism are cross-validated based on the activation value dynamic adjustment factor for each gate under various data pollution conditions for the industrial control time series data to be detected, and before the long-term dependent features are output after adjusting each gate by the optimal activation value, the method also includes: Determine the data scenario to which the industrial control time series data to be detected belongs, and use the self-supervised contrastive learning network and the generative adversarial network to jointly optimize the parameters of the new autoencoder for the determined data scenario, wherein the data scenario includes a scenario with high real-time data demand, a scenario with high data pollution, and a scenario with insufficient data.

3. The method for detecting anomalies in industrial control time series data according to claim 2, characterized in that: The method of using the self-supervised contrastive learning network and the generative adversarial network to jointly optimize the parameters of the new autoencoder for a certain data scenario includes: The total loss function of the self-supervised contrastive learning network and the generative adversarial network is constructed, and the parameters of the new autoencoder are optimized by dynamically adjusting the loss dynamic adjustment factors corresponding to the self-supervised contrastive learning network and the generative adversarial network in the constructed total loss function, wherein the total loss function is: ， is the total loss function, is the contrastive loss of the self-supervised contrastive learning network, is the adversarial loss of the generative adversarial network, is the loss dynamic adjustment factor of the self-supervised contrastive learning network, A dynamic adjustment factor for the loss of the generative adversarial network; When the data scenario to which the industrial control time series data to be detected belongs is a scenario with high real-time data requirements, the loss dynamic adjustment factor of the generative adversarial network is set to the minimum preset threshold close to 0; when the data scenario to which the industrial control time series data to be detected belongs is a scenario with high data pollution, the loss dynamic adjustment factor of the self-supervised contrastive learning network is set to be smaller than the loss dynamic adjustment factor of the generative adversarial network; and when the data scenario to which the industrial control time series data to be detected belongs is a scenario with insufficient data, the loss dynamic adjustment factor of the generative adversarial network is set to 0.

4. The method for detecting anomalies in industrial control time series data according to claim 1, characterized in that: There are multiple gates. The dynamic adjustment factor based on the activation value cross-validates the optimal activation value corresponding to each gate under various data pollution conditions of the industrial control time series data to be detected, including: For any of the various gates in the gating mechanism, based on the dynamic adjustment factor of the activation value and the optimal activation value determination formula, cross-validation is performed to obtain the optimal activation value corresponding to the gate under various data pollution conditions for the industrial control timing data to be detected, wherein the optimal activation value determination formula is: ， ， is the optimal activation value of the gate at time t, is the weight matrix, The industrial control timing data to be tested input at the current moment and the hidden state at the previous moment t-1 The splicing, is the bias vector, is the optimal activation value of the gate adjusted based on the activation value dynamic adjustment factor, The Sigmoid activation function is used to make the optimal activation value between 0 and 1.

5. The method for detecting anomalies in industrial control time series data according to claim 4, characterized in that: The gates include an input gate, a forget gate, and an output gate. The output long-term dependency features after adjusting each gate by the optimal activation value include: For any of the various gates, the output of the gate is controlled by the gating calculation formula corresponding to the gate and the optimal activation value of the gate until the output of the various gates is controlled respectively and the long-term dependent characteristics are output, wherein the gating calculation formulas corresponding to different gates are respectively: ， ， ， ， , , Respectively through the Sigmoid activation function The calculated input gate, forget gate and output gate, To pass The candidate cell states calculated by the function, , , and are the weight matrices of the input gate, forget gate, output gate, and candidate cell state, respectively. , , and They are the bias items of input gate, forget gate, output gate and candidate cell state respectively. Input industrial control timing data at the current moment and the hidden state at the previous moment t-1 The splicing, , and are the optimal activation values ​​of the input gate, forget gate and output gate at time t, respectively. The Sigmoid activation function is used to make the optimal activation value between 0 and 1.

6. The method for detecting anomalies in industrial control time series data according to claim 1, characterized in that: The long short-term memory network includes a memory unit, the memory unit is used to store and update information, and the output long-term dependency features include: The latest information stored in the memory unit and the industrial control timing data to be detected input at the current moment are used to output the long-term dependency features through the output gate, wherein the output long-term dependency features include the information at the current moment and the long-term dependency features at the previous moment.

7. The method for detecting anomalies in industrial control time series data according to claim 1, characterized in that: The temporal convolutional network includes a first convolutional layer, a second convolutional layer and a third convolutional layer, and the expansion rates corresponding to the first convolutional layer to the third convolutional layer increase layer by layer. The temporal convolutional network is used to dynamically adjust the contribution of the residual part of the temporal convolutional network according to the long-term dependency feature based on the residual connection scaling factor, and then output short-term features, including: The first convolution layer, the second convolution layer and the third convolution layer are sequentially connected through the dynamic residual, and the residual connection scaling factor is used to dynamically adjust the contribution of the residual part of the temporal convolution network according to the long-term dependency feature, and then the short-term feature is output, wherein the output short-term feature is expressed by the following formula: ， After the temporal convolutional network The convolution operation and the contribution of the residual part are dynamically adjusted to produce the short-term features of the output. is the long-term dependence characteristic at time t, is the weight of the convolution kernel of the temporal convolutional network, To determine the expansion rate of the sampling interval during wide-base operation, is the residual connection scaling factor.

8. The method for detecting anomalies in industrial control time series data according to claim 1, characterized in that: The self-attention mechanism module uses the self-attention mechanism to locate key features in the short-term features and outputs them, including: The self-attention mechanism module adopts the self-attention mechanism to dynamically identify the contamination features in the short-term features, and adjusts the attention weights of the identified contamination features to obtain and output key features.

9. The method for detecting anomalies in industrial control time series data according to claim 1, characterized in that: The abnormality detection module obtains an abnormality detection result by comparing the industrial control timing data to be detected with the industrial control timing data to be compared, including: The abnormality detection module compares the industrial control timing data to be detected with the industrial control timing data to be compared. When the comparison result of the industrial control timing data to be detected and the industrial control timing data to be compared shows that there is a difference exceeding a preset difference threshold, abnormal industrial control timing data exists.

10. The method for detecting anomalies in industrial control time series data according to any one of claims 1 to 9, characterized in that: Before obtaining the to-be-detected industrial control time series data and inputting it into the enhanced neural network framework to perform the following steps to realize data anomaly detection through the enhanced neural network framework, the method further includes: The original industrial control time series data is acquired in real time, and the industrial control time series data to be detected is obtained after data preprocessing is performed on the original industrial control time series data, wherein the data preprocessing includes at least one of data outlier removal and data normalization.

Images