Industrial process oscillation monitoring method and device based on visual-frequency dual-mode fusion network

Abstract

The application discloses an industrial process oscillation monitoring method and device based on a visual-frequency dual-mode fusion network, and relates to the field of control loop oscillation detection. The method comprises the following steps: obtaining process data output by an industrial process loop to be detected; obtaining frequency domain data and visual data according to the process data; inputting the frequency domain data and the visual data into a trained visual-frequency dual-mode fusion network to detect an oscillation type and quantify an oscillation intensity; wherein the visual-frequency dual-mode fusion network comprises an oscillation detection network and an oscillation quantification network; the network structures of the oscillation detection network and the oscillation quantification network are the same, and the activation functions adopted by the oscillation detection network and the oscillation quantification network are different; the oscillation type comprises non-oscillation, regular oscillation and irregular oscillation, and the quantification result comprises a period number value, a sparsity value and a regularity value. The application improves the precision of oscillation detection and the quantification precision of oscillation intensity in an industrial control loop process.

Description

Technical Field

[0001] This application relates to the field of control loop oscillation detection, and in particular to an industrial process oscillation monitoring method and device based on a vision-frequency domain dual-modal fusion network. Background Technology

[0002] Industrial processes encompass process industries, represented by high-energy-consuming sectors such as power, chemicals, steel, and cement, as well as discrete industries, represented by high-end manufacturing such as automobiles, electronics, shipbuilding, and precision instruments. These two sectors are the core components of the modern industrial system, playing a crucial role in supporting infrastructure construction, ensuring the stability of industrial and supply chains, and driving technological innovation. With the deep integration of industrial automation and information technology, industrial processes are rapidly evolving towards intelligence and greening, placing higher demands on the reliability, energy efficiency, safety, and product quality of production processes.

[0003] As the "nerve center" of industrial processes, control systems are the core carriers for achieving precise control, optimizing resource utilization, and ensuring production safety. Their performance directly determines the stability and economic benefits of the production process. However, during long-term operation, the performance of control systems often degrades due to factors such as equipment aging, changes in operating conditions, and external disturbances. Degraded control systems not only lead to energy waste and raw material losses but may also cause production fluctuations and even safety accidents.

[0004] Control loop oscillation is a typical characteristic of performance degradation and a persistent problem in process industries. Continuous oscillation exacerbates equipment wear, reduces product quality, increases carbon emissions, and can trigger a chain reaction of production failures. For example, in the chemical industry, abnormal oscillations in reactor temperature or pressure can lead to catalyst deactivation, causing millions in economic losses. Therefore, timely detection and elimination of oscillations is crucial for improving the economic efficiency of industrial processes. However, modern industrial sites typically deploy thousands of control loops, and traditional manual inspection methods are inefficient and prone to missed detections and misjudgments. Furthermore, after detecting oscillations, they need to be classified according to their intensity to distinguish between loops requiring immediate maintenance and those that do not affect equipment performance. Developing intelligent online detection and diagnostic technologies has become an inevitable choice for achieving refined management of industrial processes.

[0005] However, most current monitoring methods only design specific rules based on partial features of the data to detect oscillations, causing them to fail when such rules are not present in the data. Another group of machine learning-based monitoring methods uses only a single data modality to extract features for oscillation detection. While this method is not constrained by specific rules, in practical applications, inconsistencies in spatiotemporal features due to data length differences affect network performance, resulting in sparse feature distribution and poor intra-class compactness. This not only exacerbates the volatility of detection performance but also weakens the network's generalization ability. Therefore, for oscillation detection and oscillation intensity quantization in industrial control loops, related technologies suffer from low detection accuracy and low quantization accuracy. Summary of the Invention

[0006] The purpose of this application is to provide an industrial process oscillation monitoring method, device, equipment, medium, and product based on a vision-frequency domain dual-modal fusion network, which can improve the accuracy of oscillation detection and the quantification accuracy of oscillation intensity in industrial control loop processes.

[0007] To achieve the above objectives, this application provides the following solution:

[0008] In a first aspect, this application provides a method for monitoring industrial process oscillations based on a vision-frequency domain dual-modal fusion network, comprising:

[0009] Acquire process data output from the industrial process loop under test;

[0010] Frequency domain data and visual data are obtained based on the process data;

[0011] The frequency domain data and visual data are input into a trained visual-frequency domain bimodal fusion network to detect oscillation types and quantify oscillation intensity. The visual-frequency domain bimodal fusion network is trained using a sample training set and includes an oscillation detection network and an oscillation quantization network. The oscillation detection network and the oscillation quantization network have the same network structure but use different activation functions. The oscillation types include non-oscillation, regular oscillation, and irregular oscillation, and the quantization results include period count, sparsity, and regularity values.

[0012] Secondly, this application provides an industrial process oscillation monitoring device based on a vision-frequency domain dual-modal fusion network, comprising:

[0013] The acquisition module is used to acquire process data output from the industrial process loop to be tested.

[0014] The preprocessing module is used to obtain frequency domain data and visual data based on the process data;

[0015] The detection and quantization module is used to input frequency domain data and visual data into a trained visual-frequency domain bimodal fusion network to detect oscillation types and quantize oscillation intensity. The visual-frequency domain bimodal fusion network is trained using a sample training set and includes an oscillation detection network and an oscillation quantization network. The oscillation detection network and the oscillation quantization network have the same network structure but use different activation functions. The oscillation types include non-oscillation, regular oscillation, and irregular oscillation, and the quantization results include period count, sparsity, and regularity values.

[0016] According to the specific embodiments provided in this application, the following technical effects are disclosed:

[0017] This application provides a method and apparatus for monitoring industrial process oscillations based on a vision-frequency domain dual-modal fusion network. The process data output from the detected industrial process loop is preprocessed to obtain frequency domain data and visual data. The time-series data is converted to the frequency domain to ensure data length consistency and enhance noise resistance. Based on this, the trained vision-frequency domain dual-modal fusion network is used to capture features in both the vision and frequency domains, making up for the shortcomings of spatiotemporal feature inconsistency and improving the oscillation detection accuracy and oscillation intensity quantification accuracy of the industrial process loop. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is an application environment diagram of an industrial process oscillation monitoring method based on a vision-frequency domain dual-modal fusion network according to an embodiment of this application;

[0020] Figure 2 A flowchart illustrating an industrial process oscillation monitoring method based on a vision-frequency domain dual-modal fusion network, provided as an embodiment of this application;

[0021] Figure 3 A schematic diagram illustrating the training and performance evaluation process of a visual-frequency domain dual-modal fusion network provided in an embodiment of this application;

[0022] Figure 4 This is a schematic diagram illustrating a data example of a dataset provided in one embodiment of this application.

[0023] Figure 5This is a schematic diagram of an oscillation detection network structure provided in an embodiment of this application;

[0024] Figure 6 A schematic diagram illustrating the evaluation results of an oscillation detection network provided in an embodiment of this application;

[0025] Figure 7 is a schematic diagram of the evaluation results of an oscillatory quantization network provided in an embodiment of this application; wherein, (a) is the evaluation result of an oscillatory quantization network with a quantization period number, (b) is the evaluation result of an oscillatory quantization network with a quantization sparsity, and (c) is the evaluation result of an oscillatory quantization network with a quantization regularity.

[0026] Figure 8 This is a functional module diagram of an industrial process oscillation monitoring device based on a vision-frequency domain dual-modal fusion network, provided as another embodiment of this application. Detailed Implementation

[0027] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0028] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0029] The industrial process oscillation monitoring method based on a vision-frequency domain dual-modal fusion network provided in this application embodiment can be applied to, for example... Figure 1In the application environment shown, terminal 101 communicates with server 102 via a network. A data storage system can store the data that server 102 needs to process. The data storage system can be set up independently, integrated into server 102, or placed in the cloud or on another server. Terminal 101 can send the process data output from the industrial process loop to be detected to server 102. Server 102 receives the process data output from the industrial process loop to be detected and obtains the process data output from the industrial process loop. Based on the process data, server 102 obtains frequency domain data and visual data. The frequency domain data and visual data are input into a trained visual-frequency domain dual-modal fusion network to detect oscillation types and quantify oscillation intensity. Server 102 can feed back the obtained oscillation detection results and oscillation intensity quantization results to terminal 101. In addition, in some embodiments, the industrial process oscillation monitoring method based on the visual-frequency domain dual-modal fusion network can also be implemented by the server 102 or the terminal 101 separately. For example, the terminal 101 can directly detect the oscillation type and quantify the oscillation intensity of the process data output by the industrial process loop to be detected, or the server 102 can obtain the process data output by the industrial process loop to be detected from the data storage system, and detect the oscillation type and quantify the oscillation intensity.

[0030] The terminal 101 can be, but is not limited to, various desktop computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, and smart in-vehicle devices. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted devices. The server 102 can be implemented using a standalone server or a server cluster composed of multiple servers, or it can be a cloud server.

[0031] In one exemplary embodiment, such as Figure 2 As shown, an industrial process oscillation monitoring method based on a vision-frequency domain dual-modal fusion network is provided. This method is executed by a computer device, specifically by a terminal or server alone, or by both a terminal and a server. In this embodiment, the method is applied to... Figure 1 Taking server 104 as an example, the explanation includes the following steps 201 to 203. Wherein:

[0032] Step 201: Obtain the process data output from the industrial process loop to be tested.

[0033] Step 202: Obtain frequency domain data and visual data based on the process data. Specifically, this includes steps 301-302:

[0034] Step 301: The process data is imaged using a preset image resolution to obtain visual data. A preset image resolution is selected to image the process data, and the pixel matrix is ​​normalized. The calculation method is as follows:

[0035]

[0036] in, This represents the process data of the i-th input. This represents the corresponding normalization result.

[0037] The preset size selected in this embodiment is 200×2400 (height×width), but other sizes can also be selected. This embodiment does not limit the size.

[0038] Step 302: Normalize the process data, and perform discrete Fourier transform on the normalized process data to extract amplitude information, thereby obtaining frequency domain data.

[0039] To address the issue of inconsistent process data scales across different industrial process loops, the process data is first normalized to the mean:

[0040]

[0041] in, Let x(i) be the normalized process data of the i-th data point, x(i) be the time series of the i-th process data point, and mean(·), max(·), and min(·) be the mean, maximum, and minimum value operators, respectively.

[0042] The normalized process data is then subjected to a Discrete Fourier Transform (DFT), specifically represented as follows:

[0043]

[0044] in, For the i-th frequency domain discrete spectrum The complex value corresponding to the k-th frequency point Let j be the data at the nth time point in the i-th normalized process data, j be the imaginary part, N be the number of points in the discrete Fourier transform, also known as the transform length, and k = 0, 1, 2, ..., N-1.

[0045] In this embodiment, N is set to 2. 13 .

[0046] Amplitude information is extracted from the data after Discrete Fourier Transform (DFT). The amplitude extraction is performed on the entire DFT-transformed data set. The specific expression is as follows:

[0047]

[0048] in, For the i-th discrete spectrum in the frequency domain, For the i-th data after extracting the amplitude, abs(·) represents the amplitude extraction operator. Due to the symmetry of DFT, only the first half of the DFT amplitude is retained as frequency domain data.

[0049] Step 203: Input the frequency domain data and visual data into the trained visual-frequency domain bimodal fusion network to detect oscillation types and quantize oscillation intensity; wherein, the visual-frequency domain bimodal fusion network includes an oscillation detection network and an oscillation quantization network; the oscillation detection network and the oscillation quantization network have the same network structure, but use different activation functions; the oscillation types include non-oscillation, regular oscillation, and irregular oscillation, and the quantization results include period count values, sparsity values, and regularity values.

[0050] like Figure 3 As shown, the training and performance evaluation process of the vision-frequency domain bimodal fusion network is as follows:

[0051] 1. Data Acquisition

[0052] Prepare two datasets. The first is the IASDS dataset, constructed using the Artificial Simulation Algorithm for Industrial Data (ASAID) to randomly generate 10,000 time series data. The second is the ISDB open-source industrial dataset obtained from literature. Examples of data from these two datasets are shown below. Figure 4 As shown in the figure. The IASDS dataset is used for model training and performance evaluation to obtain the optimal detection model, while ISDB is used to test the model's oscillation detection performance.

[0053] Using MATLAB, variables were designed for industrial data simulation, as shown in Table 1. These variables were then used to construct the ASAID algorithm.

[0054] Table 1

[0055]

[0056] The following details are included when constructing the ASAID algorithm:

[0057] First, both the noise and the disturbance are Gaussian white noise, and the noise variance N is used. var and disturbance amplitude D ampTheir magnitudes are determined separately. Furthermore, to simulate real-world conditions, both disturbance and oscillation data need to be smoothed. The disturbance data is smoothed using the following transfer function H1(s):

[0058]

[0059] The oscillating data is smoothed using the following transfer function H2(s):

[0060]

[0061] Where s is the complex frequency of the Laplace transform.

[0062] Secondly, the amplitudes of all oscillating time series are normalized to [-1, 1] to ensure that variable N var and D amp The effectiveness.

[0063] Third, the fundamental frequency f0 of the non-frequency oscillation data is determined by the signal length variable S. L and the number of cycles N per The decision is made by the frequency f(t) of variable frequency oscillation data. Conversely, for variable frequency oscillation data, the frequency f(t) is determined by f0 and F. cf Decide.

[0064] The process of generating artificial process time series using the ASAID algorithm consists of steps 1-8:

[0065] Step 1: Initialize time series variables.

[0066] Initialize the output time series as an empty array: O ts ←0.

[0067] Construct a time series index vector: t = [0, 1, 2, ..., S] L-1 ].

[0068] Step 2: Generate Gaussian noise terms and disturbance data with a preset time length based on time series variables.

[0069] Two independent time series variables with a preset time length of S are generated. L Gaussian white noise sequence D noise Noise terms and The perturbation data, both with zero mean and variance N, are used. amp .

[0070] Step 3: Smooth the disturbance data using the first filter, and normalize the amplitude of the smoothed disturbance data; the disturbance is smoothed and its amplitude is normalized.

[0071] Use the first filter to scramble the data. After smoothing, the output is The transfer function of the first filter is H1(s).

[0072] Disturb data Standardize and control its amplitude range to D amp Amplitude-normalized perturbation data D disturbance The calculation formula is:

[0073]

[0074] Step 4: If the artificial process time series to be generated is oscillating data.

[0075] If the artificial process time series to be generated is oscillating data (i.e., O...), prob If =1), then perform the following steps:

[0076] Calculate the fundamental frequency f0 based on the preset time length:

[0077] f0 = 1 / (S) L / N per (8);

[0078] Where, N per This represents the number of times a complete waveform appears throughout the entire artificial process time series.

[0079] Step 5: If the artificial process time series needs to be generated as irregular oscillating data, i.e., VF prob =1, the frequency of the irregular oscillating signal changes with time, then the time-varying function is calculated based on the fundamental frequency f0:

[0080]

[0081] Where t is time.

[0082] Generate a periodic signal O based on a time-varying function. d (t):

[0083]

[0084] Among them, O wf =0,1,2 represent sine wave, square wave, and sawtooth wave, respectively.

[0085] Step 6: If you need to generate artificial process time series data as regular oscillating data, i.e., VF prob =0, then the fundamental frequency f0 is a constant:

[0086]

[0087] Step 7: Smooth the periodic signal data using the second filter, and then normalize the smoothed periodic signal data.

[0088] For periodic signal O d (t) The signal is smoothed using the smoothing factor SF, and the output is... Will Normalization, with a mean of zero and an amplitude between [-1, 1], yields the normalized periodic signal data.

[0089]

[0090] The transfer function of the second filter is H2(s).

[0091] The artificial process time series O is generated based on the normalized periodic signal data, the amplitude-normalized perturbation data, and the Gaussian noise term. ts :

[0092]

[0093] Step 8: If the artificial process time series to be generated is non-oscillating data, then generate the artificial process time series based on the amplitude-normalized perturbation data and the Gaussian noise term:

[0094] O ts =D noise +D disturbance (14).

[0095] A simulation dataset IASDS was constructed using time-series data randomly generated by ASAID.

[0096] 2. Data labeling and processing.

[0097] Step 301 is used to process the dataset. Specifically, the time series data from the IASDS and ISDB datasets are visualized in a two-dimensional coordinate system using the matplotlib library in Python. The horizontal axis represents the time point number, and the vertical axis represents the corresponding amplitude. A continuous image is formed by connecting the data points in chronological order using a polyline. After plotting, the image is saved locally in PNG format using the savefig function to simulate the imaging process. Subsequently, the keras.utils.img_to_array function is used to convert the image into a grayscale pixel matrix, and the pixel values ​​are normalized to the [0,1] interval to construct a standardized image feature input.

[0098] Step 302 transforms the dataset to the frequency domain. Specifically, the time series data in the IASDS and ISDB datasets are mean-normalized. Then, the numpy.fft.fft function in Python is used to perform an 8192-point Discrete Fourier Transform on each segment of normalized data and extract the amplitude. Due to the symmetry of the DFT, only the first half of the DFT amplitude is retained as the frequency domain data.

[0099] The processed IASDS dataset was randomly divided into training and validation sets in an 8:2 ratio.

[0100] like Figure 5 As shown, the visual-frequency domain bimodal fusion network for oscillation detection comprises a visual modality branch network, a frequency domain modality branch network, a feature commonality fusion network, and fully connected layers. The visual modality branch network includes multiple depthwise separable convolutional layers, multiple two-dimensional standard convolutional layers, and a global average pooling layer; the frequency domain modality branch network includes multiple one-dimensional convolutional layers and multiple max-pooling layers. The feature commonality fusion network consists of sequentially connected channel connection layers, one-dimensional convolutional layers, channel attention layers, and channel connection layers. In the feature commonality fusion network, common features are extracted through the channel attention layer, and based on this, the final feature map is obtained using the channel connection layer.

[0101] The training process of the visual-frequency domain bimodal fusion network for oscillation detection is divided into two processes: forward and backward. The forward process can be further divided into four steps (a)-(d):

[0102] (a): Processed visual data After a series of linear and nonlinear transformations of the visual modality branch The extracted visual features can be represented as follows:

[0103]

[0104] Where V represents visual features, z vc This is the c-th channel feature map extracted from the last convolutional layer, where c is the number of output channels of the last convolutional layer, and Concat(·) is used to concatenate along the channel dimension.

[0105] (b): Processed frequency domain data After a series of linear and nonlinear transformations via frequency domain modal branching Extracting frequency domain features, the output can be represented as follows:

[0106]

[0107] Where F is the frequency domain feature, z jc This is the c-th channel feature map extracted from the last convolutional layer, where c is the number of output channels of the last convolutional layer.

[0108] (c): By fusing visual features and frequency domain features through a feature commonality fusion network, the fused output feature M can be represented as follows:

[0109] M=Concat(ECA(Conv1D(Concat(V+F))),V,F)(17);

[0110] Conv1D(·) is a one-dimensional convolution operation, and ECA(·) is an efficient channel attention operation.

[0111] (d): The fused features M are mapped to output task units through a fully connected layer, and then converted into the probability distribution of the categories by the output of the activation function neurons to obtain the final prediction result. The fully connected operation can be represented as:

[0112]

[0113] Among them, Z( i F represents the prediction result for the i-th data point. l (·) represents a series of linear and nonlinear operations. The output of the j-th neuron after the i-th data point passes through the fully connected layer is given by K, where K is the number of data categories.

[0114] For the oscillation detection network, the number of hidden neurons in the fully connected layer is 3, i.e., K=3, which is consistent with the number of sample classes in the IASDS dataset. For the oscillation quantization network, the number of hidden neurons in the fully connected layer is 1, i.e., K=1, which is the final output oscillation intensity value.

[0115] In oscillation detection networks, the softmax function is used to transform the neuron's output into a probability distribution of the class. The prediction result after softmax transformation is... It can be represented as:

[0116]

[0117] In oscillatory quantization networks, a linear activation function is used to map the neuron's output to the final quantized result, which is then transformed into a prediction result. It can be represented as:

[0118]

[0119] The backward process of the visual-frequency domain bimodal fusion network for oscillation detection is a process of gradient descent and network weight parameter update. Its connection to the forward process is achieved through the loss function L(·). Here, a feature consistency loss function is designed based on the feature structure, fusion relationship, and final task. The specific operation is as follows:

[0120] L = L task +λ[1-cos(V,F)]+μ[1-cos(V+F,2M)](21);

[0121] Where L is the total loss, V and F are the visual features and frequency domain features respectively, λ and μ are the weight terms respectively, and M is the fusion feature. In this embodiment, the values ​​of λ and μ are both 0.5.

[0122] In the oscillation detection network, L task The cross-entropy loss method, L1, operates as follows:

[0123]

[0124] Where θ1 represents the network parameters of the oscillation detection network. Let N be the i-th data point, N be the total number of data points, and K be the meaning of the data.

[0125] In the oscillatory quantization network, L task The mean squared error loss (MSE) is calculated using the L2 method, which operates as follows:

[0126]

[0127] in, and They are The category label and oscillation intensity label are θ2, and the network parameters of the oscillation detection network are θ2. After sufficient iterations in the forward and backward propagation phases, the network learning is stopped, and the data is automatically saved during the learning process.

[0128] 3. Training the model. The learning rate is set to 0.001, and the batch size is 8. Furthermore, the Adaptive Moment Estimator (Adam) is selected as the optimizer. The gradient optimization process using Adam can be represented as follows:

[0129]

[0130] Where, m st v st Let β1 and β2 be the first and second moment estimates after bias correction at the t-th step, respectively, where β1 and β2 are the decay rate hyperparameters (0.9 and 0.999, respectively), α is the learning rate (0.001), ε is a small constant to prevent division by zero, and θ is the second moment estimate. st and θ st-1 represents all relevant parameters of the oscillation quantization network or the oscillation detection network at the t-th iteration.

[0131] After setting the above hyperparameters, the oscillation detection network and oscillation quantization network built on Keras and TensorFlow deep learning frameworks were trained for 100 epochs respectively, and the trained oscillation detection network and oscillation quantization network were saved.

[0132] The trained oscillation detection network and oscillation quantization network were evaluated and compared one by one using the partitioned validation set data to obtain the optimal model. The evaluation results of the trained oscillation detection network are as follows: Figure 6 As shown, Figure 6 In the coordinate system, '0' represents the non-oscillating data in the artificial simulation dataset IASDS; '1' represents the regular oscillating data in the artificial simulation dataset IASDS; and '2' represents the irregular oscillating data in the artificial simulation dataset IASDS. The evaluation results of the trained oscillation quantization network are shown in Figure 7. In Figure 7(a), the evaluation results of the oscillation quantization network with the number of quantization cycles are shown; in Figure 7(b), the evaluation results of the oscillation quantization network with quantization sparsity are shown; and in Figure 7(c), the evaluation results of the oscillation quantization network with quantization regularity are shown.

[0133] 4. Evaluation Model

[0134] The trained oscillation detection network and oscillation quantization network are loaded, and the validation set data is input into the oscillation detection network and oscillation quantization network to evaluate the model's detection performance. The evaluation metrics are the confusion matrix (oscillation detection) and the mean squared error (oscillation quantization). The mean squared error can be calculated as equation (23), and the confusion matrix is ​​shown in Table 2 below:

[0135] Table 2

[0136] Real positive samples TP FN True negative samples FP TN

[0137] Here, TP, TN, FP, and FN represent the number of correctly predicted positive samples, the number of correctly predicted negative samples, the number of incorrectly predicted positive samples, and the number of incorrectly predicted negative samples, respectively. By evaluating the model, the optimal oscillation detection network and oscillation quantization network can be obtained.

[0138] The optimal oscillation detection network and oscillation quantization network are loaded. Industrial data is randomly extracted from the ISDB dataset and processed in step 202 to obtain a data format that can be received by the oscillation detection network and oscillation quantization network. This data is then input into the optimal oscillation detection network and oscillation quantization network. After the forward process, i.e., after processing by formulas (15)-(20), the output result of the forward process can be obtained. and in, It is the predicted value of the oscillation intensity of the i-th data point. and Let q represent the probabilities that the i-th data point belongs to non-oscillatory, regular oscillatory, and irregular oscillatory patterns, respectively. (i) Let be the predicted quantization intensity value of the i-th data point. It is a vector representing the predicted probability of the oscillation type of the i-th data point.

[0139] By obtaining the exponent value IV, which represents the maximum probability, using argmax(·), the class to which the sample belongs can be determined. This operation can be represented as:

[0140]

[0141] 5. Classify according to oscillation intensity

[0142] The trained oscillation detection network is loaded, and process data from six randomly selected industrial process loops (referred to as loops) from the processed ISDB dataset are input into the oscillation detection network to obtain test results. The test results are as follows: Figure 6 .

[0143] Oscillations are classified by combining oscillation detection results and oscillation intensity quantification results. Samples with oscillation results are selected and ranked according to the predicted oscillation intensity, prioritizing the processing of loops with strong oscillations. Specifically, if the classification is based on the number of cycles, the processing priority is loop 6, loop 5, loop 4, and loop 3 in that order; if the classification is based on sparsity, the processing priority is loop 5, loop 4, loop 6, and loop 3 in that order; and if the classification is based on regularity, the processing priority is loop 3, loop 5, loop 6, and loop 4 in that order.

[0144] In an exemplary embodiment, step 203 specifically includes steps 401-402:

[0145] Step 401: Input the frequency domain data and visual data into the trained oscillation detection network to detect the oscillation type.

[0146] In an exemplary embodiment, step 401 specifically includes steps 501-504:

[0147] Step 501: Input the visual data into the visual modality branch network for feature extraction to obtain visual features.

[0148] Step 402: Input the frequency domain data and visual data into the trained oscillation quantization network to quantize the oscillation intensity.

[0149] Step 502: Input the frequency domain data into the frequency domain modal branch network for feature extraction to obtain frequency domain features.

[0150] Step 503: Input the visual features and the frequency domain features into the feature commonality fusion network for feature fusion to obtain fused features.

[0151] Step 504: The fused features are input into a fully connected layer for mapping processing and transformed by the output of an activation function neuron to obtain the detected oscillation type.

[0152] This application has the following advantages:

[0153] 1. Since the proposed industrial process oscillation monitoring method based on vision-frequency domain dual-modal fusion network does not limit the detection of oscillations to specific rules, this application solves the problem of low accuracy or even failure of the method in related technologies that are based on certain rules for oscillation detection.

[0154] 2. By combining visual and frequency domain modal data to detect and quantify oscillations, this application solves the problem of inconsistent spatiotemporal characteristics of data caused by related machine learning methods using only single modal data.

[0155] 3. Due to the rich dynamic characteristics contained in visual and frequency domain modal data and the powerful feature extraction capabilities of dual-modal fusion networks, this application solves the problems that most related technologies struggle to effectively handle noisy sequences, multiple oscillations, intermittent oscillations, time-varying oscillations, and non-stationary characteristics.

[0156] 4. Combining detection and quantification results to classify oscillations, this "detection + classification" approach significantly reduces the difficulty of subsequent maintenance and effectively improves work efficiency and resource utilization.

[0157] 5. Since the visual-frequency domain dual-modal fusion network can automatically infer and detect the presence of oscillations based on the input and can be updated, it solves the problems of instability and low reliability of related technologies.

[0158] 6. The method proposed in this application can use additional training data to update the model in order to handle new loop characteristics.

[0159] Based on the same inventive concept, this application also provides an apparatus for implementing the above-mentioned industrial process oscillation monitoring method based on a vision-frequency domain dual-modal fusion network. The solution provided by this apparatus is similar to the implementation described in the above method. Therefore, the specific limitations of one or more embodiments of the industrial process oscillation detection apparatus based on a vision-frequency domain dual-modal fusion network provided below can be found in the limitations of the industrial process oscillation monitoring method based on a vision-frequency domain dual-modal fusion network described above, and will not be repeated here.

[0160] In one exemplary embodiment, such as Figure 8As shown, an industrial process oscillation detection device based on a vision-frequency domain dual-modal fusion network is provided, comprising:

[0161] The acquisition module 81 is used to acquire the process data output by the industrial process loop to be tested.

[0162] The preprocessing module 82 is used to obtain frequency domain data and visual data based on the process data.

[0163] The detection and quantization module 83 is used to input frequency domain data and visual data into the trained visual-frequency domain bimodal fusion network to detect oscillation types and quantize oscillation intensity. The visual-frequency domain bimodal fusion network is trained using a sample training set, and includes an oscillation detection network and an oscillation quantization network. The oscillation detection network and the oscillation quantization network have the same network structure, but use different activation functions. The oscillation types include non-oscillation, regular oscillation, and irregular oscillation, and the quantization results include period count values, sparsity values, and regularity values.

[0164] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0165] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for monitoring industrial process oscillations based on a vision-frequency domain dual-modal fusion network, characterized in that, include: Acquire process data output from the industrial process loop under test; Frequency domain data is obtained based on the process data; Visual data is obtained by imaging the process data using a preset image resolution. The frequency domain data and visual data are input into a trained visual-frequency domain bimodal fusion network to monitor oscillation types and quantify oscillation intensity. The visual-frequency domain bimodal fusion network is trained using a sample training set and includes an oscillation detection network and an oscillation quantization network. The oscillation detection network and the oscillation quantization network have the same network structure but use different activation functions. Oscillation types include non-oscillation, regular oscillation, and irregular oscillation, and the quantization results include period count, sparsity, and regularity values. Specifically, inputting the frequency domain data and visual data into the trained visual-frequency domain bimodal fusion network to detect oscillation types and quantify oscillation intensity includes: inputting the frequency domain data and visual data into the trained oscillation detection network to detect oscillation types; and inputting the frequency domain data and visual data into the trained oscillation quantization network to quantify oscillation intensity. The frequency domain data and visual data are input into a trained oscillation detection network to detect oscillation types. Specifically, this includes: inputting the visual data into a visual modality branch network for feature extraction to obtain visual features; inputting the frequency domain data into a frequency domain modality branch network for feature extraction to obtain frequency domain features; inputting the visual features and the frequency domain features into a feature commonality fusion network for feature fusion to obtain fused features; and inputting the fused features into a fully connected layer for mapping processing and transformation through the output of activation function neurons to obtain the detected oscillation type. The oscillation detection network includes a visual modality branch network, a frequency domain modality branch network, a feature commonality fusion network, and a fully connected layer. The visual modality branch network includes multiple depthwise separable convolutional layers, multiple two-dimensional standard convolutional layers, and a global average pooling layer. The frequency domain modality branch network includes multiple one-dimensional convolutional layers and multiple max-pooling layers. The loss function used during the training process of the visual-frequency domain bimodal fusion network is: ； in, For the total loss, and These are visual features and frequency domain features, respectively. and These are the weight terms, To fuse features, in the oscillation detection network Cross-entropy loss is used in oscillatory quantization networks. This represents the mean square error loss.

2. The industrial process oscillation monitoring method based on a vision-frequency domain dual-modal fusion network according to claim 1, characterized in that, Frequency domain data is obtained from the process data, specifically including: The process data is normalized, and the amplitude information is extracted by discrete Fourier transform of the normalized process data to obtain frequency domain data.

3. The industrial process oscillation monitoring method based on a vision-frequency domain dual-modal fusion network according to claim 1, characterized in that, Before acquiring the process data output from the industrial process loop to be tested, the following steps are also included: Generating artificial process time series using industrial data artificial simulation algorithms; The sample visual data and sample frequency domain data are obtained based on the time series of the artificial process. The sample training set is obtained based on the sample visual data and sample frequency domain data.

4. The industrial process oscillation monitoring method based on a vision-frequency domain dual-modal fusion network according to claim 3, characterized in that, Generating artificial process time series using industrial data artificial simulation algorithms, specifically including: Initialize time series variables; Generate Gaussian noise terms and disturbance data of a preset time length based on time series variables; The perturbation data is smoothed using a first filter, and the amplitude of the smoothed perturbation data is normalized. If the artificial process time series to be generated is oscillating data, then periodic signal data is generated according to the preset time length; The periodic signal data is smoothed using a second filter, and the smoothed periodic signal data is then normalized. The artificial process time series is generated based on the normalized periodic signal data, the amplitude-normalized perturbation data, and the Gaussian noise term; If the artificial process time series to be generated is non-oscillating data, then the artificial process time series is generated based on the amplitude-normalized perturbation data and the Gaussian noise term.

5. The industrial process oscillation monitoring method based on a vision-frequency domain dual-modal fusion network according to claim 4, characterized in that, The transfer function of the first filter is: ； The transfer function of the second filter is: ； in, and Let be the transfer functions of the first filter and the second filter, respectively. The complex frequency of the Laplace transform, This is a smoothing factor.

6. An industrial process oscillation monitoring device based on a vision-frequency domain dual-modal fusion network, characterized in that, include: The acquisition module is used to acquire process data output from the industrial process loop to be tested. The preprocessing module is used to obtain frequency domain data based on the process data; Visual data is obtained by imaging the process data using a preset image resolution. The detection and quantization module is used to input frequency domain data and visual data into a trained visual-frequency domain bimodal fusion network to detect oscillation types and quantify oscillation intensity. The visual-frequency domain bimodal fusion network is trained using a sample training set and includes an oscillation detection network and an oscillation quantization network. The oscillation detection network and the oscillation quantization network have the same network structure but use different activation functions. Oscillation types include non-oscillation, regular oscillation, and irregular oscillation, and quantization results include period count, sparsity, and regularity values. Specifically, inputting frequency domain data and visual data into the trained visual-frequency domain bimodal fusion network to detect oscillation types and quantify oscillation intensity includes: inputting the frequency domain data and visual data into the trained oscillation detection network to detect oscillation types; and inputting the frequency domain data and visual data into the trained oscillation quantization network to quantize oscillation intensity. The frequency domain data and visual data are input into a trained oscillation detection network to detect oscillation types. Specifically, this includes: inputting the visual data into a visual modality branch network for feature extraction to obtain visual features; inputting the frequency domain data into a frequency domain modality branch network for feature extraction to obtain frequency domain features; inputting the visual features and the frequency domain features into a feature commonality fusion network for feature fusion to obtain fused features; and inputting the fused features into a fully connected layer for mapping processing and transformation through the output of activation function neurons to obtain the detected oscillation type. The oscillation detection network includes a visual modality branch network, a frequency domain modality branch network, a feature commonality fusion network, and a fully connected layer. The visual modality branch network includes multiple depthwise separable convolutional layers, multiple two-dimensional standard convolutional layers, and a global average pooling layer. The frequency domain modality branch network includes multiple one-dimensional convolutional layers and multiple max-pooling layers. The loss function used during the training process of the visual-frequency domain bimodal fusion network is: ； in, For the total loss, and These are visual features and frequency domain features, respectively. and These are the weight terms, To fuse features, in the oscillation detection network Cross-entropy loss is used in oscillatory quantization networks. This represents the mean square error loss.

Images