A Stability Control Method for Rolling Process Based on Multimodal Fusion Fault Diagnosis

Abstract

This invention discloses a rolling process stability control method based on multimodal fusion fault diagnosis, belonging to the field of intelligent control technology in steel production. The method includes: collecting rolling process fault data, establishing an original fault dataset, and dividing it into training and testing sets; simulating fault data using mechanism models corresponding to each fault type, and performing data augmentation on the training set to obtain a mechanism-enhanced dataset; generating fault data using a temporal generative adversarial network, and performing data augmentation on the mechanism-enhanced dataset to obtain a fused enhanced dataset; constructing a multimodal fusion fault diagnosis model based on the fused enhanced fault dataset; using the established fault diagnosis model for rolling process fault diagnosis, and formulating rolling stability control strategies based on the diagnosis results. This invention's control method fuses the time-domain and frequency-domain features of faults to achieve high-precision fault diagnosis, and then implements corresponding control strategies based on the diagnosis results, thereby achieving online control of rolling stability.

Description

Technical Field

[0001] This invention belongs to the field of intelligent control technology in steel production, and relates to a method for stability control of rolling process based on multimodal fusion fault diagnosis. Background Technology

[0002] As the requirements for steel products in various industrial sectors continue to increase, higher demands are being placed on the control level of the rolling process. The rolling process involves numerous parameters and is accompanied by various unknown disturbances, leading to decreased stability. This can result in quality defects or even major production accidents. Timely diagnosis of faults caused by unknown disturbances and the implementation of corresponding control strategies are crucial to ensuring the stability of the rolling process.

[0003] Currently, with the development of computer technology and information technology, several stability control methods for the rolling process have been applied. Chinese patent "CN104772339A" proposes a method to improve rolling stability during the edge drop control process of steel plates, proposing specific control strategies for the first few coils before start-up and the head and tail sections, and a new roll shifting strategy for the stable rolling stage. Chinese patent "CN103949481B" proposes a segmented flatness control method that balances the rolling stability and quality of hot-rolled strip steel. It segments the entire length of the hot-rolled strip steel and applies different flatness control strategies for each segment, thus addressing the impact of poor flatness during strip threading and tail-throwing on rolling stability. Chinese patent "CN112692058A" proposes a rolling process to control the thickness stability of non-oriented silicon steel. By reducing the silicon steel finishing mill inlet temperature and adjusting the production parameters of the roughing, finishing, and coiling zones, the thickness stability of the non-oriented silicon steel is ensured. Chinese patent "CN113020286B" proposes a method and device for controlling the stability of the rolling process. Based on the deviation between the calculated and measured rolling force of the previous pass, a correction coefficient for the rolling force of the current pass is obtained, and this correction is used to obtain the set rolling force for the current pass, thus improving the stability of the rolling process. Chinese patent application "CN118437771A" proposes a method for controlling the rolling stability of the head of thin strip steel under ultra-rapid cooling conditions. By adjusting the water spray volume in the upper and lower manifolds of the ultra-rapid cooling zone, it solves the problem of the head affecting the rolling quality of thin strip steel under ultra-rapid cooling mode, effectively improving the rolling quality of subsequent processes and reducing equipment wear and accident rates.

[0004] Currently, the aforementioned studies have achieved some results in controlling the stability of the rolling process, but certain shortcomings remain. Firstly, the rolling process involves numerous parameters and complex operating conditions; the studies only target a single parameter, making it difficult to systematically and comprehensively control the stability of the rolling process. Secondly, these studies only improve strip quality and thus rolling stability through various strategies, rather than directly addressing stability issues caused by unknown faults in the rolling process. Summary of the Invention

[0005] To address the aforementioned technical problems, the present invention aims to provide a rolling process stability control method based on multimodal fusion fault diagnosis. This method integrates the time-domain and frequency-domain characteristics of faults to achieve high-precision fault diagnosis, and then implements corresponding control strategies based on the diagnosis results, thereby realizing online control of rolling stability.

[0006] This invention provides a rolling process stability control method based on multimodal fusion fault diagnosis, comprising:

[0007] Step 1: Collect rolling process fault data, establish the original fault dataset, and divide it into training set and test set;

[0008] Step 2: Simulate and generate fault data using the mechanism models corresponding to each fault type, perform data augmentation on the training set, and obtain the mechanism-augmented dataset;

[0009] Step 3: Use a temporal generative adversarial network to generate fault data, perform data augmentation on the mechanism enhancement dataset, and obtain a fused enhanced dataset;

[0010] Step 4: Construct a multimodal fusion fault diagnosis model based on the fusion-enhanced fault dataset;

[0011] Step 5: Use the established fault diagnosis model for fault diagnosis in the rolling process, and formulate rolling stability control strategies based on the diagnosis results.

[0012] This invention discloses a rolling process stability control method based on multimodal fusion fault diagnosis. First, it enhances the number of fault samples for three types: mill vibration, roll eccentricity, and controller performance degradation, using a temporal generative adversarial network and a mechanistic model. Next, it extracts one-dimensional frequency domain features from the fault data using one-dimensional SE-ResNet and two-dimensional grayscale features using two-dimensional EfficientNetV2, and then fuses these features. A multimodal fusion fault diagnosis model is then established to diagnose faults in the rolling process and determine the fault type. Finally, based on the diagnosis results, corresponding stability control strategies are implemented for the rolling process, achieving fault diagnosis-based rolling process stability control. The rolling process stability control method based on multimodal fusion fault diagnosis proposed in this invention can achieve high-precision fault diagnosis in the rolling process and, according to the corresponding stability control strategies, significantly improves the stability control level of the rolling process, making it widely applicable to rolling production. Attached Figure Description

[0013] Figure 1 This is a flowchart of a rolling process stability control method based on multimodal fusion fault diagnosis according to the present invention;

[0014] Figure 2 This is a structural diagram of a multimodal fusion fault diagnosis model that incorporates data augmentation. Detailed Implementation

[0015] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples.

[0016] To verify the effectiveness of the rolling process stability control method based on multimodal fusion fault diagnosis of the present invention, the stability of the cold continuous rolling process is controlled using a cold rolling production line as an example.

[0017] like Figure 1 As shown, the present invention provides a rolling process stability control method based on multimodal fusion fault diagnosis, comprising the following steps:

[0018] Step 1: Collect fault data during the rolling process, establish an original fault dataset, and divide it into a training set and a test set, specifically:

[0019] Step 1.1: Collect rolling process fault data and establish an original fault dataset; the rolling process fault data includes: mill vibration data, roll eccentricity data and controller performance degradation data.

[0020] Step 1.2: While ensuring that the proportion of each type of fault data remains unchanged, randomly select 80% of the original fault dataset as the training set to build the fault diagnosis model, and use the remaining data as the test set to evaluate the model's diagnostic performance.

[0021] In this embodiment, the original fault dataset includes: 90 mill vibration data points, 45 roll eccentricity data points, and 30 controller performance degradation data points. 80% of the original fault dataset, namely 72 mill vibration data points, 36 roll eccentricity data points, and 24 controller performance degradation data points, are randomly selected as the training set to build the fault diagnosis model. The remaining data is used as the test set to evaluate the model's diagnostic performance.

[0022] Step 2: Simulate and generate fault data using the mechanism models corresponding to each fault type, and perform data augmentation on the training set to obtain the mechanism-augmented dataset, specifically:

[0023] Step 2.1: While ensuring that the number of samples for each type of fault data is equal, determine the type of fault data that needs to be generated by mechanistic model simulation.

[0024] Step 2.2: Use the corresponding mechanism model in the following formula to simulate and generate the corresponding fault waveform, where the mill vibration model is used to simulate and generate the mill vibration waveform, the roll eccentricity model is used to simulate and generate the roll eccentricity waveform, and the controller model is used to generate the controller performance degradation waveform.

[0025] The formula for the mill vibration model is:

[0026]

[0027] in, Roll stiffness, For the quality of the rolls, This refers to the dynamic change in rolling force. Sampling time, , The amplitude of the rolling mill vibration waveform. The waveform represents the vibration of the rolling mill; it is set randomly. , This generates the mill vibration waveform.

[0028] The formula for the roll eccentricity model is:

[0029]

[0030] in, This is the waveform of the eccentric roll; Sampling time; , , The amplitude of the eccentric waveform of the roll; The frequency is one times that of the support roller. This is twice the frequency of the support roller. The frequency of the working roller; The phase corresponding to the first harmonic frequency of the support roller. The phase corresponding to the second harmonic frequency of the support roller. The phase corresponding to the frequency of the working roller is determined by random setting. , , as well as , , The eccentric waveform of the roll is simulated and generated.

[0031] The controller model formula is:

[0032]

[0033] in, The waveform represents the degradation of controller performance. The residuals between the actual values ​​and target values ​​of each control variable. The proportional gain of the PI controller. The integral coefficient of the PI controller is set randomly. and This generates a waveform indicating a decrease in controller performance.

[0034] Step 2.3: Based on the generated fault waveform, add ±1% random error to generate the corresponding fault data.

[0035] Step 2.4: Add the simulated fault data to the training set to obtain the mechanism augmentation dataset. The number of samples for each type of fault in the mechanism augmentation dataset is equal. That is, after adding fault data to the mechanism model, ensure that the number of samples for each type of fault is equal, namely 72 mill vibration data, 72 roll eccentricity data, and 72 controller descent data.

[0036] Step 3: Use a Time-series Generative Adversarial Network (TSGAN) to generate fault data, augment the mechanism augmentation dataset, and obtain a fused augmented dataset, specifically:

[0037] Step 3.1: Map the static features and time-series features of the fault data in the mechanism-enhanced dataset to a low-dimensional latent space using an embedding function to obtain the corresponding latent codes, as shown in the following formula:

[0038]

[0039]

[0040] in, The static characteristics of fault data For the static feature space of fault data, For the embedding function of static features, For static features, the latent code in a low-dimensional latent space. for Time series characteristics of fault data at any given time. For embedding functions of time series features, for The latent code in the low-dimensional latent space for the time-series features of fault data at any given time; for The latent code in the low-dimensional latent space for the time series features of fault data at time -1.

[0041] In practice, the embedding functions for static features and time-series features employ a recurrent neural network (RNN), consisting of an input layer, two hidden layers, and an output layer connected sequentially. The output of the second hidden layer is passed to the output layer via a ReLU activation function.

[0042] Step 3.2: Generate data in the low-dimensional latent space using a generator, as shown in the following formula:

[0043]

[0044]

[0045]

[0046] in, For a low-dimensional latent space of static features, For the low-dimensional latent space of time series features, The spatial range of the random vectors generated by static features in the latent space. The spatial range of the random vectors generated by time series features in the latent space. For static features, generate random vectors in the low-dimensional latent space. A generator for static features. The latent code for the generated static features in the low-dimensional latent space. for The time series features at each time step generate random vectors in the low-dimensional latent space. A generator for time series features. For generated The latent code of time series features in a low-dimensional latent space.

[0047] In practice, the generator's network structure includes a first fully connected layer, two LSTM layers, a second fully connected layer, and an output layer connected in sequence; the output of the second fully connected layer is passed to the output layer through the tanh activation function.

[0048] Step 3.3: Determine whether to accept the generated data using a discriminator, as shown in the following formula:

[0049]

[0050]

[0051]

[0052]

[0053] in, The classification of static features for generating fault data includes: accepting the generation of potential codes and not accepting the generation of potential codes; To generate a discriminant function for static features; The classification of time-series features for generating fault data includes accepting the generation of potential codes and not accepting the generation of potential codes; To generate a discriminant function for time series features, for The forward hidden state sequence at each time step. for The backward hidden state sequence at time step It is a forward loop function. It is a backward loop function.

[0054] In practice, the discriminator's network structure consists of two LSTM layers, a fully connected layer, and an output layer connected in sequence. The output of the fully connected layer is passed to the output layer through a sigmoid activation function. The output of the output layer is the probability of accepting the generation of latent codes. The probability is then rounded to a category, i.e., whether or not to accept the generation of latent codes.

[0055] Step 3.4: Remap the generated latent codes back to the feature space of the mechanism-enhanced dataset fault data using a recovery function, as shown in the following formula:

[0056]

[0057]

[0058] in, Static features for generating fault data, For static features, To generate time series features for fault data, It is a recovery function for time series features;

[0059] In practice, the recovery function uses a feedforward neural network, which consists of an input layer, two hidden layers, and an output layer connected in sequence; the output of the second hidden layer is passed to the output layer through the ReLU activation function.

[0060] Step 3.5: Define the loss function, as shown in the following formula:

[0061]

[0062] in, The reconstruction loss is used to calculate the residual between the generated fault data features and the actual fault data features. For all static features and time series features Expected value To enhance the Euclidean distance between the static features of the fault data in the mechanism enhancement dataset and the static features of the generated fault data, To enhance the distribution of static and time-series features of fault data in the latent space, this method employs reconstruction loss. A network used for iterating over embedded functions and recovery functions.

[0063]

[0064] in, Unsupervised loss is used to calculate the loss between the static features and time-series features of the fault data generated by the generator. To enhance the classification of static features of fault data in the mechanism-based dataset, To enhance the classification of time-series features of fault data in the mechanism-based dataset, The distribution of static and time-series features of fault data in the latent space is used to generate unsupervised loss. The network is used for iterating the generator and the discriminator so that the discriminator can give the most accurate classification of the generated latent code, i.e., whether to accept the generated latent code.

[0065]

[0066] in, The supervised loss is used to calculate the residuals between the latent codes of the time series features generated by the generator and the latent codes of the true time series features. (Supervised Loss) A network used for iterating over a generator in order to minimize the generator's residuals.

[0067] In this embodiment, the augmented dataset is called the fused augmented dataset, and the number of fault samples of each type generated is twice the number of fault samples of each type in the mechanism augmented dataset. The Adam optimizer is selected, with a batch size of 128 and a sliding window size of 10. The generator has a learning rate of 0.001, a latent vector dimension of 100, 3 network layers, and tanh activation function. The discriminator has a learning rate of 0.002, a training frequency ratio of 5, leaky ReLU activation function, and dropout of 0.3. After TSGAN data augmentation, the fused augmented dataset contains 216 mill vibration data points, 216 roll eccentricity data points, and 216 controller descent data points.

[0068] Step 4: Construct a multimodal fusion fault diagnosis model based on the fusion-enhanced fault dataset, specifically:

[0069] Step 4.1: Wavelet packet decomposition is used to eliminate high-frequency noise in the fault data of the fused and enhanced fault dataset.

[0070] Step 4.2: Extract the frequency domain information of the fault data processed in Step 4.1, including frequency and amplitude, through Fast Fourier Transform.

[0071] Step 4.3: Convert the fault data processed in Step 4.1 into a two-dimensional image, and then into a grayscale image.

[0072] Step 4.4: Construct a one-dimensional frequency domain feature for extracting frequency domain information from fault data using a one-dimensional SE-ResNet. For example... Figure 2 As shown, the one-dimensional SE-ResNet includes: a first convolutional module, eight attention residual modules, and a first global average pooling layer connected in sequence. Step 4.4 specifically involves:

[0073] Step 4.4.1: Construct the first convolutional module, which includes a first convolutional layer and a max pooling layer. The kernel size of the first convolutional layer is 7×1, and the number of kernels is 16.

[0074] Step 4.4.2: Construct a first and a second attention residual module with identical structures. This includes two sequentially connected second convolutional layers and a batch normalization (BN) layer. Residual connections are established to add the output of the batch normalization layer to the module input, and then the result is passed to the attention layer (SE, Squeeze, and Excitation) via a swish activation function. The second convolutional layer has a kernel size of 3×1 and 16 kernels.

[0075] Step 4.4.3: Construct a third and fourth attention residual module with identical structures. This includes two sequentially connected third convolutional layers and a batch normalization layer. A residual connection is established, and the output of the batch normalization layer is added to the module input before being passed to the attention layer via a swish activation function. The third convolutional layer has a kernel size of 3×1 and a kernel count of 32.

[0076] Step 4.4.4: Construct a fifth and a sixth attention residual module with identical structures. This includes two sequentially connected fourth convolutional layers and a batch normalization layer. A residual connection is established, and the output of the batch normalization layer is added to the module input before being passed to the attention layer via a swish activation function. The fourth convolutional layer has a kernel size of 3×1 and a kernel count of 64.

[0077] Step 4.4.5: Construct a seventh and eighth attention residual module with identical structures; including two fifth convolutional layers and a batch normalization layer connected in sequence, establish residual connections, add the output of the batch normalization layer to the module input, and then pass it to the attention layer through the swish activation function; the kernel size of the fifth convolutional layer is 3×1, and the number of convolutional kernels is 128.

[0078] Step 4.4.6: Construct the first global average pooling layer to obtain the one-dimensional frequency domain features of the fault data.

[0079] Step 4.5: Construct a two-dimensional EfficientNetV2 to extract two-dimensional grayscale features from the fault data. For example... Figure 2 As shown, the two-dimensional EfficientNetV2 includes: a first two-dimensional convolutional layer, three Fused-MBConv modules, three MBConv modules, and a second global average pooling layer connected in sequence; step 4.5 specifically includes:

[0080] Step 4.5.1: Construct the first two-dimensional convolutional layer (cov2d1), with 1 input channel, a kernel size of 3×3, and 16 kernels.

[0081] Step 4.5.2: Construct the first Fused-MBConv module, expansion=1, the structure of which includes: a two-dimensional convolutional layer with a kernel size of 3×3 and a kernel number of 16, a batch normalization layer, an attention layer passed through the swish activation function, a two-dimensional convolutional layer with a kernel size of 1×1, and a batch normalization layer.

[0082] Step 4.5.3: Construct the second Fused-MBConv module, expansion=2, the structure of which includes: a two-dimensional convolutional layer with a kernel size of 3×3 and a kernel number of 32, a batch normalization layer, an attention layer passed through the swish activation function, a two-dimensional convolutional layer with a kernel size of 1×1, and a batch normalization layer.

[0083] Step 4.5.4: Construct the third Fused-MBConv module, expansion=4, the structure of which includes: a two-dimensional convolutional layer with a kernel size of 3×3 and a kernel number of 64, a batch normalization layer, an attention layer passed through the swish activation function, a two-dimensional convolutional layer with a kernel size of 1×1, and a batch normalization layer.

[0084] Step 4.5.5: Construct the first MBConv module, expansion=4, the structure of which includes: a 2D convolutional layer with a kernel size of 1×1, a batch normalization layer, a depthwise separable convolutional layer (with a kernel size of 3×3 and a kernel count of 64) passed through the swish activation function, a batch normalization layer, an attention layer passed through the swish activation function, a 2D convolutional layer with a kernel size of 1×1, and a batch normalization layer.

[0085] Step 4.5.6: Construct the second MBConv module, expansion=4, the structure of which includes: a 2D convolutional layer with a kernel size of 1×1, a batch normalization layer, a first depthwise separable convolutional layer (3×3 kernel size, 64 kernels) passed through the swish activation function, a batch normalization layer, a first attention layer passed through the swish activation function, a 2D convolutional layer with a kernel size of 1×1, a batch normalization layer, a second depthwise separable convolutional layer (3×3 kernel size, 64 kernels) passed through the swish activation function, a batch normalization layer, a second attention layer passed through the swish activation function, a 2D convolutional layer with a kernel size of 1×1, and a batch normalization layer.

[0086] Step 4.5.7: Construct the third MBConv module, expansion=4, whose structure includes: a 2D convolutional layer with a kernel size of 1×1, a batch normalization layer, a first depthwise separable convolutional layer (3×3 kernel size, 128 kernels) passed through the swish activation function, a batch normalization layer, a first attention layer passed through the swish activation function, a 2D convolutional layer with a kernel size of 1×1, a batch normalization layer, a second depthwise separable convolutional layer (3×3 kernel size, 128 kernels) passed through the swish activation function, a batch normalization layer, a second attention layer passed through the swish activation function, a 2D convolutional layer with a kernel size of 1×1, and a batch normalization layer.

[0087] Step 4.5.8: Construct a second global average pooling layer to obtain the two-dimensional grayscale features of the fault data;

[0088] Step 4.6: Fuse one-dimensional frequency domain features and two-dimensional grayscale image features, and input them into a fully connected neural network. The structure of the fully connected neural network is: 3 hidden layers and [512, 128, 32] hidden layer nodes.

[0089] Step 4.7: Define the loss function for the fault diagnosis model, as shown in the following formula:

[0090]

[0091] in, The probability of correctly diagnosing a faulty sample. It is a regulating factor.

[0092] The multimodal fusion fault diagnosis model of this invention uses the Adam optimizer with a learning rate of 0.001. The performance of the established multimodal fusion fault diagnosis model is evaluated using a test set. MAUC, G-Mean, and F1-Macro are the evaluation metrics, and their formulas are as follows:

[0093]

[0094] In the formula, and These are the probabilities that a sample actually belonging to category j is classified as category i and the probabilities that a sample actually belonging to category i is classified as category j, respectively.

[0095]

[0096] In the formula, Indicates the total number of categories. This represents the recall rate for each category.

[0097]

[0098] In the formula, It is the F1 score for each category.

[0099]

[0100] In the formula, and These are precision and recall for a specific category, respectively.

[0101]

[0102] In the formula, and These are the true positive rate and false positive rate for a certain category, respectively.

[0103]

[0104] In the formula, It is the false negative rate for a certain category.

[0105] The proposed multimodal fault diagnosis model based on fused data augmentation (FDA-MMFD) is compared with the single-dimensional fault diagnosis model based on fused data augmentation (FDA-2DFD), the multimodal fault diagnosis model based on TSGAN data augmentation (TSGAN-MMFD), and the multimodal fault diagnosis model without data augmentation (MMFD). The diagnostic performance comparison of each method is shown in Table 1.

[0106] Table 1. Comparison of Diagnostic Performance

[0107]

[0108] The proposed multimodal fusion fault diagnosis model with data augmentation exhibits a MAUC of 0.9561, a G-Mean of 0.9433, and an F1-Macro of 0.9428, significantly outperforming the compared models in all three metrics. Furthermore, the proposed data augmentation strategy and one-dimensional and two-dimensional multimodal feature extraction strategies substantially improve the model's diagnostic performance. In summary, the proposed multimodal fusion fault diagnosis model with data augmentation demonstrates high diagnostic accuracy and can meet the requirements for stability control in cold rolling processes.

[0109] Step 5: Use the established fault diagnosis model for fault diagnosis in the rolling process, and formulate a rolling stability control strategy based on the diagnosis results, specifically:

[0110] Step 5.1: Apply the fault diagnosis model established in Step 4 to diagnose faults in the rolling process and obtain the fault types.

[0111] Step 5.2: Based on the diagnosed fault type, implement the corresponding rolling stability control strategy for the rolling process.

[0112] If the diagnosis is mill vibration, the mill vibration can be controlled by simultaneously increasing the tension between the stands and decreasing the rolling force.

[0113] If the prediction result is roll eccentricity, then the frequency, amplitude and phase of roll eccentricity are analyzed using fast Fourier transform, and the opposite amplitude is applied to the roll gap according to the analyzed frequency and phase to compensate for the thickness deviation caused by roll eccentricity.

[0114] If the prediction result indicates a decrease in controller performance, the controller parameters are optimized using the Crayfish Optimization Algorithm (COA) to improve the system's control capabilities.

[0115] The above description is only a preferred embodiment of the present invention and is not intended to limit the ideas of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A rolling process stability control method based on multi-modal fusion fault diagnosis, characterized in that, The application relates to a rolling process fault diagnosis method based on multi-modal fusion data enhancement, and belongs to the field of rolling process fault diagnosis. The method comprises the following steps: Step 1: collecting rolling process fault data, establishing an original fault data set, and dividing the original fault data set into a training set and a test set; Step 2: generating fault data by using a mechanism model corresponding to each fault type, performing data enhancement on the training set, and obtaining a mechanism enhanced data set; Step 3: generating fault data by using a time sequence generative adversarial network, performing data enhancement on the mechanism enhanced data set, and obtaining a fusion enhanced data set; Step 4: constructing a multi-modal fusion fault diagnosis model based on the fusion enhanced data set; Step 5: using the established fault diagnosis model for rolling process fault diagnosis, and formulating a rolling stability control strategy based on the diagnosis result. The step 1 is specifically as follows: Step 1.1: collecting rolling process fault data, and establishing an original fault data set; the rolling process fault data comprises mill vibration data, roll eccentricity data and controller performance degradation data; Step 1.2: under the condition that the proportions of various fault data are unchanged, randomly selecting 80% of the original fault data set as a training set for constructing a fault diagnosis model, and using the rest of the data as a test set for evaluating the diagnosis performance of the model; The step 2 is specifically as follows: Step 2.1: under the condition that the sample numbers of various types of fault data are equal, determining the types of fault data that need to be generated by using a mechanism model; Step 2.2: generating corresponding fault waveforms by using the mechanism model corresponding to the formula, wherein the mill vibration model is used for generating mill vibration waveforms, the roll eccentricity model is used for generating roll eccentricity waveforms, and the controller model is used for generating controller performance degradation waveforms; wherein, Roll stiffness, is the roll mass, is the dynamic roll force variation, is the sampling time, , is the amplitude of the mill vibration waveform, is the mill vibration waveform; the mill vibration waveform is generated by randomly setting , the mill vibration waveform; The formula of the mill vibration model is as follows: wherein, is a roll eccentricity waveform; is a sampling time; , , is an amplitude of the roll eccentricity waveform; is a one-fold frequency of the backup roll, is a two-fold frequency of the backup roll, is a frequency of the work roll; is a phase corresponding to the one-fold frequency of the backup roll, is a phase corresponding to the two-fold frequency of the backup roll, is a phase corresponding to the frequency of the work roll, which is set randomly , , and , , , the roll eccentricity waveform is generated by simulation; The formula of the roll eccentricity model is as follows: wherein, is a controller performance degradation waveform, is a residual of each control amount actual value and target value, is a proportional coefficient of a PI controller, is an integral coefficient of a PI controller, and is set randomly and a controller performance degradation waveform is generated; The formula of the controller model is as follows: Step 2.3: on the basis of the generated fault waveforms, adding random errors of plus or minus 1%, and generating corresponding fault data; 2. The rolling process stability control method based on multi-modal fusion fault diagnosis according to claim 1, characterized in that, Step 2.4: adding the simulated fault data to the training set, and obtaining a mechanism enhanced data set; the sample numbers of various types of fault data in the mechanism enhanced data set are equal. The step 3 is specifically as follows: wherein, is a static feature of the failure data, is a static feature space of the failure data, is an embedding function of the static feature, is a latent code of the static feature in a low-dimensional latent space, is a time series feature of the failure data at time t, is a time series feature of the failure data at time t, is an embedding function of the time series feature, is a latent code of the time series feature of the failure data at time t in a low-dimensional latent space, is a latent code of the time series feature of the failure data at time t in a low-dimensional latent space, is a latent code of the time series feature of the failure data at time t in a low-dimensional latent space, is a latent code of the time series feature of the failure data at time t in a low-dimensional latent space. Step 3.1: mapping the static features and time sequence features of the fault data in the mechanism enhanced data set to a low-dimensional latent space by using an embedding function, and obtaining corresponding latent codes, as shown in the following formula: The embedding function of the static features and the embedding function of the time sequence features adopt a recurrent neural network, which comprises an input layer, two hidden layers and an output layer connected in sequence; wherein, is a low-dimensional latent space for static features, is a low-dimensional latent space for time-series features, is a space range of generated random vectors for static features in the latent space, is a space range of generated random vectors for time-series features in the latent space, is a generated random vector for static features in the low-dimensional latent space, is a generator for static features, is a latent code for the generated static features in the low-dimensional latent space, is a generated random vector for time-series features at time t in the low-dimensional latent space, is a generator for time-series features, is a generated is a latent code for the generated time-series features at time t in the low-dimensional latent space; Step 3.2: generating data in the low-dimensional latent space by using a generator, as shown in the following formula: The network structure of the generator comprises a first full connection layer, two LSTM layers, a second full connection layer and an output layer connected in sequence; wherein, a classification to generate static features of the failure data, including accepting generating latent codes and not accepting generating latent codes; a discriminant function to generate static features, a classification to generate time series features of the failure data, including accepting generating latent codes and not accepting generating latent codes; a discriminant function to generate time series features, for a sequence of forward hidden states at time t, for a sequence of backward hidden states at time t, a forward recurrent function, a backward recurrent function. Step 3.3: judging whether to accept the generated data by using a discriminator, as shown in the following formula: The network structure of the discriminator comprises two LSTM layers, a full connection layer and an output layer connected in sequence; the output of the full connection layer is transmitted to the output layer through a sigmoid activation function, and the output result of the output layer is the probability of accepting the generated latent code; Step 3.4: mapping the generated latent code back to the feature space of the fault data in the mechanism enhanced data set by using a recovery function, as shown in the following formula: wherein, a static feature for generating the failure data, a recovery function for the static feature, a time series feature for generating the failure data, a recovery function for the time series feature; The recovery function adopts a feedforward neural network: including an input layer, two hidden layers and an output layer connected in turn; Step 3.5: define the loss function, the formula is as follows: wherein, is a reconstruction loss for calculating the residual between the generated failure data feature and the real failure data feature, is the expected value of all static features and time series features of the failure data of the mechanism augmented dataset, is the Euclidean distance between the static features of the failure data of the mechanism augmented dataset and the static features of the generated failure data, is the distribution of the static features and time series features of the failure data of the mechanism augmented dataset in the latent space; wherein, is an unsupervised loss for computing the static features and the time series features of the generated failure data by the generator, is a classification of the static features of the failure data of the mechanism augmented dataset, is a classification of the time series features of the failure data of the mechanism augmented dataset, is a distribution of the static features and the time series features of the generated failure data in the latent space; wherein, is a supervisory loss for computing a residual between the latent code generated by the generator for the time series feature and the latent code for the real time series feature. 3.The rolling process stability control method based on multi-modal fusion fault diagnosis according to claim 1, characterized in that, The step 4 is specifically: Step 4.1: adopt wavelet packet decomposition to eliminate the high-frequency noise of the fault data in the fusion enhanced fault data set; Step 4.2: extract the frequency domain information of the fault data processed in step 4.1 through fast Fourier transform, including frequency and amplitude; Step 4.3: convert the fault data processed in step 4.1 into a two-dimensional image, and then into a grayscale image; Step 4.4: construct a one-dimensional SE-ResNet to extract one-dimensional frequency domain features of the frequency domain information of the fault data; Step 4.5: construct a two-dimensional EfficientNetV2 to extract two-dimensional grayscale image features of the fault data; Step 4.6: fuse the one-dimensional frequency domain features and the two-dimensional grayscale image features, and input them into a fully connected neural network, the structure of the fully connected neural network is: the number of hidden layers is 3, and the number of hidden layer nodes is [512, 128, 32]; Step 4.7: define the loss function of the fault diagnosis model, the formula is as follows: wherein, is the probability of a correct diagnosis for a fault sample, is the adjustment factor. 4.The rolling process stability control method based on multi-modal fusion fault diagnosis according to claim 3, characterized in that, The one-dimensional SE-ResNet includes: a first convolutional module, eight attention residual modules and a first global average pooling layer connected in turn, and the step 4.4 is specifically: Step 4.4.1: construct a first convolutional module, including a first convolutional layer and a maximum pooling layer, the convolutional kernel size of the first convolutional layer is 7x1, and the number of convolutional kernels is 16; Step 4.4.2: construct a first attention residual module and a second attention residual module with the same structure; including two second convolutional layers and batch normalization layers connected in turn, a residual connection is established to add the output of the batch normalization layer to the module input, and then the swish activation function is transmitted to the attention layer; the convolutional kernel size of the second convolutional layer is 3x1, and the number of convolutional kernels is 16; Step 4.4.3: construct a third attention residual module and a fourth attention residual module with the same structure; including two third convolutional layers and batch normalization layers connected in turn, a residual connection is established to add the output of the batch normalization layer to the module input, and then the swish activation function is transmitted to the attention layer; the convolutional kernel size of the third convolutional layer is 3x1, and the number of convolutional kernels is 32; Step 4.4.4: construct a fifth attention residual module and a sixth attention residual module with the same structure; including two fourth convolutional layers and batch normalization layers connected in turn, a residual connection is established to add the output of the batch normalization layer to the module input, and then the swish activation function is transmitted to the attention layer; the convolutional kernel size of the fourth convolutional layer is 3x1, and the number of convolutional kernels is 64; Step 4.4.5: construct a seventh attention residual module and an eighth attention residual module with the same structure; including two fifth convolutional layers and batch normalization layers connected in turn, a residual connection is established to add the output of the batch normalization layer to the module input, and then the swish activation function is transmitted to the attention layer; the convolutional kernel size of the fifth convolutional layer is 3x1, and the number of convolutional kernels is 128; Step 4.4.6: Construct the first global average pooling layer to obtain the one-dimensional frequency domain features of the fault data. 5.The rolling process stability control method based on multi-modal fusion fault diagnosis according to claim 1, characterized in that, The step 5 is specifically: Step 5.1: applying the fault diagnosis model established in step 4 to diagnose the fault of the rolling process to obtain the fault type; Step 5.2: according to the diagnosed fault type, implementing the corresponding rolling stability control strategy for the rolling process. 6.The rolling process stability control method based on multi-modal fusion fault diagnosis according to claim 5, characterized in that, The step 5.2 is specifically: If the diagnosis result is mill vibration, the mill vibration is controlled by increasing the inter-stand tension and reducing the rolling force synchronously; If the prediction result is roll eccentricity, the frequency, amplitude and phase of the roll eccentricity are analyzed by using fast Fourier transform, and the opposite amplitude is applied to the roll gap according to the analyzed frequency and phase to compensate the thickness deviation caused by the roll eccentricity; If the prediction result is controller performance degradation, the controller parameters are optimized by using the crayfish optimization algorithm to improve the system control ability.

Images