Early warning method for faults in the flotation process based on BiLSTM to predict the deviation degree

Abstract

The present invention provides a method for early warning of faults in the flotation process based on predicting the deviation degree by BiLSTM: First, collect foam video images and extract the color features of the video sequence to construct a sample data set; Then, construct a time series prediction model based on ResNet50 and BiLSTM, and input the spatial features extracted by ResNet50 into the BiLSTM network for time series prediction of colors; Secondly, use the normal working data set to predict the time series color features, calculate the deviation degree according to the predicted value and the actual value, and determine the early warning threshold; Finally, collect foam video images in real time for color feature prediction, and give an early warning of production faults when the predicted deviation degree exceeds the limit. The prediction method of the time series color features of the method of the present invention has a small prediction deviation and a good fitting degree, the fault early warning time is effectively advanced, the early warning of faults in the flotation process can be realized, and the waste of mineral resources and reagents can be effectively reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of froth flotation, in particular to an early warning method for faults in the flotation process based on the prediction deviation degree of BiLSTM. Background Art

[0002] Froth flotation is an important ore dressing technology. Faults occurring during the flotation process will cause waste of mineral resources and reagents. An effective fault detection technology can help the flotation plant reduce reagent consumption and labor intensity, improve the mineral recovery rate, promote the optimal control of the flotation process and save production costs. However, the fault detection and diagnosis of the flotation process are often challenging. Generally, it is completed by experienced operators through frequent visual inspections of the foam appearance. However, this manual operation is always accompanied by serious response delays and measurement errors, and is very time-consuming and laborious.

[0003] To overcome this problem, machine vision indicates the flotation performance by extracting the visual features of the foam on the surface of the flotation cell, and transmits the extracted features to the operator or as the input of the process control system to adjust the flotation process accordingly. Many fault detection methods for the flotation process based on machine vision have been successively proposed. For example, wavelet transform is used to describe the foam grayscale image, and the spatial gray-level co-occurrence matrix is calculated to obtain the texture features of the foam image. The foam image is classified through these features to realize the identification of the fault state of the flotation process. However, the fault detection accuracy is greatly affected by the recognition accuracy; the PDF of the bubble size after approximate segmentation is obtained through a non-parametric kernel estimator, a dynamic weight model of the bubble size PDF is established, and stability analysis is carried out based on the weight model, and a threshold criterion determined by the stability condition is obtained for fault detection. However, the bubble shapes are different and the distribution is uneven, so that the same segmentation algorithm cannot be applied to all cases, resulting in low fault detection accuracy; after wavelet transform and reconstruction of the foam grayscale image, an equivalent bubble size feature is designed by calculating the white area of the binary image, and the range of the normal foam image is determined for flotation fault detection. However, the direction selectivity of wavelet transform is limited, and only limited direction information can be captured, which will introduce false information and serious appearance degradation when processing the image, affecting the fault detection accuracy.

[0004] An effective fault detection technology can help the flotation plant reduce reagent consumption, lower labor intensity, and improve the mineral recovery rate. Traditionally, the fault detection of the flotation process is mostly completed by experienced personnel by observing the foam appearance. This manual operation is accompanied by serious measurement errors and response delays, and is time-consuming and laborious. Subsequently, fault detection methods for the flotation process based on foam visual features have been successively proposed. However, these methods are for the online real-time detection of faults in the flotation process. When a fault occurs, the processing time left for the operator is relatively urgent. Summary of the Invention

[0005] In view of this, the purpose of the present invention is to provide an early warning method for flotation process faults based on BiLSTM predicted deviation degree, to achieve early warning of flotation process faults, and effectively reduce the waste of mineral resources and reagents.

[0006] To achieve the above object, the present invention adopts the following technical solutions: An early warning method for flotation process faults based on BiLSTM predicted deviation degree, comprising the following steps:

[0007] Step 1, obtain the historical flotation process data set, including the normal state data set and the fault state data set; perform data preprocessing such as data cleaning on the data set, and select the model prediction quantity;

[0008] Divide the normal state data set among them into a training set, a validation set and a test set according to a ratio of 6:2:2. Use the training set data as the input of the model, extract the mean value of the a-channel of the image color and the standard deviation of the a-channel of the training set video frame, these 2 measurement points parameters related to faults, and use the time series data of the 2 measurement points parameters as the predicted output value of the model. Train the fault prediction model, and test the training effect of the model through the validation set;

[0009] Step 3: First, save the fault prediction model trained in Step 2, use the normal test set as the input data of the model to verify the generalization ability and prediction accuracy of the model. Then, use the root mean square error RMSE, the mean absolute error MAE and the fitting coefficient R2 as the indicators to evaluate the prediction effect. Finally, combine the deviation degree and the warning threshold under the normal flotation process state calculated from the color measurement point parameter values extracted from the corresponding foam video frames to obtain the fault warning model;

[0010] Step 4: Test the fault warning model through the fault test set. Input the fault test set into the fault warning model to obtain the predicted value and calculate the deviation degree according to the predicted value. Perform early warning according to whether the deviation degree exceeds the warning threshold to verify the effectiveness of the warning model.

[0011] Step 5: Real-time collect foam video images and input them into the fault warning model to obtain the predicted value and calculate the deviation degree according to the predicted value. If the deviation degree exceeds the warning threshold, it is considered that there is a tendency for the flotation process to fail, and a warning signal is given.

[0012] In a preferred embodiment, the data preprocessing is specifically to preprocess the collected flotation foam image data set; specifically including:

[0013] (1) Data cleaning, clean the data by the Grubbs criterion method, and the specific steps of this criterion method are as follows:

[0014] Step S1: Assume x i(i = 1, 2, 3,..., N) is the observation data sample of the flotation process, and the observation data model with μ as the observation object is established as shown in the following formula:

[0015] x i = μ + p i , p i N(0, σ 2 )

[0016] Step S2: Calculate the sample mean and variance of xi respectively according to the following formula;

[0017]

[0018]

[0019] Step S3: Calculate the constructed statistic G such that the statistic G follows a transformation;

[0020]

[0021] Step S4: Calculate the statistic G α value according to the following formula. When G > G α , determine that the sample is abnormal and directly eliminate it;

[0022]

[0023] Step S5: Loop and execute Step S1 to Step S4 until the sample data set is cleaned;

[0024] (2) Data normalization. Before training the model, normalize the data through the following formula to compress the data into the range interval of [0, 1];

[0025] x std = (x - x min ) / (x max - x min ).

[0026] In a preferred embodiment, the construction of the flotation process fault warning model is specifically as follows:

[0027] Use the ResNet50 network to extract the features of the input flotation foam video frames. Input the pre - processed video frame data into the ResNet50 network to extract the spatial features of the data at each time point, and then transfer it to the BiLSTM network for time - series prediction;

[0028] After the ResNet50 network extracts features from the input data, after a period of accumulation, a sequence of feature values is formed, and the waveform formed by its fluctuation over time reflects the change in the flotation process state; the data features learned from the ResNet50 network are input into the BiLSTM network, which performs temporal encoding on the data features, obtains the feature vectors in time series, and then sends them into the fully connected layer to complete the prediction of the time series.

[0029] A fault prediction model for the flotation process is constructed based on ResNet50 and BiLSTM; the fault prediction model includes two parts: a training model and a testing model, and both parts of the model mainly include three parts: data input, deep network, and prediction output; in the data input part, first, the visible light video frames of the flotation foam are preprocessed and used as the input of the prediction model. Among them, the training set and the validation set are used as the input of the training model, and the normal test set is used as the input of the testing model; the deep network includes ResNet50 and BiLSTM networks. The ResNet50 network is used to extract the spatial features of the input data at each time point, and then transfer them to the BiLSTM network to learn the correlation between data, perform temporal encoding on the data features, and extract time series features from both positive and negative directions; in the prediction output part, the extracted temporal features are sent into the fully connected layer, and the prediction value is output through the fully connected layer. Among them, in the training model, the foam video image is converted to the CIElab space to extract two color measurement points, namely the mean and variance of the a channel, and the time series data of the two measurement point parameters are used as the prediction output value of the model to train the fault prediction model, so as to obtain the fault prediction model; while in the testing model, the prediction value of the color measurement point parameters at the next moment is output by the fully connected layer to complete the prediction of the time series, and then the normal deviation degree and the warning threshold are calculated by combining the color measurement point parameter values extracted from the corresponding foam video frames to further obtain the fault warning model.

[0030] In a preferred embodiment, the definition of the deviation degree and the warning strategy are specifically as follows:

[0031] The constructed ResNet50-BiLSTM flotation process fault prediction model is trained with the normal flotation process dataset. When the model receives new time series data, it can predict the next moment's data according to the learning results; when there is a trend of fault in the flotation process, the relevant monitoring variables will show a certain deviation from the normal state data. When this deviation exceeds the set safety threshold, it is determined that an early fault has occurred in the flotation process.

[0032] Calculate the residual r between the predicted value and the actual value output by the ResNet50-BiLSTM network model in this section through the following formula ij ;

[0033]

[0034] Where: r ij is the residual of variable i at time j; y ij and are respectively the actual value of variable i and the predicted value output by the model in this section at time j; according to the above selection of the model prediction quantity, there are 4 measurement point parameters, so the residual data at each moment forms a 4-dimensional vector, and the deviation degree of the flotation process from the normal state at this moment is calculated according to the following formula;

[0035] Calculate the deviation degree at each moment, so as to form a deviation degree sequence;

[0036]

[0037] There are multiple extreme points and non-stationarity in the deviation degree sequence. The generalized extreme value theory is used to calculate the early warning threshold, and the specific steps for its solution are as follows:

[0038] Decompose the deviation degree sequence into multiple smallest intervals with the same number of data points, and select 5 points as the smallest interval; then the calculation method of the maximum value M in each interval is shown in the following formula:

[0039] M = max{x 1 ,..., x n}

[0040] Where: x i is the value in a smallest interval; {x 1 ,..., x n} is a random sequence with the same distribution and independent;

[0041] Let the distribution function of the random sequence be F, then the relationship between the distribution of M and the distribution function F of {x 1 ,..., x n} is:

[0042] P r {M ≤ z} = P r {x 1 ≤ z,..., x n ≤ z} = P r {x 1 ≤ z} ×... × P r {x n ≤ z} = {F(z)} n

[0043] Since the distribution function F is unknown, it is assumed that there exist μ and σ satisfying:

[0044] P r {(M - μ) / σ} → G(z)

[0045] Where: μ is the location parameter; σ is the scale parameter; G() is the generalized extreme value distribution function, which is calculated according to the following formula:

[0046] G(z) = exp{-[1 + ((z - μ) / σ)] -1 / ξ}

[0047] Where: The generalized extreme value distribution function is defined on the set {z: 1 + ξ(z - μ) / σ > 0}, where the parameters μ and ξ satisfy: -∞ < μ < ∞, σ > 0, -∞ < ξ < ∞;

[0048] The location parameter μ, scale parameter σ and shape parameter ξ of the generalized extreme value distribution are obtained by the maximum likelihood estimation method, and the warning threshold is calculated by the following formula;

[0049] T h = μ - σ[1 - {-ln(1 - α)} -ξ / ξ

[0050] In summary, in the flotation process, if the deviation degree sequence is maintained within the warning threshold, it is determined that the flotation process is normal. If the deviation degree exceeds the warning threshold, it is determined that there is a tendency for the flotation process to malfunction, so as to achieve fault warning.

[0051] In a preferred embodiment, BiLSTM constructs two LSTM networks, a forward one and a backward one, to extract feature information; the same input sequence is respectively connected to the forward and backward LSTM networks, and then the internal structures of the two LSTM networks are changed;

[0052] The operation process of BiLSTM is as follows:

[0053] The update formula for forward propagation is as follows:

[0054]

[0055] The update formula for backward propagation is as follows:

[0056]

[0057] The formula for the output after superimposing the forward and backward network layers is as follows:

[0058]

[0059] Where: t represents the time series; represents the hidden layer vector at time t, and the arrow represents the direction; x t and y t respectively represent the input and output at time t; W xh 、W hh and W hyrespectively represent the weight matrices of the input-hidden layer, hidden-hidden layer, and hidden-output layer; b h and b y respectively represent the bias vectors of the hidden layer and the output layer; H represents the activation function of the hidden layer.

[0060] In a preferred embodiment, the foam image is converted to the CIELab color space, digital features on the a channel that can characterize the degree of redness are extracted, and two statistics of this channel, namely the mean μ and the standard deviation σ, are calculated according to the following formula. A total of 2D statistics are obtained for each frame of the image: the a channel mean and the a channel variance, which constitute the color feature vector of the corresponding image;

[0061]

[0062]

[0063] In the formula: p ij represents the pixel value of the image at (i, j); M and N represent the width and height of the image.

[0064] Compared with the prior art, the present invention has the following beneficial effects: The present invention proposes a method for early warning of flotation process faults based on BiLSTM prediction deviation. A time series prediction model based on ResNet50 and BiLSTM is constructed. The spatial features extracted by ResNet50 are input into the BiLSTM network for time series prediction of color. The deviation is calculated based on the predicted value and the actual value, and the early warning threshold is determined. When the predicted deviation exceeds the limit, early warning of production faults is carried out. The prediction deviation of the time series color features of the method of the present invention is small and the fitting degree is good. The early warning time of faults is effectively advanced, and early warning of flotation process faults can be realized, winning more time for operators to make adjustments and effectively reducing the waste of mineral resources and reagents. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 is the BiLSTM network structure diagram of the preferred embodiment of the present invention;

[0066] Figure 2 is the prediction model framework diagram of the preferred embodiment of the present invention;

[0067] Figure 3 is the flotation process fault detection flow chart of the preferred embodiment of the present invention;

[0068] Figure 4 is the early fault warning effect diagram of the preferred embodiment of the present invention, the prediction result of the ResNet50-BiLSTM model on the test set;

[0069] Figure 5Early fault warning effect diagram of the preferred embodiment of the present invention. (a) is the warning effect of the CNN-BiLSTM model, (b) is the warning effect of the ResNet50-LSTM model, and (c) is the warning effect of the ResNet50-BiLSTM model of the present invention. Detailed implementation manners

[0070] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0071] It should be noted that the following detailed description is exemplary and is intended to provide further illustration of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present application belongs.

[0072] It should be noted that the terms used herein are only for describing specific implementation manners and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0073] The present invention relates to a method for early fault warning of a flotation process based on the prediction deviation of BiLSTM. First, foam video images are collected and the color features of the video sequence are extracted to construct a sample data set. Then, a time series prediction model based on ResNet50 and BiLSTM is constructed, and the spatial features extracted by ResNet50 are input into the BiLSTM network for time series prediction of color. Secondly, the normal working data set is used to predict the time series color features, and the deviation is calculated based on the predicted value and the actual value to determine the warning threshold. Finally, foam video images are collected in real time for color feature prediction, and early warning of production faults is carried out when the predicted deviation exceeds the limit. The prediction deviation of the time series color features of the method of the present invention is small, the fitting degree is good, the fault warning time is effectively advanced, the early warning of flotation process faults can be realized, and the waste of mineral resources and reagents can be effectively reduced.

[0074] The detailed technical solution is as follows:

[0075] 1. BiLSTM network model

[0076] The BiLSTM network overcomes the limitations of the RNN network and the LSTM network. Figure 1The structure of the BiLSTM network is shown in the figure. It can be seen that the BiLSTM constructs two LSTM networks, a forward one and a backward one, to extract feature information, and learns the temporal features of the data more fully. The main idea is to connect the same input sequence to the forward and backward LSTM networks respectively, and then connect the hidden layers of the two networks together and jointly connect them to the output layer for prediction. The BiLSTM does not change the internal structure of the LSTM network. Although it only models the data with the LSTM in the forward and backward directions and then combines the information, this network not only retains the advantages of the LSTM but also solves the problem that the importance of the front and back data information changes due to the model structure of the LSTM, resulting in a decrease in accuracy.

[0077] As can be seen from Figure 1 below, the operation process of the BiLSTM is as follows:

[0078] The update formula for forward propagation is:

[0079]

[0080] The update formula for backward propagation is:

[0081]

[0082] The formula for the output after the forward and backward network layers are stacked is:

[0083]

[0084] In the formula: t represents the time series; represents the hidden layer vector at time t, and the arrow represents the direction; x t and y t represent the input and output at time t respectively; W xh , W hh and W hy represent the weight matrices of the input-hidden layer, hidden layer-hidden layer, and hidden layer-output layer respectively; b h and b y represent the bias vectors of the hidden layer and the output layer respectively; H represents the hidden layer activation function.

[0085] 2. Extraction of flotation foam color features

[0086] Before constructing the prediction model, the model predictions are first selected. Since the flotation production process is affected by many physical and chemical factors, researchers have found that the color of the flotation foam surface is closely related to the flotation production process indicators. The color of the flotation foam surface is a direct indicator of the ore dressing production index. Experienced flotation plant operators judge the production status by observing the color of the bubble surface and make timely adjustments to the production process. Therefore, the fault warning model constructed in the present invention warns of faults based on changes in the color characteristics of the flotation foam surface.

[0087] The degree of red in the foam image can more intuitively reflect the flotation production conditions. Since the CIELab color space can better separate the color information of the image and is more in line with the visual perception characteristics of humans, the foam image is converted to the CIELab color space, and the digital features on the a-channel that can characterize the degree of red are extracted. According to equations (4) and (5), two statistics of this channel are calculated, namely the mean μ and the standard deviation σ. Therefore, a total of 2D statistics are obtained for each frame of the image: the a-channel mean and the a-channel variance, which constitute the color feature vector of the corresponding image.

[0088]

[0089]

[0090] In the formula: pij represents the pixel value of the image at (i,j); M and N represent the width and height of the image.

[0091] 3. Early warning of faults in the flotation process based on BiLSTM and prediction deviation

[0092] 3.1 Data preprocessing

[0093] During the flotation process, due to the poor environment at the mineral flotation site, weak light on the surface of the flotation cell, and interference from dust and fog, the foam images captured by the on-site camera system show phenomena such as shadows and blurs, and sometimes there are problems such as personnel recording deviations and unstable production equipment, resulting in abnormal samples that may be collected at the flotation site and cannot be directly used as inputs for the prediction model. Therefore, the collected flotation foam image dataset needs to be preprocessed.

[0094] (1) Data cleaning

[0095] To avoid interference from irrelevant data during the flotation process, in this section, the data is cleaned according to the working conditions at the flotation site by the Grubbs criterion method. The specific steps of this criterion method are as follows:

[0096] Step 1: Assume x i(i = 1, 2, 3, ..., N) is the observation data sample of the flotation process. An observation data model with μ as the observation object is established as shown in the following formula:

[0097] x i = μ + p i , p i N(0, σ 2 ) (6)

[0098] Step 2: Calculate the sample mean and variance of x i respectively according to Equation (7) and Equation (8).

[0099]

[0100]

[0101] Step 3: Construct a statistic G according to Equation (9) so that the statistic G follows the transformation shown in Equation (10).

[0102]

[0103]

[0104] Step 4: Calculate the value of the statistic G α according to Equation (11). When G > G α , determine that the sample is abnormal and directly eliminate it.

[0105]

[0106] Step 5: Loop through Steps 1 to 4 until the sample data set is cleaned up.

[0107] (2) Data normalization

[0108] Before training the model, normalize the data through Equation (12) to compress the data into the range interval of [0, 1] and eliminate the influence of different dimensions. Normalization can not only speed up the calculation speed and the convergence speed of the model, but also improve the model accuracy to a certain extent.

[0109] x std = (x - x min ) / (x max - x min ) (12)

[0110] 3.2 Construction of the flotation process fault warning model

[0111] (1) ResNet50 extracts spatial features

[0112] Through training the model, deep learning can accurately extract hidden features from input data. The ResNet50 network shows good performance on flotation foam images. Therefore, the ResNet50 network is used to extract the features of the input flotation foam video frames. The video frame data after data preprocessing is input into the ResNet50 network to extract the spatial features of the data at each time point, and then it is passed to the BiLSTM network for time series prediction.

[0113] (2) BiLSTM extracts time series features

[0114] After the ResNet50 network extracts the features of the input data, after a period of accumulation, a sequence of feature values will be formed. The waveform formed by its fluctuation over time can reflect the changes in the state of the flotation process. However, the ResNet50 network can only recognize the amplitude of local changes and cannot detect the order information of the input data. Therefore, the data features learned from the ResNet50 network can be input into the BiLSTM network, which performs time series encoding on the data features. After obtaining the feature vectors in time series, they are then sent to the fully connected layer to complete the time series prediction.

[0115] (3) Prediction model

[0116] In order to achieve early fault warning and accurate prediction of future time series, the present invention constructs a fault prediction model for the flotation process based on ResNet50 and BiLSTM. The structure of the fault prediction model is as Figure 2As shown in the figure, the model consists of two parts: a training model and a testing model. Both parts mainly include three components: data input, a deep network, and prediction output. In the data input part, first, the visible light video frames of flotation foam are preprocessed and used as the input of the prediction model. Among them, the training set and the validation set are used as the input of the training model, and the normal test set is used as the input of the testing model. The deep network includes a ResNet50 and a BiLSTM network. The ResNet50 network is used to extract the spatial features of the input data at each time point, and then it is passed to the BiLSTM network to learn the correlation between data, perform temporal encoding on the data features, and extract time series features from both the forward and reverse directions. In the prediction output part, the extracted temporal features are sent to a fully connected layer, and the prediction value is output through the fully connected layer. Among them, in the training model, the foam video images are converted to the CIElab space to extract two color measurement points, namely the mean and variance of the a channel. The time series data of the two measurement point parameters are used as the prediction output values of the model to train the fault prediction model, thereby obtaining the fault prediction model. In the testing model, the prediction value of the color measurement point parameters at the next moment is output by the fully connected layer to complete the prediction of the time series. Then, the normal deviation degree and the warning threshold are calculated by combining the color measurement point parameter values extracted from the corresponding foam video frames, and further the fault warning model is obtained.

[0117] 3.3 Definition of Deviation Degree and Warning Strategy

[0118] The constructed ResNet50-BiLSTM flotation process fault prediction model can learn the relationships between various variables in the normal flotation process state after being trained with the normal flotation process data set. When the model receives new time series data, it can predict the data at the next moment according to the learned results. When there is a trend of fault in the flotation process, the relevant monitoring variables will show a certain deviation from the normal state data. When this deviation exceeds the set safety threshold, it is determined that an early fault has occurred in the flotation process.

[0119] (1) Definition of Deviation Degree

[0120] The error between the prediction value and the actual value of the deep learning network is called the residual, and the residual can reflect the change of the target variable. The residual r between the prediction value and the actual value output by the ResNet50-BiLSTM network model in this section is calculated by Equation (13) ij .

[0121]

[0122] In the formula: rij is the residual of variable i at time j; yij and They are respectively the actual value of variable i at time j and the predicted value output by the model in this section. According to the above selection of model predictions, there are 4 measured point parameters in this chapter. Therefore, the residual data at each moment forms a 4D vector, and the deviation degree of the flotation process from the normal state at this moment is calculated according to Equation (14).

[0123] Calculate the deviation degree at each moment, thus forming a deviation degree sequence.

[0124]

[0125] (2) Early warning strategy

[0126] There are multiple extreme points and non-stationarity in the deviation degree sequence. In order to further improve the expression ability of the model, it is necessary to set an early warning threshold for the model. The present invention uses the generalized extreme value theory to calculate the early warning threshold, and the specific steps for its solution are as follows:

[0127] Decompose the deviation degree sequence into multiple smallest intervals with the same number of data points, and select 5 points as the smallest interval. Then the calculation method of the maximum value M in each interval is shown in the following formula:

[0128] M = max{x 1 ,..., x n} (15)

[0129] Where: x i is the value within a smallest interval; {x 1 ,..., x n} is a random sequence with the same distribution and independent.

[0130] Let the distribution function of the random sequence in 1) be F, then the relationship between the distribution of M and the distribution function F of {x 1 ,..., x n} is:

[0131] P r {M ≤ z} = P r {x 1 ≤ z,..., x n ≤ z} = P r {x 1 ≤ z} ×... × P r {x n ≤ z} = {F(z)} n (16)

[0132] Since the distribution function F is unknown, it is assumed that there exist μ and σ satisfying:

[0133] P r {(M - μ) / σ} → G(z) (17)

[0134] Where: μ is the location parameter; σ is the scale parameter; G() is the generalized extreme value distribution function, which is calculated according to the following formula:

[0135] G(z) = exp{-[1 + ((z - μ) / σ)] -1 / ξ} (18)

[0136] Where: The generalized extreme value distribution function is defined on the set {z: 1 + ξ(z - μ) / σ > 0}, where the parameters μ and ξ satisfy: -∞ < μ < ∞, σ > 0, -∞ < ξ < ∞.

[0137] 4) Obtain the location parameter μ, scale parameter σ, and shape parameter ξ of the generalized extreme value distribution through the maximum likelihood estimation method, and calculate the warning threshold through Equation (19).

[0138] T h = μ - σ[1 - {-ln(1 - α)} -ξ / ξ (19)

[0139] In summary, during the flotation process, if the deviation sequence remains within the warning threshold, it is determined that the flotation process is normal; if the deviation exceeds the warning threshold, it is determined that there is a tendency for the flotation process to malfunction, so as to achieve fault warning.

[0140] 3.4 Specific implementation process and steps

[0141] To achieve early identification of faults during the flotation process, first preprocess the input data set and then input it into the ResNet50 network to extract the spatial features of the data at each time point; then transfer the extracted spatial features to the BiLSTM network to extract the time series features, and transfer them to the fully connected layer, and output the predicted value for the next moment through the fully connected layer; finally, calculate the deviation degree through the obtained predicted value and the actual value, determine the alarm threshold, and achieve early fault warning of the flotation process. The process of the fault warning method for the flotation production process is as Figure 3 shown, and the specific implementation steps are as follows:

[0142] Step 1, obtain the historical flotation process data set, including the normal state data set and the fault state data set; preprocess the data set such as data cleaning, and select the model prediction quantity;

[0143] Divide the normal state data set among them into a training set, a validation set, and a test set according to the ratio of 6:2:2. Use the training set data as the input of the model, extract the two measured point parameters related to faults, namely the mean value of the a-channel of the image color and the standard deviation of the a-channel of the training set video frame. Use the time series data of the two measured point parameters as the predicted output value of the model, train the fault prediction model, and test the training effect of the model through the validation set;

[0144] Step 3: First, save the fault prediction model trained in Step 2. Use the normal test set as the input data of this model to verify the generalization ability and prediction accuracy of the model. Then, use the root mean square error (RMSE), mean absolute error (MAE), and coefficient of determination (R2) as the indicators to evaluate the prediction effect. Finally, calculate the deviation degree and warning threshold under the normal flotation process state by combining the color measurement point parameter values extracted from the corresponding foam video frames to obtain the fault warning model.

[0145] Step 4: Test the fault warning model with the fault test set. Input the fault test set into the fault warning model to obtain the predicted values and calculate the deviation degree according to the predicted values. Conduct early warning based on whether the deviation degree exceeds the warning threshold to verify the effectiveness of the warning model.

[0146] Step 5: Real-time collect foam video images and input them into the fault warning model to obtain the predicted values and calculate the deviation degree according to the predicted values. If the deviation degree exceeds the warning threshold, it is considered that there is a tendency of fault occurrence in the flotation process, and a warning signal is given.

[0147] 4 Specific embodiments and descriptions

[0148] To verify the effectiveness of the method of the present invention, the foam images collected in the lead-zinc ore flotation plant of Fujian Jindong Mining Co., Ltd. are used as experimental samples. The hardware platform of the experiment is Intel(R) Core(TM) i7-9800X CPU@3.80GHz, NVIDIA GeForce RTX 3080Ti, 128GB RAM, and the software running environment is Windows 10, Matlab2019a, Python3.7, Pytorch1.7. The method proposed in the present invention is verified through experiments.

[0149] Select the operation data of 12 hours under the normal flotation process state of a certain day in the flotation plant as the experimental data. Among them, the data sampling interval is 3s. After building the ResNet50-BiLSTM model, train the model through the training set and the validation set, and test the model with the test set. Then, the prediction results of the ResNet50-BiLSTM fault prediction model proposed in the present invention on the test set are as Figure 4 shown.

[0150] To further verify the prediction performance of the proposed ResNet50-BiLSTM model, compare the prediction effect of this model with that of the CNN-BiLSTM and ResNet50-LSTM models. Select RMSE, MAE, and R 2As an evaluation index, the average value of each parameter is taken as the evaluation result. The calculation of each evaluation index is shown in formulas (20)-(22). Among them, RMSE reflects the accuracy of the prediction result. The lower the RMSE, the higher the prediction accuracy; MAE reflects the consistency of the prediction result. The lower the MAE, the smaller the error of the prediction result; R 2 reflects the goodness of fit of the prediction curve. The closer R 2 is to 1, the better the fitting effect. The performance statistics of each model are shown in Table 1. It can be seen from this that compared with the other two models, the prediction result of the CNN-BiLSTM model has a larger deviation from the actual result, the prediction accuracy is lower, and the fitting degree is poorer; while the prediction result of the ResNet50-LSTM model has been improved in each index compared with the prediction result of the CNN-BiLSTM model, but there is still room for improvement; compared with the ResNet50-LSTM model, the root mean square error of the ResNet50-BiLSTM model proposed by the present invention is reduced by 20.6%, the mean absolute error is reduced by 23.5%, and the fitting coefficient is increased by 17.1%. Since the ResNet50-BiLSTM model combines the advantages of the two models and better synthesizes the characteristics of multivariate time series data, the prediction accuracy is higher, the deviation is small, and the fitting degree is good.

[0151]

[0152]

[0153]

[0154] In the formula: y ij , and are respectively the actual value, predicted value and average value of variable i at the j-th moment; j = 1, 2, …, N; N is the length of the time series of the data set.

[0155] Table 1 Performance statistics of prediction models

[0156]

[0157]

[0158] Next, perform the early fault warning test: Since the variation laws of the two measured point variables during the flotation process are relatively stable and cannot effectively reflect whether a fault occurs in the flotation process, first, use the ResNet50-BiLSTM model to predict the sequences of the two measured point variables in the future for a period of time, then calculate the prediction deviation degree, and determine the warning threshold. The fault data collected from a lead-zinc ore flotation plant are used as experimental data, and the CNN-BiLSTM, ResNet50-LSTM, and ResNet50-BiLSTM models are respectively used for the fault warning test. The warning effects are as Figure 5 shown. It is known that a fault occurs at the 2.8-hour time point during the flotation process. As can be seen from the figure, the CNN-BiLSTM warning model fluctuates greatly, and the deviation degree does not show an obvious upward trend before the fault occurs. Moreover, the deviation degree is still lower than the threshold a few days before the fault. The warning threshold of this model is 2.8687, and the deviation degree is higher than the threshold at the 154-minute time point, giving an alarm 14 minutes in advance; while the deviation degrees of the other two warning models show obvious upward trends before the fault occurs. The warning threshold of the ResNet50-LSTM model is 2.3217, and the deviation degree is higher than the threshold at the 140-minute time point, giving an alarm 28 minutes in advance. The warning threshold of the ResNet50-BiLSTM model of the present invention is 1.9471, and the deviation degree is higher than the threshold at the 106-minute time point, giving an alarm 62 minutes in advance. After the alarm, the deviation degree is above the threshold, and with the deepening of the fault, the upward trend of the deviation degree fluctuation is obvious.

Claims

1. An early warning method for faults in the flotation process based on BiLSTM prediction deviation, characterized in that, it includes the following steps: Step 1, obtain the historical flotation process data set, including the normal state data set and the fault state data set; perform data cleaning and preprocessing on the data set, and select the model prediction quantity; Divide the normal state data set among them into a training set, a validation set and a test set according to the ratio of 6:2:

2. Use the training set data as the input of the model, extract the mean value of the a-channel and the standard deviation of the a-channel of the image color of the training set video frames, these 2 measuring point parameters related to faults, and use the time series data of the 2 measuring point parameters as the predicted output value of the model. Train the fault prediction model, and test the training effect of the model through the validation set; Step 3: First, save the fault prediction model trained in Step 2, use the normal test set as the input data of this model to verify the generalization ability and prediction accuracy of the model. Then, use the root mean square error RMSE, the mean absolute error MAE and the fitting coefficient R2 as indicators to evaluate the prediction effect. Finally, combine the deviation and the warning threshold calculated under the normal flotation process state with the color measuring point parameter values extracted from the corresponding foam video frames to obtain the fault warning model; Step 4: Test the fault warning model through the fault test set. Input the fault test set into the fault warning model, obtain the predicted value and calculate the deviation according to the predicted value. Conduct early warning according to whether the deviation exceeds the warning threshold to verify the effectiveness of the warning model; Step 5: Real-time collect foam video images and input them into the fault warning model, obtain the predicted value and calculate the deviation according to the predicted value. If the deviation exceeds the warning threshold, it is considered that there is a tendency for the flotation process to fail, and a warning signal is given; The construction of the flotation process fault warning model is specifically as follows: Use the ResNet50 network to extract the features of the input flotation foam video frames. Input the video frame data after data preprocessing into the ResNet50 network to extract the spatial features of the data at each time point, and then transfer it to the BiLSTM network for time series prediction; After the ResNet50 network extracts the features of the input data, after a period of accumulation, a sequence of feature values is formed. The waveform formed by its fluctuation over time reflects the change of the flotation process state; input the data features learned from the ResNet50 network into the BiLSTM network, perform time series encoding on the data features by it, obtain the feature vectors in time series, and then send them into the fully connected layer to complete the prediction of the time series; Build a fault prediction model for the flotation process based on ResNet50 and BiLSTM; the fault prediction model consists of two parts, the training model and the testing model, and both parts mainly include three parts: data input, deep network, and prediction output; in the data input part, first, the visible light video frames of flotation foam are preprocessed and used as the input of the prediction model. Among them, the training set and the validation set are used as the input of the training model, and the normal test set is used as the input of the testing model; the deep network includes ResNet50 and BiLSTM networks. The ResNet50 network is used to extract the spatial features of the input data at each time point, and then it is passed to the BiLSTM network to learn the correlation between data, perform temporal encoding on the data features, and extract time series features from both positive and negative directions; in the prediction output part, the extracted temporal features are sent to the fully connected layer, and the prediction value is output through the fully connected layer. Among them, in the training model, the foam video image is converted to the CIElab space to extract two color measurement points, the mean and variance of the a channel, and the time series data of the two measurement point parameters are used as the prediction output value of the model to train the fault prediction model, so as to obtain the fault prediction model; while in the testing model, the prediction value of the color measurement point parameters at the next moment is output by the fully connected layer to complete the prediction of the time series, and then the normal deviation degree and the warning threshold are calculated by combining the color measurement point parameter values extracted from the corresponding foam video frames to further obtain the fault warning model; The definition of the deviation degree and the warning strategy are specifically as follows: The constructed ResNet50-BiLSTM fault prediction model for the flotation process is trained with the normal flotation process dataset. When the model receives new time series data, it predicts the data at the next moment according to the learning results; when there is a trend of fault in the flotation process, there will be a certain deviation in the relevant monitoring variables compared with the normal state data. When this deviation exceeds the set safety threshold, it is determined that an early fault occurs in the flotation process; Calculate the residual r between the predicted value and the actual value output by the ResNet50-BiLSTM network model in this section through the following formula ij ; Where: r ij is the residual of variable i at time j; y ij and are respectively the actual value of variable i and the predicted value output by the model in this section at time j; according to the selection of the model prediction quantity, there are 4 measuring point parameters, so the residual data at each moment forms a 4-dimensional vector, and the deviation degree of the flotation process deviating from the normal state at this moment is calculated according to the following formula; Calculate the deviation degree at each moment to form a deviation degree sequence; There are multiple extreme points and non-stationarity in the deviation degree sequence. The generalized extreme value theory is used to calculate the warning threshold, and the specific steps of its solution are as follows: Decompose the deviation degree sequence into multiple smallest intervals with the same number of data points, and select 5 points as the smallest interval; then the calculation method of the maximum value M in each interval is shown in the following formula: M = max{x 1 ,..., x n} where: x i is the value within a minimum interval; {x 1 ,..., x n} is a sequence of identically distributed and independent random variables; Let the distribution function of the random sequence be F. Then the relationship between the distribution of M and the distribution function F of {x 1 ,..., x n} is as follows: P r {M ≤ z} = P r {x 1 ≤ z, …, x n ≤ z} = P r {x 1 ≤ z} × … × P r {x n ≤ z} = {F(z)} n Since the distribution function F is unknown, it is assumed that there exist μ and σ that satisfy: P r {(M - μ) / σ} → G(z) In the formula: μ is the location parameter; σ is the scale parameter; G() is the generalized extreme value distribution function, which is calculated according to the following formula: G(z) = exp{-[1 + ((z - μ) / σ)] -1 / ξ} In the formula: the generalized extreme value distribution function is defined on the set {z: 1 + ξ(z - μ) / σ > 0}, where the parameters μ and ξ satisfy: -∞ < μ < ∞, σ > 0, -∞ < ξ < ∞; Obtain the location parameter μ, scale parameter σ, and shape parameter ξ of the generalized extreme value distribution by the maximum likelihood estimation method, and calculate the warning threshold through the following formula; T h = μ - σ[1 - {-ln(1 - α)} -ξ / ξ In summary, during the flotation process, if the deviation degree sequence remains within the warning threshold, it is determined that the flotation process is normal; if the deviation degree exceeds the warning threshold, it is determined that there is a tendency for the flotation process to malfunction, thereby achieving fault warning; BiLSTM constructs two LSTM networks, a forward one and a backward one, to extract feature information; the same input sequence is respectively connected to the forward and backward LSTM networks, and then the internal structures of the two LSTM networks are changed; The operation process of BiLSTM is as follows: The update formula for forward propagation is as follows: The update formula for backward propagation is as follows: The formula for the output after stacking the forward and backward network layers is as follows: Where: t represents the time series; represents the hidden layer vector at time t, and the arrow represents the direction; x t and y t respectively represent the input and output at time t; W xh , W hh and W hy respectively represent the weight matrices of the input-hidden layer, hidden layer-hidden layer, and hidden layer-output layer; b h and b y respectively represent the bias vectors of the hidden layer and the output layer; H represents the hidden layer activation function; The foam image is converted to the CIELab color space, and the digital features on the a channel that can characterize the degree of redness are extracted. According to the following formula, two statistical quantities of this channel are calculated, namely the mean μ and the standard deviation σ. A total of 2D statistical quantities are obtained for each frame of the image: the a channel mean and the a channel variance, which constitute the color feature vector of the corresponding image; In the formula: pij represents the pixel value of the image at (i,j); M and N represent the width and height of the image.

2. The early fault warning method for the flotation process based on BiLSTM prediction of the deviation degree according to claim 1, characterized in that, The data preprocessing is specifically to preprocess the collected flotation foam image data set; specifically including: (1) Data cleaning, the data is cleaned by the Grubbs criterion method, and the specific steps of this criterion method are as follows: Step S1: Assume x i (i = 1, 2, 3, ..., N) is the observation data sample of the flotation process. The observation data model with μ as the observation object is established as shown in the following formula: x i = μ + p i , p i N(0, σ 2 ) Step S2: Calculate the sample mean and variance of xi respectively according to the following formula; Step S3: Calculate and construct the statistic G such that the statistic G follows a transformation; Step S4: Calculate the statistic G according to the following formula α When the value of G > G α , it is determined that the sample is abnormal and is directly excluded; Step S5: Loop and execute steps S1 to S4 until the sample data set is cleaned; (2) Data normalization, before training the model, the data is normalized by the following formula to compress the data into the range interval of [0,1]; x std = (x - x min ) / (x max - x min )。

Images