Small sample sintering condition identification method based on multimodal data time-varying coupling characteristics modeling

Abstract

A small-sample sintering condition recognition method based on multimodal data modeling of variable time-delay and variable coupling characteristics includes collecting on-site rotary kiln condition data and labeling the known condition data with condition category labels; parallel splicing multiple thermal variables with multiple flame video feature sequences to extract variable time-delay and variable coupling features from known-label condition samples; inputting the variable coupling features into a classifier to perform parameter update learning for the recognition model; extracting variable time-delay and inter-variant coupling features from unlabeled condition samples; calculating the similarity of the variable time-delay and variable coupling features between known-label abnormal condition samples and unlabeled condition samples; and, based on the feature similarity, assigning corresponding abnormal condition pseudo-labels to the unlabeled condition samples or discarding the unlabeled condition samples; and adding the condition samples labeled with abnormal condition pseudo-labels to known-label samples to form a training sample set, which is then used to retrain the recognition model. The present invention achieves high condition recognition accuracy.

Description

Technical Field

[0001] The present invention relates to a sintering condition identification method, and in particular to a small sample sintering condition identification method based on multimodal data variable time lag and variable coupling characteristic modeling. Background Art

[0002] The government has set higher standards for pollution emissions from enterprises, particularly those related to carbon, nitrogen, and oxides (CO). The rotary kiln is a complex, dynamic system characterized by high energy consumption and high CO emissions. It is widely used in metallurgy, industry, cement, and other fields. The temperature of the sintering zone within the kiln is one of the most critical parameters in the production process. Depending on the sintering temperature, rotary kilns can be classified into three sintering conditions: normal, overfired, and underfired. Normal operating conditions indicate that the sintering zone temperature is within the set range, resulting in sufficient sintering of the material, high product quality, and low CO emissions. This is the ideal sintering condition. Overfired and underfired conditions indicate that the sintering temperature is above or below the required sintering temperature, respectively. Both are abnormal conditions. Overfire leads to poor material flowability, which can easily cause ringing and nodules, damaging the kiln body. Underfire results in incomplete combustion of the pulverized coal and insufficient material reaction, leading to yellowing and high CO emissions. Accurately identifying the sintering condition of a rotary kiln is crucial for improving sintering quality, reducing CO emissions, and increasing energy efficiency.

[0003] Currently, the identification of rotary kiln sintering conditions is mainly based on modeling based on flame videos captured by CCD cameras and multivariate thermal data collected by various sensors. In fact, the flame videos captured on site are usually severely interfered by the smoke and dust in the kiln, and the multivariate thermal data have complex characteristics such as variable time lag and variable coupling. In addition, the corresponding data collected during normal production have large differences in the volume of data between normal and abnormal conditions, that is, sample imbalance. In this case, traditional machine learning or deep neural network modeling methods are difficult to solve the above problems at the same time. Therefore, how to design a more efficient and robust new neural network modeling method based on the various data characteristics of the rotary kiln and the problem of sample imbalance is still a difficult problem that needs to be overcome for the automatic identification of rotary kiln conditions.

[0004] In recent years, considering the problems of large data quality fluctuations and poor model robustness in single-modal flame video or multivariate thermal data modeling, researchers have drawn on the concept of multimodal learning and proposed a variety of sintering condition identification methods based on multimodal data fusion. For example, patent CN 117150345 A discloses a method and system for identifying abnormal operating conditions of a rotary kiln based on multimodal data fusion. This method uses Bi-LSTM and convolutional neural networks (CNN) to extract operating condition features from each single-modal data and multimodal fusion data, respectively, and realizes the identification of imbalanced operating condition samples by improving the classifier model kernel parameters. As the first framework to propose multimodal data fusion for rotary kiln operating condition identification, related work has achieved good results. However, in reality, Bi-LSTM and CNN have difficulty in modeling complex nonlinear relationships such as variable time lag and variable coupling between multivariate data, and the proposed multimodal fusion framework is a three-branch network with a large model structure and too many parameters. In addition, the solution proposed in the patent to improve the classifier for the long-tail distribution problem of data is helpful for improving the accuracy of identifying abnormal working conditions of minority classes, but the impact of the imbalance of working condition samples also includes the overfitting problem of the feature learning network for majority class samples. Simply improving the classifier cannot improve the data bias of the feature learning model, and it is difficult to learn the clear classification boundaries of different working condition features, which affects the accuracy of working condition identification. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to overcome the shortcomings of the above-mentioned background technology and provide a small sample sintering condition identification method based on multimodal data variable time lag and variable coupling characteristic modeling, extract the variable time lag and variable coupling characteristics between data, expand the small sample abnormal condition data, and improve the accuracy of condition identification.

[0006] The technical solution adopted by the present invention to solve the technical problem is a small sample sintering condition identification method based on multimodal data variable time lag and variable coupling characteristic modeling, comprising the following steps:

[0007] Step S1: Collecting on-site operating data of the rotary kiln, including flame video and multivariate thermal data, constructing multimodal data operating condition samples, and labeling the data of known operating conditions with operating condition category labels;

[0008] Step S2: Extract flame image i working condition features from the flame video data in the working condition sample to form multiple flame video feature sequences right Down sampling is performed to obtain the thermal variable data Z in the working condition sample y Flame video feature sequence V with consistent dimension i , and then multiple Z y With multiple V iBy parallel splicing, we can obtain the multimodal fusion data D of the working condition samples.

[0009] Step S3: Use the known label working condition samples to train the sintering working condition recognition model: The fusion data D of the known label working condition samples l Input variable time lag feature extraction module to extract variable time lag features between each thermal variable data and flame video feature sequence in the corresponding working condition sample Time-varying hysteresis characteristics After transposition, input the variable coupling feature extraction module to extract the variable coupling features between the variables of the corresponding working condition samples Variable Coupling Features Input the classifier to perform parameter update learning of the sintering condition identification model;

[0010] Step S4: After the sintering condition recognition model is trained using known label condition samples, the fusion data D of the unlabeled condition samples is nl , input the variable time lag feature extraction module to extract the variable time lag features between the thermal variable data and the flame video feature sequence in the unlabeled working condition sample Time-varying hysteresis characteristics Input variable coupling feature extraction module to extract variable coupling features between variables of unlabeled working condition samples

[0011] Step S5: Calculate the similarity of the variable time lag features and variable coupling features of the abnormal working condition samples with known labels and the unlabeled working condition samples. According to the feature similarity, calibrate the corresponding abnormal working condition pseudo-label for the unlabeled working condition sample, or discard the unlabeled working condition sample.

[0012] Step S6: adding the working condition samples marked as abnormal working condition pseudo labels to the known label samples to form a training sample set with a more balanced sample distribution, and using the training sample set to retrain the sintering working condition recognition model.

[0013] Furthermore, in step S1, the multivariate thermal data includes the following thermal variables: coal feed rate Z1, slurry flow rate Z2, primary air volume Z3, kiln tail negative pressure Z4, kiln head temperature Z5, kiln tail temperature Z6, main engine current Z7, sintering belt temperature Z8; the operating condition category labels include normal, underburning and overburning, among which underburning and overburning are abnormal operating conditions.

[0014] Furthermore, in step S2, the multimodal fusion data D is expressed as: D = [V1...V i ; Z1...Z y ]; i represents the number of types of flame image working condition features, and y represents the number of thermal variables.

[0015] Furthermore, in step S2, the seed region growing method is used to extract flame image features. The flame image features include flame brightness V1, flame area size V2, and flame flickering degree V3. Specifically,

[0016] Step S2-1: For each point f(a, b) in each flame image frame of the flame video in the working condition sample, the flame region is divided using the seed region growing method. The initial seed point is set as the brightest point in the image. The growth region severity is set as the growth region division condition. The growth region severity T is continuously updated based on the grayscale mean m and standard deviation σ in the current growth region. The calculation process is as follows:

[0017]

[0018] Where, T σ The preset value for the harshness of the growing area;

[0019] Step S2-2: Calculate the grayscale mean m and standard deviation σ of the n known grayscale points (a, b) in the current growth area Q:

[0020]

[0021] Where a is the horizontal coordinate of the grayscale point (a, b), and b is the vertical coordinate of the grayscale point (a, b);

[0022] Step S2-3: Compare the difference between the grayscale value f(x, y) of the grayscale point to be determined (x, y) and the grayscale mean value in the grown area Q with the growth area severity T:

[0023] |f(x,y)-m|≤T (4)

[0024] If the growth conditions are met, then continue to grow, otherwise stop growing;

[0025] Step S2-4: taking the grayscale mean value in the grown area Q as the flame brightness feature V1 of the flame image: V1=m;

[0026] Step S2-5: Use the size of the grown region as the flame region feature V2 of the flame image: V2=n

[0027] Step S2-6: Sum the flame area Q at the current time t t and the flame area Q at time t+1 t+1 The grayscale difference at the same position is used as the flame flickering degree feature V3:

[0028]

[0029] Where g t+1 (a, b) is the flame area Q at time t+1 t+1 Gray value, gt (a, b) is the flame area Q at time t t+1 Grayscale value.

[0030] Further, in step S3, the sintering working condition recognition model includes a variable time-lag feature and variable coupling feature extraction module, a pseudo-label learning module and a classifier. The variable time-lag feature and variable coupling feature extraction module includes a variable time-lag feature extraction module and a variable coupling feature extraction module. The pseudo-label learning module is connected to the variable time-lag feature extraction module and the variable coupling feature extraction module. The variable time-lag feature extraction module is connected to the variable coupling feature extraction module, and the variable coupling feature extraction module is connected to the classifier. The variable time-lag feature extraction module is used to extract the variable time-lag features between each thermal variable data and the flame video feature sequence in the working condition sample; the variable coupling feature extraction module is used to extract the variable coupling features between each variable of the working condition sample; the pseudo-label learning module is used to calculate the similarity of the variable time-lag features and variable coupling features of the abnormal working condition sample with a known label and the unlabeled working condition sample, and according to the feature similarity, the corresponding abnormal working condition pseudo-label is calibrated for the unlabeled working condition sample, or the unlabeled working condition sample is discarded.

[0031] Furthermore, the variable time-delay feature extraction module adopts an autocorrelation attention module, and the variable coupling feature extraction module adopts a multi-layer perceptron MLP network.

[0032] Furthermore, in step S3, each thermal variable data Z in the fusion feature is y With the flame video feature sequence V i Use the autocorrelation attention module to extract variable time lag features. The specific steps are as follows:

[0033] Step S3-1: Multiple variable data X within time period t j (1:t)=[x j (1),...,x j (t)], which is then linearly embedded as the query input Q of the autocorrelation attention module j , with multiple variable data X in a random time period τ j (1:τ)=[x j (1),...,x j (τ)] is linearly embedded as the key input K of the autocorrelation attention module j , with multiple variable data X within the time period τ~t j (τ:t)=[x j (τ),...,x j (t)] is linearly embedded and then input into V as the value of the autocorrelation attention module j :

[0034] Q j =W Q ·Xj (1:t) (6)

[0035] K j =W K ·X j (1:τ) (7)

[0036] V j =W v ·X j (τ:t) (8)

[0037] Where, X j Represents the thermal variables or flame video feature variables in the fusion feature, W Q 、W K 、W V are the query matrix, key matrix, and value matrix of linear embedding, respectively, τ<t;

[0038] Step S3-2: Input the key into K by interpolation j , value input V j The size and query input Q j After the size is unified, enter Q for the query j Perform fast Fourier transform and input K for key j Perform Fourier conjugate operation, and the product of the two results is used as the two components Q j , K j Common frequency domain Then substitute τ into the common frequency domain Perform Fourier inverse transform to obtain two components Q j , K j Similarity

[0039]

[0040] Among them, F(x), F * (x), F -1 (x) represents Fourier transform, Fourier conjugate operation, and inverse Fourier transform, respectively; f represents the frequency domain component; and i represents the imaginary unit; Represents two components Q j , K j The common frequency domain, Represents two components Q j , K j similarity;

[0041] Step S3-3: Similarity under multiple random times τ Sort and select the k with the highest similarity Corresponding τ1,...,τ k , for k Use the Sofamax function to assign attention weights and get k attention weights k They are k attention weights They are

[0042]

[0043] Where, τ1, ..., τ k represents k random times τ, L represents the upper limit of τ, and Topk(·) represents the highest k values ​​selected;

[0044] Step S3-4: X j (1:τ)=[x j (1),...,x j (τ)] moves to X j (τ:t)=[x j (τ),...,x j (t)] The value formed by inputting V j '=W V ·X j (τ:t) end, forming Roll(V j ',τ), so that the data length is the same as X j (1:t)=[x j (1),...,x j (t)] are the same; for the k Rolls with the highest similarity (V j ',τ), k Roll(V j ',τ) are Roll(V j ',τ1),...,Roll(V j ',τ k ), k Roll(V j ',τ) and k attention weights Perform weighted summation to obtain the output of the autocorrelation attention module, and j , the output of the autocorrelation attention module is expressed as:

[0045]

[0046] Roll(x,y) represents the time series data formed by concatenating x with y, i'∈1...k.

[0047] Furthermore, step S5 specifically includes the following steps:

[0048] Step S5-1: Calculate the similarity of the time-delay characteristics of the overburning condition samples with known labels and the unlabeled condition samples Set the similarity threshold of the time-delay features between the overburning condition samples and the unlabeled condition samples Determining the similarity of time-varying delay features Is it greater than the threshold? If it is greater than, go to step S5-2 to further calculate the similarity of the variable coupling characteristics of the overburning condition sample with known labels and the unlabeled condition sample. If not, go to step S5-3 to further calculate the similarity of the variable time lag characteristics of the underfire condition sample with known labels and the unlabeled condition sample.

[0049] Step S5-2: Calculate the similarity of the variable coupling characteristics between the overburning condition sample with known labels and the unlabeled condition sample Set the similarity threshold of the variable coupling features between the overburning condition samples and the unlabeled condition samples Determine the similarity of variable coupling characteristics Is it greater than the threshold? If it is greater than, the overburning condition pseudo label is calibrated for the unlabeled working condition sample; if it is not greater than, go to step S5-3 to further calculate the similarity of the variable time lag characteristics of the underburning working condition sample with known label and the unlabeled working condition sample

[0050] Step S5-3: Calculate the similarity of the time-delay characteristics between the underburned condition samples with known labels and the unlabeled condition samples Set the similarity threshold of the variable time lag features between underfire condition samples and unlabeled condition samples Determining the similarity of time-varying delay features Is it greater than the threshold? If it is greater than, go to step S5-4 and further calculate the similarity of the variable coupling characteristics of the underfire condition sample with known labels and the unlabeled condition sample. If not, discard the unlabeled working condition sample;

[0051] Step S5-4: Calculate the similarity of the variable coupling characteristics between the underburned working condition samples with known labels and the unlabeled working condition samples Set the variable coupling feature similarity threshold between underfire condition samples and unlabeled condition samples Determine the similarity of variable coupling characteristics Is it greater than the threshold? If it is greater than, the underburning condition pseudo label is calibrated for the unlabeled working condition sample; if it is not greater than, the unlabeled working condition sample is discarded.

[0052] Furthermore, the feature similarity is calculated as follows:

[0053]

[0054] in, F represents the similarity between the i'th class abnormal working condition sample with known label and the j'th class working condition sample without label, j' nl represents the j' class feature of the unlabeled sample, represents the j' class feature of the known label sample, Represents the j' class features of the i' class abnormal working condition samples with known labels, Represents the j'-class feature set of i'-class abnormal working condition samples with known labels, Z j' Represents the j'-type feature set of known label samples, where the j'-type feature is a variable time lag feature or a variable coupling feature, and the i''-type abnormal operating condition is an over-burning condition or an under-burning condition; τ' is a constant, and its value is Square root of the mean.

[0055] Furthermore, the feature similarity threshold is set as follows: the feature similarities of class j' of all class i' abnormal working condition samples with known labels are summed up and the average value is used as the feature similarity threshold.

[0056] Compared with the prior art, the advantages of the present invention are as follows:

[0057] (1) The rotary kiln is a large-scale high-temperature production equipment with a complex mechanism. Under the same sintering conditions, the correlation between the flame video and multivariate thermal variables collected on site and the data is not exactly the same. For example, two samples were collected under the same under-burning condition. The first sample was under-burned due to insufficient primary air volume and incomplete coal powder combustion; the other sample was under-burned due to excessive slurry flow and excessive kiln speed. The two samples with the same label of under-burning have different time lag and coupling relationships between their flame brightness and primary air volume. Under the first sample working condition, increasing the primary air volume will immediately increase the flame brightness; while under the second sample working condition, increasing the primary component flame brightness will not change much or will change slowly. In view of the above characteristics presented by the rotary kiln data, the present invention adopts the Auto-correlation attention module to mine the variable time lag characteristics of multimodal data, and adopts the multi-layer perceptron MLP network to mine the variable coupling characteristics of multimodal data, laying the foundation for the extraction of highly separable working condition features (high working condition feature discrimination).

[0058] (2) In order to ensure the production efficiency and product quality of the rotary kiln, the volume of various working condition data collected at the production site varies greatly, and the number of samples of over-burning and under-burning working conditions is relatively small. In order to solve the problem of small samples of abnormal working conditions caused by the imbalance of the above-mentioned working condition data, the present invention adopts a pseudo-label learning method based on data variable time lag and variable coupling feature similarity. On the basis of augmenting the small sample abnormal working condition data, it alleviates the bias of the feature learning model for unbalanced data, thereby improving the recognition accuracy of small sample abnormal working conditions. Judging similar working conditions based on the distribution of original data and the data distribution of features extracted by the network has higher working condition credibility than direct judgment using a classifier. It also provides more information for network input, improving the accuracy of the feature extraction network and the accuracy of the classifier. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 It is a method principle diagram of an embodiment of the present invention.

[0060] Figure 2 1 is a schematic diagram of input and output of the variable time-delay feature extraction module and the variable coupling feature extraction module of the embodiment shown in the figure.

[0061] Figure 3 yes Figure 1 The illustrated embodiment is a flow chart for calibrating pseudo labels for abnormal operating conditions. DETAILED DESCRIPTION

[0062] The present invention will be described in further detail below with reference to the accompanying drawings and specific embodiments.

[0063] Reference Figure 1 , the method of this embodiment includes the following steps:

[0064] Step S1: Collecting on-site operating data of the rotary kiln, including flame video and multivariate thermal data, constructing multimodal data operating condition samples, and labeling the data of known operating conditions with operating condition category labels;

[0065] In step S1, the multivariable thermal data includes the following thermal variables: coal feed rate Z1, slurry flow rate Z2, primary air volume Z3, kiln tail negative pressure Z4, kiln head temperature Z5, kiln tail temperature Z6, main engine current Z7, and sintering zone temperature Z8. The operating condition category labels include normal, underfired, and overfired, with underfired and overfired being abnormal conditions.

[0066] Step S2: Extract flame image i working condition features from the flame video data in the working condition sample to form multiple flame video feature sequences right Down sampling is performed to obtain the thermal variable data Z in the working condition sample y Flame video feature sequence V with consistent dimension i , and then multiple Z y With multiple Vi By parallel splicing, the multimodal fusion data D of the working condition sample can be obtained, which is expressed as D = [V1...V i ; Z1...Z y ]; i represents the number of types of flame image working condition features, y represents the number of thermal variables; the multimodal fusion data D of working condition samples includes the fusion data D of working condition samples with known labels l and the fusion data D of unlabeled working condition samples nl ;

[0067] In step S2, the seed region growing method is used to extract flame image features. The flame image features include flame brightness V1, flame area size V2, and flame flickering degree V3. Specifically,

[0068] Step S2-1: For each point f(a, b) in each flame image frame of the flame video in the working condition sample, the flame region is divided using the seed region growing method. The initial seed point is set as the brightest point in the image. The growth region severity is set as the growth region division condition. The growth region severity T is continuously updated based on the grayscale mean m and standard deviation σ in the current growth region. The calculation process is as follows:

[0069]

[0070] Where, T σ The default value for the harshness of the growing area.

[0071] Step S2-2: Calculate the grayscale mean m and standard deviation σ of the n known grayscale points (a, b) in the current growth area Q:

[0072]

[0073] Where a is the horizontal coordinate of the grayscale point (a, b), and b is the vertical coordinate of the grayscale point (a, b).

[0074] Step S2-3: Compare the difference between the grayscale value f(x, y) of the grayscale point to be determined (x, y) and the grayscale mean value in the grown area Q with the growth area severity T:

[0075] |f(x,y)-m|≤T (4)

[0076] If the growth conditions are met, the plant continues to grow, otherwise it stops growing.

[0077] Step S2-4: taking the grayscale mean value in the grown area Q as the flame brightness feature V1 of the flame image: V1=m;

[0078] Step S2-5: Use the size of the grown region as the flame region feature V2 of the flame image: V2=n

[0079] Step S2-6: Sum the flame area Q at the current time t t and the flame area Q at time t+1 t+1 The grayscale difference at the same position is used as the flame flickering degree feature V3:

[0080]

[0081] Where g t+1 (a, b) is the flame area Q at time t+1 t+1 Gray value, g t (a, b) is the flame area Q at time t t+1 Grayscale value.

[0082] Reference Figure 2 , Step S3: Use the known label working condition samples to train the sintering working condition recognition model: The fusion data D of the known label working condition samples l Input the constructed variable time lag feature extraction module to extract the variable time lag features between the thermal variable data and the flame video feature sequence in the corresponding working condition sample Time-varying hysteresis characteristics After transposition, the variable coupling feature extraction module is constructed to extract the variable coupling features between the variables of the corresponding working condition samples. Variable Coupling Features Input the classifier to perform parameter update learning of the sintering condition identification model;

[0083] In step S3, the sintering condition recognition model includes a variable time-delay feature and variable coupling feature extraction module, a pseudo-label learning module, and a classifier. The variable time-delay feature and variable coupling feature extraction module includes a variable time-delay feature extraction module and a variable coupling feature extraction module. The pseudo-label learning module is connected to the variable time-delay feature extraction module and the variable coupling feature extraction module. The variable time-delay feature extraction module is connected to the variable coupling feature extraction module, and the variable coupling feature extraction module is connected to the classifier. The variable time-delay feature extraction module is used to extract the variable time-delay features between each thermal variable data and the flame video feature sequence in the working condition sample; the variable coupling feature extraction module is used to extract the variable coupling features between each variable in the working condition sample; and the pseudo-label learning module is used to calculate the similarity of the variable time-delay features and variable coupling features between abnormal working condition samples with known labels and unlabeled working condition samples. Based on the feature similarity, the unlabeled working condition sample is calibrated with a corresponding pseudo-label for the abnormal working condition, or the unlabeled working condition sample is discarded.

[0084] Among them, the variable time-delay feature extraction module adopts the auto-correlation attention module, which can be used to solve the situation where the time series variables have different lags; the variable coupling feature extraction module is implemented using the multi-layer perceptron MLP network;

[0085] In step S3, each thermal variable data Z in the fusion feature is y With the flame video feature sequence V i Use the autocorrelation attention module to extract variable time lag features. The specific steps are as follows:

[0086] Step S3-1: Multiple variable data X within time period t j (1:t)=[x j (1),...,x j (t)], which is then linearly embedded as the query input Q of the autocorrelation attention module j , with multiple variable data X in a random time period τ j (1:τ)=[x j (1),...,x j (τ)] is linearly embedded as the key input K of the autocorrelation attention module j , with multiple variable data X within the time period τ~t j (τ:t)=[x j (τ),...,x j (t)] is linearly embedded and then input into V as the value of the autocorrelation attention module j :

[0087] Q j =W Q ·X j (1:t) (6)

[0088] K j =W K ·X j (1:τ) (7)

[0089] V j =W v ·X j (τ:t) (8)

[0090] Where, X j Represents the thermal variables or flame video feature variables in the fusion feature, W Q 、W K 、W V are the query matrix, key matrix, and value matrix of linear embedding, respectively, τ<t.

[0091] Step S3-2: Input the key into K by interpolation j , value input V j The size and query input Q j After the size is unified, enter Q for the query j Perform fast Fourier transform and input K for key j Perform Fourier conjugate operation, and the product of the two results is used as the two components Qj , K j Common frequency domain Then substitute τ into the common frequency domain Perform Fourier inverse transform to obtain two components Q j , K j Similarity

[0092]

[0093] Among them, F(x), F * (x), F -1 (x) represents Fourier transform, Fourier conjugate operation, and inverse Fourier transform, respectively; f represents the frequency domain component; and i represents the imaginary unit; Represents two components Q j , K j The common frequency domain, Represents two components Q j , K j similarity.

[0094] Step S3-3: Similarity under multiple random times τ Sort and select the k with the highest similarity Corresponding τ1,...,τ k , for k Use the Sofamax function to assign attention weights and get k attention weights k They are k attention weights They are

[0095]

[0096] Where, τ1, ..., τ k represents k random times τ, L represents the upper limit of τ, and Topk(·) represents the selection of the highest k values.

[0097] Step S3-4: X j (1:τ)=[x j (1),...,x j (τ)] moves to X j (τ:t)=[x j (τ),...,x j (t)] The value formed by inputting V j '=W V ·X j (τ:t) end, forming Roll(V j ',τ), so that the data length is the same as Xj (1:t)=[x j (1),...,x j (t)] are the same; for the k Rolls with the highest similarity (V j ',τ), k Roll(V j ',τ) are Roll(V j ',τ1),...,Roll(V j ',τ k ), k Roll(V j ',τ) and k attention weights Perform weighted summation to obtain the output of the autocorrelation attention module, and j , the output of the autocorrelation attention module is expressed as:

[0098]

[0099] Roll(x,y) represents the time series data formed by concatenating x with y, i'∈1...k.

[0100] In step S3, the intervariable coupling characteristics of the corresponding working condition samples are extracted The specific steps are as follows:

[0101] Step S3-5: Calculate the output of the autocorrelation attention module for each variable in the working condition sample, and cascade the output to obtain the time-varying lag characteristics between each thermal variable data in the working condition sample and the flame video feature sequence.

[0102] Step S3-6: After transposition, it is input into the variable coupling feature extraction module according to the variable dimension direction, and the variable coupling features between each variable channel are output. The row direction is the time dimension, and the column direction is the variable channels.

[0103] Variable coupling characteristics of working condition samples The time-varying lag characteristic Based on this, It includes both the variable time lag characteristics of each variable channel of the working condition sample and the variable coupling characteristics between the variable channels.

[0104] In step S3, the classifier is implemented using a fully connected layer, and the classification loss is as follows:

[0105]

[0106] Among them, N represents the number of samples of all known label conditions, N i”is the number of i” type working condition samples in the known label working condition samples, x n Indicates the true value of the nth sample, y n Indicates the predicted value of the nth sample, w i” Indicates the classification weight of the “i” type working condition.

[0107] Step S4: After the sintering condition recognition model is trained using known label condition samples, the fusion data D of the unlabeled condition samples is nl , input the constructed variable time lag feature extraction module to extract the variable time lag features between the thermal variable data and the flame video feature sequence in the unlabeled working condition sample Time-varying hysteresis characteristics Input the constructed variable coupling feature extraction module to extract the variable coupling features between variables of the unlabeled working condition samples

[0108] Reference Figure 3 Step S5: Calculate the similarity of the variable time lag features and variable coupling features between the abnormal working condition samples with known labels and the unlabeled working condition samples. Based on the feature similarity, calibrate the corresponding abnormal working condition pseudo-label for the unlabeled working condition sample, or discard the unlabeled working condition sample. Specifically, the following steps are included:

[0109] Step S5-1: Calculate the similarity of the time-delay characteristics of the overburning condition samples with known labels and the unlabeled condition samples Set the similarity threshold of the time-delay features between the overburning condition samples and the unlabeled condition samples Determining the similarity of time-varying delay features Is it greater than the threshold? If it is greater than, go to step S5-2 to further calculate the similarity of the variable coupling characteristics of the overburning condition sample with known labels and the unlabeled condition sample. If not, go to step S5-3 to further calculate the similarity of the variable time lag characteristics of the underfire condition sample with known labels and the unlabeled condition sample.

[0110] Step S5-2: Calculate the similarity of the variable coupling characteristics between the overburning condition sample with known labels and the unlabeled condition sample Set the similarity threshold of the variable coupling features between the overburning condition samples and the unlabeled condition samples Determine the similarity of variable coupling characteristics Is it greater than the threshold? If it is greater than, the overburning condition pseudo label is calibrated for the unlabeled working condition sample; if it is not greater than, go to step S5-3 to further calculate the similarity of the variable time lag characteristics of the underburning working condition sample with known label and the unlabeled working condition sample

[0111] Step S5-3: Calculate the similarity of the time-delay characteristics between the underburned condition samples with known labels and the unlabeled condition samples Set the similarity threshold of the variable time-lag features between underfire condition samples and unlabeled condition samples Determining the similarity of time-varying delay features Is it greater than the threshold? If it is greater than, go to step S5-4 and further calculate the similarity of the variable coupling characteristics of the underfire condition sample with known labels and the unlabeled condition sample. If not, discard the unlabeled working condition sample;

[0112] Step S5-4: Calculate the similarity of the variable coupling characteristics between the underburned working condition samples with known labels and the unlabeled working condition samples Set the variable coupling feature similarity threshold between underfire condition samples and unlabeled condition samples Determine the similarity of variable coupling characteristics Is it greater than the threshold? If it is greater than, the underburning condition pseudo label is calibrated for the unlabeled working condition sample; if it is not greater than, the unlabeled working condition sample is discarded.

[0113] The feature similarity is calculated as follows:

[0114]

[0115] in, F represents the similarity between the i'th class abnormal working condition sample with known label and the j'th class working condition sample without label, j' nl represents the j' class feature of the unlabeled sample, represents the j' class feature of the known label sample, Represents the j' class features of the i' class abnormal working condition samples with known labels, Represents the j'-class feature set of i'-class abnormal working condition samples with known labels, Z j' Represents the j'-type feature set of known label samples, where the j'-type feature is a variable time lag feature or a variable coupling feature, and the i'-type abnormal operating condition is an over-burning condition or an under-burning condition. τ' is a constant, and its value is Square root of the mean.

[0116] The method for setting the feature similarity threshold is as follows: the feature similarities of class j' of all class i' abnormal working condition samples with known labels are added together and the average value is used as the feature similarity threshold.

[0117] Step S6: adding the working condition samples marked as abnormal working condition pseudo labels to the known label samples to form a training sample set with a more balanced sample distribution, and using the training sample set to retrain the sintering working condition recognition model to improve the model's recognition accuracy for small sample abnormal working conditions.

[0118] Part of the experimental data of this embodiment was collected from rotary kilns No. 5 and 6 of the Zhongzhou branch of China Aluminum Corporation. A color charge-coupled device (CCD) camera is installed at the kiln head of the rotary kiln. The signal collected by the CCD is digitized to obtain a flame video in RGB format with a speed of 25 frames per second, a size of 704*576 per frame, and a precision of 24 bits. We convert the RGB image into a grayscale image in preparation for subsequent processing. In order to test the performance of the algorithm, the experiment used multimodal data recorded within 30 days, and a total of 5083 labeled working condition samples were obtained. The specific number of working condition category samples and data set construction information are shown in Table 1:

[0119] Table 1

[0120]

[0121] Table 2 shows the ablation experiment results of the working condition identification of this embodiment. Table 2 shows different working condition identification methods using CNN network, Bi-LSTM network, M1, CNN+M2, Bi-LSTM+M2, and M1+M2 respectively. In Table 2, M1 represents the variable time-delay feature variable coupling feature extraction module, M2 represents the pseudo-label learning module, and M1+M2 represents the working condition identification method that uses the variable time-delay feature variable coupling feature extraction module and the pseudo-label learning module at the same time, which is the method of the present invention. It can be seen from Table 2 that after the working condition samples of this example are processed by the above method, the classification accuracy under normal working conditions, under-burning working conditions, and over-burning working conditions reaches 96.4%, 94.1%, and 91.3%, respectively. Compared with other working condition identification methods, the working condition classification accuracy of the method of the present invention is the highest.

[0122] Table 2

[0123]

[0124] The above experiments were simulated on a server. This server uses an NVIDIA RTX A6000 CPU running at 1455 MHz and 48 GB of memory. The software used is Python. Those skilled in the art may make various modifications and variations to the present invention. If such modifications and variations fall within the scope of the claims and their equivalents, they are also within the scope of protection of the present invention.

[0125] The contents not described in detail in the specification are prior art known to those skilled in the art.

Claims

1. A small sample sintering condition identification method based on multimodal data time-varying and time-delay-varying coupling characteristic modeling, characterized by: The following steps are involved: Step S1: Collecting on-site operating data of the rotary kiln, including flame video and multivariate thermal data, constructing multimodal data operating condition samples, and labeling the data of known operating conditions with operating condition category labels; Step S2: Extract flame image i working condition features from the flame video data in the working condition sample to form multiple flame video feature sequences right Down sampling is performed to obtain the thermal variable data Z in the working condition sample y Flame video feature sequence V with consistent dimension i , and then multiple Z y With multiple V i By parallel splicing, we can obtain the multimodal fusion data D of the working condition samples. Step S3: Use the known label working condition samples to train the sintering working condition recognition model: The fusion data D of the known label working condition samples l Input variable time lag feature extraction module to extract variable time lag features between each thermal variable data and flame video feature sequence in the corresponding working condition sample Time-varying hysteresis characteristics After transposition, input the variable coupling feature extraction module to extract the variable coupling features between the variables of the corresponding working condition samples Variable Coupling Features Input the classifier to perform parameter update learning of the sintering condition identification model; Step S4: After the sintering condition recognition model is trained using known label condition samples, the fusion data D of the unlabeled condition samples is nl , input the variable time lag feature extraction module to extract the variable time lag features between the thermal variable data and the flame video feature sequence in the unlabeled working condition sample Time-varying hysteresis characteristics Input variable coupling feature extraction module to extract variable coupling features between variables of unlabeled working condition samples Step S5: Calculate the similarity of the variable time lag features and variable coupling features of the abnormal working condition samples with known labels and the unlabeled working condition samples. According to the feature similarity, calibrate the corresponding abnormal working condition pseudo-label for the unlabeled working condition sample, or discard the unlabeled working condition sample. Step S6: adding the working condition samples marked as abnormal working condition pseudo labels to the known label samples to form a training sample set with a more balanced sample distribution, and using the training sample set to retrain the sintering working condition recognition model.

2. The small sample sintering condition identification method based on multimodal data time-varying and time-delay-varying coupling characteristic modeling according to claim 1, characterized in that: In step S1, the multivariate thermal data includes the following thermal variables: coal feed rate Z1, slurry flow rate Z2, primary air volume Z3, kiln tail negative pressure Z4, kiln head temperature Z5, kiln tail temperature Z6, main engine current Z7, sintering belt temperature Z8; the operating condition category labels include normal, underburned and overburned, among which underburned and overburned are abnormal operating conditions.

3. The small sample sintering condition identification method based on multimodal data time-varying and time-delay-varying coupling characteristic modeling according to claim 1, characterized in that: In step S2, the multimodal fusion data D is expressed as: D = [V1...V i ; Z1...Z y ]; i represents the number of types of flame image working condition features, and y represents the number of thermal variables.

4. The small sample sintering condition identification method based on multimodal data time-varying and time-delay-varying coupling characteristic modeling according to claim 1, characterized in that: In step S2, the seed region growing method is used to extract flame image features. The flame image features include flame brightness V1, flame area size V2, and flame flickering degree V3. Specifically, Step S2-1: For each point f(a, b) in each flame image frame of the flame video in the working condition sample, the flame region is divided using the seed region growing method. The initial seed point is set as the brightest point in the image. The growth region severity is set as the growth region division condition. The growth region severity T is continuously updated based on the grayscale mean m and standard deviation σ in the current growth region. The calculation process is as follows: Where, T σ The preset value for the harshness of the growing area; Step S2-2: Calculate the grayscale mean m and standard deviation σ of the n known grayscale points (a, b) in the current growth area Q: Where a is the horizontal coordinate of the grayscale point (a, b), and b is the vertical coordinate of the grayscale point (a, b); Step S2-3: Compare the difference between the grayscale value f(x, y) of the grayscale point to be determined (x, y) and the grayscale mean value in the grown area Q with the growth area severity T: |f(x,y)-m|≤T (4) If the growth conditions are met, then continue to grow, otherwise stop growing; Step S2-4: taking the grayscale mean value in the grown area Q as the flame brightness feature V1 of the flame image: V1=m; Step S2-5: Use the size of the grown area as the flame area feature V2 of the flame image: V2 = n Step S2-6: Sum the flame area Q at the current time t t and the flame area Q at time t+1 t+1 The grayscale difference at the same position is used as the flame flickering degree feature V3: Where g t+1 (a, b) is the flame area Q at time t+1 t+1 Gray value, g t (a, b) is the flame area Q at time t t+1 Grayscale value.

5. The small sample sintering condition identification method based on multimodal data time-varying and time-delay-varying coupling characteristic modeling according to claim 1, characterized in that: In step S3, the sintering condition recognition model includes a variable time-delay feature and variable coupling feature extraction module, a pseudo-label learning module and a classifier. The variable time-delay feature and variable coupling feature extraction module includes a variable time-delay feature extraction module and a variable coupling feature extraction module. The pseudo-label learning module is connected to the variable time-delay feature extraction module and the variable coupling feature extraction module. The variable time-delay feature extraction module is connected to the variable coupling feature extraction module, and the variable coupling feature extraction module is connected to the classifier. The variable time-delay feature extraction module is used to extract the variable time-delay feature between each thermal variable data in the working condition sample and the flame video feature sequence. The variable coupling feature extraction module is used to extract the variable coupling features between the variables of the working condition samples; the pseudo-label learning module is used to calculate the similarity of the variable time-delay features and variable coupling features between the abnormal working condition samples with known labels and the unlabeled working condition samples. According to the feature similarity, the corresponding abnormal working condition pseudo-label is calibrated for the unlabeled working condition samples, or the unlabeled working condition samples are discarded.

6. The small sample sintering condition identification method based on multimodal data time-varying and time-delay-varying coupling characteristic modeling according to claim 1, characterized in that: The variable time-delay feature extraction module adopts an autocorrelation attention module, and the variable coupling feature extraction module adopts a multi-layer perceptron MLP network.

7. The small sample sintering condition identification method based on multimodal data time-varying and time-delay-varying coupling characteristic modeling according to claim 6, characterized in that: In step S3, each thermal variable data Z in the fusion feature is y With the flame video feature sequence V i Use the autocorrelation attention module to extract variable time lag features. The specific steps are as follows: Step S3-1: Multiple variable data within time period t After linear embedding, it is used as the query input Q of the autocorrelation attention module j , with multiple variable data X in a random time period τ j (1:τ)=[x j (1),...,x j (τ)] is linearly embedded as the key input K of the autocorrelation attention module j , with multiple variable data X within the time period τ~t j (τ:t)=[x j (τ),...,x j (t)] is linearly embedded and then input into V as the value of the autocorrelation attention module j : Q j =W Q ·X j (1:t) (6) K j =W K ·X j (1:t) (7) V j =W v ·X j (τ:t) (8) Where, X j Represents the thermal variables or flame video feature variables in the fusion feature, W Q 、W K 、W V are the query matrix, key matrix, and value matrix of linear embedding, respectively, τ<t; Step S3-2: Input the key into K by interpolation j , value input V j The size and query input Q j After the size is unified, enter Q for the query j Perform fast Fourier transform and input K for key j Perform Fourier conjugate operation, and the product of the two results is used as the two components Q j , K j Common frequency domain Then substitute τ into the common frequency domain Perform Fourier inverse transform to obtain two components Q j , K j Similarity Among them, F(x), F * (x), F -1 (x) represents Fourier transform, Fourier conjugate operation, and inverse Fourier transform, respectively; f represents the frequency domain component; and i represents the imaginary unit; Represents two components Q j , K j The common frequency domain, Represents two components Q j , K j similarity; Step S3-3: Similarity under multiple random times τ Sort and select the k with the highest similarity Corresponding τ1,...,τ k , for k Use the Sofamax function to assign attention weights and get k attention weights k They are k attention weights They are Where τ1, ..., τ k represents k random times τ, L represents the upper limit of τ, and Topk(·) represents the highest k values ​​selected; Step S3-4: X j (1:τ)=[x j (1),...,x j (τ)] moves to X j (τ:t)=[x j (τ),...,x j (t)] The value formed by inputting V j '=W V ·X j (τ:t) end, forming Roll(V j ',τ), so that the data length is the same as X j (1:t)=[x j (1),...,x j (t)] are the same; for the k Rolls with the highest similarity (V j ',τ), k Roll(V j ',τ) are Roll(V j ',τ1),...,Roll(V j ',τ k ), k Roll(V j ',τ) and k attention weights Perform weighted summation to obtain the output of the autocorrelation attention module, and j , the output of the autocorrelation attention module is expressed as: Roll(x,y) represents the time series data formed by concatenating x with y, i'∈1...k.

8. The small sample sintering condition identification method based on multimodal data time-varying and time-delay-varying coupling characteristic modeling according to claim 1 is characterized in that: Step S5 specifically includes the following steps: Step S5-1: Calculate the similarity of the time-delay characteristics of the overburning condition samples with known labels and the unlabeled condition samples Set the similarity threshold of the time-delay features between the overburning condition samples and the unlabeled condition samples Determining the similarity of time-varying delay features Is it greater than the threshold? If it is greater than, go to step S5-2 to further calculate the similarity of the variable coupling characteristics of the overburning condition sample with known labels and the unlabeled condition sample. If not, go to step S5-3 to further calculate the similarity of the variable time lag characteristics of the underfire condition sample with known labels and the unlabeled condition sample. Step S5-2: Calculate the similarity of the variable coupling characteristics between the overburning condition sample with known labels and the unlabeled condition sample Set the similarity threshold of the variable coupling features between the overburning condition samples and the unlabeled condition samples Determine the similarity of variable coupling characteristics Is it greater than the threshold? If it is greater than, the overburning condition pseudo label is calibrated for the unlabeled working condition sample; if it is not greater than, go to step S5-3 to further calculate the similarity of the variable time lag characteristics of the underburning working condition sample with known label and the unlabeled working condition sample Step S5-3: Calculate the similarity of the time-delay characteristics between the underburned condition samples with known labels and the unlabeled condition samples Set the similarity threshold of the variable time-lag features between underfire condition samples and unlabeled condition samples Determining the similarity of time-varying delay features Is it greater than the threshold? If it is greater than, go to step S5-4 and further calculate the similarity of the variable coupling characteristics of the underfire condition sample with known labels and the unlabeled condition sample. If not, discard the unlabeled working condition sample; Step S5-4: Calculate the similarity of the variable coupling characteristics between the underburned working condition samples with known labels and the unlabeled working condition samples Set the variable coupling feature similarity threshold between underfire condition samples and unlabeled condition samples Determine the similarity of variable coupling characteristics Is it greater than the threshold? If it is greater than, the underburning condition pseudo label is calibrated for the unlabeled working condition sample; if it is not greater than, the unlabeled working condition sample is discarded.

9. The small sample sintering condition identification method based on multimodal data time-varying and time-delay-varying coupling characteristic modeling according to claim 8, characterized in that: The feature similarity is calculated as follows: in, Indicates the similarity between the i'th class abnormal working condition sample with known label and the j'th class working condition sample without label, represents the j' class feature of the unlabeled sample, represents the j' class feature of the known label sample, Represents the j' class features of the i' class abnormal working condition samples with known labels, Represents the j'-class feature set of i'-class abnormal working condition samples with known labels, Z j' Represents the j'-type feature set of known label samples, where the j'-type feature is a variable time lag feature or a variable coupling feature, and the i''-type abnormal operating condition is an over-burning condition or an under-burning condition; τ' is a constant, and its value is Square root of the mean.

10. The small sample sintering condition identification method based on multimodal data time-varying and time-delay-varying coupling characteristic modeling according to claim 8, characterized in that: The method for setting the feature similarity threshold is as follows: the feature similarities of class j' of all class i' abnormal working condition samples with known labels are added together and the average value is used as the feature similarity threshold.

Images