Blast furnace top gas flow state identification method based on time sequence image deep learning

Abstract

The invention relates to the technical field of blast furnace ironmaking, and discloses a blast furnace top gas flow state recognition method based on time sequence image deep learning, which comprises the following steps: acquiring a blast furnace top infrared video, splitting into images according to frames, and performing expert feature marking on the images to obtain a time sequence fusion sample and an expansion sample; constructing a blast furnace top gas flow state identification model, wherein the model comprises a time sequence fusion feature fusion module for extracting spatial features and fusing the spatial features with expert features, a time feature extraction module, a position coding module for fusing the time features with the position of an image in an original blast furnace top infrared video, and an attention enhancement module for integrating global information; and constructing a time sequence fusion sample corresponding to the blast furnace top infrared video to be identified by using the expanded sample training model, and inputting the time sequence fusion sample into the trained model to obtain an identification result of the gas flow state. According to the method, various characteristics of the coal gas flow in the real-time blast furnace top infrared video can be comprehensively considered, and high-precision identification of the distribution and the development state of the coal gas flow is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of blast furnace ironmaking, and in particular to a method for identifying the gas flow state of a blast furnace top based on deep learning of time series images. Background Art

[0002] The blast furnace ironmaking process is a core and critical link in the steel production process, and its energy consumption and emissions account for a significant portion of the overall steel production process. During this process, monitoring the top gas flow status allows real-time monitoring of the blast furnace's operating status. When the top gas flow is too weak, it can cause furnace temperature drop, thickening of the furnace wall, overhanging material, and material collapse, leading to serious problems such as reduced or no air flow. When the top gas flow is too strong, it can cause scouring, exacerbating furnace erosion, shortening the blast furnace's lifespan, and wasting energy. Real-time monitoring of the top gas flow status allows for timely adjustments to blast furnace operations in the event of anomalies, ensuring stable operation. Therefore, effective identification of the top gas flow status is essential for ensuring stable charge delivery, proper chemical reactions, and heat exchange, and is key to achieving blast furnace stability, energy conservation, increased production, and improved quality.

[0003] In the actual process of blast furnace ironmaking, the state of the coal gas flow at the top of the furnace is mostly judged manually, which relies heavily on the professional knowledge of the on-site operators, and the judgment results often vary from person to person, lacking objectivity and consistency. Therefore, in order to efficiently and automatically identify the state of the coal gas flow at the top of the blast furnace, there is a method in the prior art to identify the intensity of the central coal gas flow and the edge coal gas flow by comprehensively evaluating the infrared image of the furnace top. When conducting a comprehensive evaluation of the infrared image of the furnace top, the scene methods mainly include numerical modeling, data-driven method, expert system method and image method. Among them:

[0004] Numerical modeling utilizes knowledge from disciplines such as blast furnace dynamics and heat transfer to develop theoretical models describing the blast furnace smelting process. By obtaining the composition of gas flows at different locations within the blast furnace, a gas flow distribution model is established to analyze the gas flow distribution and temperature field at the furnace top. However, this method relies heavily on assumptions and simplifications, resulting in significant discrepancies between simulated and actual results. Therefore, it is only suitable for theoretical research and is not suitable for practical application in identifying gas flow conditions at the blast furnace top.

[0005] The data-driven approach relies on a cross-shaped temperature measurement device on the blast furnace roof to obtain discrete temperature data. This method uses machine learning and deep learning methods to determine the state of gas flow. However, the discrete temperature data obtained by the cross-shaped temperature measurement device only indirectly reflects the distribution of local gas flow and cannot fully reflect the state of gas flow at the furnace roof.

[0006] Expert systems are a set of rules established based on knowledge of the blast furnace ironmaking process, blast furnace operation, and the experience of blast furnace experts. They use fuzzy mathematics, fuzzy reasoning, and decision-making methods to identify the state of blast furnace top gas flow. However, this approach often lacks effective knowledge updating and automatic adjustment capabilities, making it difficult to timely acquire new patterns in blast furnace conditions, resulting in a high probability of error in the analysis results.

[0007] Image-based methods use computer vision technology to identify infrared images of blast furnace roofs. However, existing image-based methods primarily use the center point and the proportion of singular values ​​of the central gas flow as input features to classify the distribution of top gas flow, failing to fully utilize image features. Furthermore, most of these methods only consider the development of the central gas flow and ignore the development of the peripheral gas flow, making it impossible to effectively identify the distribution and development of the top gas flow. Summary of the Invention

[0008] To this end, the technical problem to be solved by the present invention is to overcome the shortcomings of the existing technology and provide a blast furnace top gas flow status recognition method based on time series image deep learning, which can comprehensively consider various characteristics of the gas flow in the real-time blast furnace top infrared video to achieve high-precision recognition of the distribution of the furnace top gas flow and its development status.

[0009] To solve the above technical problems, the present invention provides a method for identifying the gas flow state of a blast furnace top based on deep learning of time series images, comprising:

[0010] Acquire an infrared video of a blast furnace top and split it into infrared images by frame. Perform expert feature labeling on each frame of the infrared image to obtain a time-series fusion sample. Perform window sampling and random enhancement processing on the time-series fusion sample to obtain an expanded sample set.

[0011] Construct a blast furnace top coal gas flow state recognition model, the blast furnace top coal gas flow state recognition model includes a time series fusion feature fusion module, a time feature extraction module, a position coding module, an attention enhancement module, and a classification module, the time series fusion feature fusion module extracts the spatial features of the furnace top infrared image and fuses them with the expert features to obtain a time series fusion feature, the time feature extraction module extracts the time features of the time series fusion feature, the position coding module fuses the time feature with the position of the furnace top infrared image in the original blast furnace top infrared video to obtain a coding feature, the attention enhancement module integrates global information according to the coding feature to obtain a global feature, and the classification module performs category prediction according to the global feature;

[0012] The expanded sample set is used to train the blast furnace top coal gas flow state recognition model, and a time series fusion sample corresponding to the blast furnace top infrared video to be identified is constructed and input into the trained blast furnace top coal gas flow state recognition model to obtain the recognition result of the blast furnace top coal gas flow state.

[0013] Furthermore, the expert feature marking of each frame of furnace roof infrared image is performed to obtain a time series fusion sample, specifically:

[0014] Calculate the expert features of each frame of furnace top infrared image, including central coal flow area, central coal flow brightness, central coke package area, central coke package brightness, central coal flow inclination, and edge coal flow brightness.

[0015] The method for calculating the central coal gas flow area is as follows: using a threshold segmentation method to segment the furnace top infrared image to obtain a central coal gas flow area, and calculating the area of ​​the central coal gas flow area to obtain the central coal gas flow area;

[0016] The calculation method of the central coal gas flow brightness is as follows: calculating the average brightness of the central coal gas flow area to obtain the central coal gas flow brightness;

[0017] The central coke package area is calculated by performing a quadratic ellipse fitting on the central coal gas flow region to obtain a central coke package area, and calculating the area of ​​the central coke package area to obtain the central coke package area;

[0018] The calculation method of the central focal package brightness is as follows: calculating the average brightness of the central focal package area to obtain the central focal package brightness;

[0019] The central gas flow inclination is calculated by performing weighted integration of the grayscale values ​​of the image pixels in the central gas flow area using geometric moments to obtain the central gas flow inclination.

[0020] The method for calculating the edge gas flow brightness is as follows: calculating the average brightness of the remaining area after removing the central gas flow area to obtain the edge gas flow brightness;

[0021] The calculated expert features are normalized and matched with each frame of furnace top infrared image to obtain time series fusion samples.

[0022] Furthermore, window sampling and random enhancement processing are performed on the time series fusion samples to obtain an expanded sample set, specifically:

[0023] The number of samples in the i-th category is increased by M using the following formula i Times:

[0024]

[0025] Among them, N is the number of frames of the infrared video of the blast furnace top, L is the size of the time window, and S is the step size. represents the floor function, W i Represents the sampling weight of each category, W i The calculation method is:

[0026]

[0027] Among them, Max(Class) represents the number of classes with the largest number of samples among all categories, and Class i represents the original number of samples in category i;

[0028] After sampling the time window, M is improved i The number of samples is increased to 2M. i times;

[0029] Increase the number of samples to 2M i The samples after multiplication are randomly enhanced to obtain the expanded sample set.

[0030] Furthermore, the random enhancement calculation method is:

[0031]

[0032] Among them, T() represents the random enhancement operation, I(x,y) represents the pixel value of the original sample image at the coordinate (x,y) before random enhancement, α represents the enhancement contrast adjustment factor, β represents the brightness adjustment factor, and λ represents the sharpening intensity. stands for Laplace operator.

[0033] Furthermore, the temporal fusion feature fusion module extracts the spatial features of the furnace top infrared image and fuses them with the expert features, specifically:

[0034] The ResNet-18 model is used to extract the spatial features of the furnace top infrared images in the expanded sample set. The spatial features are projected onto a vector of preset length through a fully connected layer. The projected spatial features and expert features are horizontally spliced ​​to obtain temporal fusion features.

[0035] Furthermore, the position encoding module fuses the time feature with the position of the furnace top infrared image in the original blast furnace top infrared video to obtain a coding feature, specifically:

[0036] The position code is calculated based on the temporal position of the furnace top infrared image in the expanded sample set in the original blast furnace top infrared video, and the coding feature is obtained by fusing the time feature and the position code:

[0037] f=[f1,f2,...,f i ,...,ft ],

[0038] Wherein, f is the coding feature, f i is the encoding feature of the furnace top infrared image at time i, and t is the frame number of the furnace top infrared image in the original blast furnace top infrared video;

[0039] f i The calculation method is:

[0040] f i =y i +p i ,

[0041] Among them, y i is the time feature of the furnace top infrared image at time i, p i is the position code of the furnace top infrared image at time i.

[0042] Furthermore, the attention enhancement module integrates global information according to the coding features to obtain global features, specifically: setting classification marks, combining the coding features to perform attention calculation, and obtaining the global features as follows:

[0043]

[0044] Among them, F(c [cls] ,f) is the global feature, Q [cls] is the Query vector, K is the Key matrix, V is the Value matrix, d is the vector dimension, and T is the transpose operation;

[0045] The calculation method of query vector is:

[0046] Q [cls] =c [cls] W Q ,

[0047] Among them, c [cls] is the classification mark, W Q is the weight matrix of the Query vector;

[0048] The calculation method of Key matrix and Value matrix is:

[0049] K=f·W K ,

[0050] V=f·W V ;

[0051] Among them, f is the encoding feature, W K is the weight matrix of the Key matrix, W V is the weight matrix of the Value matrix.

[0052] Furthermore, when training the blast furnace top gas flow state recognition model, the total loss function constructed is:

[0053] L=L CE +γ·L align +δ·L TC ,

[0054] Among them, L represents the total loss function, L CE is the cross entropy loss function, L align is the multi-feature alignment loss function constructed based on expert features, L TC is the temporal consistency loss function constructed based on expert features, and γ and δ are weight coefficients.

[0055] Furthermore, the calculation method of the multi-feature alignment loss function constructed based on the expert features is:

[0056]

[0057] Among them, f res (i) represents the temporal fusion feature corresponding to the infrared image of the furnace top in frame i, f hand (i) represents the expert feature corresponding to the i-th frame of the furnace top infrared image, ||||2 represents the two-norm, and N is the number of frames of the furnace top infrared image.

[0058] Furthermore, the calculation method of the temporal consistency loss function constructed based on the expert features is:

[0059]

[0060] Among them, KL() represents the calculation divergence, p i,t Represents the predicted probability distribution of the i-th sample in the expanded sample set at time point t.

[0061] The above technical solution of the present invention has the following beneficial effects compared with the prior art:

[0062] This method comprehensively considers the various characteristics of coal gas flow in infrared video of blast furnace tops. It enriches the feature information by fusing image features with expert features. Furthermore, by combining temporal features and position encoding, the model can understand the corresponding positional relationships between each frame and learn global information. This enables real-time and accurate recognition of the coal gas flow status at the top of the furnace. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to make the content of the present invention more clearly understood, the present invention is further described in detail below based on specific embodiments of the present invention in conjunction with the accompanying drawings, wherein:

[0064] Figure 1 Flowchart of the method in the preferred embodiment of the present invention.

[0065] Figure 2 This is a general architecture diagram of the blast furnace top gas flow state identification model constructed in the preferred embodiment of the present invention.

[0066] Figure 3 This is an example of infrared image marking of the furnace top commonly used by blast furnace personnel. DETAILED DESCRIPTION

[0067] The present invention will be further described below with reference to the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it. However, the embodiments are not intended to limit the present invention.

[0068] Reference Figure 1 As shown, the present invention discloses a method for identifying the gas flow state of a blast furnace top based on deep learning of time series images, comprising the following steps:

[0069] S1: Obtain the infrared video of the blast furnace top and split it into furnace top infrared images by frame. Perform expert feature labeling on each frame of the furnace top infrared image to obtain a time series fusion sample.

[0070] S1-1: In order to monitor the production status of blast furnaces, a furnace top infrared monitoring system is currently installed on the top of each blast furnace. The blast furnace top infrared video is collected through the furnace top infrared monitoring system, and the furnace top infrared video is pre-processed through the data enhancement strategy. The data enhancement strategies include rotation, cropping, brightness adjustment, sharpening and contrast enhancement, etc. In this embodiment, in order to enhance the robustness of the model, two types of sharpening and contrast enhancement are selected from these data enhancement strategies based on expert experience and task requirements to process the furnace top infrared video, so that the outlines of the central coal gas flow and the edge coal gas flow are clearer, so that the model can better identify the characteristics of the coal gas flow image in complex environments. The specific selected data enhancement strategy method can be adjusted according to the actual video situation.

[0071] The preprocessed infrared video is split into furnace top infrared images by frame.

[0072] S1-2: In order to accurately reflect the state of the furnace top gas flow, each frame of the furnace top infrared image is marked with expert features to obtain a time series fusion sample, specifically:

[0073] The expert features of each frame of furnace top infrared image are calculated, and the expert features include the center coal gas flow area, center coal gas flow brightness, center coke package (i.e., the bottom area of ​​the center coal gas flow) area, center coke package brightness, center coal gas flow inclination, and edge coal gas flow brightness.

[0074] The position distribution of the central gas flow is relatively fixed, so the method for calculating the central gas flow area is as follows: using the threshold segmentation method to segment the furnace top infrared image to obtain the central gas flow area, and calculating the area of ​​the central gas flow area to obtain the central gas flow area;

[0075] The calculation method of the central coal gas flow brightness is as follows: calculating the average brightness of the central coal gas flow area to obtain the central coal gas flow brightness;

[0076] Since the central coke package is elliptical, the method for calculating the central coke package area is as follows: performing a quadratic ellipse fitting on the central coal gas flow area to obtain the central coke package area, and calculating the area of ​​the central coke package area to obtain the central coke package area;

[0077] The calculation method of the central focal package brightness is as follows: calculating the average brightness of the central focal package area to obtain the central focal package brightness;

[0078] The flame above the central gas flow reflects the strength and stability of the central gas flow. When it is strong, it exhibits a relatively stable tilt pattern, while when it is weak or unstable, it frequently changes direction. Therefore, the central gas flow tilt is calculated by performing a weighted integration of the grayscale values ​​of the image pixels in the central gas flow area using geometric moments to obtain the central gas flow tilt.

[0079] The method for calculating the edge gas flow brightness is as follows: calculating the average brightness of the remaining area after removing the central gas flow area to obtain the edge gas flow brightness.

[0080] S1-3: Because the six expert features have different scales, the calculated expert features are normalized and then mapped to each frame of the furnace top infrared image to obtain a time series fusion sample. In this embodiment, six expert features are designed to provide richer feature information for the furnace top gas flow state recognition task.

[0081] S2: Perform window sampling and random enhancement processing on the time series fusion samples to obtain an expanded sample set to increase the number of samples and achieve a more accurate recognition effect, and divide the expanded sample set into a training set and a test set.

[0082] S2-1: Use a time window for sampling, set the time window size to L, and the step size to S for sliding. In addition, since the number of samples in each category is inconsistent, a category weight adjustment mechanism and enhancement strategy are added to balance the sample distribution of various gas flow states. The number of samples in category i is increased by M using the following formula i Times:

[0083]

[0084] Among them, N is the number of frames of the infrared video of the blast furnace top, L is the size of the time window, and S is the step size. represents the floor function, W i Represents the sampling weight of each category, W i The calculation method is:

[0085]

[0086] Among them, Max(Class) represents the number of classes with the largest number of samples among all categories, and Class i Represents the original number of samples of category i; when W i When >1, the i-th class will be oversampled during the sampling process to compensate for the problem of class imbalance.

[0087] S2-2: In order to further expand the sample, the above time window is sampled and M is increased. i The number of samples is increased to 2M. i times.

[0088] S2-3: Since simply copying samples during oversampling will lead to overfitting of the model, the number of oversampled time series image samples, that is, the number of samples is increased to 2M i The samples after multiplication are randomly enhanced to obtain the expanded sample set. The calculation method of random enhancement is:

[0089]

[0090] Among them, T() represents the random enhancement operation, I(x,y) represents the pixel value of the original sample image at the coordinate (x,y) before random enhancement, α represents the enhancement contrast adjustment factor, β represents the brightness adjustment factor, and λ represents the sharpening intensity. stands for Laplace operator.

[0091] S2-4: Divide the expanded sample set into training and test sets in a ratio of 8:2. To reduce the impact of video data with similar time periods on training, ensure that the training and test sets come from data from different months when dividing the dataset.

[0092] S3: Construct a blast furnace top gas flow state recognition model. Since the infrared video of the blast furnace top before charging and the expert features are important features for judging the gas flow at the top of the blast furnace, the present invention uses time series feature fusion to perform time series feature classification to identify the state of the gas flow at the top of the blast furnace. Figure 2 As shown, the blast furnace top gas flow state recognition model includes a time series fusion feature fusion module, a time feature extraction module, a position encoding module, an attention enhancement module, and a classification module.

[0093] The temporal fusion feature fusion module extracts the spatial features of the furnace top infrared image and fuses them with the expert features to obtain the temporal fusion feature x i The time feature extraction module extracts the time feature y of the time series fusion feature i , the position encoding module converts the temporal feature y i The attention enhancement module integrates the global information according to the coding feature f to obtain the global feature, and the classification module predicts the category of the blast furnace top gas flow state according to the global feature.

[0094] S3-1: The temporal fusion feature fusion module extracts the spatial features of the furnace top infrared image and fuses them with the expert features, specifically:

[0095] The ResNet-18 model is used to extract the spatial features of the furnace top infrared image in the expanded sample set. In this embodiment, the deep image features of length 512 are extracted from the output of the last convolutional layer of the ResNet-18 model through global average pooling. In order to balance the contribution of image features and expert features and reduce computational complexity, the spatial features are projected onto a vector of preset length through a fully connected layer, and the projected spatial features and expert features are horizontally spliced ​​to obtain a temporal fusion feature. The temporal fusion feature of the furnace top infrared image at time i in the extracted expanded sample set (i.e., the i-th frame of the furnace top infrared image in the original blast furnace top infrared video) is denoted as x. i In this embodiment, the extracted spatial features are projected onto a vector of length 18 through a fully connected layer, and then horizontally concatenated with the normalized expert features to form a time-series fusion feature vector of length 24. Finally, the time-series fusion feature vector is input into the BiLSTM network to extract temporal features.

[0096] S3-2: In this embodiment, the time feature extraction module uses a bidirectional long short-term memory network (Bidirectional Long Short-Term Memory, BiLSTM) to extract time features, and records the extracted x i The time characteristic is y i BiLSTM runs two independent LSTMs at each time step. The output at each moment is the concatenation of the hidden states of the forward and backward LSTMs. It can take into account the context of the time series and provide a richer feature representation. Since the length of the time series fusion feature is 24 and the time window size is L, the input size in this embodiment is (L, 24). The hidden layer size is 24 and the number of unit layers is 1. Finally, the time feature vector of size (L, 48) is obtained. i Send to the position encoding module.

[0097] S3-3: The position encoding module fuses the time feature with the position of the furnace top infrared image in the original blast furnace top infrared video to obtain the encoding feature.

[0098] S3-3-1: Calculate the position code p according to the temporal position of the furnace top infrared image in the expanded sample set in the original blast furnace top infrared video (i.e., the position i of the furnace top infrared image of the i-th frame in the original blast furnace top infrared video). i , It is a learnable positional encoding, and the positional encoding p is calculated. i The method can be adjusted according to actual conditions and is not limited in this embodiment.

[0099] S3-3-2: The time feature y i and position code p i The fused encoding features are:

[0100] f=[f1,f2,...,f i ,...,f t ],

[0101] Among them, f is the encoding feature finally output by the position encoding module, f i is the encoding feature of the furnace top infrared image at time i, and t is the frame number of the furnace top infrared image in the original blast furnace top infrared video;

[0102] f i The calculation method is:

[0103] f i =y i +p i ,

[0104] Among them, y i is the temporal feature of the furnace top infrared image at time i, i.e., the feature output by BiLSTM, p i is the position code of the furnace top infrared image at time i.

[0105] S3-4: In this embodiment, the attention enhancement module is implemented based on the Transformer Longformer model, specifically using the temporal attention mechanism in the Longformer model to integrate the global information of the encoded feature f. Specifically:

[0106] S3-4-1: Set classification mark c [cls] , c [cls] is a randomly initialized learnable variable, d is the dimension of the hidden layer. In the input embedding layer, the subscript [CLS] is assigned a fixed position (usually 0), and the global information of the entire sequence is aggregated through the self-attention mechanism as the representation vector for the classification task.

[0107] The present invention encodes the frame positions in the original video as position embedding, so that the Longformer model can understand the corresponding position relationship between each frame. In addition, considering that the change process of gas flow, such as coke bag and flame transformation, has typical temporal position characteristics, a special classification mark c is added to reflect its temporal position information. [cls] , which will gradually learn the global information of the entire sequence during the model training process and serve as the representative vector of the entire sequence for downstream classification tasks.

[0108] S3-4-2: For c [cls] Perform attention calculation with the encoded feature f and gradually learn the global information. The specific process is as follows:

[0109] According to c [cls] Calculate the query vector:

[0110] Q [cls] =c [cls] W Q ,

[0111] Among them, Q [cls] is the Query vector, W Q is the weight matrix of the Query vector;

[0112] Calculate the Key matrix and Value matrix based on the encoded feature f:

[0113] K=f·W K ,

[0114] V=f·W V ;

[0115] Among them, K is the Key matrix, V is the Value matrix, and W K is the weight matrix of the Key matrix, W V is the weight matrix of the Value matrix,

[0116] The global features that aggregate global information are:

[0117]

[0118] Among them, F(c [cls] ,f) aggregates the information of all time points, which is the global feature, d is the vector dimension, and T is the transpose operation.

[0119] Since F(c[cls] ,f) Fusion of global classification label information c [cls] and information at all points in time, especially c [cls] It is possible to focus on other tags in the input feature sequence, so this embodiment uses F(c [cls] ,f) as the final feature representation of the video and apply it to the given classification task head.

[0120] S3-5: The classification module uses a multi-layer perceptron (MLP) head for classification. The MLP classification head includes layer normalization, two linear transformation layers, a Gaussian Error Linear Unit (GELU) activation function, and a random dropout layer. It obtains a score for each category and outputs the category with the highest probability as the top gas flow state, thereby obtaining the blast furnace top gas flow state recognition result.

[0121] F(c [cls] ,f) Input classification module, mapped to the score of 6 types of blast furnace top gas flow status through MLP classification head. In this embodiment, the center gas flow and edge gas flow are marked according to their intensity (i.e. the range and size of gas flow visible in the image), and are divided into, for example Figure 3 The target classification in this embodiment is based on the different states of moderate center, strong center, weak edge, moderate edge, and strong edge. Figure 3 For example, it can be divided into 6 categories: "moderate center, weak edges", "moderate center, moderate edges", "moderate center, strong edges", "strong center, weak edges", "strong center, moderate edges", and "strong center, strong edges".

[0122] S4: using the training set to train the blast furnace top coal gas flow state recognition model, and using the test set to test the blast furnace top coal gas flow state recognition model.

[0123] When training the blast furnace top gas flow state recognition model, the total loss function constructed is:

[0124] L=L CE +γ·L align +δ·L TC ,

[0125] Among them, L represents the total loss function, L CE is the cross entropy loss function, L align is the multi-feature alignment loss function constructed based on expert features, L TC is the temporal consistency loss function constructed based on expert features, and γ and δ are weight coefficients.

[0126] Considering that the chute calibration problem can be classified as a classification problem, the cross entropy function is first used to calculate the loss between the predicted value and the true value. CE The calculation method is:

[0127]

[0128] where y c represents the true label, p c Represents the probability that the sample predicted by the model belongs to category c.

[0129] Image features and expert features need to be as similar as possible in the latent space. In order to narrow the distance between the two features, the present invention adds a multi-feature alignment loss to improve the fusion effect. The calculation method of the multi-feature alignment loss function constructed based on the expert features is:

[0130]

[0131] Among them, f res (i) represents the temporal fusion feature corresponding to the infrared image of the furnace top in frame i, f hand (i) represents the expert feature corresponding to the i-th frame of the furnace top infrared image, ||||2 represents the two-norm, and N is the number of frames of the furnace top infrared image.

[0132] As the top gas flow video frames and expert features change, sudden changes may occur in the fused data at two adjacent time points. Therefore, it is necessary to constrain the smoothness of the model in time series to avoid drastic fluctuations. Therefore, the calculation method of the temporal consistency loss function constructed based on the expert features is:

[0133]

[0134] Among them, KL() represents the calculation divergence, which is used to measure the difference in the predicted distribution of adjacent time points, p i,t Represents the predicted probability distribution of the i-th sample in the expanded sample set at time point t.

[0135] S5: Construct a time series fusion sample corresponding to the blast furnace top infrared video to be identified and input it into the trained blast furnace top gas flow state recognition model to obtain the recognition result of the blast furnace top gas flow state.

[0136] The present invention also discloses a blast furnace top gas flow state recognition system based on deep learning of time-series images, comprising a data acquisition module, a sample processing module, a recognition model construction module, a training module, and a recognition module. The data acquisition module is used to acquire a blast furnace top infrared video and split it into top infrared images by frame. The sample processing module is used to perform expert feature labeling on each frame of the top infrared image to obtain a time-series fusion sample, perform window sampling and random enhancement processing on the time-series fusion sample to obtain an expanded sample set, and divide the expanded sample set into a training set and a test set. The recognition model construction module is used to construct a blast furnace top gas flow state recognition model. The training module is used to train the blast furnace top gas flow state recognition model using the training set and test the blast furnace top gas flow state recognition model using the test set. The recognition module is used to construct a time-series fusion sample corresponding to the blast furnace top infrared video to be recognized and input it into the trained blast furnace top gas flow state recognition model to obtain a recognition result of the blast furnace top gas flow state.

[0137] The present invention also discloses a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, a method for identifying the state of gas flow at the top of a blast furnace based on deep learning of time-series images is implemented.

[0138] The present invention also discloses a device, including a memory, a processor, and a computer program stored in the memory and runnable on the processor. When the processor executes the computer program, a method for identifying the state of gas flow at the top of a blast furnace based on deep learning of time-series images is implemented.

[0139] This invention comprehensively considers the various characteristics of coal gas flow in infrared video of blast furnace tops, fusing image features with expert features to enrich feature information. Furthermore, by combining temporal features and position encoding, the model understands the corresponding positional relationships between frames and learns global information. This enables real-time and accurate recognition of coal gas flow status at the top of the furnace. Compared with existing technologies, this invention has the following advantages:

[0140] (1) For the first time, the task of identifying the state of the furnace top gas flow is defined as a time series classification problem, in which the state is evaluated based on the key information of multiple consecutive time points, and the states of the central gas flow and the edge gas flow are identified simultaneously, thus achieving real-time performance while improving the recognition accuracy.

[0141] (2) To address the problem of limited data, this paper selects a specific data augmentation algorithm to generate diverse training data. In addition, it designs an effective time window sampling method to further increase the number of samples. This can improve the robustness of the model in different shooting environments and enhance recognition accuracy.

[0142] (3) The blast furnace top coal gas flow state recognition model in this paper fuses image features and corresponding expert features to construct time series fusion features, realizes the reuse of existing key features, provides richer feature information for the blast furnace top coal gas flow state recognition task, and improves the recognition accuracy of the model.

[0143] (4) Learnable positional encoding information is introduced and a special classification marker is added at the beginning of the feature sequence. The global information of the entire sequence is learned through the Longformer-based attention enhancement module, which further improves the recognition accuracy of the model.

[0144] The present invention identifies the infrared video of the blast furnace top obtained before charging, and comprehensively considers various characteristics of the coal gas flow in the infrared video of the blast furnace top to judge the real-time coal gas flow state, thereby providing data support for real-time monitoring of the blast furnace condition and providing important guarantee for the stable operation of the blast furnace.

[0145] To further demonstrate the advantages of the present invention, this example collects a video of the blast furnace before the charge distribution without chute obstruction, and calculates the corresponding expert features frame by frame. Through data enhancement and time window sampling, a total of 4200 time series fusion samples are obtained. The dataset is divided into training and test sets in an 8:2 ratio, and the training and test sets are ensured to be from different months. The top gas flow recognition model is built using the PyTorch framework, and the initial learning rate is set to 10 -3 The Adam optimizer was used for learning rate adjustment. The model was trained for 50 epochs with a batch size of 32. Simulation experiments were performed on a server equipped with an NVIDIA V100-SXM2 16GB GPU.

[0146] Accuracy (ACC), precision (P), recall (R), weighted F1 score (Weighted F1), and Kappa coefficient (Kappa) are used as evaluation indicators. The calculation method of each indicator is as follows:

[0147]

[0148] Among them, TP, TN, FP, and FN are the numbers of true positives, true negatives, false positives, and false negatives, respectively.

[0149] To alleviate the problem of unbalanced sample size across categories, this embodiment uses the Weighted F1 score as the evaluation metric, and its formula is as follows:

[0150]

[0151] Among them, n i is the number of samples in the i-th category, w iis the proportion of the number of samples in the i-th category to the total number of samples, F1 i represents the F1 score of the i-th category, and C is the number of categories.

[0152] In addition, the Kappa coefficient is used in the following equation to observe the degree of consistency between the model prediction results and the actual classification results:

[0153]

[0154] Among them, p0 is the sum of the number of correctly classified samples in each category divided by the total number of samples, that is, the overall classification accuracy. e The calculation method is:

[0155]

[0156] Among them, a i represents the number of samples in the i-th category, b i It represents the number of samples predicted to be of class i, and N is the total number of samples.

[0157] Blast furnace top gas flow status was identified using the proposed model, a ResNet and RNN combination model (Res-RNN), a ResNet and GRU combination model (Res-GRU), a ResNet and LSTM combination model (Res-LSTM), and a ResNet, LSTM, and Longformer combination model (Res-LSTM-Longformer). For fairness, all methods used the time series fusion feature proposed in this paper. The experimental results are shown in Table 1.

[0158] Table 1 Comparison of blast furnace top gas flow state recognition results of different models

[0159] Models ACC (%) P(％) R(％) Weight F1(%) Kappa (%) Res-RNN 90.95 89.34 88.64 90.98 88.90 Res-GRU 90.12 86.32 85.63 89.10 87.89 Res-LSTM 94.17 92.66 92.26 94.11 92.85 Res-LSTM-Longformer 92.74 91.71 89.58 92.57 91.08 The present invention 95.83 94.84 94.30 95.82 94.89

[0160] As can be seen from Table 1, the performance of the RNN-based model is significantly lower, which may be related to the gradient vanishing and gradient exploding problems of RNN. The performance of the GRU-based model is far inferior to that of the LSTM-based model, with an accuracy difference of about 4%. This may be related to the relatively simple structure of the GRU and its limitations in processing longer sequences. For the models based on Res-LSTM-Longformer and Res-BiLSTM-Longformer, the BiLSTM module of the latter makes it easier for the model to achieve comprehensive contextual understanding, with a performance improvement of about 3%. This is because BiLSTM can simultaneously use past and future information for prediction, which is more conducive to the Longformer capturing comprehensive contextual information when processing long sequences. The present invention has demonstrated excellent performance in all evaluation indicators, indicating that the method of the present invention is more suitable for the task to be solved by the present invention and can accurately diagnose the status of the furnace top gas flow.

[0161] In addition, in order to evaluate the performance of each module, this embodiment conducted an ablation experiment on each module in the model of the present invention, including the temporal fusion feature fusion module (expert feature), the temporal feature extraction module (BiLSTM), the position encoding module (position encoding) and the attention enhancement module (Longformer). The results are shown in Table 2.

[0162] Table 2 Comparison of ablation test results of various modules of the present invention

[0163]

[0164] Table 2 shows that expert features play the most important role. When the time series fusion feature fusion module is removed, performance significantly degrades, reaching only 91.79% accuracy. Furthermore, Longformer achieves higher performance than BiLSTM. For the same accuracy, Longformer achieves slightly higher precision, recall, weighted F1, and Kappa. The introduction of positional encoding improves performance across various metrics, with precision increasing by approximately 4%. This indicates that positional encoding helps the model learn positional relationships between sequences, enhancing the network's representational capabilities.

[0165] The integration of all the above modules significantly enhances the expressive power of the model, achieving an accuracy of 95.83%, a precision of 94.84%, a recall rate of 94.30%, a Weighted F1 score of 95.82% and a Kappa score of 94.89%. These positive results strongly verify the effectiveness of the present invention in the task of identifying the gas flow status of blast furnace tops.

[0166] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0167] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0168] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0169] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0170] Obviously, the above embodiments are merely examples for clarity of explanation and are not intended to limit the implementation methods. Those skilled in the art will appreciate that other variations or modifications can be made based on the above description. It is not necessary and impossible to enumerate all implementation methods here. Obvious variations or modifications arising therefrom remain within the scope of protection of the present invention.

Claims

1. A method for identifying the gas flow state of a blast furnace top based on deep learning of time series images, characterized in that: include: Acquire an infrared video of a blast furnace top and split it into infrared images by frame. Perform expert feature labeling on each frame of the infrared image to obtain a time-series fusion sample. Perform window sampling and random enhancement processing on the time-series fusion sample to obtain an expanded sample set. Construct a blast furnace top coal gas flow state recognition model, the blast furnace top coal gas flow state recognition model includes a time series fusion feature fusion module, a time feature extraction module, a position coding module, an attention enhancement module, and a classification module, the time series fusion feature fusion module extracts the spatial features of the furnace top infrared image and fuses them with the expert features to obtain a time series fusion feature, the time feature extraction module extracts the time features of the time series fusion feature, the position coding module fuses the time feature with the position of the furnace top infrared image in the original blast furnace top infrared video to obtain a coding feature, the attention enhancement module integrates global information according to the coding feature to obtain a global feature, and the classification module performs category prediction according to the global feature; The expanded sample set is used to train the blast furnace top coal gas flow state recognition model, and a time series fusion sample corresponding to the blast furnace top infrared video to be identified is constructed and input into the trained blast furnace top coal gas flow state recognition model to obtain the recognition result of the blast furnace top coal gas flow state.

2. The method for recognizing blast furnace top gas flow state based on time series image deep learning according to claim 1, characterized in that: The expert feature marking of each frame of furnace roof infrared image is performed to obtain a time series fusion sample, specifically: Calculate the expert features of each frame of furnace top infrared image, including central coal flow area, central coal flow brightness, central coke package area, central coke package brightness, central coal flow inclination, and edge coal flow brightness. The method for calculating the central coal gas flow area is as follows: using a threshold segmentation method to segment the furnace top infrared image to obtain a central coal gas flow area, and calculating the area of ​​the central coal gas flow area to obtain the central coal gas flow area; The calculation method of the central coal gas flow brightness is as follows: calculating the average brightness of the central coal gas flow area to obtain the central coal gas flow brightness; The central coke package area is calculated by performing a quadratic ellipse fitting on the central coal gas flow region to obtain a central coke package area, and calculating the area of ​​the central coke package area to obtain the central coke package area; The calculation method of the central focal package brightness is as follows: calculating the average brightness of the central focal package area to obtain the central focal package brightness; The central gas flow inclination is calculated by performing weighted integration of the grayscale values ​​of the image pixels in the central gas flow area using geometric moments to obtain the central gas flow inclination. The method for calculating the edge gas flow brightness is as follows: calculating the average brightness of the remaining area after removing the central gas flow area to obtain the edge gas flow brightness; The calculated expert features are normalized and matched with each frame of furnace top infrared image to obtain time series fusion samples.

3. The method for recognizing blast furnace top gas flow state based on time series image deep learning according to claim 1, characterized in that: The time series fusion samples are subjected to window sampling and random enhancement processing to obtain an expanded sample set, specifically: The number of samples in the i-th category is increased by M using the following formula i Times: Among them, N is the number of frames of the infrared video of the blast furnace top, L is the size of the time window, and S is the step size. represents the floor function, W i Represents the sampling weight of each category, W i The calculation method is: Among them, Max(Class) represents the number of classes with the largest number of samples among all categories, and Class i represents the original number of samples in category i; After sampling the time window, M is improved i The number of samples is increased to 2M. i times; Increase the number of samples to 2M i The samples after multiplication are randomly enhanced to obtain the expanded sample set.

4. The method for recognizing blast furnace top gas flow status based on time series image deep learning according to claim 3 is characterized in that: The calculation method of the random enhancement is: Among them, T() represents the random enhancement operation, I(x,y) represents the pixel value of the original sample image at the coordinate (x,y) before random enhancement, α represents the enhancement contrast adjustment factor, β represents the brightness adjustment factor, and λ represents the sharpening intensity. stands for Laplace operator.

5. The method for recognizing blast furnace top gas flow status based on time series image deep learning according to claim 1, characterized in that: The temporal fusion feature fusion module extracts the spatial features of the furnace top infrared image and fuses them with the expert features, specifically: The ResNet-18 model is used to extract the spatial features of the furnace top infrared images in the expanded sample set. The spatial features are projected onto a vector of preset length through a fully connected layer. The projected spatial features and expert features are horizontally spliced ​​to obtain temporal fusion features.

6. The method for recognizing blast furnace top gas flow status based on time series image deep learning according to claim 1, characterized in that: The position coding module fuses the time feature with the position of the furnace top infrared image in the original blast furnace top infrared video to obtain a coding feature, specifically: The position code is calculated based on the temporal position of the furnace top infrared image in the expanded sample set in the original blast furnace top infrared video, and the coding feature is obtained by fusing the time feature and the position code: f=[f1,f2,...,f i ,...,f t ], Wherein, f is the coding feature, f i is the encoding feature of the furnace top infrared image at time i, and t is the frame number of the furnace top infrared image in the original blast furnace top infrared video; f i The calculation method is: f i =y i +p i , Among them, y i is the time feature of the furnace top infrared image at time i, p i is the position code of the furnace top infrared image at time i.

7. The method for recognizing blast furnace top gas flow status based on time series image deep learning according to claim 1, characterized in that: The attention enhancement module integrates global information according to the coding features and obtains global features. Specifically, the classification mark is set, and attention calculation is performed in combination with the coding features to obtain the global features: Among them, F(c [cls] ,f) is the global feature, Q [cls] is the Query vector, K is the Key matrix, V is the Value matrix, d is the vector dimension, and T is the transpose operation; The calculation method of query vector is: Q [cls] =c [cls] ·W Q , Among them, c [cls] is the classification mark, W Q is the weight matrix of the Query vector; The calculation method of Key matrix and Value matrix is: K=f·W K , V=f·W V ; Among them, f is the encoding feature, W K is the weight matrix of the Key matrix, W V is the weight matrix of the Value matrix.

8. The method for recognizing blast furnace top gas flow status based on time series image deep learning according to any one of claims 1 to 7, characterized in that: When training the blast furnace top gas flow state recognition model, the total loss function constructed is: L=L CE +γ·L align +δ·L TC , Among them, L represents the total loss function, L CE is the cross entropy loss function, L align is the multi-feature alignment loss function constructed based on expert features, L TC is the temporal consistency loss function constructed based on expert features, and γ and δ are weight coefficients.

9. The method for recognizing blast furnace top gas flow status based on time series image deep learning according to claim 8, characterized in that: The calculation method of the multi-feature alignment loss function constructed based on the expert features is: Among them, f res (i) represents the temporal fusion feature corresponding to the infrared image of the furnace top in frame i, f hand (i) represents the expert feature corresponding to the i-th frame of the furnace top infrared image, ||||2 represents the two-norm, and N is the number of frames of the furnace top infrared image.

10. The method for recognizing blast furnace top gas flow status based on time series image deep learning according to claim 8, characterized in that: The calculation method of the temporal consistency loss function constructed based on the expert features is: Among them, KL() represents the calculation divergence, p i,t Represents the predicted probability distribution of the i-th sample in the expanded sample set at time point t.

Images