Converter steelmaking endpoint carbon temperature prediction method based on multi-modal characteristics of furnace mouth flame
By integrating the feature extraction framework and channel attention mechanism of ResNet and GLCM, the problems of insufficient feature extraction and weak anti-interference ability in the prediction of the final carbon temperature in converter steelmaking are solved, and high-precision and robust prediction of the final carbon temperature is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- UNIV OF SCI & TECH BEIJING
- Filing Date
- 2026-02-18
- Publication Date
- 2026-06-16
AI Technical Summary
Existing methods for predicting the final carbon temperature in converter steelmaking rely on a single feature source, which fails to adequately characterize the texture and statistical information of flame images and is sensitive to interference information, making it difficult to maintain high accuracy and robustness in complex industrial environments.
A deep fusion residual network (ResNet) and gray-level co-occurrence matrix (GLCM) feature extraction framework is adopted, combined with channel attention mechanism, to fuse the deep visual features and texture features of flame images, and to establish the mapping relationship between flame features and final carbon temperature through LSTM network for temporal modeling.
It significantly improves the accuracy and robustness of endpoint carbon temperature prediction, maintains high precision and stability in complex industrial environments, and enhances the model's adaptability to different working conditions.
Smart Images
Figure CN122220737A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of converter steelmaking technology, and in particular to a method for predicting the final carbon temperature of converter steelmaking based on the multimodal characteristics of the furnace mouth flame. Background Technology
[0002] Converter steelmaking is one of the most important steelmaking processes in modern steel production. Its core objective is to remove impurities such as carbon, silicon, and phosphorus from molten iron in a short period of time to obtain steel that meets quality requirements. The carbon content and temperature at the final stage of molten steel production are key indicators for evaluating the quality of converter steelmaking, directly affecting the mechanical properties of the steel, the stability of subsequent continuous casting processes, and the level of production cost control.
[0003] From the perspective of steel product quality, even slight differences in carbon content can significantly affect the performance of steel. When the carbon content is too high, the plasticity and toughness of the steel decrease, leading to problems such as brittle fracture during processing and use. Conversely, if the carbon content is too low, the steel's strength and hardness requirements cannot be met, affecting its practical application. Improper control of molten steel temperature can lead to abnormal solidification processes, resulting in internal defects and reducing the quality stability and reliability of the steel. In terms of production efficiency and cost, accurate control of the final parameters of molten steel can reduce unnecessary smelting time and energy consumption, improve converter production efficiency, and lower production costs.
[0004] Early predictions of the final carbon content and molten steel temperature in converter steelmaking relied primarily on operator experience. The final state was estimated by observing the furnace flame and flue gas conditions, combined with information such as blowing time and oxygen supply. This method was highly dependent on operators, subjective, and ill-suited to the stability and precision requirements of modern steel production. Subsequently, mechanistic models based on material and energy conservation were gradually introduced into converter final state prediction. These models calculated the final carbon temperature by establishing carbon-oxygen reaction kinetics and heat balance equations. However, due to the complexity of the internal reaction mechanisms in the converter and the difficulty in accurately obtaining parameters, the prediction accuracy of these mechanistic models significantly decreased under large fluctuations in operating conditions, limiting their application in actual production.
[0005] With the continuous improvement of industrial big data technology and computing power, some existing technologies have begun to adopt machine learning or deep learning methods to achieve endpoint prediction based on steelmaking process parameters or furnace flame images. Among them, research on furnace flame information has gradually attracted attention. Existing solutions acquire furnace flame images using camera equipment and extract features using image processing or convolutional neural networks to establish a mapping relationship between flame state and molten steel endpoint parameters. However, most existing methods rely mainly on a single feature source in the feature extraction stage, usually only using convolutional neural networks to extract high-level visual features of flame images, ignoring the important texture statistical information contained in the flame images. This results in insufficient depiction of flame combustion details and local structural changes, making it difficult to comprehensively reflect the smelting state inside the furnace.
[0006] Secondly, while some technologies attempt to incorporate multi-source features, they often employ simple splicing or fixed weighting methods during feature fusion, failing to differentiate the importance of different feature channels. This results in key features and redundant information being treated equally, making it difficult for the model to highlight crucial information highly correlated with the endpoint carbon content and temperature. In real industrial environments, furnace flame images are easily affected by factors such as flue gas, dust, slag splashes, and changes in lighting. Existing technologies lack effective feature selection and noise suppression mechanisms, making the models highly sensitive to interference and limiting the stability and robustness of prediction results.
[0007] Finally, the furnace flame in converter steelmaking exhibits significant temporal evolution characteristics. Its morphology, brightness, and texture continuously change with each blowing stage, closely related to the dynamic changes in molten steel temperature and carbon content. However, some existing flame image-based prediction methods primarily analyze single-frame or short-term features, failing to fully utilize the continuity and long-term dependencies of flame features over time. Even when time-series models are introduced, they are often limited by insufficient input feature quality, making it difficult to effectively capture the deep correlation between flame evolution patterns and the final state.
[0008] Therefore, integrating the multimodal characteristics of the furnace mouth flame and fully characterizing its temporal evolution has become an important issue in the field of converter steelmaking endpoint prediction. Summary of the Invention
[0009] To address the aforementioned problems, the present invention aims to provide a method for predicting the final carbon temperature of converter steelmaking based on the multimodal features of the furnace flame. By constructing a feature extraction framework of deep fusion residual network (ResNet) and gray-level co-occurrence matrix (GLCM), and introducing a channel attention mechanism to achieve adaptive feature weighting, this method overcomes the problems of insufficient feature extraction, inadequate utilization of multi-source information, lack of prominence of key features, and weak anti-interference ability in the prior art, thereby improving the accuracy of predicting the final carbon content and molten steel temperature in converter steelmaking.
[0010] To solve the above-mentioned technical problems, the present invention provides the following technical solution: On the one hand, a method for predicting the final carbon temperature of converter steelmaking based on the multimodal characteristics of the furnace mouth flame is provided, the method comprising the following steps: S1. Collect raw data during the converter steelmaking process, perform data cleaning and feature labeling preprocessing, and align the furnace flame image with the steel temperature and carbon content labels to obtain the preprocessed data. S2. Perform data augmentation on the preprocessed data, including geometric transformation, brightness and contrast adjustment, and Gaussian smoothing. S3. Build a steelmaking endpoint carbon temperature prediction model based on ResNet-LSTM. Use ResNet network to extract the depth feature vector of the furnace flame image, use GLCM method to extract the texture feature vector of the furnace flame image, and introduce channel attention mechanism for fusion. Input the fused features into LSTM network to model the temporal evolution law. S4. Use the data processed in steps S1 and S2 to train and validate the prediction model, and save the optimal model parameters. S5. Use the trained prediction model to perform inference and predict the final steel temperature and carbon content of the converter steelmaking process.
[0011] Optionally, step S1 specifically includes: S11. Data Cleaning: First, the original furnace flame video is processed by frame extraction to obtain a discrete furnace flame image sequence. Second, the furnace flame image is denoised and smoothed using median filtering or Gaussian filtering. Finally, the furnace flame image sequence is processed using the inter-frame difference method to calculate the grayscale difference of corresponding pixels to characterize the intensity of flame change in the time dimension, thereby extracting dynamic change information of the flame. S12. Feature Labeling and Alignment: Using the furnace batch as the basic labeling unit, the furnace flame video is matched with the corresponding steelmaking production time information to form a flame image sequence with a temporal relationship for each furnace batch; at the same time, using the furnace number as an index, the furnace flame image sequence collected for each furnace batch is uniformly labeled with the corresponding endpoint molten steel temperature and carbon content, and the molten steel temperature and carbon content serve as the supervision labels for the furnace batch sample.
[0012] Optionally, step S1 further includes: Multiple frames of furnace mouth flame images sampled in chronological order during each heat are combined into an image sequence and regarded as a time-series sample. Each frame in the sequence is assigned a corresponding time step index. The time-series sample corresponds to the endpoint molten steel temperature and carbon content label under the same heat number, so as to realize the mapping relationship between furnace mouth flame characteristics and endpoint carbon temperature, and provide normalized input for subsequent time series learning of LSTM network.
[0013] Optionally, step S2 specifically includes: S21. Use OpenCV's geometric transformation operators to perform random-scale horizontal flipping, vertical flipping, and small-angle rotation transformations on the furnace flame image. S22. Adjust the brightness and contrast of the furnace flame image, apply a linear transformation to the original pixel values to make the overall brightness and contrast of the image change randomly by a predetermined range, and then perform Gamma correction on the image after brightness and contrast adjustment. S23. Perform Gaussian smoothing on the furnace flame image to simulate the imaging blur caused by furnace flue gas or heat waves through Gaussian filtering.
[0014] Optionally, step S3 specifically includes: S31. Use a ResNet network to extract deep features from the furnace flame image, and obtain the furnace flame image at time [time value missing]. t Deep feature vectors The depth feature vector is used to characterize the overall shape, brightness distribution and high-level semantic information of the flame; S32. Based on the GLCM method, texture features are extracted from the furnace flame image to obtain the furnace flame image at time [time value missing]. t Texture feature vector The texture feature vector is used to characterize the texture roughness, energy distribution, and combustion stability of the flame; S33, Transfer the deep feature vector With texture feature vector The data is concatenated along the channel dimension, and a channel attention mechanism is introduced for weighted fusion to obtain the fused features. ; S34. Input the fused features into the LSTM network to model the temporal evolution of the multimodal features of the furnace mouth flame and establish the mapping relationship between the multimodal features of the furnace mouth flame and the carbon content and molten steel temperature at the end of converter steelmaking.
[0015] Optionally, step S4 specifically includes: S41. Construct training and validation sets using the data enhanced in step S2. Pre-train the ResNet-LSTM prediction model based on the training set. During the training process, iterate through different network parameter configurations and determine the initial optimal model parameters based on the prediction error of the validation set. S42. Based on step S41, the preprocessed data from step S1 and the enhanced data from step S2 are fused to obtain a training set and a validation set. The initial optimal model parameters saved in step S41 are used as initial values, and the ResNet-LSTM prediction model is further trained using the training set. During training, mean squared error is used as the loss function to calculate the error between the predicted values of molten steel temperature and carbon content at the end of the model output and the corresponding measured values. The model parameters are iteratively updated through the backpropagation algorithm. At the same time, the model performance is evaluated based on the validation set, and the model parameters with the best prediction performance during the validation process are retained as the optimal model parameters.
[0016] Optionally, in step S4, the ResNet network adopts a multi-layer residual network structure, and the number of convolutional channels and the output feature dimension in the residual block are adjusted according to the feature extraction effect; the number of hidden layers and the number of hidden units in the LSTM network are set according to the fusion feature dimension, and the relevant parameters are adaptively adjusted through validation set error feedback.
[0017] Optionally, step S5 specifically includes: S51. Preprocess the input test data, the same as step S1; S52. Load the trained prediction model from the specified path, input the preprocessed test data into the loaded prediction model, and the prediction model infers based on the learned optimal model parameters to calculate the predicted values of the final steel temperature and carbon content. S53. Obtain the true values of the test data, compare them with the model prediction results, calculate the prediction hit rate of the model for the final steel temperature and carbon content, and analyze the performance of the model based on the prediction hit rate.
[0018] On the other hand, an electronic device is provided, the electronic device comprising: processor; The memory stores computer-readable instructions, which, when loaded and executed by the processor, implement the steps of the converter steelmaking endpoint carbon temperature prediction method based on the multimodal characteristics of furnace mouth flames described above.
[0019] On the other hand, a computer-readable storage medium is provided, wherein program code is stored in the computer-readable storage medium, and the program code can be called by a processor to execute the steps of the converter steelmaking endpoint carbon temperature prediction method based on the multimodal characteristics of furnace mouth flame described above.
[0020] The beneficial effects of the technical solution provided by this invention include at least the following: (1) This invention introduces a controllable data augmentation technique based on OpenCV in the data preprocessing stage. Through controllable geometric transformations, brightness and contrast adjustments, Gamma correction, and noise and fuzzy simulation, it effectively expands the sample diversity of flame images in terms of illumination, spatial morphology, and noise distribution without destroying the intrinsic physical characteristics of the flame and the semantic information of the process. This method can significantly alleviate the model's dependence on specific working conditions under small sample conditions, improve the model's adaptability to different furnace batches, different blowing stages, and complex industrial environments, thereby enhancing the robustness and generalization performance of the endpoint carbon temperature prediction model in actual production applications.
[0021] (2) In the data preprocessing stage, this invention effectively filters out high-quality flame images through frame extraction, median or Gaussian filtering for noise reduction, and inter-frame difference temporal cleaning. In the feature fusion stage, ResNet deep visual features and GLCM texture statistical features are fused to construct a multi-level, multi-modal flame feature representation, which comprehensively characterizes the flame morphology, brightness distribution and combustion state changes, providing a reliable feature basis for the final carbon temperature prediction.
[0022] (3) This invention constructs a ResNet-LSTM hybrid model and introduces a channel attention mechanism, combining depth visual features and texture statistical features to achieve dynamic modeling of the temporal sequence of furnace flames. First, the flame depth features extracted by ResNet and the GLCM texture features are concatenated in the channel dimension. Then, the channel attention module adaptively learns the correlation between the two types of features, enhancing the sensitivity to key texture channels and high-temperature regions, thereby improving the accuracy and robustness of the final carbon temperature prediction.
[0023] (4) This invention uses an LSTM network to perform time-series modeling on the fused multimodal feature sequences, effectively capturing the dynamic evolution of flame features during the blowing process. Combined with feature weighted input using an attention mechanism, the model can more accurately establish the nonlinear mapping relationship between flame time-series features and the final carbon temperature, significantly improving the accuracy and time-series consistency of the predicted results of the final carbon content and molten steel temperature. Attached Figure Description
[0024] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0025] Figure 1 This is a flowchart of the overall process for predicting the final carbon temperature of converter steelmaking based on the multimodal characteristics of the furnace mouth flame provided in this embodiment of the invention. Figure 2 This is a schematic diagram of the overall structure of the converter steelmaking endpoint carbon temperature prediction method based on the multimodal characteristics of the furnace mouth flame provided in this embodiment of the invention. Figure 3 This is a data preprocessing flowchart provided in an embodiment of the present invention; Figure 4 This is a flowchart of the adaptive threshold filtering process provided in an embodiment of the present invention; Figure 5 This is a data augmentation flowchart provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of multimodal feature fusion provided in an embodiment of the present invention. Detailed Implementation
[0026] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the described embodiments of the present invention without creative effort are within the scope of protection of the present invention.
[0027] This invention provides a method for predicting the final carbon temperature in converter steelmaking based on multimodal characteristics of the furnace mouth flame. This method can be implemented using electronic equipment, such as a terminal or server. (Reference) Figures 1-2 As shown, the processing flow of this method may include the following steps: S1. Collect raw data during the converter steelmaking process, perform data cleaning and feature labeling preprocessing, and align the furnace flame image with the steel temperature and carbon content labels to obtain the preprocessed data.
[0028] As an optional embodiment of the present invention, refer to Figure 3 As shown, step S1 specifically includes: S11. Data cleaning: First, the original furnace flame video is processed by frame extraction. The frame extraction sampling frequency can be set to about 10fps to obtain a discrete furnace flame image sequence. Secondly, median filtering or Gaussian filtering is used to denoise and smooth the furnace flame image. Median filtering is used to remove impulse noise caused by splashing sparks and abnormal bright spots in the flame image. Let the input grayscale image be... In pixels Select a size of neighborhood window The output image after median filtering Defined as expression (1), where Defined as expression (2).
[0029] (1) (2) in, This represents the median of the gray values of the pixels in the neighborhood. Gaussian filtering is used to smooth the brightness distribution of flame images and suppress random noise, thereby improving image quality while preserving key structural features of the flame. The Gaussian filter kernel is defined by a two-dimensional Gaussian function, expressed as equation (3).
[0030] (3)
[0031] in, The standard deviation of the Gaussian distribution controls the degree of smoothing. These are the coordinates relative to the center within the filtering window. The image after Gaussian filtering. Original image With Gaussian kernel The convolution of is expressed as equation (4).
[0032] (4)
[0033] Finally, the inter-frame difference method is used to process the furnace flame image sequence and calculate the gray-scale difference of corresponding pixels to characterize the intensity of flame change in the time dimension, thereby extracting the dynamic change information of the flame.
[0034] Let the two adjacent flame images be the first and second frames respectively. frame With the frame By calculating the grayscale difference between the two frames at the corresponding pixel positions, the inter-frame difference image is obtained, and its expression is Equation (5).
[0035] (5)
[0036] in, This characterizes the magnitude of flame brightness variation between adjacent frames. Based on the statistical characteristics of the difference images between adjacent frames, an adaptive threshold determination mechanism is constructed, as shown in the flowchart below. Figure 4 As shown. The threshold is adjusted in real time according to the dynamic characteristics of the flame. This applies to inter-frame difference images. Perform statistical analysis and calculate its global mean. and standard deviation Statistical analysis was performed, and the expressions are (6) and (7).
[0037] (6) (7) An adaptive threshold is constructed based on statistical features, expressed as equation (8): (8) in, k This is a threshold adjustment coefficient used to balance the sensitivity and robustness of flame change detection, and its preferred value range is 1.5~3. The adaptive threshold... Applied to inter-frame difference images, the binary determination result of the flame change region is obtained, expressed as Equation (9).
[0038] (9)
[0039] in, This indicates the area of significant change in the flame at the furnace opening.
[0040] S12. Feature Labeling and Alignment: After cleaning and outlier processing of the furnace flame image data, feature alignment is performed on the furnace flame image data collected during the converter steelmaking process. The dataset is divided into training, validation, and test sets according to the furnace batch, with a ratio of 7:2:1.
[0041] Using the furnace batch as the basic labeling unit, the furnace flame video is matched with the corresponding steelmaking production time information to form a flame image sequence with a time sequence for each furnace batch. At the same time, using the furnace number as an index, the furnace flame image sequence collected for each furnace batch is uniformly labeled with the corresponding endpoint molten steel temperature and carbon content. The molten steel temperature and carbon content serve as the monitoring labels for the sample of that furnace batch.
[0042] For flame images within the same furnace, the parameters for GLCM texture feature extraction are uniformly set during the feature extraction stage, including gray level, pixel spacing, and orientation angle, to ensure the consistency and comparability of texture feature extraction processes across different furnaces and time steps.
[0043] Furthermore, step S1 also includes: Multiple frames of furnace mouth flame images sampled sequentially during each heat's blowing process are compiled into an image sequence and considered as a time-series sample. A corresponding time step index is assigned to each frame in the sequence to describe the evolution of flame features throughout the blowing process, thus constructing a fixed-length time-series input structure. The time-series sample corresponds to the endpoint molten steel temperature and carbon content labels for the same heat number, enabling modeling of the mapping relationship between furnace mouth flame features and endpoint carbon temperature, and providing standardized input for subsequent time-series learning by the LSTM network.
[0044] S2. Perform data augmentation on the preprocessed data, including geometric transformation, brightness and contrast adjustment, and Gaussian smoothing.
[0045] As an optional embodiment of the present invention, refer to Figure 5 As shown, step S2 specifically includes: S21. Using OpenCV's geometric transformation operators, the furnace flame image is subjected to random-proportion horizontal flipping, vertical flipping, and small-angle rotation transformation. This process uses the cv2.flip and cv2.getRotationMatrix2D functions to remap the image space, enhancing the model's robustness to changes in the converter's geometric position and ensuring accurate locking of the flame core region from different visual angles.
[0046] S22. Adjust the brightness and contrast of the furnace flame image, and adjust the original pixel values. A linear transformation is applied to randomly vary the overall brightness and contrast of the image by a predetermined amount, resulting in enhanced pixel values. The expression is Equation (10), and the processing can be achieved by calling the cv2.convertScaleAbs function.
[0047] (10)
[0048] in, Contrast coefficient, This represents the brightness offset.
[0049] Then, Gamma correction is performed on the image after brightness and contrast adjustment, and the mapping relationship is expressed as Equation (11). The radiation characteristics of the high-temperature area of the flame are enhanced by nonlinear brightness mapping, highlighting the high-temperature area of the flame, while maintaining the continuity of the overall brightness distribution of the flame. The processing can be implemented by the cv2.LUT() function, which maps the original pixel values to the Gamma correction table, thereby quickly completing the nonlinear brightness transformation.
[0050] (11)
[0051] in, The maximum value of the pixel. This is the Gamma correction parameter.
[0052] S23. Gaussian smoothing is applied to the furnace flame image to simulate the imaging blur caused by furnace smoke or heat waves through Gaussian filtering. The two-dimensional Gaussian kernel function is defined as Equation (3), and the expression for the Gaussian smoothing result of the image is Equation (4), where, This is used to control the blur intensity. This process can be implemented using the cv2.GaussianBlur() function.
[0053] To further enhance the model's robustness to random perturbations, mild random noise can be superimposed on the image matrix. This noise can be generated using NumPy and then added to the image matrix. The Gaussian random noise expression is given by equation (12). The cv2.normalize function is then used to ensure that the pixel values return to the legal range, guaranteeing that the enhanced image still retains the physical characteristics of the flame. The normalization expression is given by equation (13).
[0054] (12)
[0055] in, ) indicates that the mean is 0 and the variance is . Gaussian noise.
[0056] (13)
[0057] in, and These represent the maximum and minimum pixel values of the image, respectively.
[0058] S3. Construct a steelmaking endpoint carbon temperature prediction model based on ResNet-LSTM. Use ResNet network to extract the depth feature vector of the furnace flame image, use GLCM method to extract the texture feature vector of the furnace flame image, and introduce channel attention mechanism for fusion. Input the fused features into LSTM network to model the temporal evolution law.
[0059] As an optional embodiment of the present invention, refer to Figure 6 As shown, step S3 specifically includes: S31. Use a ResNet network to extract deep features from the furnace flame image, and obtain the furnace flame image at time [time value missing]. t Deep feature vectors The depth feature vector is used to characterize the overall shape, brightness distribution, and high-level semantic information of the flame.
[0060] Let the input image of the furnace flame be... After layer-by-layer convolution and residual mapping in the ResNet network, the th... The feature representation of the layer is expressed as equation (14).
[0061] (14)
[0062] in, This represents the residual mapping composed of convolution, batch normalization, and nonlinear activation functions. For the first Layer network parameters. After global average pooling (GAP), the flame image at time [time value missing] is obtained. The depth feature vector is expressed as Equation (15).
[0063] (15)
[0064] This deep feature vector is used to characterize the overall shape, brightness distribution, and high-level semantic information of the flame.
[0065] S32. Based on the GLCM method, texture features are extracted from the furnace flame image to obtain the furnace flame image at time [time value missing]. t Texture feature vector The texture feature vector is used to characterize the texture roughness, energy distribution, and combustion stability of the flame.
[0066] Let the gray level be N, the pixel spacing be d, and the direction be... Then GLCM is defined as Equation (16), energy is defined as Equation (17), contrast is defined as Equation (18), entropy is defined as Equation (19), and correlation is defined as Equation (20).
[0067] (16) (17) (18) (19) (20) The above statistics are combined to form a texture feature vector, which is expressed as equation (21): (twenty one) S33, Transfer the deep feature vector With texture feature vector The data is concatenated along the channel dimension, and a channel attention mechanism is introduced for weighted fusion to obtain the fused features. .
[0068] Specifically, the depth feature vector extracted by ResNet and the texture feature vector extracted by GLCM are concatenated in the channel dimension, as expressed in equation (22).
[0069] (twenty two)
[0070] A channel attention mechanism is introduced to perform global average pooling on the spliced features to obtain the channel description vector, which is expressed as Equation (23).
[0071] (twenty three)
[0072] Channel attention weights are obtained by using a two-layer fully connected network and introducing the Sigmoid function, as expressed in equation (24).
[0073] (twenty four)
[0074] in, It is the ReLU activation function. This is the Sigmoid function.
[0075] The attention weights are applied to the original spliced features to obtain the weighted fused features, which are expressed as Equation (25).
[0076] (25)
[0077] S34. Input the fused features into the LSTM network to model the temporal evolution of the multimodal features of the furnace mouth flame, and establish the mapping relationship between the multimodal features of the furnace mouth flame and the carbon content and molten steel temperature at the end of converter steelmaking, as expressed in Equation (26).
[0078] (26)
[0079] in, These are the predicted carbon content and molten steel temperature at the end of the converter steelmaking process, respectively. It is a weight matrix. It is a bias vector. It is the output feature vector of the LSTM network at time step T.
[0080] S4. Use the data processed in steps S1 and S2 to train and validate the prediction model, and save the optimal model parameters.
[0081] The furnace mouth flame image data and corresponding endpoint carbon temperature data collected during the converter steelmaking process were divided into training and validation sets. The network parameters of the ResNet-LSTM prediction model were set, with the ResNet network employing a multi-layer residual network structure, preferably a 50-layer residual network structure (ResNet-50). Based on the feature extraction performance, the number of convolutional channels and the output feature dimension in the residual blocks were adjusted. The number of hidden layers and hidden units in the LSTM network were set according to the fused feature dimension, and the relevant parameters were adaptively adjusted using validation set error feedback.
[0082] Regarding model training parameter configuration, the Adam optimization algorithm was selected to update the model parameters, with an initial learning rate set to 1×10⁻⁶. -4 The batch size is set to 8, and the maximum number of training epochs is set to 100. At the same time, a cosine annealing learning rate scheduling strategy is introduced to dynamically adjust the learning rate during training.
[0083] As an optional embodiment of the present invention, step S4 specifically includes: S41. Construct training and validation sets using the data enhanced in step S2. Pre-train the ResNet-LSTM prediction model based on the training set. During the training process, iterate through different network parameter configurations and determine the initial optimal model parameters based on the prediction error of the validation set.
[0084] S42. Based on step S41, the preprocessed data from step S1 and the enhanced data from step S2 are fused to obtain a training set and a validation set. The initial optimal model parameters saved in step S41 are used as initial values, and the ResNet-LSTM prediction model is further trained using the training set.
[0085] During the training process, mean squared error (MSE) is used as the loss function to calculate the error between the predicted values of the final steel temperature and carbon content output by the model and the corresponding measured values. The expression is given by equation (27): (27) in, Let N be the loss function, and N be the total number of steelmaking furnaces. For the first i The measured value of carbon content at the furnace endpoint. For the first i Predicted carbon content at the furnace endpoint No. i The measured value of the final temperature of molten steel in the furnace. No. i Predicted value of molten steel temperature at the furnace endpoint.
[0086] The model parameters are iteratively updated using the backpropagation algorithm, and the model performance is evaluated based on the validation set. When the validation error no longer decreases within a certain number of consecutive training rounds, an early stopping mechanism is triggered to terminate the training process prematurely. Finally, the model parameters with the best prediction performance during the validation process are retained as the optimal model parameters, resulting in the final trained prediction model, which is used in step S5 for inference prediction of the final steel temperature and carbon content in converter steelmaking.
[0087] S5. Use the trained prediction model to perform inference and predict the final steel temperature and carbon content of the converter steelmaking process.
[0088] As an optional embodiment of the present invention, step S5 specifically includes: S51. Preprocess the input test data, the same as step S1.
[0089] S52. Load the trained prediction model from the specified path, input the preprocessed test data into the loaded prediction model, and the prediction model infers based on the learned optimal model parameters to calculate the predicted values of the final steel temperature and carbon content.
[0090] S53. Obtain the true values of the test data, compare them with the model prediction results, calculate the prediction hit rate of the model for the final steel temperature and carbon content, and analyze the performance of the model based on the prediction hit rate.
[0091] In this embodiment of the invention, by constructing an extraction framework that deeply integrates ResNet visual features and GLCM texture features, and by introducing a channel attention mechanism to achieve adaptive feature weighting, the problems of insufficient feature extraction, insufficient utilization of multi-source information, lack of prominence of key features, and weak anti-interference ability in the prior art are overcome.
[0092] First, this invention introduces a deep residual network (ResNet) as the core visual feature extractor. By utilizing its deep structure and residual connections, it effectively extracts deep semantic features and micro-texture details in flame images that are closely related to the combustion state, providing richer and more discriminative visual input for time series prediction.
[0093] Secondly, this invention proposes a multimodal feature fusion mechanism. Based on the deep features extracted by ResNet, it simultaneously extracts texture statistical features (such as energy, contrast, entropy, etc.) based on GLCM, and achieves complementarity between visual and physical texture information through feature splicing, thereby enhancing the model's ability to comprehensively depict the flame state and improving its robustness under interference such as smoke and splashes.
[0094] Finally, this invention introduces a channel attention mechanism in the feature fusion stage, which automatically learns and assigns differentiated weights to different feature channels, enhancing the model's sensitivity to key flame regions and important texture dimensions, suppressing interference from irrelevant or noisy features, and using an LSTM network to perform temporal modeling on the weighted multimodal feature sequence to capture the dynamic evolution of flame features. This solves the problem of insufficient ability of existing methods to model long-term temporal dependencies, thereby optimizing the feature quality input to the temporal network and further improving the accuracy and stability of the final carbon temperature prediction.
[0095] In a more specific embodiment, the implementation process of the present invention is as follows: (1) Overall process The entire process can be divided into five steps: data preprocessing, data augmentation, model building, model training, and model inference.
[0096] (2) Prediction dataset of carbon temperature at the end point of converter steelmaking
[0097] This invention uses actual production data from the converter steelmaking process of a steel plant as the experimental dataset. The input features mainly include the amounts of raw materials, auxiliary materials, and gases added during the steelmaking process, as well as equipment information. To verify the effectiveness of the model, the final steel temperature and carbon content at the converter steelmaking endpoint were used as target variables for training.
[0098] (3) Data preprocessing
[0099] Denoising and smoothing can be achieved using median filtering or Gaussian filtering; The ResNet-LSTM model was used for training, and features were labeled according to the model metrics. The validation set error-driven adaptive parameter tuning was adopted.
[0100] (4) OpenCV Data Augmentation Details
[0101] The `flipCode` property of `cv2.flip` is randomly set to 0 or 1 to achieve vertical or horizontal flipping. The rotation angle of `cv2.getRotationMatrix2D` is randomly selected within the range of -5° to +5°. The scaling factor is set to 0.95 to 1.05, and the affine transformation is performed through `cv2.warpAffine`. The contrast coefficient α of `cv2.convertScaleAbs` is set to a value ranging from 0.8 to 1.2, the brightness offset β is set to a value ranging from -20 to 20, and the Gamma correction parameter γ is set to a value ranging from 0.8 to 1.2. Non-linear mapping is performed through `cv2.LUT`. The Gaussian kernel size of `cv2.GaussianBlur` is selected as (3,3) or (5,5), and the standard deviation is adaptively determined according to the kernel size. At the same time, Gaussian random noise with a mean of 0 and a variance of 5 to 15 is superimposed. After the noise is superimposed, the pixel values are normalized to the range of [0,255] through `cv2.normalize`.
[0102] (5) Model parameters
[0103] ①ResNet parameters
[0104] hidden_dim=512,glcm_dim=96,lstm_layers=2,output_dim=1,dropout_p=0.5
[0105] ②LSTM parameters
[0106] input_size=2048,hidden_size=hidden_dim,num_layers=lstm_layers,batch_first=True)
[0107] ③GLCM matrix parameters
[0108] GLCM_DISTANCES=[1]
[0109] GLCM_ANGLES=[0,np.pi / 4,np.pi / 2,3 np.pi / 4]
[0110] GLCM_FEATURES=['contrast','dissimilarity','homogeneity','ASM','energy','correlation']
[0111] (6) Experimental setup
[0112] The PyTorch 2.1.0 learning framework was used, with Adam as the optimizer and an initial learning rate of 1×10⁻⁶.-4 The batch size was 8, and the number of training epochs was 100. All experiments were conducted on a server with 4 NVIDIA RTX 3090 GPUs (24GB VRAM). The inference process is described in the above method implementation examples and will not be repeated here.
[0113] Compared with existing technologies, the present invention has the following advantages: (1) In the problem of predicting the endpoint of converter steelmaking, the core issue is how to improve the generalization ability of the model and avoid overfitting in the context of imbalanced data and high noise. The current mainstream approach is to directly perform routine cleaning on the original dataset and establish a single prediction model based on machine learning to predict the carbon content and molten steel temperature at the converter endpoint. This type of method can achieve good results when the data is sufficient and evenly distributed. However, in the process of steelmaking, the furnace flame image data is significantly affected by factors such as fluctuations in operating conditions, smoke obstruction, changes in illumination, and differences in equipment. The cost of obtaining high-quality samples is high and the sample distribution is uneven, which makes the traditional method of directly modeling based on the original data prone to problems such as overfitting and insufficient generalization ability.
[0114] This invention introduces a controllable data augmentation technique based on OpenCV in the data preprocessing stage. Through controllable geometric transformations, brightness and contrast adjustments, gamma correction, and noise and fuzzy simulation, it effectively expands the sample diversity of flame images in terms of illumination, spatial morphology, and noise distribution without destroying the intrinsic physical characteristics of the flame and the semantic information of the process. This method can significantly alleviate the model's dependence on specific operating conditions under small sample conditions, improve the model's adaptability to different furnace runs, different blowing stages, and complex industrial environments, thereby enhancing the robustness and generalization performance of the endpoint carbon temperature prediction model in practical production applications.
[0115] (2) Due to the common problems of invalid samples such as blurriness, obstruction and small flame area in the furnace flame video collected in the industrial field during converter steelmaking, traditional data preprocessing methods mostly rely on simple correlation analysis, which cannot accurately identify abnormal data and capture the relationship between complex high-dimensional features.
[0116] In the data preprocessing stage, this invention effectively filters high-quality flame images through frame extraction, median or Gaussian filtering for noise reduction, and inter-frame difference temporal cleaning. In the feature fusion stage, ResNet deep visual features and GLCM texture statistical features are fused to construct a multi-level, multi-modal flame feature representation, comprehensively characterizing flame morphology, brightness distribution, and combustion state changes, providing a reliable feature basis for endpoint carbon temperature prediction.
[0117] (3) A "spatial-texture" dual-path parallel feature extraction structure was designed. One path uses a ResNet deep convolutional network to automatically extract deep semantic features (such as morphology and brightness distribution) from the furnace flame image; the other path uses the GLCM method to extract texture statistical features (such as contrast, energy, and entropy) from the flame image. The key is that the two types of heterogeneous features are not simply concatenated. Instead, the visual features extracted by ResNet and the GLCM texture features are first concatenated in the channel dimension. Then, the correlation between the two types of features is adaptively learned through the channel attention module, which enhances the sensitivity to key texture channels and high-temperature regions, thereby improving the accuracy and robustness of the final carbon temperature prediction.
[0118] (4) A ResNet-LSTM hybrid model was constructed as the core predictor. The ResNet module was used to process the fused features of each frame and output high-dimensional spatial features. The spatial feature sequence of consecutive frames was input into the LSTM network. The LSTM network was used to perform temporal modeling on the fused multimodal feature sequence, effectively capturing the dynamic evolution of flame features during the blowing process. Finally, the temporal hidden state was mapped to the specific values of the endpoint carbon content and temperature through a fully connected regression layer. The model can more accurately establish the nonlinear mapping relationship between the flame temporal features and the endpoint carbon temperature, significantly improving the accuracy and temporal consistency of the prediction results of the endpoint carbon content and molten steel temperature.
[0119] In an exemplary embodiment, the present invention also provides an electronic device, the electronic device comprising: processor; The memory stores computer-readable instructions, which, when loaded and executed by the processor, implement the steps of the converter steelmaking endpoint carbon temperature prediction method based on the multimodal characteristics of furnace mouth flames described above.
[0120] In an exemplary embodiment, the present invention also provides a computer-readable storage medium storing at least one instruction, which is loaded and executed by a processor to implement the steps of the converter steelmaking endpoint carbon temperature prediction method based on the multimodal characteristics of furnace flame described above. For example, the computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, or optical data storage device, etc.
[0121] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Unless otherwise specified, an element defined by the phrase "comprising..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.
[0122] The use of terms such as "an embodiment," "an embodiment," "an exemplary embodiment," and "some embodiments" in the specification indicates that the described embodiment may include a specific feature, structure, or characteristic, but not every embodiment necessarily includes that specific feature, structure, or characteristic. Furthermore, when a specific feature, structure, or characteristic is described in connection with an embodiment, implementing such a feature, structure, or characteristic in conjunction with other embodiments (whether explicitly described or not) should be within the knowledge of those skilled in the art.
[0123] It should be understood that, in various embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.
[0124] In the embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.
[0125] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0126] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.
[0127] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0128] This invention encompasses any substitutions, modifications, equivalent methods, and solutions made within the spirit and scope of this invention. To provide the public with a thorough understanding of this invention, specific details are described in detail in the following preferred embodiments; however, those skilled in the art will fully understand the invention even without these details. Furthermore, to avoid unnecessary misunderstanding of the essence of this invention, well-known methods, processes, procedures, components, and circuits are not described in detail.
[0129] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A method for predicting the final carbon temperature in converter steelmaking based on multimodal characteristics of the furnace mouth flame, characterized in that, Includes the following steps: S1. Collect raw data during the converter steelmaking process, perform data cleaning and feature labeling preprocessing, and align the furnace flame image with the steel temperature and carbon content labels to obtain the preprocessed data. S2. Perform data augmentation on the preprocessed data, including geometric transformation, brightness and contrast adjustment, and Gaussian smoothing. S3. Build a steelmaking endpoint carbon temperature prediction model based on ResNet-LSTM. Use ResNet network to extract the depth feature vector of the furnace flame image, use GLCM method to extract the texture feature vector of the furnace flame image, and introduce channel attention mechanism for fusion. Input the fused features into LSTM network to model the temporal evolution law. S4. Use the data processed in steps S1 and S2 to train and validate the prediction model, and save the optimal model parameters. S5. Use the trained prediction model to perform inference and predict the final steel temperature and carbon content of the converter steelmaking process.
2. The method for predicting the final carbon temperature of converter steelmaking based on the multimodal characteristics of the furnace mouth flame according to claim 1, characterized in that, Step S1 specifically includes: S11. Data Cleaning: First, the original furnace flame video is processed by frame extraction to obtain a discrete furnace flame image sequence. Second, the furnace flame image is denoised and smoothed using median filtering or Gaussian filtering. Finally, the furnace flame image sequence is processed using the inter-frame difference method to calculate the grayscale difference of corresponding pixels to characterize the intensity of flame change in the time dimension, thereby extracting dynamic change information of the flame. S12. Feature Labeling and Alignment: Using the furnace batch as the basic labeling unit, the furnace flame video is matched with the corresponding steelmaking production time information to form a flame image sequence with a temporal relationship for each furnace batch; at the same time, using the furnace number as an index, the furnace flame image sequence collected for each furnace batch is uniformly labeled with the corresponding endpoint molten steel temperature and carbon content, and the molten steel temperature and carbon content serve as the supervision labels for the furnace batch sample.
3. The method for predicting the final carbon temperature of converter steelmaking based on multimodal characteristics of furnace mouth flame according to claim 2, characterized in that, Step S1 further includes: Multiple frames of furnace mouth flame images sampled in chronological order during each heat are combined into an image sequence and regarded as a time-series sample. Each frame in the sequence is assigned a corresponding time step index. The time-series sample corresponds to the endpoint molten steel temperature and carbon content label under the same heat number, so as to realize the mapping relationship between furnace mouth flame characteristics and endpoint carbon temperature, and provide normalized input for subsequent time series learning of LSTM network.
4. The method for predicting the final carbon temperature of converter steelmaking based on the multimodal characteristics of the furnace mouth flame according to claim 1, characterized in that, Step S2 specifically includes: S21. Use OpenCV's geometric transformation operators to perform random-scale horizontal flipping, vertical flipping, and small-angle rotation transformations on the furnace flame image. S22. Adjust the brightness and contrast of the furnace flame image, apply a linear transformation to the original pixel values to make the overall brightness and contrast of the image change randomly by a predetermined range, and then perform Gamma correction on the image after brightness and contrast adjustment. S23. Perform Gaussian smoothing on the furnace flame image to simulate the imaging blur caused by furnace flue gas or heat waves through Gaussian filtering.
5. The method for predicting the final carbon temperature of converter steelmaking based on multimodal characteristics of furnace mouth flame according to claim 1, characterized in that, Step S3 specifically includes: S31. Use a ResNet network to extract deep features from the furnace flame image, and obtain the furnace flame image at time [time value missing]. t Deep feature vectors The depth feature vector is used to characterize the overall shape, brightness distribution and high-level semantic information of the flame; S32. Based on the GLCM method, texture features are extracted from the furnace flame image to obtain the furnace flame image at time [time value missing]. t Texture feature vector The texture feature vector is used to characterize the texture roughness, energy distribution, and combustion stability of the flame; S33, Transfer the deep feature vector With texture feature vector The data is concatenated along the channel dimension, and a channel attention mechanism is introduced for weighted fusion to obtain the fused features. ; S34. Input the fused features into the LSTM network to model the temporal evolution of the multimodal features of the furnace mouth flame and establish the mapping relationship between the multimodal features of the furnace mouth flame and the carbon content and molten steel temperature at the end of converter steelmaking.
6. The method for predicting the final carbon temperature of converter steelmaking based on multimodal characteristics of furnace mouth flame according to claim 1, characterized in that, Step S4 specifically includes: S41. Construct training and validation sets using the data enhanced in step S2. Pre-train the ResNet-LSTM prediction model based on the training set. During the training process, iterate through different network parameter configurations and determine the initial optimal model parameters based on the prediction error of the validation set. S42. Based on step S41, the preprocessed data from step S1 and the enhanced data from step S2 are fused to obtain a training set and a validation set. The initial optimal model parameters saved in step S41 are used as initial values, and the ResNet-LSTM prediction model is further trained using the training set. During training, mean squared error is used as the loss function to calculate the error between the predicted values of molten steel temperature and carbon content at the end of the model output and the corresponding measured values. The model parameters are iteratively updated through the backpropagation algorithm. At the same time, the model performance is evaluated based on the validation set, and the model parameters with the best prediction performance during the validation process are retained as the optimal model parameters.
7. The method for predicting the final carbon temperature of converter steelmaking based on the multimodal characteristics of the furnace mouth flame according to claim 6, characterized in that, In step S4, the ResNet network adopts a multi-layer residual network structure. Based on the feature extraction effect, the number of convolutional channels and the output feature dimension in the residual block are adjusted. The number of hidden layers and the number of hidden units in the LSTM network are set according to the fusion feature dimension, and the relevant parameters are adaptively adjusted through validation set error feedback.
8. The method for predicting the final carbon temperature of converter steelmaking based on the multimodal characteristics of the furnace mouth flame according to claim 1, characterized in that, Step S5 specifically includes: S51. Preprocess the input test data, the same as step S1; S52. Load the trained prediction model from the specified path, input the preprocessed test data into the loaded prediction model, and the prediction model infers based on the learned optimal model parameters to calculate the predicted values of the final steel temperature and carbon content. S53. Obtain the true values of the test data, compare them with the model prediction results, calculate the prediction hit rate of the model for the final steel temperature and carbon content, and analyze the performance of the model based on the prediction hit rate.
9. An electronic device, characterized in that, The electronic device includes: processor; A memory storing computer-readable instructions that, when loaded and executed by the processor, implement the method as described in any one of claims 1 to 8.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program code that can be invoked by a processor to perform the method as described in any one of claims 1 to 8.