Converter on-line monitoring system based on AI flame analysis

Abstract

The present application belongs to the technical field of converter steelmaking, and particularly relates to a converter online monitoring system based on AI flame analysis, comprising: an image acquisition module, an AI visual feature extraction module, a time series analysis module, a multi-task discrimination module, an online learning module, an edge computing deployment module and a multi-modal fusion interface. The converter online monitoring system based on AI flame analysis of the present application introduces end-to-end AI visual analysis, time series dynamic modeling and online learning mechanism, overcomes the limitations of traditional manual features and fixed threshold scheme, realizes accurate, real-time and self-adaptive online monitoring of converter smelting conditions, and has significant industrial application value.

Description

Technical Field

[0001] This invention belongs to the field of converter steelmaking technology, specifically relating to an online monitoring system for converters based on AI flame analysis. Background Technology

[0002] Converter steelmaking is a core step in steel production. The quality of slag formation and the precision of endpoint control directly determine the quality of molten steel and production costs. Traditional manual methods rely on operators visually observing the furnace flame, which suffers from high subjectivity, high labor intensity, and low accuracy. With the development of machine vision technology, automatic recognition solutions based on image processing have emerged.

[0003] Patent CN111079537B proposes a converter smelting condition identification system based on a Gabor filter. It utilizes the Gabor filter to extract flame texture features and oxygen lance contour features, and then determines the slag hardness and smelting endpoint through fixed threshold rules. However, this type of solution has the following fundamental limitations: First, Gabor filters are manually designed feature operators, and their parameters (frequency, direction, scale) need to be set manually. They are extremely sensitive to image degradation caused by changes in lighting, smoke and dust interference, and furnace vibration in complex industrial environments, and their generalization ability and robustness are insufficient. Second, the existing solution processes each frame of image independently and uses a 0 / 1 hard threshold for single-frame discrimination, which completely ignores the temporal evolution law of converter smelting as a dynamic and continuous process and cannot utilize the gradual trend information of the flame changing from soft to hard and from expansion to contraction. Third, the system lacks self-learning ability. Once the model parameters and discrimination thresholds are set, they are fixed and cannot be adaptively adjusted with factors such as furnace age, raw material composition fluctuations, and seasonal changes, resulting in a decrease in accuracy after long-term use. Fourth, the endpoint determination dimension is singular, relying only on two superficial features: the oxygen lance outline and flame contraction. It does not deeply mine the high-dimensional semantic information of the flame, nor does it integrate multimodal information such as temperature and audio, making it difficult to meet the requirements of high-precision endpoint control.

[0004] To address the aforementioned issues, this application provides an AI-based online monitoring system for converters, which combines end-to-end AI vision with time-series analysis and applies it to online monitoring of converter furnace mouth flames.

[0005] The information disclosed in this background section is intended only to enhance the understanding of the overall background of the invention and should not be construed as an admission or in any way implying that the information constitutes prior art known to those skilled in the art. Summary of the Invention

[0006] The purpose of this invention is to provide an online monitoring system for converters based on AI flame analysis. This system replaces traditional manual feature extraction with deep neural networks and introduces time-series modeling and online learning mechanisms to achieve accurate, real-time, and adaptive monitoring of slag formation and smelting endpoints.

[0007] To achieve the above objectives, the present invention provides the following technical solution: An online monitoring system for converters based on AI flame analysis includes: The image acquisition module is used to acquire real-time image sequences of converter furnace mouth flames in the form of video streams; The AI ​​visual feature extraction module is used to input the flame image sequence into a pre-trained deep convolutional neural network to obtain a high-dimensional semantic feature vector of the furnace flame. The temporal analysis module is used to input the high-dimensional semantic feature vectors of multiple consecutive frames into the temporal neural network to capture the dynamic evolution of the flame during the smelting process and obtain temporal fusion features. The multi-task discrimination module is used to output the slag state evaluation result, the probability of proximity to the smelting endpoint and the probability of arrival at the smelting endpoint in parallel based on the time-series fusion features. The online learning module is used to collect flame images and corresponding laboratory feedback data under actual smelting conditions, and to incrementally fine-tune the deep convolutional neural network and / or temporal neural network so that the system can adapt to changes in furnace conditions.

[0008] Preferably, the deep convolutional neural network includes a lightweight backbone network and a feature pyramid network; the lightweight backbone network is one of MobileNetV3, EfficientNet-B0, or ShuffleNet; the feature pyramid network is used to fuse multi-scale flame features to enhance the perception of subtle flame textures and oxygen lance contours.

[0009] Preferably, the temporal neural network is one of a bidirectional long short-term memory network (Bi-LSTM), a gated recurrent unit (GRU), or a temporal convolutional network (TCN); the temporal analysis module also introduces a self-attention mechanism to assign higher weights to the flame features at key time steps.

[0010] Preferably, the multi-task discrimination module includes: The slag formation branch is used to output the probability of good / bad slag formation based on the degree of turbulence and uniformity of the flame texture. The endpoint proximity branch is used to output the probability of whether the current smelting endpoint is approaching, based on the probability of the oxygen lance outline appearing in the flame background and the trend of flame height change. The endpoint arrival branch is used to output the probability of whether the smelting endpoint has been reached, based on the degree of flame outline contraction, brightness decay trend, and abrupt changes in time sequence.

[0011] Preferably, the online learning module includes: The data caching unit is used to store newly acquired flame image sequences and corresponding secondary gun test results or manually labeled tags; The model fine-tuning unit is used to incrementally update the parameters of the deep convolutional neural network and the temporal neural network based on the new data in the data caching unit and using a transfer learning strategy. The model evaluation unit is used to evaluate the performance of the fine-tuned model on the validation set and triggers model replacement only when performance improves.

[0012] Preferably, the system also includes an edge computing deployment module; the edge computing deployment module is integrated into the industrial control computer at the converter site or an embedded AI box, and is used to perform INT8 quantization and TensorRT acceleration on the deep convolutional neural network and the temporal neural network to meet the millisecond-level real-time inference requirements.

[0013] Preferably, the system also includes a multimodal fusion interface for receiving furnace mouth audio signals, furnace body temperature signals, or flue gas analysis signals; the time series analysis module performs channel cascading or attention-weighted fusion of the high-dimensional semantic feature vector with the audio signal, temperature signal, or flue gas signal to obtain multimodal time series fusion features.

[0014] Compared with the prior art, the present invention has the following beneficial effects: (1) The converter online monitoring system based on AI flame analysis of the present invention abandons the poor robustness of Gabor manual filter, adopts deep convolutional neural network to automatically learn the high-dimensional semantic features of flame, has a strong adaptability to changes in light and smoke interference, and greatly improves the feature expression ability.

[0015] (2) The converter online monitoring system based on AI flame analysis of the present invention introduces time-series networks such as Bi-LSTM / GRU / TCN to model the evolution process of continuous multi-frame flame images, which can perceive the gradual trend of flame from soft to hard and from expansion to contraction, and significantly reduce the single-frame misjudgment rate.

[0016] (3) The converter online monitoring system based on AI flame analysis of the present invention optimizes slag judgment and endpoint prediction simultaneously through a multi-task learning framework. Each task branch shares the underlying visual features, which improves the discrimination accuracy and reduces the overall computational cost.

[0017] (4) The converter online monitoring system based on AI flame analysis of the present invention continuously absorbs new smelting data and makes incremental fine adjustments through the online learning module. The system can adapt to long-term drift such as furnace age increase and raw material changes, and solves the problem that the traditional system becomes less and less accurate with use.

[0018] (5) The converter online monitoring system based on AI flame analysis of the present invention can achieve millisecond-level real-time monitoring on industrial control computers or embedded devices at the converter site through model quantization and inference acceleration, thus meeting the low latency requirements of industrial sites.

[0019] (6) The converter online monitoring system based on AI flame analysis of the present invention, by reserving multi-modal interfaces such as audio, temperature, and flue gas, lays the foundation for building a more comprehensive and robust converter intelligent sensing system in the future. Attached Figure Description

[0020] Figure 1 This is a system block diagram of an embodiment of the present invention. Detailed Implementation

[0021] The technical solution of this invention patent will be clearly and completely described below. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of this invention.

[0022] In the description of this invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship are based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention.

[0023] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "joining" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to a connection within two components. Those skilled in the art will understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0024] See attached document Figure 1The converter online monitoring system based on AI flame analysis provided in this application is deployed near the converter mouth and specifically includes: an image acquisition module, an AI visual feature extraction module, a time series analysis module, a multi-task discrimination module, an online learning module, an edge computing deployment module, and a multimodal fusion interface. Among these, The image acquisition module uses a high-temperature resistant industrial CCD camera, which is fixed on a tripod in a safe area diagonally above the furnace opening, and acquires a 1080p video stream of the furnace flame at a frame rate of 25fps. The front end of the camera is equipped with a narrow-band filter and an automatic aperture to suppress interference from strong light and splashing slag.

[0025] The AI ​​visual feature extraction module receives the video stream output from the image acquisition module, preprocesses each frame of the image (grayscale conversion, resizing to 448×448, normalization to [0,1]), and then inputs it into a pre-trained deep convolutional neural network. In this embodiment, the backbone network is EfficientNet-B0, which is connected to a Feature Pyramid Network (FPN). The FPN fuses features from layers P3 to P5, ultimately outputting a 512-dimensional high-dimensional semantic feature vector for each frame. This network automatically learns high-level semantics such as flame texture, oxygen gun outline, and flame shape through end-to-end training, without the need for manual design of Gabor kernel parameters.

[0026] The temporal analysis module inputs high-dimensional semantic feature vectors (16×512 dimensions) from 16 consecutive frames (T=16 frames) into a bidirectional LSTM (Bi-LSTM) network. The Bi-LSTM hidden layer has a dimension of 256, enabling it to simultaneously capture both forward trends (e.g., gradual contraction) and backward dependencies (e.g., abrupt flashback) in flame evolution. To further emphasize the role of keyframes, a temporal self-attention mechanism is introduced into the Bi-LSTM output layer, outputting weighted temporal fusion features (512 dimensions).

[0027] The multi-task discrimination module, based on temporal fusion features, outputs in parallel through three independent fully connected branches: the slag state branch outputs binary classification probabilities (Softmax), P(good slag) and P(poor slag); the endpoint proximity branch outputs the probability of whether the current state is close to the endpoint, with the probability approaching 1 when the oxygen lance outline gradually appears in time and the flame height decreases; and the endpoint arrival branch outputs the probability of whether the current state is close to the endpoint, with the probability approaching 1 when the flame outline significantly shrinks in time, the brightness continuously decreases, and thinning features appear. The three branches share temporal fusion features and are jointly trained using a multi-task loss function.

[0028] in, , , The weights of each task are dynamically adjusted based on on-site needs.

[0029] The online learning module includes a data caching unit, a model fine-tuning unit, and a model evaluation unit. The data caching unit automatically saves the flame video clip and the corresponding secondary lance carbon temperature result or manually confirmed endpoint time after each smelting furnace completes. The model fine-tuning unit triggers incremental fine-tuning after accumulating M=50 furnaces of new data. During fine-tuning, the first 5 layers of EfficientNet-B0 (low-level general visual features) are frozen, and only the network backend, FPN, Bi-LSTM, and multi-task branches are fine-tuned with a low learning rate of 1e-5 to prevent catastrophic forgetting. The model evaluation unit calculates the F1-score and endpoint prediction hit rate on the validation set. Only when the new model outperforms the old model is the edge model updated via OTA (Over-The-Air).

[0030] The edge computing deployment module converts the trained PyTorch model to ONNX format, further optimizes it with INT8 quantization and layer fusion using TensorRT, and deploys it on an NVIDIA Jetson AGX Xavier embedded device. Actual single-frame inference time is << 8ms, meeting the 25fps real-time requirement.

[0031] This embodiment reserves RS485 and TCP / IP interfaces, which can be optionally connected to a furnace opening audio sensor and an infrared thermometer. After Mel-frequency conversion, the audio signal is used to extract features through a lightweight CNN, which is then channel-cascaded with visual temporal fusion features. After another layer of attention weighting, multimodal fusion features are obtained to further improve the accuracy of endpoint prediction.

[0032] In practice, a sequence of flame images from the converter furnace opening is acquired in real time as a video stream. An industrial camera captures video at 25fps, and 16 consecutive frames are selected as input samples by sliding across a time window. The flame image sequence is preprocessed, including: grayscale conversion to reduce computational load; random cropping and brightness perturbation are performed only during the training phase to enhance generalization; and fixed-size resizing and normalization are performed during the inference phase.

[0033] The preprocessed flame image sequence is input into a pre-trained deep convolutional neural network to obtain a high-dimensional semantic feature vector for each frame. Multiple consecutive frames of high-dimensional semantic feature vectors are then input into a Bi-LSTM and a temporal self-attention network to capture the dynamic evolution of the flame, resulting in temporal fusion features. For example, in the later stages of smelting, if the temporal network detects a continuous decrease in flame height and a gradually clearer oxygen lance outline, the temporal fusion features will encode a "approaching the endpoint" temporal pattern. Based on these temporal fusion features, a multi-task discrimination module outputs in parallel the slag formation state assessment results, the probability of approaching the smelting endpoint, and the probability of reaching the smelting endpoint. The on-site HMI interface can display the probability curves of these three parameters in real time for operator reference.

[0034] Flame images and corresponding laboratory feedback data under actual smelting conditions are collected, and the deep neural network is incrementally fine-tuned. This step ensures that the system maintains high accuracy even after long-term operation, achieving "the more you use it, the more accurate it becomes."

[0035] Before system deployment, offline pre-training is required, as detailed below: Dataset Construction: Historical smelting videos from N furnaces were collected, and each frame was labeled by a senior process engineer: slag formation status (soft / hard), presence of the oxygen lance outline, and whether the endpoint was reached. Sub-lance test results were also recorded as hard labels.

[0036] Pre-training: The weights of EfficientNet-B0, pre-trained on ImageNet, were used as initialization. The entire network was trained end-to-end on the Converter dataset. The Adam optimizer was used with an initial learning rate of 1e-3 and a cosine annealing strategy, for a total of 100 epochs.

[0037] Data augmentation: Random rotation, horizontal flipping, Gaussian blur, and salt-and-pepper noise were applied to the training samples to simulate on-site smoke and dust, and logarithmic transformation was used to simulate changes in illumination.

[0038] Hard sample mining: Online hard sample mining (OHEM) is performed on easily confused samples near the endpoint (such as flames that briefly shrink and then expand again) to improve the robustness of endpoint discrimination.

[0039] In another embodiment, the furnace mouth audio signal is accessed via a multimodal fusion interface. Different stages of converter smelting have characteristic audio: when slag formation is poor, splashing at the furnace mouth is accompanied by popping sounds, while the sound tends to be low and stable when the endpoint is reached. After preprocessing, 128-dimensional audio features are extracted using 1D-CNN, and concatenated with visual temporal fusion features to form a 640-dimensional vector, which is then input into the multi-task discrimination module. Experiments show that the accuracy of endpoint arrival prediction can be further improved after incorporating audio.

[0040] The converter online monitoring system based on AI flame analysis of the present invention overcomes the limitations of traditional manual feature and fixed threshold schemes by introducing end-to-end AI visual analysis, time-series dynamic modeling and online learning mechanisms, and realizes accurate, real-time and adaptive online monitoring of converter smelting conditions, which has significant industrial application value.

[0041] The foregoing description of specific exemplary embodiments of the invention is for illustrative and explanatory purposes. These descriptions are not intended to limit the invention to the precise forms disclosed, and it will be apparent that many changes and variations can be made in accordance with the foregoing teachings. The exemplary embodiments were chosen and described in order to explain the specific principles of the invention and its practical application, thereby enabling those skilled in the art to implement and utilize various different exemplary embodiments of the invention, as well as various different choices and variations. The scope of the invention is intended to be defined by the claims and their equivalents.

Claims

1. An online monitoring system for converters based on AI flame analysis, characterized in that, include: The image acquisition module is used to acquire real-time image sequences of converter furnace mouth flames in the form of video streams; The AI ​​visual feature extraction module is used to input the flame image sequence into a pre-trained deep convolutional neural network to obtain a high-dimensional semantic feature vector of the furnace flame. The temporal analysis module is used to input the high-dimensional semantic feature vectors of multiple consecutive frames into the temporal neural network to capture the dynamic evolution of the flame during the smelting process and obtain temporal fusion features. The multi-task discrimination module is used to output the slag state evaluation result, the probability of proximity to the smelting endpoint and the probability of arrival at the smelting endpoint in parallel based on the time-series fusion features. The online learning module is used to collect flame images and corresponding laboratory feedback data under actual smelting conditions, and to incrementally fine-tune the deep convolutional neural network and / or temporal neural network so that the system can adapt to changes in furnace conditions.

2. The converter online monitoring system based on AI flame analysis according to claim 1, characterized in that, The deep convolutional neural network includes a lightweight backbone network and a feature pyramid network; the lightweight backbone network is one of MobileNetV3, EfficientNet-B0, or ShuffleNet; the feature pyramid network is used to fuse multi-scale flame features to enhance the perception of subtle flame textures and oxygen lance contours.

3. The converter online monitoring system based on AI flame analysis according to claim 1, characterized in that, The temporal neural network is one of a bidirectional long short-term memory network, a gated recurrent unit, or a temporal convolutional network; the temporal analysis module also introduces a self-attention mechanism to assign higher weights to the flame features at key time steps.

4. The converter online monitoring system based on AI flame analysis according to claim 1, characterized in that, The multi-task discrimination module includes: The slag formation branch is used to output the probability of good / bad slag formation based on the degree of turbulence and uniformity of the flame texture. The endpoint proximity branch is used to output the probability of whether the current smelting endpoint is approaching, based on the probability of the oxygen lance outline appearing in the flame background and the trend of flame height change. The endpoint arrival branch is used to output the probability of whether the smelting endpoint has been reached, based on the degree of flame outline contraction, brightness decay trend, and abrupt changes in time sequence.

5. The converter online monitoring system based on AI flame analysis according to claim 1, characterized in that, The online learning module includes: The data caching unit is used to store newly acquired flame image sequences and corresponding secondary gun test results or manually labeled tags; The model fine-tuning unit is used to incrementally update the parameters of the deep convolutional neural network and the temporal neural network based on the new data in the data caching unit and using a transfer learning strategy. The model evaluation unit is used to evaluate the performance of the fine-tuned model on the validation set and triggers model replacement only when performance improves.

6. The converter online monitoring system based on AI flame analysis according to claim 1, characterized in that, It also includes an edge computing deployment module; the edge computing deployment module is integrated into the industrial control computer or embedded AI box at the converter site, and is used to perform INT8 quantization and TensorRT acceleration on the deep convolutional neural network and the temporal neural network to meet the millisecond-level real-time inference requirements.

7. The converter online monitoring system based on AI flame analysis according to claim 1, characterized in that, It also includes a multimodal fusion interface for receiving furnace inlet audio signals, furnace body temperature signals, or flue gas analysis signals; the time-series analysis module performs channel cascading or attention-weighted fusion of the high-dimensional semantic feature vector with the audio signal, temperature signal, or flue gas signal to obtain multimodal time-series fusion features.

Images