Converter steelmaking intervention warning method and device, computer equipment and medium

Abstract

The application relates to a converter steelmaking return intervention warning method and device, computer equipment and a medium. The method comprises the following steps: acquiring audio data and video data in a converter; extracting audio features in the audio data and extracting image features in the video data; inputting the audio features and the image features into a multi-modal fusion and multi-task output model; the multi-modal fusion and multi-task output model comprises an audio branch neural network, an image branch neural network, an attention fusion layer and a multi-task output head; calling the multi-modal fusion and multi-task output model to process based on the audio features and the image features, obtaining a return intervention warning probability, a return degree and measure optimization data. The whole scheme can realize accurate return intervention warning in converter steelmaking.

Description

Technical Field

[0001] This application relates to the field of converter steelmaking technology, and in particular to a method, device, computer equipment and medium for early warning of dryness in converter steelmaking. Background Technology

[0002] In converter steelmaking, slag drying is a common and serious technological problem that has long been a concern. When slag drying occurs, its viscosity increases significantly, disrupting its original beneficial physical properties. This not only drastically reduces the slag's ability to cover and protect the molten metal, leading to increased contact area between the molten metal and air, thus exacerbating the oxidation loss of metal elements and severely impacting the quality of the finished steel, but also disrupts the normal rhythm of the steelmaking process, increases energy consumption, reduces overall production efficiency, and causes economic losses to the enterprise.

[0003] In converter steelmaking, the assessment of re-drying phenomena traditionally relies primarily on manual experience. However, this method has several limitations. Due to varying levels of experience among operators, it's difficult to standardize the recognition and judgment criteria for re-drying characteristics, leading to inaccurate and inconsistent results. Furthermore, manual judgment often depends on operators' observation of apparent phenomena such as furnace flames and sounds. In the complex and variable steelmaking environment, these phenomena may not be obvious or easily confused with other conditions, making timely assessment difficult to guarantee.

[0004] Traditional methods for judging re-drying based on human experience are ill-suited to detecting and accurately assessing subtle changes in the early stages of re-drying. Therefore, there is an urgent need for an accurate early warning system for re-drying during converter steelmaking. Summary of the Invention

[0005] Based on this, it is necessary to address at least one technical problem, such as the difficulty in ensuring the timeliness of judgment due to the indistinct appearance of phenomena or their easy confusion with other situations in the complex and ever-changing steelmaking environment, and the difficulty in keenly detecting and accurately judging subtle changes in the early stage of re-drying. Therefore, it is necessary to provide a more timely and accurate method, device, computer equipment, computer-readable storage medium, and computer program product for early warning of re-drying in converter steelmaking.

[0006] Firstly, this application provides a method for early warning of dryness recurrence in converter steelmaking. The method includes:

[0007] Acquire audio and video data from inside the converter;

[0008] Extract audio features from audio data and extract image features from video data;

[0009] Audio features and image features are respectively input into the multimodal fusion and multi-task output model; the multimodal fusion and multi-task output model includes an audio branch neural network, an image branch neural network, an attention fusion layer, and a multi-task output head;

[0010] The multimodal fusion and multi-task output model is invoked, and data on audio and image features are processed to obtain the probability of drought recurrence warning, the degree of drought recurrence, and the data for optimizing measures.

[0011] The multimodal fusion and multi-task output model processing includes: an audio branch neural network extracts audio temporal features based on audio features, and an image branch neural network extracts spatial-temporal pattern features; an attention fusion layer calculates audio attention weights and image attention weights based on the real-time signal-to-noise ratio of audio and video data, and calculates a fusion vector of audio temporal features and spatial-temporal pattern features based on the audio attention weights and image attention weights; a multi-task output head performs binary classification output for dryness warning based on the fusion vector to obtain the dryness warning probability; a dryness degree regression output is performed based on the fusion vector to obtain the dryness degree; and a measure optimization process is performed based on the dryness degree to obtain measure optimization data, which includes adjusting the gun position height and adjusting the slag dosage.

[0012] Furthermore, extracting audio features from the audio data includes:

[0013] The audio data is divided into frames to obtain audio frame data;

[0014] Based on the audio frame data, extract any one or more of the fundamental frequency, fluctuation amplitude, MFCC, spectral entropy, and zero-crossing rate to construct multi-dimensional temporal features as audio features.

[0015] Furthermore, the image features in the video data include HSV color features, flame height, and Martian density; the extracted image features include:

[0016] Based on the video data, a single frame of flame image is obtained, and HSV color features are extracted;

[0017] Edge detection is performed on a single frame of a flame image to highlight the flame outline, and the pixel coordinates of the top of the flame outline are extracted to determine the flame height.

[0018] Motion analysis is performed on continuous flame image frames to identify Martians in a preset motion state, and the number of Martians per unit area is counted to obtain the Martian density.

[0019] Furthermore, based on the degree of re-drying, optimization measures were performed, resulting in optimized measure data including:

[0020] Obtain the lance height and slag dosage in the current converter steelmaking process;

[0021] The gun position adjustment amount is obtained based on the degree of drying, the gun position height, and a preset linear function of the degree of drying and the gun position height.

[0022] The adjustment amount of slag dosage is obtained based on the degree of re-drying, the dosage of slag, and a preset piecewise linear function of the degree of re-drying and the dosage of slag.

[0023] Furthermore, the training process of the multimodal fusion and multi-task output model includes:

[0024] Obtain the initial multimodal fusion and multi-task output model and training sample data; the initial multimodal fusion and multi-task output model includes the initial audio branch neural network, the initial image branch neural network, the initial attention fusion layer, and the initial multi-task output head;

[0025] The initial audio branch neural network and the initial image branch neural network are trained separately based on the training sample data until convergence, and the encoder parameters of the trained audio branch neural network and image branch neural network are frozen.

[0026] The first sub-loss function, the second sub-loss function, and the third sub-loss function are obtained. The first sub-loss function is constructed based on the cross-entropy of the binary classification of the dryness warning. The second sub-loss function is constructed based on the mean square error of the dryness degree regression. The third sub-loss function is constructed based on the gun position height adjustment and the slag dosage adjustment, and is constructed with additional constraint penalty using Huber loss.

[0027] The first, second, and third sub-loss functions are weighted to construct the total loss function;

[0028] The initial attention fusion layer and initial multi-task output head are trained based on the total loss function and training sample data to obtain the multimodal fusion and multi-task output model.

[0029] Furthermore, the audio branch neural network includes an audio input layer, a 1D-CNN encoder, and an LSTM temporal modeling layer;

[0030] The audio input layer is used to receive audio features, preprocess them, and output a standardized audio feature sequence.

[0031] The 1D-CNN encoder, connected to the output of the audio input layer, is used to extract local spatial features from the standardized audio feature sequence and output an encoded vector containing local temporal features.

[0032] The LSTM temporal modeling layer, connected to the output of the 1D-CNN encoder layer, is used to model the long and short-term temporal dependencies of the encoded vectors and output audio temporal features that fuse local and temporal features.

[0033] Furthermore, the image branching neural network includes an image input layer, a 2D-CNN encoder, and a temporal pooling layer;

[0034] The image input layer receives image features, performs preprocessing, and outputs a normalized image feature tensor.

[0035] The 2D-CNN encoder layer is connected to the output of the image input layer and is used to extract multi-level features of the spatial dimension of the image feature tensor, outputting an encoded feature map containing multi-scale spatial semantic information.

[0036] The temporal pooling layer, connected to the output of the 2D-CNN encoder layer, is used to aggregate and reduce the temporal dimension of the encoded feature map, and outputs a spatial-temporal pattern feature that fuses spatial and temporal features.

[0037] Secondly, this application also provides a device for early warning of dryness during converter steelmaking. The device includes:

[0038] The data acquisition module is used to acquire audio and video data inside the converter;

[0039] The feature extraction module is used to extract audio features from audio data and image features from video data.

[0040] The data input module is used to input audio features and image features into the multimodal fusion and multi-task output model, respectively; the multimodal fusion and multi-task output model includes an audio branch neural network, an image branch neural network, an attention fusion layer, and a multi-task output head;

[0041] The model invocation module is used to invoke the multimodal fusion and multi-task output model to process audio and image features to obtain the probability of drought return warning, the degree of drought return, and data on optimized measures.

[0042] The multimodal fusion and multi-task output model processing includes: an audio branch neural network extracts audio temporal features based on audio features, and an image branch neural network extracts spatial-temporal pattern features; an attention fusion layer calculates audio attention weights and image attention weights based on the real-time signal-to-noise ratio of audio and video data, and calculates a fusion vector of audio temporal features and spatial-temporal pattern features based on the audio attention weights and image attention weights; a multi-task output head performs binary classification output for dryness warning based on the fusion vector to obtain the dryness warning probability; a dryness degree regression output is performed based on the fusion vector to obtain the dryness degree; and a measure optimization process is performed based on the dryness degree to obtain measure optimization data, which includes adjusting the gun position height and adjusting the slag dosage.

[0043] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of any of the methods described above.

[0044] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps of any of the methods described above.

[0045] The present invention provides a method, device, computer equipment, and storage medium for early warning of dryness in converter steelmaking. On the one hand, by simultaneously acquiring audio and video data from the converter and extracting features separately, it utilizes multimodal fusion to comprehensively utilize information from multiple sources, avoiding the information loss problem that may exist with a single data source. On the other hand, the attention fusion layer in the multimodal fusion and multi-task output model calculates the attention weights of different modalities based on the real-time signal-to-noise ratio, which can reasonably fuse audio temporal features and spatial-temporal pattern features, enhancing feature effectiveness. Based on this, the multi-task output head can accurately perform binary classification of dryness warning, regression of dryness degree, and optimization of measures based on the fusion vector, thereby achieving accurate early warning effect of dryness in converter steelmaking. Attached Figure Description

[0046] Figure 1 This is an application environment diagram of the early warning method for re-drying in converter steelmaking in one embodiment;

[0047] Figure 2 This is a flowchart illustrating a method for early warning of re-drying in converter steelmaking in one embodiment;

[0048] Figure 3 This is a flowchart illustrating a method for early warning of dryness in converter steelmaking, used in a specific application example.

[0049] Figure 4 This is a structural block diagram of a re-drying early warning device in converter steelmaking in one embodiment;

[0050] Figure 5 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0052] The method for early warning of dryness in converter steelmaking provided in this application embodiment can be applied to, for example... Figure 1In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104 or placed on a cloud or other network server. Terminal 102 sends a dry-return warning request in converter steelmaking to server 104. Server 104 responds to the request, acquiring audio and video data from the converter; extracting audio features from the audio data and image features from the video data; inputting the audio and image features into a multimodal fusion and multi-task output model; the multimodal fusion and multi-task output model includes an audio branch neural network, an image branch neural network, an attention fusion layer, and a multi-task output head; the multimodal fusion and multi-task output model is called to process the audio and image features to obtain the dry-return warning probability, dry-return degree, and optimization data; wherein, the processing of the multimodal fusion and multi-task output model... The process includes: an audio branch neural network extracting audio temporal features based on audio features, and an image branch neural network extracting spatial-temporal pattern features; an attention fusion layer calculating audio attention weights and image attention weights based on the real-time signal-to-noise ratio of audio and video data, respectively, and calculating a fusion vector of audio temporal features and spatial-temporal pattern features based on the audio attention weights and image attention weights; a multi-task output head performing binary classification output for dryness warning based on the fusion vector to obtain the dryness warning probability; performing dryness degree regression output based on the fusion vector to obtain the dryness degree; and performing measure optimization processing based on the dryness degree to obtain measure optimization data, including gun position height adjustment and slag dosage adjustment. Furthermore, server 104 can feed back the dryness warning probability, dryness degree, and measure optimization data to terminal 102. Terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can be smart speakers, smart TVs, smart air conditioners, smart vehicle devices, etc. Portable wearable devices can be smartwatches, smart bracelets, head-mounted devices, etc. Server 104 can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0053] To explain in detail the technical principle and effect of the early warning method for re-drying in converter steelmaking of this application, the following will first introduce the changes in audio characteristics and flame image characteristics corresponding to the occurrence of re-drying in converter steelmaking, as well as the corresponding improvement measures.

[0054] During converter steelmaking, the audio signals inside the furnace can reflect the state of the slag. When re-drying occurs, the slag viscosity increases, and the audio signals will change significantly. Specifically, the audio frequency decreases, for example, from the normal 1000Hz to below 500Hz; at the same time, the signal stability deteriorates, with fluctuations exceeding 30%. Therefore, in-depth research has found that the occurrence of re-drying can be determined by analyzing the frequency and stability parameters of the collected audio signals.

[0055] Under normal steelmaking conditions, the furnace flame is orange-red, with a relatively high flame height and dense sparks. When re-drying occurs, the flame color changes from orange-red to dark red, the flame height decreases, and sparks become sparse. Therefore, in-depth research has found that analyzing flame images captured by high-definition cameras and extracting flame color and morphological characteristic parameters can determine whether the re-drying conditions are met.

[0056] To address the existing dry-out phenomenon, a pre-set algorithm can be used to develop an oxygen lance adjustment strategy. For example, when dry-out is detected, the oxygen lance can be lowered from its current position (e.g., 1.8m) to the slagging lance position (e.g., 1.2m). Lowering the oxygen lance position increases the contact area between oxygen and the molten pool, allowing oxygen to react more fully with the substances in the molten pool, promoting the slagging process, and thus alleviating the dry-out problem. Additionally, based on the degree of dry-out, a pre-set model can determine the appropriate amount of lightly calcined dolomite or fluorite to be added. A comprehensive model considering various factors, such as slag composition, temperature, and degree of dry-out, is used to determine the optimal amount of slagging agent to be added. After determining the amount, the slagging agent is rapidly added to the converter via the silo system to reduce slag viscosity, improve slag flowability, further promote slagging, and resolve the dry-out problem.

[0057] In one embodiment, such as Figure 2 As shown, a method for early warning of dryness in converter steelmaking is provided, which is applied to... Figure 1 Taking server 104 as an example, the following steps are included:

[0058] S200: Acquire audio and video data from inside the converter.

[0059] By rationally arranging audio acquisition devices (such as microphones) and video acquisition devices (such as cameras) at the converter steelmaking site, the audio acquisition devices collect sound signals generated during the smelting process in the converter in real time and convert them into digital audio data; the video acquisition devices capture the smelting scene in the converter in real time, obtain video image data containing information such as flames and slag surface in the furnace, and transmit these data to the server 104.

[0060] Audio data reflects dynamic sound changes during the smelting process in the converter, such as slag melting sounds and splashing sounds. These sound characteristics are related to the drying phenomenon. As mentioned above, when drying occurs, the slag viscosity increases, and the audio signal changes significantly, specifically by a decrease in audio frequency. Video data visually presents the actual smelting conditions inside the converter, such as the color and intensity of the flame. It can also provide important evidence for judging the drying situation. As mentioned above, when drying occurs, the flame color changes from orange-red to dark red, the flame height decreases, and sparks become sparse. By acquiring both types of data simultaneously, comprehensive smelting information from multiple perspectives can be obtained, laying the foundation for accurate subsequent judgment of the drying situation.

[0061] S400: Extracts audio features from audio data and extracts image features from video data.

[0062] For audio data, audio feature extraction algorithms are employed, such as fundamental frequency, zero-crossing rate, MFCC (13-dimensional), spectral entropy, and fluctuation amplitude. The zero-crossing rate reflects the frequency variation of the audio signal; MFCC effectively simulates the human ear's perception of sound, extracting representative audio feature vectors. For video data, image processing techniques, such as edge detection, are used to extract flame contour features. Further image features primarily include HSV color features, flame height, and spark density.

[0063] S600: Input audio features and image features into the multimodal fusion and multi-task output model respectively; the multimodal fusion and multi-task output model includes an audio branch neural network, an image branch neural network, an attention fusion layer, and a multi-task output head.

[0064] The multimodal fusion and multi-task output model is a specially designed deep learning model architecture. Audio branch neural networks and image branch neural networks further process audio and image features, respectively. The audio branch neural network can employ convolutional neural networks (CNNs) and their variants, such as long short-term memory networks (LSTM), to extract temporal information from audio features, obtaining audio temporal features. The image branch neural network typically uses a CNN structure, utilizing convolutional layers, pooling layers, and other operations to extract spatial-temporal pattern features from video images. The attention fusion layer fuses the features from both audio and image modalities. It calculates audio attention weights and image attention weights based on the real-time signal-to-noise ratio of the audio and video data, respectively, and then calculates a fusion vector of audio temporal features and spatial-temporal pattern features based on these weights. The multi-task output head is responsible for outputting multiple tasks based on the fusion vector, including binary classification for anti-interference warning, anti-interference degree regression, and measure optimization processing.

[0065] By inputting audio and image features into their respective branch neural networks, the advantages of different modalities can be fully utilized to extract their respective feature information. The attention fusion layer dynamically adjusts the contribution of audio and image features during the fusion process by calculating attention weights based on the real-time signal-to-noise ratio, resulting in a fused vector that more accurately reflects the actual smelting conditions within the converter. The multi-task output head design enables the model to complete multiple related tasks simultaneously, improving its practicality and efficiency.

[0066] S800: Calls the multimodal fusion and multi-task output model, processes audio and image features to obtain the probability of drought return warning, the degree of drought return, and data on measures to optimize the response.

[0067] The audio branch neural network extracts temporal features from the input audio features through operations between its internal neuron layers and activation functions. The image branch neural network processes the input image features, using convolutional layers to extract local features, pooling layers to reduce feature dimensionality and enhance robustness, and fully connected layers to extract pattern feature vectors containing spatial-temporal information. These features describe the changes in the shape and motion of objects (mainly flames / sparks) in the video image over time. The attention fusion layer first calculates audio attention weights and image attention weights separately based on the real-time signal-to-noise ratio (SNR) of the audio and video data using specific algorithms (such as soft attention mechanisms). The SNR reflects the ratio of effective information to noise in the signal, and the real-time SNR dynamically reflects the quality of the current audio and video data. Then, based on the calculated audio and image attention weights, the audio temporal features and spatial-temporal pattern features are weighted and fused to obtain a fusion vector. This fusion vector comprehensively considers the importance of both audio and video modalities at different times, more accurately reflecting the overall smelting state within the converter. The multi-task output head processes multiple tasks based on the fusion vector. For the binary classification output of the dryness recurrence warning, classification algorithms such as logistic regression are used to determine whether dryness recurrence has occurred based on the fusion vector, outputting the probability of dryness recurrence warning; the closer the probability value is to 1, the greater the likelihood of dryness recurrence. For the regression output of dryness recurrence severity, regression algorithms such as linear regression are used to predict the degree of dryness recurrence based on the fusion vector; for example, a numerical range can be used to represent the severity of dryness recurrence. Based on the predicted degree of dryness recurrence, pre-defined rules or algorithms are used for optimization measures; for example, based on different ranges of dryness recurrence severity, the adjustment amount of the gun position height and the adjustment amount of the slag dosage are determined to obtain optimization data.

[0068] Here, the multimodal fusion and multi-task output model processing can fully utilize information from both audio and video modal data to accurately assess the drying situation within the converter from multiple perspectives. The drying warning probability can promptly alert operators to the phenomenon and prompt appropriate measures; accurate prediction of the drying degree helps operators understand its severity and develop more reasonable handling plans; and the optimized measures data provides operators with specific operational guidance, enabling them to quickly and effectively adjust smelting parameters, improve the drying situation, and enhance the quality and efficiency of converter steelmaking.

[0069] In this embodiment, a method for early warning of dryness in converter steelmaking according to the present invention is presented. On the one hand, by simultaneously acquiring audio and video data from the converter and extracting features separately, a multimodal fusion approach is used to comprehensively utilize information from multiple sources, avoiding the information loss problem that may exist in a single data source. On the other hand, the attention fusion layer in the multimodal fusion and multi-task output model calculates the attention weights of different modalities based on the real-time signal-to-noise ratio, which can reasonably fuse audio temporal features and spatial-temporal pattern features, enhancing the effectiveness of features. Based on this, the multi-task output head can accurately perform binary classification of dryness warning, regression of dryness degree, and optimization of measures based on the fusion vector, thereby achieving an accurate early warning effect of dryness in converter steelmaking.

[0070] In one embodiment, extracting audio features from audio data includes:

[0071] Step 1: Divide the audio data into frames to obtain audio frame data.

[0072] Continuous audio data is divided according to preset frame lengths and frame shifts. The frame length and frame shift can be reasonably set according to the actual application scenario and audio characteristics. In practical applications, an 8000Hz sampling rate can be used to acquire the audio stream in the converter in real time. After acquiring the continuous audio signal, it needs to be segmented into segments suitable for analysis to facilitate subsequent audio feature extraction; that is, frame segmentation is performed. The acquired audio stream is segmented into frames with a duration of 256ms, and each frame of audio data is used as an independent analysis unit, thus obtaining audio frame data. The frame segmentation operation enables the audio signal to have better local characteristics in the time dimension, laying the foundation for accurate audio feature extraction in the subsequent process.

[0073] Step 2: Based on the audio frame data, extract any one or more of the fundamental frequency, fluctuation amplitude, MFCC, spectral entropy, and zero-crossing rate to construct multi-dimensional temporal features as audio features.

[0074] Specifically, for each frame of audio data, the spectrum is calculated using Fast Fourier Transform (FFT) to find the peak frequency and extract the fundamental frequency F0 of that frame. For several consecutive groups, such as 10 frames of audio data, the temporal variance of the fundamental frequency F0 of these 10 frames is calculated to extract the fluctuation amplitude. The first 13 coefficients are selected through framing → frequency domain conversion → Mel filtering → logarithmic transformation → Discrete Cosine Transform (DCT) to extract the 13-dimensional MFCC (Mel frequency cepstral coefficients, an important characteristic parameter that can capture changes in the audio spectrum envelope). The spectral entropy is extracted by calculating H = -Σpi·log(pi), where pi represents the proportion of the energy of the i-th frequency component in the spectrum to the total energy. The zero-crossing rate is extracted by statistically analyzing the frequency at which the audio signal crosses zero points per unit time.

[0075] This embodiment presents a preferred embodiment for extracting audio features, extracting any one or more of the fundamental frequency, fluctuation amplitude, MFCC, spectral entropy, and zero-crossing rate to construct multidimensional temporal features as audio features. This is not a conventional choice, but an important one made based on the study of the back-drying state, for the following reasons:

[0076] Under normal slag conditions, the fundamental frequency F0 is around 800-1200Hz. When dryness occurs, the slag becomes viscous, and the frequency of the sound it produces decreases; for example, the fundamental frequency F0 drops below 500Hz. Therefore, a dryness warning can be issued by extracting the fundamental frequency decrease characteristic (the presence of a decreasing trend) or by detecting a fundamental frequency below a first set threshold, such as below 600Hz.

[0077] Under normal slag conditions, the audio signal is relatively stable with a small variance. However, when back-drying occurs, the signal becomes unstable, and the calculated variance relative to the mean will exceed 30%. In this case, a secondary warning can be triggered. Therefore, a back-drying warning can be issued if the fluctuation amplitude increases (showing an upward trend) or exceeds a second set threshold, such as exceeding 30%. Compared to a decrease in the fundamental frequency, this increase in fluctuation amplitude can be configured with a higher weight in the multi-dimensional time series features.

[0078] Under normal slag conditions, the high-frequency components of the audio signal are relatively abundant. When dryness occurs, the slag viscosity increases, and the high-frequency components attenuate. At this time, the mean value of the 5th to 13th order MFCC will decrease by more than 0.5. Optionally, MFCC is a 13-dimensional MFCC, which extracts the attenuation characteristics of high-frequency components or the high-frequency components are below a third set threshold to issue a dryness warning.

[0079] Under normal slag conditions, the sound energy distribution is relatively uniform, and the spectral entropy value is relatively large. However, when dryness occurs, the sound energy is concentrated in the low-frequency part, and the spectral entropy value drops below 4.5. A dryness warning is issued by extracting the characteristic of reduced spectral entropy value or when the spectral entropy value is lower than a fourth set threshold, such as below 4.5.

[0080] Under normal slag conditions, the audio signal has a certain zero-crossing rate characteristic; when dryness occurs and viscous slag sounds are produced, the zero-crossing rate will be less than 40% of the normal value. Optionally, a dryness warning can be issued by extracting the zero-crossing rate decrease characteristic or the zero-crossing rate being lower than the fifth set threshold, such as below 40%.

[0081] Furthermore, after extracting features such as fundamental frequency, fluctuation amplitude, MFCC13, spectral entropy, and zero-crossing rate, other relevant features can be extracted, such as ΔF0 (the change in fundamental frequency) and Var (variance). These features of different dimensions are combined to output 18-dimensional or even more-dimensional temporal features as audio features. These multi-dimensional temporal features can more comprehensively and accurately reflect the characteristic information of the audio data, providing strong support for subsequent analysis and judgment.

[0082] In one embodiment, image features in the video data include HSV color features, flame height, and Martian density; extracting image features from the video data includes:

[0083] Step 1: Based on the video data, obtain a single-frame flame image and extract HSV color features;

[0084] Specifically, a video of the flames at the converter nozzle can be captured at a frame rate of 10 FPS. Frame extraction is then performed on this video to obtain a single-frame flame image. Because the RGB color space exhibits strong correlations between its components when describing color, it is not conducive to accurate analysis and extraction of flame color features. The HSV color space, however, is closer to human color perception and can more accurately describe flame color. Therefore, the extracted flame image frames are converted from the RGB color space to the HSV color space, and the dominant hue (H), saturation (S), and brightness (V) components are extracted, laying the foundation for subsequent accurate color feature extraction.

[0085] Step 2: Perform edge detection on a single frame of the flame image to highlight the flame outline, and extract the pixel coordinates of the top of the flame outline to determine the flame height.

[0086] Specifically, the Canny edge detection algorithm can be used first to process the flame image frame, detecting edge information in the image to highlight the flame outline. Then, the Hough transform is used to further analyze the detected flame outline, extracting the pixel coordinates of the top of the flame outline, and then calculating the flame height based on the pixel coordinates of the top of the flame outline.

[0087] Step 3: Perform motion analysis on continuous flame image frames, identify the Martians in the preset motion state, and count the number of Martians per unit area to obtain the Martian density.

[0088] Specifically, the Lucas-Kanade optical flow method can be used to perform motion analysis on consecutive flame image frames. This algorithm can calculate the motion vectors of pixels in the image, and by analyzing the motion vectors, small bright spots moving at high speeds, i.e., sparks, can be identified. This method can accurately capture the motion information of sparks, providing a basis for subsequent spark density statistics. The number of sparks per unit area can be counted, specifically when the number of sparks per unit area is <5 / m². 2 At that time, it was determined to be an anomaly in Martian density.

[0089] In this embodiment, a preferred embodiment for extracting video features is given. Similarly, HSV color features, flame height, and Martian density are not conventional choices, but important selections made based on the study of the drying state, for the following reasons:

[0090] 1. Under normal furnace flame conditions, the flame is orange-red, and the value range of the dominant hue H component is H∈[0,30]. When back-drying occurs, due to the thickening of the slag layer and insufficient oxygen contact, the flame color turns dark red, and the dominant hue H component decreases, with the value range becoming H∈[0,10]. To facilitate timely warning, the warning threshold can be set to an average H value < 12. By analyzing the dominant hue H component, the change in flame color can be intuitively understood, thereby determining whether there is an abnormality in the furnace condition. Under normal flame combustion conditions, the average value of the saturation S component is approximately 0.6. When back-drying occurs, the flame combustion is incomplete, and the average value of the saturation S component will decrease. A threshold condition of below 0.4 can be selected for the judgment scheme. The change in the saturation S component can reflect the degree of flame combustion and provide an important basis for judging the back-drying state. More preferably, brightness V attenuation can be selected. For example, when the overall flame brightness attenuates by more than 20% and lasts for more than 5 frames, it indicates that the flame condition may be abnormal. Changes in the brightness V component can reflect changes in the flame's energy. Combined with the determination of the duration of the flame, it can more accurately determine whether there are any abnormalities in the flame's state.

[0091] 2. The flame's pixel height is approximately 400px, corresponding to an actual height of 2.0m. When re-drying occurs, the flame height decreases, with the pixel height dropping to <250px, corresponding to an actual height of 1.2m. By calculating the pixel coordinates of the flame's top edge, accurate flame height information can be obtained, providing crucial data for analyzing flame morphology. Optionally, a re-drying warning can be issued when the flame drops to a set height threshold.

[0092] 3. Changes in spark density can reflect the intensity of the reaction inside the furnace. Combined with other characteristic parameters, it can provide a more comprehensive assessment of whether a drying-back situation has occurred inside the furnace. Optionally, a drying-back warning can be issued if the change in spark density exceeds a set threshold.

[0093] Furthermore, in addition to counting the number of Martian trajectories, the length of their trajectories can also be analyzed. For example, when the average pixel length of a Martian trajectory is less than 10 pixels, it is considered that the Martian trajectories are sparse. Changes in the length of the Martian trajectories can also reflect the state of the reaction inside the furnace, providing auxiliary information for judging the drying state.

[0094] In one embodiment, optimization processing is performed based on the degree of re-drying, resulting in optimization data including:

[0095] Step 1: Obtain the lance height and slag dosage in the current converter steelmaking process.

[0096] In the converter steelmaking process, sensors and monitoring equipment installed on the converter are used to acquire real-time data on the current lance height (current_lance) and the amount of slag added (slag_amount). Lance height refers to the vertical position of the oxygen lance within the converter, which significantly impacts the reaction within the furnace; slag amount is the quantity of material used to improve slag properties and promote smooth reactions within the furnace. Accurately obtaining these two parameters is fundamental for subsequent optimization of treatment measures.

[0097] Step 2: Obtain the gun position adjustment amount based on the degree of dryness, gun position height, and the preset linear function of dryness degree - gun position height.

[0098] In the converter steelmaking process, there is a specific linear relationship between the degree of re-drying (D) and the lance position adjustment (ΔH), which is represented by a preset linear function of the degree of re-drying and the lance height. More specifically, the formula is: ΔH = 0.5 + 0.8D - 0.1 × current_lance. In this formula, 0.5 is a base adjustment, 0.8 is the influence coefficient of the degree of re-drying on the lance position adjustment, and -0.1 is the correction coefficient of the current lance height on the lance position adjustment. Using this formula, the lance position adjustment can be accurately calculated based on the real-time acquired degree of re-drying and the current lance height. Furthermore, the design of this formula ensures that the lower the current lance position, the smaller the lance position drop, while setting a safety boundary to keep the lance position adjustment within the range of 0.5-1.5m, thus guaranteeing the safety and stability of the converter steelmaking process.

[0099] Step 3: Based on the degree of re-drying, the slag dosage, and the preset piecewise linear function of the degree of re-drying and the slag dosage, obtain the adjustment amount of the slag dosage.

[0100] The relationship between the degree of re-drying (D) and the slag dosage adjustment (ΔM) is determined by a preset piecewise linear function of the degree of re-drying and the slag dosage. More specifically, the formula is: ΔM = (50 + 200D) × (slag_amount / 1000). According to this formula, the slag dosage adjustment varies depending on the degree of re-drying. In practical applications, re-drying can be categorized into light, moderate, and severe re-drying. For light re-drying, the degree of re-drying (D) is relatively small, and the calculated slag dosage adjustment is between 30-50 kg; for moderate re-drying, D increases, and the slag dosage adjustment is between 80-120 kg; for severe re-drying, D further increases, and the slag dosage adjustment is between 150-200 kg. Furthermore, when determining the specific material of the slag agent, selection is based on inventory cost. When fluorite is inexpensive, its proportion is set to 70%; when dolomite is inexpensive, its proportion is also set to 70%.

[0101] Furthermore, after calculating the adjustments for lance position and slag dosage, these adjustments are applied to the converter steelmaking process. 30-40 seconds after the adjustments are implemented, the control effect is monitored by observing indicators such as audio frequency and flame color. If the audio frequency rises above 800Hz and the flame color returns to orange-red, the adjustments are considered effective, forming a closed-loop optimization. If the expected effect is not achieved, the degree of re-drying needs to be reassessed, and the adjustments need to be optimized again.

[0102] In one embodiment, the training process of the multimodal fusion and multi-task output model includes:

[0103] Step 1: Obtain the initial multimodal fusion and multi-task output model and training sample data; the initial multimodal fusion and multi-task output model includes the initial audio branch neural network, the initial image branch neural network, the initial attention fusion layer, and the initial multi-task output head.

[0104] The initial multimodal fusion and multi-task output model consists of an initial audio branch neural network, an initial image branch neural network, an initial attention fusion layer, and an initial multi-task output head. The audio branch neural network processes audio modal data and extracts audio features related to the drying process in converter steelmaking; the image branch neural network processes image modal data and extracts image features related to the drying process; the initial attention fusion layer fuses audio and image features, enabling the model to comprehensively utilize multimodal information; and the initial multi-task output head is responsible for outputting multiple drying-related task results, such as drying warning, drying degree, lance height adjustment, and slag dosage adjustment. The training sample data includes audio data, image data, and corresponding annotation information. Data annotation can be performed by experienced personnel.

[0105] Step 2: Train the initial audio branch neural network and the initial image branch neural network separately until convergence based on the training sample data, and freeze the encoder parameters of the trained audio branch neural network and image branch neural network.

[0106] A three-stage progressive training strategy is adopted, starting with single-modal pre-training. Audio training sample data is input into the initial audio branch neural network, and image training sample data is input into the initial image branch neural network. The two branch networks are trained separately using labeled information until they converge. During training, the audio branch neural network learns to extract features relevant to the audio data, and the image branch neural network learns to extract relevant features from the image data. After training convergence, the encoder parameters of both branch networks are frozen. The encoder is responsible for converting the raw input data into feature representations; freezing its parameters ensures that the learned effective feature extraction capabilities are not compromised during subsequent training, while also reducing the computational load and complexity of subsequent training.

[0107] Step 3: Obtain the first sub-loss function, the second sub-loss function, and the third sub-loss function. The first sub-loss function is constructed based on the cross-entropy of the binary classification of the dryness warning. The second sub-loss function is constructed based on the mean square error of the dryness degree regression. The third sub-loss function is constructed based on the gun position height adjustment and the slag dosage adjustment, and is constructed with additional constraint penalties using Huber loss.

[0108] Step 4: Weight the first sub-loss function, the second sub-loss function, and the third sub-loss function to construct the total loss function.

[0109] The three sub-loss functions in step 3 are weighted according to preset weights to construct the total loss function. Specifically, the total loss function can be:

[0110] L_total=0.4·L_warning+0.3·L_degree+0.3·L_action

[0111] In this model, L_warning is the first sub-loss function with a weight of 0.4, reflecting the importance of the anti-interference warning task in the overall model training; L_degree is the second sub-loss function with a weight of 0.3, reflecting the weight of the anti-interference degree regression task; and L_action is the third sub-loss function with a weight of 0.3, representing the weight of the action optimization task. This weighted combination comprehensively considers the importance of the three tasks, enabling the model to balance the learning of each task during training and improve overall performance.

[0112] Step 5: Train the initial attention fusion layer and the initial multi-task output head based on the total loss function and training sample data to obtain the multimodal fusion and multi-task output model.

[0113] After constructing the overall loss function, the initial attention fusion layer and the initial multi-task output head are trained using training sample data. During training, different loss functions are selectively chosen for each of the three sub-loss functions:

[0114] The first sub-loss function is constructed based on the binary cross-entropy of the anti-interference warning system. The anti-interference warning task is a binary classification problem, namely, determining whether an anti-interference has occurred. The binary cross-entropy loss function can measure the difference between the anti-interference probability predicted by the model and the true label. By minimizing this loss function, the model can better learn the features that distinguish between anti-interference and non-anti-interference, thereby improving the accuracy of the anti-interference warning.

[0115] The second sub-loss function is constructed based on the mean squared error of the regression of the degree of re-drying, and heavily weighted samples are weighted (e.g., weight 2.0). The re-drying degree regression task is a continuous value prediction problem, and the mean squared error loss function can measure the difference between the model's predicted degree of re-drying and the actual degree of re-drying. Weighting heavily weighted samples is to make the model pay more attention to the situation of severe re-drying, because in the actual steelmaking process, severe re-drying has a greater impact on production. By increasing the weight of heavily weighted samples in the loss function, the model can focus more on learning and predicting the features of severe re-drying during training, thereby improving its ability to identify and predict severe re-drying.

[0116] The third sub-loss function is constructed based on the gun position height adjustment and slag dosage adjustment, and employs Huber loss with additional constraint penalties. In the optimization task, the prediction of gun position height adjustment and slag dosage adjustment needs to consider the safety limitations and operating procedures in actual production. Huber loss combines the advantages of mean squared error loss and absolute error loss, exhibiting better robustness to outliers. The additional constraint penalty adds a 10-fold penalty term to the loss function when the gun position exceeds the limit, and a 5-fold penalty term when the dosage exceeds the limit. This constraint penalty mechanism guides the model to better conform to the safety requirements and operating procedures of actual production when predicting gun position height adjustment and slag dosage adjustment, avoiding prediction results exceeding reasonable ranges.

[0117] On the other hand, the gradient of the total loss function with respect to the model parameters is calculated using the backpropagation algorithm, and the model parameters are updated using optimization algorithms (such as stochastic gradient descent) to continuously reduce the total loss function. After training, the final multimodal fusion and multi-task output model is obtained. This can further improve the overall performance of the model and allows for end-to-end fine-tuning of the entire network. A small learning rate (such as 1e-5) can be used to fine-tune all the parameters of the model, enabling the model to better adapt to the requirements of multimodal data and multi-task output, thereby improving the model's generalization ability and accuracy.

[0118] In one embodiment, the audio branch neural network includes an audio input layer, a 1D-CNN encoder, and an LSTM temporal modeling layer;

[0119] The audio input layer is used to receive audio features, preprocess them, and output a standardized audio feature sequence.

[0120] The 1D-CNN encoder, connected to the output of the audio input layer, is used to extract local spatial features from the standardized audio feature sequence and output an encoded vector containing local temporal features.

[0121] The LSTM temporal modeling layer, connected to the output of the 1D-CNN encoder layer, is used to model the long and short-term temporal dependencies of the encoded vectors and output audio temporal features that fuse local and temporal features.

[0122] This embodiment presents a preferred embodiment of the audio branch neural network, which mainly consists of an audio input layer, a 1D-CNN encoder, and an LSTM temporal modeling layer. The 1D-CNN encoder is the core feature extraction part of the audio branch neural network, optionally including multiple 1D convolutional layers, pooling layers, and activation function layers. The 1D convolutional layers extract local features from the audio signal by sliding convolutional kernels across the audio time series. Different convolutional kernels can capture different aspects of the audio signal, such as frequency features and rhythm features. The pooling layers downsample the feature maps output by the convolutional layers, reducing the amount of data while retaining the main features, enhancing the model's robustness to small changes in the audio signal. Activation function layers (such as the ReLU function) introduce non-linearity, enabling the model to learn more complex audio feature patterns. Through multiple layers of 1D convolution, pooling, and activation operations, the 1D-CNN encoder can progressively extract representative high-level features from the audio signal, providing a rich feature foundation for subsequent temporal modeling. Because audio signals exhibit significant time-series characteristics, the manifestation of re-entropy in audio is also time-dependent. The LSTM (Long Short-Term Memory) temporal modeling layer can perform temporal modeling of audio features extracted by the 1D-CNN encoder. LSTM networks possess unique gating mechanisms, including input gates, forget gates, and output gates. These gating mechanisms effectively control the flow of information, remembering important information from long time sequences while forgetting irrelevant information. In the audio branch neural network, the LSTM temporal modeling layer can learn the changing patterns of audio features over time, capturing the temporal characteristics of re-entropy in audio.

[0123] More preferably, the image branching neural network includes an image input layer, a 2D-CNN encoder, and a temporal pooling layer;

[0124] The image input layer receives image features, performs preprocessing, and outputs a normalized image feature tensor.

[0125] The 2D-CNN encoder layer is connected to the output of the image input layer and is used to extract multi-level features of the spatial dimension of the image feature tensor, outputting an encoded feature map containing multi-scale spatial semantic information.

[0126] The temporal pooling layer, connected to the output of the 2D-CNN encoder layer, is used to aggregate and reduce the temporal dimension of the encoded feature map, and outputs a spatial-temporal pattern feature that fuses spatial and temporal features.

[0127] This embodiment presents a preferred embodiment of the image branching neural network, which mainly consists of an image input layer, a 2D-CNN encoder, and a temporal pooling layer. The 2D-CNN encoder is the key part of the image branching neural network for extracting image features, and optionally includes multiple 2D convolutional layers, pooling layers, and activation function layers. The 2D convolutional layers extract local features such as edges, textures, and shapes by sliding convolutional kernels across the two-dimensional space of the image. Different sizes and numbers of convolutional kernels can capture features of different levels and types. The pooling layer downsamples the feature maps output by the convolutional layers, reducing the size and computational cost of the feature maps while enhancing the model's robustness to small changes in the image. The activation function layer also introduces nonlinear factors, enabling the model to learn more complex image feature representations. Through multi-layer 2D-CNN encoding operations, key features in the image are gradually extracted, forming a high-level image feature representation, providing a foundation for subsequent temporal processing. In the converter steelmaking process, image data also changes over time; to capture the information of image features in the temporal dimension, a temporal pooling layer is set up. Temporal pooling layers perform pooling operations on the image feature sequences output by the 2D-CNN encoder, such as max pooling or average pooling, to integrate and compress image features over a period of time. Through temporal pooling, the main trends and key information of image features over time can be extracted, reducing the dimensionality of the data while retaining important temporal features related to the desiccation phenomenon, so as to facilitate subsequent fusion with audio features and multi-task output.

[0128] To illustrate the technical solution of the early warning method for re-drying in converter steelmaking as described in this application, specific application examples will be used below, combined with... Figure 3 Expand the introduction. For example... Figure 3 As shown, in one specific application example, the method for early warning of dryness in converter steelmaking according to this application includes the following steps:

[0129] Step 1: Real-time acquisition of the audio stream inside the converter at a sampling rate of 8000Hz, and simultaneous acquisition of the flame video at the converter opening at a frame rate of 10FPS (Frames Per Second).

[0130] Step 2: Divide the acquired audio stream into 256ms frames and extract the frames from the flame video.

[0131] Step 3: Extract MFCC13, ΔF0, Var and other features from the framed audio, and output 18-dimensional temporal features; extract HSV color space features, height features and Mars density features from the extracted image frames, and output 8-dimensional image features per frame.

[0132] Step 4: Use the audio branch composed of 1D-CNN and LSTM to extract temporal dynamic information from the 18-dimensional temporal features of the audio; use the image branch composed of 2D-CNN and temporal pooling to extract spatial-temporal pattern information from the 8-dimensional / frame features of the image.

[0133] Step 5: Calculate the audio weight β_audio and image weight β_image in the attention fusion layer, and fuse the audio and image information by fusing the fusion vector = β_a (audio) + β_i (image).

[0134] Step 6: Execute Task 1, perform binary classification output for backlash warning, and obtain the warning probability P_warning∈[0,1].

[0135] Step 7: Execute Task 2, perform regression output of the degree of return, obtain the degree D∈[0,1], and classify it according to the degree: when D<0.3 it is mild, when 0.3≤D<0.6 it is moderate, and when D≥0.6 it is severe.

[0136] Step 8: Perform Task 3, optimize the measures, and determine the gun position adjustment amount ΔH (m) and the slag-reducing agent dosage ΔM (kg).

[0137] Step 9: When the drying conditions are met, the system automatically performs gun position adjustment and slag agent replenishment operations based on the calculated gun position adjustment amount and slag agent dosage.

[0138] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0139] Based on the same inventive concept, this application also provides a converter steelmaking dryness early warning device for implementing the aforementioned converter steelmaking dryness early warning method. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations of one or more converter steelmaking dryness early warning device embodiments provided below can be found in the above-described limitations of the converter steelmaking dryness early warning method, and will not be repeated here.

[0140] In one embodiment, such as Figure 4As shown, a re-drying early warning device for converter steelmaking is provided, comprising:

[0141] Data acquisition module 200 is used to acquire audio and video data inside the converter;

[0142] The feature extraction module 400 is used to extract audio features from audio data and image features from video data;

[0143] The data input module 600 is used to input audio features and image features into the multimodal fusion and multi-task output model, respectively; the multimodal fusion and multi-task output model includes an audio branch neural network, an image branch neural network, an attention fusion layer, and a multi-task output head;

[0144] The model calling module 800 is used to call the multimodal fusion and multi-task output model to process audio and image features to obtain the probability of drought return warning, the degree of drought return, and data on measures optimization.

[0145] The multimodal fusion and multi-task output model processing includes: an audio branch neural network extracts audio temporal features based on audio features, and an image branch neural network extracts spatial-temporal pattern features; an attention fusion layer calculates audio attention weights and image attention weights based on the real-time signal-to-noise ratio of audio and video data, and calculates a fusion vector of audio temporal features and spatial-temporal pattern features based on the audio attention weights and image attention weights; a multi-task output head performs binary classification output for dryness warning based on the fusion vector to obtain the dryness warning probability; a dryness degree regression output is performed based on the fusion vector to obtain the dryness degree; and a measure optimization process is performed based on the dryness degree to obtain measure optimization data, which includes adjusting the gun position height and adjusting the slag dosage.

[0146] In one embodiment, the feature extraction module 400 is further used to segment the audio data into frames to obtain audio frame data; and to extract multi-dimensional temporal audio features from the audio frame data, including fundamental frequency, fluctuation amplitude, MFCC13, spectral entropy and zero-crossing rate.

[0147] In one embodiment, the image features in the video data include HSV color features, flame height, and spark density. The feature extraction module 400 is also used to convert the flame image frames in the video data from the RGB color space to the HSV color space; extract the HSV color features, including the dominant hue (H component), saturation (S component), and brightness (V component), respectively, in the HSV color space; perform Canny edge detection on the flame image frames to highlight the flame outline, and extract the top pixel coordinates of the flame outline using Hough transform, and calculate the flame height based on the top pixel coordinates of the flame outline; perform motion analysis on the continuous flame image frames using optical flow method, identify sparks in a preset motion state, and count the number of sparks per unit area to obtain the spark density.

[0148] In one embodiment, the model calling module 800 is also used to obtain the lance position height and slag dosage in the current converter steelmaking; and to obtain the lance position adjustment amount based on the degree of drying, the lance position height, and a preset linear function of the degree of drying and the lance position height.

[0149] The adjustment amount of slag dosage is obtained based on the degree of re-drying, the dosage of slag, and a preset piecewise linear function of the degree of re-drying and the dosage of slag.

[0150] In one embodiment, the training process of the multimodal fusion and multi-task output model includes:

[0151] Obtain the initial multimodal fusion and multi-task output model and training sample data; the initial multimodal fusion and multi-task output model includes the initial audio branch neural network, the initial image branch neural network, the initial attention fusion layer, and the initial multi-task output head;

[0152] The initial audio branch neural network and the initial image branch neural network are trained separately based on the training sample data until convergence, and the encoder parameters of the trained audio branch neural network and image branch neural network are frozen.

[0153] The first sub-loss function, the second sub-loss function, and the third sub-loss function are obtained. The first sub-loss function is constructed based on the cross-entropy of the binary classification of the dryness warning. The second sub-loss function is constructed based on the mean square error of the dryness degree regression. The third sub-loss function is constructed based on the gun position height adjustment and the slag dosage adjustment, and is constructed with additional constraint penalty using Huber loss.

[0154] The first, second, and third sub-loss functions are weighted to construct the total loss function;

[0155] The initial attention fusion layer and initial multi-task output head are trained based on the total loss function and training sample data to obtain the multimodal fusion and multi-task output model.

[0156] In one embodiment, the audio branch neural network includes an audio input layer, a 1D-CNN encoder, and an LSTM temporal modeling layer; the image branch neural network includes an image input layer, a 2D-CNN encoder, and a temporal pooling layer.

[0157] The modules in the aforementioned converter steelmaking dryness early warning device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the computer device's memory as software, so that the processor can call and execute the corresponding operations of each module.

[0158] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 5 As shown, the computer device includes a processor, memory, and network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database stores preset data. The network interface communicates with external terminals via a network connection. When executed by the processor, the computer program implements a method for early warning of dryness in converter steelmaking.

[0159] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0160] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the above-described method for early warning of dryness in converter steelmaking.

[0161] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the aforementioned method for early warning of dryness in converter steelmaking.

[0162] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the aforementioned method for early warning of dryness in converter steelmaking.

[0163] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0164] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0165] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for early warning of dryness during converter steelmaking, characterized in that, The method includes: acquiring audio and video data from the converter; extracting audio features from the audio data and image features from the video data; inputting the audio features and image features into a multimodal fusion and multi-task output model, respectively; the multimodal fusion and multi-task output model includes an audio branch neural network, an image branch neural network, an attention fusion layer, and a multi-task output head; calling the multimodal fusion and multi-task output model to process the audio features and image features to obtain the probability of dryness recurrence warning, the degree of dryness recurrence, and data for optimizing measures; The multimodal fusion and multi-task output model processing includes: an audio branch neural network extracts audio temporal features based on audio features, and an image branch neural network extracts spatial-temporal pattern features; an attention fusion layer calculates audio attention weights and image attention weights based on the real-time signal-to-noise ratio of audio and video data, and calculates a fusion vector of audio temporal features and spatial-temporal pattern features based on the audio attention weights and image attention weights; a multi-task output head performs binary classification output for dryness warning based on the fusion vector to obtain the dryness warning probability; a dryness degree regression output is performed based on the fusion vector to obtain the dryness degree; and a measure optimization process is performed based on the dryness degree to obtain measure optimization data, including adjustment of gun position height and slag dosage. Extracting audio features from audio data includes: segmenting the audio data into frames to obtain audio frame data; and extracting one or more of the fundamental frequency, fluctuation amplitude, MFCC, spectral entropy, and zero-crossing rate from the audio frame data to construct multi-dimensional time-series features as audio features. Image features in video data include HSV color features, flame height, and spark density. Extracting image features from video data includes: acquiring single-frame flame images based on video data and extracting HSV color features; performing edge detection on single-frame flame images to highlight the flame outline and extracting the pixel coordinates of the top of the flame outline to determine the flame height; performing motion analysis on continuous flame image frames to identify sparks in a preset motion state and counting the number of sparks per unit area to obtain the spark density. The optimization of measures based on the degree of re-drying results in the following data: obtaining the lance height and slag dosage in the current converter steelmaking process; obtaining the lance position adjustment amount based on the degree of re-drying, lance height, and a preset linear function of the degree of re-drying and lance height; and obtaining the slag dosage adjustment amount based on the degree of re-drying, slag dosage, and a preset piecewise linear function of the degree of re-drying and slag dosage.

2. The method according to claim 1, characterized in that, The audio features extracted from the audio data are specifically as follows: extracting the fundamental frequency decrease feature or the fundamental frequency being lower than the first set threshold to issue a back-dampening warning; extracting the fluctuation amplitude increase or the fluctuation exceeding the second set threshold to issue a back-dampening warning; extracting the high-frequency component attenuation feature or the high-frequency component being lower than the third set threshold to issue a back-dampening warning; extracting the spectral entropy value decrease feature or the spectral entropy value being lower than the fourth set threshold to issue a back-dampening warning; and extracting the zero-crossing rate decrease feature or the zero-crossing rate being lower than the fifth set threshold to issue a back-dampening warning.

3. The method according to claim 1, characterized in that, The specific image features extracted from the video data are as follows: the mean value of the dominant color H is <12, which is set as the desiccation warning threshold; the mean value of the saturation S component drops to below 0.4, which is determined as the desiccation warning threshold. When the brightness V decreases by more than 20% and persists for more than 5 frames, it is determined to be at the anti-dryness warning threshold.

4. The method according to claim 1, characterized in that, The training process of a multimodal fusion and multi-task output model includes: Obtain the initial multimodal fusion and multi-task output model and training sample data; the initial multimodal fusion and multi-task output model includes the initial audio branch neural network, the initial image branch neural network, the initial attention fusion layer, and the initial multi-task output head; The initial audio branch neural network and the initial image branch neural network are trained separately based on the training sample data until convergence, and the encoder parameters of the trained audio branch neural network and image branch neural network are frozen. The first sub-loss function, the second sub-loss function, and the third sub-loss function are obtained. The first sub-loss function is constructed based on the cross-entropy of the binary classification of the dryness warning. The second sub-loss function is constructed based on the mean square error of the dryness degree regression. The third sub-loss function is constructed based on the gun position height adjustment and the slag dosage adjustment, and is constructed with additional constraint penalty using Huber loss. The first, second, and third sub-loss functions are weighted to construct the total loss function; The initial attention fusion layer and initial multi-task output head are trained based on the total loss function and training sample data to obtain the multimodal fusion and multi-task output model.

5. The method according to claim 1, characterized in that, The audio branch neural network includes an audio input layer, a 1D-CNN encoder, and an LSTM temporal modeling layer; The audio input layer is used to receive audio features, preprocess them, and output a standardized audio feature sequence. The 1D-CNN encoder, connected to the output of the audio input layer, is used to extract local spatial features from the standardized audio feature sequence and output an encoded vector containing local temporal features. The LSTM temporal modeling layer, connected to the output of the 1D-CNN encoder layer, is used to model the long and short-term temporal dependencies of the encoded vectors and output audio temporal features that fuse local and temporal features.

6. The method according to claim 1, characterized in that, Image branching neural networks consist of an image input layer, a 2D-CNN encoder, and a temporal pooling layer; The image input layer receives image features, performs preprocessing, and outputs a normalized image feature tensor. The 2D-CNN encoder layer is connected to the output of the image input layer and is used to extract multi-level features of the spatial dimension of the image feature tensor, outputting an encoded feature map containing multi-scale spatial semantic information. The temporal pooling layer, connected to the output of the 2D-CNN encoder layer, is used to aggregate and reduce the temporal dimension of the encoded feature map, and outputs a spatial-temporal pattern feature that fuses spatial and temporal features.

7. A re-drying early warning device in converter steelmaking, characterized in that, The apparatus for performing the method according to any one of claims 1 to 6 comprises: The data acquisition module is used to acquire audio and video data inside the converter; The feature extraction module is used to extract audio features from audio data and image features from video data. The data input module is used to input audio features and image features into the multimodal fusion and multi-task output model, respectively; the multimodal fusion and multi-task output model includes an audio branch neural network, an image branch neural network, an attention fusion layer, and a multi-task output head; The model invocation module is used to invoke the multimodal fusion and multi-task output model to process audio and image features to obtain the probability of drought return warning, the degree of drought return, and data on optimized measures. The multimodal fusion and multi-task output model processing includes: an audio branch neural network extracts audio temporal features based on audio features, and an image branch neural network extracts spatial-temporal pattern features; an attention fusion layer calculates audio attention weights and image attention weights based on the real-time signal-to-noise ratio of audio and video data, and calculates a fusion vector of audio temporal features and spatial-temporal pattern features based on the audio attention weights and image attention weights; a multi-task output head performs binary classification output for dryness warning based on the fusion vector to obtain the dryness warning probability; a dryness degree regression output is performed based on the fusion vector to obtain the dryness degree; and a measure optimization process is performed based on the dryness degree to obtain measure optimization data, which includes adjusting the gun position height and adjusting the slag dosage.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images