An AI-based automatic optimization method and device for converter smelting process

Abstract

The present invention relates to an automatic optimization method and device for a converter smelting process based on AI, and to the technical field of converter smelting control. The present application acquires RGB images and infrared images in a converter during the smelting process frame by frame; collects audio data during the smelting process; splices the collected RGB images and infrared images along the channel dimension to obtain visual features, slices the audio data by frame, and converts each slice into a Mel-spectrogram as an audio feature; inputs the visual features and audio features into a pre-trained converter state monitoring model, and the converter state monitoring model outputs a state tensor representing the converter state based on the visual features and audio features; provides the state tensor to a pre-trained AI agent, which is trained based on a DQN reinforcement learning architecture, and is used to select an action that maximizes the Q value under the current state tensor, which is executed by the spiral filler and oxygen gun of the converter.

Description

Technical Field

[0001] The present invention relates to the technical field of converter smelting control, and in particular to an AI-based automatic optimization method and device for a converter smelting process. Background Art

[0002] The converter is a steelmaking equipment, mainly used to make steel from pig iron. Its structural characteristics are cylindrical, tiltable, and mounted on a horizontal axis frame, which can rotate. In the process of converter steelmaking, high-pressure air or oxygen is usually blown into the furnace from the top, bottom or side of the converter to oxidize and remove impurities in the pig iron, thereby obtaining pure molten steel. The specific steps of converter steelmaking include: injecting 1300 degrees Celsius liquid pig iron into the converter, adding a certain amount of quicklime, and then blowing air into the converter and turning it to make it upright. At this time, the reaction on the surface of the liquid pig iron will produce slag, and the reaction will spread throughout the furnace through the convection of molten steel and slag. During the smelting process, carbon in the steel is oxidized to produce carbon monoxide, causing the molten steel to boil violently. Phosphorus is oxidized to produce ferrous phosphate, which further reacts with quicklime to produce stable calcium phosphate and calcium sulfide, which become slag. When phosphorus and sulfur gradually decrease and the flame fades, it indicates that the steel has been smelted. At this time, the blast should be stopped immediately, the converter should be turned to a horizontal position, and deoxidizer should be added for deoxidation. During the converter smelting process, it is necessary to control the addition of quicklime, deoxidizer and other reagents according to the smelting status, and control the height, oxygen flow rate and pressure of the oxygen gun. Existing technologies mostly use digital twins to sense the converter status based on audio equipment and temperature sensors. It is difficult for digital twins to analyze the status through vision, and the stability and accuracy of the perceived status are poor. Summary of the invention

[0003] In order to solve the above technical problems or at least partially solve the above technical problems, the present invention provides an AI-based automatic optimization method and equipment for converter smelting process.

[0004] In a first aspect, the present invention provides an AI-based automatic optimization method for a converter smelting process, comprising: acquiring RGB images and infrared images of the converter during the smelting process frame by frame through a coaxial imaging industrial camera and an infrared camera; collecting audio data during the smelting process through an audio device;

[0005] The collected RGB images and infrared images are spliced ​​along the channel dimension to obtain visual features, the audio data is sliced ​​by frame, and each slice is converted into a Mel-spectrogram as an audio feature;

[0006] Inputting the visual features and the audio features into a pre-trained converter state monitoring model, wherein the converter state monitoring model outputs a state tensor representing the converter state based on the visual features and the audio features;

[0007] The state tensor is provided to a pre-trained AI agent, which is trained based on the DQN reinforcement learning architecture and is used to select the action that maximizes the Q value under the current state tensor, which is executed by the spiral feeder and oxygen lance of the converter. The converter smelting actions include: the amount of material added to the spiral feeder during the smelting process, the height of the oxygen lance, and the oxygen lance flow and pressure.

[0008] Furthermore, the converter state monitoring model includes: the converter state monitoring model includes: a visual feature extraction module, an audio feature extraction module, two self-attention Transformers based on a multi-head self-attention module, two cross-attention Transformers based on a cross-attention module and two multi-layer perceptrons; the two self-attention Transformers are used to extract the internal correlation of visual features and audio features, and then the two cross-attention Transformers are used to extract the correlation between audio features and visual features; the two cross-attention Transformers are respectively processed and fused by two multi-layer perceptrons, and the tensor representing the converter state is obtained through activation function and normalization processing.

[0009] Furthermore, the visual feature extraction module includes: a convolution-based patch embedding layer and a dilated convolution-based patch embedding layer; the patch embedding layer and the dilated convolution-based patch embedding layer process the visual features into two groups of patches from different scales respectively; the two groups of patches are processed by a linear layer to obtain the embedded features of the visual features, the embedded features of the visual features are added and combined with the position code, and then mapped into , , Three vectors, , , The three vectors are input into the self-attention transformer of the visual feature extraction module to model the internal association of the visual features, wherein the self-attention transformer adopts a multi-head self-attention mechanism module; the output features of the multi-head self-attention mechanism module are mapped into , , Three vectors; among them, , and the vector obtained based on audio features , exchange, Cross-attention is performed through the cross-attention Transformer, that is:

[0010] ; is the visual cross-attention feature, is the cross attention function of visual features.

[0011] Furthermore, the audio feature extraction module includes: a convolution-based patch embedding layer; the patch embedding layer processes the mel spectrogram as the audio feature into a patch, processes the patch through a linear layer to obtain an embedded feature of the audio feature, adds and combines the embedded feature of the audio feature with the position code, and then maps it into , , Three vectors, , , The three vectors are input into the self-attention transformer of the audio feature extraction module to model the internal association of the audio features, which is actually to model the association between the time-frequency features of the Mel spectrogram. The self-attention transformer adopts a multi-head self-attention mechanism module; the output features of the multi-head self-attention mechanism module are mapped into , , Three vectors; among them, , And the vector obtained based on video features , exchange, Cross-attention is performed through the cross-attention Transformer, that is:

[0012] ; is the audio cross-attention feature, is the cross attention function of audio features.

[0013] Furthermore, during the training process, a state type fully connected network is additionally introduced after standardization; during training, the visual features and mel-spectrograms in the training data are randomly input into the converter state monitoring model, the converter state monitoring model predicts the corresponding state tensor, the state tensor is input into the state type fully connected network, and the state type fully connected network obtains the predicted converter state label based on the state tensor; the cross entropy between the predicted converter state label and the true converter state label is used as the loss function, and based on the loss function, the parameters of the converter state monitoring model and the state type fully connected network are updated using the gradient descent algorithm, so that the loss function of the data in the training data set is stably lower than the set threshold.

[0014] Furthermore, the state tensor is used as the state in the DQN reinforcement learning architecture;

[0015] Use the converter, oxygen lance, and screw feeder as environments in the DQN reinforcement learning architecture;

[0016] Constructing an AI agent, the AI ​​agent comprising: a Q-value network and a target Q-value network based on a fully connected network, wherein parameters of the target Q-value network are periodically synchronized from the Q-value network; the Q-value network maps a state tensor to a Q-value, wherein the dimension of the Q-value is the number of actions that the AI ​​agent can decide;

[0017] Initialize the experience replay pool and Q value table; the experience replay pool stores the experience of the AI ​​agent interacting with the environment, including the current state tensor , current moment action ,award and the next moment state tensor The Q value table records the Q value of any action performed under any state tensor;

[0018] Among them, rewards , for The upper limit of the range of feasible states set in advance during the smelting stage; for The lower limit of the pre-set feasible state range in the smelting stage; represents the Euclidean distance.

[0019] Furthermore, the process of training the AI ​​agent includes:

[0020] Initialize Q value network parameters , initialize the parameters of the target Q value network ;

[0021] Iterate until the predetermined number of training rounds is reached or the loss function is less than the set threshold:

[0022] Use the Q value network to calculate the state tensor at the current moment The maximum Q value of the optional action;

[0023] Greedy algorithm based on probability Choose the action with the largest Q value: , with probability Randomly select actions ,in, It is the set of actions that the AI ​​agent can decide;

[0024] In the environment, perform the selected action , get the reward and the state tensor of the next moment , the current state tensor , current moment action ,award and the next moment state tensor The composed experience tuple is stored in the experience replay pool;

[0025] Randomly extract an experience tuple from the experience replay pool, and calculate the next state in the selected experience tuple through the target Q value network Maximum Q value:

[0026] If the next state in the experience tuple is the terminal state, then the target value is equal to the reward in the experience tuple ; Otherwise, the target value is equal to the reward in the experience tuple Add the attenuation factor Multiply the target Q value by the next state predicted by the network The maximum Q value of

[0027] Within the range of the extracted experience tuples, the mean square error between the target value and the predicted value of the Q-value network is used as the loss function, and the gradient descent algorithm is used to update the parameters of the Q-value network.

[0028] Furthermore, a counter is set for each possible action, with the initial value set to 0;

[0029] At each time step, an action is chosen according to the following rule:

[0030] By probability Randomly select an action;

[0031] By probability , select the action with the largest Q value. If there are multiple actions with the highest Q value, select them based on the counter of each action. The probability of each action being selected is ,in is the counter value of the action;

[0032] Each time an action is selected, increase the counter value of that action.

[0033] In the second aspect, the present invention provides an automatic optimization device for a converter smelting process based on an AI intelligent agent, comprising: at least one processing unit, the processing unit being connected to a storage unit, an acquisition unit and an execution unit through a bus unit, the acquisition unit comprising an industrial camera and an infrared camera for coaxial imaging, and an audio device; the execution unit comprising: a spiral filler and an oxygen gun; the storage unit stores a computer program, and when the computer program is executed by the processing unit, the automatic optimization method for the converter smelting process based on AI is implemented.

[0034] In a third aspect, the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the AI-based automatic optimization method for the converter smelting process is implemented.

[0035] The above technical solution provided by the embodiment of the present invention has the following advantages compared with the prior art:

[0036] This application is to build an AI agent that controls the smelting action of a converter. First, a converter state monitoring model is created to model the connection between visual features and audio features and the state tensor. The state tensor represents the converter state; the visual features come from the combination of RGB images and infrared images in the converter, and the audio features come from the converter audio data. After the converter state is vectorized by the converter state monitoring model, the state tensor is used as the state in the DQN reinforcement learning architecture to train the AI ​​agent, so that the AI ​​agent selects the action that maximizes the Q value under the current state tensor, thereby realizing automatic optimization of the converter smelting process.

[0037] In the converter state monitoring model, two self-attention transformers are used to extract the internal associations of visual features and audio features respectively, and then two cross-attention transformers are used to extract the associations between audio features and visual features; the correlation between the visual features and audio features of the converter is better modeled to better obtain the state of the converter smelting process through visual and audio features. In the visual feature extraction module, the patch embedding layer and the dilated convolution patch embedding layer process the visual features into two groups of patches from different scales. The two groups of patches contain feature combinations of different scale spans in the visual features. The patches of the dilated convolution patch embedding layer span the adjacent patches of the patch embedding layer. Combining the different scale span features of the two patches can more strongly model the connection between the corresponding positions of each patch in the visual features. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0039] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0040] Figure 1 A flowchart of an AI-based automatic optimization method for a converter smelting process provided by an embodiment of the present invention;

[0041] Figure 2 A schematic diagram of a converter state monitoring model provided by an embodiment of the present invention;

[0042] Figure 3A schematic diagram of a patch embedding layer and a dilated convolution patch embedding layer in a visual feature extraction module provided in an embodiment of the present invention;

[0043] Figure 4 A flow chart of a converter state monitoring model for training provided by an embodiment of the present invention;

[0044] Figure 5 A schematic diagram of a DQN reinforcement learning architecture of an AI agent provided in an embodiment of the present invention;

[0045] Figure 6 A schematic diagram of an automatic optimization device for a converter smelting process based on an AI agent provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0048] Example 1

[0049] like Figure 1 As shown, the technology of the present invention realizes an AI-based automatic optimization method for converter smelting process, including:

[0050] The RGB images and infrared images inside the converter during the smelting process are acquired frame by frame through a coaxial imaging industrial camera and an infrared camera; audio data during the smelting process is collected through audio equipment.

[0051] The collected RGB images and infrared images are spliced ​​along the channel dimension to obtain visual features, the audio data is sliced ​​by frame, and each slice is converted into a Mel-spectrogram as an audio feature, and the visual features and audio features are input into a pre-trained converter state monitoring model, which outputs a state tensor representing the converter state based on the visual features and audio features.

[0052] In the specific implementation process, Figure 2 As shown, during the application process, the converter state monitoring model includes: a visual feature extraction module, an audio feature extraction module, two self-attention Transformers based on a multi-head self-attention module, two cross-attention Transformers based on a cross-attention module, and two multi-layer perceptrons.

[0053] The visual feature extraction module includes: a convolution-based patch embedding layer and a dilated convolution patch embedding layer based on dilated convolution; the patch embedding layer and the dilated convolution patch embedding layer process the visual features into two groups of patches from different scales respectively, and the two groups of patches contain feature combinations of different scale spans in the visual features, wherein the patches of the dilated convolution patch embedding layer span the adjacent patches of the patch embedding layer; Figure 3 As shown in Figure 1, the convolution kernel size of the convolution layer of the patch embedding layer is 3×3, and the step size is always 3. The convolution kernel size of the convolution layer of the dilated convolution patch embedding layer is 3×3, and the dilation rate is 2. The step sizes in both the width and height directions are changed, and the step sizes are 1, 5, and 1 respectively. The two groups of patches are processed by the linear layer to obtain the embedded features of the visual features, and the embedded features of the visual features are combined with the position encoding, and then mapped into , , Three vectors, , , The three vectors are input into the self-attention transformer of the visual feature extraction module to model the internal association of the visual features, wherein the self-attention transformer adopts a multi-head self-attention mechanism module; the output features of the multi-head self-attention mechanism module are mapped into , , Three vectors; among them, , and the vector obtained based on audio features , exchange, Cross-attention is performed through the cross-attention Transformer, that is:

[0054] ; is the visual cross-attention feature, is the cross attention function of the visual features; through the cross attention Transformer , , Attention calculation is performed between the two layers, and the visual features are strengthened through the attention points of the audio features, so as to establish the association between the video features and the audio features.

[0055] The audio feature extraction module includes: a convolution-based patch embedding layer; the patch embedding layer processes the Mel spectrogram as the audio feature into a patch, processes the patch through a linear layer to obtain the embedded features of the audio feature, adds the embedded features of the audio feature to the position code, and then maps them into , , Three vectors, , , The three vectors are input into the self-attention transformer of the audio feature extraction module to model the internal association of the audio features, which is actually to model the association between the time-frequency features of the Mel spectrogram. The self-attention transformer adopts a multi-head self-attention mechanism module; the output features of the multi-head self-attention mechanism module are mapped into , , Three vectors; among them, , And the vector obtained based on video features , exchange, Cross-attention is performed through the cross-attention Transformer, that is:

[0056] ; is the audio cross-attention feature, is the cross attention function of the audio feature; through the cross attention Transformer , , Attention calculation is performed between the two methods, and the audio features are strengthened through the attention points of the visual features, so as to establish the association between the video features and the audio features.

[0057] Visual cross-attention features and audio cross-attention features They are sent to the multi-layer perceptron respectively, and the multi-layer perceptron performs visual cross-attention features after internal association and cross-reinforcement. and audio cross-attention features For further processing, the hidden connections between modeling features are added and fused, and then activated by the tanh activation function, and finally standardized to obtain the state vector representing the converter state.

[0058] Two self-attention Transformers are used to extract the internal correlations of visual features and audio features, and then two cross-attention Transformers are used to extract the correlations between audio features and visual features. The two cross-attention Transformers are fused through two multi-layer perceptrons respectively, and the state tensor representing the state of the converter is obtained after activation function and normalization.

[0059] like Figure 4 As shown in Figure 1, the process of training the converter condition monitoring model includes:

[0060] During the training process of the converter state monitoring model, in order to enable the state tensor to represent the converter state, a state type fully connected network is additionally introduced after normalization.

[0061] The training data is obtained by annotating the visual features and Mel-spectrograms covering the converter state with converter state labels; the data volume of the visual features and Mel-spectrograms in the training data evenly covers the converter state, such as dividing the slag degree of each smelting stage into five converter state labels: redrying, near redrying, proper slaging, near splashing, and splashing; the data volume of the visual features and Mel-spectrograms of the five converter state labels: redrying, near redrying, proper slaging, near splashing, and splashing in each smelting stage are consistent.

[0062] During training, visual features and mel-spectrograms in training data are randomly input into the converter state monitoring model, and the converter state monitoring model predicts a corresponding state tensor; the state tensor is input into the state type fully connected network, and the state type fully connected network obtains a predicted converter state label based on the state tensor; the cross entropy between the predicted converter state label and the true converter state label is used as a loss function, and based on the loss function, the parameters of the converter state monitoring model and the state type fully connected network are updated using a gradient descent algorithm, so that the loss function of the data in the training data set is stably lower than a set threshold.

[0063] The converter state monitoring model models the correlation between the converter visual features and audio features to better obtain the state of the converter during the smelting process through visual and audio features. For example, when the slag degree of the slag in the converter is weak at the beginning of smelting, the slag does not cover the surface of the molten steel. At this time, due to the low slag degree, the audio features generated are not very obvious for identifying the slag degree, but the slag degree of the slag can be well perceived through the visual features; when the slag in the converter covers the surface of the molten steel, at this time, the ability to perceive the slag degree of the slag through visual features is relatively weakened, and due to the change in slag thickness, during the smelting process, the audio features generated by the interaction with the slag can better perceive the slag degree of the slag.

[0064] The state tensor is provided to a pre-trained AI agent, which is trained based on the DQN reinforcement learning architecture and is used to select the action that maximizes the Q value under the current state tensor, which is executed by the spiral feeder and oxygen lance of the converter. The converter smelting actions include: the amount of material added to the spiral feeder during the smelting process, the height of the oxygen lance, and the oxygen lance flow and pressure.

[0065] In the specific implementation process, Figure 5 As shown, the DQN reinforcement learning architecture for building AI agents:

[0066] Use the state tensor as the state in the DQN reinforcement learning architecture;

[0067] Use the converter, oxygen lance, and screw feeder as environments in the DQN reinforcement learning architecture;

[0068] Constructing an AI agent, the AI ​​agent comprising: a Q-value network and a target Q-value network based on a fully connected network, wherein parameters of the target Q-value network are periodically synchronized from the Q-value network; the Q-value network maps a state tensor to a Q-value, wherein the dimension of the Q-value is the number of actions that the AI ​​agent can decide;

[0069] Initialize the experience replay pool and Q value table; the experience replay pool contains the current state tensor , current moment action ,award and the next moment state tensor The Q value table records the Q value of any action performed under any state tensor.

[0070] In order to enable the reward to guide the process of action optimization, this application sets the reward as:

[0071] , for The upper limit of the range of feasible states set in advance during the smelting stage; for The lower limit of the pre-set feasible state range in the smelting stage; represents the Euclidean distance. Taking the slag state as an example, for The upper limit of the appropriate slag range in the smelting stage is for The lower limit of the appropriate slag range in the smelting stage; When splashing or drying is reached, it is abnormal. In abnormal cases, the reward is negative infinity. In normal cases, the closer the state is to the median of the appropriate slag range of the corresponding smelting stage, the greater the reward.

[0072] The process of training the AI ​​agent includes:

[0073] Initialize Q value network parameters , initialize the parameters of the target Q value network ;

[0074] Iterate until the predetermined number of training rounds is reached or the loss function is less than the set threshold:

[0075] Use the Q value network to calculate the state tensor at the current moment The maximum Q value of the optional action;

[0076] Greedy algorithm based on probability Choose the action with the largest Q value: ,by The probability of randomly selecting an action is ,in, It is the set of actions that the AI ​​agent can decide.

[0077] In the specific implementation process, a counter is set for each possible action, and the initial value is set to 0; at each time step, The greedy algorithm selects actions according to the following rules: Choose an action at random; with probability , select the action with the largest Q value. If there are multiple actions with the highest Q value, select them based on the counter of each action. The probability of each action being selected is ,in is the counter value of the action; each time an action is selected, the counter value of the action is increased.

[0078] In the environment, perform the selected action , get rewarded and the state tensor at the next moment , the current state tensor , current moment action ,award and the next moment state tensor The composed experience tuple is stored in the experience replay pool;

[0079] Randomly extract an experience tuple from the experience replay pool, and calculate the next state in the selected experience tuple through the target Q value network Maximum Q value:

[0080] If the next state in the experience tuple is the terminal state, then the target value Equal to the reward in the experience tuple ; Otherwise, the target value is equal to the reward in the experience tuple Add the attenuation factor Multiply the target Q value by the next state predicted by the network Maximum Q value:

[0081] ,in, Represents the target Q value network.

[0082] Within the range of the extracted experience tuples, the target value and Q-value network prediction value The mean square error between is the loss function of DQN, and the parameters of the Q value network are updated using the gradient descent algorithm. The parameters of the Q value network are regularly updated to the target Q value network.

[0083] Example 2

[0084] See also Figure 6 As shown, an embodiment of the present invention provides an automatic optimization device for a converter smelting process based on an AI intelligent agent, comprising: at least one processing unit, the processing unit is connected to a storage unit, a collection unit and an execution unit through a bus unit; the collection unit includes an industrial camera and an infrared camera for coaxial imaging, and an audio device; the execution unit includes: a spiral filler and an oxygen gun; the storage unit, as a computer-readable storage medium, can be used to store software programs, computer executable programs and modules, such as the software programs, computer executable programs and modules corresponding to an automatic optimization method for a converter smelting process based on AI in an embodiment of the present invention. The processing unit implements the above-mentioned automatic optimization method for a converter smelting process based on AI by running the software programs, computer executable programs and modules stored in the storage unit, comprising:

[0085] The RGB image and infrared image of the converter during the smelting process are acquired frame by frame through the coaxial imaging industrial camera and infrared camera; the audio data of the smelting process is collected through the audio equipment;

[0086] The collected RGB images and infrared images are spliced ​​along the channel dimension to obtain visual features, the audio data is sliced ​​by frame, and each slice is converted into a Mel-spectrogram as an audio feature;

[0087] Inputting the visual features and the audio features into a pre-trained converter state monitoring model, wherein the converter state monitoring model outputs a state tensor representing the converter state based on the visual features and the audio features;

[0088] The state tensor is provided to a pre-trained AI agent, which is trained based on the DQN reinforcement learning architecture and is used to select the action that maximizes the Q value under the current state tensor, which is executed by the spiral feeder and oxygen lance of the converter. The converter smelting actions include: the amount of material added to the spiral feeder during the smelting process, the height of the oxygen lance, and the oxygen lance flow and pressure.

[0089] Of course, the computer program stored in the storage unit of the automatic optimization device of the converter smelting process based on AI intelligent agent provided in an embodiment of the present invention is not limited to the method operations described above, but can also execute the relevant operations in the automatic optimization method of the converter smelting process based on AI provided in any embodiment of the present invention.

[0090] Example 3

[0091] An embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program. When the computer program is executed, the method for automatically optimizing a converter smelting process based on AI is implemented, including:

[0092] The RGB image and infrared image of the converter during the smelting process are acquired frame by frame through the coaxial imaging industrial camera and infrared camera; the audio data of the smelting process is collected through the audio equipment;

[0093] The collected RGB images and infrared images are spliced ​​along the channel dimension to obtain visual features, the audio data is sliced ​​by frame, and each slice is converted into a Mel-spectrogram as an audio feature;

[0094] Inputting the visual features and the audio features into a pre-trained converter state monitoring model, wherein the converter state monitoring model outputs a state tensor representing the converter state based on the visual features and the audio features;

[0095] The state tensor is provided to a pre-trained AI agent, which is trained based on the DQN reinforcement learning architecture and is used to select the action that maximizes the Q value under the current state tensor, which is executed by the spiral feeder and oxygen lance of the converter. The converter smelting actions include: the amount of material added to the spiral feeder during the smelting process, the height of the oxygen lance, and the oxygen lance flow and pressure.

[0096] A computer-readable storage medium provided in an embodiment of the present invention stores a computer program which is not limited to the method operations described above, but can also execute related operations in an AI-based automatic optimization method for converter smelting process provided in any embodiment of the present invention.

[0097] In the embodiments provided by the present invention, it should be understood that the disclosed structures and methods can be implemented in other ways. For example, the structural embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, structures or units, which can be electrical, mechanical or other forms.

[0098] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0099] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0100] The foregoing is merely a specific embodiment of the present invention, which enables those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features claimed herein.

Claims

1. An AI-based automatic optimization method for converter smelting process, characterized in that: include: The RGB image and infrared image of the converter during the smelting process are acquired frame by frame through the coaxial imaging industrial camera and infrared camera; the audio data of the smelting process is collected through the audio equipment; The collected RGB images and infrared images are spliced ​​along the channel dimension to obtain visual features, the audio data is sliced ​​by frame, and each slice is converted into a Mel-spectrogram as an audio feature; Inputting visual features and audio features into a pre-trained converter state monitoring model, the converter state monitoring model outputs a state tensor representing the converter state based on the visual features and audio features, the converter state monitoring model comprising: a visual feature extraction module, an audio feature extraction module, two self-attention transformers based on a multi-head self-attention module, two cross-attention transformers based on a cross-attention module, and two multi-layer perceptrons; extracting the internal associations of the visual features and the audio features respectively through the two self-attention transformers, and then extracting the associations between the audio features and the visual features through the two cross-attention transformers; the two cross-attention transformers are respectively processed and fused through two multi-layer perceptrons, and the tensor representing the converter state is obtained through activation function and normalization processing; The visual feature extraction module includes: a convolution-based patch embedding layer and a dilated convolution-based patch embedding layer; the patch embedding layer and the dilated convolution-based patch embedding layer process the visual features into two groups of patches from different scales respectively; the two groups of patches are processed by a linear layer to obtain the embedded features of the visual features, the embedded features of the visual features are combined with the position code, and then mapped into , , Three vectors, , , The three vectors are input into the self-attention transformer of the visual feature extraction module to model the internal association of the visual features, wherein the self-attention transformer adopts a multi-head self-attention mechanism module; the output features of the multi-head self-attention mechanism module are mapped into , , Three vectors; among them, , and the vector obtained based on audio features , exchange, Cross-attention is performed through the cross-attention Transformer, that is: ; is the visual cross-attention feature, is the cross attention function of visual features; The audio feature extraction module includes: a convolution-based patch embedding layer; the patch embedding layer processes the Mel spectrogram as the audio feature into a patch, processes the patch through a linear layer to obtain an embedded feature of the audio feature, adds and combines the embedded feature of the audio feature with the position code, and then maps it into , , Three vectors, , , The three vectors are input into the self-attention transformer of the audio feature extraction module to model the internal association of the audio features, which is actually to model the association between the time-frequency features of the Mel spectrogram. The self-attention transformer adopts a multi-head self-attention mechanism module; the output features of the multi-head self-attention mechanism module are mapped into , , Three vectors; among them, , And the vector obtained based on video features , exchange, Cross-attention is performed through the cross-attention Transformer, that is: ; is the audio cross-attention feature, is the cross attention function of audio features; The state tensor is provided to a pre-trained AI agent, which is trained based on the DQN reinforcement learning architecture and is used to select the action that maximizes the Q value under the current state tensor, which is executed by the spiral feeder and oxygen lance of the converter. The converter smelting actions include: the amount of material added to the spiral feeder during the smelting process, the height of the oxygen lance, and the oxygen lance flow and pressure.

2. The AI-based automatic optimization method for converter smelting process according to claim 1 is characterized in that: During the training process, a state type fully connected network is additionally introduced after standardization; during training, visual features and mel spectrograms in the training data are randomly input into the converter state monitoring model, the converter state monitoring model predicts a corresponding state tensor, and the state tensor is input into the state type fully connected network, and the state type fully connected network obtains a predicted converter state label based on the state tensor; The cross entropy between the predicted converter state label and the actual converter state label is used as the loss function. Based on the loss function, the gradient descent algorithm is used to update the parameters of the converter state monitoring model and the state type fully connected network, so that the loss function of the data in the training data set is stably lower than the set threshold.

3. The AI-based automatic optimization method for converter smelting process according to claim 1 is characterized in that: Use the state tensor as the state in the DQN reinforcement learning architecture; Use the converter, oxygen lance, and screw feeder as environments in the DQN reinforcement learning architecture; Constructing an AI agent, the AI ​​agent comprising: a Q-value network and a target Q-value network based on a fully connected network, wherein parameters of the target Q-value network are periodically synchronized from the Q-value network; the Q-value network maps a state tensor to a Q-value, wherein the dimension of the Q-value is the number of actions that the AI ​​agent can decide; Initialize the experience replay pool and Q value table; the experience replay pool contains the current state tensor , current moment action ,award and the next moment state tensor The Q value table records the Q value of any action performed under any state tensor; Among them, rewards , for The upper limit of the range of feasible states set in advance during the smelting stage; for The lower limit of the pre-set feasible state range in the smelting stage; represents the Euclidean distance.

4. The AI-based automatic optimization method for converter smelting process according to claim 3 is characterized in that: The process of training the AI ​​agent includes: Initialize Q value network parameters , initialize the parameters of the target Q value network ; Iterate until the predetermined number of training rounds is reached or the loss function is less than the set threshold: Use the Q value network to calculate the state tensor at the current moment The maximum Q value of the optional action; Greedy algorithm based on probability Choose the action with the largest Q value: ,by The probability of randomly selecting an action is ,in, It is the set of actions that the AI ​​agent can decide; In the environment, perform the selected action , get the reward and the state tensor of the next moment , the current state tensor , current moment action ,award and the next moment state tensor The composed experience tuple is stored in the experience replay pool; Randomly extract an experience tuple from the experience replay pool, and calculate the next state in the selected experience tuple through the target Q value network Maximum Q value: If the next state in the experience tuple is the terminal state, then the target value is equal to the reward in the experience tuple ; Otherwise, the target value is equal to the reward in the experience tuple Add the attenuation factor Multiply the target Q value by the next state predicted by the network The maximum Q value of Within the range of the extracted experience tuples, the mean square error between the target value and the predicted value of the Q-value network is used as the loss function, and the gradient descent algorithm is used to update the parameters of the Q-value network.

5. The AI-based automatic optimization method for converter smelting process according to claim 4 is characterized in that: Set a counter for each possible action, with the initial value set to 0; At each time step, an action is chosen according to the following rule: By probability Randomly select an action; By probability , select the action with the largest Q value. If there are multiple actions with the highest Q value, select them based on the counter of each action. The probability of each action being selected is ,in is the counter value of the action; Each time an action is selected, increase the counter value of that action.

6. An automatic optimization device for converter smelting process based on AI agent, characterized in that: include: At least one processing unit, the processing unit is connected to the storage unit, the acquisition unit and the execution unit through a bus unit, the acquisition unit includes a coaxial imaging industrial camera and an infrared camera, and an audio device; the execution unit includes: a spiral filler and an oxygen gun; The storage unit stores a computer program, and when the computer program is executed by the processing unit, the AI-based automatic optimization method for the converter smelting process as described in any one of claims 1-5 is implemented.

7. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the AI-based automatic optimization method for converter smelting process as described in any one of claims 1 to 5 is implemented.

Images