Methods, devices, and equipment for converter smelting condition monitoring based on multimodal large models

Abstract

This application relates to a method, apparatus, and equipment for converter smelting condition monitoring based on a multimodal large model, belonging to the field of converter smelting condition monitoring technology. The method includes: acquiring multimodal condition data for the corresponding monitoring furnace; acquiring similar furnace record information and activation rules based on the multimodal condition data and the corresponding multimodal database; generating fusion features corresponding to the modal features based on the corroboration relationship between the modal features corresponding to the multimodal condition data, the fusion features being used to represent the smelting condition code corresponding to the monitoring furnace; generating a condition monitoring conclusion for the monitoring furnace and the corresponding reasoning basis based on the smelting condition code, similar furnace record information, and activation rules; and generating a smelting condition monitoring report for the monitoring furnace based on the condition monitoring conclusion and the reasoning basis. This application achieves intelligent monitoring of converter smelting, effectively reducing human judgment errors and production risks.

Description

Technical Field

[0001] This application relates to the field of converter smelting condition monitoring technology, specifically to a converter smelting condition monitoring method, device and equipment based on a multimodal large model. Background Technology

[0002] Converter smelting is a core component of modern steel production, its key being the separation and purification of metals from ore through physical and chemical means. The converter smelting process comprises multiple stages, each corresponding to specific temperature changes, chemical reaction rates, and other characteristics. Monitoring the furnace temperature, chemical reaction rate, and slag formation, and adjusting smelting parameters in a timely manner, can effectively ensure the production quality of converter smelting.

[0003] The relevant technologies usually rely on manual monitoring of the converter smelting status. Operators need to observe the furnace flame at close range for a long time, which poses a safety hazard in the high-temperature environment. In addition, they rely on human experience, which can lead to judgment errors. Summary of the Invention

[0004] In view of this, this application provides a method, device and equipment for monitoring the state of converter smelting based on a multimodal large model. The main purpose is to solve the technical problems that related technologies usually rely on manual monitoring of the state of converter smelting, which requires operators to observe the furnace flame at close range for a long time, which poses safety hazards in high-temperature environments, and relies on human experience, which is prone to judgment errors.

[0005] According to the first aspect of this application, a method for monitoring the condition of converter smelting based on a multimodal large model is provided, the method comprising:

[0006] Acquire multimodal status data for the corresponding monitoring furnace of the converter;

[0007] Based on multimodal state data and the multimodal database corresponding to the converter, obtain similar furnace record information and activation rules corresponding to the monitored furnace;

[0008] Based on the corroboration relationship between the modal features corresponding to the multimodal state data, a fusion feature corresponding to the modal feature is generated. The fusion feature is used to represent the smelting state code corresponding to the monitored furnace.

[0009] Based on the smelting status code, similar furnace record information and activation rules, the status monitoring conclusions corresponding to the monitored furnaces and the reasoning basis for the status monitoring conclusions are generated.

[0010] Based on the condition monitoring conclusions and reasoning, generate a smelting condition monitoring report corresponding to the monitored furnace;

[0011] The step of obtaining similar furnace record information and activation rules corresponding to the monitored furnace based on the multimodal state data and the multimodal database corresponding to the converter includes:

[0012] Extract the historical smelting process features corresponding to the historical furnaces from the smelting record database in the multimodal database;

[0013] Based on the Mahalanobis distance between the smelting process characteristics of the multimodal state data and the historical smelting process characteristics, similar furnace record information corresponding to the monitored furnace is determined from the smelting record database;

[0014] Obtain the expert rule base in the multimodal database, and determine the activation rule corresponding to the monitored furnace based on the activation intensity of the expert rules in the expert rule base at the current time step.

[0015] According to a second aspect of this application, a converter smelting condition monitoring device based on a multimodal large model is provided, the device comprising:

[0016] The acquisition module is used to acquire multimodal status data of the monitored furnace corresponding to the converter; based on the multimodal status data and the multimodal database corresponding to the converter, acquire similar furnace record information and activation rules corresponding to the monitored furnace; the step of acquiring similar furnace record information and activation rules corresponding to the monitored furnace based on the multimodal status data and the multimodal database corresponding to the converter includes: extracting historical smelting process features corresponding to historical furnaces from the smelting record database in the multimodal database; determining similar furnace record information corresponding to the monitored furnace from the smelting record database according to the Mahalanobis distance between the smelting process features of the multimodal status data and the historical smelting process features; acquiring the expert rule base in the multimodal database, and determining the activation rule corresponding to the monitored furnace from the expert rule base according to the activation intensity of the expert rules in the expert rule base at the current time step;

[0017] The generation module is used to generate fusion features corresponding to the modal features based on the corroboration relationship between the modal features corresponding to the multimodal state data. The fusion features are used to represent the smelting state code corresponding to the monitored furnace. Based on the smelting state code, similar furnace record information and activation rules, the module generates the state monitoring conclusion corresponding to the monitored furnace, as well as the reasoning basis corresponding to the state monitoring conclusion. Based on the state monitoring conclusion and reasoning basis, the module generates the smelting state monitoring report corresponding to the monitored furnace.

[0018] According to a third aspect of this application, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method of the first aspect described above.

[0019] According to a fourth aspect of this application, a computer-readable storage medium is provided that stores a computer program thereon, which, when executed by a processor, implements the method of the first aspect described above.

[0020] The converter smelting condition monitoring method, apparatus, and equipment based on a multimodal large model provided in this application, compared with related technologies, acquire multimodal condition data of the corresponding monitoring furnace; based on the multimodal condition data and the multimodal database corresponding to the converter, acquire similar furnace record information and activation rules corresponding to the monitoring furnace; generate fusion features corresponding to the modal features based on the corroboration relationship between the modal features corresponding to the multimodal condition data, and the fusion features are used to represent the smelting condition code corresponding to the monitoring furnace; generate condition monitoring conclusions corresponding to the monitoring furnace and the reasoning basis corresponding to the condition monitoring conclusions based on the smelting condition codes, similar furnace record information, and activation rules; and generate a smelting condition monitoring report corresponding to the monitoring furnace based on the condition monitoring conclusions and the reasoning basis. In this way, this application can collect multimodal state data of converter monitoring furnaces. By analyzing the multimodal state data and the multimodal database, it extracts similar furnace record information and activation rules corresponding to the monitored furnaces. Based on the corroboration relationship between the modal features corresponding to the multimodal state data, it generates fusion features corresponding to the modal features to represent the smelting state code corresponding to the monitored furnace, thereby realizing the fusion of multimodal state data. Then, based on the smelting state code, similar furnace record information, and activation rules, it generates the state monitoring conclusion corresponding to the monitored furnace, as well as the reasoning basis corresponding to the state monitoring conclusion. Finally, based on the state monitoring conclusion and reasoning basis, it generates the smelting state monitoring report corresponding to the monitored furnace, realizing intelligent monitoring of converter smelting state without manual inspection. It can also provide an interpretable state reasoning process, enhancing the interpretability and reliability of the state monitoring conclusion, and effectively reducing human judgment errors and production risks. Attached Figure Description

[0021] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0022] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, those skilled in the art can obtain other drawings based on these drawings without creative effort.

[0023] Figure 1 A flowchart illustrating a converter smelting condition monitoring method based on a multimodal large model provided in this application embodiment;

[0024] Figure 2 A flowchart illustrating an example provided in an embodiment of this application is shown;

[0025] Figure 3 This is a schematic diagram of the structure of a converter smelting condition monitoring device based on a multimodal large model, provided in an embodiment of this application. Detailed Implementation

[0026] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of this application, including various details to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0027] The following description, with reference to the accompanying drawings, describes a method, apparatus, and equipment for monitoring the smelting status of a converter based on a multimodal large model, according to embodiments of this application.

[0028] Related technologies typically involve calculating the brightness array of converter flame images captured by cameras during converter smelting, counting the number of flickering elements in the brightness array, and then analyzing the flame state based on the calculated number of flickering units, outputting the monitoring results. Alternatively, they may determine the predicted mid-to-late stage transition point of converter smelting based on the current furnace mouth flame feature set, and determine the converter blowing endpoint based on the carbon content at the mid-to-late stage transition point using a converter blowing carbon model. These methods rely solely on a single flame image, neglecting historical experience and operational information such as gas composition and concentration, or are based on a black-box model for state monitoring and identification. They fail to form reusable and logically sound identification rules, resulting in weak robustness of the constructed models. Furthermore, these technologies usually only provide monitoring results without explaining the logical reasoning process behind the results, making it difficult to interpret the generation mechanism of the results. Furthermore, the lack of effective integration of expert experience and knowledge into the system makes it difficult to guarantee the system's confidence level in practical applications. It also lacks interpretability of the identification process, making it difficult for users and operators to trust the system's judgment results. The monitoring process suffers from strong reliance on human intervention, insufficient information mining, lack of interpretability, and low reasoning ability. As a result, it is difficult to achieve highly intelligent, high-precision, and high-reliability status monitoring of the converter smelting process, and it is difficult to meet the requirements of modern smelting industry for safe production and quality improvement and efficiency enhancement.

[0029] This application provides a method, device, and equipment for monitoring the smelting status of a converter based on a multimodal large model. The main purpose is to solve the technical problems that related technologies usually rely on manual monitoring of the smelting status of the converter. Operators need to observe the furnace flame at close range for a long time, which poses safety hazards in high-temperature environments and relies on human experience, resulting in judgment errors.

[0030] like Figure 1 As shown, embodiments of this application provide a method for monitoring the state of converter smelting based on a multimodal large model, including:

[0031] Step 101: Obtain the multimodal status data of the corresponding monitoring furnace for the converter.

[0032] For example, the monitored furnace number can be the current production furnace number of the converter, which can include the complete smelting process of the converter smelting metal; the multimodal state data can be a set of information reflecting the current smelting state of the converter, which can include converter flame images corresponding to visual modalities, flue gas composition data corresponding to time-series modalities, natural language description information corresponding to text modalities, etc.

[0033] In some embodiments, when the system receives a smelting status monitoring instruction sent from an external source (such as an operator, host computer, or scheduling system), it can start monitoring the smelting status of the current furnace and obtain multimodal status data of the furnace corresponding to the current time step in real time. The multimodal status data can be collected synchronously using different devices for converter smelting status analysis.

[0034] Step 102: Based on the multimodal state data and the multimodal database corresponding to the converter, obtain the similar furnace record information and activation rules corresponding to the monitored furnace.

[0035] The multimodal database can be a database built based on multimodal state data from historical furnaces. It may include a converter flame image database, a flue gas composition database, a smelting record database, an expert rule base, etc., and can be used for knowledge accumulation and training of preset monitoring models.

[0036] In some embodiments, several historical furnaces with operating conditions closest to the monitored furnace can be found in the converter flame image library through feature matching, and their corresponding record information can be extracted as similar furnace record information corresponding to the monitored furnace for smelting status analysis. Correspondingly, knowledge rules triggered by the natural language description information of the monitored furnace can be matched in the expert rule base and determined as activation rules for generating converter smelting status prompts and operation suggestions.

[0037] Step 103: Based on the corroboration relationship between the modal features corresponding to the multimodal state data, generate the fusion feature corresponding to the modal feature. The fusion feature is used to represent the smelting state code corresponding to the monitored furnace.

[0038] In some embodiments, feature extraction can be performed on each modal state data in the multimodal state data to obtain corresponding modal features, such as extracting flame features from converter flame images and flue gas features from flue gas composition data. To integrate multiple modal features, cross-validation, matching, and other operations can be performed on multiple modal features to obtain fusion features that integrate multiple state information. For example, multiple modal features can be merged into a unified feature vector through a certain mechanism (such as attention, splicing, gating) as a fusion feature to reflect the overall converter smelting state of the monitored furnace, and a high-dimensional semantic representation of the smelting process of the monitored furnace can be performed as the smelting state code corresponding to the monitored furnace, that is, a comprehensive digital representation of the current converter smelting state.

[0039] Step 104: Based on the smelting status code, similar furnace record information, and activation rules, generate the status monitoring conclusion corresponding to the monitored furnace, as well as the reasoning basis for the status monitoring conclusion.

[0040] In some embodiments, the condition monitoring conclusion can be a comprehensive judgment of the converter smelting state of the monitored furnace, such as "poor slag formation, high risk of splashing" or "high probability of hitting the endpoint carbon temperature." The reasoning basis can be a natural language description of the reasoning process for the condition monitoring conclusion, providing an interpretable chain of evidence to support the conclusion and transparently displaying the decision-making logic to facilitate operator understanding and process adjustment. For example, a preset monitoring model (which can be called a multimodal large model) can be used to output the condition monitoring conclusion corresponding to the monitored furnace and the reasoning basis corresponding to the condition monitoring conclusion based on information such as smelting state coding, similar furnace record information, and activation rules, so as to achieve intelligent condition monitoring and reduce manual monitoring.

[0041] Step 105: Based on the status monitoring conclusions and reasoning, generate a smelting status monitoring report corresponding to the monitored furnace.

[0042] In some embodiments, a smelting condition monitoring report corresponding to the monitored furnace can be generated by integrating the condition monitoring conclusions and reasoning basis according to a preset template. The smelting condition monitoring report can be a structured or semi-structured output document used to convey monitoring results and recommendations to operators, engineers, or management systems. For example, the smelting condition monitoring report can be output according to a preset four-segment format, forming a detailed decision support report to support operators in quick understanding and decision-making.

[0043] In this way, this embodiment can collect multimodal state data of the converter's monitoring furnaces. By analyzing the multimodal state data and the multimodal database, it extracts similar furnace record information and activation rules corresponding to the monitored furnaces. Based on the corroboration relationship between the modal features corresponding to the multimodal state data, it generates fusion features corresponding to the modal features to represent the smelting state code corresponding to the monitored furnace, thus realizing the fusion of multimodal state data. Then, based on the smelting state code, similar furnace record information, and activation rules, it generates the state monitoring conclusion corresponding to the monitored furnace, as well as the reasoning basis corresponding to the state monitoring conclusion. Finally, based on the state monitoring conclusion and reasoning basis, it generates the smelting state monitoring report corresponding to the monitored furnace, realizing intelligent monitoring of the converter smelting state without manual inspection. It can also provide an interpretable state reasoning process, enhancing the interpretability and reliability of the state monitoring conclusion, and effectively reducing human judgment errors and production risks.

[0044] Based on the technical implementation shown in the above embodiments, in order to further illustrate the specific implementation process of the method in this embodiment, optionally, the multimodal state data may include converter flame images, flue gas composition data, and natural language description information.

[0045] In some embodiments, the furnace mouth flames at different smelting stages of the converter exhibit significant differences. The converter furnace mouth flame is crucial information for determining the reaction state within the furnace. Converter flame images can be used to represent flame characteristics such as color, brightness, shape, and length. These flame characteristics can indirectly reflect the intensity of the carbon-oxygen reaction within the furnace, thereby determining the decarburization rate and smelting state of the molten pool. Based on flame characteristics, monitoring furnace temperature, chemical reaction rate, and slag formation, and adjusting smelting parameters in a timely manner, can effectively ensure the production quality of converter steelmaking. Correspondingly, flue gas composition data may include, but is not limited to, gas concentration, gas temperature, and gas flow rate; natural language description information may be descriptive text corresponding to the converter flame images and flue gas composition data.

[0046] Further optionally, before step 102, the method of this embodiment may also include: extracting production information from the production reports corresponding to the historical production batches of the converter, and constructing a smelting record database in the multimodal database based on the production information; extracting the expert's description information on the smelting state and smelting state analysis process of the converter from the expert survey information corresponding to the smelting state of the converter, and constructing an expert rule base based on the description information.

[0047] In some embodiments, production reports corresponding to the production process of historical production batches in the factory system where the converter is located can be retrieved, and production information such as smelting furnace number, charging sequence, temperature curve, blowing time, final carbon content, and tapping temperature can be extracted from the production reports to obtain smelting record data for each historical production batch. A smelting record database can be constructed based on these smelting record data.

[0048] Correspondingly, expert survey information may include expert questionnaires, expert interviews, etc., which can be obtained by collecting structured questionnaires, semi-structured interviews and other materials from experts in the field of converter smelting. Then, relevant descriptions of experts' judgment and analysis processes on the state of converter smelting and the state of converter smelting can be extracted from the expert survey information. These descriptions can be saved in text form as expert experience data for the construction of an expert rule base.

[0049] Optionally, the method in this embodiment may further include: acquiring converter flame images during the blowing cycle of historical production batches based on preset observation points of the converter, performing image enhancement processing on the converter flame images, and constructing a converter flame image library based on the image enhancement processed converter flame images.

[0050] The preset observation points can be converter furnace openings, other designated locations, etc., and the preset acquisition equipment can include high-temperature industrial cameras with characteristics such as high frame rate (≥60 fps), high temperature resistance, dustproof, and shockproof.

[0051] For example, a high-temperature industrial camera is deployed at the converter mouth to capture converter flame images throughout the entire blowing cycle, covering different smelting stages such as oxygen lance blowing, steady blowing, and endpoint control. During acquisition, the acquisition timestamp can be synchronized with the smelting operation record and the converter flame image sequence. The converter flame images can be saved in PNG format to preserve image details. Intelligent monitoring technology based on converter flame images ensures the safety of the converter steelmaking process, provides auxiliary data for equipment maintenance scheduling and production optimization, effectively reduces accident risks and production costs, and is a key path to achieving high-quality industrial development.

[0052] Specifically, various image enhancement transformations can be applied to converter flame images, such as geometric transformations (rotation, scaling, translation, mirroring), lighting and color adjustments, contrast enhancement, and noise disturbance simulation, to improve the robustness of the preset monitoring model to different shooting conditions and on-site disturbances, thus constructing a highly robust converter flame image library. For example, a converter flame image library can be represented as follows:

[0053] ;

[0054] In the formula, X represents the converter flame image library, which corresponds to N production batches. For the current production batch... It can acquire converter flame images at T time steps; for the converter flame image corresponding to the current time step t, the j-th enhancement transformation processing can be performed, and the enhancement transformation processing can include n processing methods; This can represent the current production batch. In the process, the converter flame image corresponding to the current time step t is processed for the first time step. Image of converter flame after enhancement transformation.

[0055] Optionally, the method in this embodiment may further include: using different flue gas monitoring sensors of the converter to collect flue gas composition data corresponding to the blowing cycle, performing time synchronization processing on the flue gas composition data, recording the time-synchronized flue gas composition data in the form of a time series, and obtaining the flue gas composition database in the multimodal database.

[0056] In some embodiments, the flue gas monitoring sensor may include a non-dispersive infrared (NDIR) flue gas analyzer, an electrochemical sensor, an ultraviolet sensor, etc., which can be used to measure different gases (such as CO, CO2, O2, NO) produced in the converter in real time during the blowing cycle. xThe content of gases such as (etc.) is used as flue gas composition data; during the collection process, the sampling frequency can be 4Hz and the response time can be 20s to achieve high efficiency; the flue gas composition data can be recorded in time series form, recorded as time series text data, and can be stored in formats such as comma-separated values ​​(CSV) and time series database (TSDB).

[0057] For example, a certain production batch at the current time step Collected flue gas composition data It can be represented as:

[0058] ;

[0059] In the formula, It can represent production process parameters, such as the time change rate of various gas concentrations, feeding ratio information (raw material type, mass ratio and feeding time), and process stage setting time.

[0060] Specifically, by unifying the timestamps of flue gas composition data collected by different flue gas monitoring sensors, synchronous interpolation and feature standardization processing can be performed on the flue gas composition data. Let the original sampling time step of the flue gas composition data in the current production batch i be . The next time step after the original sampling time step is Obtained through linear interpolation Estimates of flue gas composition data at each time step:

[0061] ;

[0062] In the formula, and These are two adjacent sampling points (original sampling time steps). and the next time step of the original sampling time step The corresponding flue gas composition data, This is an estimated value corresponding to the time-synchronized flue gas composition data, which facilitates the standardization of flue gas composition data collected by different flue gas monitoring sensors to eliminate the influence of dimensions.

[0063] For example, the z-score standardization method can also be used to standardize the flue gas composition data (or the linearly interpolated flue gas composition data). The formula for the z-score standardization method can be expressed as:

[0064] ;

[0065] In the formula, and Data on flue gas composition The corresponding mean and variance These are dimensionless values ​​obtained after standardizing the flue gas composition data. Optionally, the estimated values ​​corresponding to the flue gas composition data can also be standardized. It can also be a dimensionless value after standardization of the estimated values ​​corresponding to flue gas composition data.

[0066] Correspondingly, the dimensionless value after standardization based on the flue gas composition data and / or the estimated values ​​corresponding to the flue gas composition data. The constructed flue gas composition database can be represented as:

[0067] ;

[0068] In the formula, G represents the flue gas composition database, which corresponds to N production batches. For the current production batch... It can collect flue gas composition data for T time steps.

[0069] This approach ensures the consistency of flue gas measurement data over time, improving the stability and accuracy of subsequent modal feature verification and fusion.

[0070] Further optional steps include cleaning the collected multimodal status data of monitoring furnaces and historical furnaces, such as removing noise interference and abnormal data during the collection process, supplementing missing information, and unifying data format and timestamps, in order to construct converter flame image library, flue gas monitoring database, smelting record database, and expert rule base, etc.

[0071] Optionally, the method in this embodiment may further include: generating natural language prompt templates for different smelting stages of the converter smelting process based on the smelting record database and the expert rule base; generating natural language description information corresponding to the converter flame image database and the flue gas composition database according to the natural language prompt templates; and constructing a description information database in the multimodal database based on the natural language description information.

[0072] For example, descriptions of different smelting stages of the converter smelting process can be formed based on smelting record data corresponding to the smelting record database and expert experience data corresponding to the expert rule base. First, a natural language prompt template can be designed to map the enhanced converter flame image and standardized aligned flue gas composition data into structured natural language descriptions, such as "The flame is bright yellowish-white, accompanied by strong fluctuations, oxygen content is decreasing, CO remains at a high level, and it appears to be approaching the final carbon content range." This constructs a description information database, forming a converter flame image database. Flue gas composition database Description information database D corresponds to A triple, wherein the descriptive information base can be represented as:

[0073] ;

[0074] In the formula, the description information database D can correspond to N production batches. For the current production batch... It can generate natural language descriptions corresponding to T time-step converter flame images and flue gas composition data. It can represent the natural language description information generated at the current time step t.

[0075] Optionally, step 102 may specifically include: extracting historical smelting process features corresponding to historical furnaces from the smelting record database in the multimodal database; determining similar furnace record information corresponding to the monitored furnace from the smelting record database based on the Mahalanobis distance between the smelting process features of the multimodal state data and the historical smelting process features; obtaining the expert rule base in the multimodal database, and determining the activation rule corresponding to the monitored furnace from the expert rule base based on the activation intensity of the expert rules in the expert rule base at the current time step.

[0076] Specifically, historical smelting records most similar to the conditions of the monitored furnace can be retrieved from the constructed smelting record database H. For example, the smelting process feature vector corresponding to the smelting process characteristics of the monitored furnace can be set as follows: Features including flame brightness, color components, flue gas indicators, feeding sequence, temperature profile, and carbon content are considered. The similarity between the smelting process characteristics and historical smelting process characteristics can be calculated using Mahalanobis distance, with the following formula:

[0077] ;

[0078] in, Historical smelting records The corresponding historical smelting process feature vector. Let be the covariance matrix of the eigenvectors of historical smelting processes. This represents the Mahalanobis distance between the smelting process feature vector and the historical smelting process feature vector. By considering the covariance structure between features based on Mahalanobis distance, this approach reflects the correlation between converter data features, thus accurately depicting the true distribution distance of the data in the feature space.

[0079] Based on the above formula, the Mahalanobis distance between the currently collected smelting process feature vector and all historical smelting process feature vectors in the smelting record database is calculated in real time. The calculated Mahalanobis distances can be sorted from largest to smallest, and the Top-m similar historical furnaces (e.g., m is set to 10) can be selected to obtain the corresponding similar furnace record information. This information is used to extract the smelting process evolution trajectory and final smelting result (e.g., control strategies for smoothly reaching the endpoint, signs before splashing) corresponding to each historical smelting record, forming a tiered prior feature to assist the subsequent preset monitoring model in reasoning and judging the converter smelting state.

[0080] In some embodiments, the expressions in expert experience are fuzzy concepts. Membership functions can be used to formalize expert rules and quantify them into computable rule activation values. A single expert rule can be represented as:

[0081] ;

[0082] in, The preconditions corresponding to the expert rule. The set of conditions corresponding to the expert rules, such as flame features and the set of flame features corresponding to the flame features; For the first Each expert rule corresponds to a fuzzy set describing the state, such as multiple fuzzy sets for converter states like "good," "normal," "to be observed," and "state alarm." When the condition is met, the variable `condition` is set to... .

[0083] Specifically, for the first The expert rule, its prerequisites Contains several atomic conditions and its corresponding condition set For each atomic condition, a corresponding membership function is set. The actual data for each atomic condition is input into the corresponding membership function to obtain its satisfaction level. Based on the logical relationships between atomic conditions (e.g., AND takes the minimum value, OR takes the maximum value), the membership function is calculated. The expert rule at the current time step activation intensity The activation values ​​of all expert rules, forming an activation value vector, can be represented as:

[0084] ;

[0085] in, Total number of rules representing experts. Represents the activation value vector. Indicates the first The expert rule at the current time step activation intensity, Indicates the first The expert rule at the current time step The activation intensity;

[0086] Furthermore, an embedding layer is used to map the activation value vectors into dense vectors of fixed dimensions. , can be represented as:

[0087] ;

[0088] in, and It can be the weight matrix and bias vector of the rule encoder corresponding to the expert rule.

[0089] Furthermore, the expert rule base obtained above is integrated with Retrieval-Augmented Generation (RAG). This expert rule base essentially constitutes a retrieval tool based on expert experience. When the preset monitoring model needs to perform reasoning, the system will activate the most relevant rules from the expert rule base according to the current working conditions, as the activation rules corresponding to the monitoring furnace, and encode them as context to provide to the preset monitoring model.

[0090] Optionally, step 103 may specifically include: generating a flame feature vector corresponding to the converter flame image based on the flame morphology evolution process corresponding to the converter flame image; generating a flue gas feature vector corresponding to the flue gas composition data based on the long-range dependency relationship of different flue gas components at different time steps in the flue gas composition data; performing cross-validation on the flame feature vector and the flue gas feature vector based on the first attention layer in the cross-modal attention fusion strategy to obtain the corroboration relationship between the flame feature vector and the flue gas feature vector, and generating flue gas association features corresponding to the first attention layer based on the corroboration relationship, wherein the first attention layer includes a flue gas attention layer based on visual features; and verifying the flue gas association features using the text features corresponding to the natural language description information according to the second attention layer in the cross-modal attention fusion strategy to generate fused features, wherein the second attention layer includes a joint attention layer based on text features.

[0091] In some embodiments, this embodiment can process three heterogeneous but crucial time-series data: converter flame images reflecting the intensity of reactions in the furnace, flue gas composition data indicating the chemical reaction path, and natural language description information carrying operator experience, thereby transforming multimodal data into a unified and robust feature representation, laying a solid foundation for intelligent reasoning of the converter smelting status.

[0092] For example, for those containing For a sequence of converter flame images at several time steps, a feature extraction architecture combining 2D-CNN and temporal modeling can be designed. Specifically, using a pre-trained ConvNeXt as the backbone network, high-level spatial features, such as flame features (brightness, color distribution, and texture), are independently extracted for each time step of the converter flame image, represented as a fixed-dimensional flame feature vector. Then... The flame feature vectors at each time step are input into a Bidirectional Gated Recurrent Unit (Bi-GRU) network. This network captures the entire evolution of the flame morphology from bright and dazzling to soft and uniform from both forward and backward directions, ultimately outputting a flame feature vector containing spatiotemporal dynamic information, denoted as […]. .

[0093] Accordingly, for flue gas composition data, an extraction architecture combining a one-dimensional convolutional neural network (1D-CNN) and a self-attention mechanism is adopted. The 1D-CNN is responsible for capturing short-term, local interactions and fluctuation patterns among flue gas components; then, the feature sequence output by the 1D-CNN is input into the Transformer encoder, where the self-attention mechanism effectively models the long-range dependencies between different time points and between different flue gas components, such as identifying a sustained CO peak pattern in the middle of the blowing process. Finally, the vector corresponding to the classification token (CLS) output by the Transformer is taken as the flue gas feature vector, denoted as... .

[0094] Secondly, for natural language description information, an extraction strategy combining text encoding and temporal modeling is designed. A lightweight Transformer pre-trained model (such as DistilBERT) is used to independently perform deep semantic encoding on the text fragments corresponding to the natural language description information at each time point, obtaining their sentence embeddings. Then, a Bi-GRU network is used to capture the evolution of the semantic patterns of the operational descriptions, such as the text pattern change from "the flame is blinding" to "smoke is coming from the stove opening," thereby transforming fragmented experiential descriptions into structured temporal semantic features. Finally, the hidden state of the last time step of the Bi-GRU is taken as the text feature vector, denoted as... .

[0095] After obtaining the vectors corresponding to the features of each modality, a two-stage cross-modal attention fusion strategy can be designed. Through a cross-modal mechanism, one modality queries and verifies relevant information in another modality, thereby generating a fused feature representation containing cross-modal interaction information. This simulates the thought process of human experts making comprehensive judgments, i.e., simultaneously observing the flame, referring to instrument data, and combining experience descriptions for cross-verification. Specifically, the first stage can construct a first attention layer, such as a smoke attention layer based on visual features for visual guidance. The flame feature vector is used as the query vector (Query, Q), and the smoke feature vector is used as the key (Key, K) and value (Value, V) to perform cross-attention calculations and obtain the smoke-related features. , can be represented as:

[0096] ;

[0097] The above calculations enable the preset monitoring model to actively search for the most relevant and corroborating evidence in the precise flue gas composition data that is most relevant to the current visual information of the flame. For example, it can associate the visual feature of "bright white flame" with the numerical feature of "high CO concentration" to generate a visually guided flue gas correlation feature.

[0098] Correspondingly, a second attention layer can be constructed in the second stage, such as a joint attention layer that guides text based on text features. In this stage, the text feature vector can be... As the query vector Q1; and simultaneously the output of the first stage and original splicing This process generates joint visual-smoke evidence, identifying smoke-related features that corroborate the flame and smoke feature vectors. These features are then used as K1 and V1 for attention calculation, outputting the final cross-modal fused feature vector. , can be represented as:

[0099] ;

[0100] In this way, the text modality acts as a "referee" or "interpreter," performing final screening, weighting, and integration of the evidence provided by the first two modalities, thereby outputting a highly condensed and synergistic fusion feature vector.

[0101] Furthermore, the final fused feature vector can be fine-tuned through a fully connected layer to serve as the smelting state code representing the comprehensive state of the current time step. This code will be one of the core inputs to the subsequent explicit inference and determination module for the smelting state. By utilizing the preset monitoring model corresponding to the two-stage cross-modal attention fusion strategy, combined with the fused features, prior features of similar furnaces, and activation rules, the final converter smelting state determination and inference text generation can be performed.

[0102] This two-stage cross-modal attention fusion mechanism simulates the cognitive logic of "observation-verification-synthesis," making the fusion process more interpretable and more effectively capturing complex nonlinear relationships between multiple modalities. This directional attention mechanism can generate attention weights and visualize them, for example, showing which time step of the flame image, which smoke components, and which text description the pre-defined monitoring model focuses on when making a certain judgment, greatly enhancing the interpretability of state monitoring.

[0103] Optionally, step 104 may specifically include: integrating smelting status codes, similar furnace record information, and activation rules to obtain prompt words corresponding to the monitored furnace; using a preset monitoring model to perform feature comparison on the prompt words, generating status monitoring conclusions, and locating the chain evidence corresponding to the status monitoring conclusions based on the feature chain corresponding to the status monitoring conclusions, describing the chain evidence through natural language, generating reasoning basis, using the preset monitoring model to evaluate the status monitoring conclusions, generating the confidence level corresponding to the status monitoring conclusions, and retrieving smelting process knowledge to generate operational suggestions corresponding to the status monitoring conclusions.

[0104] Correspondingly, based on the status monitoring conclusions, reasoning basis, confidence level, and operational suggestions, a smelting status monitoring report corresponding to the monitored furnace can be generated.

[0105] In some embodiments, the preset monitoring model can perform the highest level of semantic understanding and logical reasoning on the multimodal state data provided by the preceding modules, and finally generate a complete report containing information such as judgment results, confidence level, reasoning basis, and operation suggestions. This approach goes beyond simple converter smelting state judgment and can provide expert-level advice for comprehensive decision support.

[0106] For example, a pre-defined monitoring model can be built based on a large-scale language model (Llama) and fine-tuned according to domain instructions. This constructs a large-scale, high-quality instruction-answer pair dataset, injects smelting knowledge from a multimodal database into it, and designs a natural language prompt template containing complete context from all information sources. The smelting status monitoring report serves as the corresponding expected answer and can be strictly standardized into a complete report format containing multiple core elements, including the judgment result, confidence level, reasoning basis, and operational suggestions. Through supervised fine-tuning, the pre-defined monitoring model not only learns how to make professional judgments but also learns how to proactively and rationally propose operational suggestions after diagnosis. After the pre-defined monitoring model completes its adaptation to the smelting domain, a standardized explicit reasoning and suggestion flow can be deployed for it. When the system needs to monitor a furnace in real time, this process is activated and executed according to the following steps:

[0107] Step 1: Multimodal Data Aggregation and Context Construction. The system automatically acquires all key information about the current state, such as the smelting state code output by the cross-modal data feature extraction and fusion module, the similar furnace record information of the Top-m similar furnaces obtained by the similar furnace record information retrieval module (e.g., "Among the three most similar historical furnaces, two experienced splashing after similar features appeared"), and the quantified activation rule embedding list generated by the expert rule activation and encoding module (e.g., rule #15 "Warning of excessive flame rigidity" activation intensity 0.9). This information is then integrated into a structured prompt word.

[0108] Step Two: Collaborative Reasoning and Suggestion within the Pre-tuned Monitoring Model. Complete contextual prompts are fed into the fine-tuned pre-tuned monitoring model. The model first performs a comprehensive diagnosis, weighing fused features, similar furnace record information, and activation rules to form a preliminary judgment on the state monitoring conclusion and its confidence level. Next, it enters the strategy generation stage. In this stage, the model actively traces back the feature chain corresponding to the state monitoring conclusion: if the state monitoring conclusion indicates an anomaly, it locates the chain of evidence leading to the anomaly; if the confidence level is low, it identifies chains of evidence in the multimodal state data that contain contradictions or uncertainties. Based on this, the pre-tuned monitoring model can retrieve and generate targeted and feasible operational suggestions based on the state monitoring conclusion from the smelting process knowledge it has learned during training.

[0109] Step 3: Generate a comprehensive decision-making report. The final output of the preset monitoring model can be constrained to a preset four-segment format, forming a detailed smelting condition monitoring report:

[0110] Judgment result: Give a clear and unambiguous condition monitoring conclusion, such as "poor slag formation, high risk of splashing" or "high probability of hitting the endpoint carbon temperature";

[0111] Confidence level: Provides a quantified confidence level percentage (such as "92%" or "75%), which intuitively reflects the certainty of the judgment;

[0112] Reasoning Basis: By describing the chain of evidence in natural language, a detailed natural language explanation is generated, transparently showcasing the decision-making logic. For example: "The reasoning basis for this judgment is: 1) The current flame's 'bright and dazzling, excessively rigid' characteristics match historical splashing cases by more than 80%; 2) The flue gas CO plateau indicates that the decarbonization reaction is hindered, which is consistent with the scenario described in Rule #15 (excessively rigid flames are usually accompanied by excessively sticky slag or high oxygen pressure). However, it is worth noting that other current monitoring data are still within the normal range, and this contradiction is the main reason why the confidence level does not reach 100%."

[0113] Operational Recommendations: Key components for empowering on-site operations. The pre-set monitoring model will provide specific and actionable recommendations based on judgment results and reasoning. For example, regarding the aforementioned splash risk, operational recommendations include: "1) Adjusting oxygen lance operation: It is recommended to appropriately reduce the oxygen pressure by 0.05 MPa or slightly raise the lance position to 'soften' the flame and reduce excessive impact on the molten pool. 2) Optimizing slag-forming material addition: It is recommended to add a small amount of fluorite in batches as needed to improve slag fluidity and promote slag formation. 3) Strengthening flame morphology monitoring: It is recommended that operators focus on whether the flame changes from 'glaringly rigid' to 'soft and uniform' within the next 2 minutes; this is a key visual signal for improved slag formation."

[0114] For example, such as Figure 2 As shown, firstly, multimodal state data of the converter smelting process can be collected and integrated as the data foundation. Then, a converter smelting state monitoring framework based on a multimodal large-scale model is constructed. This framework includes four core modules: a similar furnace record information retrieval module, an expert rule activation and encoding module, a cross-modal data feature extraction and fusion module, and a large-scale model explicit reasoning and judgment module. Finally, a smelting state monitoring report is generated based on this framework, and the results are presented. The specific functions of each module are described below:

[0115] 1) Similar Furnace Record Information Retrieval Module: By using Mahalanobis distance to match the feature vector of the current furnace with the historical records, it can make full use of existing production experience, find the closest working conditions as a reference and extract its key features, thereby improving the prior rationality of smelting process monitoring.

[0116] 2) Expert rule activation and encoding module: Call the expert rule base, calculate the rule activation strength through the membership function and encode it into rule embedding, thereby formalizing and quantifying expert experience. Based on expert experience, the RAG technology is used to build an expert rule base for the preset monitoring model, ensuring the logical rationality of the inference of the preset monitoring model.

[0117] 3) Cross-modal data feature extraction and fusion module: Parallel branch extraction In the triplet, various data features are integrated using a cross-modal attention mechanism to capture the deep correlation between different modalities, thereby improving the completeness of feature representation and robustness to complex working conditions.

[0118] 4) Large Model Explicit Reasoning and Judgment Module: Based on the Llama series of open-source large model bases, a preset monitoring model for converter smelting is formed through RAG, instruction fine-tuning, and expert rule base. This gives the model powerful semantic understanding and reasoning capabilities. Taking fused features as input, it utilizes the contextual semantic understanding capabilities of the preset monitoring model, and generates state monitoring conclusions, confidence levels, and their reasoning basis based on multimodal fusion features, historical evolution prior features, and expert rules. This achieves high-precision and interpretable converter smelting state judgment, thereby transforming complex multimodal state data into interpretable state monitoring conclusions and operational suggestions. This enables efficient transformation of data, knowledge, and decision-making, and significantly improves the accuracy, versatility, and automation level of converter smelting state monitoring.

[0119] This framework construction approach balances multimodal state data-driven and multimodal state database knowledge-driven approaches. The modules cooperate with each other while remaining relatively independent, possessing incremental learning capabilities. It continuously improves the generalization performance and real-time decision-making ability of the preset monitoring model as process data accumulates. As an intelligent production partner that provides transparent diagnosis, quantifies risks, and proactively offers solutions, it not only answers "what is happening now" and "why," but more importantly, it answers "what should be done now." This truly transforms the insights of artificial intelligence into on-site productivity, achieving closed-loop optimization of converter smelting process monitoring. It effectively reduces human judgment errors and production risks, promoting the intelligent and digital development of steel smelting, and has significant economic benefits and application promotion value.

[0120] Compared with related technologies, this embodiment can comprehensively utilize multiple modal data such as converter flame images, flue gas composition data, similar furnace record information, and expert rules. Based on the first attention layer in the cross-modal attention fusion strategy, it obtains the corroboration relationship between flame feature vectors and flue gas feature vectors, and then generates flue gas association features based on the corroboration relationship. Based on the second attention layer, it uses the text features corresponding to natural language description information to verify the flue gas association features and generate fused features. Through the cross-modal attention fusion strategy, it effectively simulates the thinking process of human experts when making comprehensive judgments and performs cross-validation, realizing the capture of complex nonlinear relationships between different modalities, effectively utilizing multimodal data, and improving information utilization efficiency. At the same time, based on the large model architecture, a preset monitoring model is constructed. This model has powerful semantic understanding and reasoning capabilities, transforming multimodal state data into interpretable state monitoring conclusions and providing operational suggestions, which greatly improves the accuracy, versatility, and automation level of converter smelting state monitoring, enhances the interpretability and reliability of state monitoring conclusions, realizes full-process monitoring of converter smelting, and achieves highly intelligent, high-precision, and high-reliability state monitoring of the converter smelting process.

[0121] Based on the above Figure 1The specific implementation of the method shown in this embodiment provides a converter smelting condition monitoring device based on a multimodal large model, such as... Figure 3 As shown, the device includes: an acquisition module 31 and a generation module 32;

[0122] The acquisition module 31 is used to acquire multimodal status data of the monitored furnace corresponding to the converter; based on the multimodal status data and the multimodal database corresponding to the converter, it acquires similar furnace record information and activation rules corresponding to the monitored furnace.

[0123] The generation module 32 is used to generate fusion features corresponding to the modal features based on the corroboration relationship between the modal features corresponding to the multimodal state data. The fusion features are used to represent the smelting state code corresponding to the monitored furnace. Based on the smelting state code, similar furnace record information and activation rules, the module generates the state monitoring conclusion corresponding to the monitored furnace, as well as the reasoning basis corresponding to the state monitoring conclusion. Based on the state monitoring conclusion and reasoning basis, the module generates the smelting state monitoring report corresponding to the monitored furnace.

[0124] In some examples of this embodiment, the multimodal state data includes converter flame images, flue gas composition data, and natural language description information. The generation module 32 is specifically configured to generate a flame feature vector corresponding to the converter flame image based on the flame morphology evolution process corresponding to the converter flame image; generate a flue gas feature vector corresponding to the flue gas composition data based on the long-range dependencies of different flue gas components at different time steps in the flue gas composition data; perform cross-validation on the flame feature vector and the flue gas feature vector based on the first attention layer in the cross-modal attention fusion strategy to obtain the corroboration relationship between the flame feature vector and the flue gas feature vector, and generate flue gas association features corresponding to the first attention layer based on the corroboration relationship. The first attention layer includes a flue gas attention layer based on visual features; and verify the flue gas association features using the text features corresponding to the natural language description information according to the second attention layer in the cross-modal attention fusion strategy to generate fused features. The second attention layer includes a joint attention layer based on text features.

[0125] In some examples of this embodiment, the generation module 32 is specifically configured to integrate smelting status codes, similar furnace record information, and activation rules to obtain prompt words corresponding to the monitored furnace; use a preset monitoring model to perform feature comparison on the prompt words to generate a status monitoring conclusion; and locate the chain evidence corresponding to the status monitoring conclusion based on the feature chain corresponding to the status monitoring conclusion, describe the chain evidence through natural language, generate reasoning basis, use the preset monitoring model to evaluate the status monitoring conclusion, generate the confidence level corresponding to the status monitoring conclusion, and retrieve smelting process knowledge to generate operation suggestions corresponding to the status monitoring conclusion.

[0126] In some examples of this embodiment, the generation module 32 is specifically configured to generate a smelting status monitoring report corresponding to the monitored furnace based on the status monitoring conclusions, reasoning basis, confidence level, and operation suggestions.

[0127] In some examples of this embodiment, the acquisition module 31 is specifically configured to extract historical smelting process features corresponding to historical furnaces from the smelting record database in the multimodal database; determine similar furnace record information corresponding to the monitored furnace from the smelting record database based on the Mahalanobis distance between the smelting process features of the multimodal state data and the historical smelting process features; acquire the expert rule base in the multimodal database; and determine the activation rule corresponding to the monitored furnace from the expert rule base based on the activation intensity of the expert rules in the expert rule base at the current time step.

[0128] In some examples of this embodiment, the acquisition module 31 is specifically configured to extract production information from the production reports corresponding to the historical production batches of the converter, and construct a smelting record database in the multimodal database based on the production information; extract the expert's description information on the smelting state and smelting state analysis process of the converter from the expert survey information corresponding to the smelting state of the converter, and construct an expert rule base based on the description information.

[0129] In some examples of this embodiment, the acquisition module 31 is further configured to: acquire converter flame images within the blowing cycle of historical production batches based on preset observation points of the converter; perform image enhancement processing on the converter flame images; construct a converter flame image library based on the image enhancement processed converter flame images; collect flue gas composition data corresponding to the blowing cycle using different flue gas monitoring sensors of the converter; perform time synchronization processing on the flue gas composition data; record the time-synchronized flue gas composition data in time series form; acquire the flue gas composition database in the multimodal database; generate natural language prompt templates for different smelting stages of the converter smelting process based on the smelting record database and expert rule base; generate natural language description information corresponding to the converter flame image library and flue gas composition database according to the natural language prompt templates; and construct a description information library in the multimodal database based on the natural language description information.

[0130] Based on the above, Figure 1 Accordingly, this embodiment also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described method. Figure 1 The method shown.

[0131] Based on this understanding, the technical solution of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as CD-ROM, USB flash drive, mobile hard drive, etc.) and includes several instructions to cause a computer device (such as personal computer, server, or network device, etc.) to execute the methods of various implementation scenarios of this application.

[0132] Based on the above, Figure 1 The method shown, and Figure 3 To achieve the above objectives, the present application also provides an electronic device, comprising a storage medium and a processor; the storage medium for storing a computer program; and the processor for executing the computer program to implement the above-described virtual device embodiments. Figure 1 The method shown.

[0133] Optionally, the aforementioned physical devices may also include a user interface, a network interface, a camera, radio frequency (RF) circuitry, sensors, audio circuitry, a Wi-Fi module, etc. The user interface may include a display screen, input units such as a keyboard, etc., and optional user interfaces may also include USB interfaces, card reader interfaces, etc. The network interface may optionally include standard wired interfaces, wireless interfaces (such as Wi-Fi interfaces), etc.

[0134] Those skilled in the art will understand that the physical device structure provided in this embodiment does not constitute a limitation on the physical device, and may include more or fewer components, or combine certain components, or have different component arrangements.

[0135] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages the hardware and software resources of the aforementioned physical device, supporting the operation of information processing programs and other software and / or programs. The network communication module is used to enable communication between the various components within the storage medium, as well as communication with other hardware and software in the information processing physical device.

[0136] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platform, or it can be implemented by hardware. The proposed solution comprehensively utilizes multiple modal data, including converter flame images, flue gas composition data, similar furnace record information, and expert rules. Based on the first attention layer of the cross-modal attention fusion strategy, it obtains the corroboration relationship between flame feature vectors and flue gas feature vectors. Then, it generates flue gas association features based on the corroboration relationship. Based on the second attention layer, it verifies the flue gas association features using text features corresponding to natural language description information, generating fused features. The cross-modal attention fusion strategy effectively simulates the thought process of human experts in comprehensive judgment, performing cross-validation to capture complex nonlinear relationships between different modalities. It effectively utilizes multimodal data and improves information utilization efficiency. Simultaneously, it constructs a pre-set monitoring model based on a large model architecture. This model has powerful semantic understanding and reasoning capabilities, transforming multimodal state data into interpretable state monitoring conclusions and providing operational suggestions. This significantly improves the accuracy, versatility, and automation level of converter smelting state monitoring, enhances the interpretability and reliability of state monitoring conclusions, and realizes full-process monitoring of converter smelting, achieving highly intelligent, high-precision, and high-reliability state monitoring of the converter smelting process.

[0137] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the term "comprising" or any other variations thereof is intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0138] The above are merely specific embodiments of this application, enabling those skilled in the art to understand or implement this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to these embodiments, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A multi-modal large model-based converter smelting state monitoring method, characterized in that, The method comprises the following steps: acquiring multi-modal state data corresponding to a monitoring furnace of a converter; based on the multi-modal state data and the multi-modal database corresponding to the converter, acquiring similar furnace record information and activation rules corresponding to the monitoring furnace; generating fusion features corresponding to the modal features according to the corroboration relationship between the modal features corresponding to the multi-modal state data, the fusion features being used to represent smelting state codes corresponding to the monitoring furnace; generating a state monitoring conclusion corresponding to the monitoring furnace and reasoning basis corresponding to the state monitoring conclusion according to the smelting state codes, the similar furnace record information and the activation rules; generating a smelting state monitoring report corresponding to the monitoring furnace based on the state monitoring conclusion and the reasoning basis; the method comprises the following steps: extracting historical smelting process features corresponding to historical furnaces in a smelting record database in the multi-modal database; determining similar furnace record information corresponding to the monitoring furnace from the smelting record database according to the Mahalanobis distance between the smelting process features of the multi-modal state data and the historical smelting process features; acquiring an expert rule base in the multi-modal database, and determining activation rules corresponding to the monitoring furnace from the expert rule base according to the activation intensity of the expert rules in the expert rule base at the current time step; the multi-modal state data comprises converter flame images, flue gas composition data and natural language description information; the method comprises the following steps: generating a flame feature vector corresponding to the converter flame images according to a flame shape evolution process corresponding to the converter flame images; generating a flue gas feature vector corresponding to the flue gas composition data according to the long-range dependence relationship between different flue gas components in the flue gas composition data at different time steps; cross-verification of the flame feature vector and the flue gas feature vector is performed based on a first attention layer in a cross-modal attention fusion strategy, the corroboration relationship between the flame feature vector and the flue gas feature vector is acquired, and a flue gas correlation feature corresponding to the first attention layer is generated based on the corroboration relationship, the first attention layer comprising a flue gas attention layer based on visual features; the fusion features are generated by verifying the flue gas correlation features using text features corresponding to the natural language description information according to a second attention layer in the cross-modal attention fusion strategy, the second attention layer comprising a joint attention layer based on text features.

2. The method of claim 1, wherein, the method comprises the following steps: integrating the smelting state codes, the similar furnace record information and the activation rules to acquire prompt words corresponding to the monitoring furnace; The feature comparison is performed on the prompt word by using a preset monitoring model, a state monitoring conclusion is generated, a chain evidence corresponding to the state monitoring conclusion is located according to a feature chain corresponding to the state monitoring conclusion, the chain evidence is described by using a natural language, the reasoning basis is generated, the preset monitoring model is used to evaluate the state monitoring conclusion, generate a confidence degree corresponding to the state monitoring conclusion, and retrieve smelting process knowledge to generate an operation suggestion corresponding to the state monitoring conclusion.

3. The method of claim 2, wherein, The smelting state monitoring report corresponding to the monitored heat is generated based on the state monitoring conclusion and the reasoning basis, including: The smelting state monitoring report corresponding to the monitored heat is generated based on the state monitoring conclusion, the reasoning basis, the confidence degree and the operation suggestion.

4. The method of claim 1, wherein, Before the similar heat record information and the activation rule corresponding to the monitored heat are obtained based on the multi-modal state data and the multi-modal database corresponding to the converter, the method further includes: Production information is extracted from a production report corresponding to a historical production batch of the converter, and a smelting record database in the multi-modal database is constructed based on the production information; Description information of smelting states and smelting state analysis processes of the converter is extracted from expert investigation information corresponding to the smelting states of the converter, and the expert rule base is constructed based on the description information.

5. The method of claim 4, wherein, The method further includes: Based on a preset observation point of the converter, converter flame images of the converter in a blowing period of the historical production batch are collected, image enhancement processing is performed on the converter flame images, and a converter flame image database is constructed based on the converter flame images after image enhancement processing; By using different flue gas monitoring sensors of the converter, flue gas component data corresponding to the blowing period is collected, the flue gas component data is time-synchronized, the flue gas component data after time-synchronization is recorded in a time sequence form, and a flue gas component database in the multi-modal database is obtained; Based on the smelting record database and the expert rule base, a natural language prompt template for different smelting stages of the converter smelting process is generated; According to the natural language prompt template, natural language description information corresponding to the converter flame image database and the flue gas component database is generated, and a description information database in the multi-modal database is constructed according to the natural language description information. 6.A multi-modal large model based converter smelting state monitoring device, characterized in that, It includes: The acquisition module is configured to acquire multi-modal state data of a monitored heat corresponding to a converter. The multi-modal state data and the corresponding multi-modal database of the converter are used to obtain similar furnace record information and activation rules corresponding to the monitored furnace; the multi-modal state data and the corresponding multi-modal database of the converter are used to obtain similar furnace record information and activation rules corresponding to the monitored furnace, including: extracting historical smelting process features corresponding to historical furnaces in the smelting record database in the multi-modal database; determining similar furnace record information corresponding to the monitored furnace from the smelting record database according to the Mahalanobis distance between the smelting process features of the multi-modal state data and the historical smelting process features; obtaining an expert rule base in the multi-modal database, and determining the activation rules corresponding to the monitored furnace from the expert rule base according to the activation intensity of the expert rules in the current time step in the expert rule base; A generation module is configured to generate fusion features corresponding to the modal features according to the corroboration relationship between the modal features corresponding to the multi-modal state data, the fusion features being used to represent smelting state encoding corresponding to the monitored furnace; generate a state monitoring conclusion corresponding to the monitored furnace and reasoning basis corresponding to the state monitoring conclusion according to the smelting state encoding, the similar furnace record information and the activation rules; generate a smelting state monitoring report corresponding to the monitored furnace based on the state monitoring conclusion and the reasoning basis; the multi-modal state data includes converter flame images, flue gas composition data and natural language description information; the fusion features corresponding to the modal features are generated according to the corroboration relationship between the modal features corresponding to the multi-modal state data, including: generating a flame feature vector corresponding to the converter flame image according to a flame shape evolution process of the converter flame image; generating a flue gas feature vector corresponding to the flue gas composition data according to a long-range dependence relationship of different flue gas components in the flue gas composition data at different time steps; cross-verification is performed on the flame feature vector and the flue gas feature vector based on a first attention layer in a cross-modal attention fusion strategy, the corroboration relationship between the flame feature vector and the flue gas feature vector is obtained, the flue gas correlation feature corresponding to the first attention layer is generated based on the corroboration relationship, and the first attention layer includes a flue gas attention layer based on visual features; the flue gas correlation feature is verified by using a text feature corresponding to the natural language description information according to a second attention layer in the cross-modal attention fusion strategy, and the fusion features are generated, and the second attention layer includes a joint attention layer based on text features.

7. An electronic device, comprising: comprises: at least one processor; and a memory connected with the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 5.

8. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the method of any one of claims 1 to 5. The computer program is executed by the processor to implement the method of any one of claims 1 to 5.

Images