An anomaly detection method for fused magnesium furnaces based on knowledge distillation using a calibrated teacher model

Abstract

This invention provides an anomaly detection method for fused magnesium furnaces based on knowledge distillation with probability calibration. The method includes: collecting real-time image data during the fused magnesium furnace production process and labeling it as normal or abnormal, forming a training dataset; training a complex teacher model and generating initial soft labels; then using IVAP to accurately calibrate the soft labels output by the teacher model, obtaining the calibrated probability distribution and its uncertainty interval; and constructing a weighted knowledge distillation loss function based on this calibrated probability interval to train a lightweight student model. During real-time production, the student model predicts the anomaly probability to achieve accurate detection and real-time alarm for abnormal states of the fused magnesium furnace. This invention improves the calibration accuracy of the student model's predicted probability, enhances the ability to identify minority anomalies, reduces the risk of missed detections, and is lightweight, making it suitable for real-time applications in resource-constrained scenarios.

Description

Technical Field

[0001] This invention relates to the field of industrial process monitoring and anomaly detection technology, and more particularly to an anomaly detection method for fused magnesium furnaces based on knowledge distillation using a calibrated teacher model. Background Technology

[0002] The fused magnesium furnace is a crucial industrial piece of equipment used for smelting metallic magnesium. It produces magnesium metal by melting and reducing magnesium ore under high-temperature conditions. This process involves high temperatures, strong electric fields, and the dynamic interaction of multiple variables, creating a complex environment with rapidly changing operating conditions. It is highly susceptible to interference from factors such as fluctuations in process parameters, equipment performance degradation, and sudden malfunctions. If anomalies occur during production and are not detected and addressed promptly, they can lead to a significant decline in product quality, or even equipment failure, impacting production safety and economic efficiency. Therefore, accurate and reliable real-time anomaly monitoring and early warning systems for the fused magnesium furnace production process are of immense practical value.

[0003] Currently, anomaly detection in fused magnesium furnaces primarily relies on traditional image processing, empirical rules, or deep learning methods for intelligent diagnosis. While deep models such as convolutional neural networks have been applied to image feature extraction, and recurrent neural networks and long short-term memory networks can handle the temporal modeling of process parameters, and related research has attempted to introduce unsupervised structures such as generative adversarial networks and autoencoders to address the limitations of anomaly data, these methods still have significant limitations in practical deployment. On the one hand, most deep models are large and computationally complex, making them difficult to deploy and run on resource-constrained equipment in industrial settings. On the other hand, current models generally neglect the probability calibration of predicted outputs, easily leading to overconfident predictions and resulting in false positives or false negatives of anomalies, especially with unstable detection performance for a few types of anomalies. Furthermore, while existing knowledge distillation methods can compress models, they typically use uncalibrated soft labels directly for student model training, failing to fully reflect the potential predictive uncertainties in the teacher model, thus affecting distillation results and final performance.

[0004] Chinese patent CN202410328742.0 discloses a "Method and System for Detecting Operating Conditions of an Electric Molded Magnesium Furnace Based on Attention and Dendritic Networks," which utilizes self-attention mechanisms and dendritic networks to identify the operating conditions of the electric molten magnesium furnace. Chinese patent CN202210481109.6 proposes a "Method for Diagnosing Operating Conditions of an Electric Molded Magnesium Furnace Control System Based on an Improved Federated Distillation Algorithm," which allows for joint modeling in a multi-client environment. While these methods possess some innovation, they still fail to address the key issues of insufficient deep model calibration and high deployment resource overhead.

[0005] Faced with the complex industrial environment of fused magnesium furnaces characterized by high temperatures, strong electricity, and drastic fluctuations in operating conditions, existing methods lack a systematic design that integrates model compression, label calibration, and anomaly detection capabilities, making it difficult to balance accuracy, real-time performance, and deployment feasibility. Therefore, there is an urgent need for an anomaly detection method that can improve detection performance while also ensuring lightweight models and predictive reliability, in order to meet the real-time industrial monitoring needs under complex operating conditions. Summary of the Invention

[0006] To address the aforementioned technical problems, this invention provides an anomaly detection method for fused magnesium furnaces based on knowledge distillation of a calibrated teacher model. First, this invention utilizes IVAP to accurately calibrate the soft labels generated by the teacher model, generating highly reliable soft labels and uncertainty intervals. Then, a weighted knowledge distillation loss function is designed to incorporate the uncertainty of the soft labels into the training process of the student model, effectively reducing the negative impact of the teacher model's uncertain knowledge. This improves the calibration performance of the student model in anomaly detection tasks, particularly significantly enhancing its ability to identify minority class anomalies.

[0007] The technical means employed in this invention are as follows:

[0008] An anomaly detection method for fused magnesium furnaces based on knowledge distillation using a calibrated teacher model includes:

[0009] S1. Use industrial cameras to capture real-time image data of high-temperature furnaces to form an initial dataset;

[0010] S2. Divide the initial dataset into training set, validation set and calibration set, select teacher model and student model, and train teacher model;

[0011] S3. Construct a scoring function using a teacher model;

[0012] S4. Based on the scoring function and calibration set, the soft labels output by the teacher model are calibrated using the IVAP algorithm;

[0013] S5. Construct a new knowledge distillation loss function using calibrated soft tags and train the student model.

[0014] S6. Deploy the trained student model at the electric fused magnesium furnace site to perform anomaly detection on real-time image data.

[0015] Further, step S1 includes:

[0016] S11. Real-time image data of a high-temperature furnace is captured using an industrial camera, denoted as... x ;

[0017] S12. Based on the acquired high-temperature furnace image data x Label normal and abnormal data to generate labels containing normal operating conditions.y =0 and abnormal operating condition labels y Initial dataset = 1 D .

[0018] Further, step S2 includes:

[0019] S21. Initial dataset D The training set was divided into two parts according to a 3:1:1 ratio. D tr Validation set D val and calibration set D cal ;

[0020] S22. Select ResNet18 as the teacher model. M t Squeezenet1_1 was chosen as the student model. M s ;

[0021] S23, Based on training set D tr and verification set D val Training teacher models M t .

[0022] Further, step S3 includes:

[0023] Using teacher models M t The logit constructs a scoring function as follows:

[0024] s i = S ( x i )

[0025] in, s i Represents the scoring function. x i This represents the high-temperature furnace image data in the training set.

[0026] Further, step S4 includes:

[0027] S41. Set the sample to be calibrated as... x t Based on the scoring function constructed in step S3, the calibration set is calculated. D cal Samples in and samples to be calibrated x t The scores are as follows:

[0028]

[0029] in, This represents the score of the first sample in the calibration set. Indicates the first in the calibration set n The score of each sample, Indicates the sample to be calibrated x t The score;

[0030] S42. Using the scoring function constructed in step S3 s i = S ( x i Construct an order-preserving regression model;

[0031] S43. Calculate the values ​​of the samples to be calibrated based on the ordinal-preserving regression model. x t The probability of corresponding labels being 0 and 1;

[0032] S44. Place the sample to be calibrated. x t The probabilities of corresponding labels being 0 and 1 are fused, and the fusion probability is calculated.

[0033] S45. Calculate the calibration label based on the fusion probability.

[0034] Furthermore, in step S42, the constructed order-preserving regression model f 0 and f 1 ,as follows:

[0035]

[0036] in, This indicates the label corresponding to the sample, with 0 and 1 representing the sample to be calibrated, respectively. x t Possible tags.

[0037] Further, in step S43, the sample to be calibrated x t The formulas for calculating the probability of corresponding labels 0 and 1 are as follows:

[0038]

[0039] in, Indicates the sample to be calibrated x t The probability that the corresponding label is 0. Indicates the sample to be calibrated x t The probability of the corresponding label being 1.

[0040] Furthermore, in step S44, the formula for calculating the fusion probability is as follows:

[0041]

[0042] in, This represents the fusion probability.

[0043] Furthermore, in step S45, the calculation formula for the calibration tag is as follows:

[0044]

[0045] in, This indicates a calibration label.

[0046] Further, step S5 includes:

[0047] S51. Calculate the cross-entropy loss between the student model's probability output and the hard labels. ,as follows:

[0048]

[0049] in, This represents the probability output of the student model. Indicates a hard tag;

[0050] S52. Calculate the student model probability output and calibration labels. Kullback-Leibler (KL) divergence ,as follows:

[0051]

[0052] S53. Utilize the cross-entropy loss from step S51 Kullback-Leibler divergence with S52 Construct a new knowledge distillation loss function ,as follows:

[0053]

[0054] in, , and Indicates the sample to be calibrated corresponding p 0 and p 1 ; It is a weight used to balance the two types of loss.

[0055] Compared with the prior art, the present invention has the following advantages:

[0056] 1. The present invention provides an anomaly detection method for fused magnesium furnaces based on knowledge distillation of a calibrated teacher model. The method based on IVAP calibrated teacher model soft tags significantly improves the calibration performance of student model prediction probability and reduces the risk of false negatives in anomaly detection.

[0057] 2. The present invention provides an anomaly detection method for fused magnesium furnaces based on knowledge distillation of a calibrated teacher model. By constructing a weighted knowledge distillation loss function and utilizing the uncertainty information of the prediction probability interval, the method significantly improves the ability of the student model to identify and detect minority anomalies when dealing with highly imbalanced data, thus ensuring the accuracy of anomaly detection.

[0058] 3. The present invention provides a knowledge distillation method for anomaly detection in fused magnesium furnaces based on a calibrated teacher model, which effectively reduces model complexity and resource consumption. It is particularly suitable for resource-constrained industrial production environments and can realize online deployment and efficient operation of real-time anomaly detection in fused magnesium furnace processes.

[0059] Based on the above reasons, this invention can be widely applied in fields such as anomaly detection in fused magnesium smelting production. Attached Figure Description

[0060] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0061] Figure 1 This is a flowchart of the method of the present invention.

[0062] Figure 2 The graph shows the loss and accuracy changes of the student model during the training process, as provided in this embodiment of the invention.

[0063] Figure 3 This is a reliability diagram of the prediction results of the student model provided in an embodiment of the present invention. Detailed Implementation

[0064] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0065] It should be noted that the terms "comprising" and "having" and any variations thereof in the specification, claims and accompanying drawings of this invention are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such processes, methods, products or devices.

[0066] like Figure 1 As shown, this invention provides a method for detecting anomalies in fused magnesium furnaces based on knowledge distillation using a calibrated teacher model, comprising:

[0067] S1. Use industrial cameras to capture real-time image data of high-temperature furnaces to form an initial dataset;

[0068] S2. Divide the initial dataset into training set, validation set and calibration set, select teacher model and student model, and train teacher model;

[0069] S3. Construct a scoring function using a teacher model;

[0070] S4. Based on the scoring function and calibration set, the soft labels output by the teacher model are calibrated using the IVAP algorithm;

[0071] S5. Construct a new knowledge distillation loss function using calibrated soft tags and train the student model.

[0072] S6. Deploy the trained student model at the electric fused magnesium furnace site to perform anomaly detection on real-time image data.

[0073] In a specific implementation, as a preferred embodiment of the present invention, step S1 includes:

[0074] S11. Real-time image data of a high-temperature furnace is captured using an industrial camera, denoted as... x ;

[0075] S12. Based on the acquired high-temperature furnace image data x Label normal and abnormal data to generate labels containing normal operating conditions. y =0 and abnormal operating condition labels yInitial dataset = 1 D .

[0076] In this embodiment, the initial dataset is constructed. D It contains 9801 samples, including normal operating conditions and under-burning conditions.

[0077] In a specific implementation, as a preferred embodiment of the present invention, step S2 includes:

[0078] S21. Initial dataset D The training set was divided into two parts according to a 3:1:1 ratio. D tr Validation set D val and calibration set D cal ;

[0079] S22. Select ResNet18 as the teacher model. M t Squeezenet1_1 was chosen as the student model. M s ;

[0080] S23, Based on training set D tr and verification set D val Training teacher models M t .

[0081] In a specific implementation, as a preferred embodiment of the present invention, step S3 includes:

[0082] Using teacher models M t The logit constructs a scoring function as follows:

[0083] s i = S ( x i )

[0084] in, s i Represents the scoring function. x i This represents the high-temperature furnace image data in the training set.

[0085] In a specific implementation, as a preferred embodiment of the present invention, step S4 includes:

[0086] S41. Set the sample to be calibrated as... x tBased on the scoring function constructed in step S3, the calibration set is calculated. D cal Samples in and samples to be calibrated x t The scores are as follows:

[0087]

[0088] in, This represents the score of the first sample in the calibration set. Indicates the first in the calibration set n The score of each sample, Indicates the sample to be calibrated x t The score; in this embodiment, the number of samples in the calibration set is 3268.

[0089] S42. Using the scoring function constructed in step S3 s i = S ( x i Construct an order-preserving regression model f 0 and f 1 ,as follows:

[0090]

[0091] in, This indicates the label corresponding to the sample, with 0 and 1 representing the sample to be calibrated, respectively. x t Possible tags.

[0092] S43. Based on the ordinal-preserving regression model f 0 and f 1 Calculate the sample to be calibrated x t The probabilities of corresponding labels 0 and 1 are as follows:

[0093]

[0094] in, Indicates the sample to be calibrated x t The probability that the corresponding label is 0. Indicates the sample to be calibrated x t The probability that the corresponding label is 1;

[0095] S44. Place the sample to be calibrated.x t The probabilities of corresponding labels 0 and 1 are fused, and the fusion probability is calculated. p ,as follows:

[0096]

[0097] S45, Based on fusion probability p Calculate the calibration label as follows:

[0098]

[0099] in, This indicates a calibration label.

[0100] In this embodiment, the "soft label" output by the teacher model refers to its predicted category probability distribution. Compared to traditional hard labels (i.e., definite 0 or 1), soft labels provide more information about the uncertainty of the model's judgment. This invention uses the IVAP algorithm to calibrate the soft labels probabilities. This method establishes a monotonic probability mapping between labels and scores based on ordinal-preserving regression, thereby improving the reliability of the predicted probabilities and the calibration performance.

[0101] In a specific implementation, as a preferred embodiment of the present invention, step S5 includes:

[0102] S51. Calculate the cross-entropy loss between the student model's probability output and the hard labels. ,as follows:

[0103]

[0104] in, This represents the probability output of the student model. Indicates a hard tag;

[0105] S52. Calculate the student model probability output and calibration labels. Kullback-Leibler (KL) divergence ,as follows:

[0106]

[0107] S53. Utilize the cross-entropy loss from step S51 Kullback-Leibler divergence with S52 Construct a new knowledge distillation loss function ,as follows:

[0108]

[0109] in, , and Indicates the sample to be calibrated corresponding p 0 and p 1 ; These are the weights used to balance the two types of losses. In this embodiment, The loss and accuracy changes of the student model during the training process are as follows: Figure 2 As shown, when the student model outputs a label of 1, it indicates that under-smelting has occurred during the melting process. Figure 3 The figure shown is a reliability diagram of the student model.

[0110] In this embodiment, to measure the distribution difference between the student model output and the calibration label, the present invention further introduces KL divergence as part of the knowledge distillation loss function, which effectively enhances training stability and accuracy.

[0111] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting anomalies in an electric magnesium smelting furnace based on knowledge distillation of a calibrated teacher model, characterized by, Comprise: S1, real-time capture high-temperature furnace image data by industrial camera, constitute initial data set; S2, the initial data set is divided into training set, validation set and calibration set, the teacher model and the student model are selected, and the teacher model is trained; S3, the score function is constructed by using the teacher model, comprising: Utilizing a teacher model M t The score function is constructed as follows: s i = S ( x i ) wherein, s i denotes a scoring function, x i denotes high-temperature furnace image data in the training set; S4, based on the score function and the calibration set, the soft label output by the teacher model is calibrated by using the IVAP algorithm, comprising: S41, setting the sample to be calibrated as x t According to the score function constructed in step S3, the scores of the samples in the calibration set D cal and the sample to be calibrated x t are calculated as follows: wherein, represents the score of the 1st sample in the calibration set, represents the score of the n th sample in the calibration set, represents the score of the sample to be calibrated; x t represents the score of the sample to be calibrated; S42, using the score function constructed in step S3 s i = S ( x i ), construct a monotonic regression model and as follows: wherein, denotes the label to which the sample corresponds, 0 and 1 respectively denote a sample to be calibrated x t possible labels; S43、According to the order-preserving regression model, calculate the sample to be calibrated x t The probability of the corresponding label being 0 and 1 is calculated according to the following formula: wherein, denotes the sample to be calibrated x t the probability that the corresponding label is 0, denotes the sample to be calibrated x t the probability that the corresponding label is 1; S44, calibrating the sample to be calibrated x t The probabilities corresponding to the labels 0 and 1 are fused to calculate a fused probability, the calculation formula being as follows: wherein, represents the fusion probability; S45, according to the fusion probability, the calibration label is calculated, the calculation formula is as follows: wherein, represents a calibration tag; S5, the new knowledge distillation loss function is constructed by using the calibrated soft label, and the student model is trained; S6, the trained student model is deployed in the electric melting magnesium furnace site, and the real-time image data is executed for anomaly detection.

2. The method according to claim 1, wherein the method is characterized by, Step S1, comprising: S11, real-time capture high-temperature furnace image data by industrial camera, denoted as x ; S12, based on the acquired high-temperature smelting furnace image data x , label normal data and abnormal data, generate an initial data set containing normal condition labels y =0 and abnormal condition labels y =1 D .

3. The method according to claim 1, wherein the method is characterized by, Step S2, comprising: S21, splitting the initial data set D into a training set D tr , a validation set D val and a calibration set D cal ; S22, select resnet18 as the teacher model M t , select squeezenet1_1 as the student model M s ; S23, based on the training set D tr and the validation set D val , training the teacher model M t .

4. The method according to claim 1, wherein the method is characterized by, Step S5, comprising: S51, compute cross-entropy loss of student model probability output and hard label As follows: wherein, probabilities output representing the student model, represents a hard label; S52, compute student model probability outputs and calibration labels Kullback-Leibler divergence of as follows: S53, using the cross-entropy loss of step S51 and the Kullback-Leibler divergence of S52 , constructing a new knowledge distillation loss function as follows: wherein, , and denote the i-th sample to be calibrated corresponding to p 0 and p 1 ; is a weight for balancing the two losses.

Images