Intelligent identification method for macro defects of continuous casting billet based on multi-model fusion

Abstract

The application discloses a kind of based on multi-model fusion's continuous casting billet low-power defect intelligent identification method, its steps are: establishing image dataset;Calculate the total loss of depth network forward, the knowledge graph network extracted from expert knowledge is applied to digital image feature extraction process, auxiliary establishment identification bypass branch reference model, and the probability label of network prediction probability label of this reference model is calculated with the probability label calculated by depth learning network to calculate first loss (Loss1);The probability label calculated by depth learning network is calculated with the label of original image supervision data to calculate second loss (Loss2);Total loss (Total Loss) is calculated by weighted summation;Each depth neural network weight is updated by reverse iteration calculation;When loss function value reaches minimum, or unchanged or precision no longer improves, complete entire training and learning process;Finally learned depth network model parameter is applied to actual inspection process.The application solves the problem of sample sparsity and unclear classification standard, improves the success rate of identification.

Description

Technical Field

[0001] This invention relates to steelmaking technology and intelligent detection technology, and in particular to an intelligent identification method for low-magnification defects in continuously cast billets based on multi-model fusion, wherein the models include manual feature extraction, knowledge graph and deep learning technology. Background Technology

[0002] During the production of continuously cast steel billets, surface quality defects may appear on a small number of billets due to various factors such as equipment failure, process changes, and parameter fluctuations. If these surface defects are severe, they will adversely affect subsequent rolling processes. Some surface defects cannot be directly observed and require cleaning and etching operations to determine their type and severity. This defect detection is known as low-magnification microstructure inspection of continuously cast billets. It involves inspecting the microstructure and morphology of the billet after cleaning and etching. These defects mainly include shrinkage cavities, central cracks, subcutaneous cracks, intermediate cracks, central porosity, central segregation, subcutaneous bubbles, thin and uneven equiaxed grain zones, and non-metallic inclusions. In some companies, the low-magnification microstructure defects in continuously cast billets primarily manifest as cracks and central segregation.

[0003] Existing machine vision technologies applied to continuous casting billet surface defect quality inspection systems are all aimed at surface defects of continuous casting billets, rather than low-magnification defects. The defect characteristics and classification requirements of the two are different.

[0004] Chinese patent CN201210061901.2 discloses an online surface quality inspection device system. This system uses a linear array camera to acquire images of the surface of a high-speed running steel billet, then a processing unit calculates the feature values ​​of the image signal and compares them with standard values ​​for normal images. If the difference exceeds a certain threshold, a defect is identified on the billet surface. Chinese patent CN201310483398.4 discloses an online surface quality inspection method for continuously cast billets. This method combines a high-power laser light source, a linear array camera, and a defect detection algorithm to locate, classify, save, and alarm on image defects. The above patents address surface inspection technology.

[0005] Chinese patent CN200710044005.4 discloses a method and device for detecting surface defects in strip steel, which improves the recognition effect by using alternating illumination from multiple light sources and multi-camera acquisition technology to identify various surface defects. Chinese patent CN200710046375.1 discloses a method for visual inspection of moving strip steel using motion imaging, achieving high-quality imaging of high-speed moving strip steel. Chinese patent CN201010112103.9 discloses a method for omnidirectional online detection of strip steel defects, combining image signals and ultrasonic signals to effectively detect surface and internal defects in strip steel. These patents address strip steel production technology. Chinese patent CN201310130020.6 discloses a method and device for detecting micro-cracks in the cross-section of continuously cast billets, applying traditional digital image processing technology to identify crack features.

[0006] In summary, existing technologies directly detect surface defects of continuously cast billets (i.e., surface longitudinal cracks, transverse cracks, pits, slag inclusions, overlaps, scratches, etc.), but cannot accurately identify low-magnification structural defects such as subcutaneous cracks, segregation, and inclusions that can only be observed after cleaning and corrosion.

[0007] Northeastern University of China has applied for invention patent CN201811162804.6, proposing an automatic identification and rating method for center segregation defects in low-magnification microstructure of continuously cast billets. The method uses traditional image processing methods to extract features from the grayscale image of the low-magnification microstructure, and then applies a BP neural network classifier model to identify and rate the center segregation region. It mainly targets the classification and rating of two major defects (cracks and shrinkage cavities) in the low-magnification microstructure of continuously cast square billets. Its drawback is that the sample size is too small, and it is a laboratory result.

[0008] Existing image recognition and detection methods for low-magnification defects in continuously cast billets are limited by issues such as sparse samples and insufficient clarity of classification criteria. It is necessary to improve the success rate of low-magnification defect recognition and classification accuracy by using standard images and limited samples. Summary of the Invention

[0009] The purpose of this invention is to provide a method for intelligent identification of low-magnification defects in continuously cast billets based on multi-model fusion. This method, based on the fusion of manual feature extraction, knowledge graphs, and deep learning models, effectively solves the problems of sparse samples and unclear classification criteria, thereby improving the recognition success rate.

[0010] To achieve the above-mentioned technical objectives, the present invention adopts the following technical solution: A method for intelligent identification of low-magnification defects in continuously cast billets based on multi-model fusion, comprising the following steps: S1. Establish an image dataset and use image data augmentation on sparse samples to expand the sparse dataset and meet the requirements of statistical analysis. S2, calculate the total forward loss of the deep network. That is, during the calculation, the knowledge graph network extracted from expert knowledge is applied to the digital image feature extraction process to help establish a reference model for the recognition side branch. The first loss (Loss1) is calculated by comparing the network prediction probability label of the reference model with the probability label calculated by the deep learning network. S3, calculates the second loss (Loss2) by using the probability labels calculated by the deep learning network and the labels of the original image supervision data. S4, calculate the total loss by weighted summation, and obtain the total residual loss of the system model; where: the total loss equals the first loss multiplied by the coefficient a plus the second loss multiplied by (1 minus the coefficient a), that is: Total Loss = Loss1*a + Loss2 * (1-a); Wherein, the coefficient 'a' is between (0.0 and 1.0); S5, reverse iterative calculation to update the weights of each deep neural network; S6. When the loss function value reaches its minimum, remains unchanged, or the accuracy no longer improves, the entire training and learning process is completed. S7. Finally, the learned deep network model parameters are applied to the actual testing process.

[0011] The image data enhancement method described above is to expand the image dataset as much as possible through image twisting, mirroring, translation, random cropping, grayscale and chroma adjustment, brightness and contrast adjustment, and random noise addition, so as to meet the model requirements for statistical analysis.

[0012] The specific process for calculating the first loss (Loss1) is as follows: S2.1 Based on the characteristics of the steel grade, including composition, process parameters and microstructure, predict the probability value of the occurrence of defects, assign probability weights to the predicted defect map based on the probability value, and transform it into a low-magnification defect knowledge map, that is, link production knowledge with common low-magnification defects. S2.2 Based on the probability weight analysis results, knowledge graph, and on-site expert knowledge and experience, the digital features of the defect images are manually summarized. That is, based on the possible defects recommended by the knowledge graph, the features that need to be focused on are determined, and these digital features, including defect length, width, position, angle, and area, are divided by a smoothing factor to adjust the distribution of feature parameters before being sent into the feature classification model for classification. S2.3, the classification result is converted into a probability value through the Softmax function. This probability value is then weighted and averaged with the mechanism prediction weight of the knowledge graph to obtain the predicted label of the bypass branch reference model.

[0013] The probability weight of the components in the analysis is between 0.1 and 0.3, the probability weight of the process parameters in the analysis is between 0.3 and 0.5, and the probability weight of the tissue in the analysis is between 0.1 and 0.3.

[0014] The smoothing factor is between 10 and 150.

[0015] The feature classification models mentioned include support vector machines, multilayer perceptrons, clustering, and random forest methods.

[0016] The calculation process of the probability label of the deep learning network is as follows: low-magnification image data is directly fed into the deep neural network model for training. The input is the original image, and the label is the on-site expert judgment result. Defect identification is performed in this step. After the identification result is divided by the smoothing factor, it is converted into a probability label through softmax.

[0017] The deep learning network models mentioned include ResNet, Inception, EfficientNet, GoogleNet, and CNN models.

[0018] This invention relates to a multi-model fusion-based intelligent identification method for low-magnification defects in continuously cast billets. Essentially, when calculating the forward network loss, a knowledge graph network refined from expert knowledge is applied to the digital image feature extraction process to assist in establishing a reference model for the identification bypass branch. The network loss of this branch model is then weighted and summed with the residual loss calculated by the deep learning network to obtain the total residual of the system model. Then, the weights of each deep neural network are iterated in reverse until the loss function value is minimized and the entire training process is completed. Finally, the learned model parameters are applied to the actual inspection process.

[0019] This invention integrates expert knowledge graph, digital image feature extraction technology, and deep learning feature extraction technology, effectively solving the problems of sparse samples and unclear judgment criteria, and improving the recognition success rate.

[0020] This invention is a deep fusion technology that organically integrates a digital image feature extraction method based on manual induction, a knowledge graph method based on the experience of production mechanism experts, and a deep learning model in the process of calculating the forward loss function of the network of the system. It gives full play to the advantages of each model method and effectively improves the accuracy and practicality of the model. This invention is operable, effective, and practical in actual production processes, improves the success rate of model recognition, and can achieve the purpose of cost reduction and efficiency improvement. Attached Figure Description

[0021] Figure 1 This invention identifies a schematic diagram of the total loss calculation process. Figure 2A schematic diagram of the training and learning process of the network system of this invention. Implementation

[0022] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0023] See Figure 1 and Figure 2 A method for intelligent identification of low-magnification defects in continuously cast billets based on multi-model fusion, the steps of which are as follows: S1. Establish an image dataset and use image data augmentation methods on sparse samples to expand the sparse dataset and meet the requirements of statistical analysis.

[0024] The image data enhancement method described above is to expand the image dataset as much as possible through image twisting, mirroring, translation, random cropping, grayscale and chroma adjustment, brightness and contrast adjustment, and random noise addition, so as to meet the model requirements for statistical analysis.

[0025] S2 calculates the total forward loss of the deep network. In this calculation, the knowledge graph network extracted from expert knowledge is applied to the digital image feature extraction process to assist in the establishment of a reference model for the recognition side branch. The first loss (Loss1) is calculated by comparing the network prediction probability label of this reference model with the probability label calculated by the deep learning network.

[0026] The specific process for calculating the first loss (Loss1) is as follows: S2.1 Based on the characteristics of the steel grade, including composition, process parameters, and microstructure, the probability of defect occurrence is predicted. Based on these probability values, a probability weight is assigned to the predicted defect map, which is then converted into a low-magnification defect knowledge map, thus linking production knowledge with common low-magnification defects. The probability weight of composition in the analysis is between 0.1 and 0.3, the probability weight of process parameters is between 0.3 and 0.5, and the probability weight of microstructure is between 0.1 and 0.3.

[0027] S2.2 Based on the probability weight analysis results, knowledge graph, and on-site expert knowledge and experience, the digital features of the defect images are manually summarized. That is, based on the possible defects recommended by the knowledge graph, the features that need to be focused on are determined, and these digital features, including defect length, width, position, angle, and area, are divided by a smoothing factor, which is between 10 and 150, to adjust the distribution of feature parameters before being sent to the feature classification model for classification.

[0028] The feature classification models mentioned include methods such as support vector machines, multilayer perceptrons, clustering, and random forests.

[0029] S2.3, the classification result is converted into a probability value through the Softmax function. This probability value is then weighted and averaged with the mechanism prediction weight of the knowledge graph to obtain the predicted label of the bypass branch reference model.

[0030] The Softmax function is: Softmax(z i )=e^(Z i ) / Sum(e^(Zc)), c=[1,C) Z i Let C be the output value of the i-th node, and C be the number of output nodes, i.e., the number of categories. The Softmax function can be used to convert the output values ​​of multi-class classification into a probability distribution in the range [0, 1], with a sum of 1.

[0031] S3 calculates the second loss (Loss2) using the probability labels computed by the deep learning network and the labels of the original image supervision data.

[0032] The calculation process for the probability labels in the deep learning network is as follows: low-magnification image data is directly fed into a deep neural network model (such as ResNet) for training. The input is the original image, and the label is the on-site expert judgment result. Defect identification is performed in this step. After the identification result is divided by a smoothing factor, it is converted into probability labels via softmax. The second loss (Loss2) can then be calculated based on the probability labels and the original image supervision data labels.

[0033] Deep learning networks are a current technology. The probability labels calculated by deep learning networks are a list of intermediate calculated values, indicating the probability of each defect occurring. The deep learning network models mentioned include ResNet, Inception, EfficientNet, GoogleNet, and CNN models.

[0034] S4, calculate the total loss by weighted summation, and obtain the total residual loss of the system model; where: the total loss equals the first loss multiplied by the coefficient a plus the second loss multiplied by (1 minus the coefficient a), that is: Total Loss = Loss1*a + Loss2 * (1-a); The coefficient 'a' is between 0.0 and 1.0.

[0035] S5, iteratively calculates and updates the weights of each deep neural network in reverse.

[0036] S6. When the loss function value reaches its minimum, remains unchanged, or the accuracy no longer improves, the entire training and learning process is completed.

[0037] S7. Finally, the learned deep network model parameters are applied to the actual testing process.

[0038] The manual feature extraction methods in this invention include, but are not limited to, digital feature extraction methods based on manual induction and summarization. The feature classification models include methods such as KNN (K-Nearest Neighbor), MLP (Multilayer Perceptron), and SVM (Support Vector Machine).

[0039] The knowledge graph in this invention is a directed graph network generated through human-computer interaction based on the knowledge of experts in low-magnification defect detection. Initial defect probability weights are automatically generated according to expert experience. The application of the knowledge graph is to assist in the automatic extraction and classification of manually summarized features based on the low-magnification defect categories and characteristics corresponding to the steel tapping marks of each steel grade in continuous casting billets.

[0040] The deep learning models in this invention include, but are not limited to, ResNet, Inception, EfficientNet, GoogleNet, CNN, and other models.

[0041] The present invention provides a method for intelligent identification of low-magnification defects in continuously cast billets based on multi-model fusion. This method incorporates production mechanism knowledge from expert knowledge graphs and considers the small-data adaptability of manual feature extraction models. Furthermore, it integrates the first two advantages into the training of deep neural network models, resulting in a deeply integrated and optimized model. Example

[0042] For the standard images of defects detected at low magnification in continuously cast billets and the images collected on site, the dataset was expanded to 60,000 images using image data augmentation methods, including 50,000 images in the training set and 10,000 images in the test set. A knowledge graph for "center segregation" was established. Based on the tapping marks of commonly used steel grades, segregation defects were correlated with parameters such as the content of easily segregating elements (C, P, S, etc.), the degree of superheat, the magnitude of casting speed fluctuation, the size of the main casting machine opening, and the accuracy of arc alignment. The weight of the prediction label was given according to whether different parameters and combinations exceeded the normal limit threshold, thus linking the mechanism knowledge with the defect characteristics. A low-magnification defect knowledge graph was established, and the initial occurrence probability of defects (such as center segregation) was given as 0.65, forming probability labels [0.65, 0.3, ..., 0.65, ..., 0.5]. Among them, the probability weight of the composition in the analysis is 0.3, the probability weight of the process parameter in the analysis is 0.4, and the probability weight of the microstructure in the analysis is 0.3.

[0043] Meanwhile, based on the experience of testing experts, the digital features of defect images are manually summarized. This involves using actual data on composition, process, and microstructure (with a probability weight of 0.3 for components, 0.4 for process parameters, and 0.3 for microstructure). A low-magnification defect knowledge graph is then consulted to identify the most likely defect type, such as center segregation defects. Based on the defect characteristics of the standard center segregation graph, the sequence of digital features for this analysis is determined, including defect length, width, location, and angle. These digital features are then divided by a smoothing factor of 96 for adjustment. The feature parameter distribution is then fed into the SVM (Support Vector Machine) algorithm for classification. The classification result is then converted into probability labels [0.1, 0.1, ..., 0.8, ..., 0.2] using the Softmax function. These probability labels are weighted and averaged with the mechanistic prediction weights from the previous step, e.g., 0.65*0.5 + 0.8*0.5 = 0.725, forming probability labels [0.375, 0.2, ..., 0.725, ..., 0.35]. This yields the predicted value of the bypass branch reference model (0.725, center bias). See [link to relevant documentation]. Figure 1 .

[0044] Furthermore, the original image data is directly fed into a deep learning model (such as ResNet) for training, and divided by a smoothing factor of 96. Then, the classification result is converted into probability labels [0.2, 0.1, ..., 0.685, ..., 0.1] using the Softmax function. This probability label is then cross-entropy-calculated with the predicted label of the reference model in the previous step to obtain the first loss (Loss1), such as 0.15. Simultaneously, this probability label [0.2, 0.1, ..., 0.685, ..., 0.1] is cross-entropy-calculated with the labels of the original image supervision data to obtain the second loss (Loss2), such as 0.25.

[0045] Calculate the total loss: Total Loss = Loss1*a + Loss2*(1-a); where a = 0.5.

[0046] For example: Total Loss = 0.15*0.5+0.25*(1-0.5)=0.2.

[0047] If the total loss (e.g., 0.2) does not reach its minimum (i.e., it remains unchanged or the accuracy no longer improves), then the network weight parameters are calculated and updated, and the next iteration begins. If the total loss (e.g., 0.01) reaches its minimum, then training ends, and the model weight parameters are saved for testing and validation. See also Figure 2 .

[0048] The model trained in the previous step was used for production identification and verification. After actual testing, the center segregation identification success rate was 90.5%, and the false positive rate was 5%.

[0049] The above are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Therefore, any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for intelligent identification of low-magnification defects in continuously cast billets based on multi-model fusion, characterized in that: The steps of the intelligent recognition method are as follows: S1. Establish an image dataset and use image data augmentation on sparse samples to expand the sparse dataset and meet the requirements of statistical analysis. S2, calculate the total forward loss of the deep network. That is, during the calculation, the knowledge graph network extracted from expert knowledge is applied to the digital image feature extraction process to help establish a reference model for the recognition side branch. The first loss (Loss1) is calculated by comparing the network prediction probability label of the reference model with the probability label calculated by the deep learning network. S3, calculates the second loss (Loss2) by using the probability labels calculated by the deep learning network and the labels of the original image supervision data. S4, calculate the total loss by weighted summation, and obtain the total residual loss of the system model; where: the total loss equals the first loss multiplied by the coefficient a plus the second loss multiplied by (1 minus the coefficient a), that is: Total Loss = Loss1*a + Loss2 * (1-a); Wherein, the coefficient 'a' is between (0.0 and 1.0); S5, reverse iterative calculation to update the weights of each deep neural network; S6. When the loss function value reaches its minimum, remains unchanged, or the accuracy no longer improves, the entire training and learning process is completed. S7. Finally, the learned deep network model parameters are applied to the actual testing process. The image data enhancement processing method described above is to expand the image dataset as much as possible through image twisting, mirroring, translation, random cropping, grayscale and chroma adjustment, brightness and contrast adjustment, and random noise addition, so as to meet the model requirements for statistical analysis and processing. The calculation process of the probability label of the deep learning network is as follows: low-magnification image data is directly fed into the deep neural network model for training. The input is the original image, and the label is the on-site expert judgment result. Defect identification is performed in this step. After the identification result is divided by the smoothing factor, it is converted into a probability label through softmax.

2. The intelligent identification method for low-magnification defects in continuously cast billets based on multi-model fusion according to claim 1, characterized in that: The specific process for calculating the first loss (Loss1) is as follows: S2.1 Based on the characteristics of the steel grade, including composition, process parameters and microstructure, predict the probability value of the occurrence of defects, assign probability weights to the predicted defect map based on the probability value, and transform it into a low-magnification defect knowledge map, that is, link production knowledge with common low-magnification defects. S2.2 Based on the probability weight analysis results, knowledge graph, and on-site expert knowledge and experience, the digital features of the defect images are manually summarized. That is, based on the possible defects recommended by the knowledge graph, the features that need to be focused on are determined, and these digital features, including defect length, width, position, angle, and area, are divided by a smoothing factor to adjust the distribution of feature parameters before being sent into the feature classification model for classification. S2.3, the classification result is converted into a probability value through the Softmax function. This probability value is then weighted and averaged with the mechanism prediction weight of the knowledge graph to obtain the predicted label of the bypass branch reference model.

3. The intelligent identification method for low-magnification defects in continuously cast billets based on multi-model fusion according to claim 2, characterized in that: The probability weight of the components in the analysis is between 0.1 and 0.3, the probability weight of the process parameters in the analysis is between 0.3 and 0.5, and the probability weight of the tissue in the analysis is between 0.1 and 0.

3.

4. The intelligent identification method for low-magnification defects in continuously cast billets based on multi-model fusion according to claim 2, characterized in that: The smoothing factor is between 10 and 150.

5. The intelligent identification method for low-magnification defects in continuously cast billets based on multi-model fusion according to claim 2, characterized in that: The feature classification models mentioned include support vector machines, multilayer perceptrons, clustering, and random forest methods.

6. The intelligent identification method for low-magnification defects in continuously cast billets based on multi-model fusion according to claim 1, characterized in that: The deep learning network models mentioned include ResNet, Inception, EfficientNet, GoogleNet, and CNN models.

Images