Continuous casting blank defect classification method, system and equipment based on meta-learning

Abstract

The invention provides a meta-learning-based continuous casting blank defect classification method, system and equipment. The meta-learning-based continuous casting blank defect classification method comprises the steps of 1, preprocessing an obtained public image classification data set and an obtained continuous casting blank image containing defects; 2, randomly forming a meta-task through meta-learning task allocation on the preprocessed public image classification data set, and inputting the meta-task to a preset image meta-learning classification model to obtain a trained image meta-learning classification model; step 3, performing meta-learning task allocation on the preprocessed continuous casting blank image containing defects to form a meta-task, and performing fine adjustment on the trained image meta-learning classification model by using the obtained meta-task; 4, classifying the defects of the continuous casting blank to be processed by utilizing the final image element learning classification model; according to the method, the problem of dependence on a large amount of labeled data in a traditional defect detection method can be effectively solved, and the classification capability of the model under the condition of few samples can be improved.

Description

Technical Field

[0001] The present invention belongs to the field of continuous casting slab defect classification, and relates to a continuous casting slab defect classification method, system and equipment based on meta-learning. Background Art

[0002] Continuous casting (continuous casting) technology is a key process in modern steel production. By directly solidifying high-temperature molten metal into billets of a specific shape, it can greatly improve production efficiency and significantly reduce production costs. However, during the continuous casting process, the billet will be affected by various internal and external factors, resulting in various defects such as cracks, bubbles, inclusions and shrinkage cavities. These defects not only affect the appearance quality of the billet, but may also cause more serious problems during subsequent processing and use, thereby affecting the mechanical properties and service life of the final product. Therefore, accurate and rapid detection and classification of continuous casting billet defects is an important part of ensuring steel quality.

[0003] Traditional continuous casting slab defect detection methods mainly rely on manual inspection and rule-based image processing technology. The manual inspection method involves experienced quality inspectors visually inspecting the surface of the slab to identify defects. This method requires high experience and skills of quality inspectors, is easily affected by subjective factors, is inefficient, and is difficult to ensure consistency and accuracy. Rule-based image processing technology uses predefined image processing algorithms (such as edge detection, threshold segmentation, etc.) to identify and classify defects. Although these methods can play a certain role in specific scenarios, they have limited robustness and generalization capabilities in the face of complex and variable defect types and noise interference, and are difficult to adapt to the diverse needs in actual production.

[0004] With the rapid development of artificial intelligence technology, deep learning, especially convolutional neural network (CNN), has achieved remarkable results in the field of image processing. Compared with traditional methods, defect detection methods based on deep learning have the advantages of automatic feature extraction, high accuracy and strong adaptability. Deep learning models can automatically extract multi-level and multi-scale features from raw images without relying on manually designed features. Deep learning models trained with large-scale data perform well in image classification and target detection tasks, and can achieve high-precision defect recognition and classification. Deep learning models can maintain a high recognition rate under different defect types and complex backgrounds, and have good generalization ability.

[0005] However, the effectiveness of deep learning methods is highly dependent on large-scale, high-quality annotated datasets. In practical applications, it is often difficult and costly to obtain a large amount of annotated data, especially for data on rare defect types. In this case, how to achieve efficient defect detection and classification with limited annotated data has become an urgent problem to be solved. Summary of the invention

[0006] The purpose of the present invention is to provide a continuous casting slab defect classification method, system and equipment based on meta-learning, which solves the defects of low efficiency and low recognition rate in the existing continuous casting slab defect classification methods.

[0007] In order to achieve the above object, the technical solution adopted by the present invention is: The present invention provides a continuous casting slab defect classification method based on meta-learning, comprising the following steps: Step 1, preprocessing the obtained public image classification data set and the continuous casting ingot image containing defects respectively to obtain the preprocessed public image classification data set and the continuous casting ingot image containing defects; Step 2, randomly forming meta-tasks through meta-learning task assignment of the pre-processed public image classification data set, and inputting it into a preset image meta-learning classification model to obtain a trained image meta-learning classification model, wherein the preset image meta-learning classification model includes a preset feature extractor and a similarity measurement function module; Step 3: The preprocessed continuous casting ingot image with defects is assigned to form a meta-task through meta-learning tasks, and the trained image meta-learning classification model is fine-tuned using the obtained meta-task to obtain the final image meta-learning classification model; Step 4: Use the final image meta-learning classification model to classify the defects of the continuous casting slab to be processed.

[0008] Preferably, in step 2, the preset feature extractor is constructed by: The preprocessed public image classification dataset is input into the input image base class classification model for processing to obtain a feature fusion map; The obtained feature fusion map is input into the fully connected layer to obtain the classification probability; The obtained classification probability is used to correct the model parameters of the base class classification model, and the corrected base class classification model is used as a preset feature extractor.

[0009] Preferably, the preset feature extractor includes an encoder, which is sequentially connected to a convolutional self-attention module and a multi-scale feature fusion module, wherein: The encoder comprises four sequentially connected downsampling modules, each of which comprises a sequentially connected convolution layer, a layer normalization layer, a convolution layer, a layer normalization layer, a convolution layer, a layer normalization layer and a nonlinear activation function; The convolution self-attention module includes a first normalization module, a Token_mixer module, a second normalization module and a Channel_mixer module connected in sequence; The multi-scale feature fusion module includes multiple maximum pooling layers and multiple stacking layers, wherein the multiple stacking layers are connected in sequence, and the input end of each stacking layer is connected to a maximum pooling layer.

[0010] Preferably, the multi-scale feature fusion module further includes a channel reduction layer, and the initial fused feature map is input into the channel reduction layer to obtain the fused feature map.

[0011] Preferably, the Token_mixer module includes a first branch unit and a second branch unit; The first normalized feature map is input into the first branch unit to obtain a feature map mean and a global context output by the first branch unit; The first normalized feature map and the feature map mean are both input into the second branch unit to obtain the similarity between the first branch unit and the second branch unit calculated by the second branch unit, and the similarity is normalized; Multiplying the global context and the normalized similarity; The feature map after the multiplication process is added to the first normalized feature map for a third time.

[0012] Preferably, the Token_mixer module further includes a first DWConv unit and a first Conv unit; The feature map after the third addition process is input into the first DWConv unit; The feature map output by the first DWConv unit is input into the first Conv layer to obtain the feature map output by the first Conv unit as the initial global context attention feature map.

[0013] Preferably, the Channel_mixer module includes a second Conv unit, a second DWConv unit, a nonlinear activation function unit, a first degradation function unit, a third Conv unit and a second degradation function unit, wherein: The second normalized feature map is input into the second Conv unit; The feature map output by the second Conv unit is input into the second DWConv unit; The feature map output by the second DWConv unit is input into the nonlinear activation function unit; The feature map output by the nonlinear activation function unit is input into the first degradation function unit; The feature map output by the first degradation function unit is input into the third Conv unit; The feature map output by the third Conv unit is input to the second degradation function unit, and the feature map output by the second degradation function unit is used as the feature map of the channel mixture.

[0014] A continuous casting slab defect classification system based on meta-learning comprises the following steps: An image preprocessing unit is used to preprocess the acquired public image classification data set and the continuous casting ingot image containing defects, respectively, to obtain the preprocessed public image classification data set and the continuous casting ingot image containing defects; An image meta-learning classification model construction unit is used to randomly form meta-tasks from the pre-processed public image classification data set through meta-learning task assignment, and input it into a preset image meta-learning classification model to obtain a trained image meta-learning classification model, wherein the preset image meta-learning classification model includes a preset feature extractor and a similarity measurement function module; The pre-processed continuous casting ingot images with defects are assigned to meta-tasks through meta-learning tasks, and the trained image meta-learning classification model is fine-tuned using the obtained meta-tasks to obtain the final image meta-learning classification model. The defect classification unit is used to classify the defects of the continuous casting slab to be processed using the final image meta-learning classification model.

[0015] A computer device comprising: a processor suitable for executing a computer program; A computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, the method described is performed.

[0016] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described above is implemented.

[0017] Compared with the prior art, the present invention has the following beneficial effects: The present invention provides a continuous casting billet defect classification method based on meta-learning, which uses meta-learning technology to achieve efficient classification under the condition of a small number of defect image samples. Meta-learning technology simulates diverse task scenarios, enables the model to quickly learn and generalize under limited data conditions, and significantly improves classification efficiency. Compared with traditional methods, the present invention does not require a large amount of labeled data. Through initial training of public image classification data sets and fine-tuning containing defect images, the model can efficiently use a small number of defect image samples, which can solve the problem of data scarcity. At the same time, the present invention can more accurately measure the distance and similarity between different samples by using Mahalanobis distance as a similarity measurement function. Mahalanobis distance is a statistical measurement method that considers the distribution of data and can accurately measure the distance and similarity between different samples, thereby improving the classification effect. At the same time, combined with multi-scale feature fusion and convolution self-attention modules, the model can extract richer and more detailed features, thereby improving the accuracy of defect classification. The multi-scale feature fusion module can integrate feature information at different scales, and the convolution self-attention module can capture global context information, making the model more comprehensive and accurate in feature extraction. The meta-learning model of the present invention has the ability to quickly adapt to new tasks. When faced with different types of continuous casting defects, the model can maintain a high classification performance. Even when data is scarce, the model can maintain a stable classification effect through fine-tuning and further training. The present invention can not only effectively solve the problem of dependence on a large amount of labeled data in traditional defect detection methods, but also improve the classification ability of the model under conditions of few samples. The present invention innovatively combines deep learning and meta-learning technology, which can not only improve the accuracy and efficiency of continuous casting defect classification, but also provide new technical solutions for defect detection and image classification tasks in other fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 The overall structural design diagram of the continuous casting slab defect classification model of the present invention; Figure 2 This is the overall structural design diagram of the convolutional self-attention module in the present invention; Figure 3 This is the overall structural design diagram of the multi-scale feature fusion module in the present invention; Figure 4 The figure is a design diagram of the meta-task allocation method described in the present invention. DETAILED DESCRIPTION

[0019] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0020] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0021] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0022] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0023] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0024] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0025] Example 1 Meta-learning, also known as learning of learning, is a method that improves the learning efficiency of the model on new tasks by learning the commonalities and differences between different tasks. Meta-learning trains a model that can quickly adapt to new tasks, so that it can quickly achieve excellent performance even when faced with a small number of new task samples. This method is particularly suitable for scenarios with scarce data, such as continuous casting slab defect classification.

[0026] Therefore, the continuous casting slab defect classification method based on meta-learning provided in this application first uses a public image classification dataset to train a powerful image base class classification model, and extracts high-quality features through the model. Then, through the meta-learning framework, these features are transferred to the continuous casting slab defect classification task, so that the model can quickly learn on a small number of defect samples and achieve high-accuracy classification. This method can not only effectively solve the problem of data scarcity, but also significantly improve the robustness and accuracy of defect classification.

[0027] In summary, the continuous casting ingot defect classification method based on meta-learning can provide an efficient and reliable solution to the data scarcity problem in continuous casting ingot defect detection by combining the powerful feature extraction ability of deep learning and the rapid adaptability of meta-learning, which is of great significance to improving the quality control level of steel production.

[0028] Specifically, see Figures 1 to 4 This embodiment provides a continuous casting slab defect classification method based on meta-learning, which uses a meta-task allocation method to construct a meta-task with a balanced number of defect samples. Based on a metric-based meta-learning method, the convolutional self-attention mechanism is used to enhance the extraction of global information by a base class classifier, and the Mahalanobis distance is used on the meta-learning classifier to measure the similarity between the class prototype and the image feature tensor. Through the similarity measurement, the image feature tensor is closer to the correctly attributed prototype and away from other non-attributed prototypes, so as to accurately learn the characteristics of continuous casting slab defects and classify the defect types. Specifically, the following steps are included: Step 1: preprocess the public image classification dataset and the continuous casting ingot image containing defects. In this embodiment, the public image classification dataset adopts the Mini_ImageNet dataset. For the continuous casting ingot image containing defects, the data volume is balanced by using the data expansion method, and the classification category meets the minimum requirements of the meta-task classification.

[0029] Preprocessing may include, but is not limited to, image data cleaning, image data enhancement, label generation, and image data set partitioning in sequence. Among them, the original image is cleaned to remove possible noise, artifacts, or other interference factors. This can be achieved by applying filtering techniques, removing motion artifacts, or other image processing methods. Clean image data helps the model learn the real situation better. For image data enhancement, in order to increase the diversity and robustness of image data, various data enhancement techniques are applied, such as rotation, scaling, flipping, translation, brightness adjustment, etc., which helps to improve the generalization ability of the model and alleviate the problem of data imbalance.

[0030] Step 2: Build a base class classification model and train an excellent feature extractor using a public image classification dataset. Figure 1 , Figure 2 and Figure 3 shown.

[0031] Figure 1 The upper part is the base class classification model of this application, which includes an encoder, a convolutional self-attention module and a multi-scale feature fusion module. Specifically, by adding a convolutional self-attention module on one side of the encoder, the global attention of the feature is calculated, and then the global context information and the multi-scale feature are fused through the multi-scale feature fusion module, and finally the classification probability of the current image for each class of all categories is output through the fully connected layer. The advantage of this structure is that it can process global and local information at the same time, helping the model to better understand the image content and perform accurate segmentation, and by referring to the convolutional self-attention module and the multi-scale feature fusion module, the amount of calculation and model complexity can be reduced, thereby improving the overall classification speed.

[0032] In some embodiments, the encoder is responsible for extracting features from the input raw image and gradually reducing the image resolution. The encoder can be composed of four sequentially connected downsampling modules, each of which includes a sequentially connected convolution layer, a layer normalization layer, a convolution layer, a layer normalization layer, a convolution layer, a layer normalization layer, and a nonlinear activation function. These operations gradually convert the raw image into a high-level feature representation and increase the number of feature channels, which is helpful for extracting local features.

[0033] Specifically, a convolution operation slides a small window (convolution kernel) on the input data and performs element-by-element multiplication and summation to extract local features of the input data. The role of the convolution layer can be to learn spatial patterns in the input data, such as edges, textures, and other features. By stacking multiple convolution layers, the model can learn more abstract and complex features.

[0034] Layer normalization can be a technique used to accelerate the training of deep neural networks. When training each small batch of data, the layer normalization layer normalizes the data of each feature channel so that its mean is close to 0 and the standard deviation is adjusted to 1. This can speed up the training process, help alleviate the gradient vanishing problem, accelerate convergence, and improve the generalization performance of the model. The layer normalization layer can also have a slight regularization effect, which helps the model prevent overfitting.

[0035] Nonlinear activation functions introduce nonlinear characteristics to neural networks, allowing them to learn complex nonlinear relationships. In convolutional neural networks, commonly used nonlinear activation functions include ReLU, Sigmoid, Tanh, etc. Among them, ReLU is one of the most commonly used activation functions. It outputs the same value as the input in the positive range and outputs 0 for negative numbers. This nonlinear transformation enables neural networks to learn nonlinear patterns and increase the network's expressive power.

[0036] In some embodiments, the original image is input into the encoder, and after the downsampling operation, the intermediate downsampling resolution feature map output by each non-last downsampling module of the encoder and the last downsampling resolution feature map output by the last downsampling module can be obtained. Among them, the intermediate downsampling resolution feature map generally refers to the feature map output by the middle layer of the encoder, and the last downsampling resolution feature map refers to the feature map output by the last layer of the encoder.

[0037] Figure 2 is a schematic diagram of a convolutional self-attention module according to this embodiment, such as Figure 2 As shown, the convolutional self-attention module includes a first normalization module, a Token_mixer module, a second normalization module, and a Channel_mixer module connected in sequence, where: The first normalization module can be used to perform layer normalization operations on the input feature map, which helps to accelerate training and improve the generalization ability of the model. The first normalized feature map is obtained by inputting the intermediate downsampled resolution feature map into the first normalization module.

[0038] The first normalized feature map is input into the Token_mixer module to obtain the initial global context attention feature map.

[0039] The role of the Token_mixer module can be to provide a rich, multi-level feature representation for the entire model based on the global context by calculating the mean of global features, analyzing the similarities between different parts of features, and retaining feature details. This helps the model better understand the image and capture the global structure and local details in the image.

[0040] Each element of the initial global context attention feature map is first added to the corresponding element of the intermediate downsampled resolution feature map to obtain the first added feature map. This feature map contains information from the initial global context attention feature map and the intermediate downsampled resolution feature map, fusing global and local features.

[0041] The first added feature map is input into the second normalization module for layer normalization to obtain a second normalized feature map. This step is the same as the first normalization module. During this process, the module can calculate the mean and standard deviation of each channel in the first added feature map, and then perform layer normalization on the feature map. In this way, each channel in the second normalized feature map will have a similar value range, which helps to improve the training stability and generalization ability of the model.

[0042] The second normalized feature map is input into the Channel_mixer module to obtain a channel mixed feature map. This feature map combines the information of different channels and has a higher level of semantic information.

[0043] The Channel_mixer module can be used to fuse and transform the input second normalized feature map in the channel direction. Through a series of operations such as convolution and activation functions, the features can be made more expressive in the channel direction, so that the model can learn more complex features.

[0044] The channel mixed feature map and the first added feature map are added for the second time to obtain the final global context attention feature map. The purpose of this step is to integrate local features, global context information and channel features to obtain a more comprehensive feature representation.

[0045] In general, the design of the convolutional self-attention module can ensure that the global context information can be fully integrated into the downsampled feature map to improve the accuracy of localization target defect detection. The characteristic of this structure is that it can capture the relationship between different positions and channels, so as to better understand the semantic information of the image, which helps to improve the performance and robustness of the model.

[0046] The Token_mixer module includes a first branch unit and a second branch unit; The first normalized feature map is input into the first branch unit to obtain a feature map mean and a global context output by the first branch unit; The first normalized feature map and the feature map mean are both input into the second branch unit to obtain the similarity between the first branch unit and the second branch unit calculated by the second branch unit, and the similarity is normalized; Multiplying the global context and the normalized similarity; The feature map after the multiplication process is added to the first normalized feature map for a third time.

[0047] The Token_mixer module also includes a first DWConv unit and a first Conv unit; The feature map after the third addition process is input into the first DWConv unit; The feature map output by the first DWConv unit is input into the first Conv layer to obtain the feature map output by the first Conv unit as the initial global context attention feature map.

[0048] The principle is: The main function of the Token_mixer module is to calculate the global information of the input and tokenize the input using the global information and linear mapping. The focus is on calculating the global information of the input of the token mixer. The process of calculating the global information of the input by the token mixer can be divided into five steps: The first step is to calculate the mean in the width and height dimensions of the input through the mean operation, so as to obtain preliminary global information of the input in width and height, which is convenient for capturing important features in the input later.

[0049] The second step is to rearrange the preliminary global information and the input into a multi-head format, calculate the similarity matrix between the two tensors, use the multi-head attention mechanism to disperse the attention to different heads, and capture the relationship between different heads by calculating the similarity matrix, so as to enhance the model's perception of different features.

[0050] The third step is to calculate the mean and standard deviation of the similarity matrix and further standardize the similarity matrix. Learnable weights are added in the standardization process. The model can improve its expressiveness by freely controlling the scaling and translation of the similarity matrix.

[0051] The fourth step feeds the preliminary global information into a feedforward neural network to learn more complex global information in the input through linear and nonlinear transformations.

[0052] The fifth step is to multiply the standardized similarity matrix with the global information, and weight the similarity matrix by the global information so that the global information affects the calculation of each local feature to obtain the final global information.

[0053] After the above five steps of calculation, the global information of the input is obtained, and then the global information is integrated into the input through the residual connection. The label mixer is superimposed twice in the actual experiment to fully learn the global information in the input, thereby achieving the purpose of enhancing the model's ability to express global information.

[0054] The Channel_mixer module includes a second Conv unit, a second DWConv unit, a nonlinear activation function unit, a first degradation function unit, a third Conv unit and a second degradation function unit, wherein: The second normalized feature map is input into the second Conv unit; The feature map output by the second Conv unit is input into the second DWConv unit; The feature map output by the second DWConv unit is input into the nonlinear activation function unit; The feature map output by the nonlinear activation function unit is input into the first degradation function unit; The feature map output by the first degradation function unit is input into the third Conv unit; The feature map output by the third Conv unit is input to the second degradation function unit, and the feature map output by the second degradation function unit is used as the feature map of the channel mixture.

[0055] Figure 3 is a schematic diagram of a multi-scale feature fusion module according to this embodiment, such as Figure 3 As shown, the multi-scale feature fusion module includes multiple maximum pooling modules and multiple overlay modules, wherein the multiple overlay layers are connected in sequence, and the input end of each overlay layer is connected to a maximum pooling layer.

[0056] In some embodiments, the maximum pooling module can be used to downsample the global context attention feature map to solve the scale mismatch fusion problem between the multi-scale global context attention feature map generated by the multi-scale global context module and the low-resolution feature map obtained by normal downsampling. In each maximum pooling module, each area of ​​the feature map is divided into several sub-areas, and then the maximum value in each sub-area is retained, and the remaining values ​​are discarded. In this way, the resolution of the feature map can be reduced, the main features can be retained, and the secondary features can be discarded, thereby reducing the amount of calculation and extracting the main information of the feature.

[0057] The global context attention feature map is downsampled by the maximum pooling module in turn. After each downsampling, the feature map of the same resolution can be superimposed in the channel dimension through an overlay module. The overlay module can integrate feature information from different sources. This integration can increase the diversity and expressiveness of features, and help capture richer and more complex image features.

[0058] The input global context attention feature map is subjected to continuous maximum pooling and dimension stacking, and the feature map output by the last maximum pooling module and the feature map of the last layer downsampled resolution are stacked in the channel dimension through the last stacking module to obtain the initial fused feature map. This initial fused feature map can be a large channel low-resolution feature map, which integrates the different scale information of the global context attention feature map and the local features of the feature map of the last layer downsampled resolution, and has more comprehensive information.

[0059] Through such a design, the multi-scale feature fusion module can effectively combine global and local information to generate a fused feature map. This fused feature map integrates information at different scales and levels, and can provide a richer and more accurate feature representation for subsequent defect detection. This multi-scale feature fusion helps improve the performance of the detection model, making it more robust and generalizable.

[0060] The principle is: Through the convolution self-attention module, four feature maps with 16, 64, 128, and 256 channels are obtained. Before these four feature maps are fused with the output of ResNet12, the multi-scale feature fusion module does not change the number of channels of any feature map, and uses the maximum pooling operation to downsample the feature map of the current level to make it consistent with the scale of the feature map of the next level, and then connects it in the channel dimension to generate a new feature map. For example, the first feature map has 16 channels, and the second feature map has 64 channels. The two are fused, and the scale is consistent with the second feature map. The number of channels is the sum of the two, which is 80, and so on, until it is consistent with the feature scale of the ResNet12 output.

[0061] After the step-by-step maximum pooling operation and channel axis connection operation, a feature map with 464 channels and multi-scale global information is obtained. The feature map is consistent with the output of ResNet12 in scale. This paper also performs a channel axis connection operation on the feature map and the output of ResNet12. As shown in the figure, a feature map with 976 channels and local and global information is obtained. However, at this time, the local information and global information of the feature map connected by the channel axis are not truly fused. Therefore, this paper allows this feature map to pass through a simple channel mixer to deeply fuse the local information of the feature map with the global information, and obtain the final multi-scale fused feature map.

[0062] The multi-scale feature fusion module designed by the above-mentioned step-by-step maximum pooling operation and channel axis connection operation can effectively fuse multi-scale feature information, so that the generated feature map has rich expression capabilities in both local and global information, which helps the network better understand and capture the structure and semantic information in the image. This paper maintains the consistency of the features output by ResNet12 through the channel mixer, while ensuring that the overall performance is not too disturbed, and further deeply fuses local and global information, thereby enhancing the representation ability of the feature map and providing a more powerful feature representation.

[0063] Step 3: Construct the meta-tasks necessary for the meta-learning classification model and train the meta-learning classification model. The meta-learning classification model is as follows: Figure 1 As shown in the figure, the meta-task allocation method is as follows Figure 4 shown.

[0064] Figure 1 The lower part is the meta-learning classification model. Through learning and training on public image classification datasets, the feature extractor of the base class classification model can effectively extract feature information from the image. Combined with the classification probability output by the final fully connected layer, it can basically realize the classification of images of different categories. However, the classification result of the base class classification model is for the full training of the public image classification dataset. The limitations of the defective continuous casting ingot images themselves make it impossible to directly train the model on the base class classification model, and thus obtain the ideal classification results. To solve this problem, the present application removes the last fully connected layer in the base class classification model, and directly uses its feature extractor as the feature extractor of the meta-learning classification model, so as to enhance the meta-learning classification model's ability to extract input features.

[0065] The network structure of the meta-learning classification model is basically the same as that of the base class classifier. The only difference is that this application does not directly use the feature information extracted from the support set of the meta-task and the query set as the basis for classification, but further generates class prototypes from the extracted feature information, and then measures the distance (i.e., similarity) between the class prototypes generated by the query set and the multiple class prototypes generated by the support set through the subsequent similarity measurement function, and uses the similarity score as the basis for classification. Figure 4 As shown in Figure 1, the training of the meta-learning classification model is roughly divided into three steps, namely, meta-learning task allocation, class prototype extraction and generation, and class prototype similarity measurement, where: Meta-learning task allocation. In the meta-training stage, this embodiment randomly generates a meta-training set from a public image classification dataset and classifies it according to its corresponding class label. When generating meta-tasks, randomly sample from the meta-training set by random sampling. categories, and then from this Random sampling from samples, forming a meta-task sample set, where samples form the support set of the meta-task, For the support set generated by random sampling, this embodiment does not disrupt the order of various samples in the support set, that is, the data dimension of the support set is , where the first two dimensions represent that there are N categories in total, each category contains K samples, and the last three dimensions represent the number of channels, width, and height of the samples. The reason why the order of samples in the support set is not disrupted is that the K samples of each category will be used to generate more representative class prototypes later. For the query set generated by random sampling, the present invention does not consider the order of samples in the query set, that is, the data dimension of the query set is , which has one less data dimension than the support set. This allocation is to reduce the impact of the sample placement in the query set in the order of categories when calculating the class prototype and similarity measure later. That is, the samples in the query set are staggered and there is no interference from category clustering. In this way, it is guaranteed that the query of each sample in the query set is independent, further highlighting the randomness of the query, making the classification accuracy obtained by the meta-learning classification model more convincing.

[0066] Class prototype extraction and generation. After meta-learning task allocation, this embodiment obtains a lot of meta-tasks. Each meta-task has a support set and a query set. Now it is necessary to extract feature information from the samples in the support set and the query set, and generate class prototypes. The feature extractor of the meta-learning classification model is derived from the base class classification model. Its feature extractor has undergone full classification training of the public image classification data set. Compared with the feature extractor of a classifier trained from scratch, the migrated feature extractor has a stronger feature extraction capability. Through meta-task allocation analysis, it can be known that the data dimension of the support set is five-dimensional, and the data dimension of the query set is four-dimensional. Therefore, in order to ensure that the feature extractor performs normal feature extraction, this embodiment needs to first superimpose the data of the first two dimensions of the support set. Because the feature extractor does not modify the input batchsize dimension, there is no need to consider the problem of data order being disrupted. After the feature extractor ends, the original data dimension is restored.

[0067] The reason why this embodiment does not support each category of the The reason why the samples are mixed is that the class prototype formed by a single sample is not universal. In the meta-learning task, the samples in the support set are used to construct class prototypes, which will be used for model adaptation and generalization. However, if only the class prototype formed by a single sample is used to calculate the similarity with each sample in the query set, the similarity measurement result may be very poor. Since a single sample cannot well represent the characteristics and changes of the entire category, in order to improve the generalization performance of the model on the query set, the obtained class prototypes of each category are mixed. The sample feature information is averaged by The feature information of each sample is averaged. This embodiment can obtain a more representative class prototype, thereby improving the credibility of the subsequent similarity measurement function for similarity measurement of the feature information of the class prototype and the query set.

[0068] Similarity metric of class prototype. The similarity metric function is generally defined by a distance function or a cost function. It is a criterion for measuring the similarity between a feature and a feature prototype. It is also the key basis for metric-based meta-learning methods. It is very important to select and design an effective similarity metric function, which will directly affect the adaptability of the model to new tasks. After the samples of the meta-task support set are processed by the feature extractor and the averaging operation, The class prototype of each category, and the query set generates A tensor that condenses sample features. This embodiment requires Tensors are calculated separately to determine the similarity score between each tensor and each class prototype.

[0069] Common distance functions used for similarity measurement include cosine similarity, Euclidean distance, Manhattan distance, and Mahalanobis distance. Different distance functions are applied in different scenarios. The Mahalanobis distance considers the covariance structure of the data and is suitable for high-dimensional data. The Mahalanobis distance is used to measure the similarity of data points relative to the entire data set, which is very consistent with calculating the similarity score of a single feature tensor based on the continuous casting billet defect sample class prototype. This embodiment uses the Mahalanobis distance to measure the similarity between the query set feature tensor and the class prototype.

[0070] However, there is a necessary prerequisite for calculating the Mahalanobis distance, which is to be able to calculate the inverse matrix of the joint covariance matrix of the query set feature tensor and the class prototype, which requires the reversibility of the corresponding joint covariance matrix. This embodiment adopts a regularization method for the joint covariance matrix, by adding a very small number multiplied by the unit matrix to the joint covariance matrix, so as to ensure that the joint covariance matrix is ​​positive definite, thereby avoiding numerical instability and solving the problem of the irreversibility of the joint covariance matrix. In addition, this embodiment adds a unit matrix to the joint covariance matrix, which does not change the properties of the joint covariance matrix, because the special properties of the unit matrix cause it to act in addition operations similar to zero elements. In this way, the similarity between the query set feature tensor and the class prototype can be calculated by the Mahalanobis distance.

[0071] Step 5: Fine-tune and test. The specific process is as follows: The continuous casting ingot images containing defects are converted into new meta-tasks according to the meta-learning task allocation method. Through this task allocation method, different defect classification tasks are simulated, so that the model can adapt to the actual application scenarios of continuous casting ingot defect images during the fine-tuning process.

[0072] Furthermore, the meta-learning classification model is fine-tuned. During the fine-tuning process, a small number of defect image samples are used to further train and adjust the meta-learning classification model. A part of the samples are randomly extracted from the continuous casting billet images containing defects as the support set (Support Set), and the other part of the samples are used as the query set (Query Set), forming a new meta-task. Then the images of the support set and the query set are input into the feature extractor to extract the corresponding feature representation to ensure the accuracy of subsequent classification. Based on the features of the support set, a prototype vector of each class is generated to represent the central features of the class.

[0073] Furthermore, the Mahalanobis distance is used to calculate the similarity between each sample feature in the query set and each type of prototype vector. Based on the similarity measurement results, the samples in the query set are classified. By comparing the predicted labels of the query set samples with the true labels, the classification loss is calculated, and the optimization algorithms such as gradient descent are used to update the model parameters and reduce the classification loss. In the fine-tuning process, through multiple iterative optimizations, the model gradually improves its performance in the defect classification task. Through fine-tuning, the model can further adapt to the characteristics of continuous casting billet defect images and improve the accuracy and robustness of classification.

[0074] This embodiment provides a continuous casting slab defect classification method based on meta-learning, which can significantly improve the efficiency and accuracy of defect classification by combining deep learning and meta-learning technology. The method first uses a public image classification dataset for initial model training, and extracts high-quality image features through a convolutional self-attention module and a multi-scale feature fusion module. Subsequently, the Mahalanobis distance is used as a similarity metric function to generate class prototypes, and a variety of task scenarios are formed through meta-learning task allocation to enhance the generalization ability of the model.

[0075] In practical applications, continuous casting ingot images containing defects are preprocessed to form a new meta-task. The meta-learning classification model is fine-tuned to ensure that defective images can be accurately classified even with a small number of samples. The fine-tuning process uses a small number of defective image samples to further train and optimize the model, making it perform better in the actual defect classification task.

[0076] This embodiment can not only effectively solve the problem of data scarcity, but also has the ability to quickly adapt to new tasks. Its versatility and flexibility make it suitable for continuous casting slab defect detection and defect detection and image classification tasks in other fields. By improving the technical level and efficiency of defect classification, the present invention can significantly improve the quality control in the steel production process, reduce the generation of unqualified products, improve production efficiency and product quality, and has broad application prospects and economic benefits.

[0077] Example 2 A continuous casting slab defect classification method based on meta-learning comprises the following steps: An image preprocessing unit is used to preprocess the acquired public image classification data set and the continuous casting ingot image containing defects, respectively, to obtain the preprocessed public image classification data set and the continuous casting ingot image containing defects; An image meta-learning classification model construction unit is used to randomly form meta-tasks from the pre-processed public image classification data set through meta-learning task assignment, and input it into a preset image meta-learning classification model to obtain a trained image meta-learning classification model, wherein the preset image meta-learning classification model includes a preset feature extractor and a similarity measurement function module; The pre-processed continuous casting ingot images with defects are assigned to meta-tasks through meta-learning tasks, and the trained image meta-learning classification model is fine-tuned using the obtained meta-tasks to obtain the final image meta-learning classification model. The defect classification unit is used to classify the defects of the continuous casting slab to be processed using the final image meta-learning classification model.

[0078] Example 3 This embodiment 3 provides a computer device, including: a memory for storing a computer program; and a processor for implementing the steps of a computer method when executing the computer program.

[0079] When the processor executes the computer program, the steps of the above-mentioned computer method are implemented, or when the processor executes the computer program, the functions of each module in the above-mentioned system are implemented.

[0080] Exemplarily, the computer program may be divided into one or more modules / units, which are stored in the memory and executed by the processor to implement the present invention. The one or more modules / units may be a series of computer program instruction segments capable of implementing preset functions, and the instruction segments are used to describe the execution process of the computer program in the computer device.

[0081] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, and a cloud server. The computer device may include, but is not limited to, a processor and a memory. Those skilled in the art will appreciate that the above are examples of computer devices and do not constitute a limitation on computer devices. The computer device may include more components than the above, or a combination of certain components, or different components. For example, the computer device may also include input and output devices, network access devices, buses, etc.

[0082] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the computer device, and uses various interfaces and lines to connect various parts of the entire computer device.

[0083] The memory may be used to store the computer programs and / or modules, and the processor implements various functions of the computer device by running or executing the computer programs and / or modules stored in the memory and calling the data stored in the memory.

[0084] The memory may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (SmartMediaCard, SMC), a secure digital (SecureDigital, SD) card, a flash card (FlashCard), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0085] Example 4 This embodiment 4 also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps of the method described are implemented.

[0086] If the module / unit integrated in the computer system is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium.

[0087] Based on such understanding, the present invention implements all or part of the processes in the above method, and can also be completed by instructing related hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and when the computer program is executed by a processor, the steps of the above computer method can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or preset intermediate form, etc.

[0088] The computer-readable storage medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium, etc.

[0089] It should be noted that the content contained in the computer-readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable storage media do not include electrical carrier signals and telecommunication signals.

[0090] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A continuous casting slab defect classification method based on meta-learning, characterized in that: The following steps are involved: Step 1, preprocessing the obtained public image classification data set and the continuous casting ingot image containing defects respectively to obtain the preprocessed public image classification data set and the continuous casting ingot image containing defects; Step 2, randomly forming meta-tasks through meta-learning task assignment of the pre-processed public image classification data set, and inputting it into a preset image meta-learning classification model to obtain a trained image meta-learning classification model, wherein the preset image meta-learning classification model includes a preset feature extractor and a similarity measurement function module; Step 3: The preprocessed continuous casting ingot images with defects are assigned to meta-tasks through meta-learning tasks, and the trained image meta-learning classification model is fine-tuned using the obtained meta-tasks combined with the similarity measurement function to obtain the final image meta-learning classification model; Step 4: Use the final image meta-learning classification model to classify the defects of the continuous casting slab to be processed.

2. The continuous casting slab defect classification method based on meta-learning according to claim 1, characterized in that: In step 2, the preset feature extractor is constructed as follows: The preprocessed public image classification dataset is input into the input image base class classification model for processing to obtain a feature fusion map; The obtained feature fusion map is input into the fully connected layer to obtain the classification probability; The obtained classification probability is used to correct the model parameters of the base class classification model, and the corrected base class classification model is used as a preset feature extractor.

3. A continuous casting slab defect classification method based on meta-learning according to claim 1 or 2, characterized in that: The preset feature extractor includes an encoder, which is sequentially connected to a convolutional self-attention module and a multi-scale feature fusion module, wherein: The encoder comprises four sequentially connected downsampling modules, each of which comprises a sequentially connected convolutional layer, a layer normalization layer, a convolutional layer, a layer normalization layer, a convolutional layer, a layer normalization layer and a nonlinear activation function; The convolution self-attention module includes a first normalization module, a Token_mixer module, a second normalization module and a Channel_mixer module connected in sequence; The multi-scale feature fusion module includes multiple maximum pooling layers and multiple stacking layers, wherein the multiple stacking layers are connected in sequence, and the input end of each stacking layer is connected to a maximum pooling layer.

4. The continuous casting slab defect classification method based on meta-learning according to claim 3 is characterized in that: The multi-scale feature fusion module also includes a channel reduction layer, and the initial fused feature map is input into the channel reduction layer to obtain the fused feature map.

5. The continuous casting slab defect classification method based on meta-learning according to claim 3, characterized in that: The Token_mixer module includes a first branch unit and a second branch unit; The first normalized feature map is input into the first branch unit to obtain a feature map mean and a global context output by the first branch unit; The first normalized feature map and the feature map mean are both input into the second branch unit to obtain the similarity between the first branch unit and the second branch unit calculated by the second branch unit, and the similarity is normalized; Multiplying the global context and the normalized similarity; The feature map after the multiplication process is added to the first normalized feature map for a third time.

6. The continuous casting slab defect classification method based on meta-learning according to claim 3 is characterized in that: The Token_mixer module also includes a first DWConv unit and a first Conv unit; The feature map after the third addition process is input into the first DWConv unit; The feature map output by the first DWConv unit is input into the first Conv layer to obtain the feature map output by the first Conv unit as the initial global context attention feature map.

7. The continuous casting slab defect classification method based on meta-learning according to claim 3 is characterized in that: The Channel_mixer module includes a second Conv unit, a second DWConv unit, a nonlinear activation function unit, a first degradation function unit, a third Conv unit and a second degradation function unit, wherein: The second normalized feature map is input into the second Conv unit; The feature map output by the second Conv unit is input into the second DWConv unit; The feature map output by the second DWConv unit is input into the nonlinear activation function unit; The feature map output by the nonlinear activation function unit is input into the first degradation function unit; The feature map output by the first degradation function unit is input into the third Conv unit; The feature map output by the third Conv unit is input to the second degradation function unit, and the feature map output by the second degradation function unit is used as the feature map of the channel mixture.

8. A continuous casting slab defect classification system based on meta-learning, characterized in that: The following steps are involved: An image preprocessing unit is used to preprocess the acquired public image classification data set and the continuous casting ingot image containing defects, respectively, to obtain the preprocessed public image classification data set and the continuous casting ingot image containing defects; An image meta-learning classification model construction unit is used to randomly form meta-tasks from the pre-processed public image classification data set through meta-learning task assignment, and input it into a preset image meta-learning classification model to obtain a trained image meta-learning classification model, wherein the preset image meta-learning classification model includes a preset feature extractor and a similarity measurement function module; The preprocessed continuous casting ingot images with defects are assigned to meta-tasks through meta-learning tasks. The trained image meta-learning classification model is fine-tuned using the obtained meta-tasks combined with the similarity metric function to obtain the final image meta-learning classification model. The defect classification unit is used to classify the defects of the continuous casting slab to be processed using the final image meta-learning classification model.

9. A computer device, characterized in that: include: a processor suitable for executing a computer program; A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by the processor, the method according to any one of claims 1 to 7 is performed.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images