A steel grain size grading method based on guided self-attention mechanism

Abstract

The application provides a steel grain size grading method based on a guided self-attention mechanism. First, the method collects a steel sample slice from a steel material to be graded, and then uses an optical microscope to collect a grain size picture of the sample. After the grain size picture is preprocessed by a fuzzy logic method, it is input into a guided self-attention mechanism network. The method outputs the corresponding steel grain size grade and completes the grain size grade labeling of the steel material. The method can solve the problems of low precision and long time consumption of the current manual grading method of the steel industry, and does not require a large server as support and does not need to load a pre-trained model to achieve a steel grain size grading accuracy of 99.8%.

Description

Technical Field

[0001] This invention belongs to the field of steel and relates to a steel grain size classification method based on a guided self-attention mechanism, which is applicable to the steel grain size classification problem. Background Technology

[0002] During the steelmaking process, steel crystallization forms steel grains. Grain size is a measure of grain size, commonly expressed as the number of grains per unit volume or area, or the average linear length or diameter of the grains. Standard grain size is divided into 12 grades: grades 1-4 are coarse grains, grades 5-8 are fine grains, and grades 9-12 are ultrafine grains. Different applications have different requirements for steel grain size. At room temperature, a higher grain size grade results in greater part strength. However, at high temperatures, a smaller grain size grade results in greater high-temperature resistance. Therefore, grain size grade is an important indicator for evaluating the performance, quality, and lifespan of metallic materials, and the accuracy of the grade directly affects the judgment of the safety performance of steel materials. Currently, three national standard grading methods are commonly used in industrial production: the comparison method, the intercept method, and the area method.

[0003] The comparative method involves manually comparing the image to be rated with the labeled image at a uniform scale, selecting the grade with the closest average grain size as the rating result. The intercept method determines the grain size by calculating the number of intersections between a given length of measurement line segment or grid and the grain boundary. The area method calculates the grain size grade by counting the number of grains that fall completely within a given grid and those that are cut off, and then using a standard formula. Although the comparative, intercept, and area methods are simple to operate, they require manual intervention, and when the number of grains increases exponentially, it is impossible to accurately grade the grain size of steel by relying solely on the naked eye.

[0004] Furthermore, existing deep learning-based methods for steel grain size classification generally suffer from low accuracy and poor generalization. Gajalakshmi et al. used Otsu and Canny networks to preprocess steel grain size images, filtering out texture information and retaining grain edge information. They then used a feature extraction network to calculate a two-dimensional feature vector representing the number of grains. Finally, they used a Support Vector Regression (SVR) network to learn the extracted features and fit the steel grain size classification task. While SVR-based top-level grain size methods use image preprocessing networks to filter out redundant information and normalize pixel distribution to between 0 and 1, effectively reducing the difficulty of feature extraction, the robustness of SVR models is low. When the amount of data is too large, overfitting can easily occur, leading to incorrect classification. Lee et al. proposed a fast image classification network based on convolutional neural networks. This fast image classification network based on convolutional neural networks uses residual connection networks and local cross-level networks to extract grain size features. The fast image classification network based on convolutional neural networks has a fast operation speed, processing each image in only 14 milliseconds. However, due to the overly simple model structure, its generalization performance is poor when the number of grain size levels is large.

[0005] The Siwen converter network is an improvement on the ViT network, so it performs worse than convolutional neural networks on small datasets. Most current work on converter networks focuses on achieving significant performance on large datasets, but this often requires large servers and extensive debugging experience, resulting in poor general applicability. Summary of the Invention

[0006] This invention proposes a steel grain size classification method based on a guided self-attention mechanism. This method uses fuzzy logic to denoise steel grain size images and enhance edge features. Then, a guided self-attention mechanism helps the model identify the grain size image classification for accurate steel grain size classification. The final accuracy of this guided self-attention mechanism-based steel grain size classification method is 99.8%, exceeding the accuracy of manual classification. The steps in its use are as follows:

[0007] Step (1): Acquire grain size images of steel samples using an optical microscope, and then preprocess the grain size images using fuzzy logic method to filter out redundant texture information and noise, and enhance edge information;

[0008] Step (2): The feature encoder is proposed to combine convolutional neural network and Siwen converter network in the way of downsampling. The preprocessed grain size image will be simultaneously input into the dense network branch and Siwen converter network branch in the feature encoder. The dense network branch consists of 4 dense network modules and the Siwen converter network branch consists of 3 Siwen converter modules.

[0009] After the first Siwen converter module completes the downsampling feature extraction of the grain size image, the resulting feature map is... Feature map obtained from the first dense network module Merge along the channel dimension; the merged feature map The feature map will be input into the second dense network module. The feature map after downsampling with the second document converter module The process of merging is repeated, and finally, the fourth dense network module outputs the feature map. ;

[0010] Because the Siwen converter module contains a sliding window multi-head self-attention mechanism, this mechanism enables spatially distant pixel blocks in the grain-level image to connect, thereby generating global feature information for the grain-level image. Therefore, the feature map... Rich in global feature information, on the other hand, because the convolutional kernel of a convolutional neural network scans local pixel blocks sequentially on the image, the feature map... Rich in local feature information, therefore derived from the feature map and feature map Feature map obtained by merging It can integrate global and local feature information;

[0011] when After being processed by the second dense network module, the resulting feature map This will be used for data containing global and local feature information. Further local feature information extraction is performed, and so on, until finally, the feature map is obtained after processing by the feature encoder. It can effectively fuse global and local feature information in grain size images;

[0012] Step (3): Feature Map Linear mapping of the bond matrix Sum matrix Feature map obtained from network processing of the document converter Mapping the query matrix Since the feature encoder in step (2) processes the feature map, The key matrix obtained by mapping It is also rich in both global and local feature information. Then, let the key matrix... Vectors and query matrices The vectors in the matrix are used to calculate cosine similarity. Finally, the weight coefficients are calculated using a Softmax layer, and the value matrix is ​​then processed. By applying constraints, a matrix can be obtained. :

[0013]

[0014] in, Key matrix Sum matrix dimensionality This is used in the Siwen converter network to mark the relative positional deviation between pixel blocks. Represents the key matrix Perform a transpose operation;

[0015] In a guided self-attention mechanism-based steel grain size classification method, the bond matrix... Sum matrix Feature map obtained from feature encoder Query matrix Feature map obtained from Siwen converter network Query matrix Key matrix Sum matrix Since they do not originate from the same feature map, a guided self-attention mechanism-based steel grain size classification method can effectively guide the Siwen converter network to find the matrix. ;

[0016] Step (4): A three-stream fusion module is proposed to enhance the generalization ability of a guided self-attention mechanism-based steel grain size classification method by utilizing multiple attention mechanisms and connection strategies; final feature map for:

[0017]

[0018] in, , , , This indicates that the feature map has been processed by the average pooling layer. This represents the dot product operation, i.e., the feature map. With feature map Multiply the pixels at corresponding positions;

[0019] This represents the feature map obtained from the guided self-attention module, i.e., the matrix. ;

[0020] Processed by the channel attention module and the global attention module. This indicates that a global attention module is used to extract global feature information. This indicates that channel attention modules are used to extract channel feature information. Both channel attention modules and global attention modules are convolutional attention modules. The channel attention module dynamically assigns weights to different channels in the feature map, accelerating convergence, while the global attention module extracts global feature information, enabling... Similar to those rich in global feature information Greater compatibility facilitates subsequent feature fusion;

[0021] The average pooling layer is size-reduced; then residual connections and and The dot product results are merged to obtain the final feature map. ;

[0022] Step (5): Based on the feature map Mapped from linear layers The classification descriptor vectors of different sizes are used, and the final classification result is determined by the cross-entropy loss function and the deep learning backpropagation mechanism.

[0023] The present invention has the following advantages and effects compared with the prior art:

[0024] (1) The fuzzy logic method is used to preprocess the grain size image, effectively filtering out redundant texture information and noise, and enhancing edge information.

[0025] (2) Using downsampling combined with convolutional neural networks and Siwen converter networks can enable the model to integrate the advantages of dense networks and Siwen converters, namely, to effectively utilize local receptive fields, efficiently reuse features, effectively fuse local and global feature information, use dynamic attention mechanisms to accelerate model convergence, and achieve better model generalization ability.

[0026] (3) The proposed three-stream fusion module enables the model to quickly locate the average grain size, and the channel attention module can dynamically assign dynamic attention weights to the feature map channels, thus accelerating convergence. Attached Figure Description

[0027] Figure 1 This is an overall flowchart of the method of the present invention.

[0028] Figure 2 This is an overall framework diagram of the method of the present invention.

[0029] Figure 3 This is a framework diagram of the feature encoder of the method of the present invention.

[0030] Figure 4 This is a framework diagram of the guided self-attention module of the method of the present invention.

[0031] Figure 5 This is a three-stream fusion framework diagram of the method of the present invention. Detailed Implementation

[0032] The present invention proposes a method for steel grain size classification based on a guided self-attention mechanism, which is described in detail below with reference to the accompanying drawings:

[0033] Figure 1 This is an overall flowchart of the method of the present invention. A method for steel grain size classification based on a guided self-attention mechanism first collects steel sample slices from the steel material to be classified. Then, grain size images of the samples are acquired using an optical microscope. The acquired grain size images are preprocessed using a fuzzy logic method and then input into a guided self-attention mechanism network GSNet. Finally, the corresponding steel grain size grade is output, and the steel material is labeled.

[0034] Figure 2 This is an overall framework diagram of the method of the present invention. First, the preprocessed steel grain size image is processed by a feature encoder to generate a feature map that can efficiently fuse global and local feature information, as shown in the visualization attention mechanism map 1. and Although they come from different image regions, they can still exchange information, representing the extracted global feature information. and Individual steel grains, representing the average grain size, are used to represent the extracted local feature information. Then, a guided self-attention mechanism is used to guide the Siwen converter network to obtain the matrix. Finally, the three-stream fusion module locates individual grains by merging three feature maps processed by three different mechanisms, thereby determining the grain size level.

[0035] Figure 3 This is a framework diagram of the feature encoder of the method of the present invention. The feature encoder contains a dense network branch and a document converter network branch, wherein the dense network branch consists of 4 dense network modules and the document converter network branch consists of 3 document converter modules.

[0036] The internal structure of the four dense network modules can be referenced. Figure 3 Dense network module 1, the dense network module is composed of dense layers, and the dense layers contain convolution kernel size and The convolutional kernels are of varying sizes, and dense network modules 1-4 contain 6, 12, 24, and 16 dense layers, respectively. The internal structures of the three document converter modules can be found in [reference needed]. Figure 3The Chinese-to-English converter module 1 comprises the following modules: a region fusion module divides the pixels in the feature map and merges the divided pixels into pixel blocks; a linear layer 1 maps the pixel blocks into vector combinations and inputs them to a multi-head attention mechanism, which outputs a feature map with weighted coefficients; a linear layer 2 maps the feature map into vector combinations and inputs them to a multilayer perceptron, which outputs a feature map with weighted coefficients; a linear layer 3 maps the feature map into vector combinations and inputs them to a sliding window multi-head attention mechanism, which uses a sliding window mechanism to cause displacement deviation in the pixel blocks and outputs a feature map with displacement deviation weights; and a linear layer 4 maps the feature map into vector combinations and inputs them to a multilayer perceptron, which outputs a feature map with weighted coefficients.

[0037] This invention employs a sliding window multi-head self-attention mechanism in the document converter network module to construct information interaction between distant regions, and then utilizes a multilayer perceptron to extract global feature information. Next, convolution in the dense connection network module is used to extract local feature information from the feature map. Finally, the feature encoder merges the features extracted from the document converter network module and the dense connection network module, i.e. Figure 3 Middle feature map To effectively merge global and local feature information, feature maps... The data will be passed to the next densely connected network module, meaning that the convolution will continue to extract local feature information from the feature map containing global feature information. Therefore, after processing by the feature encoder module, the feature map can effectively and comprehensively fuse global and local feature information.

[0038] Figure 4 This is a framework diagram of the guided self-attention module of the method of the present invention. First, based on the feature map... Linear mapping of the bond matrix Sum matrix Based on the feature map obtained from the document converter network processing Mapping the query matrix Because it has been processed by the feature encoder, based on the feature map The key matrix obtained by mapping It is also rich in both global and local feature information. Then, let the key matrix... Vectors and query matrices The vectors in the matrix are used to calculate cosine similarity. Finally, the weight coefficients are calculated using a Softmax layer, and the value matrix is ​​then processed. By applying constraints, a matrix can be obtained. :

[0039]

[0040] in, Key matrix Sum matrix dimensionality This is used in the Siwen converter network to mark the relative positional deviation between pixel blocks. Represents the key matrix Perform a transpose operation. For example... Figure 4 As shown, "Visual Attention Mechanism Map 1", "Visual Attention Mechanism Map 2", and "Visual Attention Mechanism Map 3" were obtained by the feature encoder, the text converter network, and the guided self-attention module, respectively. As shown in Visual Attention Mechanism Map 1, although the feature encoder helps the model more effectively combine global and local feature information, for example... and Although they are individual steel grains, they can generate interactive information, representing the extracted global feature information. However, there are still steel grains that are too far apart to interact, such as... and On the other hand, the Siwen converter network can learn global feature information better, such as the attention mechanism graph in Visual Attention Mechanism 2. As shown, it can construct global feature information for a large area, but it cannot locate a suitable individual grain to represent the average grain size of the grain size image. With the help of a guided self-attention module, the method of this invention can not only learn the interaction information between distant grains, such as in the visualization attention mechanism map 3, but also... and As shown, it can also locate individual grains to characterize the average grain size, as in the visualization of the attention mechanism map 3. and As shown.

[0041] Figure 5 This is a diagram of the three-stream fusion framework of the method of this invention. First, feature maps are obtained through guided self-attention modules. It can effectively fuse local and global feature information. Then, the channel attention module and the global attention module process the data to obtain the feature map. Both the channel attention module and the global attention module are convolutional-based attention modules, capable of extracting channel information and global feature information, respectively. The channel attention module dynamically assigns weights to different channels in the feature map, accelerating convergence, while the global attention module extracts global feature information, enabling... Similar to those rich in global feature information Greater compatibility facilitates subsequent feature fusion. Finally, and Perform dot product, i.e., feature map With feature map The pixels at corresponding positions are multiplied, and then connected to the feature map processed by the average pooling layer via residual connections. The features are merged to obtain feature maps that can locate individual grains, which helps the model determine the final grain size level.

[0042] The above description is only a preferred embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A method for grading steel grain size based on a guided self-attention mechanism, characterized in that, The steps during use are as follows: Step (1): Use an optical microscope to acquire grain size images of steel samples, and use fuzzy logic method to preprocess the grain size images to filter out redundant texture information and noise. Step (2): The preprocessed grain size image will be simultaneously input into the dense network branch and the Siwen converter network branch in the feature encoder. The dense network branch consists of 4 dense network modules, and the Siwen converter network branch consists of 3 Siwen converter modules. After the first Siwen converter module completes the downsampling feature extraction of the grain size image, the resulting feature map is... Feature map obtained from the first dense network module Merge along the channel dimension; the merged feature map The feature map is input into the second dense network module. Feature map after downsampling with the second document converter module The process of merging is repeated, and finally, the fourth dense network module outputs the feature map. ; Step (3): Feature Map Linear mapping of the bond matrix Sum matrix Feature map obtained by the Siwen converter network processing Mapping the query matrix The weight coefficients are calculated using the Softmax layer, and the value matrix is ​​then... By applying constraints, a matrix can be obtained. : in, Key matrix Sum matrix dimensionality This is used in the Siwen converter network to mark the relative positional deviation between pixel blocks. Represents the key matrix Perform a transpose operation; Step (4): Employing a three-stream fusion module, multiple attention mechanisms and connection strategies are utilized to enhance the generalization ability of the steel grain size classification method based on guided self-attention mechanism; final feature map for: in, , , , This indicates that the feature map has been processed by the average pooling layer. This represents the dot product operation, i.e., the feature map. With feature map Multiply the pixels at corresponding positions; This represents the feature map obtained from the guided self-attention module, i.e., the matrix. ; Processed by the channel attention module and the global attention module. This indicates that a global attention module is used to extract global feature information. This indicates that channel feature information is extracted using the channel attention module; Size reduction processing is applied to the average pooling layer; Step (5): Based on the feature map Mapped from linear layers The classification descriptor vectors of different sizes are used, and the final classification result is determined by the cross-entropy loss function and the deep learning backpropagation mechanism.

Images