A Classification Method for Microcrystalline Structures in Ultra-High Carbon Steel Based on Spatial Attention and Ensemble Prediction

Abstract

This invention discloses a classification method for ultra-high carbon steel microcrystals based on spatial attention and ensemble prediction, relating to the field of metallic materials technology. The method includes: acquiring K improved EfficientNet-B7 models; these K improved EfficientNet-B7 models are obtained by training with K-fold cross-validation based on the initial improved EfficientNet-B7 model structure; the initial improved EfficientNet-B7 model adds a spatial attention module between the output of the sixth feature extraction stage and the input of the seventh feature extraction stage; classifying the microcrystal image of ultra-high carbon steel to be classified using each improved EfficientNet-B7 model to obtain the probability distribution of each category; arithmetically averaging the multiple probability distributions predicted by the multiple improved EfficientNet-B7 models to obtain a fused probability vector; and selecting the category corresponding to the maximum probability in the fused probability vector as the category of the microcrystal image to be classified. This method improves the classification accuracy of ultra-high carbon steel microcrystal structures.

Description

Technical Field

[0001] This application relates to the field of metallic materials technology, and in particular to a method for classifying microcrystalline structures of ultra-high carbon steel based on spatial attention and integrated prediction. Background Technology

[0002] Precise microstructure analysis is crucial for achieving microstructure control and customizable steel design, and analyzing the microstructure of steel is widely considered an effective strategy to improve its mechanical properties. However, accurate identification remains challenging for microstructures with highly similar morphological characteristics. For example, high-carbon martensite and lower bainite both exhibit acicular or lath morphologies, making effective differentiation difficult.

[0003] Existing automatic classification methods for ultra-high carbon steel microstructure images mostly employ pre-trained convolutional neural networks as end-to-end classifiers, obtaining image-level features through global pooling to complete category discrimination. However, traditional backbone networks tend to favor overall statistical representations during feature extraction, making them susceptible to interference from textures or noise in non-discriminatory regions. This results in insufficient separability between similar structures, leading to low classification accuracy for ultra-high carbon steel. Summary of the Invention

[0004] Therefore, it is necessary to provide a classification method for ultra-high carbon steel microcrystals based on spatial attention and ensemble prediction to address the aforementioned technical problems. This method improves the classification accuracy of ultra-high carbon steel microcrystal structures.

[0005] The following technical solution is adopted in this specification: This specification provides a method for classifying ultra-high carbon steel microcrystals based on spatial attention and ensemble prediction, including: Obtain images of the microscopic crystals of ultra-high carbon steel to be classified; Obtain K improved EfficientNet-B7 models; the K improved EfficientNet-B7 models are obtained by training with K-fold cross-validation based on the initial improved EfficientNet-B7 model structure; the initial improved EfficientNet-B7 model adds a spatial attention module between the output of the sixth feature extraction stage and the input of the seventh feature extraction stage of the EfficientNet-B7 model. By classifying the microcrystalline images of ultra-high carbon steel using each improved EfficientNet-B7 model, the probability distribution of each category predicted by each improved EfficientNet-B7 model is obtained. The probability distributions of each category predicted by multiple improved EfficientNet-B7 models are arithmetically averaged to obtain a fused probability vector. The category corresponding to the maximum probability in the fused probability vector is then selected as the category of the microcrystalline image to be classified.

[0006] Preferably, the improved EfficientNet-B7 model includes: a 3×3 convolutional layer Conv3×3, four 3×3 moving-inverted bottleneck convolutional blocks MBConv1 3×3, seven 3×3 moving-inverted bottleneck convolutional blocks MBConv6 3×3, seven 5×5 moving-inverted bottleneck convolutional blocks MBConv6 5×5, ten 3×3 moving-inverted bottleneck convolutional blocks MBConv6 3×3, ten 5×5 moving-inverted bottleneck convolutional blocks MBConv6 5×5, a spatial attention module, thirteen 5×5 moving-inverted bottleneck convolutional blocks MBConv6 5×5, four 3×3 moving-inverted bottleneck convolutional blocks MBConv6 3×3, and a classification head; the classification head includes a 1×1 convolutional layer Conv6. 1×1 pooling layer, fully connected layer (FC), and classification layer (Classification).

[0007] Preferably, the microscopic crystal images of ultra-high carbon steel are classified using an improved EfficientNet-B7 model to obtain the probability distributions of each category predicted by the improved EfficientNet-B7 model, including: The microscopic crystal image to be classified is sequentially passed through a 3×3 convolutional layer Conv3×3, a 4-kernel 3×3 moving-inverted bottleneck convolutional block MBConv1 3×3, a 7-kernel 3×3 moving-inverted bottleneck convolutional block MBConv6 3×3, a 7-kernel 5×5 moving-inverted bottleneck convolutional block MBConv6 5×5, a 10-kernel 3×3 moving-inverted bottleneck convolutional block MBConv6 3×3, and a 10-kernel 5×5 moving-inverted bottleneck convolutional block MBConv65×5, thus mapping the microscopic crystal image to be classified into a feature representation. The feature representation is input into the spatial attention module to generate a spatial attention weight map; The spatial attention weight map is multiplied element-wise with the feature representation to highlight the region response related to key discriminative textures, resulting in enhanced features; key discriminative textures include grain boundaries, phase boundaries, lath layers, and lamellae. The enhanced features are sequentially passed through 13 moving inverted bottleneck convolutional blocks MBConv6 5×5 with 5×5 kernels, 4 moving inverted bottleneck convolutional blocks MBConv6 3×3 with 3×3 kernels, and a classification head to obtain the probability distribution of each category.

[0008] Preferably, the step of inputting the feature representation into the spatial attention module to generate a spatial attention weight map specifically includes: The feature representations are subjected to max pooling and average pooling along the channel axis respectively to obtain max pooling spatial feature description maps and average pooling spatial feature description maps. The max pooling spatial feature description map and the average pooling spatial feature description map are concatenated along the channel dimension to obtain a comprehensive representation; The comprehensive representation is passed through a convolutional layer and The function generates a spatial attention weight map.

[0009] Preferably, the loss function of the improved EfficientNet-B7 model is: in, To improve the loss value of the EfficientNet-B7 model, For the current real category The number of samples, For category The number of samples, The total number of categories, As a regulating factor, To improve the EfficientNet-B7 model's prediction probability of the correct class.

[0010] This specification provides a classification device for ultra-high carbon steel microcrystals based on spatial attention and ensemble prediction, including: The first acquisition module is used to acquire images of the microscopic crystals of ultra-high carbon steel to be classified. The second acquisition module is used to acquire K improved EfficientNet-B7 models. The K improved EfficientNet-B7 models are obtained by training with K-fold cross-validation based on the initial improved EfficientNet-B7 model structure. The initial improved EfficientNet-B7 model adds a spatial attention module between the output of the sixth feature extraction stage and the input of the seventh feature extraction stage of the EfficientNet-B7 model. The generation module is used to classify the microcrystalline images of ultra-high carbon steel to be classified using each improved EfficientNet-B7 model, and obtain the probability distribution of each category predicted by each improved EfficientNet-B7 model. The prediction module is used to perform an arithmetic average of the multiple probability distributions of each category predicted by multiple improved EfficientNet-B7 models to obtain a fused probability vector, and select the category corresponding to the maximum probability in the fused probability vector as the category of the microscopic crystal image to be classified.

[0011] This specification provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method for classifying ultra-high carbon steel microcrystals based on spatial attention and ensemble prediction.

[0012] This specification provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the above-described method for classifying ultra-high carbon steel microcrystals based on spatial attention and integrated prediction.

[0013] The above-mentioned technical solutions adopted in this specification can achieve the following beneficial effects: In the ultra-high carbon steel microcrystal classification method based on spatial attention and ensemble prediction provided in this specification, the present invention embeds a spatial attention module in the EfficientNet-B7 backbone network. It aggregates spatial information through max pooling or average pooling in the channel dimension to generate a spatial weight map, enabling the network to adaptively enhance key texture regions such as grain boundaries and cementite morphology while suppressing background noise, thereby solving the problem of distinguishing highly similar categories. The probability distributions of each category predicted by multiple improved EfficientNet-B7 models are arithmetically averaged, and the category corresponding to the maximum probability is selected as the category of the microcrystal image to be classified. The probability distributions output by each sub-model are fused by arithmetic averaging, which integrates the complementary capabilities of multiple models and effectively reduces errors caused by single-model fluctuations and the randomness of data partitioning.

[0014] This invention enhances key texture features by improving the EfficientNet-B7 model and fuses the prediction probabilities of multiple models by arithmetic averaging. It combines spatial attention mechanism with ensemble prediction to improve the classification accuracy of ultra-high carbon steel microcrystal images. Attached Figure Description

[0015] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0016] Figure 1 A schematic diagram of a classification method for ultra-high carbon steel microcrystals based on spatial attention and ensemble prediction provided in this specification; Figure 2 A schematic diagram of the traditional EfficientNet-B7 model provided by this invention; Figure 3 A schematic diagram of the EfficientNet-B7 model with an embedded spatial attention module provided by the present invention; Figure 4 The overall flowchart provided for this invention; Figure 5 A schematic diagram illustrating the enhanced results provided by the present invention; Figure 6 Figures showing the ablation experiment results of different methods provided by this invention on a test set; Figure 7 The diagram showing the impact of different spatial attention feature enhancement locations on model performance provided by this invention; Figure 8 Visualization results of the feature responses of different models to crystal structure images provided by this invention; Figure 9 A schematic diagram of an ultra-high carbon steel microcrystal classification device based on spatial attention and integrated prediction, provided in this specification; Figure 10 This specification provides a schematic diagram of a computer device for implementing a classification method for ultra-high carbon steel microcrystals based on spatial attention and integrated prediction. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this specification clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments in this specification without creative effort are within the scope of protection of this application.

[0018] Devices such as desktop computers, servers, and laptops are capable of executing the solutions described in this manual. For ease of explanation, the following description will focus on servers as the primary execution method.

[0019] In recent years, the mainstream technical approach for materials image analysis has been gradually shifting from traditional machine learning to deep learning. Convolutional Neural Networks (CNNs) can directly perform end-to-end feature learning from microscopic images, significantly improving the automation level of microscopic analysis. Within the traditional machine learning framework, researchers have addressed the problems of large errors and poor repeatability in the traditional subjective classification of low-carbon steel microstructures by constructing a classification model based on Support Vector Machines (SVMs) and associating important parameters with the microstructure. Similarly, researchers have extracted texture and morphological features from Scanning Electron Microscope (SEM) images and employed SVM machine learning techniques to achieve automated, objective, and repeatable classification of the microstructures of seven carbon-rich second-phase phases in low-carbon multiphase steel, including pearlite, martensite, and five bainite subclasses.

[0020] Existing automatic classification methods for ultra-high carbon steel microstructure images mostly employ pre-trained convolutional neural networks as end-to-end classifiers, obtaining image-level features through global pooling to complete category discrimination. However, these methods often lack targeted optimization for the microstructural characteristics of steel, and under real-world data conditions, they are prone to problems such as insufficient feature representation and limited generalization ability. First, microstructure discrimination is highly dependent on local details and spatial relationships, such as texture variations near grain boundaries / phase boundaries, and the directionality and continuity of lamellar or slat morphology. Traditional backbone networks tend to focus on overall statistical representation during feature extraction, lacking explicit modeling of which regions in the image are more critical. This makes them susceptible to interference from textures or noise in non-discriminatory regions, leading to insufficient separability between similar microstructures. Second, when the data scale is limited and the category distribution is unbalanced, the model tends to output high-frequency categories, resulting in low recall and stability of minority class samples. This is manifested in a decrease in macro-average indicators (Macro-F1, Macro-Recall), and minority classes in the confusion matrix are more likely to be merged into similar microstructure categories. Finally, many solutions are evaluated based on only a single fixed training / test split or the results of a single model. The model is sensitive to the randomness of data splitting, has a large generalization variance, and is difficult to achieve reproducible and stable performance.

[0021] To address the aforementioned shortcomings, this invention aims to construct a deep learning method for classifying microscopic images of ultra-high carbon steel. This method enhances the ability to focus on key tissue regions, improves minority class recognition performance, and increases the stability and reproducibility of test results under conditions of small sample size and class imbalance. This invention uses EfficientNet-B7 as the feature extraction backbone and introduces a spatial attention module during the feature extraction stage. By spatially weighting the feature map at specified layers, it enhances the response to discriminative texture and morphological regions and suppresses interference from irrelevant regions. To solve the class imbalance problem, this invention simultaneously employs class-weighted Focal Loss and a weighted random sampling strategy to reduce the majority class dominance effect from both the loss and sampling ends, thereby improving minority class recall and macro-average metrics. To reduce performance fluctuations caused by randomness in partitioning, this invention employs a stratified sampling partitioning method with 80% used for training and validation and 20% for independent testing. Stratified K-fold cross-validation is performed in the training and validation part, and the optimal weights are saved by early stopping based on the validation accuracy within each fold. Finally, the softmax probabilities output by the best model in each fold on the test set are averaged and fused to achieve soft voting ensemble prediction, thereby reducing single-model bias and improving overall stability.

[0022] Through the above design, the present invention aims to realize a classification method that is reproducible, engineering-deployable, and has higher key region expression ability and better minority class recognition performance on imbalanced small sample microscopic tissue data.

[0023] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0024] Figure 1 This is a flowchart illustrating a method for classifying ultra-high carbon steel microcrystals based on spatial attention and ensemble prediction, as described in this specification. The method includes the following steps: S101: Obtain images of the microcrystalline structures of ultra-high carbon steel to be classified.

[0025] This invention constructs a deep learning framework for classifying the crystal structure of microscopic images of ultra-high carbon steel. It uses EfficientNet-B7 as the backbone network and integrates spatial attention analysis to realize the main classification model. At the same time, it introduces data augmentation and imbalanced learning methods to enhance robustness. Finally, it improves the prediction accuracy by using multi-fold model soft voting ensemble prediction.

[0026] In the application phase of improving the EfficientNet-B7 model, the first step is to obtain images of the microcrystalline structures of ultra-high carbon steel to be classified.

[0027] S102: Obtain K improved EfficientNet-B7 models; the K improved EfficientNet-B7 models are obtained by training with K-fold cross-validation based on the initial improved EfficientNet-B7 model structure; the initial improved EfficientNet-B7 model adds a spatial attention module between the output of the sixth feature extraction stage and the input of the seventh feature extraction stage of the EfficientNet-B7 model.

[0028] In an exemplary embodiment, the improved EfficientNet-B7 model includes: a 3×3 convolutional layer Conv3×3, four 3×3 moving-inverted bottleneck convolutional blocks MBConv1 3×3, seven 3×3 moving-inverted bottleneck convolutional blocks MBConv6 3×3, seven 5×5 moving-inverted bottleneck convolutional blocks MBConv6 5×5, ten 3×3 moving-inverted bottleneck convolutional blocks MBConv6 3×3, ten 5×5 moving-inverted bottleneck convolutional blocks MBConv6 5×5, a spatial attention module, thirteen 5×5 moving-inverted bottleneck convolutional blocks MBConv6 5×5, four 3×3 moving-inverted bottleneck convolutional blocks MBConv6 3×3, and a classification head; the classification head includes a 1×1 convolutional layer Conv6. 1×1 pooling layer, fully connected layer (FC), and classification layer (Classification).

[0029] Specifically, this invention improves the structure of the EfficientNet-B7 baseline network by retaining the MBConv stacking and phased feature extraction process of the original network backbone, and adding a spatial attention enhancement branch after the output of the sixth feature extraction stage to enhance the response to salient spatial regions. Figure 2 A schematic diagram of the traditional EfficientNet-B7 model provided by this invention, as shown below. Figure 2As shown, the traditional EfficientNet-B7 model includes, in sequence, a 3×3 convolutional layer Conv3×3, four 3×3 moving-inverted bottleneck convolutional blocks MBConv1 3×3, seven 3×3 moving-inverted bottleneck convolutional blocks MBConv6 3×3, seven 5×5 moving-inverted bottleneck convolutional blocks MBConv6 5×5, ten 3×3 moving-inverted bottleneck convolutional blocks MBConv6 3×3, ten 5×5 moving-inverted bottleneck convolutional blocks MBConv6 5×5, thirteen 5×5 moving-inverted bottleneck convolutional blocks MBConv6 5×5, four 3×3 moving-inverted bottleneck convolutional blocks MBConv6 3×3, and a classification head; the classification head includes a 1×1 convolutional layer Conv6. 1×1 pooling layer, fully connected layer (FC), and classification layer (Classification).

[0030] Figure 3 A schematic diagram of the EfficientNet-B7 model with embedded spatial attention module provided by this invention is shown below. Figure 3 As shown, a spatial attention module is embedded between the output of the sixth feature extraction stage (a 3×3 moving inverted bottleneck convolutional block MBConv63×3 with 10 convolutional kernels) and the input of the seventh feature extraction stage (a 5×5 moving inverted bottleneck convolutional block MBConv65×5 with 13 convolutional kernels).

[0031] S103: Classify the microcrystalline images of ultra-high carbon steel by using each improved EfficientNet-B7 model, and obtain the probability distribution of each category predicted by each improved EfficientNet-B7 model.

[0032] In an exemplary embodiment, the improved EfficientNet-B7 model is used to classify the microcrystalline images of ultra-high carbon steel, obtaining the probability distributions of each category predicted by the improved EfficientNet-B7 model. This includes: sequentially passing the microcrystalline image to be classified through a 3×3 convolutional layer (Conv3×3), four 3×3 moving-inverted bottleneck convolutional blocks (MBConv1 3×3), seven 3×3 moving-inverted bottleneck convolutional blocks (MBConv6 3×3), seven 5×5 moving-inverted bottleneck convolutional blocks (MBConv6 5×5), ten 3×3 moving-inverted bottleneck convolutional blocks (MBConv6 3×3), and ten 5×5 moving-inverted bottleneck convolutional blocks (MBConv6 5×5). A 5×5 mapping method is used to map the microscopic crystal image to be classified into a feature representation. The feature representation is input into a spatial attention module to generate a spatial attention weight map. The spatial attention weight map is multiplied element-wise with the feature representation to highlight the region response related to key discriminative textures, resulting in enhanced features. Key discriminative textures include grain boundaries, phase boundaries, lath layers, and lamellae. The enhanced features are then sequentially passed through 13 moving inversion bottleneck convolutional blocks MBConv6 5×5 with 5×5 kernels, 4 moving inversion bottleneck convolutional blocks MBConv6 3×3 with 3×3 kernels, and a classification head to obtain the probability distribution of each category.

[0033] Specifically, for feature representation First, perform max pooling along the channel axis. and average pooling Aggregate channel information to obtain the max pooling spatial feature map. and average pooling spatial feature description diagram ,Will and The composite representation is obtained by concatenating along the channel dimension, and then passed through a convolutional layer and... Function generates spatial attention weight graph Finally, the spatial attention weight map is combined with the feature representation. Element-wise multiplication is performed to achieve spatial adaptive enhancement of features. This process can be formally expressed as formulas (1) and (2):

[0034] (1); in, This is a spatial attention weight map. For average pooling, For max pooling, This represents the convolution operation. This is the sigmoid function.

[0035] The formula for calculating the enhanced features is formula (2): (2); in, To enhance features, For feature representation, This indicates element-wise multiplication.

[0036] EfficientNet-B7, with its embedded spatial attention, effectively represents the structure in microscopic images of ultra-high carbon steel, demonstrating strong capture capabilities from low-level texture features to high-level morphology. Compared to large-scale models based on self-attention mechanisms, such as the Vision Transformer (ViT), it achieves a better balance between computational efficiency and model capacity, making it suitable for applications in industrial microscopic image tasks with limited data.

[0037] Experiments show that embedding the spatial attention module into the sixth layer enables the EfficientNet-B7 model to focus on more discriminative regions, such as tissue boundaries, grain morphology, and precipitate distribution, at the high-level semantic feature stage. This structural design allows the model to extract high-level semantic information while exhibiting region selectivity, thereby improving its ability to perceive complex microscopic organizational patterns. Furthermore, introducing the attention module only in a single layer avoids redundant computation and parameter growth caused by multi-layer embedding, maintaining the model's lightweight nature and inference efficiency.

[0038] In the application phase, the first The logits of the predicted probability vectors for each class output by the improved EfficientNet-B7 model are given by formula (3): (3); in, For the first k The predicted probability vectors for each class output by the improved EfficientNet-B7 model. for C 3D real vector space, C This represents the total number of categories.

[0039] The predicted probability vectors of each category are processed The function yields the corresponding category probability distribution as shown in formula (4): (4); in, For the first k The probability distribution of each class in the improved EfficientNet-B7 model. .

[0040] S104: The probability distributions of each category predicted by multiple improved EfficientNet-B7 models are arithmetically averaged to obtain a fused probability vector, and the category corresponding to the maximum probability value in the fused probability vector is selected as the category of the microcrystalline image to be classified.

[0041] The class probability distributions from the K embedded spatial attention modules of the EfficientNet-B7 sub-models are arithmetically averaged to obtain the fused probability vector as shown in formula (5): (5); in, This is the fused probability vector. K The number of EfficientNet-B7 sub-models that embed the spatial attention module.

[0042] The final predicted label is obtained by selecting the category corresponding to the maximum value in the fusion probability, as shown in formula (6): (6); in, To classify the category of the microscopic crystal images to be classified, The fused probability vector The c Each component represents a sample. i Belongs to the c The fusion probability of classes.

[0043] In one exemplary embodiment, during the training phase of the EfficientNet-B7 model with embedded spatial attention, the input is a labeled dataset of microscopic crystal images. ,in, Indicates the first Zhang's microscopic crystal images, Indicates the corresponding organization category, C To determine the number of categories; firstly, the microscopic crystal images are preprocessed to meet the input specifications of the backbone network, including size unification, channel format conversion, and pixel intensity normalization, and this is further processed during the training phase. Apply random data augmentation (such as flipping, rotating, random erasing, etc.) to expand the effective sample, reduce overfitting, and improve robustness to imaging fluctuations.

[0044] Then, to address the issue of imbalanced class distribution in the dataset, an imbalanced learning strategy is adopted: on the one hand, minority class samples are resampled at the sampling end to make the samples of each class more balanced in the training batch; on the other hand, class adaptive focus loss (Focal Loss, FL) is used at the optimization end to reduce the dominant role of easily classified samples in training, making the model pay more attention to difficult samples and minority class samples, thereby improving minority class recall and overall stability. In terms of model structure, EfficientNet-B7 is used as the backbone feature extraction network to map the input image into feature representation. Furthermore, a spatial attention module is fused into the specified feature layer of the backbone to generate a spatial weight map. ,use right Region-level weighting is performed to highlight the regional responses related to key discriminative textures such as grain boundaries / phase boundaries and laths / lamellae, while suppressing irrelevant background and noise interference, resulting in enhanced features. These enhanced features are then passed through a classification head to output predicted probability vectors for each category and the final predicted category. .

[0045] Finally, to further improve the prediction accuracy and consistency under small sample conditions, the following approach was adopted: Cross-validation training yielded An improved EfficientNet-B7 model During the inference phase, each model generates probability outputs for the same test sample. The integration probability is obtained through probabilistic soft voting. ,by The category corresponding to the largest component is used as the final output, along with the confidence score, to form a complete classification method flow from "preprocessing and enhancement - imbalanced learning - spatial attention-enhanced feature extraction - multi-fold soft voting ensemble prediction".

[0046] This invention targets the classification of microcrystalline structures in ultra-high carbon steel and mainly consists of four parts: data preprocessing, handling data imbalance, classification model training, and soft-voting integrated prediction and evaluation.

[0047] The training data is fed into the improved EfficientNet-B7 model to extract microstructural features. Five-fold cross-validation is used to obtain five weights, which are then used for joint soft-voting prediction to obtain the final classification result. The overall flowchart is as follows: Figure 4 As shown:

[0048] (1) Data preprocessing: In order to enhance the robustness of the classification method, the original microscopic crystal images were randomly flipped, rotated and erased to generate more samples with different shapes, so as to enable the model to fully learn the effective features of various crystals.

[0049] To improve the model's generalization ability and training stability, this invention designs different processing procedures for the training set and the validation / test set during the data preprocessing stage. Multiple random augmentation strategies are introduced during the training stage to expand the data distribution, while only scaling and standardization operations are performed during the validation and testing stages to ensure the fairness and repeatability of the evaluation process.

[0050] To improve the model's robustness to geometric transformations and local occlusion, the following enhancement operations are randomly applied to each image during the training phase: Figure 5 As shown, Figure 5 Figure (a) in the image is the original image. Figure 5 Image (b) in the image is a randomly flipped image. Figure 5 Image (c) is a randomly rotated image. Figure 5 In Figure (d), random erasure is shown, where the black rectangle represents the erasure area.

[0051] Random horizontal flip: The image is flipped horizontally with a 50% probability to simulate the orientation change and symmetry characteristics of the sample, thereby improving the model's adaptability to different arrangement orientations.

[0052] Random rotation: For each image, a random angle is selected, and the image is rotated within a range of ±15°. Specifically, a random angle is generated (ranging from -15° to +15°), and the image is rotated accordingly. This operation helps the model adapt to different orientations of objects within the image.

[0053] Random erasure: This simulates the loss of some image information by randomly selecting a rectangular region in the image and setting its pixel values ​​to random values ​​or 0. During training, random erasure is applied to each image with a 50% probability. In practice, the size and location of the erasure region are randomized, which helps the model learn more robust features.

[0054] During the validation and testing phases, no random augmentation operations were used; only resampling and normalization steps were retained. This ensures the consistency of the evaluation images, making the experimental results more reproducible and objective.

[0055] All samples were first resampled to 600×600 pixels and converted into three-channel images to fit the input requirements of the pre-trained backbone network. Subsequently, the images were standardized using a mean of 0.5 and a standard deviation of 0.5 to normalize the pixel values ​​of each channel as shown in Formula (7):

[0056] (7); in, These are the normalized pixel values. These are the pixel values ​​before normalization.

[0057] This operation can scale pixel values ​​to the [−1,1] range, accelerating model convergence and reducing gradient oscillations.

[0058] This training-enhancement-test consistency strategy can effectively expand the distribution range of training data without compromising evaluation fairness, thereby improving the model's ability to cope with changes in orientation, morphological differences, and local missing features in real-world scenarios. For images of steel microstructures, this type of geometric and occlusion enhancement is particularly helpful for the model to focus on the overall distribution of texture and structure, rather than relying on local features at certain fixed locations, thus significantly improving the model's generalization performance in classifying complex microstructures.

[0059] (2) Imbalanced processing method: The microstructure data of ultra-high carbon steel exhibits a significant imbalance in class distribution, with some microstructure types having very few samples, causing the model to tend to favor the majority class during training. To alleviate this problem, this invention employs a class-adaptive loss function, Focal Loss, combined with a sampling strategy to improve the model's discriminative ability in imbalanced scenarios.

[0060] To address the class imbalance problem in the data, FocalLoss with adaptive class weights is used as the loss function during the training phase to alleviate the class imbalance in the dataset. Its standard form is Equation (8):

[0061] (8); in, The loss value for the EfficientNet-B7 model with embedded spatial attention module. For fixed weights, The probability of the EfficientNet-B7 model with embedded spatial attention module predicting the correct class. It is a regulating factor.

[0062] To address the severe class imbalance problem in the classification of steel microstructures, the fixed weights in the standard Focal Loss will be adjusted. Change to category adaptive weights For those containing A dataset of categories, categories Adaptive weights The calculation is performed using formula (9):

[0063] (9); in, It is the total number of samples in the training set. It is a category The number of samples, It is the total number of categories. For adaptive weights.

[0064] To maintain the scaling stability of the loss function, the weights are further normalized as shown in formula (10): (10); in, For category Normalized weights, For category j The weights are determined by normalization. Normalization ensures that the average weights of all classes are 1.0, thus maintaining the stability of the overall scale of the loss function while preserving the weight enhancement of minority classes.

[0065] The improved adaptive weight loss function of the EfficientNet-B7 model is given by formula (11): (11); in, To improve the loss value of the EfficientNet-B7 model, For the current real category The number of samples, For category The number of samples, The total number of categories, As a regulating factor, To improve the EfficientNet-B7 model's prediction probability of the correct class.

[0066] Furthermore, during model training, a data sampling strategy was employed to further balance the training process. WeightedRandomSampler, a sampler provided by PyTorch, performs weighted random sampling based on the weights assigned to each sample. The probability of a sample being drawn is proportional to its weight, allowing categories that originally had a smaller proportion in the dataset to be sampled more frequently.

[0067] This combined strategy creates a complementary effect: the sampling strategy addresses the issue of sample contact frequency, while the loss function addresses the issue of sample learning difficulty. Their synergistic effect enables the model to learn the characteristics of various steel microstructures more comprehensively and effectively.

[0068] (3) Spatial Attention Classification Model: A classification model is built using EfficientNet-B7 as the backbone network. The separable convolution and SE attention of the MBConv module are used to effectively capture the subtle differences in morphology and texture of different crystals. At the same time, a Spatial Attention Estimation (SAE) module is embedded in the backbone network to enhance the spatial saliency expression in high-level features, focusing on the boundary division of crystals and crystal phases, and effectively improving the crystal recognition ability.

[0069] This invention improves the structure of the EfficientNet-B7 baseline network by retaining the original network backbone's MBConv stacking and staged feature extraction process, and adding a spatial attention enhancement branch after the output of the sixth feature extraction stage to strengthen the response to salient spatial regions. (4) Multi-fold model ensemble prediction: In order to improve the stability of model prediction and overall discrimination ability, a multi-fold model ensemble prediction strategy based on soft voting is introduced. The prediction results of the trained multi-fold sub-models are averaged by probability, which effectively reduces the bias and variance of the single model without changing the network structure and inference process.

[0070] In practical applications of deep neural networks, the prediction results of a single model are often affected by factors such as parameter initialization, training data partitioning, and stochastic gradient optimization, leading to performance fluctuations and unstable generalization. To improve the stability and overall discriminative ability of the model during the testing phase, this invention introduces a prediction fusion strategy based on soft voting. This method effectively reduces the bias and variance of a single model by averaging the prediction results of multiple independent sub-models without changing the network structure and inference process, thus achieving more stable classification performance.

[0071] During the training phase, five-fold cross-validation (K=5) was used on the training data, with each fold training an independent classification model with spatial attention. Each sub-model employed an early stopping strategy based on the validation set performance and saved the weight parameters with the optimal validation accuracy, thereby ensuring complementarity between models in feature representation and decision boundaries.

[0072] The soft voting method preserves the confidence information of each sub-model during the fusion process, thus achieving more refined weighting at the probabilistic level. This strategy can mitigate the impact of outlier predictions by averaging multiple models, even when local overfitting or bias exists in the model, making the overall prediction more robust.

[0073] To comprehensively evaluate the model's performance in the classification of ultra-high carbon steel microstructures, this invention employs multiple metrics from different dimensions for quantification, including confusion matrix, accuracy, precision, recall, F1 score, and geometric mean (G-mean). These metrics can reflect the model's performance in terms of overall accuracy, class discriminancy, and class balance, and are particularly suitable for multi-class classification tasks with imbalanced class distributions.

[0074] A confusion matrix is ​​a square matrix used to represent the performance of a classification model on a test dataset. Each element in the matrix represents the correspondence between the model's predicted class and the actual class. For binary classification problems, the confusion matrix contains four core elements: TP: True Positive Examples (TP), the number of samples predicted as positive and actually positive; TN: True Negative Examples (TN), the number of samples predicted as negative and actually negative; FP: False Positive Examples (FP), the number of samples predicted as positive but actually negative; and FN: False Negative Examples (FN), the number of samples predicted as negative but actually positive.

[0075] The matrix form can be expressed as formula (12): (12); in, TP For true examples, the number of samples that were predicted to be positive and actually were positive. TN The number of samples that are predicted to be negative and are actually negative, representing true counterexamples. FP The number of false positives: samples predicted as positive but actually negative. FN This is a false negative example, representing the number of samples predicted as negative but actually being positive.

[0076] Accuracy: Accuracy is the most common classification evaluation metric, representing the proportion of samples correctly predicted by the model out of the total number of samples. The formula for calculating accuracy is formula (13): (13); in, For accuracy.

[0077] Precision: Precision refers to the proportion of samples that are actually positive out of all samples predicted as positive by the model. Precision measures the accuracy of the model's prediction of positive classes. In cases of class imbalance, precision is often used to measure the impact of false positives on model performance. The formula for calculating precision is formula (14):

[0078] (14); in, For accuracy.

[0079] Recall: Recall rate refers to the proportion of samples that are actually positive that the model correctly predicts as positive. Recall rate measures the model's ability to capture positive classes. The formula for calculating recall rate is formula (15):

[0080] (15); in, This refers to the recall rate.

[0081] F1 score: The F1 score is the harmonic mean of precision and recall, which combines the trade-offs between the two. Because it considers the prediction and recall of positive samples, it is particularly effective for imbalanced tasks. The formula for the F1 score is Equation (16):

[0082] (16); in, The score is the F1 score.

[0083] G-means: To measure the model's ability to equally identify classes under imbalanced conditions, this invention employs G-means. The idea is to simultaneously examine sensitivity (recall, TPR) and specificity (True Negative Rate, TNR) for each class. A low score in either category significantly lowers the score for that class, thus suppressing the distortion of sacrificing minority classes for overall accuracy. For multi-class tasks, a one-to-many strategy is used to calculate the metrics for each class. c Class sensitivity The calculation formula is formula (17):

[0084] (17); in, For the first c The sensitivity of the class, representing the true value as the first. c The sample of class was correctly identified as c Class ratio, For the first c The number of true positives in a class, i.e., the number of true labels that are the first positives. c The prediction is also the first. c The number of samples in each class For the first c The number of false negatives in a class, i.e., the number of true labels that are the first negatives. c Class but predicted as non-first c The number of samples in a class.

[0085] The calculation formula is formula (18): (18); in, For the first c The specificity of a class, representing the fact that the truth is not the first class. c Samples of class were correctly identified as non-first class. c Class ratio, For the first c The number of true negatives in a class, i.e., the number of true negatives whose actual labels are not the first. c The prediction is also non-first. c The number of samples in each class For the first cThe number of false positives in a class, i.e., the number of true labels that are not the first class. c Class but predicted as the first c The number of samples in a class.

[0086] The geometric mean of each category is given by formula (19): (19); in, The geometric mean for each category.

[0087] The overall index is obtained by taking the macro average as formula (20): (20); in, For macro average, C This represents the total number of categories (the number of categories for the classification task).

[0088] These evaluation criteria provide a comprehensive perspective for assessing model performance, especially when dealing with imbalanced or multi-class classification tasks, enabling a better understanding of the model's strengths and weaknesses.

[0089] In one exemplary embodiment, experiments have shown that the present invention achieves ideal results and is consistent with the design expectations for the classification model of ultra-high carbon steel crystals. To evaluate the overall performance and effectiveness of the proposed ultra-high carbon steel microcrystalline classification method, experiments were conducted to compare it with six existing methods, including LBP, Textons, ViT_B16, MobileNetV2, GoogLeNet, and ViT_L32.

[0090] Table 1 shows in detail the comparison of the overall classification accuracy of the method proposed in this invention with that of various mainstream feature extraction and classification models on the same dataset.

[0091] Table 1 Comparison of Overall Classification Accuracy Table 1 shows the overall classification accuracy comparison between the proposed method and various mainstream feature extraction and classification models on the same dataset. The comparison models cover traditional texture feature extraction methods (such as LBP and Textons) and current advanced deep learning models (such as the ViT series, MobileNetV2, and GoogLeNet). The EfficientNet-B7 model with embedded spatial attention module constructed in this invention achieves an overall classification accuracy of 93.26%, surpassing not only traditional methods but also improving by nearly 2 percentage points compared to the second-best ViT_L32 model. This result fully demonstrates the effectiveness of the proposed method in feature extraction and the superiority of its classification performance, exhibiting excellent generalization ability.

[0092] Table 2 further illustrates the detailed performance metrics of the method of the present invention across various specific categories (A to G), including accuracy, recall, and F1 score. The data in the table demonstrates that the method exhibits excellent classification ability across most categories.

[0093] Table 2. Classification accuracy analysis of the method of the present invention for each category As shown in Table 2, the overall results indicate that the model achieved relatively ideal performance in classes A, E, and F. The accuracy, recall, and F1 score for these three classes all reached or approached 1, demonstrating that the EfficientNet-B7 model with the embedded spatial attention module can effectively capture the feature information of these classes. In class C, although the accuracy was slightly lower than other high-performance classes, the recall reached 0.9545, and the F1 score was 0.8936, indicating that the model's recognition of class C samples remained relatively stable. In contrast, the metrics for classes B, D, and G were lower. However, overall, the model's performance metrics after weighted averaging remained excellent. This shows that the method of this invention can not only handle feature recognition of large sample classes but also, to a certain extent, address the challenges posed by sample imbalance between classes, demonstrating good practical value.

[0094] The comparison models cover traditional texture feature extraction methods (such as LBP and Textons) as well as current state-of-the-art deep learning models (such as the ViT series, MobileNetV2, and GoogLeNet). The EfficientNet-B7 model with embedded spatial attention module constructed in this invention achieves an overall classification accuracy of 93.26%, which not only surpasses traditional methods but also improves by nearly 2 percentage points compared to the second-best ViT_L32 model. This result fully demonstrates the effectiveness of the method provided in this invention in feature extraction and the superiority of its classification performance, exhibiting excellent generalization ability.

[0095] This invention proposes a spatial attention-enhanced EfficientNet-B7 framework for classifying the microcrystalline structure of ultra-high carbon steel. Addressing the challenge of subtle microstructure features and class imbalance in images, innovative strategies including data augmentation, fine-grained feature learning, imbalance handling, and multi-mode ensemble prediction are designed. The effectiveness of this method is validated through extensive experiments and analysis. First, the introduction of a spatial attention mechanism enhances the model's ability to identify key feature regions in the microstructure of ultra-high carbon steel. Second, a dual class imbalance handling strategy (weighted sampling and adaptive Focal Loss) addresses the uneven distribution of class distributions among microstructures in the dataset. Finally, a soft-voting ensemble learning method based on cross-validation significantly improves the model's generalization ability and robustness. By integrating the prediction probability distributions of multiple models, this method effectively reduces the prediction bias of a single model.

[0096] This invention relates to the classification of microcrystalline structures in ultra-high carbon steel (UHCS), and is an image classification method based on EfficientNet-B7 feature extraction combined with spatial attention enhancement and ensemble prediction. It primarily addresses the challenges in UHCS microcrystalline structure classification research, such as difficulty in capturing subtle local textures, small sample size, and imbalanced data distribution. This invention utilizes a publicly available dataset of UHCS microcrystalline structures (UHCSDB dataset) as its research foundation, aiming to achieve high-precision feature extraction and classification of seven types of UHCS microcrystalline structures (including pearlite, spheroidal carbides, martensite, and their mixed structures) through an improved deep learning algorithm.

[0097] This invention proposes a classification technique that enhances learned crystal structure features through a spatial attention module and combines it with ensemble predictive soft voting. The invention mainly comprises two core components: a feature enhancement method based on a spatial attention module and a decision fusion method based on ensemble predictive soft voting. First, in the feature extraction stage, a deep convolutional neural network with EfficientNet-B7 as its backbone is constructed. Considering that microscopic crystal images of ultra-high carbon steel (such as pearlite, Widmanstätten structure, etc.) often exhibit high similarity in local texture details, a spatial attention module is embedded at the output of the sixth feature extraction stage of the EfficientNet-B7 backbone network. The module's working mechanism is as follows: by performing channel-dimensional average pooling and max pooling on the input feature map, the global and saliency information of the feature map in the spatial dimension is captured; then, a spatial attention weight map is generated using a convolution operation with a large receptive field; finally, this weight map is element-wise multiplied with the original feature map. This process enables the recalibration of the feature map, allowing the network to adaptively focus on highly discriminative grain boundaries and precipitated phase regions while suppressing unwanted background noise, thereby extracting more robust microstructural features.

[0098] Secondly, in the ensemble prediction stage, to address the issue of poor generalization ability of a single model due to the small sample size and imbalanced distribution of the ultra-high carbon steel dataset, an ensemble strategy based on five-fold cross-validation was adopted. During training, the dataset was divided into five parts, and five optimal sub-models with different weight parameters were trained through a round-robin validation process. During prediction, the microscopic crystal images of the ultra-high carbon steel to be classified were input into the five optimal sub-models respectively, resulting in five independent prediction probability distribution vectors.

[0099] Subsequently, a soft voting mechanism was used to fuse these five probability distribution vectors: the arithmetic mean of the predicted probabilities of each model for the same category was calculated. This fusion method can effectively reduce the prediction variance of a single model and correct misjudgments by individual models.

[0100] Finally, the category with the highest average probability after fusion was selected as the final classification result for the ultra-high carbon steel microcrystal. To verify that the algorithm proposed in this invention can learn relevant crystal features, experiments were conducted using multiple deep neural networks on the classification problem of ultra-high carbon steel microcrystal structures, and the effectiveness and superiority of the method were verified on multiple evaluation metrics.

[0101] To verify the contribution of the spatial attention-enhanced feature extraction method and the soft-voting ensemble prediction to the technology of this invention, the following experiments were conducted under the same experimental environment: Experiment 1: Feature extraction using only EfficientNet-B7.

[0102] Experiment 2: Feature extraction method based on spatial attention enhancement.

[0103] Experiment 3: The algorithm proposed in this invention is an integrated prediction method based on spatial attention-enhanced feature extraction and soft voting.

[0104] Experiment 4: Analysis of the effectiveness of the spatial attention module.

[0105] Figure 6 The following are graphs showing the ablation experimental results of different methods provided by this invention on a test set, such as... Figure 6 As shown, introducing a spatial attention mechanism improved the model accuracy by 1.77 percentage points, and further adopting a model prediction fusion strategy improved the accuracy by another 3.93 percentage points, reaching 93.26%. This indicates that both proposed improvement strategies effectively enhanced model performance.

[0106] Figure 7 The diagram showing the impact of different spatial attention feature enhancement locations on model performance provided by this invention is as follows: Figure 7As shown, to eliminate the influence of feature enhancement location factors, Experiment 4 inserted the same spatial attention module in the 2nd to 8th feature extraction stages of the network, while keeping the rest of the training and inference settings completely consistent (data partitioning, enhancement, optimizer, learning rate strategy, and early stopping criterion unchanged), and evaluated on the same test set. This setup ensures univariate control: no factors are changed except for the insertion layer index.

[0107] To further explore the role of the spatial attention module in the feature representation process, this invention uses a visualization method based on gradient-weighted class activation mapping (Grad-CAM) to analyze the spatial response distribution of the model on the last layer feature map. Figure 8 The following are visualization results of the feature responses of different models to crystal structure images provided by this invention, such as... Figure 8 As shown, the first column is the original image, the second column is the Grad-CAM heatmap generated by the ViT-L32 model, the third column is the Grad-CAM heatmap generated by the EfficientNet-B7 model, and the fourth column is the attention visualization result of the model of this invention.

[0108] After introducing the spatial attention module, the model's response region is significantly concentrated in key feature regions such as grain boundaries, phase interfaces, and internal structures, which are often closely related to the classification of microstructures. Visual comparisons of different models reveal that the constructed model has advantages in identifying boundary details and texture orientation, and the introduced spatial attention module is effective in the task of microstructure classification.

[0109] A fine-grained feature extraction method based on an improved EfficientNet-B7 and spatial attention mechanism. This invention addresses the challenges of complex microcrystalline textures and subtle grain boundary features in ultra-high carbon steel by innovatively embedding a spatial attention module at the output of the sixth feature extraction stage of the EfficientNet-B7 backbone network. This module aggregates spatial information through max pooling and average pooling in the channel dimension and utilizes convolutional layers to construct spatial dependencies, generating a spatial weight map. Through this mechanism, the network can adaptively enhance the feature response to key texture regions such as grain boundaries and cementite morphology, while suppressing background noise, thus effectively solving the problem of distinguishing highly similar categories such as pearlite and Widmanstätten structure.

[0110] An ensemble prediction method based on five-fold cross-validation and soft voting. Addressing the issues of small sample size and imbalanced distribution in materials science datasets, this invention employs an ensemble learning strategy. First, five-fold cross-validation is used to divide the dataset into five subsets for round-robin training, resulting in five optimal sub-models with different weight parameters. During the prediction phase, a soft voting mechanism is used to merging the probability distributions output by these five sub-models through arithmetic averaging. This method leverages the capabilities of multiple models, effectively reducing the prediction variance caused by the randomness of data partitioning in a single model, and significantly improving the robustness and generalization ability of the model in ultra-high carbon steel classification tasks.

[0111] Compared with the best existing technology, the advantages of the present invention are as follows: (1) It has a stronger ability to characterize key areas such as fine-grained texture and grain boundaries, thus improving the ability to distinguish highly similar categories.

[0112] Existing technologies largely rely on the global feature representation of the backbone network. However, when faced with scenarios involving complex textures, subtle grain boundary features, and high class similarity in the microstructure of ultra-high carbon steel, they are prone to insufficient response in key areas and misjudgments due to background noise interference. This invention embeds a spatial attention module at the output of the sixth feature extraction stage of the EfficientNet-B7 backbone network. By aggregating spatial information through max / average pooling along the channel dimension to generate a spatial weight map, the network can adaptively enhance key texture regions such as grain boundaries and cementite morphology while suppressing background noise. This makes it more effective in distinguishing highly similar categories such as pearlite and Widmanstätten structure.

[0113] (2) It is more robust under conditions of small sample size and unbalanced distribution, reducing prediction bias caused by accidental partitioning.

[0114] Existing technologies often employ one-time training / validation splits and single-model prediction, which are easily affected by the randomness of data splitting and are more prone to overfitting when the sample size is limited and the classes are imbalanced. This invention uses five-fold cross-validation to train multiple optimal sub-models and then uses soft voting to arithmetically average the probability distributions of each sub-model's output during the prediction phase. This integrates the complementary capabilities of multiple models, effectively reducing errors caused by single-model fluctuations and the randomness of data splitting, resulting in a more stable generalization ability in the ultra-high carbon steel classification task.

[0115] (3) The project is more user-friendly, and the modular improvement does not rely on additional annotations, making it easy to reuse and expand.

[0116] The spatial attention module of this invention is embedded in the backbone network as a pluggable structure, without the need to introduce additional artificial features or additional annotation information; the integrated prediction strategy is based on a general cross-validation and probabilistic fusion process, which is easy to transfer and use in similar material systems or similar microscopic tissue identification tasks, reducing the cost of scheme iteration.

[0117] (4) More balanced overall performance: It takes into account recognition accuracy, stability and noise resistance.

[0118] By combining the improvements of "fine-grained feature enhancement (spatial attention) + prediction variance reduction (five-fold integrated soft voting)," this invention is more likely to maintain stable discrimination in complex textures, noisy backgrounds, and subtle inter-class differences compared to existing best technologies. Overall, it achieves more reliable classification output and is more suitable for practical detection and analysis applications of material microstructures.

[0119] When applying the ultra-high carbon steel microcrystal classification method based on spatial attention and ensemble prediction provided in this manual, it is not necessary to rely on... Figure 1 The steps shown are executed in sequence. The specific execution order of each step can be determined as needed, and this manual does not impose any restrictions on it.

[0120] The above describes one or more embodiments of the ultra-high carbon steel microcrystal classification method based on spatial attention and ensemble prediction provided in this specification. Based on the same idea, this specification also provides a corresponding ultra-high carbon steel microcrystal classification device based on spatial attention and ensemble prediction, such as... Figure 9 As shown.

[0121] Figure 9 A schematic diagram of an ultra-high carbon steel microcrystal classification device based on spatial attention and ensemble prediction, provided in this specification, includes: The acquisition module 901 is used to acquire images of the microscopic crystals of ultra-high carbon steel to be classified.

[0122] Extraction module 902 is used to obtain K improved EfficientNet-B7 models; the K improved EfficientNet-B7 models are obtained by training with K-fold cross-validation based on the initial improved EfficientNet-B7 model structure; the initial improved EfficientNet-B7 model adds a spatial attention module between the output of the sixth feature extraction stage and the input of the seventh feature extraction stage of the EfficientNet-B7 model.

[0123] The generation module 903 is used to classify the microcrystalline images of ultra-high carbon steel to be classified using each improved EfficientNet-B7 model, and to obtain the probability distribution of each category predicted by each improved EfficientNet-B7 model.

[0124] The determination module 904 is used to perform an arithmetic average of the multiple probability distributions of each category predicted by multiple improved EfficientNet-B7 models to obtain a fused probability vector, and select the category corresponding to the maximum probability in the fused probability vector as the category of the microscopic crystal image to be classified.

[0125] Specific limitations regarding the classification device for ultra-high carbon steel microcrystals based on spatial attention and ensemble prediction can be found in the limitations of the classification method for ultra-high carbon steel microcrystals based on spatial attention and ensemble prediction mentioned above, and will not be repeated here. Each module in the aforementioned classification device for ultra-high carbon steel microcrystals based on spatial attention and ensemble prediction can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0126] This specification also provides a computer-readable storage medium storing a computer program that can be used to execute the above-described... Figure 1 A method for classifying ultra-high carbon steel microcrystals based on spatial attention and ensemble prediction is provided.

[0127] This instruction manual also provides Figure 10 The schematic diagram of the computer device shown is as follows: Figure 10 At the hardware level, the computer device includes a processor, internal bus, network interface, memory, and non-volatile memory, and may also include other hardware required for business operations. The processor reads the corresponding computer program from the non-volatile memory into memory and then runs it to achieve the above-mentioned functions. Figure 1 A method for classifying ultra-high carbon steel microcrystals based on spatial attention and ensemble prediction is provided.

[0128] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0129] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

Claims

1. A method for classifying ultra-high carbon steel microcrystals based on spatial attention and ensemble prediction, characterized in that, include: Obtain images of the microscopic crystals of ultra-high carbon steel to be classified; Obtain K improved EfficientNet-B7 models; the K improved EfficientNet-B7 models are obtained by training with K-fold cross-validation based on the initial improved EfficientNet-B7 model structure; the initial improved EfficientNet-B7 model adds a spatial attention module between the output of the sixth feature extraction stage and the input of the seventh feature extraction stage of the EfficientNet-B7 model. By classifying the microcrystalline images of ultra-high carbon steel using each improved EfficientNet-B7 model, the probability distribution of each category predicted by each improved EfficientNet-B7 model is obtained. The probability distributions of each category predicted by multiple improved EfficientNet-B7 models are arithmetically averaged to obtain a fused probability vector. The category corresponding to the maximum probability in the fused probability vector is then selected as the category of the microcrystalline image to be classified.

2. The method for classifying ultra-high carbon steel microcrystals based on spatial attention and ensemble prediction as described in claim 1, characterized in that, The improved EfficientNet-B7 model includes: a cascaded 3×3 convolutional layer (Conv3×3), four 3×3 moving-inverted bottleneck convolutional blocks (MBConv1 3×3), seven 3×3 moving-inverted bottleneck convolutional blocks (MBConv6 3×3), seven 5×5 moving-inverted bottleneck convolutional blocks (MBConv6 5×5), ten 3×3 moving-inverted bottleneck convolutional blocks (MBConv6 3×3), ten 5×5 moving-inverted bottleneck convolutional blocks (MBConv6 5×5), a spatial attention module, thirteen 5×5 moving-inverted bottleneck convolutional blocks (MBConv6 5×5), four 3×3 moving-inverted bottleneck convolutional blocks (MBConv6 3×3), and a classification head; the classification head includes a 1×1 convolutional layer (Conv6). 1×1 pooling layer, fully connected layer (FC), and classification layer (Classification).

3. The method for classifying ultra-high carbon steel microcrystals based on spatial attention and ensemble prediction as described in claim 2, characterized in that, The improved EfficientNet-B7 model was used to classify microscopic crystal images of ultra-high carbon steel, and the probability distributions of each category predicted by the improved EfficientNet-B7 model were obtained, including: The microscopic crystal image to be classified is sequentially passed through a 3×3 convolutional layer Conv3×3, a 4-kernel 3×3 moving-inverted bottleneck convolutional block MBConv1 3×3, a 7-kernel 3×3 moving-inverted bottleneck convolutional block MBConv6 3×3, a 7-kernel 5×5 moving-inverted bottleneck convolutional block MBConv6 5×5, a 10-kernel 3×3 moving-inverted bottleneck convolutional block MBConv6 3×3, and a 10-kernel 5×5 moving-inverted bottleneck convolutional block MBConv6 5×5, thus mapping the microscopic crystal image to be classified into a feature representation. The feature representation is input into the spatial attention module to generate a spatial attention weight map; The spatial attention weight map is multiplied element-wise with the feature representation to highlight the region response related to key discriminative textures, resulting in enhanced features; key discriminative textures include grain boundaries, phase boundaries, lath layers, and lamellae. The enhanced features are sequentially passed through 13 moving inverted bottleneck convolutional blocks MBConv6 5×5 with 5×5 kernels, 4 moving inverted bottleneck convolutional blocks MBConv6 3×3 with 3×3 kernels, and a classification head to obtain the probability distribution of each category.

4. The method for classifying ultra-high carbon steel microcrystals based on spatial attention and ensemble prediction as described in claim 3, characterized in that, The step of inputting the feature representation into the spatial attention module to generate a spatial attention weight map specifically includes: The feature representations are subjected to max pooling and average pooling along the channel axis respectively to obtain max pooling spatial feature description maps and average pooling spatial feature description maps. The max pooling spatial feature description map and the average pooling spatial feature description map are concatenated along the channel dimension to obtain a comprehensive representation; The comprehensive representation is passed through a convolutional layer and The function generates a spatial attention weight map.

5. The method for classifying ultra-high carbon steel microcrystals based on spatial attention and ensemble prediction as described in claim 2, characterized in that, The improved loss function for the EfficientNet-B7 model is as follows: in, To improve the loss value of the EfficientNet-B7 model, For the current real category The number of samples, For category The number of samples, The total number of categories, As a regulating factor, To improve the EfficientNet-B7 model's prediction probability of the correct class.

6. A classification device for ultra-high carbon steel microcrystals based on spatial attention and ensemble prediction, characterized in that, include: The first acquisition module is used to acquire images of the microscopic crystals of ultra-high carbon steel to be classified. The second acquisition module is used to acquire K improved EfficientNet-B7 models. The K improved EfficientNet-B7 models are obtained by training with K-fold cross-validation based on the initial improved EfficientNet-B7 model structure. The initial improved EfficientNet-B7 model adds a spatial attention module between the output of the sixth feature extraction stage and the input of the seventh feature extraction stage of the EfficientNet-B7 model. The generation module is used to classify the microcrystalline images of ultra-high carbon steel to be classified using each improved EfficientNet-B7 model, and obtain the probability distribution of each category predicted by each improved EfficientNet-B7 model. The prediction module is used to perform an arithmetic average of the multiple probability distributions of each category predicted by multiple improved EfficientNet-B7 models to obtain a fused probability vector, and select the category corresponding to the maximum probability in the fused probability vector as the category of the microscopic crystal image to be classified.

7. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the method described in any one of claims 1 to 5.

8. A computer device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described in any one of claims 1 to 5.

Images