Steel microstructure identification method and system based on deep learning driving and medium

Abstract

The invention discloses a steel microstructure identification method and system based on deep learning driving, and a medium. The method comprises the steps of obtaining a steel microstructure image, performing data enhancement, and performing division to obtain a training set and a verification set; based on the MobileNetV2 pre-training model, using a transfer learning method to construct a SteelIDNet model; using the training set to train the SteelIDNet model, and using the verification set to verify and evaluate the trained SteelIDNet model to obtain a final SteelIDNet model; and deploying the final SteelIDNet model at a mobile terminal to carry out steel microstructure identification. According to the method, the SteelIDNet model is constructed through deep learning and transfer learning technologies, the advantage (CNN) of the convolutional neural network is brought into full play, and the recognition efficiency of the steel microstructure is remarkably improved; the method is not only suitable for PC terminal development, but also convenient to deploy on a mobile terminal, can realize millisecond-level response speed, and is particularly suitable for high-efficiency requirements of the mobile terminal on real-time detection and analysis.

Description

Technical Field

[0001] The present invention belongs to the technical field of steel microstructure detection, and specifically relates to a steel microstructure identification method, system and medium driven by deep learning. Background Art

[0002] As a cornerstone material of modern industry, steel's performance and widespread application are largely influenced by its microstructure. Steel microstructure typically consists of pearlite, ferrite, austenite, bainite, and martensite. The proportions, distribution, and morphology of these microstructures play a decisive role in steel's strength, toughness, wear resistance, and corrosion resistance. Furthermore, steel microstructures are diverse and varied, and the features of different microstructures in images may overlap (e.g., blurred boundaries between pearlite and ferrite), complicating classification. Traditionally, microstructural analysis relies primarily on metallographic and electron microscopes, with classification and quantitative analysis performed through manual observation or simple image processing techniques. While accurate, this method has the following limitations: 1. Expertise reliance: analysis requires experienced technicians and is highly subjective; 2. Time-consuming and labor-intensive: manual analysis of a single microscopic image typically takes a long time and is inefficient; 3. Limited processing capacity: in large-scale production scenarios, it is difficult to meet the needs of real-time analysis.

[0003] Deep learning, especially convolutional neural networks (CNNs), has performed well in the fields of image classification, target detection, and semantic segmentation. It can automatically extract image features through large-scale data training, avoiding the complex manual feature design process in traditional image processing technology. It can efficiently and automatically extract the morphology, texture, and distribution characteristics of steel structures, improving analysis accuracy. It is also highly robust to noise, sample contrast changes, and sample diversity, and performs stably in complex microscopic images. It can also quickly process large quantities of microscopic images to meet the needs of real-time detection in the industrial field. It is applicable to a variety of steel grades and complex structures and has strong scalability and practical value. For example, the invention with patent number CN202011391287.7 proposes a method for evaluating banded structure in steel, but its main limitation is that it only focuses on the evaluation of banded structure, making it difficult to effectively identify other types of metallographic structures (such as austenite, bainite, ferrite, etc.). Patent number CN201910802572.4 uses Keras to improve the existing convolutional neural network Vgg16, successfully constructing an improved deep learning framework CNN-ICAM, which has excellent recognition accuracy across different tissue structures. However, this model is still only suitable for testing and training small samples, and is difficult to efficiently process when faced with massive amounts of training materials of tens of thousands or more. Patent number CN202310149249.8 uses the TensorFlow framework to build a Resnet50 model for the recognition of microstructures such as ferrite, pearlite, martensite, and bainite. However, the application of this model is still limited to a specific test environment on the PC side, and it is highly dependent on computing resources, making it difficult to adapt to mobile devices. The invention with patent number CN202211507567.9 provides a steel microstructure and device based on deep learning, and combines computer vision technology to extract image information. However, its trained model is mainly deployed in computer-usable media with available program code. In practical applications, it has certain limitations, such as the need for specific hardware support, low flexibility, and difficulty in meeting portability requirements. Summary of the Invention

[0004] The main purpose of the present invention is to overcome the shortcomings and deficiencies of the existing technology and provide a steel microstructure identification method, system and medium driven by deep learning, aiming to solve the problems of manual errors, low efficiency and inaccurate analysis in traditional steel microstructure detection methods.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] In one aspect, the present invention provides a method for identifying steel microstructure based on deep learning, comprising the following steps:

[0007] Obtain steel microstructure images for data enhancement and division to obtain training and validation sets;

[0008] Build the SteelIDNet model using transfer learning based on the MobileNetV2 pre-trained model;

[0009] Use the training set to train the SteelIDNet model, and use the validation set to verify and evaluate the trained SteelIDNet model to obtain the final SteelIDNet model;

[0010] The final SteelIDNet model is deployed on the mobile terminal for steel microstructure identification.

[0011] As a preferred technical solution, the data enhancement is specifically as follows:

[0012] Determine the filling size of the steel microstructure image and fill it to obtain a filling image of uniform size;

[0013] Randomly flip and rotate the filled image to obtain the enhanced image;

[0014] The filled image and the enhanced image are mixed to obtain the data enhanced steel microstructure image.

[0015] As a preferred technical solution, the padding size is calculated as follows:

[0016] The original size of the steel microstructure image is (H orig ,W orig ), set the target size (H target ,W target );

[0017] Calculate the image scaling ratio scale = min(H target / H orig ,W target / W orig );

[0018] Adjust the size of the microscopic tissue image according to the image scaling ratio to obtain the new size (H new ,W new ), where W new =W orig ×scale,H new =H orig ×scale;

[0019] Calculate the padding of the steel microstructure image left =(W target -W new ) / 2 and padding top =(H target -H new) / 2, where padding left The number of pixels that need to be filled on the left and right sides of the steel microstructure image; padding top The number of pixels that need to be filled above and below the steel microstructure image.

[0020] As a preferred technical solution, the SteelIDNet model includes a feature extraction layer based on a pre-trained model and an additional added layer; the additional added layer includes an input layer, a channel expansion layer, a feature compression layer, a feature flattening layer, a deep feature extraction layer, a batch normalization layer, and an output layer;

[0021] The input layer is used to read the input image;

[0022] The feature extraction layer is used to extract the feature map of the image;

[0023] The channel expansion layer is used to reduce the spatial dimension of the feature map and increase the number of feature expression channels;

[0024] The feature compression layer is used to perform a global average pooling operation on the feature map;

[0025] The feature flattening layer is used to flatten the pooled feature map into a one-dimensional vector;

[0026] The depth feature extraction layer is used to extract the depth features of the one-dimensional vector;

[0027] The batch normalization layer is used to normalize the deep feature extraction layer;

[0028] The output layer is used to perform classification tasks and identify the type of steel microstructure based on the normalized deep features.

[0029] As a preferred technical solution, the input layer is constructed using a standard convolutional layer with a convolution kernel size of 3×3 and a sliding step size of 2, expanding the input image to 32 channels;

[0030] The feature extraction layer uses multiple bottleneck convolution groups, each of which contains n stacked bottleneck convolutions, and each bottleneck convolution contains a channel expansion module, a spatial feature extraction module, and a linear compression module; the channel expansion module expands the number of channels to t times the original number through 1×1 convolution according to the expansion factor t; the spatial feature extraction module uses 3×3 depthwise separable convolution;

[0031] The channel expansion layer uses a 1×1 convolution layer to expand the number of channels of the feature map to 1280 and compress the size to 5×5;

[0032] The feature compression layer uses a global average pooling layer to compress the 5×5 feature map into a 1×1 feature map;

[0033] The deep feature extraction layer uses two fully connected layers to compress the feature number from 1280 to 512, and uses the RELU activation function for nonlinear mapping;

[0034] The batch normalization layer includes two layers; each batch normalization layer is connected after the fully connected layer to improve the stability of training;

[0035] The output layer uses the Softmax activation function to perform classification and output the classification results.

[0036] As a preferred technical solution, the final SteelIDNet model is obtained as follows:

[0037] Freeze a set number of feature extraction layers in the pre-trained model and fine-tune the parameters of the remaining layers;

[0038] Randomly select samples from the training set, shuffle them, and process them in batches to obtain sub-training sets of multiple batches;

[0039] The sub-training sets are fed into the SteelIDNet model in batches for iterative training. Classification cross entropy is used as the loss function for parameter update, and the Adam optimization algorithm is used to accelerate training.

[0040] The validation set is input into the trained SteelIDNet model for prediction and verification of the prediction accuracy. If the verification fails, iterative training continues. If the verification passes, the final SteelIDNet model is obtained.

[0041] As a preferred technical solution, the loss function is expressed as:

[0042]

[0043] Among them, y i represents the true category label corresponding to the i-th steel microstructure image, is the predicted probability of the image classification by the i-th representation model, and size is the total number of training samples;

[0044] The prediction accuracy calculation formula is:

[0045]

[0046] Among them, total_train_correct_sum is the number of images predicted correctly after each training, and total_train_sum is the total number of images in a single training.

[0047] As a preferred technical solution, the final SteelIDNet model is deployed on the mobile terminal to perform steel microstructure identification, specifically:

[0048] The final SteelIDNet model was lightweighted using TensorFlow Lite and developed for mobile devices using Android Studio, resulting in a steel microstructure identification application that included the final SteelIDNet model.

[0049] Install the steel microstructure identification application on the mobile device and use the mobile device to capture the steel microstructure image in real time;

[0050] The steel microstructure recognition application adjusts the image data format to the input format of the final SteelIDNet model, loads the final SteelIDNet model and recognizes the input image;

[0051] Update the recognition results to the new UI and display the predicted category or confidence information of the image.

[0052] On the other hand, the present invention provides a steel microstructure identification system based on deep learning, which is applied to the steel microstructure identification method based on deep learning, and includes a data processing module, a model building module, a model training module, and a deployment identification module;

[0053] The data processing module is used to obtain steel microstructure images, perform data enhancement and division, and obtain a training set and a validation set;

[0054] The model building module is used to build a SteelIDNet model using a transfer learning method based on the MobileNetV2 pre-training model;

[0055] The model training module is used to train the SteelIDNet model using the training set, and to verify and evaluate the trained SteelIDNet model using the verification set to obtain the final SteelIDNet model;

[0056] The deployment and identification module is used to deploy the final SteelIDNet model on the mobile terminal to perform steel microstructure identification.

[0057] On the other hand, a computer-readable storage medium is provided, which stores a program. When the program is executed by a processor, the steel microstructure identification method driven by deep learning is implemented.

[0058] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0059] (1) The present invention builds a SteelIDNet model based on MobileNetV2, which significantly improves the recognition efficiency of steel microstructure. Compared with traditional manual analysis and manual feature extraction algorithms, the SteelIDNet model of the present invention fully utilizes the advantages of convolutional neural networks (CNN) in steel microscopic classification and recognition tasks through the application of deep learning and transfer learning technology, achieving efficient and accurate microstructure recognition.

[0060] (2) The SteelIDNet model constructed by the present invention has a powerful feature extraction capability. The feature extraction layer constructed based on the MobileNetV2 pre-training model can fully explore the texture, edge and shape features of the microstructure image, thereby enhancing the model's ability to extract the local structure of the image. The channel expansion layer uses a 1×1 convolution layer to expand the number of feature channels to 1280, thereby enhancing the feature expression capability. The deep feature extraction layer uses two fully connected layers to compress the number of features from 1280 to 512, and introduces a Batch Normalization layer to alleviate the gradient disappearance and improve the training stability. In particular, the classification accuracy is high when facing complex microstructures (such as pearlite, ferrite, bainite, etc.).

[0061] (3) With the help of the SteelIDNet model, when faced with massive images (>10,000), especially those with complex features, the initial convolutional layer extracts low-level features (such as edges, textures, and interfaces), and then extracts high-level semantic features through multiple bottleneck layers. At the same time, the pooling layer is used to sample less and retain global information while reducing computation. A reasonable fully connected layer design is used to refine global features and gradually map feature vectors to the classification space.

[0062] (3) The SteelIDNet model is lightweight and supports cross-platform deployment. It is not only suitable for PC development, but also easy to deploy on mobile devices (such as smartphones or embedded detection devices). By converting the model to the TensorFlow Lite format (.tflite) and importing it into a mobile application developed in Android Studio, it can achieve millisecond-level response speeds, which is particularly suitable for the efficient needs of mobile devices for real-time detection and analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0064] Figure 1This is an overall flow chart of the steel microstructure identification method driven by deep learning in an embodiment of the present invention.

[0065] Figure 2 This is a diagram showing the difference between direct scaling and proportional scaling of the ferrite microscopic image input in an embodiment of the present invention.

[0066] Figure 3 Schematic diagram of the random rotation process of the ferrite microscopic image input for the present invention.

[0067] Figure 4 This is a comparison chart of the multi-model loss function Loss in an embodiment of the present invention.

[0068] Figure 5 This is a comparison chart of the multi-model prediction accuracy trainAcc in an embodiment of the present invention.

[0069] Figure 6 This is a network architecture diagram of the SteelIDNet model in an embodiment of the present invention.

[0070] Figure 7 This is a test sample image of the steel material microstructure image dataset in an embodiment of the present invention.

[0071] Figure 8 This is a test data result diagram of the steel material microstructure image dataset in an embodiment of the present invention.

[0072] Figure 9 This is an overall block diagram of the steel microstructure identification system driven by deep learning in an embodiment of the present invention.

[0073] Figure 10 Schematic diagram of the structure of a computer-readable storage medium in an embodiment of the present invention. DETAILED DESCRIPTION

[0074] In order to enable those skilled in the art to better understand the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0075] References to "embodiments" in this application mean that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described in this application may be combined with other embodiments.

[0076] like Figure 1 As shown, this embodiment provides a steel microstructure identification method based on deep learning, including the following steps:

[0077] S1. Obtain steel microstructure images for data enhancement and division to obtain training and validation sets, providing a data basis for subsequent model training.

[0078] In order to ensure the accuracy and effectiveness of subsequent model training, the acquired image data is first enhanced to increase data diversity, prevent overfitting in subsequent model training, and improve the generalization ability of the model. The specific data enhancement is as follows:

[0079] (1) Determine the filling size of the steel microstructure image and fill it to obtain a filling image of uniform size.

[0080] Furthermore, the padding size in this application is calculated as follows:

[0081] The original size of the steel microstructure image is (H orig ,W orig ), set the target size (H target ,W target );

[0082] Calculate the image scaling ratio scale = min(H target / H orig ,W target / W orig );

[0083] Adjust the size of the microscopic tissue image according to the image scaling ratio to obtain the new size (H new ,W new ), where W new =W orig ×scale,H new =H orig ×scale;

[0084] Calculate the padding of the steel microstructure image left =(W target -W new ) / 2 and padding top=(H target -H new ) / 2, where padding left Padding is the number of pixels that need to be filled on the left and right sides of the steel microstructure image to center the image. top The number of pixels that need to be filled above and below the steel microstructure image to center the image above and below.

[0085] For example, if the original image is (500×300), it must first be scaled proportionally. The height and width scaling ratios are calculated based on the target size (160×160). The width scaling ratio is 0.32, so the image width is scaled to 160 pixels, while the height is scaled to 96 pixels using the same ratio. Since the target size is 160×160 pixels and the scaled size is 160×96 pixels, padding is required. The calculation shows that 32 pixels of padding are required at the top and bottom to ensure that the target size is adjusted to 160×160 pixels and the image content remains centered in the target area.

[0086] (2) Randomly flip and rotate the filled image to obtain an enhanced image, enhance data diversity, and improve the generalization ability of the subsequent model. Usually, the image is randomly flipped horizontally by transforming it left and right, and then the rotation radius is set to [-0.2, 0.2] to randomly rotate the image.

[0087] (3) The filled image and the enhanced image are mixed to obtain the data enhanced steel microstructure image. Figure 3 The image is shown after data processing. After random flipping and rotation, the image generates grain and grain boundary distributions with different arrangements and morphologies. This transformation increases the number of training sets while maintaining the original structure, reduces overfitting, and further enhances the model's generalization ability.

[0088] In this embodiment, the data set after data augmentation is randomly divided into a ratio of 9:1 to obtain a 90% training set and a 10% validation set.

[0089] S2. The SteelIDNet model is constructed using the transfer learning method based on the MobileNetV2 pre-trained model. Additional layers are added to the MobileNetV2 pre-trained model, such as the global average pooling layer (average_pool), the fully connected layer (dense), the batch normalization layer (batch normalization), and other model structures to enhance the model's expressiveness and adapt it to the task of identifying the microstructure of steel.

[0090] For the selection of pre-trained models, this application compares and selects by loading different mainstream pre-trained models: MobileNetV2, InceptionV3, Resnet50, and InceptionResNetV2. First, ensure that the input shape and data pre-processing method of all models are consistent to ensure fairness in comparison. Based on the training results, use the pre-prepared comparison data set to evaluate the error and accuracy of different models. Figure 4 As shown in the figure, in terms of error, both Resnet50 and MobileNetV2 have lower initial errors; however, as the number of iterations increases, MobileNetV2 shows lower error levels than other models. Figure 5 As shown in the results, MobileNetV2 also showed higher model accuracy than other models. This result not only highlights the characteristics of MobileNetV2 in maintaining efficient feature extraction capabilities despite its lightweight design, but also verifies its good adaptability to classification accuracy in complex tasks. After comparison, MobileNetV2 was finally selected as the pre-training model.

[0091] Further, such as Figure 4 As shown in the figure, the SteelIDNet (Steel Microstructure Identification Network) model includes a feature extraction layer based on a pre-trained model, as well as additional layers: input layer, channel expansion layer, feature compression layer, feature flattening layer, deep feature extraction layer, batch normalization layer, and output layer.

[0092] Among them, the input layer is used to read the input image; further, the input layer in this application is constructed using a standard convolution layer with a convolution kernel size of 3×3 and a sliding step size of 2, which expands the input image (RGB channels, i.e., 3 channels) to 32 channels.

[0093] The feature extraction layer is used to extract the features of the image; further, the feature extraction layer includes multiple bottleneck convolution groups, each bottleneck convolution group includes a bottleneck convolution (Bottleneck) with a stacking number of n, and each bottleneck convolution includes a channel expansion module, a spatial feature extraction module and a linear compression module. Among them, the channel expansion module expands the number of channels to t times the original through 1×1 convolution according to the expansion factor t; the spatial feature extraction module uses 3×3 depth-separable convolution to extract spatial features. In this embodiment, the expansion factor t=6, the bottleneck convolution group is 5, and the stacking number of bottleneck convolutions in each bottleneck convolution group is 1, 2, 3, 3, and 3 respectively; therefore, the feature extraction layer inputs the feature map into the 5 bottleneck convolution groups in sequence, and the spatial size of the feature map becomes 80×80, 40×40, 20×20, 10×10, and 5×5 in sequence, and the number of channels is gradually expanded to 16, 24, 32, 96, and 160. It should be noted that the number of stacks n, the expansion factor t, and the number of channels in each layer in the bottleneck convolution group of the present invention can be flexibly configured according to the specific application scenario and computing resources. This embodiment only describes a typical implementation method.

[0094] The channel expansion layer is used to reduce the spatial dimension of the feature map and increase the number of feature expression channels. Furthermore, the channel expansion layer uses a 1×1 convolutional layer to expand the number of channels of the feature map to 1280 and compress the size to 5×5, compressing the spatial information while increasing the feature expression capability.

[0095] The feature compression layer is used to perform a global average pooling operation on the feature map; further, the feature compression layer uses a global average pooling layer to compress the 5×5 feature map into a 1×1 feature map.

[0096] The feature flattening layer is used to flatten the pooled feature map into a one-dimensional vector to facilitate input into the deep feature extraction layer.

[0097] The deep feature extraction layer is used to extract the deep features of the one-dimensional vector. Furthermore, the deep feature extraction layer uses two fully connected layers to further compress the features and adopts the RELU activation function for nonlinear mapping. The number of features is compressed from 1280 to 512.

[0098] The batch normalization layer is used to normalize the deep feature extraction layer. Furthermore, the batch normalization layer consists of two layers. After each batch normalization layer is connected to a fully connected layer, it alleviates the problem of gradient disappearance, improves the stability of training, and accelerates the convergence speed.

[0099] The output layer is used to perform classification tasks and identify the type of steel microstructure based on the normalized deep features; further, the output layer uses the Softmax activation function for classification and outputs the classification results.

[0100] S3. Use the training set to train the SteelIDNet model, and use the validation set to verify and evaluate the trained SteelIDNet model to obtain the final SteelIDNet model.

[0101] Furthermore, the constructed SteelIDNet model is trained and evaluated on the training set and validation set to obtain a stable model with good performance. The process is as follows:

[0102] S3.1. First, to preserve the feature extraction capabilities of the pre-trained model as much as possible while avoiding over-adjustment of underlying features, a set number of feature extraction layers in the pre-trained model are frozen, and the parameters of the remaining layers (i.e., the additional layers) are fine-tuned to avoid changing the learned effective features during training. In this embodiment, the first 100 feature extraction layers are frozen.

[0103] S3.2. Next, randomly select samples from the training set and shuffle them to reduce the impact of sample order on the model and improve the model's generalization ability. The samples are then batched to obtain multiple batches of sub-training sets. In this example, 1000 samples are shuffled and batched, with each batch containing 64 samples.

[0104] S3.3. Subsequently, the sub-training sets are fed into the SteelIDNet model in batches for iterative training. Classification cross entropy is used as the loss function for parameter update, and the Adam optimization algorithm is used to accelerate training.

[0105] For each training sub-set, the image data in the sub-set is read and fed into the model for forward propagation to generate a prediction result. The prediction result generated by the model is compared with the actual steel microstructure label to calculate the loss function. Then, using the set learning rate learning_ratio, the model parameters are backpropagated and optimized through the optimizer (Adam optimizer). The loss function is expressed as:

[0106]

[0107] Among them, y i represents the true category label corresponding to the i-th steel microstructure image, is the predicted probability of the image classification by the i-th representation model, and size is the total number of training samples;

[0108] S3.4. Finally, the validation set is input into the trained SteelIDNet model for prediction and verification of the prediction accuracy. If the verification fails, iterative training is continued. If the verification passes, the final SteelIDNet model is obtained.

[0109] After each iterative training, the validation set is input into the trained SteelIDNet model to generate prediction results. The number of correctly predicted images is recorded to calculate the prediction accuracy. The formula is:

[0110]

[0111] Among them, total_train_correct_sum is the number of models that predict correctly after each training, and total_train_sum is the total number of images in a single training.

[0112] In this embodiment, Figure 7 The following is a partial dataset of steel microstructure images with different inputs, and the prediction results are as follows: Figure 8 shown.

[0113] S4. Deploy the final SteelIDNet model on the mobile terminal for steel microstructure identification.

[0114] To meet the needs of rapid detection in industrial sites, this application deploys the final SteelIDNet model on mobile devices, which facilitates real-time detection and improves work efficiency. Specifically:

[0115] S4.1. First, use the TensorFlow Lite tool to lightweight the final SteelIDNet model and convert it into a .tflite file in the TensorFlow Lite format for mobile devices. Then use Android Studio to develop mobile devices and import the .tflite file model to obtain a steel microstructure identification application containing the final SteelIDNet model.

[0116] S4.2. Next, install the steel microstructure identification application in the mobile device and use the mobile device to capture the steel microstructure image in real time.

[0117] S4.3. Subsequently, the steel microstructure recognition application adjusts the image data format to the input format of the final SteelIDNet model, loads the final SteelIDNet model and recognizes the input image. In this embodiment, the input format is 160×160.

[0118] S4.4. Finally, the recognition results are updated to the new UI, and information such as the predicted category or confidence level of the image is displayed.

[0119] It should be noted that, for the sake of convenience, the aforementioned method embodiments are all expressed as a series of action combinations, but those skilled in the art should know that the present invention is not limited to the described order of actions, because according to the present invention, certain steps can be performed in other orders or simultaneously.

[0120] Based on the same concept as the deep learning-driven steel microstructure identification method in the above-mentioned embodiment, the present invention also provides a deep learning-driven steel microstructure identification system, which can be used to implement the above-mentioned deep learning-driven steel microstructure identification method. For ease of explanation, the structural diagram of the embodiment of the deep learning-driven steel microstructure identification system only shows the parts related to the embodiment of the present invention. Those skilled in the art will understand that the illustrated structure does not constitute a limitation of the device, and it may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0121] like Figure 9 As shown, another embodiment of the present invention provides a steel microstructure identification system driven by deep learning, including a data processing module, a model building module, a model training module and a deployment identification module;

[0122] The data processing module is used to obtain steel microstructure images for data enhancement and division to obtain training sets and validation sets.

[0123] The model building module is used to build the SteelIDNet model using the transfer learning method based on the MobileNetV2 pre-trained model;

[0124] The model training module is used to train the SteelIDNet model using the training set and to verify and evaluate the trained SteelIDNet model using the validation set to obtain the final SteelIDNet model;

[0125] The deployment and identification module is used to deploy the final SteelIDNet model on the mobile terminal for steel microstructure identification.

[0126] It should be noted that the steel microstructure identification system based on deep learning driving of the present invention corresponds one-to-one to the steel microstructure identification method based on deep learning driving of the present invention. The technical features and beneficial effects described in the embodiment of the above-mentioned steel microstructure identification method based on deep learning driving are applicable to the embodiment of the steel microstructure identification system based on deep learning driving. For specific contents, please refer to the description in the embodiment of the method of the present invention. No further details will be given here. This is hereby declared.

[0127] In addition, in the implementation of the deep learning-driven steel microstructure identification system in the above embodiment, the logical division of each program module is only an example. In actual application, the above functions can be assigned to different program modules as needed, for example, for the convenience of corresponding hardware configuration requirements or software implementation. That is, the internal structure of the deep learning-driven steel microstructure identification system can be divided into different program modules to complete all or part of the functions described above.

[0128] like Figure 10 As shown, in one embodiment, a computer-readable storage medium is provided, which stores a program in a memory. When the program is executed by a processor, the steel microstructure identification method based on deep learning is implemented, specifically:

[0129] Obtain steel microstructure images for data enhancement and division to obtain training and validation sets;

[0130] Build the SteelIDNet model using transfer learning based on the MobileNetV2 pre-trained model;

[0131] Use the training set to train the SteelIDNet model, and use the validation set to verify and evaluate the trained SteelIDNet model to obtain the final SteelIDNet model;

[0132] The final SteelIDNet model is deployed on the mobile terminal for steel microstructure identification.

[0133] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program. The program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0134] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0135] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be considered as equivalent replacement methods and are included in the scope of protection of the present invention.

Claims

1. A steel microstructure identification method based on deep learning, characterized in that: The steps include: Obtain steel microstructure images for data enhancement and division to obtain training and validation sets; Build the SteelIDNet model using transfer learning based on the MobileNetV2 pre-trained model; Use the training set to train the SteelIDNet model, and use the validation set to verify and evaluate the trained SteelIDNet model to obtain the final SteelIDNet model; The final SteelIDNet model is deployed on the mobile terminal for steel microstructure identification.

2. The steel microstructure identification method based on deep learning drive according to claim 1, characterized in that: The data enhancement is specifically as follows: Determine the filling size of the steel microstructure image and fill it to obtain a filling image of uniform size; Randomly flip and rotate the filled image to obtain the enhanced image; The filled image and the enhanced image are mixed to obtain the data enhanced steel microstructure image.

3. The steel microstructure identification method based on deep learning drive according to claim 2, characterized in that: The padding size is calculated as follows: The original size of the steel microstructure image is (H orig ,W orig ), set the target size (H target ,W target ); Calculate the image scaling ratio scale = min(H target / H orig ,W target / W orig ); Adjust the size of the microscopic tissue image according to the image scaling ratio to obtain the new size (H new ,W new ), where W new =W orig ×scale,H new =H orig ×scale; Calculate the padding of the steel microstructure image left =(W target -W new ) / 2 and padding top =(H target -H new ) / 2, where padding left The number of pixels that need to be filled on the left and right sides of the steel microstructure image; padding top The number of pixels that need to be filled above and below the steel microstructure image.

4. The steel microstructure identification method based on deep learning drive according to claim 1, characterized in that: The SteelIDNet model includes a feature extraction layer based on a pre-trained model and additional layers; the additional layers include an input layer, a channel expansion layer, a feature compression layer, a feature flattening layer, a deep feature extraction layer, a batch normalization layer, and an output layer; The input layer is used to read the input image; The feature extraction layer is used to extract the feature map of the image; The channel expansion layer is used to reduce the spatial dimension of the feature map and increase the number of feature expression channels; The feature compression layer is used to perform a global average pooling operation on the feature map; The feature flattening layer is used to flatten the pooled feature map into a one-dimensional vector; The depth feature extraction layer is used to extract the depth features of the one-dimensional vector; The batch normalization layer is used to normalize the deep feature extraction layer; The output layer is used to perform classification tasks and identify the type of steel microstructure based on the normalized deep features.

5. The steel microstructure identification method based on deep learning drive according to claim 4 is characterized in that: The input layer is constructed using a standard convolutional layer with a convolution kernel size of 3×3 and a sliding step size of 2, which expands the input image to 32 channels; The feature extraction layer uses multiple bottleneck convolution groups, each of which contains n stacked bottleneck convolutions, and each bottleneck convolution includes a channel expansion module, a spatial feature extraction module, and a linear compression module; the channel expansion module expands the number of channels to t times the original number through 1×1 convolution according to the expansion factor t; the spatial feature extraction module uses 3×3 depthwise separable convolution; The channel expansion layer uses a 1×1 convolution layer to expand the number of channels of the feature map to 1280 and compress the size to 5×5; The feature compression layer uses a global average pooling layer to compress the 5×5 feature map into a 1×1 feature map; The deep feature extraction layer uses two fully connected layers to compress the feature number from 1280 to 512, and uses the RELU activation function for nonlinear mapping; The batch normalization layer includes two layers; each batch normalization layer is connected after the fully connected layer to improve the stability of training; The output layer uses the Softmax activation function to perform classification and output the classification results.

6. The steel microstructure identification method based on deep learning drive according to claim 1, characterized in that: The final SteelIDNet model is obtained as follows: Freeze a set number of feature extraction layers in the pre-trained model and fine-tune the parameters of the remaining layers; Randomly select samples from the training set, shuffle them, and process them in batches to obtain sub-training sets of multiple batches; The sub-training sets are fed into the SteelIDNet model in batches for iterative training. Classification cross entropy is used as the loss function for parameter update, and the Adam optimization algorithm is used to accelerate training. The validation set is input into the trained SteelIDNet model for prediction and verification of the prediction accuracy. If the verification fails, iterative training continues. If the verification passes, the final SteelIDNet model is obtained.

7. The steel microstructure identification method based on deep learning drive according to claim 6, characterized in that: The loss function is expressed as: Among them, y i represents the true category label corresponding to the i-th steel microstructure image, is the predicted probability of the image classification by the i-th representation model, and size is the total number of training samples; The prediction accuracy calculation formula is: Among them, total_train_correct_sum is the number of images predicted correctly after each training, and total_train_sum is the total number of images in a single training.

8. The steel microstructure identification method based on deep learning drive according to claim 1, characterized in that: The final SteelIDNet model is deployed on the mobile terminal to perform steel microstructure identification, specifically: The final SteelIDNet model was lightweighted using TensorFlow Lite, and developed on mobile devices using Android Studio, resulting in a steel microstructure identification application that included the final SteelIDNet model. Install the steel microstructure identification application on the mobile device and use the mobile device to capture the steel microstructure image in real time; The steel microstructure recognition application adjusts the image data format to the input format of the final SteelIDNet model, loads the final SteelIDNet model and recognizes the input image; Update the recognition results to the new UI and display the predicted category or confidence information of the image.

9. A steel microstructure recognition system driven by deep learning, characterized by: The steel microstructure identification method based on deep learning as described in any one of claims 1 to 8 comprises a data processing module, a model building module, a model training module and a deployment identification module; The data processing module is used to obtain steel microstructure images, perform data enhancement and division, and obtain a training set and a validation set; The model building module is used to build a SteelIDNet model using a transfer learning method based on the MobileNetV2 pre-training model; The model training module is used to train the SteelIDNet model using the training set, and to verify and evaluate the trained SteelIDNet model using the verification set to obtain the final SteelIDNet model; The deployment and identification module is used to deploy the final SteelIDNet model on the mobile terminal to perform steel microstructure identification.

10. A computer-readable storage medium storing a program, characterized in that: When the program is executed by a processor, the steel microstructure identification method based on deep learning driving as described in any one of claims 1 to 8 is implemented.

Images