Sem image automatic classification method based on improved ConvNeXt network model and electronic device

Abstract

This invention provides an automatic SEM image classification method and electronic device based on an improved ConvNeXt network model. The method involves acquiring a target image using a scanning electron microscope (SEM); the target image is a microscopic electron microscope image; the target image is input into an improved ConvNeXt network model to obtain the probability of the target image belonging to each image category; the ConvNeXt network model includes a channel attention mechanism (ECA) module and a global attention mechanism (GAM) module; the image category corresponding to the highest probability is determined as the target image category. Compared to existing technologies, by introducing the channel attention module (ECA) and the global attention mechanism (GAM), the interaction information between different channels can be effectively captured during training, and the network pays more attention to important information and suppresses interference from irrelevant information during learning, thus improving the image classification ability.

Description

Technical Field

[0001] This invention relates to the field of image classification technology, and more specifically, to an automatic SEM image classification method and electronic device based on an improved ConvNeXt network model. Background Technology

[0002] With the increasing maturity of science and technology, more and more polymer materials are being integrated into daily life. However, most polymer products are flammable, and the combustion process produces toxic gases, posing a threat to the environment and public health. Therefore, research on polymer flame-retardant materials has become a hot topic. Based on their flame-retardant composition, polymer flame-retardant materials are divided into two main categories: halogenated flame retardants and halogen-free flame retardants.

[0003] Currently, various methods and instruments have been used to extensively study the performance of IFR coatings. Thermogravimetric analysis (TGA) records the combustion properties of IFR coatings, Fourier transform infrared spectroscopy (FTIR) studies the chemical structure and composition of the char layer, X-ray power spectroscopy (XPS) is used for chemical state analysis, and scanning electron microscopy (SEM) observes the microstructure of the char layer. With the increasing maturity and widespread application of image processing technology, SEM images are also being converted into digital signals for further processing.

[0004] Currently, the identification of SEM images of halogen-free flame-retardant materials with different flame-retardant grades relies primarily on manual identification by experts in the field. Identifiers, supported by their professional knowledge and personal experience, observe the morphology of the substances in the images and analyze their composition. However, this method has significant drawbacks: it requires substantial investment in training sufficiently experienced experts, and the results of manual identification are susceptible to subjective influences. Summary of the Invention

[0005] In view of this, the purpose of this invention is to provide an application of deep learning to the field of SEM image classification of flame retardant materials. Based on the ConvNeXt network which incorporates attention mechanism and global attention mechanism, it effectively captures the interaction information of the channel, improves the model accuracy, and realizes the automatic detection of the flame retardant level of halogen-free flame retardant materials.

[0006] A first aspect of the present invention provides an automatic classification method for SEM images based on an improved ConvNeXt network model, the method comprising:

[0007] The target image was acquired by scanning electron microscopy (SEM); the target image was a microscopic electron microscope image.

[0008] The target image is input into an improved ConvNeXt network model to obtain the probability that the target image belongs to each image category; the ConvNeXt network model includes a channel attention mechanism (ECA) module and a global attention mechanism (GAM) module.

[0009] The image category corresponding to the highest probability is determined as the image category of the target image.

[0010] Preferably, the channel attention mechanism (ECA) module includes:

[0011] For the input H×W×C feature map, global average pooling is used to compress the spatial features in the spatial dimension to obtain the initial feature map; channel feature learning is achieved through 1×1 convolution to output a 1×1×C feature map.

[0012] The 1×1×C feature map is multiplied channel-by-channel with the original H×W×C input image to output a feature map with channel attention.

[0013] Preferably, the step of learning channel features through 1×1 convolution to output a 1×1×C feature map includes:

[0014] The compressed initial feature map is input into a multilayer perceptron (MLP), and information interaction between channels is achieved by using a one-dimensional convolution with a kernel size of k, generating channel weights w. The channel weights w are then input into the initial feature map, thereby adaptively weighting the internal features.

[0015] Where w=σ(C1D) k (y)); C1D represents one-dimensional convolution, and σ is the Sigmoid function.

[0016] Preferably, the Global Attention Mechanism (GAM) module includes a Channel Attention Submodule (CAM) and a Spatial Attention Submodule (SAM).

[0017] Among them, the Channel Attention Submodule (CAM) uses a three-dimensional arrangement to preserve three-dimensional information. First, the input image is transformed in dimension. Then, the feature map after dimension transformation is input into a multilayer perceptron (MLP) with a two-layer encoder-decoder with a reduction ratio r to amplify the cross-dimensional channel spatial relationship. Then, it is converted back to the original dimension and finally output as a Sigmoid.

[0018] The Spatial Attention (SAM) submodule uses two convolutional layers to fuse spatial information, giving full attention to spatial information. First, it reduces the number of channels by using a convolution with a kernel of 7 to reduce the amount of computation. Then, it increases the number of channels by using another convolution with a kernel of 7. Finally, it outputs the result via a sigmoid function.

[0019] Preferably, the method further includes a ConvNeXt network model training process; the training process includes a warm-up mechanism;

[0020] The preheating mechanism includes:

[0021] The model gradually adapts to the dataset at the start of training, reducing the risk of overfitting and gradient explosion.

[0022] After the model has been trained for a period of time, the learning rate is gradually reduced to refine the training process and improve the generalization performance of the model. Cosine annealing continuously reduces the initial learning rate until it reaches the preset minimum value, and then gradually increases the learning rate back to the maximum value, forming a periodically changing curve.

[0023] Preferably, the step of inputting the target image into an improved ConvNeXt network model to obtain the probability that the target image belongs to each image category includes:

[0024] The target image is input into an improved ConvNeXt network model for data processing; the improved ConvNeXt network model performs a first convolution on the target image, then passes it through four ConvNeXt blocks and combines downsampling operations to obtain the processed feature map; the processed feature map is then subjected to global average pooling, layer normalization, and fully connected layer operations.

[0025] The fully connected layer maps the number of channels to the number of classes to be classified, thus obtaining the probability that the input is classified as each image class.

[0026] Furthermore, a second aspect of the present invention provides an electronic device comprising: one or more processors, and a memory for storing one or more computer programs; characterized in that the computer programs are configured to be executed by the one or more processors, the programs comprising steps for performing the SEM image automatic classification method based on the improved ConvNeXt network model as described in the first aspect above.

[0027] Furthermore, a third aspect of the present invention provides a storage medium storing a computer program; the program is loaded and executed by a processor to implement the steps of the SEM image automatic classification method based on the improved ConvNeXt network model as described in the first aspect above.

[0028] In this invention, an image of charred slag formed by calcining flame-retardant materials is obtained using scanning electron microscopy (SEM). The target image is a microscopic SEM image. The target image is input into an improved ConvNeXt network model to obtain the probability of the target image belonging to each image category. The ConvNeXt network model includes a channel attention mechanism (ECA) module and a global attention mechanism (GAM) module. The image category corresponding to the highest probability is determined as the image category of the target image. Compared with existing technologies, by introducing the channel attention module (ECA), the interaction information between different channels can be effectively captured during training, and the dimensionality of the input feature map is avoided from being excessively reduced. Secondly, the addition of the global attention mechanism (GAM) makes the network pay more attention to important information and suppress the interference of irrelevant information during the learning process, amplifying the cross-dimensional global interaction, thereby improving the image classification ability. A warm-up mechanism is used in the early stage of model training, and the learning rate is decayed using a cosine function value during the process, which improves the training stability and convergence speed of the model, while reducing the risk of overfitting and gradient explosion. Attached Figure Description

[0029] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0030] Figure 1 This is a schematic diagram of the overall ConvNeXt network structure disclosed in an embodiment of the present invention;

[0031] Figure 2 This is a schematic diagram of the ConvNeXt Block network structure disclosed in an embodiment of the present invention;

[0032] Figure 3 This is a flowchart illustrating the automatic classification method for SEM images of flame-retardant materials disclosed in an embodiment of the present invention.

[0033] Figure 4 This is a schematic diagram of the overall structure of the Channel Attention Mechanism (ECA) module disclosed in an embodiment of the present invention;

[0034] Figure 5 This is a schematic diagram of the overall structure of the Global Attention Mechanism (GAM) module disclosed in an embodiment of the present invention;

[0035] Figure 6 This is a schematic diagram of the channel attention submodule structure in the Global Attention Mechanism (GAM) module disclosed in an embodiment of the present invention;

[0036] Figure 7This is a schematic diagram of the spatial attention submodule in the Global Attention Mechanism (GAM) module disclosed in an embodiment of the present invention;

[0037] Figure 8 This is a schematic diagram of the ConvNeXt Block network structure in the improved ConvNeXt network model that integrates ECA and GAM, as disclosed in an embodiment of the present invention. Detailed Implementation

[0038] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this application more comprehensive and complete, and to fully convey the concept of the exemplary embodiments to those skilled in the art.

[0039] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this application. However, those skilled in the art will recognize that the technical solutions of this application can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this application.

[0040] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0041] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0042] It should be noted that "multiple" as mentioned in this article refers to two or more.

[0043] When traditional convolutional neural networks (CNNs) handle more complex visual tasks, they face several challenges, such as a large number of parameters, high computational cost, and difficulty in capturing long-range dependencies. Furthermore, with the emergence and success of the Transformer in natural language processing, researchers have begun exploring the application of the Transformer's attention mechanism to computer vision, hoping to alleviate these problems.

[0044] ConvNeXt is based on the ideas of ResNeXt and Transformer, fusing the two to propose a new convolutional neural network architecture (see the paper "Liu Z, Mao H, Wu CY, et al. A convnet for the 2020s[C] / / Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition.2022:11976-11986."). The ConvNeXt network further improves classification accuracy through the layer structure of the Swin Transformer, downsampling methods, activation functions, data processing methods, inverse bottlenecks, and depthwise convolutions. The overall network structure is similar to ResNet, consisting of multiple residual blocks (ConvNeXt Blocks). The number of module stacks is changed from (3,4,6,3) to (3,3,9,3). Each residual block consists of three convolutional layers, including grouped convolutional layers, dilated convolutional layers, and residual connections. The overall structure is as follows: Figure 1 As shown, an image of size 224×224×3 (height×width×number of channels) becomes a feature map of 56×56×96 after the first convolution; then, after four ConvNeXt Blocks combined with a series of downsampling operations, the feature map becomes 7×7×768; finally, global average pooling, layer normalization and fully connected operations are performed.

[0045] Unlike traditional convolutional neural networks, ConvNeXt introduces two main mechanisms: group convolution and dilated convolution. In ConvNeXt, group convolution involves dividing the input and output channels into multiple groups during the convolution operation. Channels within each group share a single convolution kernel, thereby reducing the number of parameters and computational cost, while also improving the model's generalization ability. Dilated convolution involves inserting gaps within the convolution kernel during the convolution operation. After inserting gaps, the effective receptive field of the convolution kernel increases, while the output resolution decreases, but more local information is preserved.

[0046] The ConvNeXt Block network structure is as follows: Figure 2As shown, its input feature map is h×w×dim. The DepthwiseConvolution layer uses group convolution for convolution operations. The convolution kernel in each channel group only convolves with the corresponding input channel, thereby reducing the amount of computation and parameters. Layer Norm normalizes the convolution results, thereby accelerating the convergence speed and stability of the network. Layer Scale automatically learns the weights between different channels, thereby scaling the feature map, enhancing the importance of certain channels, and thus adjusting and optimizing the feature representation of the model. DropPath randomly discards some feature maps during training, increasing the generalization performance of the model.

[0047] The ConvNeXt activation layer uses Gaussian error linear units (GELUs) and adds them only to the ConvNeXt Block. The derivative of the GELU function is continuous, which makes it easier to propagate gradients when training deep neural networks. This avoids the problem of the derivative of the ReLU function being discontinuous at x=0, thereby reducing the gradient vanishing problem during training. Its approximate calculation formula is shown below:

[0048]

[0049] The implementation details of the technical solutions in the embodiments of this application are described in detail below:

[0050] The first aspect of this embodiment provides an automatic classification method for SEM images of flame-retardant materials. Figure 3 The diagram shown is a flowchart of the automatic classification method for SEM images of flame-retardant materials in this embodiment. The method includes:

[0051] The first aspect of this embodiment provides an automatic classification method for SEM images of flame-retardant materials, the method comprising:

[0052] S301, the target image is obtained by scanning with a scanning electron microscope (SEM); the target image is a microscopic electron microscope image.

[0053] Specifically, in this embodiment, the image of the charred slag formed by calcining the flame-retardant material can be obtained by scanning with a scanning electron microscope (SEM); the image of the charred slag is a microscopic electron microscope image.

[0054] S302, the target image is input into an improved ConvNeXt network model to obtain the probability that the target image belongs to each image category; the ConvNeXt network model includes a channel attention mechanism (ECA) module and a global attention mechanism (GAM) module.

[0055] Specifically, in this embodiment, the traditional ConvNeXt is improved to obtain an improved ConvNeXt network model. In the ConvNeXt network structure, there are four types of blocks based on the number of channels in the output feature map. Each type of block has a different number of iterations: the first type iterates 3 times, outputting a 96-dimensional feature map; the second type iterates 3 times, outputting a 192-dimensional feature map; the third type iterates 3 times, outputting a 384-dimensional feature map; and the fourth type iterates 3 times, outputting a 768-dimensional feature map. The feature map is downsampled before each block, and the final output size (W, H) of the block is halved, doubling the output dimension under the effect of convolution. In this case, adding weight values ​​to the channels seems very important.

[0056] The improved ConvNeXt network model includes a channel attention mechanism (ECA) module and a global attention mechanism (GAM) module.

[0057] Among them, ECA (Efficient Channel Attention) is a lightweight channel attention mechanism. ECA addresses the issue of the Squeeze-and-Excitation (SE) module ignoring the relationship between features within channels. It introduces a one-dimensional convolutional operation to learn the global correlation between channels and uses a sigmoid function to weight the features, ultimately generating an attention matrix. This matrix is ​​then applied to the input feature map, thereby enhancing the network's expressive power. The overall structure is as follows: Figure 4 As shown.

[0058] First, input an H×W×C feature map and compress the spatial features using global average pooling to obtain a 1×1×C feature map. Second, channel feature learning is achieved through 1×1 convolution, outputting a 1×1×C feature map. Finally, the 1×1×C feature map is multiplied channel-wise with the original H×W×C input image to output a feature map with channel attention.

[0059] ECA averages the input feature map and then compresses each channel into a scalar, which is then input into a multilayer perceptron (MLP). One-dimensional convolution with a kernel size of k is used to realize information interaction between channels and generate channel weights w. The calculation formula is as shown in (2). The learned channel weight coefficients are input into the feature map, and then the internal features are adaptively weighted.

[0060] w=σ(C1D k (y)) (2)

[0061] Where C1D represents one-dimensional convolution, and σ is the Sigmoid function.

[0062] The Global Attention Module (GAM) is an attention mechanism capable of capturing salient features across all three dimensions. It redesigns the channel attention and spatial attention submodules based on the Convolutional Block Attention Module (CBAM), improving the performance of deep neural networks by reducing information loss and enhancing the interaction of global features. The GAM attention mechanism consists of two modules: the Convolutional Block Attention Module (CAM) and the Spatial Attention Module (SAM). Channel attention learns the weights of different channels and uses these weights to perform multiple partitions of the channels. Spatial attention focuses on the positional information of the target in the image and selectively focuses on features in each space through weighted spatial feature calculation. Its overall structure is as follows: Figure 5 As shown.

[0063] The channel attention submodule uses a 3D arrangement to preserve 3D information. First, the input image undergoes dimensionality transformation. Then, the transformed feature map is input into a two-layer multilayer perceptron (MLP) with a reduction ratio r, which amplifies the cross-dimensional channel spatial relationships. It is then converted back to its original dimension and finally output using a sigmoid function. The channel attention submodule is as follows: Figure 6 As shown.

[0064] In the spatial attention submodule, two convolutional layers are used to fuse spatial information, giving full attention to spatial information. First, a convolution with a kernel of 7 is used to reduce the number of channels and decrease computational cost. Then, another convolution with a kernel of 7 is used to increase the number of channels, and finally, a sigmoid function is applied for output. Since max pooling reduces information and has a negative impact, the pooling operation is removed to preserve feature mapping. To prevent a significant increase in parameters, group convolutions with channel shuffling are used. The spatial attention submodule is as follows: Figure 7 As shown.

[0065] Regarding the location of the channel attention mechanism (ECA) module and the global attention mechanism (GAM) module in the improved ConvNeXt network model, this embodiment describes it as follows: Figure 2 In the ConvNeXt Block network structure shown, this embodiment inserts the corresponding ECA and GAM modules at different locations. See details... Figure 8This diagram illustrates the ConvNeXt Block network structure in the improved ConvNeXt network model that integrates ECA and GAM in this embodiment. Introducing both ECA and GAM modules improves the classification performance of the ConvNeXt network, and the simultaneous addition of both modules also positively impacts classification performance. The ECA module adaptively weights information between channels, enabling the model to better utilize global inter-channel information; while the GAM module weights global spatial location, allowing the model to better focus on important image regions. Combining both modules leverages their advantages in both channel and spatial dimensions to further enhance the model's perceptual capabilities.

[0066] In some embodiments, a warm-up mechanism for model training is also proposed.

[0067] Preferably, the method further includes a ConvNeXt network model training process; the training process includes a warm-up mechanism;

[0068] The warm-up mechanism includes: the model gradually adapts to the dataset when training begins to reduce the risk of overfitting and gradient explosion; after the model has been trained for a period of time, the learning rate is gradually reduced to refine the training process of the model, thereby improving the generalization performance of the model; cosine annealing continuously reduces the initial learning rate until it reaches a preset minimum value, and then gradually increases the learning rate back to the maximum value, forming a periodically changing curve.

[0069] Specifically, in this embodiment, the model's weights are randomly initialized at the start of training, and for most samples, there is a large error between the model's output and the target. If a large learning rate is used at the start of training, the model may skip local minima, causing the model to diverge or get trapped in undesirable local minima in the early stages of training.

[0070] The warm-up mechanism refers to using a small learning rate in the first few epochs of training and gradually increasing it to avoid model instability caused by large parameter updates at the beginning. In deep learning, training a model requires updating its parameters, which is achieved through gradients calculated using the backpropagation algorithm. At the start of training, because the model's weights are randomly initialized, the model's output is often quite random. If a large learning rate is used for parameter updates at this time, it may cause the model to become unstable or even fail to converge. Therefore, the warm-up mechanism allows the model to gradually adapt to the dataset at the beginning of training, reducing the risk of overfitting and gradient explosion.

[0071] After training the model for a period of time, the learning rate is gradually reduced to refine the training process, thereby improving the model's generalization performance. Cosine annealing continuously reduces the initial learning rate until it reaches a preset minimum value, and then gradually increases the learning rate back to its maximum value, forming a periodically changing curve. By periodically adjusting the learning rate while keeping the update direction unchanged, the model can better adapt to complex datasets and avoid getting trapped in local optima, thus improving its generalization ability and performance.

[0072] In some embodiments, inputting the target image into an improved ConvNeXt network model to obtain the probability that the target image belongs to each image category includes:

[0073] The target image is input into an improved ConvNeXt network model for data processing; the improved ConvNeXt network model performs a first convolution on the target image, then passes it through four ConvNeXt blocks and combines downsampling operations to obtain the processed feature map; the processed feature map is then subjected to global average pooling, layer normalization, and fully connected layer operations.

[0074] The fully connected layer maps the number of channels to the number of classes to be classified, thus obtaining the probability that the input is classified as each image class.

[0075] S303, determine the image category corresponding to the highest probability as the image category of the target image.

[0076] Specifically, in this embodiment, the image with the highest probability among the probabilities of each predicted image category is classified as the image category of the target image, and then output.

[0077] Compared to existing technologies, this embodiment introduces a channel attention module (ECA), which effectively captures interaction information between different channels during training and avoids excessive reduction of the dimensionality of the input feature map. Secondly, it incorporates a global attention mechanism (GAM), which enables the network to focus more on important information and suppress interference from irrelevant information during learning, amplifying the cross-dimensional global interaction and thus improving the image classification ability. A warm-up mechanism is used in the early stage of model training, and the learning rate is decayed using a cosine function value during the process, which improves the training stability and convergence speed of the model, while reducing the risk of overfitting and gradient explosion.

[0078] Furthermore, a second aspect of this embodiment provides an electronic device comprising: one or more processors, and a memory for storing one or more computer programs; characterized in that the computer programs are configured to be executed by the one or more processors, and the programs include steps for performing the SEM image automatic classification method based on the improved ConvNeXt network model as described in the first aspect above.

[0079] Furthermore, a third aspect of this embodiment provides a storage medium storing a computer program; the program is loaded and executed by a processor to implement the steps of the SEM image automatic classification method based on the improved ConvNeXt network model as described in the first aspect above.

[0080] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0081] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices or units, or may be electrical, mechanical or other forms of connection.

[0082] The units described as separate components may or may not be physically separate. As will be appreciated by those skilled in the art, the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0083] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0084] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or grid device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0085] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An SEM image automatic classification method based on an improved ConvNeXt network model, characterized in that, The method includes: The target image was acquired by scanning electron microscopy (SEM); the target image was a microscopic electron microscope image. The target image is input into an improved ConvNeXt network model to obtain the probability that the target image belongs to each image category; the ConvNeXt network model includes a channel attention mechanism (ECA) module and a global attention mechanism (GAM) module. The image category corresponding to the highest probability is determined as the image category of the target image; The Global Attention (GAM) mechanism module includes a Channel Attention (CAM) submodule and a Spatial Attention (SAM) submodule. The CAM submodule uses a three-dimensional arrangement to preserve three-dimensional information. First, it transforms the input image's dimensions. Then, it inputs the transformed feature map into a multilayer perceptron (MLP) with a two-layer encoder-decoder structure with a reduction ratio r, amplifying the cross-dimensional channel spatial relationships. It then transforms the image back to its original dimensions and finally outputs a sigmoid function. The SAM submodule uses two convolutional layers to fuse spatial information, paying close attention to spatial information. It first reduces the number of channels using a convolution with a kernel of 7 to decrease computation, then increases the number of channels using another convolution with a kernel of 7, and finally outputs a sigmoid function.

2. The SEM image automatic classification method based on the improved ConvNeXt network model according to claim 1, characterized in that, The channel attention mechanism (ECA) module includes: For the input H×W×C feature map, global average pooling is used to compress the spatial features in the spatial dimension to obtain the initial feature map; channel feature learning is achieved through 1×1 convolution to output a 1×1×C feature map. The 1×1×C feature map is multiplied channel-by-channel with the original H×W×C input image to output a feature map with channel attention.

3. The SEM image automatic classification method based on the improved ConvNeXt network model according to claim 2, characterized in that, The method of learning channel features through 1×1 convolution and outputting a 1×1×C feature map includes: The compressed initial feature map is input into a multilayer perceptron (MLP), and one-dimensional convolution with a kernel size of k is used to realize information interaction between channels to generate channel weights w. The channel weights w are then input into the initial feature map to adaptively weight the internal features. in, C1D represents one-dimensional convolution. This is the Sigmoid function.

4. The automatic SEM image classification method based on the improved ConvNeXt network model according to claim 3, characterized in that, The method also includes the ConvNeXt network model training process; The training process includes a warm-up mechanism; The preheating mechanism includes: The model gradually adapts to the dataset at the start of training, reducing the risk of overfitting and gradient explosion. After the model has been trained for a period of time, the learning rate is gradually reduced to refine the training process and improve the generalization performance of the model. Cosine annealing continuously reduces the initial learning rate until it reaches the preset minimum value, and then gradually increases the learning rate back to the maximum value, forming a periodically changing curve.

5. The automatic SEM image classification method based on the improved ConvNeXt network model according to claim 4, characterized in that, The step of inputting the target image into an improved ConvNeXt network model to obtain the probability that the target image belongs to each image category includes: The target image is input into an improved ConvNeXt network model for data processing; the improved ConvNeXt network model performs a first convolution on the target image, then passes it through four ConvNeXt blocks and combines downsampling operations to obtain the processed feature map; the processed feature map is then subjected to global average pooling, layer normalization, and fully connected layer operations. The fully connected layer maps the number of channels to the number of classes to be classified, thus obtaining the probability that the input is classified as each image class.

6. An electronic device, the electronic device comprising: One or more processors, a memory for storing one or more computer programs; characterized in that the computer programs are configured to be executed by the one or more processors, the programs including steps for performing the SEM image automatic classification method based on the improved ConvNeXt network model as described in any one of claims 1-5.

7. A storage medium storing a computer program; the program being loaded and executed by a processor to implement the steps of the automatic SEM image classification method based on the improved ConvNeXt network model as described in any one of claims 1-5.

Images