A method and model for recognizing microscopic image information of residual aluminum alloys incorporating CBAM
By introducing residual modules and CBAM attention mechanisms into the UNet network, the ResUNet_CBAM model is constructed, which solves the problems of low efficiency and insufficient accuracy in traditional aluminum alloy microstructure analysis and achieves efficient and accurate aluminum alloy microstructure segmentation.
Patent Information
- Application Number
- CN202411115668.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-14
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2044-08-14
AI Technical Summary
Traditional methods for analyzing the microstructure of aluminum alloys rely on human experience, resulting in low efficiency and inaccurate results. UNet networks have high computational complexity, making it difficult to maintain high accuracy in cases of data imbalance and large-size image processing.
By incorporating residual modules and the CBAM attention mechanism into the UNet network, the ResUNet_CBAM model is constructed. The semantic segmentation accuracy of aluminum alloy microstructure is improved through the encoder-decoder structure and attention mechanism.
It improves the accuracy and efficiency of image segmentation of aluminum alloy microstructure, reduces computational complexity, enhances the model's ability to focus on specific segmentation tasks, and alleviates the gradient vanishing problem.
Smart Images

Figure CN119068484B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of alloy microscopic image processing technology, and particularly relates to a method and model for recognizing residual aluminum alloy microscopic image information incorporating CBAM. Background Technology
[0002] Due to their high specific strength, good corrosion resistance, excellent plasticity and toughness, and good machinability, aluminum alloys are increasingly widely used in engineering construction, aerospace, and transportation. In aerospace applications, aluminum alloys can reduce aircraft weight, thereby reducing fuel consumption, increasing payload, improving performance, and reducing emissions. To ensure the reliability and stability of aircraft operation, aluminum alloys must meet certain mechanical property requirements. The microstructure of metallic materials is inextricably linked to their macroscopic mechanical properties. Therefore, how to analyze metallographic structures has become one of the important research issues for understanding and modifying the intrinsic factors of materials. Traditional metallographic analysis mostly uses qualitative and semi-quantitative methods, which have many shortcomings. They largely rely on human experience to judge the content and properties, resulting in low efficiency and significant influence from subjective factors, leading to inaccurate analysis results. However, in recent years, deep learning has made progress in image segmentation, especially the UNet network. The UNet network has shown excellent performance in image segmentation, effectively solving the problems of traditional microstructure analysis and improving image segmentation efficiency and accuracy. The UNet network requires high computational resources and has excessive redundancy, which can lead to slow network training and the inability to achieve both classification accuracy and localization precision. Furthermore, it may present some challenges in handling imbalanced data and large-size image processing.
[0003] Integrating attention mechanisms into neural networks can quickly filter out valuable information, improving the model's ability to focus on and process input data. The CBAM attention mechanism, in particular, combines channel and spatial attention and is applicable to more comprehensive feature representations. While CBAM has achieved good results, it increases algorithm complexity and computational cost. Therefore, how to more effectively utilize attention mechanisms to improve the accuracy of aluminum alloy semantic segmentation while reducing computational complexity and avoiding model degradation remains a hot research topic. Summary of the Invention
[0004] The purpose of this invention is to provide a method and model for recognizing microscopic image information of aluminum alloys incorporating CBAM. This invention integrates residual modules, channel and spatial attention mechanisms into the UNet network, thereby enhancing the detailed features of the aluminum alloy microstructure and improving the accuracy of semantic segmentation of the aluminum alloy microstructure. To achieve the above objective, the technical solution adopted by this invention is as follows:
[0005] This invention proposes a method for recognizing microscopic image information of residual aluminum alloys incorporating CBAM, characterized in that the image information extraction includes the following steps:
[0006] Step 1: Data acquisition and preprocessing. Microstructure images of aluminum alloys are acquired using a metallographic microscope, and the acquired microstructure images are converted to grayscale to form a dataset of aluminum alloy microstructure images.
[0007] Step 2: Construct an aluminum alloy microstructure segmentation model. Based on the deep learning framework PyTorch, a network is built to analyze the aluminum alloy microstructure. A residual network model, ResUNet, is constructed using the encoder-decoder structure in the UNet network. An attention mechanism model, CBAM, is added to the decoder part of the ResUNet model to construct the aluminum alloy microstructure segmentation model. The aluminum alloy microstructure segmentation model is then trained to obtain the trained aluminum alloy microstructure segmentation model, ResUNet_CBAM.
[0008] Step 3: Aluminum alloy microstructure image segmentation. Input the aluminum alloy microstructure image dataset into the trained aluminum alloy microstructure segmentation model ResUNet_CBAM for recognition processing to obtain the aluminum alloy microstructure segmentation results.
[0009] A further preferred embodiment of the above method involves inputting the microstructure image of the aluminum alloy to be segmented into the trained aluminum alloy microstructure segmentation model ResUNet_CBAM for recognition processing, including the following steps:
[0010] Step 21: Set up the PyTorch neural network training environment and set the image input training parameters;
[0011] Step 22: The aluminum alloy microstructure image dataset is preprocessed by grayscale conversion, cropping and rotation using the weighted average method. Then, the preprocessed aluminum alloy microstructure images are standardized to a size of 256×256 to meet the input requirements of the residual UNet network.
[0012] Step 23: Input the aluminum alloy microstructure image dataset into the trained aluminum alloy microstructure segmentation model ResUNet_CBAM for encoding and decoding recognition processing, and output the segmentation result image.
[0013] A further preferred embodiment of the above method involves inputting the aluminum alloy microstructure image dataset into the trained aluminum alloy microstructure segmentation model ResUNet_CBAM for sequential encoding and decoding recognition processing, including the following steps:
[0014] Step 24, during encoding and recognition, the number of image channels is gradually increased to 1024 dimensions in the encoder through convolution, and then the size of the aluminum alloy microstructure image is reduced to 1 / 16 of the original size through pooling layer. Features are gradually extracted from the input aluminum alloy microstructure image to obtain the encoding and segmentation effect feature map.
[0015] Step 25: During decoding and recognition, the encoder segmentation effect feature map from the lower layer is upsampled through the upsampling layer and convolutional layer in the decoder. The attention mechanism model CBAM first performs channel attention mechanism to reduce the number of channels of the input encoder segmentation effect feature map, and then performs spatial attention mechanism operation. The encoder segmentation effect feature map copy, the feature map copy after spatial attention mechanism operation, and the feature map copy after channel attention mechanism operation are concatenated radially along a single channel to generate encoder segmentation effect features of different scales, thereby realizing multi-scale feature fusion.
[0016] Step 26: Finally, a 1×1 convolution and a Sigmoid activation layer are used to change the number of channels in the image to achieve a multi-scale feature fusion encoding segmentation effect. The segmentation result is then output, and the segmentation result is multiplied by the input aluminum alloy microstructure image. The resulting output image is then added to the image output by the skipped link layer. The image obtained after addition is then used sequentially with the Sigmoid function for prediction calculation and cross-entropy calculation for the final classification loss to obtain the segmentation performance of the segmented image.
[0017] In a further preferred embodiment of the above method, the residual network model ResUNet inserts three residual structures ResBlock into both the encoder and decoder of the residual UNet network. Before each residual structure ResBlock in the decoder of the residual network model ResUNet, the features from the lower layer are upsampled, and after each upsampling, an attention mechanism model CBAM is introduced.
[0018] In a further preferred embodiment of the above method, each ResBlock residual structure includes a convolutional layer CONV with a continuous kernel size of 3×3, a ReLU activation function layer, and a batch normalization layer BN. The attention mechanism model CBAM is introduced into the decoder part of the ResUNet residual network model after the upsampling module Upsample in the decoder upsampling stage and before the batch normalization layer BN. CBAM is used to process the input encoded segmentation feature map and then concatenate the features from the encoder.
[0019] In a further preferred embodiment of the above method, the attention mechanism model CBAM includes a channel attention module (CAM) and a spatial attention module (SAM). The channel attention module CAM and the spatial attention module SAM are sequentially inserted after the convolution of the residual UNet network model. The aluminum alloy microstructure segmentation model ResUNet_CBAM, formed by inserting the channel attention module CAM and the spatial attention module SAM into the decoder part of the residual network model ResUNet after upsampling, processes the encoded segmentation effect feature map. The specific steps are as follows:
[0020] The feature map of the upsampled output encoding segmentation effect first enters the Max Pool and Avg Pool of the Channel Attention Module (CAM), and then enters the Multilayer Perceptron Module (MPM). In the MPM module, the number of channels is first compressed to 1 / n of the original number, and then expanded to the original number of channels.
[0021] In the Spatial Attention Module (SAM), two single-channel feature maps are first obtained through a Max Pool and an Avg Pool layer. Then, a tensor concatenation layer (Concat) is used to concatenate the two single-channel feature maps. After that, a 7×7 convolution is used to transform them into single-channel feature maps. The output result is multiplied with the original image to obtain the output image of the attention mechanism model (CBAM). Then, this output image is added to the image of the skipped link layer and then fed into the next residual structure (ResBlock) for further processing.
[0022] In a further preferred embodiment of the above method, a skip link layer is added between the encoder and the decoder, which concatenates the decoded feature map from the decoding stage and the encoded feature map obtained in the same layer from the encoding stage, thereby achieving multi-scale feature fusion.
[0023] In a further preferred embodiment of the above method, the prediction calculation using the Sigmoid function and the final classification loss calculated using cross-entropy H respectively satisfy the following:
[0024]
[0025] y is considered as the model's original prediction of the sample image, and y is the true label value of the sample image, with a value of 0 or 1.
[0026] In a further preferred embodiment of the above method, setting the image input training parameters includes setting the image input size to 256×256 and the initial learning rate to 5×10. -5 The minimum learning rate is 1×10. -5 The model training batch size is 4, and the number of training iterations is 100,000.
[0027] According to another aspect of this application, the present invention provides a residual aluminum alloy microscopic image information recognition model incorporating CBAM. The recognition model includes an encoder and a decoder. Several layers of residual structures ResBlock distributed from top to bottom are respectively arranged between the input and output of the encoder and between the input and output of the decoder. The residual structures ResBlock in the encoder and the residual structures ResBlock in the decoder are mirror-symmetrically distributed. An attention mechanism model CBAM is added to the input end of each layer of residual structure ResBlock in the decoder. The input end of each residual structure ResBlock in the decoder is connected to the output end of the corresponding layer of residual structure ResBlock in the encoder through a skip link layer. The input image and output image of each attention mechanism model CBAM in the decoder are fed into the upper layer of residual structure ResBlock through multiplication.
[0028] In a further preferred embodiment of the above method, the attention mechanism model CBAM includes a channel attention module CAM and a spatial attention module SAM, which are sequentially inserted after the convolution of each residual structure ResBlock within the decoder.
[0029] Because the present invention adopts the above-described technical solution, the present invention has the following significant effects:
[0030] (1) Compared with the residual UNet model without CBAM, the residual UNet aluminum alloy microscopic image information extraction algorithm with CBAM has high segmentation accuracy and makes the network more focused on features related to specific segmentation tasks. However, at present, scholars at home and abroad have not inserted the CBAM attention mechanism into the residual UNet network, while the residual UNet aluminum alloy microscopic image information extraction algorithm with CBAM in this invention.
[0031] (2) This invention designs a residual UNet algorithm for extracting microscopic image information of aluminum alloys by incorporating CBAM. Based on the residual UNet network incorporating CBAM, this network mainly utilizes the CBAM attention mechanism to suppress irrelevant background features and improve the expressive power, discriminative power and localization power of ResUnet.
[0032] (3) This invention adds residual connections to UNet. Each module in the UNet encoder and decoder contains multiple residual connections. In each residual block, a shortcut is introduced to realize cross-layer connections, which can better extract low-frequency information, alleviate the problem of missing semantic information and gradient vanishing problem, thereby improving the accuracy of image segmentation. Attached Figure Description
[0033] Figure 1This is a schematic diagram of the recognition process structure of a residual aluminum alloy microscopic image information recognition method incorporating CBAM according to the present invention;
[0034] Figure 2 This is a schematic diagram of the model structure of a residual aluminum alloy microscopic image information recognition model incorporating CBAM according to the present invention.
[0035] Figure 3 This is a comparison diagram of the segmentation effect of the present invention. Detailed Implementation
[0036] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and preferred embodiments. However, it should be noted that many details listed in the specification are merely to provide the reader with a thorough understanding of one or more aspects of the present invention, and these aspects of the invention can be implemented even without these specific details.
[0037] Combination Figure 1 This invention proposes a method for recognizing microscopic image information of residual aluminum alloys incorporating CBAM, characterized in that the image information extraction includes the following steps:
[0038] Step 1: Data acquisition and preprocessing. Microstructure images of aluminum alloys are acquired using a metallographic microscope, and the acquired microstructure images are converted to grayscale to form a dataset of aluminum alloy microstructure images.
[0039] Step 2: Construct an aluminum alloy microstructure segmentation model. Based on the deep learning framework PyTorch, a network is built to analyze the aluminum alloy microstructure. A residual network model, ResUNet, is constructed using the encoder-decoder structure in the UNet network. An attention mechanism model, CBAM, is added to the decoder part of the ResUNet model to construct the aluminum alloy microstructure segmentation model. The aluminum alloy microstructure segmentation model is then trained to obtain the trained aluminum alloy microstructure segmentation model, ResUNet_CBAM.
[0040] Step 3: Aluminum alloy microstructure image segmentation. Input the aluminum alloy microstructure image dataset into the trained aluminum alloy microstructure segmentation model ResUNet_CBAM for recognition processing to obtain the aluminum alloy microstructure segmentation results.
[0041] In this invention, the ResUNet residual network model inserts three ResBlock structures into both the encoder and decoder of the ResUNet network. Before each ResBlock in the decoder of the ResUNet residual network model, features from lower layers are upsampled, and an attention mechanism model (CBAM) is introduced after each upsampling. In this invention, each ResBlock residual structure includes a 3×3 convolutional layer (CONV), a ReLU activation function layer, and a batch normalization layer (BN). The CBAM attention mechanism model is introduced into the decoder part of the ResUNet residual network model after the upsampling module (Upsample) in the decoder upsampling stage. Furthermore, an attention mechanism model CBAM is introduced before the batch normalization layer (BN) to process the input encoded segmentation feature map before concatenating the features from the encoder. A skip link layer (Bridge) is added between the encoder and decoder to concatenate the decoded feature map from the decoding stage and the encoded feature map obtained in the same layer from the encoding stage, thereby achieving multi-scale feature fusion. The decoder parses the encoded data into the original data form. It receives the encoded data stream and restores it to the original audio, video, or image data for playback or further processing. The decoder decodes the context vector generated by the encoder into an output sequence. The initial state of the decoder is usually initialized by the final state of the encoder to ensure information transmission. At each time step, the decoder generates the output of the current time step based on the output and hidden state of the previous time step, and gradually constructs the entire output sequence. The decoder is allowed to dynamically focus on different parts of the input sequence when generating each image, thereby improving the performance and interpretability of the model. The attention mechanism model CBAM includes a channel attention module (CAM), a spatial attention module (SAM), an input module, and an output module. The channel attention module CAM and the spatial attention module SAM are inserted sequentially after the convolution of the residual UNet network model.
[0042] In this invention, the microstructure image of the aluminum alloy to be segmented is input into a trained aluminum alloy microstructure segmentation model for recognition processing, including the following steps:
[0043] Step 21: Set up the PyTorch neural network training environment and configure the image input training parameters. This includes setting the image input size to 256×256 (resolution) and the initial learning rate to 5×10⁻⁶. -5 The minimum learning rate is 1×10. -5 The model training batch size is 4, and the number of training iterations is 100,000.
[0044] Step 22: The segmentation result image is preprocessed by grayscale conversion, cropping and rotation using the weighted average method. The preprocessed segmentation result image is then standardized to a size (resolution) of 256×256 to meet the input requirements of the residual UNet network. The grayscale processing is performed by weighted average of the RGB three components to obtain the component values of the R, G and B channels.
[0045] Step 23: Input the aluminum alloy microstructure image dataset into the trained aluminum alloy microstructure segmentation model ResUNet_CBAM for encoding and decoding processing, and output the segmentation result image. The process of inputting the aluminum alloy microstructure image dataset into the trained aluminum alloy microstructure segmentation model ResUNet_CBAM for encoding and decoding processing includes the following steps:
[0046] Step 24, during encoding and recognition, the encoder progressively increases the number of image channels to 1024 dimensions through convolution, and then reduces the size of the aluminum alloy microstructure image to 1 / 16 of its original size through pooling layers. Features are then extracted progressively from the input aluminum alloy microstructure image to obtain the encoded segmentation feature map. In the ResUNet residual network model, three ResBlock residual structures are inserted into both the encoder and decoder of the UNet network. In the ResBlock residual structure, image downsampling is no longer performed; instead, the stride of the first convolutional layer in the convolutional module is set to 2. In the encoder, the number of image channels is progressively increased to 1024 dimensions through convolution, and the feature map size is reduced to 1 / 16 of its original size through pooling layers. Features are then extracted progressively from the input segmentation image, and the dimensionality of the feature map is adjusted using reshape. A skip link is added between the encoder and decoder to concatenate the feature maps from the decoding stage and the feature information obtained in the same layer from the encoding stage, thereby achieving multi-scale feature fusion.
[0047] Step 25: During decoding and recognition, the encoder segmentation effect feature map from the lower layer is upsampled in the decoder through upsampling layer and convolutional layer. The attention mechanism model CBAM first performs channel attention mechanism to reduce the number of channels of the input encoder segmentation effect feature map, and then performs spatial attention mechanism operation. The encoder segmentation effect feature map copy obtained by encoder processing, the feature map copy after spatial attention mechanism operation, and the feature map copy after channel attention mechanism operation are concatenated radially along a single channel to generate encoder segmentation effect features of different scales, thereby realizing multi-scale feature fusion.
[0048] The decoder contains four upsampling layers. Through upsampling and convolution, it reduces the number of feature map channels and enlarges the image size, effectively fusing features to ultimately restore an output image with the same resolution as the input image. The decoder also uses three ResBlock residual structures. However, unlike the encoder, before each ResBlock, features from the lower layer are upsampled. After each upsampling, a CBAM attention mechanism is added. Then, features from the encoder are concatenated. After passing through the final decoder layer, a 1×1 convolution and a sigmoid activation layer are used to change the number of image channels.
[0049] Step 26: Finally, the image is output after passing through the sigmoid activation function and convolution. Specifically, the sigmoid activation layer changes the number of channels in the image, and a 1×1 convolution is used to achieve multi-scale feature fusion and coded segmentation. The segmentation result is then output, multiplied by the input aluminum alloy microstructure image. The resulting output image is added to the image output from the skipped link layer. The image obtained after addition is then used sequentially for prediction calculation using the sigmoid function and for calculating the final classification loss using cross-entropy H, thus obtaining the segmentation performance of the segmented image. The prediction calculation using the sigmoid function and the final classification loss calculated using cross-entropy H satisfy the following conditions:
[0050]
[0051] When dealing with classification problems, cross-entropy is often used in conjunction with the sigmoid function, assuming y is considered as the true label of the sample (0 or 1). It is considered as the model's original prediction of the sample.
[0052] The Sigmoid function is used to process the output image, and then cross-entropy is used to calculate the final classification loss. The main role of the loss function is to supervise the model's learning process. First, the difference between the model's predicted data and the labeled data is calculated, which is the loss value. After calculating the loss value, the model updates its parameters through backpropagation to reduce the difference between the true and predicted values, making the model's predicted values closer to the true values, and enabling the model to better fit the data. The cross-entropy loss function is used to measure the amount of information; cross-entropy mainly measures the difference between two probability distributions. In classification tasks, it measures the difference distribution between the predicted results and the true labels. The smaller the cross-entropy value, the better the model's prediction performance.
[0053] In this invention, the aluminum alloy microstructure segmentation model ResUNet_CBAM, formed by inserting channel attention module CAM and spatial attention module SAM into the decoder part of the residual network model ResUNet after upsampling, processes the feature map of the encoded segmentation effect. The specific steps are as follows:
[0054] Step 31: The feature map of the upsampled output encoding segmentation effect first enters the Max Pool and Avg Pool of the Channel Attention Module (CAM), and then enters the Multilayer Perceptron Module (MPM). In the MPM module, the number of channels is first compressed to 1 / n of the original number, and then expanded to the original number of channels.
[0055] Step 32: Add the outputs of the Max Pool and Avg Pool element by element, and then perform prediction calculations on the sum to obtain the output of the Channel Attention Module (CAM). Multiply the output of the CAM with the original segmentation result and then send it to the Spatial Attention Module (SAM) to generate feature maps of different scales.
[0056] Step 33: In the Spatial Attention Module (SAM), two single-channel feature maps are first obtained through a Max Pool layer and an AvgPool layer. Then, the two single-channel feature maps are concatenated using a tensor concatenation layer (Concat). After that, a 7×7 convolution is used to transform the feature maps into single-channel feature maps. The output result is multiplied with the original image (the image input to the input side of the Spatial Attention Module (SAM)) to obtain the output image of the attention mechanism model (CBAM). Then, this output image is added to the image of the skipped link layer and then fed into the next residual structure (ResBlock) to enter the next residual structure.
[0057] In this invention, the aluminum alloy microstructure segmentation model ResUNet_CBAM is achieved by adding an attention mechanism model CBAM after the upsampling part. These attention mechanism models CBAM are responsible for feature extraction. The image output from the upsampling module (Upsample) first enters the channel attention module (CAM) for max pooling and average pooling, then enters the multilayer perceptron module (MLP). In the MLP, the number of channels is compressed to 1 / n of the original number and then expanded back to the original number. Next, the outputs of max pooling and average pooling are element-wise summed, and the output of the channel attention module CAM is calculated using a sigmoid activation function. This output is then multiplied by the original encoded image (the image input to the channel attention module CAM) and enters the spatial attention module (SAM). In the spatial attention module (SAM), max pooling and average pooling are first performed to obtain... Two single-channel feature maps are concatenated using a Concat tensor, then a 7×7 convolution is applied to transform them into a single-channel feature map. Finally, the output is multiplied by the original image (the image input to the spatial attention module SAM at this layer) to obtain the output image of the attention mechanism model CBAM. In this invention, the channel attention mechanism compresses the feature map by using global average pooling and global max pooling to convert a size of (C,H,W) to (C,1,1), achieving global contextual information fusion. This results in a feature map with contextual information at different scales. This feature map is then processed through a convolutional layer to generate spatial attention weights. The ReLU activation function is applied to the generated spatial attention weights, limiting the weights to between 0 and 1. The spatial attention module weights the feature maps at different locations, strengthening important locations and suppressing unimportant ones. Then, the results of global max pooling and global average pooling are processed by a multilayer perceptron (MLP). The MLP consists of two fully connected layers with ReLU activation in between. The two layers are summed and then activated using the sigmoid function. Finally, the reshaped weights are multiplied with the original feature maps using Python's broadcasting mechanism to obtain feature maps with different weights. The spatial attention mechanism then performs max pooling and average pooling on the channel dimension based on the channel attention mechanism's results. This transforms the feature maps from (C, H, W) to (1, H, W), achieving the effect of fusing information from different channels. The max pooling and average pooling results are then superimposed on the channel dimension using `torch.cat()`. Next, the superimposed results from the two channels are convolved, with an output channel of 1 and a kernel size of 7. The output is then activated using the sigmoid function. Finally, the weights are multiplied with the original feature maps using Python's broadcasting mechanism to obtain feature maps with different weights.The attention mechanism model CBAM uses the reshape function to transform a specified matrix into a matrix of a specific dimension while maintaining the total number of elements in the matrix. This operation allows for the reshaping of the number of rows, columns, or dimensions of the matrix to adapt to specific computational needs, thereby adjusting the dimension of the feature map for subsequent attention mechanism calculations or network layer connections. This output image is then added to the image with skipped links before entering the next residual structure. Further deep network structures are built through BN layers, ReLU layers, and 3×3 convolutional layers without encountering gradient vanishing or exploding problems. Residual units simplify network training, and skipped links within residual units and between low-level and high-level layers help efficiently transfer low-level features to higher levels, thus improving the model's expressive power.
[0058] In this invention, combined with Figure 1 and Figure 2This invention provides a microscopic image information recognition model for aluminum alloys. The recognition model includes an encoder and a decoder. Several layers of residual structures (ResBlocks) distributed from top to bottom are arranged between the input and output of the encoder and between the input and output of the decoder, respectively. The residual structures (ResBlocks) in the encoder and decoder are mirror-symmetrically distributed. An attention mechanism model (CBAM) is added to the input of each layer of residual structures (ResBlocks) in the decoder. The input of each residual structure (ResBlock) in the decoder is connected to the output of the corresponding layer of residual structures (ResBlocks) in the encoder through a skip link layer. The input and output of each attention mechanism model (CBAM) in the decoder are fed into the previous layer of residual structures (ResBlocks) through multiplication. The attention mechanism model (CBAM) includes a channel attention module (CAM) and a spatial attention module (SAM). The channel attention module (CAM) and the spatial attention module (SAM) are sequentially inserted after the convolution of each residual structure (ResBlock) in the decoder. The ResBlock model consists of a Residual Network (ResUNet) and a Sequential Annotation Model (SAM). Each ResBlock includes a 3×3 convolutional layer (CONV), a ReLU activation function layer, and a batch normalization layer (BN). The ResUNet decoder incorporates a Concentration Mechanism (CBAM), which is introduced after the Upsample module in the decoder's upsampling stage and before the BN. CBAM processes the input encoded segmentation feature map and then concatenates the features from the encoder. A Bridge layer is added between the encoder and decoder to concatenate the decoded feature map from the decoding stage and the encoded feature map obtained in the same layer during the encoding stage, thus achieving multi-scale feature fusion. The decoder parses the encoded data into its original form. It receives the encoded data stream and restores it to the original audio, video, or image data for playback or further processing. The decoder decodes the context vector generated by the encoder into an output sequence. The decoder's initial state is usually initialized by the encoder's final state to ensure information transmission. In this invention, three layers of residual structures ResBlock distributed from top to bottom are respectively set between the input and output of the encoder and between the input and output of the decoder. The preprocessed aluminum alloy microstructure image dataset is input, and the feature map copy obtained after quick connection and identity mapping in the first residual structure ResBlock of the encoder is then connected to the feature map copy processed by the spatial attention module and the feature map copy processed by the channel attention module along the single channel radial splice and input to the residual structure ResBlock of the decoder. Similarly, the feature maps processed by the other two residual structures ResBlock of the encoder are all connected to the symmetrical residual structure ResBlock in the decoder through the link path.At each level between the encoder and decoder, skip link layers are used to preserve and pass feature information from higher levels. This allows lower-level feature map information to be directly passed to higher levels, helping the decoder to better recover details and edge information. In this way, the decoder can simultaneously utilize low-level features from the encoder and its own high-level features for reconstruction; the data processed by the encoder is then input into the decoder through the link bridge, and finally the result is output.
[0059] This invention constructs an aluminum alloy microstructure segmentation model by adding a CBAM attention mechanism model to the decoder part of the ResUNet residual network model. This model effectively improves the accuracy of image segmentation for aluminum alloy microstructure image recognition. Table 1 shows the segmentation and recognition performance of the ResUNet residual network model and the ResUNet_CBAM aluminum alloy microstructure segmentation model.
[0060] Table 1. Experimental data on recognition comparison effect
[0061]
[0062] This invention constructs an aluminum alloy microstructure segmentation model by adding a CBAM attention mechanism model to the decoder part of the ResUNet residual network model. This model effectively improves the accuracy of image segmentation for aluminum alloy microstructure image recognition. Table 1 shows the segmentation and recognition performance of the ResUNet residual network model and the ResUNet_CBAM aluminum alloy microstructure segmentation model.
[0063] The table above summarizes the segmentation performance of ResUNet and ResUNet_CBAM. As can be seen from the table, the ResUNet_CBAM constructed in this patent achieves improved segmentation performance compared to ResUNet, with a 1.69% increase in IoU and a 0.015 increase in F1 score. In terms of model size, ResUNet_CBAM only increases the original model size by 0.18M. The performance evaluation metrics are detailed below:
[0064] Accuracy (Acc): The ratio of correctly predicted positive metallographic samples and negative background samples to all samples, representing the proportion of correctly classified pixels. The formula is as follows:
[0065]
[0066] Intersection over Union (IoU): Calculates the proportion of overlap between the intersection and union of two sets, used to measure the accuracy of metallographic structure identification. The formula is as follows:
[0067]
[0068] F1-Score: The harmonic mean of precision and recall, calculated using the following formula:
[0069]
[0070] True Positives (TP): TP refers to the region where the metallographic structure is correctly identified as metallographic structure during the identification process. True Negatives (TN): TN refers to the region where the background is correctly identified as background during the identification process. False Positives (FP): FP refers to the region where a negative sample is identified as a positive sample during the identification process, indicating that the background is detected as metallographic structure, i.e., a false detection. False Negatives (FN): FN refers to the region where a positive sample is identified as a negative sample during the identification process, indicating that the metallographic structure is identified as background, i.e., a false negative. To ensure the objectivity of the experimental results, all comparative experiments in this chapter were conducted on the same workstation. Figure 3 The segmentation results were captured using a super depth-of-field microscope. The comparison revealed that the addition of the CBAM attention mechanism also revealed the minute aluminum alloy metallographic features, improving the segmentation accuracy of the model.
[0071] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. It should be noted that for those skilled in the art, any modifications, equivalent substitutions, and improvements made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.
Claims
1. A method for identifying residual aluminum alloy micrograph information integrated with CBAM, characterized in that, The image information extraction includes the following steps: Step 1: Data acquisition and preprocessing. Microstructure images of aluminum alloys are acquired using a metallographic microscope, and the acquired microstructure images are converted to grayscale to form a dataset of aluminum alloy microstructure images. Step 2: Construct an aluminum alloy microstructure segmentation model. Based on the deep learning framework PyTorch, a network is built to analyze the aluminum alloy microstructure. A residual network model, ResUNet, is constructed using the encoder-decoder structure in the UNet network. An attention mechanism model, CBAM, is added to the decoder part of the ResUNet model to construct the aluminum alloy microstructure segmentation model. The aluminum alloy microstructure segmentation model is then trained to obtain the trained aluminum alloy microstructure segmentation model, ResUNet_CBAM. The ResUNet residual network model inserts three ResBlock residual structures into both the encoder and decoder of the ResUNet residual network. Before each ResBlock residual structure in the decoder of the ResUNet residual network model, the features from the lower layer are upsampled, and after each upsampling, the attention mechanism model CBAM is introduced. Each ResBlock residual structure includes a 3×3 convolutional layer (CONV), a ReLU activation function layer, and a batch normalization layer (BN). The attention mechanism model CBAM is introduced into the decoder part of the ResUNet residual network model. It is introduced after the upsampling module (Upsample) in the decoder upsampling stage and before the batch normalization layer (BN). CBAM is used to process the input encoded segmentation feature map and then concatenate the features from the encoder. Step 3: Aluminum alloy microstructure image segmentation. Input the aluminum alloy microstructure image dataset into the trained aluminum alloy microstructure segmentation model ResUNet_CBAM for recognition processing to obtain the aluminum alloy microstructure segmentation results.
2. The method of claim 1, wherein the method is a method of identifying microstructure information of a residual aluminum alloy integrated with CBAM, characterized by, The microstructure image of the aluminum alloy to be segmented is input into the trained aluminum alloy microstructure segmentation model ResUNet_CBAM for recognition processing, including the following steps: Step 21: Set up the PyTorch neural network training environment and configure the image input training parameters, including setting the image input size to 256. 256, initial learning rate 5 10 -5 The minimum learning rate is 1. 10 -5 The model training batch size is 4, and the number of training iterations is 100,000. Step 22: The aluminum alloy microstructure image dataset is preprocessed by grayscale conversion, cropping, and rotation using a weighted average method. The preprocessed aluminum alloy microstructure images are then standardized to 256 pixels to meet the input requirements of the residual UNet network. 256 size; Step 23: Input the aluminum alloy microstructure image dataset into the trained aluminum alloy microstructure segmentation model ResUNet_CBAM for encoding and decoding recognition processing, and output the segmentation result image.
3. The method for recognizing microscopic image information of residual aluminum alloys incorporating CBAM according to claim 2, characterized in that, The aluminum alloy microstructure image dataset is input into the trained aluminum alloy microstructure segmentation model ResUNet_CBAM, and encoding and decoding recognition processes are performed sequentially, including the following steps: Step 24, during encoding and recognition, the number of image channels is gradually increased to 1024 dimensions in the encoder through convolution, and then the size of the aluminum alloy microstructure image is reduced to 1 / 16 of the original size through pooling layer. Features are gradually extracted from the input aluminum alloy microstructure image to obtain the encoding and segmentation effect feature map. Step 25: During decoding and recognition, the encoder segmentation effect feature map from the lower layer is upsampled through the upsampling layer and convolutional layer in the decoder. The attention mechanism model CBAM first performs channel attention mechanism to reduce the number of channels of the input encoder segmentation effect feature map, and then performs spatial attention mechanism operation. The encoder segmentation effect feature map copy, the feature map copy after spatial attention mechanism operation, and the feature map copy after channel attention mechanism operation are concatenated radially along a single channel to generate encoder segmentation effect features of different scales, thereby realizing multi-scale feature fusion. Step 26: Finally, a 1×1 convolution and a Sigmoid activation layer are used to change the number of channels in the image to achieve a multi-scale feature fusion encoding segmentation effect. The segmentation result is then output, and the segmentation result is multiplied by the input aluminum alloy microstructure image. The resulting output image is then added to the image output by the skipped link layer. The image obtained after addition is then used sequentially with the Sigmoid function for prediction calculation and cross-entropy calculation to calculate the final classification loss, thus obtaining the segmentation performance of the segmented image.
4. The method for recognizing microscopic image information of residual aluminum alloys incorporating CBAM according to claim 3, characterized in that, The attention mechanism model CBAM includes a channel attention module (CAM) and a spatial attention module (SAM). After convolution in the residual UNet network model, the channel attention module CAM and the spatial attention module SAM are sequentially inserted. In the decoder part of the residual network model ResUNet, after upsampling, the channel attention module CAM and the spatial attention module SAM are inserted to form the aluminum alloy microstructure segmentation model ResUNet_CBAM. The specific steps for processing the encoded segmentation effect feature map are as follows: The upsampled output encoded segmentation feature map first enters the channel attention module CAM's max pooling layer MaxPool and average pooling layer Avg Pool, and then enters the multilayer perceptron module. In the perceptron module, the number of channels is first compressed to 1 / n of the original number, and then expanded to the original number of channels. In the Spatial Attention (SAM) module, two single-channel feature maps are first obtained through a max pooling layer and an average pooling layer (Avg Pool). Then, a tensor concatenation layer (Concat) is used to concatenate the two single-channel feature maps. Finally, 7... The convolution calculation of step 7 is transformed into a single-channel feature map. The output result is multiplied with the original image to obtain the output image of the attention mechanism model CBAM. Then, this output image is added to the image of the skipped link layer and then fed into the next residual structure ResBlock.
5. A method for recognizing microscopic image information of residual aluminum alloys incorporating CBAM according to any one of claims 2 to 4, characterized in that, A skip link layer is added between the encoder and decoder to concatenate the decoded feature map from the decoding stage and the encoded feature map obtained in the same layer from the encoding stage, thereby achieving multi-scale feature fusion.
6. The method for recognizing microscopic image information of residual aluminum alloys incorporating CBAM according to claim 3, characterized in that, The prediction calculation using the Sigmoid function and the final classification loss calculated using cross-entropy H respectively satisfy the following: ; ; Considered as the model's original prediction of the sample image, The true label value of the sample image. The value can be 0 or 1.
7. A residual aluminum alloy microscopic image information recognition model incorporating CBAM, characterized in that, The recognition model is used to execute the residual aluminum alloy microscopic image information recognition method incorporating CBAM as described in any one of claims 1 to 6. The recognition model includes an encoder and a decoder. Several layers of residual structures ResBlock are respectively arranged between the input and output of the encoder and between the input and output of the decoder, and the residual structures ResBlock in the encoder and the residual structures ResBlock in the decoder are mirror-symmetrically distributed. The input end of each residual structure ResBlock in the decoder is connected to the output end of the corresponding layer of residual structure ResBlock in the encoder through a skip link layer. The input image and output image of each attention mechanism model CBAM in the decoder are fed into the residual structure ResBlock of the previous layer by multiplication.
8. A residual aluminum alloy microscopic image information recognition model incorporating CBAM according to claim 7, characterized in that, The attention mechanism model CBAM includes a channel attention module (CAM) and a spatial attention module (SAM).
