Residual aluminum alloy microscopic image information identification method and model integrated with CBAM

By integrating residual modules and CBAM attention into the UNet network, the method enhances the accuracy and efficiency of aluminum alloy microstructure segmentation, addressing the limitations of traditional methods and improving feature representation.

CN119068484APending Publication Date: 2024-12-03GUANGXI UNIV

Patent Information

Application Number
CN202411115668.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-14
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

Traditional metallographic analysis of aluminum alloys relies heavily on human experience, leading to inaccurate and inefficient results, and existing deep learning methods like UNet networks face challenges with high computational complexity, data imbalance, and incompatibility between classification and positioning accuracy.

Method used

Integrate the residual module and channel and spatial attention mechanism (CBAM) into the UNet network to enhance feature representation and accuracy of aluminum alloy semantic segmentation, using a method that includes data preprocessing, constructing a ResUNet_CBAM model, and implementing multi-scale feature fusion.

Benefits of technology

Improves the accuracy of aluminum alloy microstructure segmentation while reducing computational complexity, achieving higher segmentation accuracy and better feature expression.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119068484A_ABST
    Figure CN119068484A_ABST
Patent Text Reader

Abstract

The invention discloses a residual aluminum alloy microscopic image information identification method and model integrated with CBAM, and the method comprises the steps: carrying out the graying processing of a collected microstructure image, and forming an aluminum alloy microstructure image data set; the method comprises the following steps: carrying out network construction on an aluminum alloy microstructure, constructing a residual network model ResUNet, adding an attention mechanism model CBAM to a decoder part in the residual network model ResUNet to construct an aluminum alloy microstructure segmentation model, inputting an aluminum alloy microstructure image data set into the aluminum alloy microstructure segmentation model for training, and obtaining an aluminum alloy microstructure segmentation result. A trained aluminum alloy microstructure segmentation model ResUNetCBAM is obtained; and inputting a to-be-segmented aluminum alloy microstructure image into the trained aluminum alloy microstructure segmentation model for identification processing to obtain an aluminum alloy microstructure segmentation result. According to the method, a residual module and a channel and space attention mechanism are integrated in the UNet network, so that the detail features of the aluminum alloy microstructure are enhanced, and the semantic segmentation precision of the aluminum alloy microstructure is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the technical field of alloy microscopic image processing, and in particular relates to a residual aluminum alloy microscopic image information recognition method and model integrated with CBAM. Background Art

[0002] Due to the characteristics of high specific strength, good corrosion resistance, excellent plasticity and toughness, and good processing performance, aluminum alloys are more widely used in engineering construction, aerospace, and transportation. The use of aluminum alloys in aerospace applications can reduce the weight of aircraft, thereby reducing fuel consumption, increasing payload, improving performance, and reducing emissions. In order to ensure the reliability and stability of aircraft operation, aluminum alloys must meet certain relevant mechanical performance requirements. The microstructure of metal materials is closely related to their macroscopic mechanical properties. Therefore, how to analyze the metallographic structure has become one of the important research issues for understanding and changing the intrinsic factors of materials. Traditional metallographic analysis mostly uses qualitative and semi-quantitative methods with many defects. Most of them rely on human experience to judge their content and performance. The work efficiency is low and is greatly affected by human subjective factors, resulting in inaccurate analysis results. However, in recent years, deep learning has made progress in the field of image segmentation, especially the UNet network. The UNet network has a good performance in image segmentation, which can better solve the problems of traditional microstructure analysis and improve the efficiency and accuracy of image segmentation. However, the UNet network requires higher computing resources and has too much redundancy, which will lead to slow network training and the problem of incompatibility between classification accuracy and positioning accuracy. There may also be some challenges in data imbalance and large-size image processing.

[0003] Incorporating attention mechanisms into neural networks can quickly filter out valuable information and improve the model's attention and processing capabilities for input data. The CBAM attention mechanism not only combines channel attention and spatial attention, but is also suitable for more comprehensive feature representation. Although the CBAM attention mechanism has achieved good results, it will increase the complexity of the algorithm and the computational cost is high. Therefore, how to more effectively use the attention mechanism to improve the accuracy of aluminum alloy semantic segmentation while reducing computational complexity and avoiding model degradation is still one of the hot topics of research today. Summary of the invention

[0004] The purpose of the present invention is to provide a residual aluminum alloy microscopic image information recognition method and model integrated with CBAM. The present invention integrates the residual module and the channel and spatial attention mechanism into the UNet network, thereby enhancing the detailed features of the aluminum alloy microstructure and improving the accuracy of the semantic segmentation of the aluminum alloy microstructure. In order to achieve the above purpose, the technical solution adopted by the present invention is as follows:

[0005] The present invention proposes a residual aluminum alloy microscopic image information recognition method incorporating CBAM, characterized in that the image information extraction comprises the following steps:

[0006] Step 1: Data acquisition and preprocessing: using a metallographic microscope to acquire microstructure images of aluminum alloys, and graying the acquired microstructure images to form an aluminum alloy microstructure image dataset;

[0007] Step 2: Construct an aluminum alloy microstructure segmentation model. Based on the deep learning framework Pytorch, a network of aluminum alloy microstructures is constructed. The encoder-decoder structure in the UNet network is used to construct a residual network model ResUNet. The attention mechanism model CBAM is added to the decoder part of the residual network model ResUNet to construct an aluminum alloy microstructure segmentation model. The aluminum alloy microstructure segmentation model is trained to obtain the trained aluminum alloy microstructure segmentation model ResUNet_CBAM.

[0008] Step 3: Aluminum alloy microstructure image segmentation: Input the aluminum alloy microstructure image dataset into the trained aluminum alloy microstructure segmentation model ResUNet_CBAM for recognition processing to obtain the aluminum alloy microstructure segmentation result.

[0009] The above method is further preferred, wherein the aluminum alloy microstructure image to be segmented is input into the trained aluminum alloy microstructure segmentation model ResUNet_CBAM for recognition processing, comprising the following steps:

[0010] Step 21, build a Pytorch neural network training environment and set image input training parameters;

[0011] Step 22, using a weighted average method to pre-process the aluminum alloy microstructure image dataset by grayscale, cropping, and rotating, and then standardizing the pre-processed aluminum alloy microstructure image to a size of 256×256 that meets the input requirements of the residual UNet network;

[0012] Step 23, input the aluminum alloy microstructure image data set into the trained aluminum alloy microstructure segmentation model ResUNet_CBAM to perform encoding recognition and decoding recognition processing in sequence, and output the segmentation effect diagram.

[0013] The above method is further preferred, wherein the aluminum alloy microstructure image dataset is input into the trained aluminum alloy microstructure segmentation model ResUNet_CBAM to sequentially perform encoding recognition and decoding recognition processing, including the following steps:

[0014] Step 24, during encoding recognition, the number of image channels is gradually increased to 1024 dimensions through convolution in the encoder, and then the size of the aluminum alloy microstructure image is reduced to 1 / 16 of the original through the pooling layer, and features are gradually extracted from the input aluminum alloy microstructure image to obtain a coding segmentation effect feature map;

[0015] Step 25, during decoding recognition, the coding segmentation effect feature map from the lower layer is upsampled in the decoder through the upsampling layer and the convolution layer, and the attention mechanism model CBAM first performs a channel attention mechanism operation to reduce the number of feature map channels on the input coding segmentation effect feature map, and then performs a spatial attention mechanism operation, and radially splices the coding segmentation effect feature map copy output after the encoder processing, the feature map copy processed by the spatial attention mechanism operation, and the feature map copy processed by the channel attention mechanism operation along a single channel to generate coding segmentation effect features of different scales, thereby realizing multi-scale feature fusion;

[0016] Step 26. Finally, use 1×1 convolution and Sigmoid activation layer to change the number of channels of the image to achieve the segmentation result output of the encoded segmentation effect image with multi-scale feature fusion. Finally, multiply the segmentation result output by the input aluminum alloy microstructure image, add the output image to the image output by the skip link layer, and use Sigmoid function to perform prediction calculation and cross entropy calculation on the image obtained after addition to obtain the final classification loss, so as to obtain the segmentation performance of the segmented image.

[0017] The above method is further preferred, in which the residual network model ResUNet inserts 3 residual structures ResBlock into the encoder and decoder of the residual UNet network, upsamples the features from the lower layer before each residual structure ResBlock in the decoder of the residual network model ResUNet, and introduces the attention mechanism model CBAM after each upsampling.

[0018] The above method is further preferred that each residual structure ResBlock includes a convolution layer CONV with a continuous convolution kernel size of 3×3, a ReLU activation function layer and a batch normalization layer BN; the attention mechanism model CBAM is introduced in the decoder part of the residual network model ResUNet after the upsampling module Upsample in the upsampling stage of the decoder and before the batch normalization layer BN, which is used to process the input encoding segmentation effect feature map and then splice the features from the encoder.

[0019] The above method is further preferred, the attention mechanism model CBAM includes a channel attention module CAM and a spatial attention module SAM, the channel attention module CAM and the spatial attention module SAM are sequentially inserted after the convolution of the residual UNet network model, and the decoder part in the residual network model ResUNet is inserted after upsampling. The channel attention module CAM and the spatial attention module SAM are formed to form an aluminum alloy microstructure segmentation model ResUNet_CBAM to process the encoding segmentation effect feature map. The specific steps are as follows:

[0020] The up-sampled output encoding segmentation effect feature map first enters the maximum pooling layer Max Pool and the average pooling layer Avg Pool of the channel attention module CAM, and then enters the multi-layer perceptron module. In the perceptron module, the number of channels is first compressed to 1 / n of the original number, and then expanded to the original number of channels;

[0021] In the spatial attention module SAM, the maximum pooling layer Max Pool and the average pooling layer Avg Pool are first used to obtain two single-channel feature maps, and then the tensor concatenation layer Concat is used to concatenate the two single-channel feature maps. Then, a 7×7 convolution calculation is used to transform them into single-channel feature maps. The output result is multiplied by the original image to obtain the output image of the attention mechanism model CBAM. Subsequently, this output image is added to the image of the skip link layer and then sent to the next residual structure ResBlock to enter the next residual structure.

[0022] The above method is further preferred to add a skip link layer between the encoder and the decoder, and concatenate the decoding feature map in the decoding stage and the encoding feature map obtained in the same layer in the encoding stage, so as to achieve multi-scale feature fusion.

[0023] The above method is further preferred, using the Sigmoid function for prediction calculation and the cross entropy H to calculate the final classification loss respectively satisfies:

[0024]

[0025] is regarded as the original prediction value of the model for the sample image, y is the true label value of the sample image, and the value of y is 0 or 1.

[0026] The above method is further preferably configured such that the image input training parameters are configured to include setting the image input size to 256×256 and the initial learning rate to 5×10 -5 , the minimum learning rate is 1×10 -5 , the model training batch size is 4, and the number of training iterations is 100000.

[0027] According to another aspect of the present application, the present invention provides a residual aluminum alloy micro-image information recognition model incorporating CBAM, the recognition model comprising an encoder and a decoder, wherein several layers of residual structures ResBlock distributed from top to bottom are respectively arranged between the input and output of the encoder and between the input and output of the decoder, and the residual structure ResBlock in the encoder and the residual structure ResBlock in the decoder are mirror-symmetrically distributed; an attention mechanism model CBAM is added to the input end of each layer of residual structure ResBlock in the decoder, the input end of each residual structure ResBlock in the decoder is connected to the output end of the residual structure ResBlock of a corresponding layer in the encoder through a skip link layer, and the input image and output image of each attention mechanism model CBAM in the decoder are sent to the residual structure ResBlock of the previous layer by multiplication.

[0028] The above method is further preferred, wherein the attention mechanism model CBAM includes a channel attention module CAM and a spatial attention module SAM, and the channel attention module CAM and the spatial attention module SAM are sequentially inserted after the convolution of each residual structure ResBlock in the decoder.

[0029] Due to the adoption of the above technical solution, the present invention has the following significant effects:

[0030] (1) Compared with the residual UNet model without CBAM, the residual UNet aluminum alloy microscopic image information extraction algorithm with CBAM has high segmentation accuracy and makes the network more focused on features related to specific segmentation tasks. However, domestic and foreign scholars have not yet inserted the CBAM attention mechanism into the residual UNet network, while the residual UNet aluminum alloy microscopic image information extraction algorithm with CBAM is integrated in the present invention.

[0031] (2) The present invention designs a residual UNet aluminum alloy microscopic image information extraction algorithm that incorporates CBAM. Based on the residual UNet network that incorporates CBAM, the network mainly uses the CBAM attention mechanism to suppress irrelevant background features and improve the expression ability, differentiation ability and positioning ability of ResUnet.

[0032] (3) The present invention adds residual connections based on UNet, and each module in the UNet encoder and decoder contains multiple residual connections. In each residual block, a shortcut is introduced to achieve cross-layer connection, which can better extract low-frequency information, alleviate the problem of missing semantic information and the problem of gradient disappearance, thereby improving the accuracy of image segmentation. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1It is a schematic diagram of the structure of the identification process of a residual aluminum alloy microscopic image information identification method incorporating CBAM of the present invention;

[0034] Figure 2 It is a model structure principle diagram of a residual aluminum alloy microscopic image information recognition model integrated with CBAM of the present invention;

[0035] Figure 3 It is a comparison diagram of the segmentation effect of the present invention. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and preferred embodiments. However, it should be noted that many details listed in the specification are only for the purpose of enabling the reader to have a thorough understanding of one or more aspects of the present invention, and these aspects of the present invention can be implemented even without these specific details.

[0037] Combination Figure 1 The present invention proposes a residual aluminum alloy microscopic image information recognition method incorporating CBAM, characterized in that the image information extraction comprises the following steps:

[0038] Step 1: Data acquisition and preprocessing: using a metallographic microscope to acquire microstructure images of aluminum alloys, and graying the acquired microstructure images to form an aluminum alloy microstructure image dataset;

[0039] Step 2: Construct an aluminum alloy microstructure segmentation model. Based on the deep learning framework Pytorch, a network of aluminum alloy microstructures is constructed. The encoder-decoder structure in the UNet network is used to construct a residual network model ResUNet. The attention mechanism model CBAM is added to the decoder part of the residual network model ResUNet to construct an aluminum alloy microstructure segmentation model. The aluminum alloy microstructure segmentation model is trained to obtain the trained aluminum alloy microstructure segmentation model ResUNet_CBAM.

[0040] Step 3: Aluminum alloy microstructure image segmentation: Input the aluminum alloy microstructure image dataset into the trained aluminum alloy microstructure segmentation model ResUNet_CBAM for recognition processing to obtain the aluminum alloy microstructure segmentation result.

[0041] In the present invention, the residual network model ResUNet inserts three residual structures ResBlock in the encoder and decoder of the residual UNet network, upsamples the features from the lower layer before each residual structure ResBlock in the decoder of the residual network model ResUNet, and introduces the attention mechanism model CBAM after each upsampling; in the present invention, each residual structure ResBlock includes a convolution layer CONV with a continuous convolution kernel size of 3×3, a ReLU activation function layer and a batch normalization layer BN; the attention mechanism model CBAM is introduced in the decoder part of the residual network model ResUNet after the upsampling module Upsample in the upsampling stage of the decoder The attention mechanism model CBAM is introduced before the batch normalization layer BN to process the input encoding segmentation effect feature map and then splice the features from the encoder. A skip link layer Bridge is added between the encoder and the decoder to splice the decoding feature map in the decoding stage and the encoding feature map obtained in the same layer in the encoding stage, thereby realizing multi-scale feature fusion; the decoder parses the encoded data into the original data form, receives the encoded data stream, and restores it to the original audio, video or image data for playback or further processing. The decoder decodes the context vector generated by the encoder into an output sequence. The initial state of the decoder is usually initialized by the final state of the encoder to ensure the transmission of information. The decoder generates the output of the current time step based on the output and hidden state of the previous time step at each time step, and gradually constructs the entire output sequence, and allows the decoder to dynamically pay attention to different parts of the input sequence when generating each image, thereby improving the performance and interpretability of the model; the attention mechanism model CBAM includes a channel attention module CAM, a spatial attention module SAM, a transmission module and an output module, and the channel attention module CAM and the spatial attention module SAM are inserted in sequence after the convolution of the residual UNet network model.

[0042] In the present invention, the aluminum alloy microstructure image to be segmented is input into the trained aluminum alloy microstructure segmentation model for recognition processing, which includes the following steps:

[0043] Step 21: Build a Pytorch neural network training environment and set image input training parameters; setting image input training parameters includes setting the image input size to 256×256 (resolution) and the initial learning rate to 5×10 -5 , the minimum learning rate is 1×10 -5 , the model training batch size is 4, and the number of training iterations is 100000;

[0044] Step 22, the weighted average method is used to pre-process the segmentation effect image by grayscale, cropping and rotation, and then the pre-processed segmentation effect image is standardized to a size (resolution) of 256×256 that meets the input requirements of the residual UNet network. The grayscale processing is weighted averaged according to the RGB three components to obtain the component values ​​of the R, G, and B channels;

[0045] Step 23, inputting the aluminum alloy microstructure image data set into the trained aluminum alloy microstructure segmentation model ResUNet_CBAM to perform encoding recognition and decoding recognition processing in sequence, and outputting a segmentation effect diagram; inputting the aluminum alloy microstructure image data set into the trained aluminum alloy microstructure segmentation model ResUNet_CBAM to perform encoding recognition and decoding recognition processing in sequence includes the following steps:

[0046] Step 24, during coding recognition, the number of image channels is gradually increased to 1024 dimensions in the encoder through convolution, and then the size of the aluminum alloy microstructure image is reduced to 1 / 16 of the original through the pooling layer, and features are gradually extracted from the input aluminum alloy microstructure image to obtain the coding segmentation effect feature map; in the residual network model ResUNet, three residual structures ResBlock are inserted in the encoder and decoder in the UNet network. In the residual structure ResBlock, the image is no longer downsampled, but the step size of the first convolution layer in the convolution module is set to 2. In the encoder, the number of image channels is gradually increased to 1024 dimensions through convolution, and the size of the feature map is reduced to 1 / 16 of the original through the pooling layer, and features are gradually extracted from the input segmentation effect map (image), and the dimension of the feature map is adjusted by using reshape. A skip link is added between the encoder and the decoder, and the feature map of the decoding stage and the feature information obtained in the same layer of the encoding stage are spliced ​​to achieve multi-scale feature fusion;

[0047] Step 25, during decoding recognition, the coding segmentation effect feature map from the lower layer is upsampled in the decoder through the upsampling layer and the convolution layer, and the attention mechanism model CBAM first performs a channel attention mechanism operation to reduce the number of feature map channels on the input coding segmentation effect feature map, and then performs a spatial attention mechanism operation, and radially splices the coding segmentation effect feature map copy obtained by the encoder, the feature map copy processed by the spatial attention mechanism, and the feature map copy processed by the channel attention mechanism along a single channel to generate coding segmentation effect features of different scales, thereby realizing multi-scale feature fusion;

[0048] The decoder contains four upsampling layers. Through upsampling and convolution, the number of feature map channels is reduced, the image size is expanded, and the features are fused, and finally the output image with the same resolution as the input image is restored. The decoder also uses three residual structures ResBlock. Unlike the encoder, before each residual structure ResBlock, the features from the lower layer are upsampled. After each upsampling, the attention mechanism model CBAM is added, and then the features from the encoder are spliced. After passing through the last layer of the decoder, a 1×1 convolution and Sigmoid activation layer are used to change the number of channels of the image.

[0049] Step 26, finally, output after passing through the sigmiod activation function and convolution; that is, using the Sigmoid activation layer to change the image and the number of channels of the 1×1 convolution to achieve the segmentation result output of the coded segmentation effect image of the multi-scale feature fusion, and finally multiplying the segmentation result output result with the input aluminum alloy microstructure image, adding the output image to the image output by the skip link layer, and using the Sigmoid function to perform prediction calculation and cross entropy calculation on the image obtained after the addition to obtain the segmentation performance of the segmented image, and using the Sigmoid function to perform prediction calculation and cross entropy H to calculate the final classification loss respectively to meet the following requirements:

[0050]

[0051] When dealing with classification problems, cross entropy is usually used together with the Sigmoid function. Assume that y is regarded as the true label (0 or 1) of the sample. is considered as the original prediction value of the model for the sample.

[0052] The output image is processed using the Sigmoid function, and then the cross entropy is used to calculate the final classification loss. The main function of the loss function is to supervise the learning process of the model. First, the difference between the result data predicted by the model and the label data is calculated, which is the loss value. After calculating the loss value, the model updates each parameter through back propagation to reduce the difference between the true value and the predicted value, so that the predicted value generated by the model is closer to the true value, so that the model can better fit the data. The cross entropy loss function is used to measure the amount of information. The cross entropy is mainly used to measure the difference between two probability distributions. In the classification task, it is used to measure the difference distribution between the predicted result and the true label. The smaller the cross entropy value, the better the prediction effect of the model.

[0053] In the present invention, the decoder part in the residual network model ResUNet is inserted into the aluminum alloy microstructure segmentation model ResUNet_CBAM formed by the channel attention module CAM and the spatial attention module SAM after upsampling to process the encoding segmentation effect feature map. The specific steps are as follows:

[0054] Step 31, the up-sampled output encoding segmentation effect feature map first enters the maximum pooling layer Max Pool and the average pooling layer Avg Pool of the channel attention module CAM, and then enters the multi-layer perceptron module, where the number of channels is first compressed to 1 / n of the original number of channels, and then expanded to the original number of channels;

[0055] Step 32, add the output results of the maximum pooling layer Max Pool and the average pooling layer Avg Pool element by element, and then perform prediction calculation on the added result to obtain the output result of the channel attention module CAM, multiply the output result of the channel attention module CAM with the original segmentation effect map and send it to the spatial attention module SAM to generate feature maps of different scales;

[0056] Step 33, in the spatial attention module SAM, first use the maximum pooling layer Max Pool and the average pooling layer AvgPool to obtain two single-channel feature maps, and then use the tensor concatenation layer Concat to concatenate the two single-channel feature maps. Then use 7×7 convolution calculation to change them into single-channel feature maps, multiply the output result with the original image (the image input to the input side of the spatial attention module SAM) to obtain the output image of the attention mechanism model CBAM, and then add this output image to the image of the skip link layer, and then send it to the next residual structure ResBlock to enter the next residual structure;

[0057] In the present invention, the aluminum alloy microstructure segmentation model ResUNet_CBAM is achieved by adding an attention mechanism model CBAM after the upsampling part, and these attention mechanism models CBAM are responsible for extracting features. The image output by the upsampling module Upsample first enters the channel attention module CAM for maximum pooling and average pooling, and then enters the multi-layer perceptron module, where the number of channels is compressed to 1 / n of the original number, and then expanded to the original number of channels; then the output results of the maximum pooling and average pooling are added element by element, and the output result of the channel attention module CAM is calculated through a Sigmoid activation function, and then this output result is multiplied with the originally input encoded original image (the image input on the input side of the channel attention module CAMS) and enters the spatial attention module SAM; in the spatial attention module SAM, the maximum pooling and average pooling are first used to obtain Two single-channel feature maps, then use the Concat tensor to splice the two feature maps, then use 7×7 convolution to make it a single-channel feature map, and finally multiply the output result with the original image (the image input to the input side of the spatial attention module SAM of this layer) to obtain the output image of the attention mechanism model CBAM; In the present invention, the channel attention mechanism compresses the feature map, and uses global average pooling and global maximum pooling to convert the size of (C, H, W) to (C, 1, 1) to achieve global context information fusion; A feature map with context information of different scales is obtained, and then this feature map is processed by a convolution layer to generate a spatial attention weight. The ReLU activation function is applied to the generated spatial attention weight to limit the weight between 0 and 1. The spatial attention module weights the feature maps at different positions, strengthens important positions, and suppresses unimportant positions; then the global maximum pooling and global average pooling results are respectively subjected to MLP multi-layer perceptron operations. MLP is a two-layer fully connected layer with a ReLU activation function in the middle. The two are added and activated by the Sigmoid function; finally, the reshaped weight value is multiplied by the original feature map using Python's broadcast mechanism to obtain feature maps under different weights. The spatial attention mechanism performs maximum pooling and average pooling on the channel dimension for the results of the channel attention mechanism operation; that is, the feature map after the channel attention mechanism is converted from (C, H, W) to (1, H, W) to achieve the effect of fusing information from different channels, and then the maximum pooling and average pooling results are superimposed on the channel dimension, that is, torch.cat() is used; secondly, the results of the superimposed two channels are convolved, the output channel is 1, the convolution kernel size is 7, and the output result is activated by the Sigmoid function; finally, the weight value is multiplied by the original feature map using Python's broadcast mechanism to obtain feature maps under different weights.In the attention mechanism model CBAM, the reshape function is used to transform the specified matrix into a matrix of a specific dimension while keeping the total number of elements in the matrix unchanged. This operation allows the number of rows, columns, or dimensions of the matrix to be readjusted to suit specific computing requirements, thereby adjusting the dimension of the feature map for subsequent attention mechanism calculations or network layer connections. This output image is then added to the image of the skip link, and then enters the next residual structure, and then passes through the BN layer, ReLU layer, and 3×3 convolution layer to build a deeper network structure without encountering the problem of gradient disappearance or gradient explosion. The residual unit can simplify the training of the network, and the skip links within the residual unit and between the low and high levels of the network can help effectively transfer low-level features to high levels, thereby improving the expressiveness of the model.

[0058] In the present invention, combined with Figure 1 and Figure 2The present invention provides an aluminum alloy microscopic image information recognition model, the recognition model includes an encoder and a decoder, and a plurality of layers of residual structures ResBlock distributed from top to bottom are respectively arranged between the input and output of the encoder and between the input and output of the decoder, and the residual structure ResBlock in the encoder and the residual structure ResBlock in the decoder are mirror-symmetrically distributed; an attention mechanism model CBAM (CBAM model) is added to the input end of each layer of residual structure ResBlock in the decoder, the input end of each residual structure ResBlock in the decoder is connected to the output end of the residual structure ResBlock of a corresponding layer in the encoder through a skip link layer, and the input end and output of each attention mechanism model CBAM in the decoder are sent to the residual structure ResBlock of the previous layer by multiplication; the attention mechanism model CBAM includes a channel attention module CAM and a spatial attention module SAM, and the channel attention module CAM and the spatial attention module SAM are sequentially inserted after the convolution of each residual structure ResBlock in the decoder. Block SAM; wherein each residual structure ResBlock includes a convolution layer CONV with a continuous convolution kernel size of 3×3, a ReLU activation function layer and a batch normalization layer BN; the attention mechanism model CBAM is introduced in the decoder part of the residual network model ResUNet after the upsampling module Upsample in the upsampling stage of the decoder and before the batch normalization layer BN. The attention mechanism model CBAM is used to process the input encoding segmentation effect feature map and then splice the features from the encoder. A skip link layer Bridge is added between the encoder and the decoder to splice the decoding feature map in the decoding stage and the encoding feature map obtained in the same layer in the encoding stage, thereby realizing multi-scale feature fusion; the decoder parses the encoded data into the original data form. It receives the encoded data stream and restores it to the original audio, video or image data for playback or further processing. The decoder decodes the context vector generated by the encoder into an output sequence. The initial state of the decoder is usually initialized by the final state of the encoder to ensure the transmission of information. In the present invention, three layers of residual structures ResBlock distributed from top to bottom are respectively arranged between the input and output of the encoder and between the input and output of the decoder; the preprocessed aluminum alloy microstructure image data set is input, and a feature map copy is obtained after quick connection and identity mapping in the first residual structure ResBlock of the encoder, and then the feature map copy processed by the spatial attention module and the feature map copy processed by the channel attention module are radially spliced ​​along a single channel to input the residual structure ResBlock of the decoder. Similarly, the feature maps processed by the other two residual structures ResBlock of the encoder all enter the symmetrical residual structure ResBlock in the decoder through the link path.At each level between the encoder and decoder, a skip link layer is used to retain and pass higher-level feature information. This allows low-level feature map information to be passed directly to high-level levels, helping the decoder to better recover details and edge information. In this way, the decoder can use both low-level features from the encoder and its own high-level features for reconstruction; the data processed by the encoder is then input into the decoder through the connection bridge, and the result is finally output.

[0059] The present invention constructs an aluminum alloy microstructure segmentation model by adding an attention mechanism model CBAM to the decoder part of the residual network model ResUNet. It can effectively improve the accuracy of image segmentation when used for identifying aluminum alloy microstructure images. As shown in Table 1, the segmentation and recognition effects of the residual network model ResUNet and the aluminum alloy microstructure segmentation model ResUNet_CBAM are:

[0060] Table 1 Recognition comparison effect experimental data

[0061]

[0062] The present invention constructs an aluminum alloy microstructure segmentation model by adding an attention mechanism model CBAM to the decoder part of the residual network model ResUNet. It can effectively improve the accuracy of image segmentation when used for identifying aluminum alloy microstructure images. As shown in Table 1, the segmentation and recognition effects of the residual network model ResUNet and the aluminum alloy microstructure segmentation model ResUNet_CBAM are:

[0063] The above table summarizes the segmentation effects of ResUNet and ResUNet_CBAM. As can be seen from the above table, the segmentation effect of ResUNet_CBAM constructed by this patent is better than that of ResUNet, with the intersection over union (IoU) increased by 1.69% and the F1 value increased by 0.015. In terms of model size, ResUNet_CBAM only increases by 0.18M on the basis of the original model. The performance evaluation index description is as follows:

[0064] Accuracy Acc: The ratio of the predicted correct metallographic positive samples and background negative samples to all samples among all metallographic positive samples and background negative samples, that is, the ratio of pixels classified correctly. The calculation formula is:

[0065]

[0066] Intersection over Union (IoU): Calculates the overlap ratio between the intersection of two sets and their union to measure the accuracy of metallographic structure recognition. The calculation formula is:

[0067]

[0068] F1-Score: It is the harmonic mean of precision and recall, calculated by:

[0069]

[0070] True Positives (TP): TP refers to the area where the metallographic structure is correctly identified as the metallographic structure during the recognition process. True Negatives (TN): TN refers to the area where the background is correctly identified as the background during the recognition process. False Positives (FP): FP refers to the area where the negative sample is identified as the positive sample during the recognition process, indicating that the background is detected as the metallographic structure, that is, a false detection. False Negatives (FN): FN refers to the area where the positive sample is identified as the negative sample during the recognition process, indicating that the metallographic structure is identified as the background, that is, a missed detection. In order to ensure the objectivity of the experimental results, the comparative experiments in this chapter were all carried out on the same workstation. Figure 3 This is the segmentation effect picture, which was taken by a super depth of field microscope. By comparison, it is found that after adding the CBAM attention mechanism, the tiny aluminum alloy metallographic structure is also segmented out, which improves the segmentation accuracy of the model.

[0071] The above is only a preferred embodiment of the present invention and is not intended to limit the present invention. It should be noted that for those skilled in the art, several modifications, equivalent substitutions and improvements made without departing from the principles of the present invention should also be considered as the protection scope of the present invention.

Claims

1. A method for identifying residual aluminum alloy microscopic image information incorporating CBAM, characterized in that: The image information extraction comprises the following steps: Step 1: Data acquisition and preprocessing: using a metallographic microscope to acquire microstructure images of aluminum alloys, and graying the acquired microstructure images to form an aluminum alloy microstructure image dataset; Step 2: Construct an aluminum alloy microstructure segmentation model. Based on the deep learning framework Pytorch, a network of aluminum alloy microstructures is constructed. The encoder-decoder structure in the UNet network is used to construct a residual network model ResUNet. The attention mechanism model CBAM is added to the decoder part of the residual network model ResUNet to construct an aluminum alloy microstructure segmentation model. The aluminum alloy microstructure segmentation model is trained to obtain the trained aluminum alloy microstructure segmentation model ResUNet_CBAM. Step 3: Aluminum alloy microstructure image segmentation: Input the aluminum alloy microstructure image dataset into the trained aluminum alloy microstructure segmentation model ResUNet_CBAM for recognition processing to obtain the aluminum alloy microstructure segmentation result.

2. The residual aluminum alloy microscopic image information recognition method incorporating CBAM according to claim 1 is characterized in that: The aluminum alloy microstructure image to be segmented is input into the trained aluminum alloy microstructure segmentation model ResUNet_CBAM for recognition processing, including the following steps: Step 21: Build the Pytorch neural network training environment and set the image input training parameters. Setting the image input training parameters includes setting the image input size to 256×256 and the initial learning rate to 5×10 -5 , the minimum learning rate is 1×10 -5 , the model training batch size is 4, and the number of training iterations is 100000; Step 22, using a weighted average method to pre-process the aluminum alloy microstructure image dataset by grayscale, cropping, and rotating, and then standardizing the pre-processed aluminum alloy microstructure image to a size of 256×256 that meets the input requirements of the residual UNet network; Step 23, input the aluminum alloy microstructure image data set into the trained aluminum alloy microstructure segmentation model ResUNet_CBAM to perform encoding recognition and decoding recognition processing in sequence, and output the segmentation effect diagram.

3. The residual aluminum alloy microscopic image information recognition method incorporating CBAM according to claim 2 is characterized in that: Inputting the aluminum alloy microstructure image dataset into the trained aluminum alloy microstructure segmentation model ResUNet_CBAM for encoding recognition and decoding recognition processing in sequence includes the following steps: Step 24, during encoding recognition, the number of image channels is gradually increased to 1024 dimensions through convolution in the encoder, and then the size of the aluminum alloy microstructure image is reduced to 1 / 16 of the original through the pooling layer, and features are gradually extracted from the input aluminum alloy microstructure image to obtain a coding segmentation effect feature map; Step 25, during decoding recognition, the coding segmentation effect feature map from the lower layer is upsampled in the decoder through the upsampling layer and the convolution layer, and the attention mechanism model CBAM first performs a channel attention mechanism operation to reduce the number of feature map channels on the input coding segmentation effect feature map, and then performs a spatial attention mechanism operation, and radially splices the coding segmentation effect feature map copy output after the encoder processing, the feature map copy processed by the spatial attention mechanism operation, and the feature map copy processed by the channel attention mechanism operation along a single channel to generate coding segmentation effect features of different scales, thereby realizing multi-scale feature fusion; Step 26. Finally, use 1×1 convolution and Sigmoid activation layer to change the number of channels of the image to achieve the segmentation result output of the encoded segmentation effect image with multi-scale feature fusion. Finally, multiply the segmentation result output by the input aluminum alloy microstructure image, add the output image to the image output by the skip link layer, and use Sigmoid function to perform prediction calculation and cross entropy calculation on the image obtained after addition to obtain the final classification loss, so as to obtain the segmentation performance of the segmented image.

4. The residual aluminum alloy microscopic image information recognition method incorporating CBAM according to claim 1 is characterized in that: The residual network model ResUNet inserts three residual structures ResBlock in the encoder and decoder of the residual UNet network, upsamples the features from the lower layer before each residual structure ResBlock in the decoder of the residual network model ResUNet, and introduces the attention mechanism model CBAM after each upsampling.

5. The method for identifying residual aluminum alloy microscopic image information incorporating CBAM according to claim 4 is characterized in that: Each residual structure ResBlock includes a convolution layer CONV with a continuous convolution kernel size of 3×3, a ReLU activation function layer and a batch normalization layer BN; the attention mechanism model CBAM is introduced in the decoder part of the residual network model ResUNet after the upsampling module Upsample in the upsampling stage of the decoder and before the batch normalization layer BN, which is used to process the input encoding segmentation effect feature map and then splice the features from the encoder.

6. The method for identifying residual aluminum alloy microscopic image information incorporating CBAM according to claim 5 is characterized in that: The attention mechanism model CBAM includes a channel attention module CAM and a spatial attention module SAM. The channel attention module CAM and the spatial attention module SAM are sequentially inserted after the convolution of the residual UNet network model. The decoder part in the residual network model ResUNet is inserted with the channel attention module CAM and the spatial attention module SAM after upsampling to form an aluminum alloy microstructure segmentation model ResUNet_CBAM. The specific steps for processing the encoding segmentation effect feature map are as follows: The upsampled output encoding segmentation effect feature map first enters the maximum pooling layer MaxPool and the average pooling layer Avg Pool of the channel attention module CAM, and then enters the multi-layer perceptron module. In the perceptron module, the number of channels is first compressed to 1 / n of the original number, and then expanded to the original number of channels; In the spatial attention module SAM, the maximum pooling layer Max Pool and the average pooling layer Avg Pool are first used to obtain two single-channel feature maps, and then the tensor concatenation layer Concat is used to concatenate the two single-channel feature maps. Then, a 7×7 convolution calculation is used to transform them into single-channel feature maps. The output result is multiplied by the original image to obtain the output image of the attention mechanism model CBAM. Subsequently, this output image is added to the image of the skip link layer and sent to the next residual structure ResBlock to enter the next residual structure.

7. A method for identifying residual aluminum alloy microscopic image information incorporating CBAM according to any one of claims 2 to 6, characterized in that: A skip link layer is added between the encoder and the decoder to concatenate the decoded feature map in the decoding stage and the encoded feature map obtained in the same layer in the encoding stage, thereby achieving multi-scale feature fusion.

8. The method for identifying residual aluminum alloy microscopic image information incorporating CBAM according to claim 3 is characterized in that: The prediction calculation using Sigmoid function and the final classification loss calculated by cross entropy H satisfy: H(,p)=-[ylogp+(1-y)log(1-p)]; is regarded as the original prediction value of the model for the sample image, y is the true label value of the sample image, and the value of y is 0 or 1.

9. A residual aluminum alloy microscopic image information recognition model incorporating CBAM, characterized in that: The recognition model includes an encoder and a decoder, and several layers of residual structures ResBlock distributed from top to bottom are respectively arranged between the input and output of the encoder and between the input and output of the decoder, and the residual structure ResBlock in the encoder and the residual structure ResBlock in the decoder are distributed in a mirror-symmetrical manner; an attention mechanism model CBAM is added to the input end of each layer of residual structure ResBlock in the decoder, and the input end of each residual structure ResBlock in the decoder is connected to the output end of the residual structure ResBlock of a corresponding layer in the encoder through a skip link layer, and the input image and output image of each attention mechanism model CBAM in the decoder are sent to the residual structure ResBlock of the previous layer by multiplication.

10. The residual aluminum alloy microscopic image information recognition model incorporating CBAM according to claim 9 is characterized in that: The attention mechanism model CBAM includes a channel attention module CAM and a spatial attention module SAM, which are sequentially inserted after the convolution of each residual structure ResBlock in the decoder.

Citation Information

Patent Citations

  • Unet network brain tumor MRI image segmentation method for improving attention module

    CN113554669A

  • MRI brain tumor image segmentation method and system based on improved U-Net network

    CN113628220A

  • Alloy powder inclusion identification method based on microscopic imaging

    CN113705531A

  • Complex image segmentation method based on residual connection and attention mechanism fusion

    CN117495875A

  • Fiber bundle segmentation method based on U-NET network double-encoder structure

    CN117611602A

Cited By

  • Iron-carbon alloy microscopic structure identification method based on deep learning

    CN119625728A

  • Enteroscope image enhancement method based on improved generative adversarial network

    CN119941561A

  • A colonoscopy image enhancement method based on improved generative adversarial network

    CN119941561B

  • Analysis method and system for dynamic recrystallization structure morphology of titanium alloy based on machine learning and medium

    CN120544757A

  • Tissue characteristic quantitative extraction method and system for titanium alloy microstructure image

    CN120823598A