Impurity identification method and device for steel microstructure based on improved FCN network

Abstract

This invention discloses a method and apparatus for impurity identification in steel microstructures based on an improved FCN network. The method includes: obtaining a low-semantic feature map and a high-semantic feature map based on features from a microscopic image of the steel microstructure to be identified; performing channel-wise weighted summation between the channel feature map obtained from the high-semantic feature map and the high-semantic feature map to obtain a channel attention feature map, and obtaining a spatial attention feature map based on the channel attention feature map; obtaining a fused feature map based on the spatial attention feature map and the channel attention feature map; obtaining a final feature map by outputting classification results according to the multi-layer cascade; fusing and stitching the low-semantic feature map and the final feature map to obtain a restored feature map; and classifying and identifying the pixels of the restored feature map to output the final target identification result containing impurities. This invention addresses the problems of insufficient multi-scale feature information extraction by the YOLO algorithm, leading to insufficient ability to extract detailed features of small particles and low accuracy in identifying areas near grain boundaries and other boundaries.

Description

Technical Field

[0001] This invention relates to the field of steel microstructure segmentation technology, and in particular to a method and apparatus for impurity identification in steel microstructures based on an improved FCN network. Background Technology

[0002] The mechanical and physical properties of steel are influenced by its microstructure, making microstructure identification essential for controlling steel properties and performance. However, due to the variability of steel microstructure, non-uniform grayscale values, and blurred boundaries, manual image recognition suffers from problems such as high workload, long processing time, and significant susceptibility to subjective factors. In recent years, with the continuous development of deep learning, many scholars have applied deep learning methods to image recognition of microstructures, overcoming the shortcomings of manual feature extraction which lacks visual recognition information and demonstrating powerful performance.

[0003] Deep learning-based image recognition algorithms include R-CNN, SPP-Net, FastR-CNN, FasterR-CNN, YOLO, etc.

[0004] The R-CNN algorithm first divides the input image into candidate regions, then extracts features from these regions using a CNN network model for classification. However, R-CNN has limitations; it requires normalizing the selected regions to a uniform size before feeding them into the CNN, leading to information loss. The SPP-Net algorithm feeds the entire image into convolutional layers to extract feature images, then maps these feature maps to candidate regions, obtaining convolutional features that are then input into a fully connected network. The SPP layer can change the size of the convolutional feature images, avoiding image normalization and solving information loss and storage problems. However, the SPP-Net algorithm's training process is isolated, requiring the storage of a large amount of intermediate results and consuming significant disk space. The Fast R-CNN algorithm uses regions of interest and multi-task loss functions to unify classification and regression, reducing disk space usage. However, Fast R-CNN still requires extracting candidate regions from the image, which is time-consuming. The Faster R-CNN algorithm first generates a feature image from the input image through convolutional pooling layers, then generates candidate regions through the RPN network, which are then fed into the RoI layer to generate a fixed-size feature vector. Finally, it performs classification and regression for recognition. Although the Faster R-CNN algorithm has high recognition accuracy, it relies on the features of the candidate regions for classification, which cannot meet real-time requirements.

[0005] The YOLO algorithm first divides the input image into a grid, then calculates the confidence score and classification probability of the presence of a target object within each grid cell. Simultaneously, it removes grid cells without target objects by applying a threshold. This allows for the simultaneous identification of the categories and locations of multiple items within an image, achieving end-to-end image recognition.

[0006] The YOLO algorithm runs fast, but when it comes to recognizing steel microstructures with uneven grayscale values ​​and blurred boundaries, it cannot effectively extract multi-scale information from the image, resulting in low recognition accuracy. Summary of the Invention

[0007] The present invention aims to at least partially solve one of the technical problems in the related art.

[0008] To address the problems existing in the identification of steel microstructures in current technologies, this invention proposes an impurity identification method for steel microstructures based on an improved FCN network. Through semantic feature extraction, channel feature fusion, cascaded feature output, and image fusion recognition modules, this method fully considers the relationships between pixels, makes full use of contextual information and multi-scale feature information, enhances pixel classification of micro-phase structures, and effectively improves the detection accuracy of micro-particles as well as the precision of identifying areas near grain boundaries and other boundaries.

[0009] Another objective of this invention is to provide an impurity identification device for steel microstructures based on an improved FCN network.

[0010] To achieve the above objectives, this invention proposes a method for impurity identification in steel microstructures based on an improved FCN network, comprising:

[0011] The microscopic image of the steel microstructure to be identified is input into the FCN network recognition model, and the features of the extracted steel microstructure microscopic image are sequentially processed by multiple residual modules to perform a first feature classification operation to obtain a low semantic feature map and a high semantic feature map.

[0012] The high semantic feature map is input into the CBAM attention mechanism module for a second feature classification operation of multi-dimensional pooling to obtain a channel feature map. This channel feature map is then combined with the high semantic feature map through a third feature classification operation of channel-wise weighting to obtain a channel attention feature map. The channel attention feature map is then classified and concatenated with the feature maps obtained by multi-dimensional pooling along the channel dimension to obtain a spatial attention feature map. Finally, a fourth feature classification operation is performed based on the spatial attention feature map and the channel attention feature map to obtain a fused feature map.

[0013] The fused feature map is input to the ASPP multi-layer concatenation module, so that the output features of the convolutional layer with low dilation rate are input to the convolutional layer with high dilation rate as input features. The input features of each dilated convolutional layer and the output features of all the previous parallel dilated convolutional layers are combined to perform the fifth feature classification operation as the input features of the current dilated convolutional layer, so as to obtain the final feature map based on the multi-layer concatenation output classification results.

[0014] The low semantic feature map and the final feature map are fused and stitched together using the sixth feature classification operation to obtain the restored feature map. The pixels of the restored feature map are then classified and identified to output the final target recognition result containing impurities.

[0015] In addition, the impurity identification method for steel microstructures based on the improved FCN network according to the above embodiments of the present invention may also have the following additional technical features:

[0016] Further, in one embodiment of the present invention, the plurality of residual modules include a first residual block, a second residual block, a third residual block, a fourth residual block, and a fifth residual block; the step of sequentially performing a first feature classification operation on the features of the extracted steel microstructure microimage through the plurality of residual modules to obtain a low semantic feature map and a high semantic feature map includes:

[0017] The microscopic image of the steel microstructure to be identified is input into the first residual block for feature extraction and max pooling operations to output a first size feature map;

[0018] A first-dimensional feature map is input to the second residual block to output a second-dimensional feature map. The second-dimensional feature map is input to the third residual block to output a third-dimensional feature map. The third-dimensional feature map is input to the fourth residual block to output a fourth-dimensional feature map. The fourth-dimensional feature map is input to the fifth residual block to output a fifth-dimensional feature map. The fifth-dimensional feature maps include the low-semantic feature map and the high-semantic feature map.

[0019] Further, in one embodiment of the present invention, the CBAM attention mechanism module includes a spatial attention module and a channel attention mechanism module; the second feature classification operation, which inputs the high semantic feature map into the CBAM attention mechanism module for multi-dimensional pooling, yields a channel feature map, which is then subjected to a third feature classification operation involving channel-wise weighting of the high semantic feature map to obtain a channel attention feature map; the feature maps obtained by multi-dimensional pooling along the channel dimension of the channel attention feature map are then classified and concatenated to obtain a spatial attention feature map; and a fourth feature classification operation is performed based on the spatial attention feature map and the channel attention feature map to obtain a fused feature map, including:

[0020] The high semantic feature map is input into the channel attention mechanism module, and global average pooling and max pooling operations are performed on the width and height respectively to obtain the initial channel feature map;

[0021] The initial channel feature map is input into two fully connected layers with shared weights, and the features output by the fully connected layers are stacked and then multiplied and weighted onto the high semantic feature map using the sigmoid activation function to obtain the channel feature map.

[0022] The channel feature map is input into the spatial attention module, and global average pooling and max pooling operations are performed on the channel dimension to output the channel attention feature map.

[0023] After concatenating the channel attention feature maps, a spatial attention feature map is generated through a convolutional layer and a sigmoid function. The spatial attention feature map is then multiplied with the features output by the channel attention mechanism module to obtain the fused feature map.

[0024] Furthermore, in one embodiment of the present invention, the fifth feature classification operation, which fuses and concatenates the low-semantic feature map and the final feature map to obtain the recovered feature map, includes:

[0025] The fused feature map is concatenated with the fifth-size feature map, and then an upsampling operation is performed to obtain the sixth-size feature map;

[0026] The sixth-size feature map is concatenated with the fourth-size feature map, and then an upsampling operation is performed to obtain the seventh-size feature map.

[0027] The seventh-size feature map is concatenated with the third-size feature map, and then an upsampling operation is performed to obtain the eighth-size feature map.

[0028] The ninth size feature map is obtained by concatenating the eighth size feature map with the second size feature map and then performing an upsampling operation.

[0029] The ninth-size feature map is stitched together with the first-size feature map, and the restored feature map is obtained based on the stitching result.

[0030] Furthermore, in one embodiment of the present invention, the sizes of the first size feature map, the second size feature map, the third size feature map, the fourth size feature map, and the fifth size feature map are respectively 1 / 2, 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the input microstructure micrograph; the sizes of the sixth size feature map, the seventh size feature map, the eighth size feature map, and the ninth size feature map are respectively 1 / 16, 1 / 8, 1 / 4, and 1 / 2 of the input microstructure micrograph.

[0031] To achieve the above objectives, another aspect of the present invention proposes an impurity identification device for steel microstructures based on an improved FCN network, comprising:

[0032] The initial feature classification module is used to input the microscopic image of the steel microstructure to be identified into the FCN network recognition model, so that the features of the extracted steel microstructure microscopic image are sequentially classified through multiple residual modules to obtain low semantic feature map and high semantic feature map.

[0033] The first feature classification module is used to input the high semantic feature map into the CBAM attention mechanism module for a second feature classification operation of multi-dimensional pooling to obtain a channel feature map, and perform a third feature classification operation of channel-wise weighting on the high semantic feature map to obtain a channel attention feature map. The feature maps after multi-dimensional pooling along the channel dimension of the channel attention feature map are classified and concatenated to obtain a spatial attention feature map. A fourth feature classification operation is performed based on the spatial attention feature map and the channel attention feature map to obtain a fused feature map.

[0034] The second feature classification module is used to input the fused feature map into the ASPP multi-layer cascade module, so that the output features of the convolutional layer with low dilation rate are input into the convolutional layer with high dilation rate as input features, and the features after performing the fifth feature classification operation on the input features of each dilated convolutional layer and the output features of all the previous parallel dilated convolutional layers are used as the input features of the current dilated convolutional layer, so as to obtain the final feature map according to the multi-layer cascade output classification results.

[0035] The target classification and recognition module is used to fuse and stitch the low semantic feature map and the final feature map using the sixth feature classification operation to obtain the restored feature map, and to classify and recognize the pixels of the restored feature map to output the final target recognition result containing impurities.

[0036] The impurity identification method and apparatus for steel microstructures based on the improved FCN network in this invention increases the ability to extract small-scale information, captures contextual information of targets and images at multiple scales, improves the reuse rate of feature information, and enhances the ability to extract detailed features of tiny particles and improves recognition accuracy.

[0037] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0038] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0039] Figure 1This is a flowchart of an impurity identification method for steel microstructures based on an improved FCN network according to an embodiment of the present invention;

[0040] Figure 2 This is a structural diagram of the improved FCN network identification model according to an embodiment of the present invention;

[0041] Figure 3 This is a sub-flowchart of step S1 according to an embodiment of the present invention;

[0042] Figure 4 This is a structural diagram of the block residual module according to an embodiment of the present invention;

[0043] Figure 5 This is a sub-flowchart of step S2 according to an embodiment of the present invention;

[0044] Figure 6 This is a structural diagram of the CBAM attention mechanism module according to an embodiment of the present invention;

[0045] Figure 7 This is a structural diagram of the ASPP multi-layer cascaded module according to an embodiment of the present invention;

[0046] Figure 8 This is a sub-flowchart of step S4 according to an embodiment of the present invention;

[0047] Figure 9 This is a schematic diagram of the structure of an impurity identification device for steel microstructures based on an improved FCN network according to an embodiment of the present invention. Detailed Implementation

[0048] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0049] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0050] The following description, with reference to the accompanying drawings, describes a steel microstructure segmentation method based on an improved FCN network, according to an embodiment of the present invention.

[0051] Figure 1 This is a flowchart of a steel microstructure segmentation method based on an improved FCN network, according to an embodiment of the present invention.

[0052] like Figure 1 As shown, the method includes, but is not limited to, the following steps:

[0053] S1, input the microscopic image of the steel microstructure to be identified into the FCN network recognition model, so that the features of the extracted steel microstructure microscopic image are sequentially classified through multiple residual modules to obtain low semantic feature map and high semantic feature map;

[0054] S2, input the high semantic feature map into the CBAM attention mechanism module to perform the second feature classification operation of multi-dimensional pooling to obtain the channel feature map, and perform the third feature classification operation of channel weighting on the high semantic feature map to obtain the channel attention feature map. Then, perform multi-dimensional pooling operation along the channel dimension on the feature map to obtain the spatial attention feature map. Finally, perform the fourth feature classification operation based on the spatial attention feature map and the channel attention feature map to obtain the fused feature map.

[0055] S3, input the fused feature map into the ASPP multi-layer concatenation module, so that the output features of the convolutional layer with low dilation rate are input into the convolutional layer with high dilation rate as input features, and the features after performing the fifth feature classification operation on the input features of each dilated convolutional layer and the output features of all the previous parallel dilated convolutional layers are used as the input features of the current dilated convolutional layer, so as to obtain the final feature map based on the multi-layer concatenation output classification results;

[0056] S4. The low semantic feature map and the final feature map are fused and stitched together using the sixth feature classification operation to obtain the restored feature map. The pixels of the restored feature map are then classified and identified to output the final target recognition result containing impurities.

[0057] Specifically, such as Figure 2As shown, the improved FCN network recognition model structure of this invention consists of two parts: an encoder and a decoder. The entire architecture comprises 5 residual modules, 5 pooling layers, 1 attention mechanism module (CBAM), 1 densely connected dilated convolutional pyramid module (ASPP), and 5 upsampling layers. The encoder replaces the convolutional layers in the original FCN network with residual modules and adds batch normalization layers to eliminate the gradient vanishing problem and promote faster convergence. The sixth convolutional layer is replaced with the CBAM attention mechanism module, effectively improving the model's ability to extract small-scale information. The seventh convolutional layer is replaced with the ASPP (dilated spatial pyramid pooling module). The densely connected dilated convolutional pyramid module uses cascaded combinations of convolutional kernels with multiple sampling rates and effective fields of view to increase the receptive field, capture contextual information of the target and image at multiple scales, and improve the reuse rate of feature information. The decoder fuses feature maps output from different downsampling layers at the encoder through skip connections, improving the model's perception of minute particle details. By replacing the first 5 convolutional layers with a residual module, the sixth convolutional layer with a CBAM attention mechanism module, and the seventh convolutional layer with an ASPP module, the ability to extract detailed features of tiny particles and the recognition accuracy are enhanced.

[0058] Furthermore, the multiple residual modules in step S1 above include a first residual block, a second residual block, a third residual block, a fourth residual block, and a fifth residual block; as shown... Figure 3 The diagram shown is a detailed flowchart of step S1 in an embodiment of the present invention, including:

[0059] S11, input the microscopic image of the steel microstructure to be identified into the first residual block for feature extraction and max pooling operation, so as to output the first size feature map;

[0060] S12, input a first-size feature map to the second residual block to output a second-size feature map, input the second-size feature map to the third residual block to output a third-size feature map, input the third-size feature map to the fourth residual block to output a fourth-size feature map, and input the fourth-size feature map to the fifth residual block to output a fifth-size feature map; wherein, the fifth-size feature maps include low-semantic feature maps and high-semantic feature maps.

[0061] Understandably, the microscopic images of steel microstructures undergo feature extraction via a backbone feature extraction network, outputting low-semantic and high-semantic feature maps of sizes corresponding to 1 / 2, 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the original image, respectively. The high-semantic feature map is then input into the CBAM attention mechanism module.

[0062] Specifically, Figure 4 This is a structural diagram of the Block module of the present invention, as shown below. Figure 4As shown, after the microscopic image of the steel microstructure undergoes residual block feature extraction and max pooling in the first block module, the output feature map size is 1 / 2 of the original image size. The reduced feature map is then used as the input for the second block, at which point the feature map becomes 1 / 4 of the original size. After inputting the third block, the feature map shrinks to 1 / 8 of the original size. After inputting the fourth block, the feature map shrinks to 1 / 16 of the original size. Finally, after inputting the fifth block, the feature map shrinks to 1 / 32 of the original size.

[0063] Furthermore, if Figure 5 The diagram shown is a detailed flowchart of step S2 in an embodiment of the present invention, including:

[0064] S21, input the high semantic feature map into the channel attention mechanism module, and perform global average pooling and max pooling operations on the width and height respectively to obtain the initial channel feature map;

[0065] S22, the initial channel feature map is input into two fully connected layers with shared weights, and the features output by the fully connected layers are stacked and then multiplied and weighted onto the high semantic feature map using the sigmoid activation function to obtain the channel feature map;

[0066] S23, input the channel feature map into the spatial attention module, and perform global average pooling and max pooling operations on the channel dimension to output the channel attention feature map;

[0067] S24. After concatenating the channel attention feature maps, a spatial attention feature map is generated through a convolutional layer and a sigmoid function. The spatial attention feature map is then multiplied with the features output by the channel attention mechanism module to obtain a fused feature map.

[0068] Specifically, Figure 6 This is a structural diagram of the CBAM attention mechanism module of the present invention, as shown below. Figure 6 As shown, the CBAM attention mechanism module consists of two parts: spatial attention and channel attention. The channel attention part performs global average pooling and max pooling operations on the input feature map for both width and height, then feeds it into two fully connected layers with shared weights. The features output from the shared network are then stacked and activated by a sigmoid function to obtain the channel feature map. This feature map then enters the spatial attention module, where global average pooling and max pooling operations are performed along the channel dimension. The two output feature maps are then concatenated, and finally, a convolutional layer is used to generate the spatial attention feature map. This attention map is then multiplied by the input feature map to obtain the final feature map. Introducing the attention mechanism results in richer and higher-level semantic features, and enhances the model's perception of subtle granular details.

[0069] Furthermore, Figure 7 This is a structural diagram of the ASPP multi-layer cascade module of the present invention, as shown below. Figure 7 As shown, the feature map result of the CBAM attention mechanism is input into the densely connected ASPP. In the densely connected ASPP structure, dilated convolutional layers with different dilation rates (6, 12, 18, 24) are combined in a cascaded manner. The output of the convolutional layer with a low dilation rate is fed into the input of the convolutional layer with a high dilation rate. The input of each dilated convolutional layer is concatenated with the outputs of all preceding parallel dilated convolutional layers as the input of the current dilated convolutional layer. Then, the multi-layer cascaded results are stacked and subjected to 1*1 convolution for dimensionality reduction to obtain the final feature map. The densely connected ASPP increases the receptive field while improving pixel utilization and passes detailed features to deeper network layers, thus improving the model's target recognition ability.

[0070] Furthermore, if Figure 8 The diagram shown is a detailed flowchart of step S4 in an embodiment of the present invention, including:

[0071] S41, after concatenating the fused feature map with the fifth-size feature map, an upsampling operation is performed to obtain the sixth-size feature map;

[0072] S42, after concatenating the sixth-size feature map with the fourth-size feature map, an upsampling operation is performed to obtain the seventh-size feature map;

[0073] S43, after concatenating the seventh-size feature map with the third-size feature map, an upsampling operation is performed to obtain the eighth-size feature map;

[0074] S44, after concatenating the eighth-size feature map with the second-size feature map, an upsampling operation is performed to obtain the ninth-size feature map;

[0075] S45, the ninth-size feature map is stitched together with the first-size feature map, and the restored feature map is obtained based on the stitching result.

[0076] Understandably, the decoding part introduces more low-level semantic features to obtain more detailed features, gradually fusing the feature information output from different downsampling layers in the encoding part and upsampling it to finally restore the image size. Finally, a softmax layer is used to classify the pixels in the restored feature image, and the final target recognition result is output.

[0077] Specifically, the feature map output by the densely connected ASPP is concatenated with the feature map output by the fifth block, and then upsampled to restore the feature map to 1 / 16 of the original image; it is then concatenated with the feature map output by the fourth block, and then upsampled to restore the feature map to 1 / 8 of the original image; it is then concatenated with the feature map output by the third block, and then upsampled to restore the feature map to 1 / 4 of the original image; it is then concatenated with the feature map output by the second block, and then upsampled to restore the feature map to 1 / 2 of the original image; finally, it is concatenated with the feature map output by the first block to restore the image size.

[0078] According to embodiments of the present invention, the impurity identification method for steel microstructures based on an improved FCN network replaces the convolutional layers in the original FCN network with residual modules and adds batch normalization layers to eliminate the gradient vanishing problem and promote faster convergence. The sixth convolutional layer is replaced with a CBAM attention mechanism module, effectively improving the model's ability to extract small-scale information. The seventh convolutional layer is replaced with an ASPP (Spatial Pyramid Pooling Module), a densely connected spatial pyramid convolutional module that uses cascaded convolutional kernels with multiple sampling rates and effective fields of view to increase the receptive field, capture contextual information of the target and image at multiple scales, and improve the reuse rate of feature information. The decoder fuses feature maps output from different downsampling layers at the encoder through skip connections, improving the model's perception of minute particle details. By replacing the first five convolutional layers with residual modules, the sixth convolutional layer with a CBAM attention mechanism module, and the seventh convolutional layer with an ASPP module, the ability to extract minute particle details and the recognition accuracy are enhanced.

[0079] To achieve the above embodiments, such as Figure 9 As shown, this embodiment also provides an impurity identification device 10 for steel microstructures based on an improved FCN network. The device 10 includes an initial feature classification module 100, a first feature classification module 200, a second feature classification module 300, and a target classification and identification module 400.

[0080] The initial feature classification module 100 is used to input the microscopic image of the steel microstructure to be identified into the FCN network recognition model, so that the features of the extracted steel microstructure microscopic image are sequentially classified through multiple residual modules to obtain low semantic feature map and high semantic feature map.

[0081] The first feature classification module 200 is used to input the high semantic feature map into the CBAM attention mechanism module for multi-dimensional pooling of the second feature classification operation to obtain the channel feature map, and perform a channel-wise weighted third feature classification operation on the high semantic feature map to obtain the channel attention feature map. The feature map after multi-dimensional pooling of the channel attention feature map along the channel dimension is classified and concatenated to obtain the spatial attention feature map. The fourth feature classification operation is performed based on the spatial attention feature map and the channel attention feature map to obtain the fused feature map.

[0082] The second feature classification module 300 is used to input the fused feature map into the ASPP multi-layer cascade module, so that the output features of the convolutional layer with low dilation rate are input into the convolutional layer with high dilation rate as input features, and the features after performing the fifth feature classification operation on the input features of each dilated convolutional layer and the output features of all the previous parallel dilated convolutional layers are used as the input features of the current dilated convolutional layer, so as to obtain the final feature map based on the multi-layer cascade output classification results.

[0083] The target classification and recognition module 400 is used to fuse and stitch the low semantic feature map and the final feature map using the sixth feature classification operation to obtain the restored feature map, and to classify and recognize the pixels of the restored feature map to output the final target recognition result containing impurities.

[0084] Furthermore, multiple residual modules, including a first residual block, a second residual block, a third residual block, a fourth residual block, and a fifth residual block; the aforementioned initial feature classification module 100 is also used for:

[0085] The microscopic image of the steel microstructure to be identified is input into the first residual block for feature extraction and max pooling operations to output a first-size feature map;

[0086] The first-size feature map is input into the second residual block to output the second-size feature map. The second-size feature map is input into the third residual block to output the third-size feature map. The third-size feature map is input into the fourth residual block to output the fourth-size feature map. The fourth-size feature map is input into the fifth residual block to output the fifth-size feature map. The fifth-size feature maps include low-semantic feature maps and high-semantic feature maps.

[0087] Furthermore, the CBAM attention mechanism module includes a spatial attention module and a channel attention mechanism module; the aforementioned first feature classification module 200 is also used for:

[0088] The high semantic feature map is input into the channel attention mechanism module, and global average pooling and max pooling operations are performed on the width and height respectively to obtain the initial channel feature map;

[0089] The initial channel feature map is input into two fully connected layers with shared weights. The features output from the fully connected layers are stacked and then multiplied and weighted onto the high semantic feature map using the sigmoid activation function to obtain the channel feature map.

[0090] The channel feature map is input into the spatial attention module, and global average pooling and max pooling operations are performed on the channel dimension to output the channel attention feature map.

[0091] After concatenating the channel attention feature maps, a spatial attention feature map is generated through a convolutional layer and a sigmoid function. The spatial attention feature map is then multiplied with the features output by the channel attention mechanism module to obtain a fused feature map.

[0092] Furthermore, the aforementioned target classification and recognition module 400 is also used for:

[0093] The fused feature map is concatenated with the fifth-size feature map and then upsampled to obtain the sixth-size feature map.

[0094] The seventh-size feature map is obtained by concatenating the sixth-size feature map with the fourth-size feature map and then performing an upsampling operation.

[0095] The seventh-size feature map is concatenated with the third-size feature map, and then upsampling is performed to obtain the eighth-size feature map.

[0096] The ninth-size feature map is obtained by concatenating the eighth-size feature map with the second-size feature map and then performing an upsampling operation.

[0097] The ninth-size feature map is stitched together with the first-size feature map, and the restored feature map is obtained based on the stitching result.

[0098] Furthermore, the sizes of the first, second, third, fourth, and fifth size feature maps are respectively 1 / 2, 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the input microstructure microimage; the sizes of the sixth, seventh, eighth, and ninth size feature maps are respectively 1 / 16, 1 / 8, 1 / 4, and 1 / 2 of the input microstructure microimage.

[0099] According to an embodiment of the present invention, an impurity identification device for steel microstructures based on an improved FCN network replaces the convolutional layers in the original FCN network with a residual module in the encoder and adds a batch normalization layer to eliminate the gradient vanishing problem and promote faster convergence. The sixth convolutional layer is replaced with a CBAM attention mechanism module, effectively improving the model's ability to extract small-scale information. The seventh convolutional layer is replaced with an ASPP (Spatial Pyramid Pooling Module), a densely connected spatial pyramid convolutional module that uses cascaded convolutional kernels with multiple sampling rates and effective fields of view to increase the receptive field, capture contextual information of the target and image at multiple scales, and improve the reuse rate of feature information. The decoder fuses feature maps output from different downsampling layers at the encoder through skip connections, improving the model's perception of minute particle details. By replacing the first five convolutional layers with a residual module, the sixth convolutional layer with a CBAM attention mechanism module, and the seventh convolutional layer with an ASPP module, the ability to extract minute particle details and the recognition accuracy are enhanced.

[0100] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0101] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

Claims

1. A method for impurity identification in steel microstructures based on an improved FCN network, characterized in that, Includes the following steps: The microscopic image of the steel microstructure to be identified is input into the FCN network recognition model, and the features of the extracted steel microstructure microscopic image are sequentially processed by multiple residual modules to perform a first feature classification operation to obtain a low semantic feature map and a high semantic feature map. The high semantic feature map is input into the CBAM attention mechanism module for a second feature classification operation of multi-dimensional pooling to obtain a channel feature map. This channel feature map is then combined with the high semantic feature map through a third feature classification operation of channel-wise weighting to obtain a channel attention feature map. The channel attention feature map is then classified and concatenated with the feature maps obtained by multi-dimensional pooling along the channel dimension to obtain a spatial attention feature map. Finally, a fourth feature classification operation is performed based on the spatial attention feature map and the channel attention feature map to obtain a fused feature map. The fused feature map is input to the ASPP multi-layer concatenation module, so that the output features of the convolutional layer with low dilation rate are input to the convolutional layer with high dilation rate as input features. The input features of each dilated convolutional layer and the output features of all the previous parallel dilated convolutional layers are combined to perform the fifth feature classification operation as the input features of the current dilated convolutional layer, so as to obtain the final feature map based on the multi-layer concatenation output classification results. The low semantic feature map and the final feature map are fused and stitched together using the sixth feature classification operation to obtain the restored feature map. The pixels of the restored feature map are then classified and identified to output the final target recognition result containing impurities.

2. The method according to claim 1, characterized in that, The plurality of residual modules includes a first residual block, a second residual block, a third residual block, a fourth residual block, and a fifth residual block; the step of sequentially performing a first feature classification operation on the features of the extracted steel microstructure microimage through the plurality of residual modules to obtain a low semantic feature map and a high semantic feature map includes: The microscopic image of the steel microstructure to be identified is input into the first residual block for feature extraction and max pooling operations to output a first size feature map; A first-dimensional feature map is input to the second residual block to output a second-dimensional feature map. The second-dimensional feature map is input to the third residual block to output a third-dimensional feature map. The third-dimensional feature map is input to the fourth residual block to output a fourth-dimensional feature map. The fourth-dimensional feature map is input to the fifth residual block to output a fifth-dimensional feature map. The fifth-dimensional feature maps include the low-semantic feature map and the high-semantic feature map.

3. The method according to claim 2, characterized in that, The CBAM attention mechanism module includes a spatial attention module and a channel attention mechanism module. The second feature classification operation, which inputs the high semantic feature map into the CBAM attention mechanism module for multi-dimensional pooling, yields a channel attention feature map. This channel attention feature map is then subjected to a third feature classification operation, which weights the high semantic feature map channel by channel, and the channel attention feature map is further multi-dimensionally pooled along the channel dimension before being concatenated to obtain a spatial attention feature map. Finally, a fourth feature classification operation is performed based on the spatial attention feature map and the channel attention feature map to obtain a fused feature map. This process includes: The high semantic feature map is input into the channel attention mechanism module, and global average pooling and max pooling operations are performed on the width and height respectively to obtain the initial channel feature map; The initial channel feature map is input into two fully connected layers with shared weights, and the features output by the fully connected layers are stacked and then multiplied and weighted onto the high semantic feature map using the sigmoid activation function to obtain the channel feature map. The channel feature map is input into the spatial attention module, and global average pooling and max pooling operations are performed on the channel dimension to output the channel attention feature map. After concatenating the channel attention feature maps, a spatial attention feature map is generated through a convolutional layer and a sigmoid function. The spatial attention feature map is then multiplied with the features output by the channel attention mechanism module to obtain the fused feature map.

4. The method according to claim 3, characterized in that, The fifth feature classification operation, which fuses and concatenates the low-semantic feature map and the final feature map to obtain the recovered feature map, includes: The fused feature map is concatenated with the fifth-size feature map, and then an upsampling operation is performed to obtain the sixth-size feature map; The sixth-size feature map is concatenated with the fourth-size feature map, and then an upsampling operation is performed to obtain the seventh-size feature map. The seventh-size feature map is concatenated with the third-size feature map, and then an upsampling operation is performed to obtain the eighth-size feature map. The ninth size feature map is obtained by concatenating the eighth size feature map with the second size feature map and then performing an upsampling operation. The ninth-size feature map is stitched together with the first-size feature map, and the restored feature map is obtained based on the stitching result.

5. The method according to claim 4, characterized in that, The sizes of the first, second, third, fourth, and fifth size feature maps are 1 / 2, 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the input microstructure microimage, respectively; the sizes of the sixth, seventh, eighth, and ninth size feature maps are 1 / 16, 1 / 8, 1 / 4, and 1 / 2 of the input microstructure microimage, respectively.

6. An impurity identification model for steel microstructures based on an improved FCN network, characterized in that, include: The initial feature classification module is used to input the microscopic image of the steel microstructure to be identified into the FCN network recognition model, so that the features of the extracted steel microstructure microscopic image are sequentially classified through multiple residual modules to obtain low semantic feature map and high semantic feature map. The first feature classification module is used to input the high semantic feature map into the CBAM attention mechanism module for a second feature classification operation of multi-dimensional pooling to obtain a channel feature map, and perform a third feature classification operation of channel-wise weighting on the high semantic feature map to obtain a channel attention feature map. The feature maps after multi-dimensional pooling along the channel dimension of the channel attention feature map are classified and concatenated to obtain a spatial attention feature map. A fourth feature classification operation is performed based on the spatial attention feature map and the channel attention feature map to obtain a fused feature map. The second feature classification module is used to input the fused feature map into the ASPP multi-layer cascade module, so that the output features of the convolutional layer with low dilation rate are input into the convolutional layer with high dilation rate as input features, and the features after performing the fifth feature classification operation on the input features of each dilated convolutional layer and the output features of all the previous parallel dilated convolutional layers are used as the input features of the current dilated convolutional layer, so as to obtain the final feature map according to the multi-layer cascade output classification results. The target classification and recognition module is used to fuse and stitch the low semantic feature map and the final feature map using the sixth feature classification operation to obtain the restored feature map, and to classify and recognize the pixels of the restored feature map to output the final target recognition result containing impurities.

7. The model according to claim 6, characterized in that, The plurality of residual modules include a first residual block, a second residual block, a third residual block, a fourth residual block, and a fifth residual block; the initial feature classification module is further configured to: The microscopic image of the steel microstructure to be identified is input into the first residual block for feature extraction and max pooling operations to output a first size feature map; A first-dimensional feature map is input to the second residual block to output a second-dimensional feature map. The second-dimensional feature map is input to the third residual block to output a third-dimensional feature map. The third-dimensional feature map is input to the fourth residual block to output a fourth-dimensional feature map. The fourth-dimensional feature map is input to the fifth residual block to output a fifth-dimensional feature map. The fifth-dimensional feature maps include the low-semantic feature map and the high-semantic feature map.

8. The model according to claim 7, characterized in that, The CBAM attention mechanism module includes a spatial attention module and a channel attention mechanism module; the first feature classification module is further used for: The high semantic feature map is input into the channel attention mechanism module, and global average pooling and max pooling operations are performed on the width and height respectively to obtain the initial channel feature map; The initial channel feature map is input into two fully connected layers with shared weights, and the features output by the fully connected layers are stacked and then multiplied and weighted onto the high semantic feature map using the sigmoid activation function to obtain the channel feature map. The channel feature map is input into the spatial attention module, and global average pooling and max pooling operations are performed on the channel dimension to output the channel attention feature map. After concatenating the channel attention feature maps, a spatial attention feature map is generated through a convolutional layer and a sigmoid function. The spatial attention feature map is then multiplied with the features output by the channel attention mechanism module to obtain the fused feature map.

9. The model according to claim 8, characterized in that, The target classification and recognition module is also used for: The fused feature map is concatenated with the fifth-size feature map, and then an upsampling operation is performed to obtain the sixth-size feature map; The sixth-size feature map is concatenated with the fourth-size feature map, and then an upsampling operation is performed to obtain the seventh-size feature map. The seventh-size feature map is concatenated with the third-size feature map, and then an upsampling operation is performed to obtain the eighth-size feature map. The ninth size feature map is obtained by concatenating the eighth size feature map with the second size feature map and then performing an upsampling operation. The ninth-size feature map is stitched together with the first-size feature map, and the restored feature map is obtained based on the stitching result.

10. The model according to claim 9, characterized in that, The sizes of the first, second, third, fourth, and fifth size feature maps are 1 / 2, 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the input microstructure microimage, respectively; the sizes of the sixth, seventh, eighth, and ninth size feature maps are 1 / 16, 1 / 8, 1 / 4, and 1 / 2 of the input microstructure microimage, respectively.

Images