Sheet metal part visual identification method based on improved Mask R-CNN

Abstract

The invention discloses a sheet metal part visual identification method based on an improved MaskR-CNN, and relates to a sheet metal part visual identification method. The method comprises the following steps: a PA-CBAM segmentation model: optimizing the focus of the model, and generating a feature map with higher quality by an improved network to improve the recognition precision of a sheet metal part; multi-scale feature fusion: the model can still accurately identify the part under the condition that the edge is not clear; and the channel space is mixed with attention, so that feature channels which are more important to the current task are highlighted. According to the method, paths from bottom to top and cross-layer fast connection are added in an original feature pyramid network (FPN), and a space channel mixed attention module (CABAM) is integrated, so that the multi-scale feature fusion capability and the edge detection precision of the model are remarkably improved. In the visual identification task of sheet metal parts, the proposed method realizes significant performance improvement.

Description

Technical Field

[0001] The present invention relates to a visual recognition method for sheet metal parts, and in particular to a visual recognition method for sheet metal parts based on an improved Mask R-CNN. Background Art

[0002] Visual recognition of sheet metal parts is an important task in industrial inspection. Since sheet metal parts usually have complex geometric shapes and fine edge features, traditional instance segmentation methods often face challenges in such tasks. In practical applications, factors such as changing lighting conditions and background interference can cause edge blur, thus affecting the accuracy of recognition. Therefore, how to effectively detect and segment sheet metal parts, especially in complex environments, has become a challenging research problem.

[0003] In the actual sorting process of sheet metal parts, visual recognition technology is often used in automatic sorting systems, which rely on precise object detection and instance segmentation algorithms to identify, classify and locate parts. Sheet metal parts sorting is particularly prominent in vision-guided sorting on steel production lines. Vision-guided precision sorting technology can significantly reduce manual intervention, improve production efficiency and increase accuracy. As the steel plate passes through the production line, the sorting system uses a camera to capture images of sheet metal parts at different angles and lighting conditions, which are often affected by reflections, shadows and background complexity. Due to complex geometric shapes, surface gloss changes and interference from the surrounding environment, sheet metal parts in industrial production may produce blurred edges and reflection noise in actual applications. These factors make it difficult for traditional instance segmentation algorithms to accurately capture the edges and details of parts, especially under extreme lighting conditions.

[0004] To address these issues, a PA-CBAM model based on an improved Mask R-CNN is proposed. The traditional FPN architecture often exhibits a high false detection rate under extreme lighting conditions, while the proposed model strengthens multi-scale feature fusion by adding a bottom-up path and introducing cross-layer "shortcut" connections, further enhancing the information flow and fusion between feature maps and improving the detection of multi-scale objects. In addition, the improved multi-scale fusion output feature map is integrated with CBAM (Convolutional Block Attention Module). This module improves the model's attention to key feature channels and important spatial locations through channel and spatial attention mechanisms. Combining CBAM enables the model to better capture the edges and details of sheet metal parts, especially under complex lighting and background interference conditions. Summary of the invention

[0005] The purpose of the present invention is to provide a sheet metal parts visual recognition method based on an improved Mask R-CNN, which improves the multi-scale feature fusion capability and edge detection accuracy of the model by adding bottom-up paths and cross-layer "shortcut" connections in the original feature pyramid network (FPN) and integrating a spatial channel hybrid attention module (CABAM), thereby achieving significant performance improvement in the visual recognition task of sheet metal parts.

[0006] The objective of the present invention is achieved through the following technical solutions:

[0007] A sheet metal parts visual recognition method based on improved Mask R-CNN, the method comprising the following components:

[0008] (1) PA-CBAM segmentation model:

[0009] The original Feature Pyramid Network (FPN) is enhanced by adding a bottom-up path and introducing cross-layer "shortcut" connections, improving the model's ability to fuse multi-scale features, allowing better capture of objects of various scales, and by aggregating features at different levels, enhancing the model's ability to identify the edges and fine details of sheet metal parts. The Convolutional Block Attention Module (CBAM) is integrated into the output feature map of the improved FPN, and the focus of the model is further optimized through the channel and spatial attention mechanisms. The improved network generates higher quality feature maps to improve the recognition accuracy of sheet metal parts;

[0010] (2) Multi-scale feature fusion:

[0011] The introduction of the Path Aggregation Network (PANet) further enhances the multi-scale fusion capability of feature maps by adding a bottom-up path and adding "short-cut" connections between different layers of the feature pyramid. This enables the model to obtain stronger feature expression capabilities. For the visual recognition of sheet metal parts, illumination changes often lead to unclear edges of parts. The multi-scale feature fusion strategy effectively enhances the ability to capture the edge of the target by combining features of different scales, so that the model can still accurately identify parts even when the edges are unclear.

[0012] Through the top-down pathway, high-level features (such as features with rich semantic information) are effectively fused into low-level features (such as features with high resolution), making the model more robust in detecting and segmenting objects of different sizes.

[0013] The bottom-up path further integrates low-level features (features rich in detail information) into high-level features. This process enhances the detail information in the feature map and helps the model better capture the edges and detail features of sheet metal parts.

[0014] (3) Channel-space mixed attention:

[0015] After enhancing multi-scale feature fusion, CBAM (Convolutional Block Attention Module) is introduced to further optimize the model's attention allocation to important features; the goal of the channel attention module is to assign an attention weight to each channel of the feature map, thereby highlighting the feature channels that are more important to the current task.

[0016] The sheet metal parts visual recognition method based on the improved Mask R-CNN, the output feature map formula of the improved network is as follows:

[0017]

[0018] in is the output feature map after multi-scale fusion and attention mechanism; M s is the spatial attention map; M c is the channel attention map. Fusion is the feature map after multi-scale fusion.

[0019] In the sheet metal parts visual recognition method based on improved Mask R-CNN, the top-down path feature fusion formula is:

[0020]

[0021] in and They are the feature maps of the lth and l+1th layers of the top-down path, respectively, and F l It is the residual input feature map of the lth layer.

[0022] The sheet metal parts visual recognition method based on improved Mask R-CNN, the bottom-up path feature fusion formula is:

[0023]

[0024] in and They are the feature maps of the lth layer and the l-1th layer of the bottom-up path, respectively. is the feature map of the lth layer of the top-down path;

[0025] The final formula of the multi-scale fusion feature map is as follows:

[0026]

[0027] The described sheet metal parts visual recognition method based on improved Mask R-CNN, the CBAM module includes a channel attention module (CAM) and a spatial attention module (SAM), which suppresses unimportant features while retaining important features.

[0028] The advantages and effects of the present invention are:

[0029] This paper proposes a sheet metal parts visual recognition method based on improved Mask R-CNN, called PA-CBAM. By adding bottom-up paths and cross-layer "shortcut" connections to the original feature pyramid network (FPN) and integrating the spatial channel hybrid attention module (CABAM), this method significantly improves the model's multi-scale feature fusion capability and edge detection accuracy. In the visual recognition task of sheet metal parts, the proposed method achieves significant performance improvements. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 The multi-scale feature fusion map of the present invention

[0031] Figure 2 This is the channel space mixed attention module diagram of the present invention. DETAILED DESCRIPTION

[0032] The present invention will be described in detail below with reference to the embodiments shown in the accompanying drawings.

[0033] The present invention discloses a sheet metal parts visual recognition model PA-CBAM based on an improved Mask R-CNN. The original feature pyramid network (FPN) is enhanced by adding a bottom-up path and introducing a cross-layer "shortcut" connection, which improves the model's ability to fuse multi-scale features. This improvement allows for better capture of objects of various scales, and by aggregating features at different levels, significantly enhances the model's ability to identify edges and fine details of sheet metal parts. In addition, a convolutional block attention module (CBAM) is integrated into the output feature map of the improved FPN, and the focus of the model is further optimized through channel and spatial attention mechanisms. This attention mechanism is particularly suitable for the task of sheet metal parts classification, enabling the model to effectively handle uneven lighting and complex backgrounds, thereby reducing the false detection rate caused by edge blur.

[0034] The improved network is able to generate higher quality feature maps, which helps improve the recognition accuracy of sheet metal parts. Specifically, under different lighting conditions, the model is better able to adapt to different lighting environments and accurately capture edge information. The output feature map formula of the improved network is as follows:

[0035]

[0036] in It is the output feature map after multi-scale fusion and attention mechanism. s is the spatial attention map. c is the channel attention map. Fusion is the feature map after multi-scale fusion.

[0037] The output feature map of the improved network has enhanced feature selection and localization capabilities, providing more accurate feature input for the subsequent ROIHead. This improves the performance of object detection and instance segmentation. The following section describes the multi-scale feature fusion and channel space hybrid attention mechanism in detail.

[0038] Multi-scale feature fusion of the present invention

[0039] like Figure 1 As shown in the figure, compared with the original FPN, the introduction of the path aggregation network (PANet) further enhances the multi-scale fusion capability of the feature map by adding a bottom-up path and adding "short-cut" connections between different layers of the feature pyramid. This enables the model to obtain stronger feature expression capabilities. For the visual recognition of sheet metal parts, illumination changes often lead to unclear edges of parts. The multi-scale feature fusion strategy effectively enhances the ability to capture the edge of the target by combining features of different scales, so that the model can still accurately identify parts even when the edges are unclear.

[0040] Through the top-down path, high-level features (such as features with rich semantic information) can be effectively fused into low-level features (such as features with high resolution), which makes the model more robust when detecting and segmenting objects of different sizes. The formula for feature fusion in the top-down path is:

[0041]

[0042] in and They are the feature maps of the lth and l+1th layers of the top-down path, respectively, and F l It is the residual input feature map of the lth layer.

[0043] The bottom-up path further fuses low-level features (features rich in detail information) into high-level features. This process enhances the detail information in the feature map and helps the model better capture the edges and detail features of sheet metal parts. Bottom-up path feature fusion formula:

[0044]

[0045] in and They are the feature maps of the lth layer and the l-1th layer of the bottom-up path, respectively. is the feature map of the lth layer in the top-down path.

[0046] The final formula of the multi-scale fusion feature map is as follows:

[0047]

[0048] The present invention's channel space mixed attention:

[0049] Figure 1 Multi-scale feature fusion graph, after enhancing multi-scale feature fusion, CBAM (Convolutional Block Attention Module) is introduced to further optimize the model's attention allocation to important features. For the visual recognition of sheet metal parts, by better allocating attention to useful feature channels and spatial positions, the model's response to edges and important features is enhanced, and background noise is effectively suppressed, so that the model can still maintain high-precision recognition and segmentation performance under complex backgrounds. The CBAM module includes a channel attention module (CAM) and a spatial attention module (SAM), and its main purpose of design is to suppress unimportant features while retaining important features.

[0050] The goal of the channel attention module is to assign an attention weight to each channel of the feature map, thereby highlighting the feature channels that are more important to the current task. Where C is the number of channels, H and W are the height and width of the feature map, respectively. Global average pooling (GAP) and global maximum pooling (GMP) are performed on the feature map to obtain two 1D feature vectors, which are then passed through a shared two-layer fully connected network (MLP). The output formula is:

[0051]

[0052] Among them, σ(·) is the Sigmoid activation function, M c ∈R C is the channel attention map.

[0053] Use channel attention map M c The formula for weighting the input feature map channel by channel is:

[0054]

[0055] F′ is the feature map after channel attention is applied. Figure 2 It is a channel-space mixed attention module.

[0056] The goal of the spatial attention module is to assign an attention weight to each spatial position of the feature map, thereby highlighting the feature positions that are more important to the current task. Input the feature map F′∈R after channel attention weighting CxHxW , perform global average pooling and global maximum pooling operations (along the channel dimension) on the feature map to generate two 2D feature maps, concatenate the above two feature maps in the channel dimension, and generate a spatial attention map through a 7x7 convolution operation. The output formula is:

[0057] M s =σ(Conv 7x7 ([GAP(F′);GMP(F′)])) (7)

[0058] Among them, Conv 7x7 (·) is the 7x7 convolution operation, σ(·) is the Sigmoid activation function, M c ∈R HxW is the spatial attention map.

[0059] Use spatial attention map M s The feature map F′ weighted by channel attention is weighted position by position, and the formula is:

[0060]

[0061] is the final CBAM output feature map after applying channel and spatial attention.

Claims

1. A sheet metal parts visual recognition method based on improved MaskR-CNN, characterized in that: The method comprises the following components: (1) PA-CBAM segmentation model: The original Feature Pyramid Network (FPN) is enhanced by adding a bottom-up path and introducing cross-layer "shortcut" connections, improving the model's ability to fuse multi-scale features, allowing better capture of objects of various scales, and by aggregating features at different levels, enhancing the model's ability to identify the edges and fine details of sheet metal parts. The Convolutional Block Attention Module (CBAM) is integrated into the output feature map of the improved FPN, and the focus of the model is further optimized through the channel and spatial attention mechanisms. The improved network generates higher quality feature maps to improve the recognition accuracy of sheet metal parts; (2) Multi-scale feature fusion: Introduction The Path Aggregation Network (PANet) further enhances the multi-scale fusion capability of feature maps by adding a bottom-up path and adding "short-cut" connections between different layers of the feature pyramid. This enables the model to obtain stronger feature expression capabilities. For the visual recognition of sheet metal parts, illumination changes often lead to unclear edges of parts. The multi-scale feature fusion strategy effectively enhances the ability to capture the edge of the target by combining features of different scales, so that the model can still accurately identify parts even when the edges are unclear. Through the top-down path, high-level features (such as features with rich semantic information) are effectively fused into low-level features (such as features with high resolution), making the model more robust in detecting and segmenting objects of different sizes. The bottom-up path further integrates low-level features (features rich in detail information) into high-level features. This process enhances the detail information in the feature map, which helps the model better capture the edges and detail features of sheet metal parts. (3) Channel-space mixed attention: After enhancing multi-scale feature fusion, CBAM (Convolutional Block Attention Module) is introduced to further optimize the model's attention allocation to important features; the goal of the channel attention module is to assign an attention weight to each channel of the feature map, thereby highlighting the feature channels that are more important to the current task.

2. According to claim 1, a sheet metal parts visual recognition method based on improved MaskR-CNN is characterized in that: The output feature map formula of the improved network is as follows: in is the output feature map after multi-scale fusion and attention mechanism; M s is the spatial attention map; M c is the channel attention map; Fusion is the feature map after multi-scale fusion.

3. The sheet metal parts visual recognition method based on improved MaskR-CNN according to claim 1, characterized in that: The top-down path feature fusion formula is: in and They are the feature maps of the lth and l+1th layers of the top-down path, respectively, and F l It is the residual input feature map of the lth layer.

4. The sheet metal parts visual recognition method based on improved MaskR-CNN according to claim 1, characterized in that: The bottom-up path feature fusion formula: in and They are the feature maps of the lth layer and the l-1th layer of the bottom-up path, respectively. is the feature map of the lth layer of the top-down path; The final formula of the multi-scale fusion feature map is as follows:

5. The sheet metal parts visual recognition method based on improved MaskR-CNN according to claim 1, characterized in that: The CBAM module includes a channel attention module (CAM) and a spatial attention module (SAM), which suppresses unimportant features while retaining important features.

Images