Leather defect detection method based on parallel branch feature extraction

Abstract

The invention relates to the technical field of deep learning model algorithms, in particular to a leather defect detection method based on parallel branch feature extraction, which is characterized by comprising the following steps of: firstly, integrating a parallel branch in an original SegFormer encoder-decoder framework; a specially designed feature extraction module based on an EMA attention mechanism in the parallel branch can convert an original input image into a global feature map; secondly, adopting a staged feature fusion strategy of progressive context guide fusion to realize layered and progressive up-sampling feature fusion, fusing multi-scale features in a cascade mode by using PCGF, and then, further integrating the features with global features through parallel branches so as to reserve detail information on different scales; a leather surface defect image data set is constructed based on collected original leather data. According to the method, the problem of semantic discontinuity or detail loss occurring when an existing deep learning model faces a global leather surface defect segmentation task is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of deep learning model algorithms, and particularly relates to a leather defect detection method based on parallel branch feature extraction. BACKGROUND

[0002] Leather surface defects have diverse morphologies and complex textures, making them extremely difficult to identify using traditional methods. Therefore, it is crucial to find advanced methods for defect detection to ensure quality.

[0003] When dealing with complex backgrounds or high-noise leather images, performance is often poor. Threshold segmentation and edge detection are suitable for handling defects with clear contrasts, while texture analysis and morphological operations help extract leather surface features. However, these methods have limited effectiveness when faced with complex textures and small defects. Additionally, machine learning-based classification methods, while improving accuracy, rely on manual feature extraction and have limitations in the presence of new defects and complex backgrounds.

[0004] Models based on deep learning technology can automatically learn features from data through the multi-layer structure of neural networks without human intervention. This allows deep learning to handle complex, non-linear relationships and adaptively extract the most relevant features from data. However, the computational complexity of deep learning grows quadratically with input resolution, making it inefficient for high-resolution image processing.

[0005] While this local attention helps capture fine-grained features, it weakens the model's ability to perceive global context, leading to semantic discontinuity or loss of detail in segmentation results in complex background or fuzzy boundary regions. SUMMARY

[0006] To address the problem of semantic discontinuity or loss of detail in existing deep learning models when faced with global leather surface defect segmentation tasks, the present application proposes a leather defect detection method based on parallel branch feature extraction.

[0007] To achieve the above object and other related objects, the present application adopts the following technical solution:

[0008] A leather defect detection method based on parallel branch feature extraction,

[0009] First, a parallel branch is integrated into the original SegFormer encoder-decoder framework. The specially designed feature extraction module based on EMA attention mechanism in the parallel branch can convert the original input image into a global feature map.

[0010] Secondly, the gradual context-guided fusion of the phased feature fusion strategy is adopted to realize the hierarchical and gradual up-sampling feature fusion, and the PCGF is used to fuse the multi-scale features in a cascaded manner. Subsequently, these features are further integrated with the global features through parallel branches, so as to retain the detail information at different scales.

[0011] Based on the collected original leather data, a leather surface defect image dataset is constructed.

[0012] Preferably, the parallel branch first performs linear mapping and down-sampling on the input image through PatchEmbedding. This process divides the input image into several small blocks, each of which contains part of the information of the image.

[0013] Each small block is mapped to a high-dimensional vector, so that the spatial information of the original image becomes a low-resolution feature representation. This process not only reduces the size of the input image, but also converts the spatial information of the image into a format suitable for processing by the Transformer network.

[0014] Secondly, the EMA attention mechanism is introduced to enhance the global modeling capability of the network for spatial features.

[0015] After the EMA output, a depth separable convolution and an MLP layer are introduced.

[0016] Preferably, the EMA module is introduced in the feature extraction stage, including: the EMA module first divides the input feature map into G channel groups to balance the distribution of semantic information; subsequently, the 1×1 branch adopts direction-aware one-dimensional global average pooling to capture long-range dependencies through parallel subnetwork processing, while the 3×3 branch extracts local multi-scale interactions through convolution. Finally, cross-spatial dot product fusion is used to generate pixel-level attention weights.

[0017] Preferably, the features output by the parallel branch will be fused with the multi-scale features output by the backbone network in the PCGF.

[0018] Preferably, first, the low-resolution feature map is interpolated through up-sampling, and then the feature maps of different sizes are fused to achieve more accurate leather surface defect segmentation. A convolution layer is used to adjust the number of channels of a feature map so that the number of channels is the same as that of another feature map. After that, the two feature maps are concatenated along the channel dimension to form a new feature map The concatenated feature map will contain feature information from two different sources. By using SEAttention, a channel-level weight w is generated, which represents the importance of each channel in the feature map. The output of SEAttention is a weighted feature map with the same shape The module will use the weighted feature map The feature is divided into two parts based on the number of channels and corresponding channel weights. By applying the weights to the original features, the weighted features are obtained. Finally, a feature fusion operation is performed, which adds the weighted features to the original features to obtain the final fused features.

[0019] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0020] 1. This invention utilizes a parallel branch, within which a specially designed feature extraction module based on the EMA attention mechanism can convert the original input image into a global feature map. Therefore, the parallel branch can effectively capture global information for subsequent feature fusion.

[0021] 2. In this invention, the progressive context-guided fusion (PCGF) staged feature fusion strategy utilizes PCGF to fuse multi-scale features in a cascaded manner. Subsequently, these features are further integrated with global features through parallel branches, thereby preserving detailed information at different scales. This integration of "local-global" information allows the model to simultaneously focus on pixel-level details and the overall image context. Therefore, it significantly improves segmentation performance in challenging scenarios such as large-scale object segmentation or complex background scenes.

[0022] 3. A parallel feature extraction branch is incorporated, embedding an efficient multi-scale attention module (EMA) to enhance global information modeling capabilities. Simultaneously, the self-attention branch of the backbone network retains high sensitivity to local structure and texture information. The feature maps generated by the two branches, after fusion, achieve complementary representations of global and local information, enabling the model to maintain a lightweight structure while possessing stronger edge detail recovery and complex defect recognition capabilities.

[0023] 4. To further enhance the network's global modeling capability of spatial features while reducing the computational overhead of traditional attention mechanisms, this invention introduces an EMA module in the feature extraction stage. This module, through channel grouping and cross-spatial learning mechanisms, can effectively capture multi-scale global context at each layer, avoiding channel compression and information loss in traditional attention mechanisms (such as SE or CBAM), thereby improving the model's ability to model long-range dependencies. Compared to traditional attention mechanisms, EMA avoids channel compression by aggregating contextual information in parallel through grouping, further optimizing the computational complexity from O(C×H×W) to a lower overhead.

[0024] Other additional advantages and benefits of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0025] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0026] Figure 1 These are comparative illustrations of leather detection effects in embodiments of this application. Detailed Implementation

[0027] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings. The following specific examples illustrate the embodiments of the present invention, and those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, unless otherwise specified, the following embodiments and features in the embodiments can be combined with each other.

[0028] Example:

[0029] This invention discloses a leather defect detection method based on parallel branch feature extraction.

[0030] First, a parallel network branch is introduced, embedding an EMA attention mechanism to assist the model in acquiring richer global features. Second, to mitigate the loss of target feature information during decoder reconstruction, a hierarchical context-guided fusion is proposed to achieve layered and progressive upsampling feature fusion. This application evaluates the proposed method on a self-built leather defect dataset. Compared to the SegFormer model, in a certain experimental measurement, with only a 10% increase in the number of parameters, the proposed method improves the average intersection-union ratio (MIOU) and pixel accuracy (Acc) by 17.2% and 1.68%, respectively.

[0031] The architecture of this solution integrates a parallel branch into the original SegFormer encoder-decoder framework. The specially designed feature extraction module based on the EMA attention mechanism in this parallel branch can convert the original input image into a global feature map. Therefore, the parallel branch can effectively capture global information for subsequent feature fusion.

[0032] Secondly, a phased feature fusion strategy using Progressive Context-Guided Fusion (PCGF) is employed to achieve hierarchical and progressive upsampling feature fusion. PCGF is used to fuse multi-scale features in a cascaded manner. Subsequently, these features are further integrated with global features through parallel branches, thereby preserving detailed information at different scales. This integration of "local-global" information allows the model to simultaneously focus on pixel-level details and the overall image context. Therefore, it significantly improves segmentation performance in challenging scenarios such as large-scale object segmentation or complex background scenes.

[0033] To meet the company's application needs and verify the effectiveness of the proposed method, this application constructs a leather surface defect image dataset based on the raw leather data collected by the company. Experimental results show that the proposed model achieves excellent performance in practical leather surface defect detection tasks.

[0034] Specifically, this application designs a parallel feature extraction branch in the encoder stage of SegFormer, embedding an efficient multi-scale attention module (EMA) to enhance global information modeling capabilities. Simultaneously, the self-attention branch of the backbone network retains high sensitivity to local structure and texture information. The feature maps generated by the two branches, after fusion, achieve complementary representations of global and local information, enabling the model to maintain a lightweight structure while possessing stronger edge detail recovery and complex defect recognition capabilities. Specifically, the parallel network branch first performs linear mapping and downsampling on the input image through PatchEmbedding. This process divides the input image into several small patches, each containing a portion of the image's information. Each patch is mapped to a high-dimensional vector, thus transforming the spatial information of the original image into a low-resolution feature representation. This process not only reduces the size of the input image but also transforms the image's spatial information into a format suitable for Transformer network processing. Secondly, this application enhances the network's global modeling capability for spatial features by introducing an EMA attention mechanism. After the EMA output, depthwise separable convolution (DWConv) and MLP layers are introduced to enhance the local spatial receptive field while maintaining computational efficiency.

[0035] To further enhance the network's ability to model spatial features globally while reducing the computational overhead of traditional attention mechanisms, this paper introduces an EMA module in the feature extraction stage. This module can effectively capture multi-scale global context at each layer through channel grouping and cross-spatial learning mechanisms, avoiding channel compression and information loss in traditional attention mechanisms (such as SE or CBAM), thereby improving the model's ability to model long-range dependencies.

[0036] Specifically, the EMA module first takes the input feature map The semantic information is divided into G channelgroups (typically G=32) to distribute semantic information evenly. Then, it is processed through parallel sub-networks: the 1×1 branch uses direction-aware one-dimensional global average pooling to capture long-range dependencies, while the 3×3 branch extracts local multi-scale interactions through convolution. Finally, pixel-level attention weights are generated using cross-spatial dot product fusion. The core operation can be described as follows:

[0037] ;

[0038] Wherein, GAP represents global average pooling. and The outputs are 1×1 and 3×3 branches respectively. "·" represents sigmoid activation, and "·" represents matrix dot product. Through this fusion, EMA ensures the model continuously focuses on important global and local information at different levels and reduces sensitivity to noise and inconsistencies. This allows the model to more stably maintain its perception of multi-scale features even in the face of input noise or structural instability, thus improving its robustness. Furthermore, the EMA mechanism effectively captures long-distance dependencies when processing large-scale images (such as high-resolution semantic segmentation), avoiding the limitations of traditional attention mechanisms with multi-scale information. Therefore, EMA significantly improves the model's performance on large-scale image tasks, especially in detail recovery and global consistency. Through parallel branching and grouping strategies, the EMA module can "integrate" multi-scale information at each layer, which is particularly important for tasks requiring global consistency and fine segmentation. Compared to traditional attention mechanisms, EMA avoids channel compression by aggregating contextual information in parallel through grouping, further optimizing computational complexity from O(C×H×W) to a lower overhead. Features output by parallel branches... In PCGF, the multi-scale features output by the backbone network will be combined. , , , To integrate.

[0039] A hierarchical context-guided fusion strategy is proposed to enhance the feature representation capability of the network. First, low-resolution feature maps are upsampled and interpolated. Then, feature maps of different sizes are fused to achieve more accurate segmentation of leather surface defects.

[0040] Suppose the two input feature maps are respectively and ,That , , representing feature maps from different network layers, and B is the batch size. and H is the number of channels in the feature map, and H and W are the height and width of the feature map. If... This application adjusts through a convolutional layer. The number of channels, making it consistent with same.

[0041] After adjusting the number of channels through convolutional layers, the features and They are concatenated along the channel dimension to form a new feature map. This concatenated feature map will contain feature information from two different sources. To make the fusion process more discriminative, the module introduces a channel attention mechanism. Channel-level weights w are generated using SEAttention, representing the importance of each channel in the feature map. The output of SEAttention is a... Weighted feature maps with the same shape. (Assume...) The weighted feature map is obtained by assigning weights to each channel. Weighted feature maps were generated using SEAttention. Then, the module divides the feature map into two parts according to the number of channels: corresponding to... and The channel weights are then applied to the original features to obtain the weighted features:

[0042] ;

[0043] “ The symbol "" indicates element-wise multiplication.

[0044] Finally, a feature fusion operation is performed, adding the weighted features to the original features to obtain the final fused features: ;

[0045] This operation ensures that the fused features retain both the spatial information of low-level features and the semantic information of high-level features. In this way, the model can effectively integrate features from different levels.

[0046] Ablation experiments were conducted on a self-built leather surface defect dataset. This application also compared several excellent classic and advanced algorithms. In the lateral occlusion type, RT-former, SFNet, DeepLabv3P, and HardNet all exhibited semantic discontinuities, such as... Figure 1As shown in the third column, Seg-former and DeepLabv3p also exhibited misclassification issues. In surface damage defects on small targets, apart from the method in this application and Seg-former, the remaining methods failed to accurately segment the edge portions of the defects. For example... Figure 1 As shown in column 1.

[0047] The designed parallel network branches, through the embedding of the EMA attention mechanism, can effectively extract global semantic information. By progressively fusing the additional features acquired by the network branches with the features acquired by the backbone network, a more representative global feature representation can be obtained, improving the accuracy of semantic segmentation. The hierarchical context-guided fusion strategy effectively enhances the representational power after feature fusion. Furthermore, the proposed method is more applicable and effective in resource-constrained scenarios compared to other networks.

[0048] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. For those skilled in the art, the present invention can have various modifications and variations. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A leather defect detection method based on parallel branch feature extraction, characterized in that: First, a parallel branch is integrated into the original SegFormer encoder-decoder framework. The specially designed feature extraction module based on the EMA attention mechanism in this parallel branch can convert the original input image into a global feature map. Secondly, a phased feature fusion strategy with progressive context-guided fusion is adopted to achieve hierarchical and progressive upsampling feature fusion. PCGF is used to fuse multi-scale features in a cascade manner. Subsequently, these features are further integrated with global features through parallel branches, thereby preserving detailed information at different scales. A dataset of images of leather surface defects was constructed based on the collected raw leather data.

2. The leather defect detection method based on parallel branch feature extraction according to claim 1, characterized in that, The parallel branch first performs linear mapping and downsampling on the input image through PatchEmbedding. This process divides the input image into several small blocks, each of which contains a portion of the image information. Each small block is mapped to a high-dimensional vector, so that the spatial information of the original image becomes a low-resolution feature representation. This process not only reduces the size of the input image, but also transforms the spatial information of the image into a format suitable for processing by the Transformer network. Secondly, this application enhances the network's global modeling capability of spatial features by introducing the EMA attention mechanism; After the EMA output, depthwise separable convolutional and MLP layers are introduced.

3. The leather defect detection method based on parallel branch feature extraction according to claim 2, characterized in that, An EMA module is introduced in the feature extraction stage, which includes: First, the input feature map is divided into G channel groups to evenly distribute semantic information; then, a parallel sub-network processes the 1×1 branch using direction-aware one-dimensional global average pooling to capture long-range dependencies, while the 3×3 branch extracts local multi-scale interactions through convolution. Finally, cross-spatial dot product fusion is used to generate pixel-level attention weights.

4. The leather defect detection method based on parallel branch feature extraction according to claim 3, characterized in that, The features output by the parallel branches will be fused with the multi-scale features output by the backbone network in PCGF.

5. The leather defect detection method based on parallel branch feature extraction according to claim 4, characterized in that, First, low-resolution feature maps are interpolated by upsampling. Then, feature maps of different sizes are fused to achieve more accurate segmentation of leather surface defects. A convolutional layer is used to adjust the number of channels in one feature map so that it is the same as the number of channels in another feature map. The two feature maps are then concatenated along the channel dimension to form a new feature map. The concatenated feature map will contain feature information from two different sources. Channel-level weights w are generated using SEAttention, representing the importance of each channel in the feature map. The output of SEAttention is a sum of... Weighted feature maps with the same shape The module will use the weighted feature map The feature is divided into two parts based on the number of channels and corresponding channel weights. By applying the weights to the original features, the weighted features are obtained. Finally, a feature fusion operation is performed, which adds the weighted features to the original features to obtain the final fused features.

Images