Leather defect identification method based on double-branch frequency domain feature decoupling and dynamic channel enhancement network

Abstract

The invention provides a leather defect identification method based on double-branch frequency domain feature decoupling and a dynamic channel enhancement network. For the problem that the characteristics of a leather image are difficult to correctly extract due to the characteristics of complexity and non-uniformity of the leather image, a high and low frequency domain decoupling thought is introduced in the invention to extract main body and detail information respectively; a DCT (Discrete Cosine Transformation) guided high / low-frequency double-branch network is designed to respectively capture a local structure (high-frequency branch) of a microdefect and a continuity feature (low-frequency branch) of a macroscopic texture, a structure in which an SE (Square-and-Excitation) module is embedded in a feature fusion stage is designed, and channel weight and spatial position response are jointly optimized, so that the feature fusion is realized. And the significance expression of the key defect area is enhanced. In some public data sets, the method has better classification performance and higher diagnosis speed.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and artificial intelligence technology, specifically relating to an automatic identification method for leather defects based on deep learning and frequency domain feature decoupling, which is applicable to the real-time detection and classification of surface defects in the leather manufacturing process. Background Technology

[0002] In 2024, the global leather manufacturing industry was valued at over $394.8 billion. The surface quality of leather directly determines product grade, playing a crucial role, especially in the luxury goods and automotive interior sectors. However, natural biological variations in leather (such as complex textures and microporous structures) and mechanical processing (such as cutting and embossing) easily lead to defects such as loose grain, growth marks, and pinholes. Industry data shows that the average defect rate in cowhide production reaches 6.2%.

[0003] In actual industrial production quality inspection, traditional visual inspection relies on manual operation, with a speed as low as 0.5m / s. 2 The detection rate is slow and the false positive rate is as high as 18%, which cannot meet the demands of Industry 4.0 for efficient and automated quality inspection. The introduction of computer-aided identification aims to realize the industrial upgrading of automated production through artificial intelligence and significantly improve production efficiency. Currently, only a small portion of computer-aided identification is used for leather defect detection, and the research methods mainly include machine learning and deep learning.

[0004] In traditional machine learning, single-stream convolutional neural networks (CNNs) automatically extract features through end-to-end learning, avoiding the limitations of manual design. For example, models like ResNet and DenseNet capture local texture patterns through multi-layer convolutions; block processing strategies divide images into sub-regions for separate detection, improving the sensitivity to local features; Gray-Level Co-occurrence Matrix (GLCM) distinguishes defect regions by statistically analyzing texture contrast, energy, and other parameters; adaptive edge detection algorithms enhance defect contours using local gradient directions; and classifiers like Support Vector Machines (SVMs) rely on manually designed features (such as geometry and edge density) for classification. However, existing networks mostly use single-scale convolutional kernels, which are insufficient for extracting differentiated features from microscopic pores (50-200μm) and macroscopic scratches (>5mm), leading to multi-scale information confusion and a classification accuracy upper limit of only 86.4%.

[0005] The core limitations of the aforementioned methods lie in ignoring the complexity and non-uniformity of leather texture: First, the grain texture and defect morphology of natural leather are highly similar, making it difficult for traditional manual features and single-scale convolution to distinguish between local anomalies and normal variations; second, defects exhibit random spatial distribution (e.g., sparse pinholes and discontinuous scratches), and the block-based strategy disrupts global continuity, while the attention mechanism is insufficient in suppressing spatial noise; third, changes in lighting and surface reflections in industrial environments significantly interfere with the process, and existing models lack robust feature decoupling mechanisms, resulting in false detection rates exceeding 10%. Therefore, a novel recognition framework that balances multi-scale feature decoupling with dynamic environmental adaptability is still needed. Summary of the Invention

[0006] To address the complexity, non-uniformity, and significant interference from industrial environments in leather textures, this invention discloses a leather defect identification method (DFSNet) based on a dual-branch frequency domain feature decoupling and dynamic channel enhancement network. Through frequency domain decoupling and dynamic multi-scale feature fusion, it adaptively captures both local details and global texture features of the leather. A high / low-frequency dual-branch network guided by DCT transform captures the local structure of micro-defects (high-frequency branch) and the continuity features of macro-texture (low-frequency branch), respectively, solving the information confusion problem in multi-scale feature fusion of traditional single-branch CNNs. An SE (Squeeze-and-Excitation) module is embedded in the feature fusion stage to enhance the saliency representation of key defect regions by jointly optimizing channel weights and spatial location responses.

[0007] The overall process and specific steps of this invention are as follows:

[0008] Step 1: Leather image preprocessing and frequency domain decomposition;

[0009] (1) Data Acquisition:

[0010] Collect leather images containing 5 types of defects (resolution ≥1024×1024), with ≥600 images for each type.

[0011] (2) DCT block segmentation:

[0012] Convert the image to grayscale and divide it into 64×64 blocks. Perform DCT transformation on each block to separate the low-frequency (first 25% coefficients) and high-frequency components (last 75% coefficients) to generate a dual-channel feature map.

[0013] (3) Data augmentation:

[0014] Random rotation (±30°), brightness adjustment (±20%), and Gaussian noise (σ=0.01) enhance generalization ability.

[0015] Step 2: Construct the DFSNet model based on dual-branch frequency domain feature decoupling and dynamic channel enhancement network;

[0016] To address the multi-scale characteristics and complex texture interference of leather defects, this model employs a dual-branch architecture, processing the low-frequency and high-frequency components decomposed by Discrete Cosine Transform (DCT) separately. The input image is first divided into 64×64 pixel blocks. After DCT transformation, each block separates the low-frequency component (retaining the first 25% of coefficients to capture macroscopic texture continuity) and the high-frequency component (remaining 75% of coefficients to extract microscopic defect structures), generating dual-channel feature maps as model input. Both the low-frequency and high-frequency branches are based on the ResNet18 backbone network, with the first convolutional layer adapted (input channel set to 1, convolutional kernel 7×7, stride 2, padding 3). The first 16 layers of ResNet18 are retained to extract multi-level abstract features while avoiding excessive loss of high-level semantic information.

[0017] The feature maps output from the dual branches are concatenated along the channel dimension and then input into the dynamic channel enhancement module (SE module). First, global average pooling compresses the spatial dimensionality information to generate channel descriptors. Then, two 1×1 convolutional layers (the first layer compresses the number of channels to half the number of concatenated channels and activates ReLU; the second layer restores the original number of channels and activates Sigmoid) generate a channel weight matrix, performing channel-level recalibration on the concatenated features and significantly enhancing the response intensity of defect-related channels. The calibrated features are then reduced to a 1×1 spatial size using adaptive average pooling and finally input into a fully connected layer to complete classification, outputting the probability distribution of five defect classes.

[0018] Step 3: Model Training and Optimization

[0019] The Adam optimizer (initial learning rate 1e-4, weight decay 1e-5) and cross-entropy loss function are used. The dynamic learning rate is adjusted using a cosine annealing strategy. Training is conducted for 50 epochs with a batch size of 32. The dataset is divided into training and test sets in an 8:2 ratio, and 5-fold cross-validation is used to ensure generalization. Dropout (p=0.3) and L2 regularization are embedded during training to prevent overfitting. During the model inference stage, the weighted and fused bi-branch features are input to the fully connected layer for classification after global average pooling, and the output is the probability of the defect category. The final model has only 2.39M parameters and achieves an inference speed of 16ms / frame on embedded devices (such as NVIDIA Jetson AGX Xavier), supporting real-time detection (latency <50ms).

[0020] Compared with traditional methods, the beneficial effects of this invention are as follows: (i) Through frequency domain decoupling and dynamic channel-space attention, macroscopic and microscopic defect features of leather are accurately extracted, achieving an accuracy of 98.66% on a dataset of 3600 samples, which is 12.2% higher than that of single-stream CNN; (ii) The MSRCM module adaptively fuses multi-scale information, achieving a recall rate of over 97.8% for detecting microscopic pinholes of 50μm to macroscopic scratches of 5mm; (iii) The lightweight design of the model supports industrial-grade deployment, with a false detection rate of <2%, which is 16% lower than that of the traditional GLCM method, meeting the needs of intelligent manufacturing for high-precision and low-latency quality inspection. Attached Figure Description

[0021] Figure 1 This is a flowchart of the method;

[0022] Figure 2 This is a diagram of the neural network structure designed in this invention;

[0023] Figure 3 This is the dual-branch frequency domain feature decoupling diagram designed in this invention. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this invention clearer, the following detailed descriptions of the various specific details involved in the technical invention are provided:

[0025] This invention proposes a leather defect identification method based on dual-branch frequency domain feature decoupling and dynamic channel enhancement network. The electrocardiogram signal data used are from the 2018 Chinese Physiological Signal Challenge (2018CPSC) and the 2021 Computational Challenge in Cardiology (2021CinC). The 2018 CPSC dataset comes from 11 hospitals and contains 6,877 publicly available ECG records. Each record contains 12 standard leads, with a signal sampling frequency of 500 Hz and a duration of 6-60 seconds. This dataset includes 8 types of arrhythmias and 1 type of normal rhythm, for a total of 9 ECG types. The 2021 CinC dataset consists of six sources from four countries across three continents, with over 88,000 publicly available ECG records. Each record contains 12 standard leads. Due to the diverse data sources, the signal sampling frequency and duration vary. This dataset includes 133 diagnostic categories, of which 26 are relatively common and clinically significant for algorithm evaluation. Therefore, the scoring only evaluates the classification models for these 26 categories. Specific details are shown in Table 1. Each ECG record in the above datasets contains one or more labels, indicating that the patient has one or more arrhythmia conditions. The goal of this invention is to build a multi-label classification algorithm to correctly identify all types of arrhythmias in patients.

[0026] The overall process of this invention is as described in the appendix to the invention document. Figure 1As shown, the specific implementation method is as follows:

[0027] Step 1: Preprocess and frequency domain decompose the leather image data;

[0028] (1) Data Acquisition and Labeling: High-resolution images (≥1024×1024 pixels) containing 5 types of defects (fold marks, grain shedding, growth marks, loose grains, pinholes) were collected from the leather manufacturing line, with no less than 600 images for each type. All images were labeled with defect categories by professional quality inspectors to ensure label accuracy. (2) DCT Frequency Domain Decomposition: Each RGB image was converted to a grayscale image and divided into 64×64 pixel blocks. Two-dimensional discrete cosine transform (DCT) was performed on each block to generate a frequency domain coefficient matrix. According to the preset low-frequency retention ratio (25%), the coefficient matrix was divided into low-frequency components (retaining the 16×16 coefficients in the upper left corner to represent the macroscopic texture continuity) and high-frequency components (retaining the remaining 48×48 coefficients to extract the microscopic defect structure), and finally stitched together to form a dual-channel feature map (channel 0 for low frequency and channel 1 for high frequency). (3) Data Augmentation: To improve the model's robustness to changes in the industrial environment, the following augmentation operations were performed on the training set: random rotation (±30°), brightness adjustment (±20%), and Gaussian noise injection (standard deviation σ = 0.01). The total amount of data after augmentation was expanded to 5 times that of the original data to ensure sample diversity during training.

[0029] Step 2: Construct the DFSNet model based on dual-branch frequency domain feature decoupling and dynamic channel enhancement network;

[0030] As shown in the attached diagram of the instruction manual. Figure 2 As shown, the DFSNet model is based on a dual-branch architecture, processing the low-frequency and high-frequency components after Discrete Cosine Transform (DCT) decomposition separately, and combining it with a dynamic channel enhancement module (SE module) to achieve accurate extraction and fusion of multi-scale defect features. The model as a whole consists of a dual-branch feature extraction network, a dynamic channel enhancement module, and a classification output layer, as shown in the following structure:

[0031] The dual-branch feature extraction network structure is shown in the attached diagram in the manual. Figure 3 As shown, the low-frequency branch input is the low-frequency component of DCT (retaining the first 25% of coefficients). Based on the ResNet18 backbone network, the first layer convolution is adjusted to have 1 input channel (originally 3), the convolution kernel size is 7×7, the stride is 2, and the padding is 3. The formula is:

[0032] F low =ResNet18 low (X low )

[0033] Where X low For low-frequency component feature maps, ResNet18 lowIt includes the first 16 layers (up to the fourth residual block) and focuses on extracting global texture continuity features (such as macroscopic geometric distortions of growth traces).

[0034] The high-frequency branch input is the DCT high-frequency component (remaining 75% coefficients), and its structure is the same as the low-frequency branch:

[0035] F high =ResNet18 high (X high )

[0036] Where X high This is a high-frequency component feature map, focusing on the local structural features of micro-defects (such as the sharpness of pinhole edges and discrete distortion of particle shedding).

[0037] The Dynamic Channel Enhancement (SE) module dynamically adjusts the weights of channels in the feature map to enhance the response of task-relevant channels and suppress the influence of irrelevant or noisy channels, thereby improving the model's ability to capture key features. In DFSNet...

[0038] In this process, the SE module is embedded in the dual-branch feature fusion stage, and its core process consists of the following three steps:

[0039] (1) Concatenate the feature maps of the low-frequency and high-frequency branches into a 1024-channel feature tensor:

[0040] R concat =Concat(F low， F high )

[0041] (2) Spatial dimensionality information is compressed using Global Average Pooling (GAP) to generate a channel-level global descriptor. This descriptor represents the global importance of each channel. Then, channel weights are dynamically learned through two 1×1 convolutional layers: the first convolutional layer compresses the number of channels from 1024 to 512 (ReLU).

[0042] (Activation), the next convolution restores the original number of channels (Sigmoid activation), generating a normalized channel weight matrix:

[0043] Z = Gap(F) concat )

[0044] W = σ(W2·δ(W1·Z))

[0045] W1 and W2 are learnable parameters that perform dimensionality reduction and restoration operations, respectively.

[0046] (3) Finally, the weight matrix is ​​multiplied with the concatenated features channel by channel to achieve feature recalibration:

[0047]

[0048] The calibrated features are reduced to a 1×1 spatial size using adaptive average pooling and input into a fully connected layer to generate six types of defect probability distributions:

[0049] P=Softmax(FC(AdaptiveAvgPool(F calibrated ))

[0050] The fully connected layer has an input dimension of 1024 and an output dimension of 6, corresponding to 6 types of leather (normal, fold marks, grain shedding, growth marks, loose grain, and pinholes).

[0051] Step 3: Optimize and train the constructed DFSNet model;

[0052] To efficiently train the DFSNet model and improve the accuracy of leather defect classification, this step employs the cross-entropy loss function and an adaptive optimization strategy, combined with strict regularization and performance monitoring mechanisms. The calculation process is as follows:

[0053]

[0054] Where N is the batch size, c = 6 is the number of defect categories, and y i，c To encode the category of the real label, p i,c Let $c$ be the predicted class c probability by the model. The optimizer chosen is Adam, with an initial learning rate of 1e-4 and weight decay of 1e$. - 5. It gradually converges through 50 rounds of iteration.

[0055] The dataset was then divided into training and validation sets in an 8:2 ratio, with a batch size of 32. During training, a Dropout layer (dropout rate of 0.3) was embedded to suppress overfitting, and the validation set accuracy was monitored. If there was no improvement after 10 consecutive rounds, an early stopping mechanism was triggered. Each training round included the following steps:

[0056] (1) Forward propagation: Input the dual-channel feature map (low-frequency and high-frequency components) decomposed by DCT, extract features through a dual-branch ResNet18, and output the six-class classification probability P after dynamic calibration by the SE module. n ={p n 1 p n 2 ..., p6 6 The category with the highest probability is selected using torch.max.

[0057] (2) Loss calculation: Calculate the cross-entropy loss between the predicted probability and the true label;

[0058] (3) Backpropagation: The network parameters are updated by gradient descent, and the optimizer executes optimizer.step().

[0059] Table 1 Comparison of results from different methods

[0060]

Claims

1. A leather defect identification method based on dual-branch frequency domain feature decoupling and dynamic channel enhancement network, characterized in that: The method includes the following steps; Step 1: Leather image preprocessing and frequency domain decomposition; (1) Data Acquisition: Collect leather images containing 5 types of defects (resolution ≥1024×1024), with ≥600 images for each type. (2) DCT block segmentation: Convert the image to grayscale and divide it into 64×64 blocks. Perform DCT transformation on each block to separate the low-frequency (first 25% coefficients) and high-frequency components (last 75% coefficients) to generate a dual-channel feature map. (3) Data augmentation: Random rotation (±30°), brightness adjustment (±20%), and Gaussian noise (σ=0.01) enhance generalization ability. Step 2: Construct the DFSNet model based on dual-branch frequency domain feature decoupling and dynamic channel enhancement network; To address the multi-scale characteristics and complex texture interference of leather defects, this model employs a dual-branch architecture, processing the low-frequency and high-frequency components decomposed by Discrete Cosine Transform (DCT) separately. The input image is first divided into 64×64 pixel blocks. After DCT transformation, each block separates the low-frequency component (retaining the first 25% of coefficients to capture macroscopic texture continuity) and the high-frequency component (remaining 75% of coefficients to extract microscopic defect structures), generating dual-channel feature maps as model input. Both the low-frequency and high-frequency branches are based on the ResNet18 backbone network, with the first convolutional layer adapted (input channel set to 1, convolutional kernel 7×7, stride 2, padding 3). The first 16 layers of ResNet18 are retained to extract multi-level abstract features while avoiding excessive loss of high-level semantic information. The feature maps output from the dual branches are concatenated along the channel dimension and then input into the dynamic channel enhancement module (SE module). First, global average pooling compresses the spatial dimensionality information to generate channel descriptors. Then, two 1×1 convolutional layers (the first layer compresses the number of channels to half the number of concatenated channels and activates ReLU; the second layer restores the original number of channels and activates Sigmoid) generate a channel weight matrix, performing channel-level recalibration on the concatenated features and significantly enhancing the response intensity of defect-related channels. The calibrated features are then reduced to a 1×1 spatial size using adaptive average pooling and finally input into a fully connected layer to complete classification, outputting the probability distribution of five defect classes. Step 3: Model Training and Optimization The Adam optimizer (initial learning rate 1e-4, weight decay 1e-5) and cross-entropy loss function are used. The dynamic learning rate is adjusted using a cosine annealing strategy. Training is conducted for 50 epochs with a batch size of 32. The dataset is divided into training and test sets in an 8:2 ratio, and 5-fold cross-validation is used to ensure generalization. Dropout (p=0.3) and L2 regularization are embedded during training to prevent overfitting. During the model inference stage, the weighted and fused bi-branch features are input to the fully connected layer for classification after global average pooling, and the output is the probability of the defect category. The final model has only 2.39M parameters and achieves an inference speed of 16ms / frame on embedded devices (such as NVIDIA Jetson AGX Xavier), supporting real-time detection (latency <50ms).

Images