Leather surface defect detection and classification method based on improved CSPNet

Abstract

How to accurately, reliably and intelligently detect leather surface defects has become an important subject in the field of industrial detection. In order to further improve the accuracy of leather defect identification, the invention provides a CSPCMNet neural network which is lighter and more efficient and seeks to improve the calculation performance. The CSPCMNet is based on the CSPNet, a convolution block of the CSPCMNet is replaced by a convolution modulation module, and Transform is combined with a traditional convolution technology. In addition, aiming at the problems of insufficient number and low quality of the currently researched leather data set, the invention also provides a leather data set with better quality and more number. StyleGan2 is selected as a data enhancement model. And generating enough training samples through the adversarial network.

Description

Technical Field

[0001] The present invention relates to the field of leather surface defect recognition, and in particular to a leather surface defect classification method, which realizes leather surface defect detection and classification based on a convolutional neural network of an improved CSPNet. Background Art

[0002] Leather and leather products are one of the commodities with the largest trade volume in the world today. Accurate, reliable and fast intelligent inspection of leather surface defects has become an important topic in industrial inspection and a very challenging task. With the development of computer technology, intelligent machine vision systems have been the core of industrial inspection and monitoring in the past two decades and have been widely used in the field of product surface defect detection. Leather surface defect detection technology based on machine vision has gradually become mainstream. This technology is not limited by manpower and time, and will not cause fatigue and discomfort. It is very suitable for enterprises to achieve large-scale production. Although domestic and foreign scholars and automatic inspection equipment suppliers have begun to pay attention to the automatic detection of leather surface defects based on machine vision since the 1990s. However, in actual applications, many companies still maintain traditional manual defect inspection and grading, and some companies have achieved semi-automatic and semi-manual, and a truly fully automated defect detection system has not yet been realized. Leather samples contain different types of defects and have a high degree of variability. Even without defects, the surface of raw leather is very complex, showing different textures, colors, and shapes. This brings challenges to the development of algorithms with high robustness and good generalization performance. In the field of leather surface defect detection, many researchers have carried out some exploratory work, but from the publicly published literature, there are few teams that continue to conduct in-depth research. In order to implement intelligent detection of leather surface defects, in-depth research is urgently needed.

[0003] Since the birth of AlexNet in 2012, the development of deep learning in the field of computer vision has shown a rapid trend and has achieved remarkable results. On various image data sets, deep learning has shown excellent performance and performance, and its effect is significantly better than traditional machine learning classification algorithms. After that, many different types of neural networks have been proposed one after another. At present, the leather panel defect detection technology based on machine vision is constantly improving and has achieved some excellent results. For example, the method using neural networks has shown high efficiency in defect detection. Therefore, the development of leather surface defect detection technology based on deep learning has become the focus of attention in the field of leather inspection.

[0004] Deep learning has great potential in leather surface defect detection, but it still faces challenges. In practical applications, leather defect detection usually needs to focus on both localization and recognition. However, existing research often focuses on defect type recognition or segmentation, rarely considering localization and recognition as first-stage tasks of equal priority, and rarely considering real-time requirements. In addition, current research is mainly concentrated in the field of traditional machine learning, and the number of related studies on deep learning is relatively small, and the research depth and experimental design are not detailed enough.

[0005] This work focuses on how to detect abnormal characteristics of leather products more efficiently and accurately. By analyzing the surface texture, color and gloss, scratches, indentations, uneven colors and other defects can be identified to improve product quality, reduce scrap rates, and automate quality control to reduce the cost and inconsistency caused by manual inspection.

[0006] At the same time, in multi-type defect detection, we hope that the model maintains high accuracy and pays attention to the balance between complexity and computational efficiency. The goal is to avoid overly complex network structures when accurately identifying and classifying defects (such as holes, scratches, pinholes, and rotten surfaces) to reduce computing resource consumption and improve real-time processing capabilities. Summary of the invention

[0007] In view of the low recognition rate of leather surface defect detection, the present invention proposes an improved neural network model based on CSPNet to obtain a neural network with better performance and higher accuracy. In addition, in view of the insufficient number and low quality of current research leather data sets, the present invention also proposes a leather data set with better quality and larger quantity.

[0008] The present invention proposes a CSPMCNet neural network: including 5 stages: stage 1 to stage 5, each stage is added with a 3x3 convolution block after downsampling; the output channel of the 3x3 convolution block is divided into two parts, the first part enters L1 residual blocks, and the residual blocks are connected to the modulation convolution Convolutional modulation module; the second part and the modulation convolution Convolutional modulation module are jointly input into the 1x1 convolution block. The L1 values ​​of the 5 stages are 1, 2, 8, 8, and 4 respectively.

[0009] The cross-stage transfer calculation includes: several residual blocks and modulated convolution modules

[0010] The present invention proposes that a leather surface defect detection and classification method based on an improved CSPNet comprises the following steps:

[0011] Step 1: Construct a dataset of leather surface defects. This dataset uses an ultra-high-definition whole-leather one-shot imaging system to collect a total of 2,000 whole-leather images of tanned sheepskin and cowhide surfaces.

[0012] Step 2. Construct a CSPMCNet neural network: With the CSPNet neural network as the backbone, there are 5 stages in total. A 3x3 convolution block is added after downsampling in each stage. The output channel of the 3x3 convolution block is divided into two parts. The first part enters L1 residual blocks, which are then connected to the Convolutional modulation module. The second part and the Convolutional modulation module are input into the 1x1 convolution block together.

[0013] For the input leather defect image, it is first processed to a uniform pixel size of 224x224 and has three RGB layers. Then it passes through the first convolution layer to become a feature map of size 112x112x32, with 32 being the number of channels. The feature map is then sent to the first stage and calculation begins.

[0014] After the feature map enters the first stage, it first passes through a 3x3 convolution block with 64 groups of convolution kernels for feature amplification, changing the number of channels to 64. It is then divided into two parts for calculation according to the number of channels, one part for direct mapping calculation and the other for cross-stage transfer.

[0015] According to the CSPNet concept, the direct mapping calculation part remains unchanged. The cross-stage transfer calculation part needs to pass through a 1x1 ordinary convolution block after the L1 residual blocks, and the L1 values ​​of the five stages are 1, 2, 8, 8, and 4 respectively. Here, the present invention introduces a modulated convolution Convolutional modulation module after the L1 residual block, because the modulated convolution Convolutional modulation module does not change the number of channels of the feature map, so there is no need to worry about the problem of unequal number of channels.

[0016] Convolutional modulation module: The feature map X after the residual block is divided into two parts according to the number of channels. One part is obtained after two consecutive Conv1x1 convolutions to obtain X1, and the other part is obtained after one Conv1x1 convolution to obtain X2.

[0017] Then, the Hadamard product of X1 and X2 is taken, and the elements corresponding to the feature map are multiplied to obtain X3;

[0018] X3 then undergoes a Conv1x1 convolution to adjust the number of channels to obtain X4, and finally X4 is added to the original X to complete the modulation; the number of channels of the modulated convolution at each stage is consistent with the residual block to which it is connected.

[0019] The modulated convolution Convolutional modulation is calculated as follows:

[0020] A=Conv 1×1 (GeLU(Conv 7×7 (X)) (1)

[0021] Z=X+Conv 1×1 (A*V) (2)

[0022] Among them: the input X is the output feature map of the residual block in the previous part. X first passes through a 7x7 convolution layer Conv7x7 to obtain a new feature map X1, and then X1 is input into the GeLU activation function to obtain the X2 feature map, and X2 is input into Conv 1x1 The number of channels is adjusted to obtain A; then the result of multiplying A and V is added to the output feature map X of the original residual block to obtain Z, which is the output of the modulated convolution.

[0023] Step 3: Combine the expanded total data set with the CSPMCNet network for training, test and classify the test set based on the trained model, and finally obtain the test classification accuracy, F1, recall, and precious indicators for practical application.

[0024] The leather surface defect dataset in step 1 is divided into 9 categories, including no defects, holes, scratches, rotten surfaces, wrinkles, growth lines, fungal injuries, and healed injuries. In order to construct such a leather surface defect dataset, an ultra-high-definition whole leather one-shot imaging system is used. The resolution of the ultra-high-definition CCD camera is 8688x5792 pixels. In order to distinguish the leather from the background, the color of the leather fixing platform is fixed to blue after counting the color of the leather. After being labeled and selected by experienced engineers in the factory, 2,000 leather surface defect images with a size of 2268*4432pixs were finally obtained.

[0025] Step 1 also includes: using traditional data enhancement technology and generative adversarial network technology to enhance the data of these 2,000 images, and the total number of images finally reached more than 17,000. At the current stage, the mainstream means of expanding image data sets is traditional data enhancement technology. Traditional data enhancement technology is based on physical changes to the original image, such as cropping, rotation, etc. These transformed images have a strong similarity with the original image, while the use of generative adversarial network technology generates completely new images. Therefore, the new data set has better diversity and the trained model has better robustness.

[0026] The dataset expansion of leather surface defects, through comparative experiments, experiments were conducted on DcGAN, FastGan, StyleGan2, and StyleGan3, and their FIDs were analyzed. Finally, StyleGan2 was selected as the data augmentation model. StyleGan2 is an improved version of the generative adversarial network GAN. It is a further development and optimization of the earlier version of StyleGan. StyleGAN2 performs well in image generation tasks, especially in generating high-quality and high-resolution images.

[0027] StyleGan2 consists of a discriminator and a generator. Its improvements over StyleGanv1 include: (1) It introduces a multi-scale feature fusion mechanism in the generator, making the generated images more coordinated and natural in both global structure and local details. (2) It improves the pixel-by-pixel style control mechanism in StyleGAN, making the generated images highly realistic and consistent at different scales. The steps of training and generating new images include: training 3000 rounds with the original 2000 images and obtaining new weights, in which the generator continuously reduces the loss and improves the authenticity of the generated images, and the discriminator continuously improves the accuracy of the discrimination. Finally, the optimal weights of the generator are used to generate images, and images close to the real images are obtained, thereby expanding the data set.

[0028] In StyleGan2, the generator loss is the generator loss plus the path regularization loss, where z represents a random vector sampled from the latent space, G(z) is the sample generated using the random vector, and a is the target path length. It represents the norm of the gradient of the generator output to the input potential vector w, and E represents the expected value. The total loss is as follows:

[0029]

[0030] Therefore, in StyleGan2, the generator plays a decisive role in the process of generating images. There is a problem in the original loss calculation, that is, in order to distinguish the authenticity of high-resolution images, a larger receptive field is required, that is, a deeper network and a larger convolution kernel are required. In order to solve this problem, the present invention introduces feature matching loss to ensure that feature maps of different sizes are compared, which successfully solves the problem. The k in the figure is mainly used to distinguish the multi-scale of high-resolution images. k = 1, 2, 3 represents the original image, 2 times downsampling, and 4 times downsampling. T represents the number of iterations, N i represents the number of samples in the i-th iteration, (s,x) and (s,G(s)) represent the real samples and the generated samples respectively. The feature matching loss is calculated as follows:

[0031]

[0032] Finally, the total loss of the generator can be expressed as, where λ fm is a weight coefficient used to balance the importance of feature matching loss:

[0033] L t +λ fm L FM (G,D K ) (3)

[0034] The modulated convolution module in step 2 reduces the complexity of the Transformer while absorbing its advantages to improve the performance of the model.

[0035] In step three, when training the model, the Top-1 and Top-5 accuracy, precision, recall and F1 Score values ​​of the output model training and testing are used as evaluation indicators, and the accuracy curve, loss curve and confusion matrix are used to verify and evaluate the obtained model.

[0036] Compared with the prior art, the advantages and positive effects of the present invention are:

[0037] 1) This model is aimed at more efficient and accurate detection of abnormal features in leather products. This goal involves in-depth analysis of the leather's surface texture, color, gloss, etc. to identify various possible defects such as scratches, indentations, uneven color or other irregular surface features. It helps to improve the overall quality of leather products, reduce scrap rates, and to a certain extent automate the quality control process, reducing the cost and inconsistency caused by manual inspection.

[0038] 2) For multi-type defect detection, we hope that the model maintains high accuracy while also focusing on the balance between model complexity and computational efficiency. This means that our goal is to ensure that the network can accurately identify and classify various defects, such as holes, scratches, pinholes, and rotten surfaces, while also avoiding overly complex network structures to reduce computing resource consumption and improve real-time processing capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is a structural diagram of the CSPMCNet model described in an embodiment of the present invention;

[0040] Figure 2 Schematic diagram of the structure of each stage of traditional CSPNet.

[0041] Figure 3 This is a schematic diagram of the structure of each stage of the improved CSPMCNet of the present invention.

[0042] Figure 4 The following are sample diagrams of each leather defect of the present invention.

[0043] Figure 5 This is a comparison chart of several common module structures.

[0044] Figure 6 The confusion matrix of leather dataset recognition by CSPMCNet is improved by the present invention. DETAILED DESCRIPTION

[0045] In order to better understand the purpose, characteristics and advantages of the present invention, now in conjunction with the accompanying drawings and specific embodiments, detailed description is given. It should be noted that the embodiments of the present application and the features therein can be combined with each other. Although many specific details are provided hereinafter to fully understand the present invention, the present invention can also be implemented by methods different from those described, and is therefore not limited to the specific embodiments disclosed below.

[0046] 1. Improved design of backbone network:

[0047] The original CSPNet was designed to solve the gradient vanishing problem in deep neural network training and improve the learning ability and generalization performance of the model. CSPNet can improve the learning ability of the network while reducing the amount of computation. In the classification task on the ImageNet dataset and the object detection task on the MSCOCO dataset, CSPNet has shown performance that is better than or close to the state-of-the-art methods. Its main idea is to divide the feature map into two parts, one part is directly connected to the end of the stage, and the other part enters a dense block or residual block. This can reduce the duplication of gradient information, increase the diversity of gradient paths, and improve the optimization efficiency of the network. And it can be combined with network structures such as ResNet, ResNeXt and DenseNet to form variants such as CSPResNet, CSPResNeXt and CSPDenseNet, which retains the advantages of the original network while reducing the amount of computation and memory cost. In the original CSPNet, two 1x1 convolutional layers are used after the downsampling module. However, in order to better capture the complex features of the feature map and increase the receptive field, after a large number of experimental verifications, this work proposes to replace these two 1x1 convolutional layers with a 3x3 convolutional layer. Such changes are aimed at improving the model's ability to extract features and perceive space, thereby improving overall performance.

[0048] Table 1 The impact of different sizes of convolution kernels on accuracy in the downsampling module

[0049] Kernel Top-1 Top-5 Precision Recall F1 1x1 85.83 98.33 86.16 86.11 85.97 3x3 86.16 98.55 86.65 86.08 85.99

[0050] In recent years, the self-attention mechanism in Transformer has developed rapidly in the field of vision and has become one of the hot topics in the field of computer vision. The core idea of ​​the self-attention mechanism is to capture important features in the input data through the model's own attention mechanism, so as to better understand and process the data. In the field of vision, the application of this mechanism has achieved remarkable results, including image classification, object detection, semantic segmentation and many other aspects.

[0051] However, in our field of leather surface defect detection, the application of self-attention is limited because its training time is longer than that of traditional convolutional networks and requires more computing resources. In addition, since the leather defect datasets are relatively small, the available training data is not sufficient. Therefore, for leather defect detection, it is necessary to reduce the complexity of the self-attention mechanism.

[0052] like Figure 1As shown in the figure, the modulated convolution Convolutional modulation reduces the complexity of Transformer while absorbing its advantages to improve the performance of the model.

[0053] The Transformer model uses Q (Query) which is a query vector, used to match the key to determine which input information to focus on, K (Key) which is a key vector, matched with the query to calculate the attention weight, and V (Value) which is a value vector, which is output after weighted summation and represents the information in the input sequence. They are all linearly transformed from the input vector X, where W Q and W K , W V are weight matrices, which will be gradually optimized according to the loss value during training to achieve the optimal value. The calculation of Q, K, and V is shown in the following formula:

[0054] Q=XW Q ,K=XW K ,V=XW V (4)

[0055] Softmax is a normalization function that converts a set of real numbers into a probability distribution. It amplifies the input value through an exponential function and then normalizes it so that the sum of all output values ​​is 1. The self-attention mechanism allows the model to focus on information at different positions in the input sequence when processing sequence data. Multi-HeadAttention is used, and the final result vector processed by Multi-HeadAttention is Z. C is the dimension of the key vector. K T is the transposed vector of the key vector K(Key). It is expressed in matrix as follows:

[0056]

[0057] In convolution modulation, the self-attention mechanism is simplified by using the feature modulation value V. Different from the original convolution modulation, it is proposed to replace the linear layer with a convolution layer with a convolution kernel size of 7x7, which is conducive to better extracting effective features without increasing a lot of calculations. The specific implementation is shown in equations (6) and (7): Given an input feature map X, it first passes through a 7x7 convolution layer Conv 7x7 , and then use the Gelu activation function to calculate it. The Gelu formula is as follows. Φ(X) is the cumulative distribution function of the standard normal distribution, as shown below:

[0058] Gelu(X)=X·Φ(X) (6)

[0059]

[0060] Then after a Conv 1x1 The convolutional layer obtains the vector A, and finally multiplies A and the feature modulation value V and passes through a Conv 1x1 After the convolution layer, the final Z value is obtained by adding the initial value X. The modulated convolution Convolutional modulation is calculated as follows:

[0061] A=Conv 1×1 (GeLU(Conv 7×7 (X)) (8)

[0062] Z=X+Conv 1×1 (A*V) (9)

[0063] Where * is matrix multiplication, representing the convolution operation of a 1x1 convolution kernel.

[0064] 2. Dataset design: After being labeled and preprocessed by engineers, the images obtained are still as high as 2268x4432 pixels, and many images contain other defects in addition to the labeled defects. If the input image is too large, it will reduce the training efficiency of the network and consume more time and computing resources. To this end, we removed some controversial labeled images, and under the guidance of engineers, we re-annotated the images and cut out 640x640 pixel images (such as Figure 4 As shown in Table 2, the dataset is used as the model input. Based on this, Gaussian blur, horizontal flip, vertical flip, 90-degree rotation, and Gaussian noise are first used to enhance the dataset, and then StyleGAN2 is used to enhance it again. The constructed dataset is shown in Table 2. The dataset has the following characteristics:

[0065] (1) The whole skin is imaged at one time. There is no need to stitch the images together. It has the characteristics of fast imaging, simple imaging process, stable imaging quality and good consistency.

[0066] (2) The dataset contains defects of both cowhide and sheepskin, with the ratio of cowhide to sheepskin being approximately 1.6:1. Most datasets used in the literature are derived from a single species, either cowhide or sheepskin.

[0067] (3) The images collected are of surface defects of tanned leather, including nine common leather surface defects. The defects in each image collected are different in size, position, shape, color, and texture, which ensures the diversity of the data set and poses greater challenges to the algorithm.

[0068] 3. Model training: We tested the effectiveness of the neural network on our own leather dataset and Cifar10 dataset. Our experimental hardware configuration uses Inspur Yingxin server NF5280M6, GPU graphics card is NVIDIA A40GPU; the software environment is Ubuntu18.04LTS operating system. All models are built and trained using Pytorch. Model evaluation indicators include Top-1, Top-5, Mean Precision, Mean Recall, and MeanF1.

[0069] In terms of the choice of optimizer, we used the stochastic gradient descent (SGD) optimizer, with the learning rate (lr) set to 0.03, the momentum set to 0.9, and the weight decay coefficient set to 0.0001. All models were trained for 200 epochs. Finally, we used the standard Top-1 and Top-5 indicators to evaluate the performance of the network. In order to ensure a fair comparison, we will not use any additional data augmentation strategies and only use random flipping for data augmentation.

[0070] In order to verify the network performance, we selected Figure 5 The 7 common neural network models (CSPDarkNet, CSPResNet, Res2Net, ResNext, EfficientNet, Repvgg, DenseNet) are shown, and comparative experiments are conducted on the custom leather dataset and the public Cifar10 dataset. The number of samples of each category of defects in the leather test set is 100.

[0071] The present invention selects several common neural network models, and in the test of leather data set, the performance of each model is recorded in detail in Table 2. By comparison, it can be clearly seen that the present model surpasses other test models in all key indicators.

[0072] Table 2 Performance evaluation of various models for leather surface defect detection

[0073] Model Top-1 Top-5 Precision Recall F1 OursNet-T 86.33 98.56 86.49 86.33 86.19 CSPDarkNet-53 85.83 98.33 86.16 86.11 85.97 CSPResNet-50 85.67 98.56 85.88 85.67 85.59 Res2Net-50 85.33 98.33 85.66 85.33 85.30 ResNeXt-50 85.11 98.67 85.29 85.11 85.02 EfficientNet-B3 83.67 98.33 84.12 83.67 83.60 RepVgg-A2 84.56 98.44 84.98 84.56 84.50 DenseNet-201 84.26 98.88 84.39 84.19 84.08

[0074] The present invention selected several common neural network models and tested them on the Cifar10 dataset. The results are shown in Table 3.

[0075] Table 3 Performance evaluation of each model on the Cifar10 dataset

[0076]

[0077]

[0078] As shown in the table above, the Top-1 accuracy of the model proposed in this invention is 86.3%, ranking first among all models and outperforming other models. The Top-5 accuracy is 98.56%, also ahead of other models, which shows that the model has stronger performance in classification tasks. The precision rate is 86.49%, which performs well among all models, indicating that it has high accuracy when predicting positive samples.

[0079] In order to further observe the recognition distribution of each category, the following Figure 6 The confusion matrix of the model shown is a tool for evaluating the performance of classification models. It shows the comparison between the model prediction results and the actual situation in the form of a matrix. In the confusion matrix, the first row and the first column represent the number of samples that identify the normal category image as the normal category, and the first row and the second column represent the number of samples that identify the normal category image as the cavity category. Therefore, in each category, the model's recognition rate for scratch defects is poor. Among the 100 scratch samples, only 69 samples are correctly identified, and they are misdiagnosed as any other type. Among them, the number of samples that are misdiagnosed as holes and wrinkles is the highest, both of which are 9. The precision and recall rates of defect-free images and pinhole defects are relatively high.

[0080] like Figure 2 and Figure 3 As shown in the figure, this application builds a new network structure based on the CSPNet model. Convolutional modulation is added after residual block processing, and the modulation convolution Convolutional modulation is improved. The linear layer is set to a convolution layer with a convolution kernel size of 7x7 to achieve the effect of improving classification accuracy. The two 1x1 convolution layers in the downsampling module in the CSPNet model are replaced with a 3x3 convolution layer, as shown in the figure. Figure 3 . The present invention comprehensively compares the performance of various mainstream neural networks in leather defect detection. In order to further verify the generalization performance of the model of the present invention, a generalization performance experiment was conducted based on the public dataset Cifar10. The experimental results show that the proposed hybrid model exhibits superior performance in both leather surface defect detection and Cifar10 datasets.

Claims

1. Leather surface defect detection and classification method based on improved CSPNet, characterized by The following steps are involved: Step 1: Construct a dataset of leather surface defects. This dataset uses an ultra-high-definition whole-leather imaging system to collect a total of 2,000 images of tanned sheepskin and cowhide surfaces. Use traditional data enhancement technology and generative adversarial network technology to enhance the data of these 2,000 images to form a training dataset. Step 2. Construct CSPMCNet neural network: With CSPNet neural network as the backbone, it includes 5 stages: stage 1 to stage 5. After downsampling in each stage, a 3x3 convolution block is added; the output channel of the 3x3 convolution block is divided into two parts. The first part enters L1 residual blocks, which are then connected to the Convolutional modulation module; the second part and the Convolutional modulation module are jointly input into the 1x1 convolution block; the L1 values ​​of the 5 stages are 1, 2, 8, 8, and 4 respectively; For the input leather defect image, it is first processed uniformly to a pixel size of 224x224 with three RGB layers, and then it passes through the first convolution layer to become a feature map of size 112x112x32, where 32 is the number of channels; Send the feature map to the first stage and start calculation; After entering the first stage, the feature map first passes through a 3x3 convolution block with 64 convolution kernels for feature amplification, which changes the number of channels to 64. Then, the calculation is divided into two parts according to the number of channels, one of which is used for direct mapping calculation and the other for cross-stage transfer calculation. The cross-stage transfer calculation includes: several residual blocks and modulated convolution modules; Step 3: Combine the expanded total data set with the CSPMCNet network for training, test and classify the test set based on the trained model, and finally obtain the test classification accuracy, F1, recall, and precious indicators for practical application.

2. The leather surface defect detection and classification method based on improved CSPNet according to claim 1 is characterized in that: Convolutional modulation module: The feature map X after the residual block is divided into two parts according to the number of channels. One part is obtained after two consecutive Conv1x1 convolutions to obtain X1, and the other part is obtained after one Conv1x1 convolution to obtain X2. Then, the Hadamard product of X1 and X2 is taken, and the elements corresponding to the feature map are multiplied to obtain X3; X3 then undergoes a Conv1x1 convolution to adjust the number of channels to obtain X4, and finally X4 is added to the original X to complete the modulation; the number of channels of the modulated convolution at each stage is consistent with the residual block to which it is connected.

3. The leather surface defect detection and classification method based on improved CSPNet according to claim 2 is characterized in that: The modulated convolution Convolutional modulation is calculated as follows: A=Conv 1×1 (GeLU(Conv 7×7 (X)) (1) Z=X+Conv 1×1 (A*V) (2) Among them: the input X is the output feature map of the residual block in the previous part. X first passes through a 7x7 convolution layer Conv7x7 to obtain a new feature map X1, and then X1 is input into the GeLU activation function to obtain the X2 feature map, and X2 is input into Conv 1x1 The number of channels is adjusted to obtain A; then the result of multiplying A and V is added to the output feature map X of the original residual block to obtain Z, which is the output of the modulated convolution.

4. The leather surface defect detection and classification method based on improved CSPNet according to claim 1 is characterized in that: In step one, traditional data enhancement techniques include Gaussian noise, brightness transformation, vertical symmetry transformation, scaling, and cropping to expand the original image; the generative adversarial network uses the StyleGan2 model.

5. The leather surface defect detection and classification method based on improved CSPNet according to claim 4 is characterized in that: In StyleGan2, the generator loss is the generator loss plus the path regularization loss, and the total loss is as follows: z represents a random vector sampled from the latent space, G(z) is a sample generated using the random vector, D(G(z)) represents the result of the discriminator's judgment on the generated fake sample, a is the target path length, It represents the norm of the gradient of the generator output to the input potential vector w, and E represents the expected value; Feature matching loss is introduced to ensure that feature maps of different sizes are compared; The feature matching loss is calculated as follows: in The k in the figure is mainly used to distinguish the multi-scale of high-resolution images. k = 1, 2, 3 represents the original image, 2 times downsampling, and 4 times downsampling. T represents the number of iterations, N i represents the number of samples in the i-th iteration, (s,x) and (s,G(s)) represent the real samples and the generated samples respectively; Finally, the total loss of the generator can be expressed as: L t +λ fm L FM (G,D K ) (5) where λ fm is a weight coefficient used to balance the importance of feature matching loss.

Images