Detection model repeatability enhancement method for microfiber leather defect detection model

Abstract

The invention discloses a detection model repeatability enhancement method for a microfiber leather defect detection model, which comprises the following steps: acquiring a plurality of rolls of images for microfiber leather of the same model, and segmenting the acquired images into flawless area images and defective area images; processing the flawless area image through a double-branch variational auto-encoder network to extract common features and disturbance features, and storing the disturbance features as noise vectors; injecting the extracted disturbance noise to the defective area image at the bottleneck layer through feature modulation by using a feature injection U-Net network, and generating disturbance defect images in one-to-one correspondence with the original images; a YOLO series network structure is adopted, a confidence coefficient consistency constraint loss function is introduced, joint training is performed on an original flaw image and a generated disturbance flaw image, and the consistent prediction capability of the model on the same flaw under different environment conditions is enhanced. According to the method, the repetition rate of microfiber leather defect detection can be remarkably improved, and meanwhile, the detection precision and the real-time performance are kept.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer vision and artificial intelligence, in particular to a detection model repeatability enhancement method for a microfiber leather defect detection model, which is applied to quality detection in industrial production processes. BACKGROUND

[0002] As a new type of synthetic leather material, microfiber leather has been widely used in furniture, clothing, automotive interiors and other fields due to its excellent physical properties and environmental characteristics. In the production process of microfiber leather, surface quality is one of the key factors determining product grade and performance. However, with the expansion of production scale and the improvement of quality requirements, traditional manual visual inspection methods have been unable to meet the needs of modern industrial production for microfiber leather surface defect detection, so automatic defect detection technology based on computer vision has emerged.

[0003] In recent years, deep learning technology has made significant progress in industrial visual inspection, especially the YOLO series algorithm, due to its efficient real-time performance and high detection accuracy, has become one of the mainstream methods for industrial surface defect detection. YOLOv3, YOLOv5 and YOLOv7 variants have been widely used in the detection of leather, fabric and other surface defects, and have achieved excellent performance in multiple studies. However, current research mainly focuses on improving detection accuracy and recall rate, and pays less attention to the repeatability of detection results.

[0004] Detection repeatability refers to the ability of a detection system to produce consistent results under the same detection object and different imaging conditions. In actual industrial applications, especially in the detection of microfiber leather defects, environmental factors have a greater impact on detection results, resulting in greater fluctuations in the model in multiple detections, thereby reducing the stability of quality control. Therefore, how to improve the detection repeatability of the microfiber leather surface defect detection model is a major challenge currently faced by technology.

[0005] Currently, there are still several shortcomings in the technology for detecting defects on the surface of microfiber leather:

[0006] Firstly, changes in environmental factors, such as fluctuations in light in industrial sites, changes in material tension and camera parameters, will affect the consistency of detection results. Even under controlled environmental conditions, the results of the detection system still have some fluctuations. In addition, microfiber leather as a flexible material is affected by the tension changes of the conveying equipment during the detection process, resulting in stretching and deformation of the material, making the detection results of the same defect different at different times. Especially in the re-inspection process, due to the roll form of microfiber leather, reverse feeding is usually used in re-inspection, and the change in imaging direction also increases the inconsistency of detection results.

[0007] Secondly, existing simple data augmentation techniques such as rotation, flipping and brightness adjustment, while they can improve the generalization ability of the model, have limited ability to simulate disturbance factors in complex industrial environments, and cannot effectively improve the stability and repeatability of the model in a variable environment.

[0008] Finally, traditional deep learning model training focuses more on the optimization of detection accuracy, ignoring the important indicator of detection repeatability. Optimizing the accuracy loss function may make the model sensitive to environmental changes, thereby affecting the adaptability of the model in the actual production environment.

[0009] Therefore, the prior art fails to effectively solve the repeatability problem in microfiber leather surface defect detection, especially in a variable industrial site environment, the detection result of the prior art fluctuates greatly, causing unstable quality judgment, increasing production cost, and affecting customer satisfaction.

[0010] Therefore, the present application provides a detection model repeatability enhancement method for a microfiber leather defect detection model. SUMMARY

[0011] In view of the deficiencies of the prior art, the purpose of the present application is to provide a detection model repeatability enhancement method for a microfiber leather defect detection model, which significantly improves the consistency and reliability of the detection result by simulating various interference factors in the industrial environment and using a special training strategy, thereby providing more stable technical support for the microfiber leather quality detection system and improving its practical applicability in industrial applications.

[0012] The above invention purpose of the present application is realized by the following technical scheme:

[0013] A detection model repeatability enhancement method for a microfiber leather defect detection model, comprising the following steps:

[0014] Step 1, prepare a microfiber leather surface image training set, collect several volumes of images of the same type of microfiber leather, and divide the collected images into non-defect area images and defect area images;

[0015] Step 2, process the non-defect area images through a double-branch variational autoencoder network to extract common features and disturbance features, and store the disturbance features as noise vectors;

[0016] Step 3, use a feature injection U-Net generation network, input the original image of the defect area, inject the extracted disturbance noise vector at the network bottleneck layer, and generate a disturbance defect image corresponding to the original image;

[0017] Step four, using the YOLO series network structure, training while using the original defect image and the generated perturbation defect image, and introducing a confidence consistency constraint loss function, so that the model produces consistent prediction results for the same defect in the original image and the corresponding perturbation image.

[0018] The application is further provided that the double-branch variational autoencoder network comprises:

[0019] The shared encoder part is used for preliminary extraction of image features.

[0020] The branch path comprises a content encoder branch and a perturbation encoder branch.

[0021] The total loss function of the double-branch variational autoencoder network is defined as:

[0022]

[0023] Wherein, X represents an input image, c represents a content feature vector, z represents a perturbation feature vector, N represents a sample number, represents an i-th feature layer of a feature extraction network, and represents a mean vector and a standard deviation vector output by the perturbation encoder branch respectively, and m is a boundary parameter. , , , and are weight coefficients of each loss term respectively.

[0024] The application is further provided that the feature injection U-Net generation network comprises:

[0025] The encoder part maps the defect image into a deep feature representation.

[0026] The bottleneck layer receives a perturbation noise vector and completes noise injection through a feature modulation (FiLM) method.

[0027] The decoder part reconstructs the fused features into a perturbation defect image.

[0028] The total loss function of the feature injection U-Net generation network is defined as:

[0029]

[0030] Wherein, is an original image, is a generated image, M is a defect region mask, and SSIM is a structural similarity index. represents an i-th feature layer of a feature extraction network, , and are the weight coefficients of each loss term.

[0031] The present invention is further configured as follows: the confidence consistency constraint loss function is defined as the cross entropy of the original defect image and the corresponding perturbed defect image in each category prediction confidence, which is used to constrain the model's prediction consistency for the same defect under different perturbation conditions and improve the detection repetition rate; the confidence consistency constraint loss function is defined as:

[0032]

[0033] Among them, N represents the number of sample pairs, M represents the number of defect categories, and are the original prediction confidence and the expanded sample prediction confidence of the i-th sample in the j-th category, respectively.

[0034] The present invention is further configured as follows: the total loss function of the YOLO series network is:

[0035] in, is the classification loss, is the regression loss, is the confidence consistency constraint loss, 、 and are the weight coefficients of each loss function.

[0036] In summary, the beneficial technical effects of the present invention are:

[0037] 1. Accurately modeling real-world disturbances. By extracting structured disturbance features from multiple samples of the same material, we more accurately simulate the actual variations in industrial sites compared to random or hand-designed disturbances, ensuring that the disturbance distribution of the generated samples is consistent with the real environment.

[0038] 2. It provides a precise balance between defect feature stability and environmental variations. By injecting features into the U-Net deep semantic fusion layer, it ensures that the basic shape of the defect remains stable while introducing changes in details, lighting, and texture that are consistent with real scenes, avoiding the limitation of simple geometric transformations that cannot simulate complex optical effects.

[0039] 3. Enhanced detection repeatability. The innovative confidence consistency constraint training strategy directly optimizes the model's prediction consistency under different input conditions, enabling the detection system to maintain stable output in the face of changes in the industrial environment and significantly improving the repeat detection rate;

[0040] 4. High precision and real-time performance are maintained. Since the noise extraction and image generation process is only performed in the training stage, the model structure remains unchanged in the inference stage, thus significantly improving the repetition rate without increasing the detection delay, meeting the real-time detection requirements of industry. BRIEF DESCRIPTION OF DRAWINGS

[0041] Figure 1 is the overall flowchart of the method of the present application;

[0042] Figure 2 is a double-branch variational autoencoder network structure diagram;

[0043] Figure 3 is a feature injection U-Net generation network structure diagram. DETAILED DESCRIPTION

[0044] The present application will be further described in detail below in conjunction with the accompanying drawings.

[0045] REFERENCE Figure 1 The flowchart described is a detection model repetition enhancement method for the super-fiber leather defect detection model disclosed by the present application, comprising the following steps:

[0046] Step 1: Construction of super-fiber leather surface defect dataset. According to the on-site production process, a line array true color camera is used to collect images of super-fiber leather. In order to ensure uniform illumination, the system is equipped with a LED linear light source. During the collection process, M rolls of samples of the same type of super-fiber leather are collected to ensure that super-fiber leather images under different environmental conditions can be obtained.

[0047] The collected original images are preprocessed, including super-fiber leather region segmentation and subgraph division, and finally N subgraphs are obtained. These N subgraphs are classified by artificial classification into N1 non-defect area images and N2 defect area images. The N2 defect area images are labeled to record the position, size and category information of the defects, and are randomly divided into training set and test set according to the ratio of 8:2.

[0048] In the model training process, N1 non-defect area images are used to train a double-branch variational autoencoder network to extract environmental disturbance feature vectors. The labeled images of the N2 defect area are used as the input of the feature injection U-Net generation network, combined with the extracted disturbance feature vectors, to generate N2 disturbance defect images corresponding to the original images. Finally, the original N2 defect images and the generated N2 disturbance defect images are used together to train the YOLOv8 detection network, so that the detection model learns the original defect features and the defect features under disturbance conditions at the same time, thereby improving the repetition of detection.

[0049] Through this organized data set construction and utilization mode, it is ensured that the model can learn the essence characteristics of the microfiber leather and the disturbance characteristics of the environmental changes, thereby providing a solid foundation for subsequent repetitive reinforcement.

[0050] Step two, disturbance feature extraction based on double-branch variational autoencoder.

[0051] (1) Double-branch variational autoencoder network architecture. As shown in Figure 2 Fig. 1, the present application designs a dedicated double-branch variational autoencoder network for separating the common content features and environmental disturbance features in the microfiber leather image. Among them, the common features represent the basic texture and structural characteristics of the microfiber leather, and the disturbance features reflect the environmental changes (such as illumination changes, material stretching, and camera noise, etc.) under different acquisition conditions. The network structure includes:

[0052] Shared encoder: ResNet architecture is used as the backbone network, the last fully connected classification layer is removed, and the feature extraction part is retained. The encoder contains multiple convolution stages, each stage contains multiple residual blocks, which are used to extract basic feature representations from the input flawless microfiber leather image;

[0053] Branch path: the feature map output by the shared encoder is processed through two parallel branches:

[0054] The content encoder branch gradually processes the output features of the shared encoder through multiple residual blocks, and finally maps them into a content feature vector c through global average pooling and fully connected layers. The design goal of the content encoder branch is to capture the basic texture structure and material characteristics of the microfiber leather, and to ensure that these features remain highly consistent among different acquisition images of the same type of material.

[0055] The disturbance encoder branch adopts a variational design, which processes the output features of the shared encoder through residual blocks, and then outputs the mean vector μ and the logarithmic variance vector log(σ²) through two independent fully connected layers. Through the reparameterization technique, the disturbance feature vector z is generated by the formula (where ). This variational design makes the disturbance features form a continuous distribution, which can better express the diversity and continuity of environmental changes.

[0056] The decoder part first connects the content features c and the disturbance features z into a comprehensive feature vector, maps it through a fully connected layer and reshapes it into a feature map. Then, through multiple upsampling blocks, the image resolution is gradually restored, and finally the feature map is mapped to an RGB image through a convolution layer. The design of the decoder enables the network to reconstruct the original image from the separated content features and disturbance features, verifying the effectiveness of the feature separation.

[0057] (2) Training process and loss function design. To achieve effective separation of content features and perturbation features, DB-VAE adopts a multi-objective optimization strategy, and the loss function combination includes reconstruction loss, content consistency loss, perturbation difference loss, and KL divergence regularization. The definitions of various symbols are as follows: X represents the input image, c represents the content feature vector, z represents the perturbation feature vector, N represents the total number of flawless samples, i and j represent the indexes of different images, and respectively represent the mean vector and standard deviation vector output by the perturbation encoder branch. The reconstruction loss adopts a combination of L2 norm and perceptual loss, ensuring that the decoded image is similar to the input image at the pixel level and feature level:

[0058] ,

[0059] where represents the i-th feature layer of the feature extraction network. The content consistency loss forces the texture images of the same type of super leather to cluster in the content feature space: . The perturbation difference loss increases the difference in perturbation features of images under different acquisition conditions using a contrast learning method: where m is the margin parameter, which is used to ensure that the perturbation features of different images maintain a minimum distance. The KL divergence regularization makes the distribution of perturbation features close to the standard normal distribution: . The total loss function is the weighted sum of the above items: .

[0060] (3) Perturbation feature extraction. After training, N1 background images of super leather are input into the network to extract their perturbation feature vectors and store them. These vectors represent the real perturbation patterns that may occur in the industrial environment and will be used as noise input for generating perturbation defect images in the future.

[0061] Step three, perturbation defect image generation based on feature injection U-Net.

[0062] (1) Feature injection U-Net network architecture. The application designs an improved U-Net structure, which is specially used for converting the original defect image into a defect image containing specific environmental disturbance, while maintaining the basic form of the defect. The network adopts a typical U-shaped structure, and the encoder path contains multiple encoding blocks, each of which contains two convolution layers, each of which is followed by LeakyReLU and BatchNorm, and the spatial size is reduced by maximum pooling. The number of feature channels increases layer by layer, and the spatial size decreases layer by layer. The decoder path contains a corresponding number of decoding blocks, each of which is upsampled in space by transposed convolution, and the feature map of the corresponding layer of the encoder is spliced through jump connection, and then processed through the convolution layer. The number of feature channels decreases layer by layer, and the spatial size gradually recovers. Finally, the feature map is mapped to an RGB image through convolution, and the Sigmoid activation function is used to ensure that the pixel value is in the range of [0, 1].

[0063] The key innovation of the network lies in the noise injection mechanism of the bottleneck layer. The bottleneck layer receives the final feature map F of the encoder, and also receives the disturbance feature vector z extracted in step two. The vector is mapped into scaling parameter γ and offset parameter β through two fully connected layers. In order to avoid excessive changes to the basic form of the defect, the application uses a residual form of feature modulation:

[0064] , ,

[0065] Wherein α is used to control the disturbance intensity. This residual form ensures that the original feature F is still dominant, and the disturbance part is supplemented, effectively avoiding excessive changes to the basic form of the defect. The modulated feature is used as the starting input of the decoder.

[0066] (2) Multi-objective loss function design. In order to ensure that the generated disturbance defect image meets the requirements, a multi-objective loss function combination is designed: the global reconstruction loss uses L1 norm to measure the overall similarity between the generated image and the original image: The defect area consistency loss adopts the structural similarity index (SSIM) to measure, which emphasizes the structure preservation of the defect area:

[0067] ,

[0068] Wherein M is the defect area mask, which is used to focus on the structural similarity evaluation of the defect area. This loss ensures that the defect area in the generated image is consistent with the original defect in visual perception, while allowing reasonable changes in non-critical areas. The perceptual loss uses the intermediate layer features of the pre-trained network to calculate the semantic level similarity: , wherein The i-th feature layer of the feature extraction network. In this implementation, higher weights are assigned to deep features to give more importance to the preservation of high-level semantic features. The total loss function is:

[0069] wherein , and are the weight coefficients of each loss term, respectively.

[0070] (3) Disturbance defect image generation process. After training, for N2 training images with defect regions, select appropriate vectors z from the N1 disturbance feature vectors obtained in step two, input the original defect image into the encoder of the feature injection U-Net, and obtain the bottleneck layer feature F. The disturbance feature vector z generates scaling and offset parameters through the mapping network, and is applied to F in the form of residual modulation. After processing by the decoder, the generated disturbance defect image is output. This image maintains the basic morphology of the defect, but reflects changes in the industrial environment in terms of illumination, detail texture, etc. In this way, one corresponding disturbance defect image is generated for each original defect image, forming N2 pairs of image pairs, which are used for subsequent confidence consistency training of the YOLO network. The disturbance generated by this feature injection method is more realistic, and compared with simple geometric transformation or random noise, it can more accurately simulate actual changes in the industrial environment, such as uneven illumination, material stretching, and camera noise, etc.

[0071] Step four, YOLO network training with confidence consistency constraint.

[0072] (1) YOLO model architecture. The confidence consistency constraint method proposed in this application is applicable to detection models of all versions of the YOLO series, including but not limited to YOLOv5, YOLOv7, YOLOv8, and YOLOv10, etc. This embodiment adopts YOLOv8, and taking YOLOv8 as an example, the network is composed of the following main components:

[0073] Backbone network: used to extract multi-scale features of the image;

[0074] Feature pyramid: fuses features of different scales to enhance the detection ability of different size defects;

[0075] Detection head: predicts defect categories, confidence, and bounding box coordinates.

[0076] (2) Confidence consistency constraint loss function. To enhance the consistency of the detection results of the YOLO series models under different interference conditions, this application introduces a confidence consistency constraint loss function, which is defined as follows:

[0077]

[0078] wherein, N represents the number of samples, M represents the number of categories of defects, With respectively, the original prediction confidence of the ith sample in the jth category and the extended sample prediction confidence.

[0079] (3) Total loss function. The total loss function of the YOLO network is defined as:

[0080]

[0081] In this example, is set to , and are the weight coefficients of each loss function.

[0082] (4) Training strategy and implementation. The present application adopts a two-stage training strategy:

[0083] The first stage (pre-training): using N2 original defect training images, optimizing the standard YOLO loss function ( ), so that the model learns the basic defect detection ability;

[0084] The second stage (consistency training): using N2 original defect images and the corresponding generated N2 perturbed defect images as input in pairs, optimizing the total loss function containing confidence consistency constraint , guiding the model to learn consistent prediction results under different interference conditions.

[0085] During the second stage of training, the original image and the corresponding perturbed image in each batch are input into the network in pairs, and the confidence consistency loss between them is calculated to strengthen the model's adaptability to environmental disturbances. Through this training strategy, the model not only can accurately detect defects, but also can maintain the consistency of the detection results under various environmental disturbances, significantly improving the detection repetition rate.

[0086] This confidence consistency constraint training method is a general improvement for YOLO series models, and can be implemented according to the specific application scenario by selecting the appropriate YOLO version.

[0087] The embodiments of the specific implementation mode are the preferred embodiments of the present application, not limited to the protection scope of the present application, so: any equivalent changes made according to the structure, shape, principle of the present application should be covered within the protection scope of the present application.

Claims

1. A method for enhancing the repeatability of a microfiber leather defect detection model, characterized by: The following steps are involved: Step 1: Prepare a training set of microfiber leather surface images, collect several rolls of images of the same model of microfiber leather, and segment the collected images into images of flawless areas and images of defective areas; Step 2: Process the defect-free area image through a two-branch variational autoencoder network to extract common features and perturbation features, and store the perturbation features as a noise vector; Step 3: Use the feature injection U-Net to generate a network. Take the original image with defective areas as input and inject the extracted perturbation noise vector into the bottleneck layer of the network to generate a perturbation defect image that corresponds one-to-one with the original image. In step 4, the YOLO series network structure is used, and the original defect image and the generated perturbed defect image are used for training at the same time. The confidence consistency constraint loss function is introduced to make the model produce consistent prediction results for the same defect in the original image and the corresponding perturbation image.

2. The method according to claim 1, wherein: The dual-branch variational autoencoder network includes: The shared encoder part is used to preliminarily extract image features; A branch path, including a content encoder branch and a perturbation encoder branch; The total loss function of the two-branch variational autoencoder network is defined as: Where X represents the input image, c represents the content feature vector, z represents the perturbation feature vector, and N represents the number of samples. represents the i-th feature layer of the feature extraction network, μ and They represent the mean vector and standard deviation vector of the disturbance encoder branch output, m is the boundary parameter, 、 、 、 and are the weight coefficients of each loss term.

3. The method according to claim 1, wherein: The feature injection U-Net generation network includes: The encoder part maps the defect image into a deep feature representation; The bottleneck layer receives the perturbation noise vector and completes the noise injection through feature modulation (FiLM); The decoder reconstructs the fused features into a perturbed defect image; The total loss function of the feature injection U-Net generation network is defined as: in, is the original image, To generate an image, M is the defect area mask, SSIM is the structural similarity index, represents the i-th feature layer of the feature extraction network, 、 and are the weight coefficients of each loss term.

4. The method according to claim 1, wherein: The confidence consistency constraint loss function is defined as the cross entropy of the original defect image and the corresponding perturbed defect image on the prediction confidence of each category, which is used to constrain the model's prediction consistency for the same defect under different perturbation conditions and improve the detection repetition rate. The confidence consistency constraint loss function is defined as: Among them, N represents the number of sample pairs, M represents the number of defect categories, and are the original prediction confidence and the expanded sample prediction confidence of the i-th sample in the j-th category, respectively.

5. The method according to claim 1, wherein: The total loss function of the YOLO series network is: in, is the classification loss, is the regression loss, is the confidence consistency constraint loss, 、 and are the weight coefficients of each loss function.

Images